JP3890838B2 - Encoded stream conversion apparatus, encoded stream conversion method, and recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present inventionEncoded stream conversion apparatus, encoded stream conversion method, and recording mediumIn particular, the present invention is suitable for use in a transcoding device for changing the GOP (Group of Pictures) structure of an encoded bitstream encoded based on the MPEG standard or changing the bit rate of the encoded bitstream. NaEncoded stream conversion apparatus, encoded stream conversion method, and recording mediumAbout.
[0002]
[Prior art]
In recent years, MPEG (Moving Picture Experts Group) technology has been commonly used in broadcasting stations that produce and broadcast television programs in order to compress / encode video data. In particular, this MPEG technology is becoming the de facto standard when recording video data on a randomly accessible recording medium material such as a tape and when transmitting video data via a cable or satellite.
[0003]
An example of processing in the broadcasting station until the video program produced in the broadcasting station is transmitted to each home will be briefly described. First, source video data is encoded and recorded on a magnetic tape by an encoder provided in a camcorder in which a video camera and a VTR (Video Tape Recorder) are integrated. At this time, the encoder of the camcorder encodes the source video data so as to be suitable for the recording format of the VTR tape. For example, the GOP structure of an MPEG bit stream recorded on this magnetic tape is a structure (for example, I, B, I, B, I, B,...) Composed of 2 frames and 1 GOP. Is done. The bit rate of the MPEG bit stream recorded on the magnetic tape is 18 Mbps.
[0004]
Next, in the main broadcasting station, editing processing for editing the video bit stream recorded on the magnetic tape is performed. For this purpose, the GOP structure of the video bit stream recorded on the magnetic tape is converted into a GOP structure suitable for editing processing. A GOP structure suitable for editing processing is a GOP structure in which one GOP is composed of one frame and all pictures are I pictures. This is because an I picture having no correlation with other pictures is most suitable for editing in frame units. In actual operation, the video stream recorded on the magnetic tape is once decoded and returned to the baseband video data. Then, the baseband video signal is re-encoded so that all pictures become I pictures. By performing the decoding process and the re-encoding process in this way, it is possible to generate a bitstream having a GOP structure suitable for the editing process.
[0005]
Next, in order to transmit the edited video program generated by the editing process described above to the local station from the main station, the bit stream of the edited video program is converted into a GOP structure and a bit rate suitable for the transmission process. A GOP structure suitable for transmission between broadcast stations is, for example, a GOP structure in which 1 GOP is composed of 15 frames (for example, I, B, B, P, B, B, P,...). The bit rate suitable for transmission between broadcasting stations is generally a high bit rate of 50 Mbps or more because a dedicated line having a high transmission capacity such as an optical fiber is provided between broadcasting stations. It is desirable. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for transmission between the broadcasting stations described above.
[0006]
In the local station, editing processing is performed in order to insert a commercial unique to the local area in the video program transmitted from the main station. That is, as in the editing process described above, the video stream transmitted from the main station is once decoded and returned to the baseband video data. Then, by re-encoding the baseband video signal so that all the pictures become I pictures, a bitstream having a GOP structure suitable for editing processing can be generated.
[0007]
Subsequently, the video program edited in the local station is converted into a GOP structure and bit rate suitable for the transmission process in order to transmit it to each home via a cable or a satellite. For example, a GOP structure suitable for transmission processing for transmission to each home is a GOP structure in which 1 GOP is composed of 15 frames (for example, I, B, B, P, B, B, P,...). Thus, a bit rate suitable for transmission processing for transmission to each home is a low bit rate of about 5 Mbps. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for the transmission processing described above.
[0008]
[Problems to be solved by the invention]
As can be understood from the above description, the decoding process and the encoding process are repeated a plurality of times while the video program is transmitted from the broadcast station to each home. Actually, the processing at the broadcasting station requires various signal processing in addition to the signal processing described above, and the decoding processing and the encoding processing must be repeated each time.
[0009]
However, it is well known that encoding processing and decoding processing based on the MPEG standard are not 100% reversible processing. That is, the baseband video data before being encoded and the video data after being decoded are not 100% the same, and the image quality is degraded by this encoding process and decoding process. That is, as described above, there is a problem in that when the decoding process and the encoding process are repeated, the image quality deteriorates every time the process is performed. In other words, image quality deterioration accumulates every time decoding / encoding processing is repeated.
[0010]
The present invention has been made in view of such a situation, and performs decoding and encoding processing to change the GOP (Group of Pictures) structure of an encoded bitstream encoded based on the MPEG standard. It is intended to realize a transcoding system that does not cause image quality degradation even if it is repeated.
[0011]
[Means for Solving the Problems]
  An encoded stream conversion apparatus according to a first aspect of the present invention is an encoded stream conversion apparatus that converts an encoded stream into a re-encoded stream, and in the past encoding process or decoding process for the encoded stream History encoding parameters used, andIt can be used for a conversion process for converting the encoded stream into the re-encoded stream.Input means for inputting identification information for identifying a selective combination of the history encoding parameters together with the encoded stream;Encoding parameter calculation means for calculating a current encoding parameter that can be used in a conversion process for converting the encoded stream into the re-encoded stream;The picture type included in the history coding parameter is used for the conversion process.Re-encodingIf it matches the picture type,Of the history coding parameters that match the picture type,The identification informationShown inHistory encoding parametersIs the encoding parameter used in the conversion processChoose asIf the picture type included in the history encoding parameter does not match the re-encoded picture type in the conversion process, the current encoding parameter calculated by the encoding parameter calculation unit is used for the conversion process. Selected as the coding parameter usedAnd selecting means to be selected by the selecting meansUtilize coding parameters usedConversion means for converting the encoded stream into the re-encoded stream.
[0012]
  The history coding parameter is a motion vector.includingCan be.
[0014]
An output means for outputting the re-encoded stream converted by the conversion means can be further provided.
[0015]
  The identification information isUsing the history coding parameter indicated in the identification informationConversion process of the conversion meansIs executedThe information can be distinguished according to the degree of image quality degradation.
[0016]
The identification information may be information that is distinguished according to a transmission path through which the encoded stream is transmitted or a capacity of a recording medium that records the encoded stream.
[0017]
  Selected by the selecting meansUse encoding parameters orAn output means for outputting any of the history coding parameters together with the re-encoded stream converted by the conversion means can be further provided.
[0018]
  The conversion means includes the selection means selected by the selection means.Use encoding parameters orAny of the history encoding parameters can be described in the re-encoded stream
[0021]
  The identification information includes the history encoding parameter.Parameters that can be used for the conversion process for converting the encoded stream into the re-encoded stream.Or the history coding parameterParameters that can be used for the conversion process for converting the encoded stream into the re-encoded stream.First identification information for identifying whether a part of the input is input;Available for conversion processThe history encoding parameter when some of the parameters are inputAvailable parameters included inAnd second identification information for identifying the combination of the two.
[0023]
  The conversion means performs a conversion process in an MPEG system having a sequence layer, a GOP layer, a picture layer, a slice layer, and a macroblock layer, and the first identification information and the second identification informationBut,Described in the re-encoded stream as a descriptor represented by red_bw_flag and red_bw_indicatorHas beenCan be.
[0024]
The conversion means includes a bit rate or GOP The conversion process can be performed so as to change the structure.
[0025]
The conversion means includes a sequence layer, GOP Has a layer, a picture layer, a slice layer, and a macroblock layer MPEG The conversion process can be performed by a method.
[0026]
The history encoding parameter may be an encoding parameter used in past multiple generation encoding processes or decoding processes for the encoded stream.
[0027]
The history encoding parameter may be an encoding parameter used in encoding processing or decoding processing for the last four generations with respect to the encoded stream.
[0028]
  The encoded stream conversion method according to the first aspect of the present invention, and the program recorded on the recording medium include history encoding parameters used in past encoding processing or decoding processing for the encoded stream, andIt can be used for a conversion process for converting the encoded stream into the re-encoded stream.Identification information identifying a selective combination of the history encoding parameters, and the encoded streamInput step that receives inputWhen,An encoding parameter calculation step for calculating a current encoding parameter that can be used in a conversion process for converting the encoded stream into the re-encoded stream;The picture type included in the history coding parameter is used for the conversion process.Re-encodingIf it matches the picture type,Of the history coding parameters that match the picture type,The identification informationShown inHistory encoding parametersIs the encoding parameter used in the conversion processChoose asIf the picture type included in the history encoding parameter does not match the re-encoded picture type in the conversion process, the current encoding parameter calculated by the process of the encoding parameter calculation step is used for the conversion process. Select as the used encoding parameter to useA selection step to be performed, and the selection selected by the processing of the selection stepUsing the use coding parameters,Converting the encoded stream into the re-encoded stream.
[0029]
  In the first aspect of the present invention, history encoding parameters used in past encoding processing or decoding processing for an encoded stream, andCan be used for conversion processing to convert encoded stream to re-encoded streamIdentification information identifying a selective combination of historical coding parameters, as well as an encoded stream are input,The current encoding parameters available for the conversion process are calculated,The picture type included in the history coding parameter is used for the conversion process.Re-encodingIf it matches the picture type,Of the history coding parameters that match the picture type,Identification informationShown inHistory encoding parametersIs the encoding parameter used in the conversion process.Selected asWhen the picture type included in the history encoding parameter does not match the re-encoded picture type in the conversion process, the calculated current encoding parameter is selected as the use encoding parameter used for the conversion process,chosenUtilization coding parameters are used,The encoded stream is converted into a re-encoded stream.
[0200]
DETAILED DESCRIPTION OF THE INVENTION
The transcoder to which the present invention is applied will be described below, but before that, compression coding of a moving image signal will be described. In this specification, the term “system” means an overall apparatus constituted by a plurality of apparatuses and means.
[0201]
For example, in a system that transmits a moving image signal to a remote place such as a video conference system and a video phone system, in order to efficiently use a transmission path, the line correlation of video signals and the correlation between frames are used. An image signal is compressed and encoded.
[0202]
When line correlation is used, an image signal can be compressed by, for example, DCT (discrete cosine transform) processing.
[0203]
Further, when the inter-frame correlation is used, the image signal can be further compressed and encoded. For example, as shown in FIG. 1, when frame images PC1 to PC3 are respectively generated at times t1 to t3, the difference between the image signals of the frame images PC1 and PC2 is calculated to generate PC12. The difference between the images PC2 and PC3 is calculated to generate PC23. Normally, images of frames that are temporally adjacent do not have such a large change. Therefore, when the difference between them is calculated, the difference signal has a small value. Therefore, if this difference signal is encoded, the code amount can be compressed.
[0204]
However, if only the differential signal is transmitted, the original image cannot be restored. Therefore, the image of each frame is set to one of three types of pictures, ie, I picture, P picture, or B picture, and the image signal is compressed and encoded.
[0205]
That is, for example, as shown in FIG. 2, the image signals of 17 frames from frames F1 to F17 are set as a group of pictures (GOP), which is a unit of processing. The image signal of the leading frame F1 is encoded as an I picture, the second frame F2 is processed as a B picture, and the third frame F3 is processed as a P picture. Hereinafter, the fourth and subsequent frames F4 to F17 are alternately processed as a B picture or a P picture.
[0206]
As an image signal of an I picture, the image signal for one frame is transmitted as it is. On the other hand, as the picture signal of the P picture, basically, as shown in FIG. 2, the difference from the picture signal of the I picture or P picture preceding in time is transmitted. Further, as an image signal of a B picture, basically, as shown in FIG. 3, a difference from the average value of both the temporally preceding frame and the succeeding frame is obtained, and the difference is encoded.
[0207]
FIG. 4 shows the principle of a method for encoding a moving image signal in this way. As shown in the figure, since the first frame F1 is processed as an I picture, it is transmitted as it is to the transmission path as transmission data F1X (intra-picture coding). In contrast, since the second frame F2 is processed as a B picture, the difference between the temporally preceding frame F1 and the average value of the temporally following frame F3 is calculated, and the difference is calculated as It is transmitted as transmission data F2X.
[0208]
However, there are four types of processing as the B picture in more detail. The first processing is to transmit the data of the original frame F2 as it is as transmission data F2X (SP1) (intra coding), and is the same processing as in the case of an I picture. The second process is to calculate a difference from the temporally subsequent frame F3 and transmit the difference (SP2) (backward predictive coding). The third process is to transmit a difference (SP3) from the temporally preceding frame F1 (forward prediction coding). Further, the fourth process is to generate a difference (SP4) between the average value of the temporally preceding frame F1 and the succeeding frame F3 and transmit this as transmission data F2X (bidirectional predictive coding). .
[0209]
In practice, the method that minimizes the transmission data among the four methods described above is employed.
[0210]
Note that when transmitting difference data, a motion vector x1 (motion vector between frames F1 and F2) between the frame image (reference image) whose difference is to be calculated (in the case of forward prediction), or x2 (Motion vector between frames F3 and F2) (for backward prediction) or both x1 and x2 (for bidirectional prediction) are transmitted along with the difference data.
[0211]
Also, the frame F3 of the P picture is obtained by calculating a difference signal (SP3) from this frame and the motion vector x3 using the frame F1 temporally preceding as a reference image, and transmitting this as transmission data F3X (forward prediction). Coding). Alternatively, the data of the original frame F3 is transmitted as it is as data F3X (SP1) (intra coding). Among these methods, as in the case of the B picture, a method with less transmission data is selected.
[0212]
FIG. 5 shows an example of the configuration of an apparatus that encodes and transmits a moving image signal and decodes it based on the principle described above. The encoding device 1 encodes an input video signal and transmits it to a recording medium 3 as a transmission path. The decoding device 2 reproduces the signal recorded on the recording medium 3, decodes it, and outputs it.
[0213]
In the encoding device 1, the input video signal is input to the preprocessing circuit 11, where the luminance signal and the color signal (in this embodiment, the color difference signal) are separated, and each of the A / D converter 12, 13 converts the analog signal into a digital signal. The video signals converted into digital signals by the A / D converters 12 and 13 are supplied to the frame memory 14 and stored therein. The frame memory 14 stores the luminance signal in the luminance signal frame memory 15 and the color difference signal in the color difference signal frame memory 16, respectively.
[0214]
The format conversion circuit 17 converts the frame format signal stored in the frame memory 14 into a block format signal. That is, as shown in FIG. 6, the video signal stored in the frame memory 14 is frame format data as shown in FIG. 6A in which V lines of H dots are collected per line. Yes. The format conversion circuit 17 divides the signal of one frame into M slices in units of 16 lines as shown in FIG. 6B. Each slice is divided into M macroblocks. As shown in FIG. 6C, the macro block is composed of a luminance signal corresponding to 16 × 16 pixels (dots), and this luminance signal is further divided into blocks Y [1 in units of 8 × 8 dots. ] To Y [4]. The 16 × 16 dot luminance signal corresponds to an 8 × 8 dot Cb signal and an 8 × 8 dot Cr signal.
[0215]
In this way, the data converted into the block format is supplied from the format conversion circuit 17 to the encoder 18 where encoding (encoding) is performed. Details thereof will be described later with reference to FIG.
[0216]
The signal encoded by the encoder 18 is output to the transmission path as a bit stream. For example, it is supplied to the recording circuit 19 and recorded on the recording medium 3 as a digital signal.
[0217]
Data reproduced from the recording medium 3 by the reproduction circuit 30 of the decoding device 2 is supplied to the decoder 31 and decoded. Details of the decoder 31 will be described later with reference to FIG.
[0218]
The data decoded by the decoder 31 is input to the format conversion circuit 32 and converted from the block format to the frame format. The luminance signal in the frame format is supplied to and stored in the luminance signal frame memory 34 of the frame memory 33, and the color difference signal is supplied to and stored in the color difference signal frame memory 35. The luminance signal and the color difference signal read from the luminance signal frame memory 34 and the color difference signal frame memory 35 are converted into analog signals by the D / A converters 36 and 37, respectively, and supplied to the post-processing circuit 38. The post-processing circuit 38 synthesizes and outputs the luminance signal and the color difference signal.
[0219]
Next, the configuration of the encoder 18 will be described with reference to FIG. The encoded image data is input to the motion vector detection circuit 50 in units of macro blocks. The motion vector detection circuit 50 processes the image data of each frame as an I picture, P picture, or B picture according to a predetermined sequence set in advance. It is predetermined whether the image of each frame that is sequentially input is processed as an I, P, or B picture (for example, as shown in FIGS. 2 and 3, frames F1 to F17). Are processed as I, B, P, B, P,... B, P).
[0220]
Image data of a frame processed as an I picture (for example, frame F1) is transferred from the motion vector detection circuit 50 to the front original image portion 51a of the frame memory 51, stored, and processed as a B picture (for example, a frame) The image data of F2) is transferred and stored in the original image portion 51b, and the image data of a frame (for example, frame F3) processed as a P picture is transferred and stored in the rear original image portion 51c.
[0221]
At the next timing, when an image of a frame to be further processed as a B picture (frame F4) or a P picture (frame F5) is input, the first P picture stored in the rear original image portion 51c until then. The image data of (frame F3) is transferred to the forward original image portion 51a, the image data of the next B picture (frame F4) is stored (overwritten) in the reference original image portion 51b, and the next P picture (frame F5) ) Is stored (overwritten) in the rear original image portion 51c. Such an operation is sequentially repeated.
[0222]
The signal of each picture stored in the frame memory 51 is read therefrom, and the prediction mode switching circuit 52 performs frame prediction mode processing or field prediction mode processing.
[0223]
Furthermore, under the control of the prediction determination circuit 54, the arithmetic unit 53 performs intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Which of these processes is performed is determined corresponding to a prediction error signal (a difference between a reference image to be processed and a predicted image corresponding thereto). For this reason, the motion vector detection circuit 50 generates the absolute value sum (or sum of squares) of the prediction error signal used for this determination.
[0224]
Here, the frame prediction mode and the field prediction mode in the prediction mode switching circuit 52 will be described.
[0225]
When the frame prediction mode is set, the prediction mode switching circuit 52 supplies the four luminance blocks Y [1] to Y [4] supplied from the motion vector detection circuit 50 to the arithmetic unit 53 in the subsequent stage. Output. In other words, in this case, as shown in FIG. 8, the data of the odd-numbered field lines and the data of the even-numbered field lines are mixed in each luminance block. In this frame prediction mode, prediction is performed in units of four luminance blocks (macro blocks), and one motion vector corresponds to the four luminance blocks.
[0226]
On the other hand, in the field prediction mode, the prediction mode switching circuit 52 receives a signal input from the motion vector detection circuit 50 with the configuration shown in FIG. 8 among four luminance blocks as shown in FIG. The luminance blocks Y [1] and Y [2] are composed of, for example, only odd-field line dots, and the other two luminance blocks Y [3] and Y [4] are even-field line dots. And output to the arithmetic unit 53. In this case, one motion vector corresponds to the two luminance blocks Y [1] and Y [2], and the other two luminance blocks Y [3] and Y [4]. Thus, one other motion vector is associated.
[0227]
The motion vector detection circuit 50 outputs the sum of absolute values of prediction errors in the frame prediction mode and the sum of absolute values of prediction errors in the field prediction mode to the prediction mode switching circuit 52. The prediction mode switching circuit 52 compares the absolute value sum of the prediction errors in the frame prediction mode and the field prediction mode, performs a process corresponding to the prediction mode having a small value, and outputs the data to the calculator 53.
[0228]
However, such processing is actually performed by the motion vector detection circuit 50. That is, the motion vector detection circuit 50 outputs a signal having a configuration corresponding to the determined mode to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 outputs the signal as it is to the arithmetic unit 53 in the subsequent stage.
[0229]
In the frame prediction mode, the color difference signal is supplied to the computing unit 53 in a state where odd-numbered-field line data and even-numbered-field line data are mixed, as shown in FIG. In the case of the field prediction mode, as shown in FIG. 9, the upper half (four lines) of the color difference blocks Cb and Cr are the odd field color difference signals corresponding to the luminance blocks Y [1] and Y [2]. The lower half (four lines) is the color difference signal of the even field corresponding to the luminance blocks Y [3] and Y [4].
[0230]
In addition, the motion vector detection circuit 50 uses a prediction error for determining whether to perform intra prediction, forward prediction, backward prediction, or bidirectional prediction in the prediction determination circuit 54 as shown below. Generate sum of absolute values.
[0231]
That is, as the sum of absolute values of prediction errors of intra-picture prediction, the absolute value | ΣAij | of the sum ΣAij of the macroblock signal Aij of the reference image and the sum Σ | Aij | of the absolute value | Aij | of the macroblock signal Aij Find the difference. Further, as the absolute value sum of the prediction errors of the forward prediction, the sum Σ | Aij− of the absolute value | Aij−Bij | of the difference Aij−Bij between the macroblock signal Aij of the reference image and the macroblock signal Bij of the predicted image Bij | is obtained. Also, the absolute value sum of the prediction errors of the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the prediction image to a prediction image different from that in the forward prediction).
[0232]
These absolute value sums are supplied to the prediction determination circuit 54. The prediction determination circuit 54 selects the smallest one of the absolute value sums of the prediction errors of the forward prediction, the backward prediction, and the bidirectional prediction as the absolute value sum of the prediction errors of the inter prediction. Further, the absolute value sum of the prediction error of the inter prediction and the absolute value sum of the prediction error of the intra prediction are compared, and the smaller one is selected, and the mode corresponding to the selected absolute value sum is set as the prediction mode. select. That is, if the sum of the absolute values of the prediction errors of intra prediction is smaller, the intra prediction mode is set. If the absolute value sum of the prediction errors of inter prediction is smaller, the mode with the smallest corresponding absolute value sum among the forward prediction, backward prediction, and bidirectional prediction modes is set.
[0233]
As described above, the motion vector detection circuit 50 has the configuration corresponding to the mode selected by the prediction mode switching circuit 52 out of the frame or field prediction modes for the signal of the macroblock of the reference image. The motion vector between the prediction image corresponding to the prediction mode selected by the prediction determination circuit 54 of the four prediction modes and the reference image is detected, and the variable-length encoding circuit 58 is detected. And output to the motion compensation circuit 64. As described above, the motion vector having the minimum absolute value sum of the corresponding prediction errors is selected.
[0234]
When the motion vector detection circuit 50 reads I-picture image data from the front original image portion 51a, the prediction determination circuit 54 uses a frame or field (image) prediction mode (a mode in which motion compensation is not performed) as a prediction mode. And the switch 53d of the computing unit 53 is switched to the contact a side. As a result, the I-picture image data is input to the DCT mode switching circuit 55.
[0235]
As shown in FIG. 10 or FIG. 11, the DCT mode switching circuit 55 is configured to separate the data of four luminance blocks in a state where odd-numbered field lines and even-numbered field lines coexist (frame DCT mode). Any one of the states (field DCT mode) is output to the DCT circuit 56.
[0236]
That is, the DCT mode switching circuit 55 compares the coding efficiency when DCT processing is performed with mixed odd-numbered field data and even-numbered field data, and the coding efficiency when DCT processing is performed in a separated state. Choose a good mode.
[0237]
For example, as shown in FIG. 10, the input signal has a configuration in which odd-numbered field and even-numbered field lines coexist, and calculates the difference between the signal of the odd-numbered line and the even-numbered line adjacent to each other. Further, the sum (or sum of squares) of the absolute values is obtained.
[0238]
Further, as shown in FIG. 11, the input signal has a configuration in which the odd-numbered field and even-numbered field lines are separated, and the signal difference between the odd-numbered adjacent field lines and the signals of the even-numbered line lines are separated. The sum of absolute values (or the sum of squares) of each is calculated.
[0239]
Furthermore, both (absolute value sum) are compared, and a DCT mode corresponding to a small value is set. That is, if the former is smaller, the frame DCT mode is set, and if the latter is smaller, the field DCT mode is set.
[0240]
Then, data having a configuration corresponding to the selected DCT mode is output to the DCT circuit 56, and a DCT flag indicating the selected DCT mode is output to the variable length encoding circuit 58 and the motion compensation circuit 64.
[0241]
As is apparent from a comparison between the prediction mode (FIGS. 8 and 9) in the prediction mode switching circuit 52 and the DCT mode (FIGS. 10 and 11) in the DCT mode switching circuit 55, the luminance block has both of them. The data structure in the mode is substantially the same.
[0242]
When the frame prediction mode (mode in which odd lines and even lines are mixed) is selected in the prediction mode switching circuit 52, the frame DCT mode (mode in which odd lines and even lines are mixed) also in the DCT mode switching circuit 55. When the prediction mode switching circuit 52 selects the field prediction mode (the mode in which the data of the odd field and the even field are separated), the DCT mode switching circuit 55 selects the field DCT mode ( There is a high possibility that the mode in which the data of the odd field and the even field are separated is selected.
[0243]
However, the mode is not always selected in this way, and the prediction mode switching circuit 52 determines the mode so that the absolute value sum of the prediction errors is small, and the DCT mode switching circuit 55 performs the encoding. The mode is determined so that the efficiency is good.
[0244]
The I-picture image data output from the DCT mode switching circuit 55 is input to the DCT circuit 56, subjected to DCT processing, and converted into DCT coefficients. This DCT coefficient is input to the quantization circuit 57, quantized with a quantization scale corresponding to the data storage amount (buffer storage amount) of the transmission buffer 59, and then input to the variable length encoding circuit 58.
[0245]
The variable length encoding circuit 58 corresponds to the quantization scale (scale) supplied from the quantization circuit 57, and converts image data (in this case, I picture data) supplied from the quantization circuit 57, for example, It is converted into a variable length code such as a Huffman code and output to the transmission buffer 59.
[0246]
The variable length encoding circuit 58 also determines whether a quantization scale (scale) is set by the quantization circuit 57 and a prediction mode (intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction) is set by the prediction determination circuit 54. Mode), a motion vector from the motion vector detection circuit 50, a prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode is set) from the prediction mode switching circuit 52, and the DCT output from the DCT mode switching circuit 55 A flag (a flag indicating whether the frame DCT mode or the field DCT mode is set) is input, and these are also variable-length encoded.
[0247]
The transmission buffer 59 temporarily stores input data and outputs data corresponding to the storage amount to the quantization circuit 57. When the remaining data amount increases to the allowable upper limit value, the transmission buffer 59 increases the quantization scale of the quantization circuit 57 by the quantization control signal, thereby reducing the data amount of the quantized data. On the other hand, when the remaining data amount decreases to the allowable lower limit value, the transmission buffer 59 reduces the quantization scale of the quantization circuit 57 by the quantization control signal, thereby reducing the data amount of the quantized data. Increase. In this way, overflow or underflow of the transmission buffer 59 is prevented.
[0248]
The data stored in the transmission buffer 59 is read at a predetermined timing, output to the transmission path, and recorded on the recording medium 3 via the recording circuit 19, for example.
[0249]
On the other hand, the I picture data output from the quantization circuit 57 is input to the inverse quantization circuit 60 and inversely quantized in accordance with the quantization scale supplied from the quantization circuit 57. The output of the inverse quantization circuit 60 is input to an IDCT (Inverse Discrete Cosine Transform) circuit 61, subjected to inverse discrete cosine transform processing, and then supplied to and stored in the forward prediction image unit 63a of the frame memory 63 via the calculator 62. Is done.
[0250]
The motion vector detection circuit 50, when processing the image data of each frame sequentially input as, for example, pictures of I, B, P, B, P, B. After the image data is processed as an I picture, the image data of the next input frame is further processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture is accompanied by backward prediction, and therefore cannot be decoded unless a P picture as a backward predicted image is prepared first.
[0251]
Therefore, the motion vector detection circuit 50 starts processing the image data of the P picture stored in the rear original image portion 51c after the processing of the I picture. As in the case described above, the absolute value sum of the inter-frame difference (prediction error) in units of macroblocks is supplied from the motion vector detection circuit 50 to the prediction mode switching circuit 52 and the prediction determination circuit 54. The prediction mode switching circuit 52 and the prediction determination circuit 54 correspond to the absolute value sum of the prediction errors of the macroblock of this P picture, or the frame / field prediction mode, or intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction. Set the prediction mode.
[0252]
When the in-picture prediction mode is set, the computing unit 53 switches the switch 53d to the contact a side as described above. Accordingly, this data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59, similarly to the I picture data. Further, this data is supplied to and stored in the backward predicted image unit 63b of the frame memory 63 via the inverse quantization circuit 60, the IDCT circuit 61, and the calculator 62.
[0253]
When the forward prediction mode is set, the switch 53d is switched to the contact point b, and image (in this case, an I picture image) data stored in the forward prediction image portion 63a of the frame memory 63 is read. Then, the motion compensation circuit 64 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 50. That is, the motion compensation circuit 64, when the setting of the forward prediction mode is instructed from the prediction determination circuit 54, the read address of the forward prediction image unit 63a, and the macroblock currently being output by the motion vector detection circuit 50. Data is read out from the position corresponding to the position by an amount corresponding to the motion vector, and predicted image data is generated.
[0254]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53a. The computing unit 53a subtracts the prediction image data corresponding to the macroblock supplied from the motion compensation circuit 65 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and the difference (prediction error). ) Is output. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59. The difference data is locally decoded by the inverse quantization circuit 60 and the IDCT circuit 61 and input to the calculator 62.
[0255]
The calculator 62 is also supplied with the same data as the predicted image data supplied to the calculator 53a. The calculator 62 adds the predicted image data output from the motion compensation circuit 64 to the difference data output from the IDCT circuit 61. As a result, image data of the original (decoded) P picture is obtained. The image data of the P picture is supplied to and stored in the backward predicted image unit 63b of the frame memory 63.
[0256]
As described above, the motion vector detection circuit 50 stores the data of the I picture and the P picture in the forward predicted image unit 63a and the backward predicted image unit 63b, respectively, and then executes the process of the B picture. The prediction mode switching circuit 52 and the prediction determination circuit 54 set a frame / field mode corresponding to the magnitude of the absolute value sum of inter-frame differences in units of macroblocks, and set the prediction mode to an intra-picture prediction mode, Set to either the forward prediction mode, backward prediction mode, or bidirectional prediction mode.
[0257]
As described above, in the intra prediction mode or the forward prediction mode, the switch 53d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and data is transmitted.
[0258]
On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 53d is switched to the contact c or d, respectively.
[0259]
In the backward prediction mode in which the switch 53d is switched to the contact point c, the image data (in this case, a P picture image) stored in the backward prediction image unit 63b is read out, and the motion compensation circuit 64 Motion compensation is performed corresponding to the motion vector output from the vector detection circuit 50. That is, the motion compensation circuit 64, when the setting of the backward prediction mode is instructed by the prediction determination circuit 54, the read address of the backward prediction image unit 63b, the macro block currently output by the motion vector detection circuit 50. Data is read out from the position corresponding to the position by an amount corresponding to the motion vector, and predicted image data is generated.
[0260]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53b. The computing unit 53b subtracts the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the prediction mode switching circuit 52, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.
[0261]
In the bidirectional prediction mode in which the switch 53d is switched to the contact point d, the image (in this case, the image of I picture) data stored in the forward predicted image unit 63a and the backward predicted image unit 63b are stored. Image (in this case, P picture image) data is read out, and motion compensation is performed by the motion compensation circuit 64 corresponding to the motion vector output from the motion vector detection circuit 50.
[0262]
That is, in the motion compensation circuit 64, when the setting of the bidirectional prediction mode is instructed by the prediction determination circuit 54, the motion vector detection circuit 50 now outputs the read addresses of the forward prediction image unit 63a and the backward prediction image unit 63b. The data is read out from the position corresponding to the position of the macroblock being read by shifting the motion vector by the amount corresponding to the motion vector (in this case, the motion vector is for the forward prediction image and the backward prediction image), and prediction image data is generated To do.
[0263]
The predicted image data output from the motion compensation circuit 64 is supplied to the calculator 53c. The computing unit 53c subtracts the average value of the predicted image data supplied from the motion compensation circuit 64 from the macroblock data of the reference image supplied from the motion vector detection circuit 50, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 55, the DCT circuit 56, the quantization circuit 57, the variable length coding circuit 58, and the transmission buffer 59.
[0264]
The B picture image is not stored in the frame memory 63 because it is not a predicted image of another image.
[0265]
Note that, in the frame memory 63, the forward predicted image unit 63a and the backward predicted image unit 63b are subjected to bank switching as necessary, and those stored in one or the other with respect to a predetermined reference image, It can be switched and output as a predicted image or a backward predicted image.
[0266]
In the above description, the luminance block is mainly described, but the color difference block is also processed and transmitted in units of macroblocks shown in FIGS. Note that the motion vector when processing the color difference block is obtained by halving the motion vector of the corresponding luminance block in the vertical direction and the horizontal direction, respectively.
[0267]
FIG. 12 is a block diagram showing a configuration of the decoder 31 of FIG. The encoded image data transmitted through the transmission path (recording medium 3) is received by a receiving circuit (not shown), reproduced by a reproducing device, temporarily stored in the receiving buffer 81, and then decoded by a decoding circuit 90. To the variable length decoding circuit 82. The variable length decoding circuit 82 performs variable length decoding on the data supplied from the reception buffer 81, outputs a motion vector, a prediction mode, a prediction flag, and a DCT flag to the motion compensation circuit 87, and dequantizes the quantization scale. The decoded image data is output to the inverse quantization circuit 83.
[0268]
The inverse quantization circuit 83 inversely quantizes the image data supplied from the variable length decoding circuit 82 according to the quantization scale supplied from the variable length decoding circuit 82 and outputs the result to the IDCT circuit 84. The data (DCT coefficient) output from the inverse quantization circuit 83 is subjected to inverse discrete cosine transform processing by the IDCT circuit 84 and supplied to the computing unit 85.
[0269]
When the image data supplied from the IDCT circuit 84 to the computing unit 85 is I picture data, the data is output from the computing unit 85, and image data (P or B picture data that is input later to the computing unit 85). ) Is supplied to and stored in the forward predicted image portion 86a of the frame memory 86. The data is output to the format conversion circuit 32 (FIG. 5).
[0270]
When the image data supplied from the IDCT circuit 84 is P picture data having the image data of the previous frame as predicted image data and is data in the forward prediction mode, the image data is stored in the forward predicted image unit 86a of the frame memory 86. The stored image data of the previous frame (I picture data) is read out, and motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 is performed by the motion compensation circuit 87. The calculator 85 adds the image data (difference data) supplied from the IDCT circuit 84 and outputs the result. This added data, that is, the decoded P picture data is stored in the rear of the frame memory 86 in order to generate predicted picture data of the picture data (B picture or P picture data) to be input later to the calculator 85. The predicted image unit 86b is supplied and stored.
[0271]
Even in the case of P picture data, the intra prediction mode data is not processed by the computing unit 85 and stored in the backward predicted image unit 86b as is the case with the I picture data.
[0272]
Since this P picture is an image to be displayed next to the next B picture, at this point of time, it is not yet output to the format conversion circuit 32 (as described above, the P picture input after the B picture is Processed and transmitted before B picture).
[0273]
When the image data supplied from the IDCT circuit 84 is B picture data, the I stored in the forward predicted image unit 86 a of the frame memory 86 corresponding to the prediction mode supplied from the variable length decoding circuit 82. Image data of a picture (in the case of forward prediction mode), image data of a P picture stored in the backward prediction image unit 86b (in the case of backward prediction mode), or both of them (in the case of bidirectional prediction mode) The motion compensation circuit 87 performs the motion compensation corresponding to the motion vector output from the variable length decoding circuit 82 and generates a predicted image. However, when motion compensation is not required (in the case of intra-picture prediction mode), a predicted image is not generated.
[0274]
In this way, the data subjected to motion compensation by the motion compensation circuit 87 is added to the output of the IDCT circuit 84 in the computing unit 85. This addition output is output to the format conversion circuit 32.
[0275]
However, since this addition output is B picture data and is not used for generating a predicted image of another image, it is not stored in the frame memory 86.
[0276]
After the B picture image is output, the P picture image data stored in the backward predicted image unit 86 b is read out and supplied to the computing unit 85 via the motion compensation circuit 87. However, at this time, motion compensation is not performed.
[0277]
The decoder 31 does not show circuits corresponding to the prediction mode switching circuit 52 and the DCT mode switching circuit 55 in the encoder 18 of FIG. 5, but processes corresponding to these circuits, that is, odd fields and even numbers. The motion compensation circuit 87 executes a process of returning the configuration in which the signal of the field line is separated to the original configuration as necessary.
[0278]
In the above description, the luminance signal processing has been described, but the color difference signal processing is performed in the same manner. However, the motion vector in this case is obtained by halving the luminance signal motion vector in the vertical and horizontal directions.
[0279]
FIG. 13 shows the quality of the encoded image. Image quality (SNR: Signal to Noise Ratio) is controlled according to the picture type, I picture and P picture are of high quality, and B picture is inferior to I and P pictures. Is transmitted. This is a technique using human visual characteristics, and the visual image quality is better when the quality is oscillated than when all the image qualities are averaged. The image quality control corresponding to this picture type is executed by the quantization circuit 57 of FIG.
[0280]
14 and 15 show the configuration of the transcoder 101 to which the present invention is applied, and FIG. 15 shows the more detailed configuration of FIG. The transcoder 101 converts the GOP structure and bit rate of the encoded video bit stream input to the decoding apparatus 102 into the GOP structure and bit rate desired by the operator. In order to explain the function of the transcoder 101, three transcoders having substantially the same function as the transcoder 101 are connected to the front stage of the transcoder 101, although not shown in FIG. 15. It shall be. That is, in order to variously change the GOP structure and bit rate of the bit stream, the first transcoder, the second transcoder, and the third transcoder are connected in series, and the third transcoder's It is assumed that the fourth transcoder shown in FIG. 15 is connected behind.
[0281]
In the following description of the present invention, the encoding process performed in the first transcoder is defined as the first generation encoding process, and the second transcoder connected after the first transcoder. The encoding process performed is defined as the second generation encoding process, and the encoding process performed in the third transcoder connected after the second transcoder is defined as the third generation encoding process. The encoding process performed in the fourth transcoder (transcoder 101 shown in FIG. 15) defined and connected after the third transcoder is the fourth generation encoding process or the current encoding process. We will define
[0282]
In addition, the encoding parameter generated in the first generation encoding process is referred to as a first generation encoding parameter, and the encoding parameter generated in the second generation encoding process is referred to as a second generation encoding parameter. The encoding parameter generated in the third generation encoding process is referred to as the third generation encoding parameter, and the encoding parameter generated in the fourth generation encoding process is referred to as the fourth generation encoding parameter or This is called the current encoding parameter.
[0283]
First, the encoded video stream ST (3rd) supplied to the transcoder 101 shown in FIG. 15 will be described. ST (3rd) represents a third generation encoded stream generated in the third generation encoding process in the third transcoder provided in the preceding stage of the transcoder 101. In the encoded video stream ST (3rd) generated in the third generation encoding process, the third generation encoding parameter generated in the third encoding process includes the encoded video stream ST (3rd ) Sequence layer, GOP layer, picture layer, slice layer, and macroblock layer, sequence_header () function, sequence_extension () function, group_of_pictures_header () function, picture_header () function, picture_coding_extension () function, picture_data () function, It is described as slice () function and macroblock () function. The description of the third encoding parameter used in the third encoding process in the third encoded stream generated by the third encoding process is defined in the MPEG2 standard. There is no novelty.
[0284]
The unique point in the transcoder 101 of the present invention is that not only the third encoding parameter is described in the third encoded stream ST (3rd) but also the first generation and second generation encodings. The first generation and second generation encoding parameters generated in the process are also described.
[0285]
Specifically, the first generation and second generation encoding parameters are described as a history stream history_stream () in the user data area of the picture layer of the third generation encoded video stream ST (3rd). . In the present invention, the history stream described in the user data area of the picture layer of the third generation encoded video stream ST (3rd) is called “history information” or “history information”. The described encoding parameters are called “history parameters” or “history parameters”.
[0286]
Alternatively, if the third generation encoding parameter described in the third generation encoded stream ST (3rd) is called “current encoding parameter”, the third generation code In view of the encoding process, the first generation and second generation encoding processes are encoding processes performed in the past, and are therefore described in the user data area of the picture layer of the third generation encoded stream ST (3rd). The encoding parameter described as a history stream is also called “past encoding parameter”.
[0287]
Thus, not only the third encoding parameter is described in the third encoded stream ST (3rd) but also the first generation generated in the first generation and second generation encoding processes. The reason why the second generation encoding parameters are described is that even if the GOP structure and bit rate of the encoded stream are repeatedly changed by the transcoding process, the image quality can be prevented from being deteriorated.
[0288]
For example, a picture is encoded as a P picture in the first generation encoding process, and the picture is encoded as a B picture in the second generation encoding process in order to change the GOP structure of the first generation encoded stream. In order to further change the GOP structure of the second generation encoded stream, it may be possible to encode the picture again as a P picture in the third generation encoding process. Since encoding processing and decoding processing based on the MPEG standard are not 100% reversible processing, it is known that image quality deteriorates every time encoding and decoding processing is repeated.
[0289]
In such a case, in the third generation encoding process, the encoding parameters such as the quantization scale, the motion vector, and the prediction mode are not calculated again, but generated in the first generation encoding process. Reuse coding parameters such as quantization scale, motion vector, and prediction mode. Compared to encoding parameters such as quantization scale, motion vector, and prediction mode newly generated by the third generation encoding process, quantization scale, motion vector, and prediction mode newly generated by the first generation encoding process. Since the encoding parameters such as are clearly more accurate, the image quality degradation can be reduced by reusing the first generation parameters even if the encoding and decoding processes are repeated.
[0290]
In order to describe the processing according to the present invention described above, the processing of the fourth generation transcoder 101 shown in FIG. 15 will be described in more detail as an example.
[0291]
The decoding apparatus 102 decodes the encoded video included in the third generation encoded bitstream ST (3rd) using the third generation encoding parameter, and decodes the decoded baseband digital video data. It is a device for generating. Further, the decoding apparatus 102 decodes the first generation and second generation encoding parameters described as the history stream in the user data area of the picture layer of the third generation encoded bit stream ST (3rd). It is also a device.
[0292]
Specifically, as shown in FIG. 16, the decoding device 102 has basically the same configuration as the decoder 31 (FIG. 12) of the decoding device 2 in FIG. 5, and buffers the supplied bit stream. A reception buffer 81 for ringing, a variable-length decoding circuit 112 for variable-length decoding the encoded bitstream, and an inverse quantization unit that inverse-quantizes the variable-length decoded data according to the quantization scale supplied from the variable-length decoding circuit 112 A quantization circuit 83, an IDCT circuit 84 that performs inverse discrete cosine transform on the inversely quantized DCT coefficients, an arithmetic unit 85 for performing motion compensation processing, a frame memory 86, and a motion compensation circuit 87 are provided.
[0293]
The variable length decoding circuit 112 performs decoding processing on the third generation encoded bitstream ST (3rd) in the picture layer, slice layer, and macroblock layer of the third generation encoded bitstream ST (3rd). The described third generation encoding parameters are extracted. For example, the third generation encoding parameters extracted in the variable length decoding circuit 112 are picture_coding_type indicating the picture type, quantizer_scale_code indicating the quantization scale step size, macroblock_type indicating the prediction mode, motion_vector indicating the motion vector, and frame prediction. Frame / field_motion_type indicating the mode or the field prediction mode, dct_type indicating the frame DCT mode or the field DCT mode, or the like. The quatntiser_scale_code extracted by the variable length decoding circuit 112 is supplied to the inverse quantization circuit 83, and parameters such as picture_coding_type, quatntiser_scale_code, macroblock_type, motion_vector, frame / field_motion_type, and dct_type are supplied to the motion compensation circuit 87.
[0294]
The variable length decoding circuit 112 not only provides these encoding parameters necessary for decoding the third generation encoded bit stream ST (3rd), but also adds the third generation history to the subsequent fifth generation transcoder. Coding parameters to be transmitted as information are extracted from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third generation coded bitstream ST (3rd). Of course, third-generation encoding parameters such as picture_coding_type, quatntiser_scale_code, macroblock_type, motion_vector, frame / field_motion_type, and dct_type used for the third-generation decoding process are included in the third-generation history information. The encoding parameters to be extracted as history information are set in advance by the operator or the host computer according to the transmission capacity.
[0295]
Further, the variable length decoding circuit 112 extracts user data described in the user data area of the picture layer of the third generation encoded bit stream ST (3rd) and supplies the user data to the history decoding device 104. To do.
[0296]
The history decoding apparatus 104 uses the first generation encoding parameters and second generation described as history information from user data described in the picture layer of the third generation encoded bitstream ST (3rd). This is a circuit for extracting the encoding parameter (the encoding parameter of the generation before the previous generation). Specifically, the history decoding apparatus 104 detects the unique History_Data_Id described in the user data by analyzing the syntax of the received user data, and thereby extracts converted_history_stream (). be able to. Furthermore, the history decoding apparatus 104 obtains history_stream () by taking 1 bit of marker bit (marker_bit) inserted at a predetermined interval in converted_history_stream (), and then obtains the history_stream () synth. By analyzing the tax, the first generation and second generation encoding parameters described in the history_stream () can be obtained. Detailed operation of the history decoding device 104 will be described later.
[0297]
The history information multiplexing apparatus 103 is decoded by the decoding apparatus 102 to supply the first generation, second generation, and third generation encoding parameters to the encoding apparatus 106 that performs the fourth generation encoding process. This is a circuit for multiplexing the first generation, second generation and third generation encoding parameters into the baseband video data. Specifically, the history information multiplexing apparatus 103 includes baseband video data output from the arithmetic unit 85 of the decoding apparatus 102, and third-generation encoding parameters output from the variable length decoding apparatus 112 of the decoding apparatus 102. , And the first generation encoding parameter and the second generation encoding parameter output from the history decoding device 104 are received, and the first generation, second generation, and second generation parameters are received in the baseband video data. Multiplex the 3rd generation encoding parameters. Baseband video data in which the first generation, second generation, and third generation encoding parameters are multiplexed is supplied to the history information separation apparatus 105 via a transmission cable.
[0298]
Next, a method of multiplexing the first generation, second generation, and third generation encoding parameters into the baseband video data will be described with reference to FIGS. FIG. 17 shows one macroblock of 16 pixels × 16 pixels defined in the MPEG standard. This macro block of 16 pixels × 16 pixels has four sub-blocks (Y [0], [1], [2] and Y [3]) consisting of four 8 pixels × 8 pixels with respect to the luminance signal, and a color difference signal. Consists of four sub-blocks (Cr [0], r [1], b [0], and Cb [1]) each consisting of 8 pixels × 8 pixels.
[0299]
FIG. 18 shows a format of video data. This format is a format defined in the ITU recommendation-RDT 601 and represents a so-called “D1 format” used in the broadcasting industry. Since this D1 format has been standardized as a format for transmitting 10-bit video data, one pixel of the video data can be expressed by 10 bits.
[0300]
Since the baseband video data decoded according to the MPEG standard is 8 bits, in the transcoder of the present invention, as shown in FIG. 18, the upper 8 bits (D9 to D2) out of 10 bits of the D1 format are used. It is used to transmit baseband video data decoded based on the MPEG standard. As described above, when the decoded 8-bit video data is written in the D1 format, the lower 2 bits (D1 and D0) become unallocated bits. In the transcoder of the present invention, history information is transmitted using this unallocated area.
[0301]
The data block shown in FIG. 18 includes sub-blocks (Y [0], Y [1], Y [2], Y [3], Cr [0], Cr [1], Cb [0], Since this is a data block for transmitting one pixel in Cb [1]), 64 data blocks shown in FIG. 18 are transmitted to transmit one macroblock data. If the lower 2 bits (D1 and D0) are used, a total of 1024 (= 16 × 64) bits of history information can be transmitted for one macroblock of video data. Accordingly, since history information for one generation is generated to be 256 bits, history information for the past 4 (= 1024/256) generations may be superimposed on video data of one macroblock. it can. In the example shown in FIG. 18, the first generation history information, the second generation history information, and the third generation history information are superimposed.
[0302]
The history information separation device 105 is a circuit for extracting baseband video data from the upper 8 bits of data transmitted as the D1 format and extracting history information from the lower 2 bits. In the example illustrated in FIG. 15, the history information separation device 105 extracts baseband video data from transmission data, supplies the video data to the encoding device 106, and generates first and second generations from the transmission data. The generation and third generation history information is extracted and supplied to the encoding device 106 and the history encoding device 107, respectively.
[0303]
The encoding device 106 is a device for encoding the baseband video data supplied from the history information separation device 105 into a bit stream having a GOP structure and a bit rate specified by an operator or a host computer. is there. Note that changing the GOP structure means, for example, the number of pictures included in the GOP, the number of P pictures existing between I pictures and I pictures, and between I pictures and P pictures (or I pictures). This means that the number of B pictures is changed.
[0304]
In the example shown in FIG. 15, since the history information of the first generation, the second generation, and the third generation is superimposed on the supplied baseband video data, the encoding device 106 performs re-encoding. The history information is selectively reused to perform the fourth generation encoding process so that the image quality degradation due to the conversion process is reduced.
[0305]
FIG. 19 is a diagram illustrating a specific configuration of the encoder 121 provided in the encoding device 106. The encoder 121 is basically configured in the same manner as the encoder 18 shown in FIG. 7, and includes a motion vector detection circuit 50, a frame / field prediction mode switching circuit 52, a calculator 53, a DCT mode switching circuit 55, and a DCT circuit. 56, a quantization circuit 57, a variable length coding circuit 58, a transmission buffer 59, an inverse quantization circuit 60, an inverse DCT circuit 61, an arithmetic unit 62, a frame memory 63, and a motion compensation circuit 64. The functions of these circuits are almost the same as those in the encoder 18 described with reference to FIG. The following description will focus on differences between the encoder 121 and the encoder 18 described with reference to FIG.
[0306]
The encoder 121 has a controller 70 for controlling the operation and function of each circuit described above. The controller 70 receives instructions regarding the GOP structure from the operator or the host computer, and determines the picture type of each picture so as to correspond to the GOP structure. The controller 70 receives information on the target bit rate from the operator or the host computer, and controls the quantization circuit 57 so that the bit rate output from the encoder 121 becomes the specified target bit rate. To do.
[0307]
Further, the controller 70 receives history information of a plurality of generations output from the history information separation device 105, and performs reference picture encoding processing by reusing these history information. This will be described in detail below.
[0308]
First, the controller 70 determines whether or not the picture type of the reference picture determined from the GOP structure designated by the operator matches the picture type included in the history information. That is, it is determined whether or not this reference picture has been encoded in the past with the same picture type as the designated picture type.
[0309]
If the example shown in FIG. 15 is used for easier understanding, the controller 70 determines that the picture type assigned to the reference picture is the first generation encoding process as the fourth generation encoding process. It is determined whether or not the picture type of the reference picture in the second generation encoding process matches the picture type of the reference picture in the second generation encoding process or the picture type of the reference picture in the third generation encoding process.
[0310]
If the picture type designated as the reference picture as the fourth generation encoding process does not match any picture type in the past encoding process, the controller 70 performs the “normal encoding process”. . That is, in this case, in any of the first generation, second generation, or third generation encoding processing, this reference picture is encoded using the picture type assigned as the fourth generation encoding processing. It has never been done. On the other hand, if the picture type specified for the reference picture as the fourth generation encoding process matches any picture type in the past encoding process, the controller 70 determines “parameter reuse”. Encoding process "is performed. In other words, in this case, the reference picture is encoded with the picture type assigned as the fourth generation encoding process in the first generation, second generation, or third generation encoding process. It means that it has been processed.
[0311]
First, the normal encoding process of the controller 70 will be described.
[0312]
The motion vector detection circuit 50 detects the prediction error in the frame prediction mode and the prediction error in the field prediction mode in order to determine whether the frame prediction mode or the field prediction mode should be selected, and the value of the prediction error Is supplied to the controller 70. The controller 70 compares these prediction error values, and selects the prediction mode with the smaller prediction error value. The prediction mode switching circuit 52 performs signal processing so as to correspond to the prediction mode selected by the controller 70 and supplies it to the computing unit 53.
[0313]
Specifically, when the frame prediction mode is selected, the prediction mode switching circuit 52 outputs the luminance signal to the arithmetic unit 53 in the input state as described with reference to FIG. The signal processing is performed so that the color difference signal is processed so that the odd-numbered field lines and the even-numbered field lines are mixed. On the other hand, when the field prediction mode is selected, as described with reference to FIG. 9, with respect to the luminance signal, the luminance blocks Y [1] and Y [2] are composed of odd field lines, and the luminance block Y [3] and Y [4] are signal-processed so as to be composed of even field lines, and regarding the color difference signal, the upper 4 lines are composed of odd field lines and the lower 4 lines are composed of even field lines. Signal processing.
[0314]
Furthermore, the motion vector detection circuit 50 determines the prediction error in each prediction mode in order to determine which prediction mode is selected from the intra prediction mode, the forward prediction mode, the backward prediction mode, or the bidirectional prediction mode. The prediction error in each prediction mode is supplied to the controller 70. The controller 70 selects the smallest prediction error of the forward prediction, backward prediction, and bidirectional prediction as the prediction error of the inter prediction. Further, the prediction error of the inter prediction and the prediction error of the intra-picture prediction are compared, the smaller one is selected, and the mode corresponding to the selected prediction error is selected as the prediction mode. That is, if the prediction error of intra prediction is smaller, the intra prediction mode is set. If the prediction error of inter prediction is smaller, the mode in which the corresponding prediction error is the smallest of the forward prediction, backward prediction, and bidirectional prediction modes is set. The controller 70 controls the calculator 53 and the motion compensation circuit 64 so as to correspond to the selected prediction mode.
[0315]
In order to select either the frame DCT mode or the field DCT mode, the DCT mode switching circuit 55 uses the signal form (frame DCT mode) in which the data of the four luminance blocks are mixed in the odd and even field lines. ) And a signal form (field DCT mode) in which the odd and even field lines are separated, and the respective signals are supplied to the DCT circuit 56. The DCT circuit 56 calculates the encoding efficiency when the odd-numbered field and the even-numbered field are mixed and DCT processing and the coding efficiency when the odd-numbered field and the even-numbered field are separated and the DCT processing is performed. 70. The controller 70 compares the coding efficiencies supplied from the DCT circuit 56, selects the DCT mode with the better coding efficiency, and controls the DCT mode switching circuit 55 so that the selected DCT mode is obtained. .
[0316]
The controller 70 receives the target bit rate indicating the target bit rate supplied from the operator or the host computer and the signal indicating the bit amount buffered in the transmission buffer 59, that is, the signal indicating the remaining amount of the buffer. Based on the bit rate and the buffer remaining amount, feedback_q_scale_code for controlling the quantization step size of the quantization circuit 57 is generated. This feedback_q_scale_code is a control signal generated in accordance with the remaining buffer capacity of the transmission buffer 59 so that the transmission buffer 59 does not overflow or underflow, and the bits of the bit stream output from the transmission buffer 59 It is also a signal that controls the rate to be the target bit rate.
[0317]
Specifically, for example, when the bit amount buffered in the transmission buffer 59 is reduced, the quantization step size is reduced so that the generated bit amount of the next picture to be encoded is increased. On the other hand, if the amount of bits buffered in the transmission buffer 59 has increased, the quantization step size is increased so that the generated bit amount of the next picture to be encoded decreases. Note that feedback_q_scale_code and the quantization step size are proportional to each other. When feedback_q_scale_code is increased, the quantization step size is increased, and when feedback_q_scale_code is decreased, the quantization step size is decreased.
[0318]
Next, parameter reuse encoding processing, which is one of the features of the transcoder 101, will be described. In order to explain this process more clearly, the reference picture is encoded as a P picture in the first generation encoding process, encoded as an I picture in the second generation encoding process, It is assumed that the B picture was encoded in the encoding process, and this reference picture must be encoded as a P picture in the current fourth generation encoding process.
[0319]
In this case, since the reference picture is encoded in the first generation encoding process with the same picture type (I picture) as the picture type assigned as the fourth generation picture type, the controller 70 Rather than creating new encoding parameters from the supplied video data, encoding processing is performed using the first generation encoding parameters. Typical encoding parameters to be reused in the fourth encoding process include quantizer_scale_code indicating the quantization scale step size, macroblock_type indicating the prediction direction mode, motion_vector indicating the motion vector, Frame prediction mode or Field. Frame / field_motion_type indicating the prediction mode, dct_type indicating the Frame DCT mode or the Field DCT mode, and the like.
[0320]
The controller 70 does not reuse all the encoding parameters transmitted as history information, but reuses the encoding parameters as described above, which are supposed to be reused, and does not reuse them. The encoding parameters that are considered desirable are generated anew.
[0321]
Next, the encoding parameter reuse encoding process will be described focusing on differences from the above-described normal encoding process.
[0322]
In the normal encoding process described above, the motion vector detection circuit 50 detects the motion vector of the reference picture. In this parameter reuse encoding process, the motion vector motion_vector detection process is not performed, The motion vector motion_vector supplied as history information of one generation is reused. The reason will be described.
[0323]
Since the baseband video data obtained by decoding the third generation encoded stream is subjected to at least three decoding and encoding processes, the image quality is clearly degraded as compared with the original video data. Even if a motion vector is detected from video data with degraded image quality, an accurate motion vector cannot be detected. That is, the motion vector supplied as the first generation history information is clearly a more accurate motion vector than the motion vector detected in the fourth generation encoding process. That is, by reusing the motion vector transmitted as the first generation encoding parameter, the image quality does not deteriorate even if the fourth generation encoding process is performed. The controller 70 uses the motion compensation circuit 64 and variable length coding as the motion vector information of the reference picture encoded in the fourth generation encoding process, using the motion vector motion_vector supplied as the first generation history information. Supply to circuit 58.
[0324]
Further, the motion vector detection circuit 50 detects the prediction error in the frame prediction mode and the prediction error in the field prediction mode in order to determine whether the frame prediction mode or the field prediction mode is selected. In the use encoding process, the process of detecting the prediction error in the frame prediction mode and the prediction error in the field prediction mode is not performed, and the frame prediction mode or the field prediction mode supplied as the first generation history information is determined. Reuse the indicated frame / field_motion_type. This is because the prediction error in each prediction mode detected in the first generation is higher in accuracy than the prediction error in each prediction mode detected in the fourth generation encoding process. This is because a more optimal encoding process can be performed when the selected prediction mode is selected.
[0325]
Specifically, the controller 70 supplies a control signal corresponding to the frame / field_motion_type supplied as the first generation history information to the prediction mode switching circuit 52, and the prediction mode switching circuit 52 is reused. Signal processing corresponding to frame / field_motion_type is performed.
[0326]
Furthermore, in the normal encoding process, the motion vector detection circuit 50 predicts any prediction mode (hereinafter, this prediction mode) from among the intra-picture prediction mode, the forward prediction mode, the backward prediction mode, and the bidirectional prediction mode. The prediction error in each prediction direction mode is calculated in order to determine whether to select (which is also referred to as a direction mode). In this parameter reuse encoding process, the prediction error in each prediction direction mode is calculated. First, the prediction direction mode is determined based on the macroblock_type supplied as the first generation history information. This is because the prediction error in each prediction direction mode in the first generation encoding process is more accurate than the prediction error in each prediction direction mode in the fourth generation encoding process. This is because a more efficient encoding process can be performed by selecting the prediction direction mode determined by the above. Specifically, the controller 70 selects the prediction direction mode indicated by the macroblock_type included in the first generation history information, and the arithmetic unit 53 and the motion compensation circuit so as to correspond to the selected prediction direction mode. 64 is controlled.
[0327]
In the normal encoding process, the DCT mode switching circuit 55 compares the frame DCT mode encoding efficiency with the field DCT mode encoding efficiency, the field DCT mode signal format, and the field DCT mode Both of the signals converted to the signal format of the mode were supplied to the DCT circuit 56, but in this parameter reuse encoding process, the signal converted to the signal format of the frame DCT mode and the signal format of the field DCT mode are converted. The processing for generating both signals is not performed, and only the processing corresponding to the DCT mode indicated by dct_type included in the history information of the first generation is performed. Specifically, the controller 70 reuses the dct_type included in the first generation history information, and the DCT so that the DCT mode switching circuit 55 performs signal processing corresponding to the DCT mode indicated by the dct_type. The mode switching circuit 55 is controlled.
[0328]
In the normal encoding process, the controller 70 controls the quantization step size of the quantization circuit 57 based on the target bit rate specified by the operator and the remaining amount of the transmission buffer. This parameter reuse encoding process Then, the quantization step size of the quantization circuit 57 is controlled based on the target bit rate, the remaining amount of transmission buffer, and the past quantization scale included in the history information. In the following description, the past quantization scale included in the history information is described as history_q_scale_code. Further, in the history stream described later, this quantization scale is described as quantizer_scale_code.
[0329]
First, the controller 70 generates the current quantization scale feedback_q_scale_code, as in the normal encoding process. The feedback_q_scale_code is a value determined according to the remaining buffer capacity of the transmission buffer 59 so that the transmission buffer 59 does not overflow or underflow. Subsequently, the previous quantization scale history_q_scale_code value included in the first generation history stream is compared with the current quantization scale feedback_q_scale_code value to determine which quantization scale is larger. . A large quantization scale means a large quantization step. If the current quantization scale feedback_q_scale_code is larger than the past quantization scale history_q_scale_code, the controller 70 supplies the current quantization scale feedback_q_scale_code to the quantization circuit 57. On the other hand, if the past quantization scale history_q_scale_code is larger than the current quantization scale feedback_q_scale_code, the controller 70 supplies the past quantization scale history_q_scale_code to the quantization circuit 57.
[0330]
That is, the controller 70 selects the largest quantization scale code among the plurality of past quantization scales included in the history information and the current quantization scale calculated from the remaining amount of the transmission buffer. In other words, the controller 70 is used in the quantization step in the past (first, second, and third generation) encoding process or the current (fourth generation) encoding process. The quantization circuit 57 is controlled to perform quantization using the largest quantization step among the quantization steps. The reason for this will be described below.
[0331]
For example, the bit rate of the stream generated in the third generation encoding process is 4 [Mbps], and the target bit rate set for the encoder 121 that performs the fourth generation encoding process is 15 [Mbps]. Mbps]. At this time, since the target bit rate is increased, it is not actually the case that the quantization step should be simply reduced. Even if a picture encoded with a large quantization step in the past encoding process is encoded with a smaller quantization step in the current encoding process, the picture quality of this picture is improved. No. That is, encoding with a quantization step smaller than the quantization step in the past encoding process simply increases the bit amount and does not improve the image quality. Therefore, the largest quantization step among the quantization steps used in the past (first, second, and third generation) encoding processes or the current (fourth generation) encoding process is selected. When used and quantized, the most efficient encoding process can be performed.
[0332]
Next, the history decoding device 104 and the history encoding device 107 in FIG. 15 will be further described. As shown in the figure, the history decoding device 104 includes a user data decoder 201 that decodes user data supplied from the decoding device 102, a converter 202 that converts the output of the user data decoder 201, and a history from the output of the converter 202. A history VLD 203 for reproducing information is used.
[0333]
The history encoding device 107 also formats the history VLC 211 that formats the encoding parameters for three generations supplied from the history information separation device 105, the converter 212 that converts the output of the history VLC 211, and the output of the converter 212 as the user data format. It is constituted by a user data formatter 213 that formats it.
[0334]
The user data decoder 201 decodes the user data supplied from the decoding device 102 and outputs it to the converter 202. Although details will be described later with reference to FIG. 51, user data (user_data ()) is composed of user_data_start_code and user_data. In the MPEG standard, 23 bits of “0” (the same code as start_code) is included in user_data. ) Is prohibited. This is to prevent the data from being erroneously detected as start_code. The history information (history_stream ()) is described in the user data area (as a kind of user_data of the MPEG standard), and there may be such “0” having 23 or more consecutive bits. Therefore, it is necessary to insert “1” at a predetermined timing and convert it into converted_history_stream () (FIG. 38 to be described later) so that consecutive “0” s of 23 bits or more do not occur. It is the converter 212 of the history encoding apparatus 107 that performs this conversion. The converter 202 of the history decoding apparatus 104 performs a conversion process reverse to that of the converter 212 (removes “1” inserted so as not to generate “0” of 23 or more consecutive bits).
[0335]
The history VLD 203 generates history information (in this case, a first generation encoding parameter and a second generation encoding parameter) from the output of the converter 202, and outputs the history information to the history information multiplexer 103.
[0336]
On the other hand, in the history encoding device 107, the history VLC 211 converts the encoding parameters for the three generations (first generation, second generation, and third generation) supplied from the history information separation device 105 into a history information format. To do. This format includes a fixed-length format (FIGS. 40 to 46 described later) and a variable-length format (FIG. 47 described later). Details of these will be described later.
[0337]
The history information formatted by the history VLC 211 is converted into converted_history_stream () by the converter 212. As described above, this is a process for preventing start_code of user_data () from being erroneously detected. That is, although “0” having 23 or more consecutive bits exists in the history information, since “0” having 23 or more consecutive bits cannot be arranged in user_data, do not touch this prohibited item. Data is converted by the converter 212 ("1" is inserted at a predetermined timing).
[0338]
The user data formatter 213 adds the History_Data_ID to the converted_history_stream () supplied from the converter 212 based on FIG. 38 to be described later, and further adds the user_data_stream_code to generate user_data of the MPEG standard that can be inserted into the video stream. And output to the encoding device 106.
[0339]
FIG. 20 shows a configuration example of the history VLC 211. In the codeword converter 301 and the code length converter 305, an encoding parameter (an encoding parameter to be transmitted this time as history information) (item data) and information for specifying a stream in which the encoding parameter is arranged (for example, The name of the syntax (for example, the name of sequence_header described later) (item NO.) Is supplied from the history information separating apparatus 105. The codeword converter 301 converts the input encoding parameter into a codeword corresponding to the instructed syntax, and outputs the codeword to the barrel shifter 302. The barrel shifter 302 shifts the code word input from the code word converter 301 by an amount corresponding to the shift amount supplied from the address generation circuit 306, and outputs the code word to the switch 303 as a byte-by-byte code word. A switch 303 that is switched by a bit select signal output from the address generation circuit 306 is provided for each bit, and the code word supplied from the barrel shifter 302 is supplied to the RAM 304 for storage. The write address at this time is designated from the address generation circuit 306. In addition, when a read address is designated from the address generation circuit 306, data (codeword) stored in the RAM 304 is read and supplied to the subsequent converter 212 and, if necessary, via the switch 303. The data is again supplied to the RAM 304 and stored.
[0340]
The code length converter 305 determines the code length of the encoding parameter from the input syntax and the encoding parameter, and outputs it to the address generation circuit 306. The address generation circuit 306 generates the above-described shift amount, bit select signal, write address, or read address corresponding to the input code length, and supplies them to the barrel shifter 302, the switch 303, or the RAM 304, respectively. .
[0341]
As described above, the history VLC 211 is configured as a so-called variable length encoder, and performs variable length encoding on the input encoding parameter and outputs the result.
[0342]
FIG. 21 shows a configuration example of the history VLD 203 that decodes the data formatted in the history format as described above. In the history VLD 203, the encoding parameter data supplied from the converter 202 is supplied to the RAM 311 and stored therein. The write address at this time is supplied from the address generation circuit 315. The address generation circuit 315 also generates a read address at a predetermined timing and supplies it to the RAM 311. At this time, the RAM 311 reads the data stored at the read address and outputs it to the barrel shifter 312. The barrel shifter 312 shifts the input data by an amount corresponding to the shift amount output from the address generation circuit 315 and outputs the shifted data to the inverse code length converter 313 and the inverse codeword converter 314.
[0343]
The inverse code length converter 313 is also supplied from the converter 202 with the name of the syntax (item No.) of the stream in which the encoding parameter is arranged. Based on the syntax, the inverse code length converter 313 obtains the code length from the input data (code word) and outputs the obtained code length to the address generation circuit 315.
[0344]
Further, the inverse codeword converter 314 decodes the data supplied from the barrel shifter 312 based on the syntax (inverse codeword) and outputs the decoded data to the history information multiplexing apparatus 103.
[0345]
Further, the inverse codeword converter 314 extracts information necessary for specifying what codeword is included (information necessary for determining a codeword delimiter), and generates an address generation circuit. It outputs to 315. The address generation circuit 315 generates a write address and a read address based on this information and the code length input from the inverse code length converter 313, outputs it to the RAM 311, generates a shift amount, and outputs it to the barrel shifter 312. To do.
[0346]
FIG. 22 shows a configuration example of the converter 212. In this example, 8-bit data is read from the read address output from the controller 326 in the buffer memory 320 arranged between the history VLC 211 and the converter 212, and is read into a D-type flip-flop (D-FF) 321. Supplied and held. The data read from the D-type flip-flop 321 is supplied to the stuff circuit 323 and also supplied to and held in the 8-bit D-type flip-flop 322. The 8-bit data read from the D-type flip-flop 322 is combined with the 8-bit data read from the D-type flip-flop 321 and supplied to the stuff circuit 323 as 16-bit parallel data.
[0347]
The stuff circuit 323 inserts a code “1” at the position of a stuff position signal (stuff position) supplied from the controller 326 (stuffing), and outputs the data to the barrel shifter 324 as 17-bit data.
[0348]
The barrel shifter 324 shifts the input data based on the signal (shift) indicating the shift amount supplied from the controller 326, extracts 8-bit data, and outputs it to the 8-bit D-type flip-flop 325. Data held in the D-type flip-flop 325 is read from the data and supplied to the user data formatter 213 in the subsequent stage via the buffer memory 327. At this time, the controller 326 generates a write address together with the output data and supplies the write address to the buffer memory 327 interposed between the converter 212 and the user data formatter 213.
[0349]
FIG. 23 illustrates a configuration example of the stuff circuit 323. The 16-bit data input from the D-type flip-flops 322 and 321 are input to the contacts a of the switches 331-16 to 331-1, respectively. The data of the switch adjacent to the MSB side (upper side in the figure) is supplied to the contact c of the switch 331-i (i = 0 to 15). For example, the thirteenth data from the LSB supplied to the contact a of the switch 331-13 adjacent to the MSB side is supplied to the contact c of the switch 331-12, and the contact c of the switch 331-13 is supplied to the contact c of the switch 331-13. The 14th data from the LSB side supplied to the contact a of the switch 331-14 adjacent to the MSB side is supplied.
[0350]
However, the contact a of the switch 331-0 further below the switch 331-1 corresponding to the LSB is open. In addition, the contact c of the switch 331-16 corresponding to the MSB is open because there is no higher-order switch.
[0351]
Data “1” is supplied to the contact b of each of the switches 331-0 to 331-16.
[0352]
The decoder 332 switches one of the switches 331-0 to 331-16 to the contact b side in response to the signal stuff position indicating the position to insert the data “1” supplied from the controller 326, Further, the switch on the LSB side is switched to the contact c side, and the switch on the MSB side is switched to the contact a side.
[0353]
FIG. 23 shows an example in which data “1” is inserted in the thirteenth from the LSB side. Accordingly, in this case, the switches 331-0 to 331-12 are all switched to the contact c side, the switch 331-13 is switched to the contact b side, and the switches 331-14 to 331-16 are switched to the contact c side. It has been switched to the a side.
[0354]
The converter 212 shown in FIG. 22 converts the 22-bit code into 23-bit and outputs the result.
[0355]
FIG. 24 shows the timing of the output data of each part of the converter 212 of FIG. When the controller 326 of the converter 212 generates a read address (FIG. 24A) in synchronization with the byte-unit clock, the corresponding data is read from the buffer memory 320 in byte units, and the D-type flip-flop 321 is temporarily held. The data read from the D-type flip-flop 321 (FIG. 24B) is supplied to the stuff circuit 323 and is also supplied to and held by the D-type flip-flop 322. The data held in the D-type flip-flop 322 is further read therefrom (FIG. 24C) and supplied to the stuff circuit 323.
[0356]
Therefore, the input of the stuff circuit 323 (FIG. 24D) is the first 1-byte data D0 at the timing of the read address A1, and the 1-byte data D0 and 1 byte at the timing of the next read address A2. 2 bytes of data D1 and at the timing of the read address A3, it becomes 2 bytes of data composed of data D1 and data D2.
[0357]
The stuff circuit 323 is supplied from the controller 326 with a signal stuff position (FIG. 24E) indicating a position where data “1” is inserted. The decoder 332 of the stuff circuit 323 switches the switch corresponding to this signal stuff position to the contact b among the switches 331-16 to 331-0, switches the switch on the LSB side to the contact c side, and further switches the MSB from it. Switch the side switch to the contact a side. As a result, since data “1” is inserted, the stuff circuit 323 outputs data in which data “1” is inserted at the position indicated by the signal stuff position (FIG. 24F).
[0358]
The barrel shifter 324 barrel-shifts the input data by the amount indicated by the signal shift (FIG. 24 (G)) supplied from the controller 326 and outputs it (FIG. 24 (H)). This output is further held once by the D-type flip-flop 325 and then output to the subsequent stage (FIG. 24I).
[0359]
Data “1” is inserted into the data output from the D-type flip-flop 325 after the 22-bit data. Therefore, even if all the bits between the data “1” and the next data “1” are 0, the number of consecutive 0 data is 22.
[0360]
FIG. 25 illustrates a configuration example of the converter 202. The configuration including the D-type flip-flop 341 to the controller 346 of the converter 202 is basically the same as the configuration of the D-type flip-flop 321 to the controller 326 of the converter 212 shown in FIG. Instead of the converter 212, the point that the delete circuit 343 is inserted is different from that in the converter 212. Other configurations are the same as those in the converter 212 of FIG.
[0361]
That is, in this converter 202, the delete circuit 343 deletes the bit (data “1” inserted by the stuff circuit 323 in FIG. 22) according to the signal delete position output from the controller 346 and indicating the position of the bit to be deleted. Is done.
[0362]
Other operations are the same as those in the converter 212 of FIG.
[0363]
FIG. 26 shows a configuration example of the discrete circuit 343. In this configuration example, among the 16-bit data input from the D-type flip-flops 342 and 341, 15 bits on the LSB side are supplied to the contacts a of the corresponding switches 351-0 to 351-14, respectively. . MSB side data is supplied to the contact b of each switch by 1 bit. The decoder 352 deletes the bit specified by the signal delete position supplied from the controller 346 and outputs it as 15-bit data.
[0364]
FIG. 26 shows a state where the 13th bit from the LSB is deleted. Accordingly, in this case, the switches 351-0 to 351-11 are switched to the contact a side, and 12 bits from the LSB to the 12th are selected and output as they are. Further, since the switches 351-12 to 351-14 are respectively switched to the contact b side, the 14th to 16th data are selected and output as the 13th to 15th bit data. The
[0365]
The input of the stuff circuit 323 of FIG. 23 and the input of the delete circuit 343 of FIG. 26 is 16 bits. The input of the stuff circuit 323 of the converter 212 of FIG. 22 is supplied from the D-type flip-flops 322 and 321, respectively. This is because, in the converter 202 of FIG. 25, the input of the delete circuit 343 is 16 bits by the D-type flip-flops 342 and 341. In FIG. 22, the 17 bits output from the stuff circuit 323 are barrel-shifted by the barrel shifter 324, so that, for example, 8 bits are finally selected and output. In the converter 202 of FIG. The 15-bit data output from 343 is barrel-shifted by a predetermined amount by the barrel shifter 344 to obtain 8-bit data.
[0366]
FIG. 27 illustrates another configuration example of the converter 212. In this configuration example, the counter 361 counts the number of consecutive 0 bits in the input data and outputs the count result to the controller 326. For example, when the counter 361 counts 22 consecutive zero bits, the controller 326 outputs the signal stuff position to the stuff circuit 323. At this time, the controller 326 resets the counter 361 and causes the counter 361 to count the number of consecutive 0 bits again.
[0367]
Other configurations and operations are the same as those in FIG.
[0368]
FIG. 28 illustrates another configuration example of the converter 202. In this configuration example, the counter 371 counts the number of consecutive 0s in the input data and outputs the count result to the controller 346. When the count value of the counter 371 reaches 22, the controller 346 outputs the signal delete position to the delete circuit 343, resets the counter 371, and causes the counter 371 to count the number of new consecutive 0 bits again. . Other configurations are the same as those in FIG.
[0369]
Thus, in this configuration example, data “1” as a marker bit is inserted and deleted based on a predetermined pattern (the number of consecutive data “0”).
[0370]
The configuration shown in FIGS. 27 and 28 enables more efficient processing than the configuration shown in FIGS. 22 and 25. However, the converted length depends on the original history information.
[0371]
FIG. 29 shows a configuration example of the user data formatter 213. In this example, when the controller 383 outputs a read address to a buffer memory (not shown) disposed between the converter 212 and the user data formatter 213, the data read from the read address is converted to the user data formatter 213. The switch 382 is supplied to the contact a side. The ROM 381 stores data necessary for generating user_data () such as a user data start code and a data ID. At a predetermined timing, the controller 313 switches the switch 382 to the contact a side or the contact b side, and appropriately selects and outputs data stored in the ROM 381 or data supplied from the converter 212. As a result, data in the format of user_data () is output to the encoding device 106.
[0372]
Although not shown, the user data decoder 201 can be realized by outputting input data through a switch that is read from the ROM 381 in FIG. 29 and deletes inserted data. .
[0373]
FIG. 30 shows a state where a plurality of transcoders 101-1 to 101-N are connected in series and used in a video editing studio, for example. The history information multiplexing apparatus 103-i of each transcoder 101-i (i = 1 to N) is used by the self in the section in which the oldest encoding parameter in the above-described encoding parameter area is recorded. Overwrite the latest encoding parameters. As a result, the encoding parameters (generation history information) for the latest four generations corresponding to the same macroblock are recorded in the baseband image data (FIG. 18).
[0374]
The encoder 121-i (FIG. 19) of each encoding device 106-i has a quantization circuit based on the encoding parameter used this time supplied from the history information separation device 105-i in the variable length encoding circuit 58. The video data supplied from 57 is encoded. In the bit stream generated in this way (for example, picture_header ()), the current encoding parameter is multiplexed.
[0375]
The variable-length encoding circuit 58 also multiplexes user data (including generation history information) supplied from the history encoding device 107-i in the output bit stream (instead of the embedding process as shown in FIG. 18). Multiplex in the bitstream). The bit stream output from the encoding device 106-i is input to the subsequent transcoder 101- (i + 1) via the SDTI (Serial Data Transfer Interface) 108-i.
[0376]
The transcoder 101-i and the transcoder 101- (i + 1) are configured as shown in FIG. Therefore, the process is the same as that described with reference to FIG.
[0377]
When it is desired to change what is currently encoded as an I picture to P or B picture as the encoding using the actual encoding parameter history, the past encoding parameter history is referred to and P or A case where the picture is a B picture is searched. If these histories exist, the picture type is changed using parameters such as a motion vector. On the other hand, if there is no history in the past, the change of the picture type without motion detection is given up. Of course, even if there is no history, the picture type can be changed by performing motion detection.
[0378]
In the format shown in FIG. 18, encoding parameters for four generations are embedded. However, parameters of picture types of I, P, and B can be embedded. FIG. 31 shows an example of the format in this case. In this example, one generation of encoding parameters (picture history information) is recorded for each picture type when the same macroblock has been encoded with a change in picture type in the past. Therefore, the decoder 111 shown in FIG. 16 and the encoder 121 shown in FIG. 19 use the I picture, P picture instead of the current (latest), third generation, second generation, and first generation encoding parameters. , And the encoding parameters for one generation corresponding to the B picture are input / output.
[0379]
In this example, the free space of Cb [1] [x] and Cr [1] [x] is not used, so the Cb [1] [x] and Cr [1] [x] areas are available. The present invention can also be applied to 4: 2: 0 format image data.
[0380]
In this example, the decoding apparatus 102 extracts the encoding parameter at the same time as decoding, determines the picture type, and writes (multiplexes) the encoding parameter in a location corresponding to the picture type of the image signal to separate history information. Output to the device 105. The history information separating apparatus 105 can separate the encoding parameters, and can perform re-encoding while changing the picture type in consideration of the picture type to be encoded from now and the input past encoding parameters.
[0381]
Next, processing for determining a picture type that can be changed in each transcoder 101 will be described with reference to the flowchart of FIG. Note that since the picture type change in the transcoder 101 uses a past motion vector, it is assumed that this process is performed without performing motion detection. Further, the process described below is executed by the history information separating apparatus 105.
[0382]
In step S <b> 1, one generation of encoding parameters (picture history information) is input to the history information separating apparatus 105 for each picture type.
[0383]
In step S2, the history information separating apparatus 105 determines whether or not there is an encoding parameter when the picture history information is changed to the B picture. If it is determined that there is an encoding parameter when the picture history information is changed to the B picture, the process proceeds to step S3.
[0384]
In step S3, the history information separating apparatus 105 determines whether or not there is an encoding parameter when the picture history information is changed to the P picture. If it is determined that there is an encoding parameter when the picture history information is changed to the P picture, the process proceeds to step S4.
[0385]
In step S4, the history information separating apparatus 105 determines that the picture types that can be changed are I picture, P picture, and B picture.
[0386]
If it is determined in step S3 that there is no coding parameter when the picture history information is changed to the P picture, the process proceeds to step S5.
[0387]
In step S5, the history information separating apparatus 105 determines that the picture types that can be changed are I picture and B picture. Furthermore, the history information separation apparatus 105 can be changed to a P picture in a pseudo manner by performing a special process (only the forward prediction vector is used instead of the backward prediction vector included in the B picture history information). to decide.
[0388]
If it is determined in step S2 that there is no coding parameter when the picture history information is changed to the B picture, the process proceeds to step S6.
[0389]
In step S6, the history information separating apparatus 105 determines whether there is an encoding parameter when the picture history information is changed to the P picture. If it is determined that there is an encoding parameter when the picture history information is changed to the P picture, the process proceeds to step S7.
[0390]
In step S7, the history information separating apparatus 105 determines that the changeable picture types are I picture and P picture. Further, the history information separating apparatus 105 determines that the picture can be changed to the B picture by performing a special process (using only the forward prediction vector included in the history information for the P picture).
[0390]
If it is determined in step S6 that there is no coding parameter when the picture history information is changed to the P picture, the process proceeds to step S8. In step S8, since there is no motion vector, the history information separating apparatus 105 determines that the picture type that can be changed is only an I picture (since it is an I picture, it cannot be changed to other than an I picture).
[0392]
Following step S4, S5, S7, and S8, in step S9, the history information separation device 105 displays a changeable picture type on a display device (not shown) and notifies the user.
[0393]
FIG. 33 shows an example of changing the picture type. When the picture type is changed, the number of frames constituting the GOP is changed. That is, in this example, a 4-Mbps Long GOP (N = 15 (GOP frame number N = 15)) and M = 3 (I or G picture appearance period M = 3) frames are included. The first generation) is converted to a 50 Mbps Short GOP (second generation) composed of N = 1, M = 1 frames, and again a 4 Mbps Long composed of N = 15, M = 3 frames. It has been converted to GOP (3rd generation). In the figure, a broken line indicates a GOP boundary.
[0394]
When the picture type is changed from the first generation to the second generation, as is clear from the description of the changeable picture type determination process described above, it is possible to change the picture type to I picture for all frames. is there. When the picture type is changed, all motion vectors calculated when the moving image (0th generation) is converted to the 1st generation are stored (remained) in the picture history information. Next, when converting to Long GOP again (the picture type is changed from the second generation to the third generation), the motion vector for each picture type when converted from the zeroth generation to the first generation is stored. Therefore, by reusing this, it is possible to suppress degradation in image quality and convert it back to Long GOP again.
[0395]
FIG. 34 shows another example of picture type change. In this example, N = 14, M = 2 4 Mbps Long GOP (1st generation) is converted to N = 2, M = 2 18 Mbps Short GOP (2nd generation). = 1, M = 1 frame number is converted to 50 Mbps Short GOP (3rd generation), 1 Mbps frame number N is converted to random GOP (4th generation).
[0396]
Also in this example, the motion vector for each picture type when converted from the 0th generation to the 1st generation is stored until the conversion from the 3rd generation to the 4th generation. Therefore, as shown in FIG. 34, even when the picture type is changed in a complicated manner, the stored image parameters can be reused, so that image quality deterioration can be suppressed to a small level. Furthermore, if the stored quantization parameter quantization scale is used effectively, encoding with less image quality degradation can be realized.
[0397]
The reuse of the quantization scale will be described with reference to FIG. FIG. 35 shows that a predetermined frame is always converted into an I picture from the first generation to the fourth generation, and only the bit rate is changed to 4 Mbps, 18 Mbps, or 50 Mbps.
[0398]
For example, when converting from the first generation (4 Mbps) to the second generation (18 Mbps), the image quality does not improve even if re-encoding is performed with a fine quantization scale as the bit rate increases. This is because data quantized in the past in a coarse quantization step is not restored. Therefore, as shown in FIG. 35, even if the bit rate is increased in the middle, the quantization in fine quantization steps is accompanied by an increase in the amount of information and does not lead to an improvement in image quality. Therefore, if control is performed so as to maintain the coarsest (larger) quantization scale in the past, the most efficient and efficient encoding is possible.
[0399]
Note that when changing from the third generation to the fourth generation, the bit rate is reduced from 50 Mbps to 4 Mbps, but in this case as well, the coarsest (larger) quantization scale in the past is maintained.
[0400]
As described above, when the bit rate is changed, it is very effective to encode using the history of the past quantization scale.
[0401]
This quantization control process will be described with reference to the flowchart of FIG. In step S11, the history information separating apparatus 105 determines whether or not the input picture history information includes a coding parameter of a picture type to be converted. If it is determined that there is a coding parameter of the picture type to be converted, the process proceeds to step S12.
[0402]
In step S12, the history information separating apparatus 105 extracts history_q_scale_code from the encoding parameter that is the target of the picture history information.
[0403]
In step S <b> 13, the history information separating apparatus 105 calculates feedback_q_scale_code based on the remaining buffer amount fed back from the transmission buffer 59 to the quantization circuit 57.
[0404]
In step S14, the history information separating apparatus 105 determines whether history_q_scale_code is larger (coarse) than feedback_q_scale_code. When it is determined that history_q_scale_code is larger than feedback_q_scale_code, the process proceeds to step S15.
[0405]
In step S15, the history information separating apparatus 105 outputs history_q_scale_code to the quantization circuit 57 as a quantization scale. The quantization circuit 57 performs quantization using history_q_scale_code.
[0406]
In step S16, it is determined whether all macroblocks included in the frame have been quantized. If it is determined that all the macroblocks have not been quantized, the process returns to step S12, and the processes of steps S12 to S16 are repeated until all the macroblocks are quantized.
[0407]
If it is determined in step S14 that history_q_scale_code is not larger (fine) than feedback_q_scale_code, the process proceeds to step S17.
[0408]
In step S <b> 17, the history information separating apparatus 105 outputs feedback_q_scale_code to the quantization circuit 57 as a quantization scale. The quantization circuit 57 performs quantization using feedback_q_scale_code.
[0409]
If it is determined in step S11 that the encoding parameter of the picture type to be converted does not exist in the history information, the process proceeds to step S18.
[0410]
In step S <b> 18, the history information separation device 105 calculates feedback_q_scale_code based on the remaining buffer amount fed back from the transmission buffer 59 to the quantization circuit 57.
[0411]
In step S19, the quantization circuit 57 performs quantization using Feedback_q_scale_code.
[0412]
In step S20, it is determined whether all macroblocks included in the frame have been quantized. If it is determined that all the macroblocks have not been quantized, the process returns to step S18, and the processes of steps S18 to S20 are repeated until all the macroblocks are quantized.
[0413]
In the transcoder 101 in the present embodiment, as described above, the decoding side and the code side are roughly coupled, and the encoding parameters are multiplexed and transmitted to the image data. As shown, the decoding apparatus 102 and the encoding apparatus 106 may be directly connected (tightly coupled).
[0414]
The transcoder 101 described in FIG. 15 multiplexes and transmits the past coding parameters to the baseband video data in order to supply the first to third generation past coding parameters to the coding device 106. It was like that. However, in the present invention, a technique for multiplexing past coding parameters on baseband video data is not essential, and as shown in FIG. 37, a transmission path (for example, a data transfer bus) different from baseband video data is used. ) May be used to transmit past coding parameters.
[0415]
That is, the decoding apparatus 102, history decoding apparatus 104, encoding apparatus 106, and history encoding apparatus 107 shown in FIG. 37 are the same as the decoding apparatus 102, history decoding apparatus 104, encoding apparatus 106, and history described in FIG. It has exactly the same function and configuration as the encoding device 107.
[0416]
The variable length decoding circuit 112 of the decoding apparatus 102 extracts the third generation encoding parameters from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third generation encoded stream ST (3rd). , And supplies it to the controller 70 of the history encoding device 107 and the encoding device 106, respectively. The history encoding apparatus 107 converts the received third-generation encoding parameter into converted_history_stream () so that it can be described in the user data area of the picture layer, and variable length encoding of the encoding apparatus 106 using converted_history_stream () as user data Supply to circuit 58.
[0417]
Further, the variable length decoding circuit 112 extracts user data user_data including the first generation encoding parameter and the second encoding parameter from the user data area of the picture layer of the third generation encoded stream, This is supplied to the variable length coding circuit 58 of the decoding device 104 and the coding device 106. The history decoding device 104 extracts the first generation encoding parameter and the second generation encoding parameter from the history stream described as converted_history_stream () in the user data area, and sends it to the controller of the encoding device 106. Supply.
[0418]
The controller 70 of the encoding device 106 performs coding based on the first generation and second generation encoding parameters received from the history decoding device 104 and the third generation encoding parameters received from the encoding device 102. The encoding process of the encoding device 106 is controlled.
[0419]
The variable length encoding circuit 58 of the encoding device 106 receives the user data user_data including the first generation encoding parameter and the second encoding parameter from the decoding device 102, and receives the third generation from the history encoding device 107. The user data user_data including the following encoding parameters is received, and the user data is described as history information in the user data area of the picture layer of the fourth generation encoded stream.
[0420]
FIG. 38 is a diagram illustrating a syntax for decoding an MPEG video stream. The decoder extracts a plurality of meaningful data items (data elements) from the bit stream by decoding the MPEG bit stream according to this syntax. In the drawing, the syntax described below has functions and conditional statements expressed in small letters, and data elements are shown in bold letters. The data item is described by a mnemonic indicating its name, bit length, type, and transmission order.
[0421]
First, functions used in the syntax shown in FIG. 38 will be described.
[0422]
The next_start_code () function is a function for searching for a start code described in the bitstream. In the syntax shown in FIG. 38, the sequence_header () function and the sequence_extension () function are arranged in this order next to the next_start_code () function. Therefore, in this bitstream, the sequence_header () function and The data element defined by the sequence_extension () function is described. Therefore, when decoding the bitstream, the next_start_code () function uses the next_start_code () function to find the start code (a type of data element) described at the beginning of the sequence_header () function and sequence_extension () function from the bitstream. Then, the sequence_header () function and the sequence_extension () function are further found, and each data element defined by them is decoded.
[0423]
The sequence_header () function is a function for defining the header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is a function for defining the extension data of the sequence layer of the MPEG bit stream. .
[0424]
The do {} while syntax placed next to the sequence_extension () function is a data element written based on the function in {} of the do statement while the condition defined by the while statement is true. This is a syntax for extracting from the stream. That is, with the do {} while syntax, while the condition defined by the while statement is true, a decoding process is performed to extract the data element described based on the function in the do statement from the bit stream.
[0425]
The nextbits () function used in the while statement is a function for comparing a bit or a bit string appearing in the bit stream with a data element to be decoded next. In the syntax example of FIG. 38, the nextbits () function compares the bit string in the bit stream with sequence_end_code indicating the end of the video sequence, and when the bit string in the bit stream does not match sequence_end_code, The sentence condition is true. Therefore, the do {} while syntax placed next to the sequence_extension () function means that the data element defined by the function in the do statement is not bitstreamed while the sequence_end_code indicating the end of the video sequence does not appear in the bitstream. It shows that it is described in.
[0426]
In the bitstream, after each data element defined by the sequence_extension () function, a data element defined by the extension_and_user_data (0) function is described. The extension_and_user_data (0) function is a function for defining extension data and user data in the sequence layer of the MPEG bit stream.
[0427]
The do {} while syntax placed next to this extension_and_user_data (0) function is a data element written based on the function in {} of the do statement while the condition defined by the while statement is true. Is a function for extracting from the bitstream. The nextbits () function used in this while statement is a function for determining a match between a bit or a bit string appearing in the bit stream and a picture_start_code or group_start_code, and the bit or bit string appearing in the bit stream, If picture_start_code or group_start_code matches, the condition defined by the while statement is true. Therefore, in this do {} while syntax, when picture_start_code or group_start_code appears in the bitstream, the code of the data element defined by the function in the do statement is described next to the start code. By searching for the start code indicated by this picture_start_code or group_start_code, the data element defined in the do statement can be extracted from the bitstream.
[0428]
The if statement described at the beginning of the do statement indicates a condition that group_start_code appears in the bitstream. When the condition by this if statement is true, the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function are sequentially described in the bitstream after this group_start_code.
[0429]
The group_of_picture_header (1) function is a function for defining the header data of the GOP layer of the MPEG bit stream. The extension_and_user_data (1) function is the extension data (extension_data) and user data (extension_data) of the GOP layer of the MPEG bit stream. This is a function for defining (user_data).
[0430]
Furthermore, in this bitstream, the data elements defined by the picture_header () function and the picture_coding_extension () function are described after the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function. Yes. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function is not described, so it is defined by the extension_and_user_data (0) function. The data element defined by the picture_header () function and the picture_coding_extension () function is described after the data element.
[0431]
This picture_header () function is a function for defining the header data of the picture layer of the MPEG bit stream, and the picture_coding_extension () function is a function for defining the first extension data of the picture layer of the MPEG bit stream It is.
[0432]
The next while statement is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement is a function for determining a match between a bit string appearing in the bitstream and extension_start_code or user_data_start_code, and a bit string appearing in the bitstream and extension_start_code or user_data_start_code and If they match, the condition defined by this while statement is true.
[0433]
The first if statement is a function for determining whether the bit string appearing in the bitstream matches extension_start_code. When the bit string appearing in the bitstream matches the 32-bit extension_start_code, the data element defined by the extension_data (2) function is described next to the extension_start_code in the bitstream.
[0434]
The second if statement is a syntax for determining a match between the bit string appearing in the bitstream and user_data_start_code. If the bit string appearing in the bitstream matches the 32-bit user_data_start_code, the third if statement Condition judgment of if statement is performed. This user_data_start_code is a start code for indicating the start of the user data area of the picture layer of the MPEG bit stream.
[0435]
The third if statement is a syntax for determining whether the bit string appearing in the bitstream matches History_Data_ID. If the bit string appearing in the bitstream matches this 32-bit History_Data_ID, then in the user data area of the picture layer of this MPEG bitstream, after the code indicated by this 32-bit History_Data_ID, the converted_history_stream () function Describes data elements defined by.
[0436]
The converted_history_stream () function is a function for describing history information and history data for transmitting all the encoding parameters used at the time of MPEG encoding. Details of the data element defined by the converted_history_stream () function will be described later as history_stream () with reference to FIGS. 40 to 47. The History_Data_ID is a start code indicating the history information and history data described in the user data area of the picture layer of the MPEG bit stream.
[0437]
The else statement is a syntax for indicating that the condition is not true in the third if statement. Therefore, when the data element defined by the converted_history_stream () function is not described in the user data area of the picture layer of the MPEG bit stream, the data element defined by the user_data () function is described.
[0438]
In FIG. 38, history information is described in converted_history_stream () and is not described in user_data (), but this converted_history_stream () is described as a kind of user_data in the MPEG standard. Therefore, in this specification, it is also described that history information is described in user_data depending on the case, but this means that it is described as a kind of user_data of the MPEG standard.
[0439]
The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer after the user data in the picture layer of the MPEG bit stream. Normally, the data element indicated by the picture_data () function is the data element defined by the converted_history_stream () function described in the user data area of the picture layer of the bit stream or the data element defined by the user_data () function. As described below, when there is no extension_start_code or user_data_start_code in the bitstream indicating the data element of the picture layer, the data element indicated by this picture_data () function is defined by the picture_coding_extension () function It is described after the data element.
[0440]
Next to the data element indicated by the picture_data () function, data elements defined by the sequence_header () function and the sequence_extension () function are arranged in order. The data elements described by the sequence_header () function and the sequence_extension () function are exactly the same as the data elements described by the sequence_header () function and the sequence_extension () function described at the beginning of the video stream sequence. The reason why the same data is described in the stream in this way is that the data of the sequence layer is received when the reception starts from the middle of the data stream (for example, the bit stream portion corresponding to the picture layer) on the bit stream receiver side. This is to prevent the stream from being able to be decoded and the stream from being decoded.
[0441]
Following the data element defined by the last sequence_header () function and sequence_extension () function, that is, at the end of the data stream, 32-bit sequence_end_code indicating the end of the sequence is described.
[0442]
An outline of the basic configuration of the above syntax is as shown in FIG.
[0443]
Next, a history stream defined by the converted_history_stream () function will be described.
[0444]
This converted_history_stream () is a function for inserting a history stream indicating history information into the user data area of the MPEG picture layer. The meaning of “converted” is a conversion process that inserts a marker bit (1 bit) at least every 22 bits of a history stream composed of history data to be inserted into the user area in order to prevent start emulation. It means that it is a stream.
[0445]
This converted_history_stream () is described in either a fixed-length history stream (FIGS. 40 to 46) or a variable-length history stream (FIG. 47) described below. When a fixed-length history stream is selected on the encoder side, there is an advantage that a circuit and software for decoding each data element from the history stream on the decoder side are simplified. On the other hand, when a variable length history stream is selected on the encoder side, history information (data elements) described in the user area of the picture layer can be arbitrarily selected in the encoder as needed. Can be reduced, and as a result, the data rate of the entire encoded bitstream can be reduced.
[0446]
The “history stream”, “history stream”, “history information”, “history information”, “history data”, “history data”, “history parameter”, and “history parameter” described in the present invention are past codes. It means the encoding parameter (or data element) used in the encoding process, and does not mean the encoding parameter used in the current (final stage) encoding process. For example, in the first generation encoding process, a certain picture is encoded and transmitted with an I picture, and in the next second generation encoding process, this picture is encoded and transmitted as a P picture. In the third generation encoding process, an example will be described in which this picture is encoded with a B picture and transmitted.
[0447]
The encoding parameters used in the third generation encoding process are set to predetermined positions in the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the encoded bitstream generated in the third generation encoding process. is described. On the other hand, the encoding parameters used in the first generation and second generation encoding processes, which are past encoding processes, are sequence layers and GOP layers in which the encoding parameters used in the third generation encoding process are described. Is described in the user data area of the picture layer as the history information of the encoding parameter according to the syntax described above.
[0448]
First, the fixed-length history stream syntax will be described with reference to FIGS.
[0449]
The user data area in the picture layer of the bitstream generated in the final stage (for example, third generation) encoding process is first used in the past (for example, first generation and second generation) encoding processes. The encoding parameter included in the sequence header of the sequence layer that has been stored is inserted as a history stream. Note that history information such as the sequence header of the sequence layer of the bit stream generated in the past encoding process is not inserted into the sequence header of the sequence layer of the bit stream generated in the encoding process of the final stage. It should be noted that.
[0450]
The data elements included in the sequence header (sequence_header) used in the past encoding process are sequence_header_code, sequence_header_present_flag, horizontal_size_value, marker_bit, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, VBV_buffer_size_value, constant_in_traiter, Composed.
[0451]
The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. The marker_bit is bit data inserted to prevent start code emulation. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle.
[0452]
bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantiser_matrix is data indicating the presence of intra MB quantization matrix data. load_non_intra_quantiser_matrix is data indicating the presence of non-intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the intra MB quantization matrix. non_intra_quantiser_matrix is data representing a value of a non-intra MB quantization matrix.
[0453]
In the user data area of the picture layer of the bit stream generated in the encoding process at the final stage, a data element representing a sequence extension of the sequence layer used in the past encoding process is described as a history stream.
[0454]
Data elements representing the sequence extensions (sequence_extension) used in the past encoding process are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_ext_, extension_delay_lay_rate_extension_, It is.
[0455]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data.
[0456]
The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data to be added to the vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.
[0457]
Subsequently, in the user area of the picture layer of the bit stream, a data element representing a sequence layer sequence display extension used in the past encoding process is described as a history stream.
[0458]
The data element described as this sequence display extension (sequence_display_extension) includes extension_start_code, extension_start_code_identifier, sequence_display_extension_present_flag, video_format, colour_description, colour_primaries, transfer_characteristics, matrix_coeffients, display_horizontal_size, and display_vertical_size.
[0459]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is a code indicating which extension data is sent. The sequence_display_extension_present_flag is data indicating whether the data element in the sequence display extension is valid or invalid. video_format is data representing the video format of the original signal. color_description is data indicating that there is detailed data of the color space. color_primaries is data indicating details of the color characteristics of the original signal. transfer_characteristics is data indicating details of how photoelectric conversion is performed. Matrix_coeffients is data indicating details of how the original signal is converted from the three primary colors of light. display_horizontal_size is data representing the active area (horizontal size) of the intended display. display_vertical_size is data representing the active area (vertical size) of the intended display.
[0460]
Subsequently, macroblock assignment data (macroblock_assignment_in_user_data) indicating the phase information of the macroblock generated in the past encoding process is stored in the user area of the picture layer of the bitstream generated in the final stage encoding process. It is described as a history stream.
[0461]
Macroblock_assignment_in_user_data indicating the phase information of the macroblock is composed of data elements such as macroblock_assignment_present_flag, v_phase, and h_phase.
[0462]
This macroblock_assignment_present_flag is data indicating whether the data element in macroblock_assignment_in_user_data is valid or invalid. v_phase is data indicating vertical phase information when a macroblock is cut out from image data. h_phase is data indicating horizontal phase information when a macroblock is cut out from image data.
[0463]
Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the GOP header of the GOP layer used in the past encoding process is described as a history stream. Yes.
[0464]
A data element representing this GOP header (group_of_picture_header) is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.
[0465]
group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. Broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.
[0466]
Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the picture header of the picture layer used in the past encoding process is described as a history stream. Yes.
[0467]
Data elements relating to this picture header (picture_header) are composed of picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, and backward_f_code.
[0468]
Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range.
[0469]
Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the final stage, the picture coding extension of the picture layer used in the past encoding process is described as a history stream.
[0470]
The data elements for this picture coding extension (picture_coding_extension) are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_carrier, burst_amplitude, burst_amplitude
[0471]
extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction.
[0472]
intra_dc_precision is data representing the precision of the DC coefficient. Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, the data indicates whether the upper field or the lower field. top_field_first is data indicating whether the first field is upper or lower in the case of a frame structure. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock.
[0473]
q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock. The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal.
[0474]
v_axis is data used when the source signal is PAL. The field_sequence is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.
[0475]
Subsequently, the quantization matrix extension used in the past encoding process is described as a history stream in the user area of the picture layer of the bit stream generated by the encoding process at the final stage.
[0476]
Data elements related to the quantization matrix extension (quant_matrix_extension) are, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantiser_matrix, intra_quantiser_matrix [64], load_non_intra_quantiser_matrix, non_intra_quantiser_matrix [64], load_chroma_intra_quantiser_matrix, chroma_intra_quantiser_matrix [64], is composed of Load_chroma_non_intra_quantiser_matrix, and chroma_non_intra_quantiser_matrix [64] The
[0477]
extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantiser_matrix is data indicating the value of a quantization matrix for an intra macroblock.
[0478]
load_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantiser_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantiser_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantiser_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for color difference non-intra macroblocks. chroma_non_intra_quantiser_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.
[0479]
Subsequently, the copyright extension used in the past encoding process is described as the history stream in the user area of the picture layer of the bit stream generated by the encoding process in the final stage.
[0480]
Data elements related to this copyright extension (copyright_extension) are composed of extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.
[0481]
extension_start_code is a start code indicating the start of the copyright extension. This code indicates which extension data of extension_start_code_itentifier is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid. copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end.
[0482]
The copyright_identifier is data for identifying the registration organization of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.
[0483]
Subsequently, the picture display extension (picture_display_extension) used in the past encoding process is described as a history stream in the user area of the picture layer of the bitstream generated by the encoding process at the final stage.
[0484]
Data elements representing this picture display extension include extension_start_code, extension_start_code_identifier, picture_display_extension_present_flag, frame_center_horizontal_offset_1, frame_center_vertical_offset_1, frame_center_horizontal_offset_2, frame_center_vertical_offset_2, frame_center_horizontal_offset_3, and frame_center_horizontal_offset_3.
[0485]
extension_start_code is a start code for indicating the start of the picture display extension. extension_start_code_identifier is a code indicating which extension data is sent. picture_display_extension_present_flag is data indicating whether a data element in the picture display extension is valid or invalid. The frame_center_horizontal_offset is data indicating a horizontal offset of the display area, and can be defined up to three offset values. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and can be defined up to three offset values.
[0486]
In the user area of the picture layer of the bitstream generated in the encoding process of the final stage, user data (user_data) used in the past encoding process is next to the history information indicating the picture display extension described above. , Described as a history stream.
[0487]
Next to the user data, information on the macroblock layer used in the past encoding process is described as a history stream.
[0488]
Information about the macroblock layer includes data elements related to macroblock (macroblock) positions such as macroblock_address_h, macroblock_address_v, slice_header_present_flag, skipped_macroblock_flag, macroblock_quant, macroblock_motion_forward, macroblock_motion_backward, macroblock_pattern, macro_block_frame, (Macroblock_modes []), data elements related to quantization step control such as quantizer_scale_code, PMV [0] [0] [0], PMV [0] [0] [1], motion_vertical_field_select [0] [0 ], PMV [0] [1] [0], PMV [0] [1] [1], motion_vertical_field_select [0] [1], PMV [1] [0] [0], PMV [1] [0] Motion compensation data elements such as [1], motion_vertical_field_select [1] [0], PMV [1] [1] [0], PMV [1] [1] [1], motion_vertical_field_select [1] [1] Macroblock such as coded_block_pattern And data elements related to the turn, num_mv_bits, is configured Num_coef_bits, and the data elements relating to the generated code amount of such Num_other_bits.
[0489]
Hereinafter, data elements related to the macroblock layer will be described in detail.
[0490]
macroblock_address_h is data for defining the absolute position of the current macroblock in the horizontal direction. macroblock_address_v is data for defining the absolute position of the current macroblock in the vertical direction. The slice_header_present_flag is data indicating whether or not this macroblock is the head of the slice layer and is accompanied by a slice header. skipped_macroblock_flag is data indicating whether or not to skip this macroblock in the decoding process.
[0491]
The macroblock_quant is data derived from a macroblock type (macroblock_type) shown in FIGS. 63 and 64 described later, and indicates whether quantizer_scale_code appears in the bitstream. The macroblock_motion_forward is data derived from the macroblock type shown in FIGS. 63 and 64, and is data used in the decoding process. The macroblock_motion_backward is data derived from the macroblock type shown in FIGS. 63 and 64, and is data used in the decoding process. mocroblock_pattern is data derived from the macroblock type shown in FIGS. 63 and 64, and indicates whether or not coded_block_pattern appears in the bitstream.
[0492]
The macroblock_intra is data derived from the macroblock type shown in FIGS. 63 and 64, and is data used in the decoding process. spatial_temporal_weight_code_flag is data derived from the macroblock type shown in FIGS. 63 and 64, and spatial_temporal_weight_code indicating the upsampling method of the lower layer image with temporal scalability is data indicating whether or not the bitstream exists. It is.
[0493]
frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”. field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime. dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode. quantiser_scale_code is data indicating the quantization step size of the macroblock.
[0494]
Next, data elements relating to motion vectors will be described. The motion vector is encoded as a difference with respect to the previously encoded vector in order to reduce the motion vector required during decoding. In order to perform motion vector decoding, the decoder must maintain four motion vector prediction values (with horizontal and vertical components, respectively). This predicted motion vector is expressed as PMV [r] [s] [v]. [r] is a flag indicating whether the motion vector in the macroblock is the first vector or the second vector, and is “0” when the vector in the macroblock is the first vector. Thus, when the vector in the macroblock is the second vector, “1” is obtained. [s] is a flag indicating whether the direction of the motion vector in the macroblock is the forward direction or the backward direction, and is “0” in the case of the forward motion vector, and the backward motion vector In this case, it is “1”. [v] is a flag indicating whether the vector component in the macroblock is the horizontal direction or the vertical direction, and is “0” in the case of the horizontal component, and in the case of the vertical component Becomes “1”.
[0495]
Therefore, PMV [0] [0] [0] represents the horizontal component data of the forward motion vector of the first vector, and PMV [0] [0] [1] represents the first vector. PMV [0] [1] [0] represents the vertical component data of the forward motion vector, PMV [0] [1] [0] represents the horizontal component data of the backward motion vector of the first vector, and PMV [0] [ 1] [1] represents the data of the vertical component of the backward motion vector of the first vector, and PMV [1] [0] [0] represents the horizontal of the forward motion vector of the second vector. Represents the direction component data, PMV [1] [0] [1] represents the vertical component data of the forward motion vector of the second vector, and PMV [1] [1] [0] PMV [1] [1] [1] represents the vertical component data of the backward motion vector of the second vector, and represents the horizontal component data of the backward motion vector of the second vector. Yes.
[0496]
motion_vertical_field_select [r] [s] is data indicating which reference field is used for the prediction format. When the motion_vertical_field_select [r] [s] is “0”, the top reference field is used, and when it is “1”, the bottom reference field is used.
[0497]
Therefore, motion_vertical_field_select [0] [0] indicates a reference field for generating a forward motion vector of the first vector, and motion_vertical_field_select [0] [1] indicates a backward motion vector of the first vector. , Motion_vertical_field_select [1] [0] indicates a reference field when generating a forward motion vector of the second vector, and motion_vertical_field_select [1] [1] indicates the second A reference field for generating a backward motion vector of the vector is shown.
[0498]
The coded_block_pattern is variable-length data indicating which DCT block has a significant coefficient (non-zero coefficient) among a plurality of DCT blocks storing DCT coefficients. num_mv_bits is data indicating the code amount of the motion vector in the macroblock. num_coef_bits is data indicating the code amount of the DCT coefficient in the macroblock. num_other_bits is data indicating the code amount of the macroblock and the code amount other than the motion vector and the DCT coefficient.
[0499]
Next, a syntax for decoding each data element from a variable-length history stream will be described with reference to FIGS.
[0500]
This variable length history stream consists of next_start_code () function, sequence_header () function, sequence_extension () function, extension_and_user_data (0) function, group_of_picture_header () function, extension_and_user_data (1) function, picture_header () function, picture_coding_extension () function, It consists of data elements defined by the re_coding_stream_info () function, extension_and_user_data (2) function, and picture_data () function.
[0501]
Since the next_start_code () function is a function for searching for a start code existing in the bit stream, the top of the history stream is a data element used in the past encoding process as shown in FIG. A data element defined by the sequence_header () function is described.
[0502]
The data elements defined by the sequence_header () function are sequence_header_code, sequence_header_present_flag, horizontal_size_value, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, marker_bit, VBV_buffer_size_value, constrained_parameter_flag, load_intra_intra_intra_iser_iser, etc.
[0503]
The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle. bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits.
[0504]
The marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantiser_matrix is data indicating the presence of intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the intra MB quantization matrix. load_non_intra_quantiser_matrix is data indicating the presence of non-intra MB quantization matrix data. non_intra_quantiser_matrix is data representing a value of a non-intra MB quantization matrix.
[0505]
Next to the data element defined by the sequence_header () function, the data element defined by the sequence_extension () function as shown in FIG. 49 is described as a history stream.
[0506]
Data elements defined by the sequence_extension () function are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_extension, bit_rate_extension, _delay_rate_rate_delay_rate_delay_rate
[0507]
extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data. The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header.
[0508]
low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.
[0509]
Next to the data element defined by the sequence_extension () function, the data element defined by the extension_and_user_data (0) function as shown in FIG. 50 is described as a history stream. The extension_and_user_data (i) function describes only the data element defined by the user_data () function as a history stream without describing the data element defined by the extension_data () function when “i” is other than 1. . Therefore, the extension_and_user_data (0) function describes only the data element defined by the user_data () function as a history stream.
[0510]
The user_data () function describes user data as a history stream based on the syntax as shown in FIG.
[0511]
Next to the data element defined by the extension_and_user_data (0) function, the data element defined by the group_of_picture_header () function as shown in FIG. 52 and the data element defined by the extension_and_user_data (1) function are used as a history stream. is described. However, the data element defined by the group_of_picture_header () function and the data element defined by the extension_and_user_data (1) function are described only when group_start_code indicating the GOP layer start code is described in the history stream. ing.
[0512]
The data element defined by the group_of_picture_header () function is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.
[0513]
group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. Broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.
[0514]
Similar to the extension_and_user_data (0) function, the extension_and_user_data (1) function describes only the data element defined by the user_data () function as a history stream.
[0515]
If group_start_code indicating the GOP layer start code does not exist in the history stream, the data elements defined by these group_of_picture_header () and extension_and_user_data (1) functions are not described in the history stream. Absent. In this case, after the data element defined by the extension_and_user_data (0) function, the data element defined by the picture_headr () function is described as a history stream.
[0516]
Data elements defined by the picture_headr () function are picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, backward_f_code, extra_bit_picture, and extra_information_picture, as shown in FIG.
[0517]
Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range. extra_bit_picture is a flag indicating the presence of subsequent additional information. When this extra_bit_picture is “1”, there is next extra_information_picture, and when extra_bit_picture is “0”, it indicates that there is no subsequent data. extra_information_picture is information reserved in the standard.
[0518]
Next to the data element defined by the picture_headr () function, the data element defined by the picture_coding_extension () function as shown in FIG. 54 is described as a history stream.
[0519]
The data elements defined by this picture_coding_extension () function are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_mplitude, phase_st, sub_carrier, burst
[0520]
extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction. intra_dc_precision is data representing the precision of the DC coefficient.
[0521]
Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, the data indicates whether the upper field or the lower field. top_field_first is data indicating whether the first field is upper or lower in the case of a frame structure. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock.
[0522]
The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal. v_axis is data used when the source signal is PAL. The field_sequence is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.
[0523]
Next to the data element defined by the picture_coding_extension () function, the data element defined by the re_coding_stream_info () function is described as a history stream. The re_coding_stream_info () function is mainly used when describing a combination of history information, and details thereof will be described later with reference to FIG.
[0524]
Next to the data element defined by the re_coding_stream_info () function, the data element defined by extensions_and_user_data (2) is described as a history stream. As shown in FIG. 50, the extension_and_user_data (2) function describes data elements defined by the extension_data () function when an extension start code (extension_start_code) exists in the bitstream. Next to this data element, when a user data start code (user_data_start_code) exists in the bitstream, a data element defined by the user_data () function is described. However, when the extension start code and the user data start code do not exist in the bit stream, the data elements defined by the extension_data () function and the user_data () function are not described in the bit stream.
[0525]
As shown in FIG. 55, the extension_data () function records a data element indicating extension_start_code and a data element defined by the quant_matrix_extension () function, copyright_extension () function, and picture_display_extension () function in the bitstream. This is a function to describe as a stream.
[0526]
Data elements defined by the quant_matrix_extension () function, as shown in FIG. 56, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantiser_matrix, intra_quantiser_matrix [64], load_non_intra_quantiser_matrix, non_intra_quantiser_matrix [64], load_chroma_intra_quantiser_matrix, chroma_intra_quantiser_matrix [64], load_chroma_non_intra_quantiser_matrix, and chroma_non_intra_quantiser_matrix [64].
[0527]
extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantiser_matrix is data indicating the value of a quantization matrix for an intra macroblock.
[0528]
load_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantiser_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantiser_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantiser_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for color difference non-intra macroblocks. chroma_non_intra_quantiser_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.
[0529]
As shown in FIG. 57, the data element defined by the copyright_extension () function includes extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.
[0530]
extension_start_code is a start code indicating the start of the copyright extension. extension_start_code_itentifier This code indicates which extension data is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid.
[0531]
copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end. The copyright_identifier is data for identifying the registration organization of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.
[0532]
Data elements defined by the picture_display_extension () function are extension_start_code_identifier, frame_center_horizontal_offset, frame_center_vertical_offset, etc., as shown in FIG.
[0533]
extension_start_code_identifier is a code indicating which extension data is sent. The frame_center_horizontal_offset is data indicating the horizontal offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined.
[0534]
Returning again to FIG. 47, after the data element defined by the extension_and_user_data (2) function, the data element defined by the picture_data () function is described as a history stream. However, this picture_data () function exists when red_bw_flag is not 1 or red_bw_indicator is 2 or less. The red_bw_flag and red_bw_indicator are described in the re_coding_stream_info () function, which will be described later with reference to FIGS. 71 and 72.
[0535]
The data element defined by the picture_data () function is a data element defined by the slice () function as shown in FIG. At least one data element defined by the slice () function is described in the bit stream.
[0536]
As shown in FIG. 60, the slice () function includes data elements such as slice_start_code, slice_quantiser_scale_code, intra_slice_flag, intra_slice, reserved_bits, extra_bit_slice, extra_information_slice, and extra_bit_slice, as well as data elements defined by the macroblock () function. It is a function to describe as.
[0537]
The slice_start_code is a start code indicating the start of the data element defined by the slice () function. The slice_quantiser_scale_code is data indicating the quantization step size set for the macroblock existing in this slice layer. However, when quantiser_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used preferentially.
[0538]
intra_slice_flag is a flag indicating whether or not intra_slice and reserved_bits exist in the bitstream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_slice is “0”, and if all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. Become. reserved_bits is 7-bit data and takes a value of “0”. extra_bit_slice is a flag indicating that additional information exists as a history stream, and is set to “1” when extra_information_slice exists next. If there is no additional information, it is set to “0”.
[0539]
Next to these data elements, data elements defined by the macroblock () function are described as a history stream.
[0540]
As shown in FIG. 61, the macroblock () function includes data elements such as macroblock_escape, macroblock_address_increment, macroblock_quantiser_scale_code, and marker_bit, and data elements defined by the macroblock_modes () function, motion_vectors (s) function, and code_block_pattern () function. Is a function for describing
[0541]
macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, 33 is added to the value of macroblock_address_increment. The macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If there is one macroblock_escape before this macroblock_address_increment, the value obtained by adding 33 to the value of the macroblock_address_increment is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock. .
[0542]
The macroblock_quantiser_scale_code is a quantization step size set for each macroblock, and exists only when the macroblock_quant is “1”. In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set. When macroblock_quantiser_scale_code is set for the reference macroblock, this quantization step size is selected.
[0543]
Next to macroblock_address_increment, a data element defined by the macroblock_modes () function is described. As shown in FIG. 62, the macroblock_modes () function is a function for describing data elements such as macroblock_type, frame_motion_type, field_motion_type, and dct_type as a history stream.
[0544]
macroblock_type is data indicating the coding type of the macroblock. Details thereof will be described later with reference to FIGS. 65 to 67.
[0545]
If macroblock_motion_forward or macroblock_motion_backward is “1”, the picture structure is a frame, and frame_pred_frame_dct is “0”, a data element representing frame_motion_type is described after the data element representing macroblock_type. The frame_pred_frame_dct is a flag indicating whether or not the frame_motion_type exists in the bitstream.
[0546]
frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”.
[0547]
If the condition describing the frame_motion_type is not satisfied, the data element representing the field_motion_type is described next to the data element representing the macroblock_type.
[0548]
field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime.
[0549]
If the picture structure is a frame, frame_pred_frame_dct indicates that frame_motion_type is present in the bitstream, and frame_pred_frame_dct indicates that dct_type is present in the bitstream, then the data element representing the macroblock_type Describes a data element representing dct_type. Dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode.
[0550]
Referring back to FIG. 61, if the reference macroblock is a forward prediction macroblock or the reference macroblock is an intra macroblock and is a concealment macroblock, motion_vectors (0) Describes a data element defined by a function. When the reference macroblock is a backward prediction macroblock, a data element defined by the motion_vectors (1) function is described. The motion_vectors (0) function is a function for describing a data element related to the first motion vector, and the motion_vectors (1) function is a function for describing a data element related to the second motion vector. It is.
[0551]
The motion_vectors (s) function is a function for describing data elements relating to motion vectors, as shown in FIG.
[0552]
If there is one motion vector and the dial prime prediction mode is not used, a data element defined by motion_vertical_field_select [0] [s] and motion_vector (0, s) is described.
[0553]
In this motion_vertical_field_select [r] [s], the first motion vector (which may be either forward or backward vector) is a vector created by referring to the bottom field or refers to the top field. Is a flag indicating whether the vector is a This index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an indicator that shows.
[0554]
As shown in FIG. 64, the motion_vector (r, s) function includes a data string related to motion_code [r] [s] [t], a data string related to motion_residual [r] [s] [t], and dmvector [t ] Is a function for describing the data representing [].
[0555]
motion_code [r] [s] [t] is variable-length data representing the magnitude of the motion vector in the range of −16 to +16. motion_residual [r] [s] [t] is variable-length data representing a motion vector residual. Therefore, a detailed motion vector can be described by the values of motion_code [r] [s] [t] and motion_residual [r] [s] [t]. dmvector [t] is an existing value according to the time distance in order to generate a motion vector in one field (for example, the top field is one field with respect to the bottom field) in the dual prime prediction mode. The motion vector is scaled and the vertical direction is corrected in order to reflect the vertical shift between the top field and the bottom field lines. This index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is whether the prediction direction is forward or backward prediction. It is an indicator that shows. “S” is data indicating whether the motion vector is a vertical component or a horizontal component.
[0556]
64. First, a data string representing motion_coder [r] [s] [0] in the horizontal direction is described as a history stream by the motion_vector (r, s) function shown in FIG. Since the number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], f_code [s] [t] is not 1 In this case, motion_residual [r] [s] [t] is present in the bitstream. The motion_residual [r] [s] [0] of the horizontal component is not “1” and the motion_code [r] [s] [0] of the horizontal component is not “0”. Since there is a data element representing [r] [s] [0], which means that there is a horizontal component of the motion vector, in that case, motion_residual [r] [s ] A data element representing [0] is described.
[0557]
Subsequently, a data string representing motion_coder [r] [s] [1] in the vertical direction is described as a history stream. Similarly, since the number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], f_code [s] [t] Is not 1, it indicates that motion_residual [r] [s] [t] is present in the bitstream. motion_residual [r] [s] [1] is not “1” and motion_code [r] [s] [1] is not “0”. This means that motion_residual [r] [s] [1] Means that there is a vertical component of the motion vector, and in this case, the data element that represents the vertical component motion_residual [r] [s] [1] Is described.
[0558]
Next, macroblock_type will be described with reference to FIGS. The macroblock_type is variable length data generated from flags such as macroblock_quant, dct_type_flag, macroblock_motion_forward, and macroblock_motion_backward. macroblock_quant is a flag indicating whether or not macroblock_quantiser_scale_code for setting the quantization step size for the macroblock is set. When macroblock_quantiser_scale_code exists in the bitstream, macroblock_quant is a value of “1”. I take the.
[0559]
dct_type_flag is a flag for indicating whether or not dct_type indicating whether the reference macroblock is encoded in the frame DCT or the field DCT is present (in other words, a flag indicating whether or not the DCT is DCT), and is a bit. When dct_type exists in the stream, this dct_type_flag takes a value of “1”. The macroblock_motion_forward is a flag indicating whether or not the reference macroblock is predicted forward, and takes a value of “1” when the reference macroblock is predicted forward. macroblock_motion_backward is a flag indicating whether or not the reference macroblock is predicted backward, and takes a value of “1” when backward prediction is performed.
[0560]
In the variable length format, history information can be reduced in order to reduce the transmission bit rate.
[0561]
That is, when macroblock_type and motion_vectors () are transferred but quantizer_scale_code is not transferred, the bit rate can be reduced by setting slice_quantiser_scale_code to “00000”.
[0562]
In addition, when only macroblock_type is transferred and motion_vectors (), quantizer_scale_code, and dct_type are not transferred, the bit rate can be reduced by using “not coded” as macroblock_type.
[0563]
Furthermore, when only picture_coding_type is transferred and all information below slice () is not transferred, the bit rate can be reduced by using picture_data () without slice_start_code.
[0564]
In the above, “1” is inserted every 22 bits in order to prevent the continuous “0” of 23 bits in user_data from being output. However, it may not be every 22 bits. Further, instead of counting the number of consecutive “0” s and inserting “1”, it is possible to check and insert Byte_allign.
[0565]
Furthermore, in MPEG, the generation of 23 bits of continuous “0” is prohibited, but in reality, only the case where 23 bits are continued from the beginning of the byte is a problem. If 0 continues for 23 bits, this is not a problem. Therefore, for example, “1” may be inserted at a position other than the LSB every 24 bits.
[0566]
In the above description, the history information is in a format close to a video elementary stream, but may be in a format close to a packetized elementary stream or a transport stream. In addition, although the location of user_data in Elementary Stream is in front of picture_data, it can be other locations.
[0567]
In the transcoder 101 of FIG. 15, encoding parameters for four generations are output to the subsequent stage as history information. However, not all of the history information is actually required and is required for each application. History information will be different. In addition, the actual transmission path or recording medium (transmission medium) has a limited capacity, and although it is compressed, if all history information is transmitted, it will be a burden on capacity, and as a result The bit rate of the image bit stream is suppressed, and the effectiveness of history information transmission is impaired.
[0568]
Therefore, a descriptor describing a combination of items to be transmitted as history information is incorporated into the history information and transmitted to the subsequent stage so that information corresponding to various applications is transmitted instead of transmitting all history information. Can be. FIG. 68 shows a configuration example of the transcoder 101 in such a case.
[0569]
In FIG. 68, portions corresponding to those in FIG. 15 are denoted with the same reference numerals, and description thereof will be omitted as appropriate. In the configuration example of FIG. 68, an encoding parameter selection circuit 501 is inserted between the history information separating apparatus 105 and the encoding apparatus 106 and between the history encoding apparatus 107 and the encoding apparatus 106.
[0570]
An encoding parameter selection circuit 501 includes an encoding parameter calculation unit 512 that calculates an encoding parameter from a baseband video signal output by the history information separation device 105, and an output of the history information separation device 105 in the transcoder 101. Coding parameters and descriptors (red_bw_flag, red_bw_indicator) (described later with reference to FIG. 72) are separated from information on coding parameters determined to be optimal (for example, second generation coding parameters). The combination descriptor separation unit 511 outputs either one of the encoding parameters output from the combination descriptor separation unit 511 and the encoding parameter calculation unit 512 and the encoding parameter output from the combination descriptor separation unit 511. A switch is selected corresponding to the separated descriptor and output to the encoding device 106. It has a switch 513. Other configurations are the same as those in FIG.
[0571]
Here, combinations of items to be transmitted as history information will be described. The history information can be divided into information in picture units and information in macroblock units. Information in units of slices can be obtained by collecting information on macroblocks included therein, and information in units of GOPs can be obtained by collecting information in units of pictures included therein.
[0572]
Since information in picture units is transmitted only once per frame, the bit rate occupied in information transmission is not so large. On the other hand, since information in units of macroblocks is transmitted for each macroblock, for example, in the case of a video system having 525 scanning lines per frame and a field rate of 60 fields / second, the number of pixels per frame Is 720 × 480, it is necessary to transmit information in units of macroblock 1350 (= (720/16) × (480/16)) times per frame. For this reason, a considerable part of the history information is occupied by information for each macroblock. Therefore, as history information, at least information in units of pictures is always transmitted, but information in units of macroblocks can be selected and transmitted according to the application, thereby suppressing the amount of information to be transmitted.
[0573]
The macroblock unit information transferred as history information includes, for example, num_coef_bits, num_mv_bits, num_other_bits, q_scale_code, q_scale_type, motion_type, mv_vert_field_sel [] [], mv [] [] [], mb_mfwd, mb_mbwd, mb_pattern, _coded_block_pattern, There are slice_start, dct_type, mb_quant, skipped_mb, etc. These are expressed using the element of macroblock rate information.
[0574]
num_coef_bits represents the code amount required for the DCT coefficient among the code amounts of the macroblock. num_mv_bits represents the code amount required for the motion vector among the code amounts of the macroblock. num_other_bits represents a code amount other than num_coef_bits and num_mv_bits among the code amounts of the macroblock.
[0575]
q_scale_code represents q_scale_code applied to the macroblock. motion_type represents the type of motion vector applied to the macroblock. mv_vert_field_sel [] [] represents a field select of a motion vector applied to a macroblock.
[0576]
mv [] [] [] represents a motion vector applied to a macroblock. mb_mfwd is a flag indicating that the prediction mode of the macroblock is forward prediction. mb_mbwd is a flag indicating that the prediction mode of the macroblock is backward prediction. The mb_pattern is a flag indicating whether or not there is a non-zero DCT coefficient of the macroblock.
[0577]
The coded_block_pattern is a flag indicating the presence or absence of non-zero macroblock DCT coefficients for each DCT block. mb_intra is a flag indicating whether the macroblock is intra_macro or not. slice_start is a flag indicating whether or not the macroblock is the head of the slice. dct_type is a flag indicating whether the macroblock is field_dct or flame_dct.
[0578]
mb_quant is a flag indicating whether or not the macroblock transmits quantizer_scale_code. skipped_mb is a flag indicating whether or not the macroblock is a skipped macroblock.
[0579]
All of these items are not always necessary, and the necessary items vary depending on the application. For example, items such as num_coef_bits and slice_start are necessary in an application having a request for transparent to restore the bitstream at the time of re-encoding as much as possible. In other words, these items are not necessary in an application that changes the bit rate. In addition, there are applications in which it is only necessary to know the coding type of each picture when the transmission path is extremely limited. From such a situation, for example, a combination shown in FIG. 69 can be considered as an example of a combination of items for transmitting history information.
[0580]
In FIG. 69, the value “2” corresponding to the item in each combination means that the information exists and can be used, and “0” means that the information does not exist. “1” indicates that the information itself has no meaning, for example, for the purpose of assisting the existence of other information, or syntactically, but not related to the original bitstream information. . For example, slice_start is “1” in the first macroblock of the slice when transmitting history information, but if the slice is not necessarily in the same positional relationship with the original bitstream, It becomes meaningless as information.
[0581]
In the example of FIG. 69, (num_coef_bits, num_mv_bits, num_other_bits), (q_scale_code, q_scale_type), (motion_type, mv_vert_field_sel [] [], mv [] [] []), (mb_mfwd, mb_mbwd), (mb_pattern), (mb_pattern), Depending on the presence or absence of each item of coded_block_pattern), (mb_intra), (slice_start), (dct_type), (mb_quant), and (skipped_mb), five combinations of combinations 1 to 5 are prepared.
[0582]
Combination 1 is a combination intended to reconstruct a completely transparent bit stream. According to this combination, highly accurate transcoding can be realized by using the generated code amount information. Combination 2 is also a combination intended to reconstruct a completely transparent bit stream. The combination 3 is a combination for allowing a completely transparent bit stream to be visually reconstructed although a completely transparent bit stream cannot be reconstructed. Combination 4 is inferior to combination 3 from the viewpoint of transparent, but is a combination that can reconstruct a bitstream with no visual problem. The combination 5 is inferior to the combination 4 from the viewpoint of transparent, but is a combination capable of incomplete reconstruction of the bitstream with a small amount of history information.
[0583]
Of these combinations, the smaller the combination number, the higher the function, but the larger the capacity required to transfer the history. Therefore, it is necessary to determine the combination to be transmitted by considering the assumed application and the capacity available for the history.
[0584]
Next, the operation of the transcoder 101 of FIG. 68 will be described with reference to the flowchart of FIG. In step S41, the decoding device 102 of the transcoder 101 decodes the input bitstream, extracts the encoding parameter (4th) used when encoding the bitstream, and extracts the encoding parameter (4th). ) To the history information multiplexer 103 and the decoded video data is also output to the history information multiplexer 103. In step S <b> 42, the decoding apparatus 102 also extracts user_data from the input bitstream and outputs it to the history decoding apparatus 104. In step S43, the history decoding apparatus 104 extracts combination information (descriptor) from the input user_data, and further uses it to extract encoding parameters (1st, 2nd, 3rd) as history information. And output to the history information multiplexing apparatus 103.
[0585]
In step S44, the history information multiplexing apparatus 103 sends the current encoding parameter (4th) supplied from the decoding apparatus 102 extracted in step S41 and the past encoding output from the history decoding apparatus 104 in step S43. The parameters (1st, 2nd, 3rd) are multiplexed on the baseband video data supplied from the decoding device 102 according to the format shown in FIG. 18 or FIG. 31, and output to the history information separation device 105.
[0586]
In step S45, the history information separation device 105 extracts coding parameters from the baseband video data supplied from the history information multiplexing device 103, and from among them, the coding parameters most suitable for the current coding ( For example, the second generation encoding parameter) is selected and output to the combination descriptor separating unit 511 together with the descriptor. Further, the history information separating apparatus 105 encodes an encoding parameter other than the encoding parameter determined to be optimal for the current encoding (for example, when the optimal encoding parameter is determined to be the second generation encoding parameter). Are output to the history encoding apparatus 107 (the other first generation, third generation, and fourth generation encoding parameters). In step S46, the history encoding apparatus 107 describes the encoding parameter input from the history information separation apparatus 105 in user_data, and outputs the user_data (converted_history_stream ()) to the encoding apparatus 106.
[0587]
The combination descriptor separation unit 511 of the coding parameter selection circuit 501 separates the coding parameter and the descriptor from the data supplied from the history information separation device 105, and sets the coding parameter (2nd) as one contact point of the switch 513. To supply. The encoding parameter calculation unit 512 calculates and supplies encoding parameters from the baseband video data output from the history information separation device 105 to the other contact of the switch 513. In step S48, the switch 513 corresponds to the descriptor output from the combination descriptor separation unit 511, or the encoding parameter output from the combination descriptor separation unit 511 or the encoding parameter output from the encoding parameter calculation unit 512. Is selected and output to the encoding device 106. That is, in the switch 513, when the encoding parameter supplied from the combination descriptor separation unit 511 is valid, the encoding parameter output from the combination descriptor separation unit 511 is selected. If it is determined that the encoding parameter output by the 511 is invalid, the encoding parameter calculated by the encoding parameter calculation unit 512 processing the baseband video is selected. This selection is performed according to the capacity of the transmission medium.
[0588]
In step S49, the encoding device 106 encodes the baseband video signal supplied from the history information separation device 105 based on the encoding parameter supplied from the switch 513. In step S50, the encoding device 106 multiplexes user_data supplied from the history encoding device 107 into the encoded bitstream and outputs the multiplexed data.
[0589]
In this way, even when the combination of encoding parameters obtained by each history is different, transcoding can be performed without any trouble.
[0590]
Thus, as shown in FIG. 38, the history information is transmitted by history_stream () (more precisely, converted_history_stream ()) as a kind of user_data () function of the video stream. The syntax of the history_stream () is as shown in FIG. A descriptor (red_bw_flag, red_bw_indicator) representing a combination of items of history information, and items (num_other_bits, num_mv_bits, num_coef_bits) not supported in the MPEG stream are transmitted by the re_coding_stream_info () function in FIG.
[0591]
As shown in FIG. 71, the re_coding_stream_info () function includes data elements such as user_data_start_code, re_coding_stream_info_ID, red_bw_flag, red_bw_indicator, marker_bit, num_other_bits, num_mv_bits, and num_coef_bits.
[0592]
user_data_start_code is a start code indicating that user_data starts. The re_coding_stream_info_ID is a 16-bit integer and is used for identifying the re_coding_stream_info () function. Specifically, the value is “1001 0001 1110 1100” (0x91ec).
[0593]
The red_bw_flag is a 1-bit flag, and is set to 0 when the history information transmits all items. When the value of this flag is 1, the red_bw_indicator following this flag is checked, as shown in FIG. Of the five combinations, it is possible to determine which combination is sending the item.
[0594]
red_bw_indicator is a 2-bit integer and describes a combination of items as shown in FIG.
[0595]
That is, among the five combinations shown in FIG. 69, in the case of the combination 1, red_bw_flag is 0, and in the case of the combinations 2 to 5, red_bw_flag is 1. On the other hand, red_bw_indicator is 0 for combination 2, 1 for combination 3, 2 for combination 4, and 3 for combination 5.
[0596]
Therefore, red_bw_indicator is defined when red_bw_flag is 1 (in the case of combination 2 to combination 5).
[0597]
Furthermore, as shown in FIG. 71, when red_bw_flag is 0 (in the case of combination 1), marker_bit, num_other_bits, num_mv_bits, and num_coef_bits are described for each macroblock. These four data elements are not defined for combinations 2 to 5 (when red_bw_flag is 1).
[0598]
As shown in FIG. 59, the picture_data () function is composed of one or more slice () functions. However, in the case of the combination 5, the syntax elements below it including the picture_data () function are not transmitted (FIG. 69). In this case, the history information is intended to transmit information in units of picture such as picture_type.
[0599]
In the case of the combination 1 to the combination 4, the slice () function shown in FIG. 60 exists. However, the position information of the slice determined by the slice () function and the position information of the slice of the original bitstream depend on the combination of items of history information. In the case of the combination 1 or the combination 2, the position information of the slice of the bit stream that is the source of the history information and the position information of the slice determined by the slice () function need to be the same.
[0600]
The syntax element of the macroblock () function shown in FIG. 61 depends on the combination of history information items. The macroblock_escape, macroblock_address_increment, and macroblock_modes () functions are always present. However, the effectiveness of macroblock_escape and macroblock_address_increment as information is determined by the combination. When the combination of history information items is combination 1 or combination 2, the same information as the skipped_mb information of the original bitstream needs to be transmitted.
[0601]
In the case of the combination 4, there is no motion_vectors () function. In the case of the combination 1 to the combination 3, the presence of the motion_vectors () function is determined by the macroblock_type of the macroblock_modes () function. In the case of the combination 3 or the combination 4, the coded_block_pattern () function does not exist. In the case of the combination 1 and the combination 2, the presence of the coded_block_pattern () function is determined by the macroblock_type of the macroblock_modes () function.
[0602]
The syntax element of the macroblock_modes () function shown in FIG. 62 depends on the combination of history information items. macroblock_type is always present. When the combination is combination 4, flame_motion_type, field_motion_type, and dct_type do not exist.
[0603]
The effectiveness of the parameter obtained from macroblock_type as information is determined by a combination of items of history information.
[0604]
When the combination of history information items is combination 1 or combination 2, macroblock_quant needs to be the same as the original bitstream. For combination 3 or combination 4, macroblock_quant represents the presence of quantizer_scale_code in the macroblock () function and need not be the same as the original bitstream.
[0605]
When the combination is combination 1 to combination 3, macroblock_motion_forward and macroblock_motion_backward need to be the same as the original bitstream. If the combination is combination 4 or combination 5, this is not necessary.
[0606]
When the combination is combination 1 or combination 2, macroblock_pattern needs to be the same as the original bitstream. For combination 3, macroblock_pattern is used to indicate the presence of dct_type. When the combination is combination 4, the relationship as in combination 1 to combination 3 is not established.
[0607]
When the combination of history information items is combination 1 to combination 3, macroblock_intra needs to be the same as the original bitstream. In the case of combination 4, this is not the case.
[0608]
The history_stream () in FIG. 47 has a syntax when the history information has a variable length. However, as shown in FIGS. 40 to 46, when the syntax has a fixed length, the history information has a fixed length in the history information. Descriptors (red_bw_flag and red_bw_indicator) as information indicating which of the items to be transmitted are valid are superimposed on the baseband image and transmitted. As a result, by examining this descriptor, it is possible to determine that it exists as a field but its contents are invalid.
[0609]
Therefore, as shown in FIG. 44, user_data_start_code, re_coding_stream_info_ID, red_bw_flag, red_bw_indicator, and marker_bit are arranged as re_coding_stream_information. Each meaning is the same as that in FIG.
[0610]
Thus, by transmitting the elements of the encoding parameter transmitted as the history in a combination according to the application, it is possible to transmit the history according to the application with an appropriate amount of data.
[0611]
As described above, when history information is transmitted as a variable-length code, the re_coding_stream_info () function is configured as shown in FIG. 71, and is transmitted as part of the history_stream () function as shown in FIG. On the other hand, when the history information is transmitted as a fixed-length code, re_coding_stream_information () is transmitted as part of the history_stream () function, as shown in FIG. In the example of FIG. 44, user_data_start_code, re_coding_stream_info_ID, red_bw_flag, and red_bw_indicator are transmitted as re_coding_stream_information.
[0612]
Also, a Re_Coding information Bus macroblock format as shown in FIG. 73 is defined for transmission of history information in the baseband signal output from the history information multiplexing apparatus 103 in FIG. This macro block is composed of 16 × 16 (= 256) bits. In FIG. 73, 32 bits shown in the third and fourth rows from the top are picrate_element. In this picrate_element, picture rate elements shown in FIGS. 74 to 76 are described. In FIG. 74, 1-bit red_bw_flag is defined in the second line from the top, and 3-bit red_bw_indicator is defined in the third line. That is, these flags red_bw_flag and red_bw_indicator are transmitted as picrate_element in FIG.
[0613]
The other data in FIG. 73 will be described. SRIB_sync_code is a code indicating that the first row of the macroblock of this format is aligned left-justified, and specifically, is set to “11111”. fr_fl_SRIB is set to 1 when picture_structure has a frame picture structure (when its value is “11”), indicating that Re_Coding Information Bus macroblock is transmitted over 16 lines, and picture_structure is not a frame structure In this case, it is set to 0, which means that Re_Coding Information Bus is transmitted over 16 lines. This mechanism locks the Re_Coding Information Bus to the corresponding pixel of the video frame or field decoded spatially and temporally.
[0614]
SRIB_top_field_first is set to the same value as top_field_first held in the original bitstream, and represents the temporal alignment of Re_Coding Information Bus of the related video together with repeat_first_field. SRIB_repeat_first_field is set to the same value as repeat_first_field held in the original bitstream. The content of Re_Coding Information Bus in the first field needs to be repeated as indicated by this flag.
[0615]
422_420_chroma represents whether the original bit stream is 4: 2: 2 or 4: 2: 0. The value of 0 indicates that the bitstream is 4: 2: 0 and that the upsampling of the color difference signal is performed so that 4: 2: 2 video is output. The value 0 indicates that the color difference signal filtering process is not executed.
[0616]
rolling_SRIB_mb_ref represents a 16-bit modulo 65521, and this value is incremented for each macroblock. This value must be continuous across frames of the frame picture structure. Otherwise, this value must be continuous across the field. This value is initialized to a predetermined value between 0 and 65520. This allows the incorporation of a unique Re_Coding Information Bus identifier into the recorder system.
[0617]
The meaning of the other data of the Re_Coding Information Bus macroblock is as described above, and is omitted here.
[0618]
As shown in FIG. 77, the 256-bit Re_Coding Information Bus data in FIG. 73 is Cb [0] [0], Cr [0] [0], Cb [1] which are LSBs of color difference data bit by bit. [0], Cr [1] [0]. Since the 4-bit data can be sent with the format shown in FIG. 77, the 256-bit data in FIG. 73 can be transmitted by sending 64 (= 256/4) formats in FIG.
[0619]
According to the transcoder of the present invention, since the encoding parameters generated in the past encoding process are reused in the current encoding process, the image quality can be improved even if the decoding process and the encoding process are repeated. No deterioration occurs. That is, accumulation of image quality degradation due to repetition of decoding processing and encoding processing can be reduced.
[0620]
78 and 79 show a configuration example when the transcoder of the present invention is applied to a video tape recorder. 78 shows a configuration example of the recording system of the video tape recorder 601, and FIG. 79 shows a configuration example of the playback system of the video tape recorder 601.
[0621]
A video tape recorder 601 shown in FIG. 78 includes a transcoder 101R, a channel encoding device 602, and a recording head 603. The configuration of the transcoder 101R is basically the same as that of the transcoder shown in FIG. In this configuration example, the transcoder 101R converts a long GOP bit stream ST into a short GOP bit stream ST.
[0622]
The fourth generation encoded stream ST output from the encoding device 106 of the transcoder 101R is supplied to the channel encoding device 602. As described above, the user data user_data including the first generation to third generation encoding parameters is recorded in the user data area of the picture layer of the fourth generation encoded stream ST.
[0623]
The channel encoding device 602 attaches a parity code for error correction to the input fourth generation encoded stream, and then performs channel encoding using, for example, the NRZI modulation method, and supplies it to the recording head 603. The recording head 603 records the input encoded stream on the magnetic tape 604.
[0624]
As shown in FIG. 79, in the reproducing system, a signal is generated from the magnetic tape 604 by the reproducing head 611 and supplied to the channel decoding device 612. The channel decoding device 612 channel-decodes the signal supplied from the reproducing head 611 and corrects errors using parity.
[0625]
The fourth generation encoded stream ST output from the channel decoding apparatus 612 is input to the transcoder 101P. The basic configuration of the transcoder 101P is the same as that of the transcoder shown in FIG.
[0626]
The decoding device 102 of the transcoder 101P extracts the user data user_data including the first generation to third generation encoding parameters from the fourth generation encoded stream, and sends it to the history decoding device 104 and the encoding device 106. Supply. The history decoding apparatus 104 decodes the input user data user_data and supplies the obtained first generation to third encoding parameters to the encoding apparatus 106.
[0627]
The decoding apparatus 102 also decodes the fourth generation encoded stream ST and outputs a baseband video signal and a fourth generation encoding parameter. The baseband video signal is supplied to the encoding device 106, and the fourth generation encoding parameters are supplied to the encoding device 106 and the history encoding device 107.
[0628]
The history encoding device 107 converts the input fourth-generation encoding parameter into user data user_data and supplies it to the encoding device 106.
[0629]
As described above, the controller 70 of the encoding device 106 determines whether or not the picture type of each picture determined from the GOP structure designated by the operator matches the picture type included in the history information (user data user_data). Judging. Then, in accordance with the determination result, the “normal encoding process” or “parameter reuse encoding process” described above is executed. Through this process, the encoding device 106 outputs a fourth generation encoded stream ST converted from the Short GOP to the Long GOP. The first generation to fourth generation encoding parameters are recorded as history information in the user data user_data of the encoded stream ST.
[0630]
In the video tape recorder 601 shown in FIGS. 78 and 79, the history information is recorded in the user_data of the picture layer. However, the history information can also be recorded in an area different from the video data of the magnetic tape 604. It is. 80 and 81 show a configuration example of the video tape recorder 601 in this case. FIG. 80 shows a configuration example of the recording system of the video tape recorder 601, and FIG. 81 shows a configuration example of the playback system.
[0631]
As shown in FIG. 80, in this video tape recorder 601, user data user_data output from the decoding device 102 of the transcoder 101R is input to the history decoding device 104, where the past encoding parameters (in this example) In this case, first generation and second generation encoding parameters) are decoded and supplied to the encoding device 106. In this example, since it is not necessary to record history information as user data user_data on the magnetic tape 604, only the history VLC 211 is employed in the history encoding device 107 shown in FIG. Then, in this history VLC 211, the encoding parameter output by the decoding device 102 (in this case, the third generation encoding parameter) and the encoding parameter decoded and output from the user data user_data by the history decoding device 104 (this In the case of the example, first generation and second generation encoding parameters) are supplied. The history VLC 211 performs variable length coding on the first generation to third generation encoding parameters, generates history_stream shown in FIG. 40 to FIG. 46 or FIG. 47, and supplies it to the multiplexer 621.
[0632]
The multiplexer 621 also receives the fourth generation encoded stream ST output from the encoding device 106. The multiplexer 621 multiplexes the encoded stream (bit stream) supplied from the encoding device 106 into a safer area than the history supplied from the history VLC 211.
[0633]
For example, as shown in FIG. 82, in the magnetic tape 604, the video stream output from the encoding device 106 is recorded at a position close to the sync code, and the history_stream output from the history VLC 211 is obtained from the sync code from the video stream. , Recorded at a more distant position. When searching for a video stream during special playback or the like, a sync code is first detected, and the subsequent video stream is searched based on the sync code. Accordingly, when the video stream is arranged at a position close to the sync code, the video data can be more reliably reproduced even during high-speed reproduction. history_stream is not required information during high-speed playback. Therefore, even if this history_stream is arranged at a position farther from the sync code, there is no problem.
[0634]
The signal multiplexed by the multiplexer 621 is input to the channel encoding device 602, channel-encoded, and then recorded on the magnetic tape 604 by the recording head 603.
[0635]
In this way, in this example, history_stream is multiplexed at a position different from that of video data, so even if a start code appears there, it can be sufficiently distinguished from video data. Therefore, in this example, it is not necessary to insert a marker bit and set histroy_stream to converted_history_stream.
[0636]
In addition, there is data that can be supplied to the multiplexer 621 as it is without being encoded in the history_stream format and multiplexed, but if so, the data amount of the encoding parameter increases because it is not compressed. In addition, the utilization efficiency of the magnetic tape 604 decreases. Therefore, it is preferable that the data is compressed by the history VLC 211 and multiplexed in the format of history_stream.
[0637]
As shown in FIG. 81, in the reproduction system of the video tape recorder 601, a signal reproduced by the reproducing head 611 from the magnetic tape 604 is channel-decoded by the channel decoding device 612. The demultiplexer 631 is channel-decoded by the channel decoding device 612. The demultiplexer 631 separates the fourth generation encoded stream ST supplied from the channel decoding device 612 into a video stream and history_stream, supplies the video stream to the decoding device 102, and supplies history_stream to the history VLD 203. To do.
[0638]
That is, in this example, only the history VLD 203 is employed in the history decoding apparatus 104 shown in FIG.
[0639]
The history VLD 203 performs variable-length decoding processing on history_stream, and outputs the obtained first to third generation encoding parameters to the encoding device 106.
[0640]
The history_stream output from the demultiplexer 631 is input to the converter 212 ′. The converter 212 ′ and the user data formatter 213 ′ at the subsequent stage are separate from the converter 212 and the user data formatter 213 (see FIG. 15) built in the history encoding apparatus 107, but are the same as those. It fulfills the functions of
[0641]
That is, the converter 212 'adds a marker bit to the history_stream input from the demultiplexer 631, generates a converted_history_stream, and outputs it to the user data formatter 213'. The user data formatter 213 ′ converts the input converted_history_stream into user_data and outputs it to the encoding device 106. This user_data includes the first generation to third generation encoding parameters.
[0642]
The decoding device 102 decodes the video stream input from the demultiplexer 631 and outputs a baseband video signal to the encoding device 106. Also, the decoding apparatus 102 supplies the fourth generation encoding parameter to the encoding apparatus 106 and outputs it to the history circle coding apparatus 107. The history encoding device 107 generates user_data from the input fourth-generation encoding parameter and outputs it to the encoding device 106.
[0643]
Encoding apparatus 106 performs “normal encoding processing” or “parameter reuse encoding processing” in the same manner as encoding apparatus 106 in FIG. 79, and outputs a fifth generation encoded stream ST. In the fifth generation encoded stream ST, the first generation to fourth generation encoding parameters are recorded in the user_data of the picture layer.
[0644]
According to the transcoder of the present invention, the encoding parameter generated in the past encoding process is described in the user data area of the encoded stream generated in the current encoding process, and the generated bit stream Since the encoded stream conforms to the MPEG standard, any existing decoder can perform the decoding process. Furthermore, according to the transcoder of the present invention, since it is not necessary to provide a dedicated line for transmitting the encoding parameters in the past encoding process, the conventional data stream transmission environment is used as it is. Past coding parameters can be transmitted.
[0645]
According to the transcoder of the present invention, the encoding parameter generated in the past encoding process is selectively described in the encoded stream generated in the current encoding process. It is possible to transmit past coding parameters without extremely increasing the bit rate of the bit stream.
[0646]
According to the transcoder of the present invention, since the most suitable encoding parameter for the current encoding process is selected from the past encoding parameter and the current encoding parameter, the encoding process is performed. Even if the decoding process and the encoding process are repeated, the image quality deterioration is not accumulated.
[0647]
According to the transcoder of the present invention, since the most suitable encoding parameter for the current encoding process is selected from the past encoding parameters according to the picture type, the encoding process is performed. Even if the process and the encoding process are repeated, the image quality deterioration is not accumulated.
[0648]
According to the transcoder of the present invention, since it is determined whether or not to reuse the past coding parameter based on the picture type included in the past coding parameter, the optimum coding process is performed. Can do.
[0649]
The computer program for performing each of the above processes is provided by being recorded on a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. It can be provided by recording on a medium.
[0650]
【The invention's effect】
  According to the first aspect of the present invention, an encoded stream can be converted.Re-encodingIdentification information if it matches the picture typeShown inHistory coding parameters are selectedIf the picture type included in the history encoding parameter does not match the re-encoded picture type in the conversion process, the calculated current encoding parameter is selected as the use encoding parameter used in the conversion process.Therefore, it is possible to suppress deterioration in image quality.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining the principle of high-efficiency encoding.
FIG. 2 is a diagram for explaining a picture type when image data is compressed.
FIG. 3 is a diagram illustrating picture types when image data is compressed.
FIG. 4 is a diagram illustrating the principle of encoding a moving image signal.
FIG. 5 is a block diagram illustrating a configuration of an apparatus that encodes and decodes a moving image signal.
FIG. 6 is a diagram illustrating a configuration of image data.
7 is a block diagram showing a configuration of the encoder 18 of FIG. 5. FIG.
8 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
9 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
10 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
11 is a diagram for explaining the operation of a prediction mode switching circuit 52 in FIG. 7;
12 is a block diagram showing a configuration of the decoder 31 in FIG. 5. FIG.
FIG. 13 is a diagram illustrating SNR control corresponding to a picture type.
FIG. 14 is a block diagram showing a configuration of a transcoder 101 to which the present invention is applied.
15 is a block diagram showing a more detailed configuration of the transcoder 101 of FIG.
16 is a block diagram showing a configuration of a decoder 111 built in the decoding device 102 of FIG.
FIG. 17 is a diagram illustrating macroblock pixels.
FIG. 18 is a diagram for explaining an area in which an encoding parameter is recorded.
FIG. 19 is a block diagram showing a configuration of an encoder 121 built in the encoding device 106 of FIG.
20 is a block diagram illustrating a configuration example of a history VLC 211 in FIG.
FIG. 21 is a block diagram illustrating a configuration example of a history VLD 203 in FIG.
22 is a block diagram illustrating a configuration example of a converter 212 in FIG.
23 is a block diagram illustrating a configuration example of a stuff circuit 323 in FIG.
24 is a timing chart illustrating the operation of converter 212 in FIG.
25 is a block diagram illustrating a configuration example of a converter 202 in FIG.
26 is a block diagram showing a configuration example of a discrete circuit 343 in FIG. 25. FIG.
27 is a block diagram showing another configuration example of the converter 212 in FIG.
28 is a block diagram showing another configuration example of the converter 202 in FIG.
29 is a block diagram illustrating a configuration example of a user data formatter 213 in FIG.
30 is a diagram showing a state in which the transcoder 101 in FIG. 14 is actually used.
FIG. 31 is a diagram for explaining an area in which an encoding parameter is recorded.
Fig. 32 is a flowchart for describing changeable picture type determination processing of the encoding device 106 of Fig. 14;
FIG. 33 is a diagram illustrating an example in which a picture type is changed.
FIG. 34 is a diagram illustrating another example in which the picture type is changed.
FIG. 35 is a diagram for describing quantization control processing of the encoding device 106 in FIG. 14;
36 is a flowchart for describing quantization control processing of the encoding device 106 in FIG. 14; FIG.
FIG. 37 is a block diagram showing a configuration of the transcoder 101 that is tightly coupled.
Fig. 38 is a diagram for describing the syntax of a stream of a video sequence.
FIG. 39 is a diagram for explaining the configuration of the syntax of FIG. 38;
Fig. 40 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 41 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 42 is a diagram illustrating the syntax of history_stream () for recording history information of fixed length.
FIG. 43 is a diagram illustrating the syntax of history_stream () for recording fixed-length history information.
Fig. 44 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
Fig. 45 is a diagram for describing the syntax of history_stream () for recording history information of fixed length.
FIG. 46 is a diagram for describing the syntax of history_stream () for recording fixed-length history information.
Fig. 47 is a diagram for describing the syntax of history_stream () for recording variable-length history information.
Fig. 48 is a diagram for describing the syntax of sequence_header ().
Fig. 49 is a diagram for describing the syntax of sequence_extension ().
Fig. 50 is a diagram for describing the syntax of extension_and_user_data ().
Fig. 51 is a diagram illustrating the syntax of user_data ().
Fig. 52 is a diagram for describing the syntax of group_of_pictures_header ().
Fig. 53 is a diagram for describing the syntax of picture_header ().
Fig. 54 is a diagram for describing the syntax of picture_coding_extension ().
[Fig. 55] Fig. 55 is a diagram for describing the syntax of extension_data ().
Fig. 56 is a diagram for describing the syntax of quant_matrix_extension ().
Fig. 57 is a diagram for describing the syntax of copyright_extension ().
Fig. 58 is a diagram for describing the syntax of picture_display_extension ().
Fig. 59 is a diagram for describing the syntax of picture_data ().
Fig. 60 is a diagram for describing the syntax of slice ().
Fig. 61 is a diagram for describing the syntax of macroblock ();
Fig. 62 is a diagram for describing the syntax of macroblock_modes ().
Fig. 63 is a diagram for describing the syntax of motion_vectors (s).
Fig. 64 is a diagram for describing the syntax of motion_vector (r, s).
Fig. 65 is a diagram for describing a variable length code of macroblock_type for an I picture.
Fig. 66 is a diagram for describing a variable length code of macroblock_type for a P picture.
[Fig. 67] Fig. 67 is a diagram for describing a variable length code of macroblock_type for a B picture.
FIG. 68 is a block diagram showing another configuration of the transcoder 101 to which the present invention is applied.
FIG. 69 is a diagram illustrating a combination of items of history information.
70 is a flowchart for explaining the operation of the transcoder 101 of FIG. 68. FIG.
Fig. 71 is a diagram for describing the syntax of re_coding_stream_info ().
FIG. 72 is a diagram illustrating red_bw_flag and red_bw_indicator.
FIG. 73 is a diagram illustrating Re_Coding Information Bus macroblock formation.
Fig. 74 is a diagram for explaining Picture rate elements.
Fig. 75 is a diagram for explaining Picture rate elements.
FIG. 76 is a diagram for explaining Picture rate elements.
Fig. 77 is a diagram for describing an area in which Re_Coding Information Bus is recorded.
78 is a block diagram illustrating a configuration example of a recording system of a video tape recorder. FIG.
Fig. 79 is a block diagram illustrating a configuration example of a playback system of a video tape recorder.
Fig. 80 is a block diagram illustrating another configuration example of the recording system of the video tape recorder.
Fig. 81 is a block diagram illustrating another configuration example of the playback system of the video tape recorder.
Fig. 82 is a diagram for describing a recording position of a video stream and history_stream.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Encoding apparatus, 2 Decoding apparatus, 3 Recording medium, 12, 13 A / D converter, 14 Frame memory, 15 Luminance signal frame memory, 16 Color difference signal frame memory, 17 Format conversion circuit, 18 Encoder, 31 Decoder, 32 Format conversion circuit, 33 frame memory, 34 luminance signal frame memory, 35 color difference signal frame memory, 36, 37 D / A converter, 50 motion vector detection circuit, 51 frame memory, 52 prediction mode switching circuit, 53 computing unit, 54 Prediction determination circuit, 55 DCT mode switching circuit, 56 DCT circuit, 57 quantization circuit, 58 variable length coding circuit, 59 transmission buffer, 60 inverse quantization circuit, 61 IDCT circuit, 62 arithmetic unit, 63 frame memory, 64 motion Compensation times , 81 reception buffer, 82 variable length decoding circuit, 83 inverse quantization circuit, 84 IDCT circuit, 85 arithmetic unit, 86 frame memory, 87 motion compensation circuit, 101 transcoder, 102 decoding device, 103 history information multiplexing device, 105 History information separation device, 106 encoding device, 111 decoder, 112 variable length decoding circuit, 121 encoder,

Claims (15)

In an encoded stream conversion apparatus that converts an encoded stream into a re-encoded stream,
Selection of history coding parameters used in past coding processing or decoding processing for the coded stream, and history coding parameters usable in conversion processing for converting the coded stream into the re-coded stream Input means for inputting identification information for identifying a specific combination together with the encoded stream;
Encoding parameter calculation means for calculating a current encoding parameter that can be used in a conversion process for converting the encoded stream into the re-encoded stream;
If the picture type included in the history encoding parameter matches a picture type of the re-encoding definitive conversion processing, the history encoding parameters the picture type is indicated in the matching the identification information of the history encoding parameter Is selected as a use coding parameter to be used in the conversion process, and when the picture type included in the history coding parameter does not match the re-encoded picture type in the conversion process, the calculation is performed by the coding parameter calculation unit. Selection means for selecting the current encoding parameter as the use encoding parameter to be used for the conversion process ;
An encoded stream conversion device comprising: conversion means for converting the encoded stream into the re-encoded stream using the use encoding parameter selected by the selection means. The history encoding parameter, the encoded stream converting apparatus according to claim 1 including a motion vector. The encoded stream conversion apparatus according to claim 1, further comprising an output unit that outputs the re-encoded stream converted by the conversion unit. The identification information is information that is distinguished according to the degree of image quality degradation when the conversion process of the conversion unit is executed using the history coding parameter indicated in the identification information. Encoded stream conversion apparatus. The encoded stream conversion apparatus according to claim 1, wherein the identification information is information that is distinguished according to a transmission path through which the encoded stream is transmitted or a capacity of a recording medium that records the encoded stream. . The output means which outputs either the use coding parameter selected by the selection means or the history coding parameter together with the re-encoded stream converted by the conversion means. Encoded stream conversion apparatus. The conversion unit describes either the use coding parameter or the history coding parameter selected by the selection unit in the re-encoded stream.
The encoded stream conversion apparatus according to claim 6 . In the identification information, all the parameters that can be used for the conversion process for converting the encoded stream into the re-encoded stream are input as the history encoding parameter, or the encoding is used as the history encoding parameter . First identification information for identifying whether or not a part of parameters that can be used for the conversion process for converting a stream into the re-encoded stream is input, and a part of the parameters that can be used for the conversion process The encoded stream conversion apparatus according to claim 1, further comprising: second identification information that identifies a combination of available parameters included in the history encoding parameter when input. The converting means performs a conversion process in an MPEG system having a sequence layer, a GOP layer, a picture layer, a slice layer, and a macroblock layer, and the first identification information and the second identification information are red_bw_flag and red_bw_indicator wherein it is described in re-encoding stream in a descriptor represented
The encoded stream conversion apparatus according to claim 8 . The encoded stream conversion apparatus according to claim 1, wherein the conversion unit performs conversion processing so as to change a bit rate or a GOP structure. The encoded stream conversion apparatus according to claim 1, wherein the conversion means performs conversion processing by an MPEG method having a sequence layer, a GOP layer, a picture layer, a slice layer, and a macroblock layer. The encoded stream conversion apparatus according to claim 1, wherein the history encoding parameter is an encoding parameter used in encoding processing or decoding processing of a plurality of past generations with respect to the encoded stream. The history encoding parameter is an encoding parameter used in encoding processing or decoding processing for the last four generations with respect to the encoded stream.
The encoded stream conversion apparatus according to claim 12 . In an encoded stream conversion method of an encoded stream conversion apparatus for converting an encoded stream into a re-encoded stream,
Selection of history coding parameters used in past coding processing or decoding processing for the coded stream, and history coding parameters usable in conversion processing for converting the coded stream into the re-coded stream Identification information for identifying a specific combination, and an input step for receiving the input of the encoded stream;
An encoding parameter calculation step for calculating a current encoding parameter that can be used in a conversion process for converting the encoded stream into the re-encoded stream;
If the picture type included in the history encoding parameter matches a picture type of the re-encoding definitive conversion processing, the history encoding parameters the picture type is indicated in the matching the identification information of the history encoding parameter Is selected as a use coding parameter to be used in the conversion process, and when the picture type included in the history coding parameter does not match the re-encoded picture type in the conversion process, the calculation is performed by the process of the coding parameter calculation step. A selection step of selecting the current encoding parameter that has been used as the use encoding parameter to be used in the conversion process ;
An encoded stream conversion method comprising: a conversion step of converting the encoded stream into the re-encoded stream using the use encoding parameter selected by the selection step. A computer-executable program that executes processing for converting an encoded stream into a re-encoded stream,
Selection of history coding parameters used in past coding processing or decoding processing for the coded stream, and history coding parameters usable in conversion processing for converting the coded stream into the re-coded stream Identification information for identifying a specific combination, and an input step for receiving the input of the encoded stream;
An encoding parameter calculation step for calculating a current encoding parameter that can be used in a conversion process for converting the encoded stream into the re-encoded stream;
If the picture type included in the history encoding parameter matches a picture type of the re-encoding definitive conversion processing, the history encoding parameters the picture type is indicated in the matching the identification information of the history encoding parameter Is selected as a use coding parameter to be used in the conversion process, and when the picture type included in the history coding parameter does not match the re-encoded picture type in the conversion process, the calculation is performed by the process of the coding parameter calculation step. A selection step of selecting the current encoding parameter that has been used as the use encoding parameter to be used in the conversion process ;
A recording in which a program for causing a computer to execute a process including a conversion step of converting the encoded stream into the re-encoded stream using the use encoding parameter selected in the selection step is recorded. Medium.

Images