JP2007259492A - Information processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To transmit data so as to easily extract significant information by changing a method of superimposing data in accordance with a degree of significance of information. <P>SOLUTION: To a video data signal outputted from a history information multiplexer of a video decoding system, an encoding parameter, above a picture layer, with a high degree of necessity for a number of applications is inserted into a V Blanking region and an encoding parameter, below a slice layer, is inserted into an H Blanking region after being converted into an ancillary packet. In a video encoding system, the ancillary packet superimposed on H Blanking and the ancillary packet superimposed on V Blanking are extracted from an inputted baseband video signal. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and method, and in particular, transcoding that does not cause image quality degradation even when decoding and encoding processing are repeated to change the structure of an encoded bitstream that is encoded based on the MPEG standard. The present invention relates to an information processing apparatus and method capable of transmitting and reading out necessary information easily according to the importance of information in a system.

In recent years, MPEG (Moving Picture Experts Group) technology has been generally used in broadcasting stations that produce and broadcast television programs in order to compress / encode video data. In particular, this MPEG technology is becoming the de facto standard when recording video data on a randomly accessible recording medium material such as a tape and when transmitting video data via a cable or satellite.

  An example of processing in the broadcasting station until the video program produced in the broadcasting station is transmitted to each home will be briefly described. First, source video data is encoded and recorded on a magnetic tape by an encoder provided in a camcorder in which a video camera and a VTR (Video Tape Recorder) are integrated. At this time, the encoder of the camcorder encodes the source video data so as to be suitable for the recording format of the VTR tape. For example, the GOP (Group of Pictures) structure of an MPEG bit stream recorded on the magnetic tape is a structure in which one frame is composed of two frames (for example, I, B, I, B, I, B,...). ...). The bit rate of the MPEG bit stream recorded on the magnetic tape is 18 Mbps.

  Next, in the main broadcasting station, editing processing for editing the video bit stream recorded on the magnetic tape is performed. For this purpose, the GOP structure of the video bit stream recorded on the magnetic tape is converted into a GOP structure suitable for editing processing. A GOP structure suitable for editing processing is a GOP structure in which one GOP is composed of one frame and all pictures are I pictures. This is because an I picture having no correlation with other pictures is most suitable for editing in frame units. In actual operation, the video stream recorded on the magnetic tape is once decoded and returned to the baseband video data. Then, the baseband video signal is re-encoded so that all pictures become I pictures. By performing the decoding process and the re-encoding process in this way, it is possible to generate a bitstream having a GOP structure suitable for editing processing.

  Next, in order to transmit the edited video program generated by the above-described editing process from the main station to the local station, the bit stream of the edited video program is converted into a GOP structure and a bit rate suitable for the transmission process. A GOP structure suitable for transmission between broadcast stations is, for example, a GOP structure in which 1 GOP is composed of 15 frames (for example, I, B, B, P, B, B, P,...). The bit rate suitable for transmission between broadcasting stations is generally a high bit rate of 50 Mbps or more because a dedicated line having a high transmission capacity such as an optical fiber is provided between broadcasting stations. Is desirable. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for transmission between the broadcasting stations described above.

  In the local station, editing processing is performed in order to insert a commercial unique to the local area in the video program transmitted from the main station. That is, as in the editing process described above, the video stream transmitted from the main station is once decoded and returned to the baseband video data. Then, by re-encoding the baseband video signal so that all the pictures become I pictures, a bitstream having a GOP structure suitable for editing processing can be generated.

  Subsequently, the video program edited in the local station is converted into a GOP structure and a bit rate suitable for the transmission process to be transmitted to each home via a cable or a satellite. For example, a GOP structure suitable for transmission processing for transmission to each home is a GOP structure in which 1 GOP is composed of 15 frames (for example, I, B, B, P, B, B, P,...). The bit rate suitable for transmission processing for transmission to each home is a low bit rate of about 5 Mbps. Specifically, the bit stream of the edited video program is once decoded and returned to baseband video data. Then, the baseband video data is re-encoded so as to have a GOP structure and a bit rate suitable for the transmission processing described above.

  As can be understood from the above description, the decoding process and the encoding process are repeated a plurality of times while the video program is transmitted from the broadcast station to each home. Actually, the processing at the broadcasting station requires various signal processing in addition to the signal processing described above, and the decoding processing and the encoding processing must be repeated each time.

  However, it is well known that encoding processing and decoding processing based on the MPEG standard are not 100% reversible processing. That is, the baseband video data before being encoded and the video data after being decoded are not 100% the same, and the image quality is degraded by this encoding process and decoding process. That is, as described above, there is a problem in that when the decoding process and the encoding process are repeated, the image quality deteriorates every time the process is performed. In other words, image quality deterioration is accumulated every time decoding / encoding processing is repeated.

  In order to solve this problem, there is a technique for preventing deterioration in image quality by transmitting an encoding history together with video data and performing encoding and decoding using the transmission history.

  However, depending on the application that processes the image, the transcoding system may not need all the parameters when executing the decoding process and the encoding process. However, since it is not known in which region the necessary data is located, in the transcoding system, it is necessary to assemble hardware so that all data can be accessed. For this reason, the hardware becomes complicated and the scale of the apparatus increases. Furthermore, if a packet is lost due to the influence of noise or the like, it is difficult to recover the data thereafter, and thus a large amount of data may be lost.

  The present invention has been made in view of such a situation, and even if decoding and encoding processes are repeated to change the GOP structure of an encoded bitstream encoded based on the MPEG standard, the image quality is improved. In a transcoding system where degradation does not occur, change the data superposition method according to the importance of information, etc., and describe it in a predetermined position (for example, vertical blanking area and horizontal blanking area), so that necessary information can be It is transmitted so that it can be easily taken out, and it can be read out.

  An information processing apparatus according to an aspect of the present invention is an information processing apparatus that inserts an encoding parameter into image data obtained by decoding an encoded stream, the encoding parameter of the encoded stream being the encoding In the re-encoding process of the encoded stream, an acquisition unit that is acquired together with the stream, an encoding parameter that is higher than a picture layer and an encoding parameter that is lower than a slice layer included in the encoding parameter acquired by the acquisition unit are selected. In order to be used, an encoding parameter equal to or higher than the picture layer is inserted into a vertical blanking area of the image data and transmitted, or an encoding parameter equal to or lower than the slice layer is transmitted as a horizontal blanking area of the image data. And transmission means for selectively performing transmission after insertion.

  Decoding means for decoding the encoded stream to generate the image data can be further provided.

  The encoding parameter may include a current encoding parameter used when generating the encoded stream.

  The encoding parameter may include a history encoding parameter used in a past encoding process or decoding process for the encoded stream.

  The encoding parameter may be multiplexed in the encoded stream, and the acquisition unit may acquire the encoding parameter from the encoded stream.

  The encoded stream may be encoded according to the MPEG standard.

  The re-encoding process may conform to the MPEG standard.

  An information processing method according to one aspect of the present invention is an information processing method of an information processing apparatus that inserts an encoding parameter into image data obtained by decoding an encoded stream, the encoding parameter of the encoded stream being , Obtained together with the encoded stream, and selectively used in the re-encoding process of the encoded stream, the encoding parameter higher than the picture layer and the encoding parameter lower than the slice layer included in the acquired encoding parameter As described above, an encoding parameter higher than the picture layer is inserted into the vertical blanking region of the image data for transmission, or an encoding parameter lower than the slice layer is inserted into the horizontal blanking region of the image data. Or selectively executing whether to transmit.

  In one aspect of the present invention, an encoding parameter of an encoded stream is acquired together with the encoded stream, and an encoding parameter equal to or higher than a picture layer and an encoding parameter equal to or lower than a slice layer are included in the acquired encoding parameter. In order to be selectively used in the re-encoding process of the encoded stream, an encoding parameter higher than the picture layer is inserted and transmitted in the vertical blanking region of the image data, or an encoding parameter lower than the slice layer is transmitted. It is inserted into the horizontal blanking area of image data and transmitted.

  As described above, according to one aspect of the present invention, it is possible to transmit an encoding parameter, and in particular, it is possible to prevent image degradation by using an encoding parameter transmitted by a subsequent apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a transcoding system of the present invention. The video decoding system 1 receives the input of the bit stream, decodes the input signal by a method to be described later with reference to FIG. 5, and if necessary, performs V blanking area and H blanking area to be described later with reference to FIG. 7. Information is described, converted into a baseband video signal and output. The video encoding system 2 receives the input of the baseband video signal, extracts necessary information from a V Blanking area and an H Blanking area, which will be described later using FIG. 7, and is input by a method described later using FIG. The encoded signal is encoded, converted into a bit stream, and output. The video processing system (VTR) 3 executes editing processing on the baseband video signal as necessary.

  FIG. 2 is a block diagram showing a detailed configuration of the video decoding system 1.

  The decoding device 11 extracts a video signal from the input bit stream, outputs the video signal to the history information multiplexing device 12, and outputs the encoding parameter one generation before (for example, if this transcoding system is the fourth generation, 3 generation encoding parameters) are extracted and output to the history information multiplexing device 12 and the data classification device 13, and the user data indicating the conversion history of the previous data input to the video decoding system (that is, from the previous generation) Further, the previous generation encoding parameter) is input to the history decoding device 14.

  The coding parameters input to the data classification device 13 are separated into, for example, coding parameters higher than the picture layer, which are highly necessary in many applications, and coding parameters below the slice layer that are not necessary for all parameters. Are output to the format converter 15 and the format converter 17, respectively. The format converter 15 and the format converter 17 convert the input encoding parameters into ancillary packets and output them to the V blanking insertion device 16 and the H blanking insertion device 18, respectively.

  The history decoding device 14 includes a user data decoder 21 that decodes user data supplied from the decoding device 11, a converter 22 that converts the output of the user data decoder 21, and a history VLD that reproduces history information from the output of the converter 22 ( Variable Length Decoder) 23.

  The user data decoder 21 decodes the user data supplied from the decoding device 11 and outputs it to the converter 22. Although details will be described later with reference to FIG. 28, user data (user_data ()) is composed of user_data_start_code and user_data. In the MPEG standard, 23 bits of “0” (the same code as start_code) is included in user_data. ) Is prohibited. This is to prevent the data from being erroneously detected as start_code. The history information (history_stream ()) is described in the user data area (as a kind of user_data in the MPEG standard), and there may be such “0” of 23 bits or more in succession. Therefore, it is necessary to insert “1” at a predetermined timing and convert it into converted_history_stream () (FIG. 15 to be described later) so that consecutive “0” s of 23 bits or more do not occur. This conversion is performed by a converter of the history encoding apparatus described later. The converter 22 of the history decoding apparatus 14 performs a conversion process reverse to that of the converter (removes “1” inserted so as not to generate “0” of 23 or more consecutive bits).

  The history VLD 23 generates history information (in this case, a first generation encoding parameter and a second generation encoding parameter) from the output of the converter 22 and outputs the history information to the history information multiplexer 12.

  That is, the history decoding apparatus 14 can detect the unique History_Data_Id described in the user data by analyzing the syntax of the received user data, and thereby extract the converted_history_stream (). . Furthermore, the history decoding device 14 can obtain history_stream () by taking 1 marker bit (marker_bit) inserted at a predetermined interval in the converted_history_stream (), and the history_stream () The first generation and second generation encoding parameters described in history_stream () can be obtained by analyzing the syntax).

  The history information multiplexer 12 records all data change histories before being input to the video decoding system (for example, if this is a fourth generation change, the first generation, second generation and third generation encodings). Parameter) to the video encoding system 2 that performs the fourth generation encoding process, the baseband video data decoded by the decoding device 11 is added to the first generation, second generation, and third generation. This is a circuit for multiplexing the encoding parameters.

  The signal output from the history information multiplexing device 12 is input to the V Blanking insertion device 16. The V Blanking insertion device 16 inserts an encoding parameter equal to or higher than the picture layer input from the format converter 15 into a V Blanking area, which will be described later with reference to FIG. 7, and outputs it to the H Blanking insertion device 18. The H Blanking insertion device 18 inserts an encoding parameter below the slice layer input from the format converter 17 into an H Blanking area, which will be described later with reference to FIG.

  FIG. 3 is a block diagram showing a detailed configuration of the video encoding system 2.

  The H Blanking extraction device 31 extracts the ancillary packet superimposed on the H Blanking area from the input baseband video signal and outputs it to the format converter 32. The format converter 32 converts the format of the input ancillary packet and outputs it to the multiplexer 33. The V Blanking extraction device 34 extracts the ancillary packet superimposed on the V Blanking area from the input baseband video signal, and outputs it to the format converter 35. The format converter 35 converts the format of the input ancillary packet and outputs it to the multiplexer 33. The multiplexer 33 multiplexes the input signals and outputs them to the parameter reproducing unit 36.

  When the packet is missing, the parameter reproducing unit 36 corrects the lost data by a method described later with reference to FIG. 8 and outputs the corrected data to the encoding device 39.

  After the ancillary packet is extracted by the H Blanking extraction device 31 and the H Blanking extraction device 34, the data is input to the history information separation device 37. The history information separation device 37 extracts baseband video data from the transmission data, supplies the video data to the encoding device 39, and the first generation, second generation, and third generation history information from the transmission data. Are extracted and supplied to the encoding device 39 and the history encoding device 38, respectively.

  In the history encoding device 38, a history VLC (Variable Length Coder) 43 records the three generations (first generation, second generation, and third generation) of encoding parameters supplied from the history information separation device 37. Convert to information format. There are a fixed length format (FIGS. 17 to 23 described later) and a variable length format (FIG. 24 described later). Details of these will be described later.

  The history information formatted by the history VLC 43 is converted into converted_history_stream () by the converter 42. As described above, this is a process for preventing the start_code of user_data () from being erroneously detected. That is, although “0” having 23 or more consecutive bits exists in the history information, since “0” having 23 or more consecutive bits cannot be arranged in user_data, do not touch this prohibited item. Data is converted by the converter 42 ("1" is inserted at a predetermined timing).

  The user data formatter 41 adds the History_Data_ID to the converted_history_stream () supplied from the converter 42 based on FIG. 15 to be described later, and further adds the user_data_stream_code to generate user_data of the MPEG standard that can be inserted into the video stream. And output to the encoding device 39.

  The encoding device 39 is a device for encoding the baseband video data supplied from the history information separation device 37 into a bit stream having a GOP structure and a bit rate specified by an operator or a host computer. is there. Note that changing the GOP structure means, for example, the number of pictures included in the GOP, the number of P pictures existing between I pictures and I pictures, and between I pictures and P pictures (or I pictures). This means that the number of B pictures is changed.

  In this case, since the history information of the first generation, the second generation, and the third generation is superimposed on the supplied baseband video data, the encoding device 39 causes image quality degradation due to re-encoding processing. The history information is selectively reused so as to reduce the fourth generation encoding process.

  For example, in the case of handling only data above the picture layer, not all the hardware described with reference to FIGS. 2 and 3 is necessary. That is, a video decoding system in which the format converter 17 and the H Blanking insertion device 18 are deleted in FIG. 2 and the H Blanking extraction device 31 and the format converter 32 are deleted in FIG. 3 may be used.

  Also, when data below the slice layer is handled with the same frequency as data above the picture layer, a video comprising a decoding device 61, a history decoding device 62, and a history information multiplexing device 63, as shown in FIG. You may use the video encoding system 52 which consists of the decoding system 51 and the history information separation apparatus 64, the history encoding apparatus 65, and the encoding apparatus 66. FIG.

  The decoding device 61 extracts a video signal and encoding parameters from the input bit stream and outputs them to the history information multiplexing device 63, and also extracts user data from the input bit stream and outputs it to the history decoding device 62. To do. The history decoding device 62 decodes the user data by the same method as the history decoding device 14 and outputs it to the history information multiplexing device 63. As will be described later with reference to FIG. 7, the history information multiplexing device 63 describes the encoding parameters in the designated area and outputs them to the history information separating device 64.

  The history information separation device 64 extracts baseband video data from the input data, supplies the video data to the encoding device 66, and is described at a predetermined position of the input data. The 1st generation, 2nd generation, and 3rd generation history information is extracted and supplied to the encoding device 66 and the history encoding device 65, respectively. The history encoding device 65 encodes the input history information by the same process as the history encoding device 38 and outputs the encoded history information to the encoding device 66. The encoding device 66 encodes the input data so as to be a bit stream having a predetermined GOP structure and bit rate.

  FIG. 5 is a block diagram showing a detailed configuration of the decoder 81 included in the decoding device 11 described with reference to FIG. 2 (or the decoding device 61 described with reference to FIG. 4).

  The decoder 81 receives a received buffer 91 for buffering the supplied bit stream, a variable length decoding circuit 92 for variable length decoding of the encoded bit stream, and supplies variable length decoded data from the variable length decoding circuit 92. An inverse quantization circuit 93 that performs inverse quantization according to the quantized scale, an IDCT circuit 94 that performs inverse discrete cosine transform on the inversely quantized DCT (discrete cosine transform) coefficient, and an arithmetic unit 95 for performing motion compensation processing; A frame memory 96 and a motion compensation circuit 97 are provided.

  The variable length decoding circuit 92 is described in the picture layer, slice layer, and macroblock layer of this encoded bitstream in order to decode the encoded bitstream of the immediately previous generation (in this case, the third generation). Extract encoding parameters. The extracted encoding parameters are also supplied to the data classification device 13 as described with reference to FIG. 2, and are classified into the picture layer and the slice layer.

  For example, the third generation encoding parameters extracted in the variable length decoding circuit 92 are picture_coding_type indicating the picture type, quantizer_scale_code indicating the quantization scale step size, macroblock_type indicating the prediction mode, motion_vector indicating the motion vector, and frame prediction. Frame / field_motion_type indicating the mode or field prediction mode, and dct_type indicating the frame DCT mode or the field DCT mode. The quatntiser_scale_code extracted by the variable length decoding circuit 92 is supplied to the inverse quantization circuit 93, and parameters such as picture_coding_type, quatntiser_scale_code, macroblock_type, motion_vector, frame / field_motion_type, and dct_type are supplied to the motion compensation circuit 97.

  The variable length decoding circuit 92 transmits not only these encoding parameters necessary for decoding the third generation encoded bitstream but also the third generation history information to the subsequent fifth generation transcoding system. The encoding parameters to be performed are extracted from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third generation encoded bitstream. Of course, third-generation encoding parameters such as picture_coding_type, quatntiser_scale_code, macroblock_type, motion_vector, frame / field_motion_type, and dct_type used for the third-generation decoding process are included in the third-generation history information. The encoding parameters to be extracted as history information are set in advance by the operator or the host computer according to the transmission capacity.

  Further, the variable length decoding circuit 92 extracts user data described in the user data area of the picture layer of the third generation encoded bit stream, and supplies the user data to the history decoding device 14.

  As described above, the history decoding device 14 uses the first generation encoding parameters described in the history information and the second information from the user data described in the picture layer of the third generation encoded bitstream. This is a circuit for extracting the encoding parameter of the generation (the encoding parameter of the generation before the previous generation).

  The variable length decoding circuit 92 performs variable length decoding on the data supplied from the reception buffer 91, outputs the motion vector, the prediction mode, the prediction flag, and the DCT flag to the motion compensation circuit 97, and sets the quantization scale to the inverse quantum. And outputs the decoded image data to the inverse quantization circuit 93.

  The inverse quantization circuit 93 inversely quantizes the image data supplied from the variable length decoding circuit 92 according to the quantization scale supplied from the variable length decoding circuit 92 and outputs the result to the IDCT circuit 94. The data (DCT coefficient) output from the inverse quantization circuit 93 is subjected to inverse discrete cosine transform processing by the IDCT circuit 94 and supplied to the computing unit 95.

  When the image data supplied from the IDCT circuit 94 to the computing unit 95 is I picture data, the data is output from the computing unit 95, and image data (P or B picture data) to be input later to the computing unit 95. ) Is supplied to and stored in the forward predicted image unit 96a of the frame memory 96.

  When the image data supplied from the IDCT circuit 94 is P picture data having the image data of the previous frame as predicted image data and is data in the forward prediction mode, the image data is stored in the forward predicted image unit 96a of the frame memory 96. The stored image data of the previous frame (I picture data) is read out, and motion compensation corresponding to the motion vector output from the variable length decoding circuit 92 is performed by the motion compensation circuit 97. The arithmetic unit 95 adds the image data (difference data) supplied from the IDCT circuit 94 and outputs the result. The added data, that is, the decoded P picture data is stored in the rear of the frame memory 96 in order to generate predicted picture data of picture data (B picture or P picture data) to be input later to the arithmetic unit 95. The predicted image unit 96b is supplied and stored.

  In this way, even in the case of P picture data, the intra prediction mode data is not processed in the computing unit 95 and stored in the backward predicted image unit 96b as it is in the I picture data.

  When the image data supplied from the IDCT circuit 94 is B-picture data, I corresponding to the prediction mode supplied from the variable length decoding circuit 92 is stored in the forward predicted image unit 96a of the frame memory 96. Image data of a picture (in the case of forward prediction mode), image data of a P picture stored in the backward prediction image unit 96b (in the case of backward prediction mode), or both of them (in the case of bidirectional prediction mode) The motion compensation circuit 97 reads out and performs motion compensation corresponding to the motion vector output from the variable length decoding circuit 92, thereby generating a predicted image. However, when motion compensation is not required (in the case of intra-picture prediction mode), a predicted image is not generated.

  In this way, the data subjected to motion compensation in the motion compensation circuit 97 is added to the output of the IDCT circuit 94 in the computing unit 95 and output. However, since this addition output is B picture data and is not used for generating a predicted image of another image, it is not stored in the frame memory 96. Further, in the motion compensation circuit 97, processing for returning the configuration in which the signals of the odd-numbered field and even-numbered line lines are separated to the original configuration is performed as necessary.

  After the B picture image is output, the P picture image data stored in the backward predicted image unit 96 b is read and supplied to the computing unit 95 via the motion compensation circuit 97. However, motion compensation is not performed at this time.

  The output from the decoder 81 is input to the history information multiplexer 12. Specifically, the history information multiplexing device 12 includes baseband video data output from the arithmetic unit 95 of the decoder 81, a third generation encoding parameter output from the variable length decoding circuit 92 of the decoder 81, and The first generation encoding parameter and the second generation encoding parameter output from the history decoding device 14 are received, and the first generation, second generation, and third generation are included in the baseband video data. Then, as will be described later with reference to FIG. 7, the encoding parameters are described in the H Blanking area and the V Blanking area, and output data is generated.

  Next, a method of multiplexing these first generation, second generation, and third generation encoding parameters into baseband video data will be described with reference to FIGS.

  FIG. 6 shows one macro block of 16 pixels × 16 pixels defined in the MPEG standard. This macro block of 16 pixels × 16 pixels has four sub-blocks (Y [0], [1], [2] and Y [3]) consisting of four 8 pixels × 8 pixels with respect to the luminance signal, and with respect to the color difference signal. Consists of four sub-blocks (Cr [0], r [1], b [0], and Cb [1]) each consisting of 8 pixels × 8 pixels.

  Then, as shown in FIG. 7, for example, encoding parameters higher than the picture layer, which are highly necessary in many applications, are described in the V Blanking area, and are not required in all applications. The parameters are inserted by the above-described V Blanking insertion device 16 and H Blanking insertion device 18 so as to be described in the H Blanking area.

  Further, in the correction of the missing packet in the parameter reproducing unit described above, for example, there is a method of storing previous data and supplementing data corresponding to the location of the missing packet. Here, as shown in FIG. 8A, when start_code is written at the start of each data regardless of packet delimitation, all data must be read to find start_code. Therefore, in the information above the picture layer included in the V Blanking area, as shown in FIG. 8B, the information of 1 macroblock is closed within an ancillary packet of 1 stripe, and the start_code is always a packet. By writing at the head of the packet, it is easy to correct the missing packet.

  Further, in the macro block described with reference to FIG. 6, the information in band units that is decomposed in the vertical direction is called a stripe. In MPEG2, when processing an interlaced image, the field structure and the frame structure shown in FIG. 9 are appropriately switched and used. In the field structure, an image is processed in units of fields, whereas in a frame structure, a frame is composed of two fields, and an image is processed in units of frames. Therefore, in the field structure, a stripe is formed every 16 lines, whereas in the frame structure, a stripe is formed every 8 lines.

  The encoding parameters below the slice layer forming each stripe are transmitted in a closed form in the H blanking area of the stripe. Thus, since the video data of each stripe can be obtained and the encoding parameter can be obtained at the same time, it is not necessary to have a mechanism such as a buffer for adjusting the time. When start_code exists in the payload of the packet, start_code is always written at the head of the packet as shown in FIG. 10 in order to facilitate recovery when the packet is lost.

  As shown in FIG. 11, the data block includes sub-blocks (Y [0], Y [1], Y [2], Y [3], Cr [0], Cr [1], Cb [0]). , Cb [1]) is a data block for transmitting one pixel, 64 data blocks shown in FIG. 11 are transmitted to transmit one macroblock data. If the lower 2 bits (D1 and D0) are used, a total of 1024 (= 16 × 64) bits of history information can be transmitted for one macroblock of video data. Accordingly, since history information for one generation is generated to be 256 bits, history information for the past 4 (= 1024/256) generations may be superimposed on video data of one macroblock. it can. In the example shown in FIG. 11, the first generation history information, the second generation history information, and the third generation history information are superimposed.

  FIG. 12 is a diagram illustrating a specific configuration of the encoder 101 provided in the encoding device 39 described with reference to FIG. 3 (or the encoding device 66 described with reference to FIG. 4). The encoder 101 includes a controller 125, a motion vector detection circuit 111, a frame memory 112, a frame / field prediction mode switching circuit 113, a calculator 114, a DCT mode switching circuit 115, a DCT circuit 116, a quantization circuit 117, and a variable length coding. A circuit 118, a transmission buffer 119, an inverse quantization circuit 120, an inverse DCT (IDCT) circuit 121, an arithmetic unit 122, a frame memory 123, and a motion compensation circuit 124 are provided.

  The encoder 101 has a controller 125 for controlling the operation and function of each circuit described above. The controller 125 receives an instruction regarding the GOP structure from an operator or a host computer, and determines the picture type of each picture so as to correspond to the GOP structure. Further, the controller 125 receives target bit rate information from the operator or the host computer, and makes the quantum rate so that the bit rate output from the encoder 101 becomes the target bit rate specified by the operator or the host computer. The control circuit 117 is controlled.

  Further, the controller 125 receives a plurality of generations of history information output from the history information separation device 37, and performs a reference picture encoding process by reusing these history information. This will be described in detail below.

  First, the controller 125 determines whether or not the picture type of the reference picture determined from the GOP structure designated by the operator matches the picture type included in the history information. That is, it is determined whether or not this reference picture has been encoded in the past with the same picture type as the designated picture type. That is, the controller 125 determines that the picture type assigned to the reference picture as the fourth generation encoding process is the picture type of the reference picture in the first generation encoding process and the picture type assigned to the reference picture in the second generation encoding process. It is determined whether the picture type of the reference picture matches the picture type of the reference picture in the third generation encoding process.

  If the picture type specified for the reference picture as the fourth generation encoding process does not match any picture type in the past encoding process, the controller 125 performs the “normal encoding process”. . That is, in this case, in any of the first generation, second generation, or third generation encoding processing, this reference picture is encoded using the picture type assigned as the fourth generation encoding processing. It has never been done. On the other hand, if the picture type specified in the reference picture as the fourth generation encoding process matches any picture type in the past encoding process, the controller 125 will execute “parameter reuse”. Encoding process "is performed. In other words, in this case, the reference picture is encoded with the picture type assigned as the fourth generation encoding process in the first generation, second generation, or third generation encoding process. It means that it has been processed.

  First, the normal encoding process of the controller 125 will be described.

  The encoded image data is input to the motion vector detection circuit 111 in units of macroblocks. The motion vector detection circuit 111 processes the image data of each frame as an I picture, P picture, or B picture according to a predetermined sequence set in advance. It is determined in advance whether the image of each frame that is sequentially input is processed as an I, P, or B picture.

  Image data of a frame processed as an I picture is transferred and stored from the motion vector detection circuit 111 to the front original image unit 112a of the frame memory 112, and image data of a frame processed as a B picture is stored in the original image unit 112b. The image data of the frame transferred and stored and processed as the P picture is transferred and stored in the rear original image unit 112c.

  At the next timing, when an image of a frame to be further processed as a B picture or a P picture is input, the image data of the first P picture that has been stored in the rear original image unit 112c until then is the front original image. The image data of the next B picture is stored (overwritten) in the reference original image unit 112b, and the image data of the next P picture is stored (overwritten) in the rear original image unit 112c. Such an operation is sequentially repeated.

  The signal of each picture stored in the frame memory 112 is read therefrom, and the prediction mode switching circuit 113 performs frame prediction mode processing or field prediction mode processing.

  The motion vector detection circuit 111 detects a prediction error in the frame prediction mode and a prediction error in the field prediction mode in order to determine whether the frame prediction mode or the field prediction mode should be selected, and the value of the prediction error Is supplied to the controller 125.

  That is, the motion vector detection circuit 111 calculates the absolute value sum of the prediction errors of the intra-picture prediction as the absolute value sum of the macroblock signals Aij of the reference image | ΣAij | and the absolute value | Aij of the macroblock signal Aij. Find the difference of the sum Σ | Aij | Further, as the absolute value sum of the prediction errors of the forward prediction, the sum Σ | Aij− of the absolute value | Aij−Bij | of the difference Aij−Bij between the macroblock signal Aij of the reference image and the macroblock signal Bij of the predicted image Bij | is obtained. Also, the absolute value sum of the prediction errors of the backward prediction and the bidirectional prediction is obtained in the same manner as in the forward prediction (by changing the prediction image to a prediction image different from that in the forward prediction).

  These absolute value sums are supplied to the controller 125. Further, the controller 125 compares the absolute value sum of the prediction errors of the prediction and the absolute value sum of the prediction errors of the intra prediction, selects the smaller one, and selects a mode corresponding to the selected absolute value sum. Select as prediction mode. That is, if the sum of the absolute values of the prediction errors of intra prediction is smaller, the intra prediction mode is set. If the absolute value sum of the prediction errors of the prediction is smaller, the mode in which the corresponding absolute value sum of the forward prediction, backward prediction, or bidirectional prediction mode is the smallest is set.

  The prediction mode switching circuit 113 performs signal processing so as to correspond to the prediction mode selected by the controller 125 and supplies it to the computing unit 114. Specifically, when the frame prediction mode is selected, the prediction mode switching circuit 113 performs signal processing so that the luminance signal is output to the calculator 114 in the input state, and the color difference signal is output. Performs signal processing so that odd field lines and even field lines are mixed. On the other hand, when the field prediction mode is selected, of the four luminance blocks, two luminance blocks are composed of, for example, only the dots of the odd field lines, and the other two luminance blocks are the even field. Are composed of only the dots in the line and output to the computing unit 114. With respect to the color difference signal, signal processing is performed so that the upper four lines are constituted by odd field lines and the lower four lines are constituted by even field lines.

  Further, the motion vector detection circuit 111 determines the prediction error in each prediction mode in order to determine which prediction mode is selected from the intra-picture prediction mode, the forward prediction mode, the backward prediction mode, or the bidirectional prediction mode. The prediction error in each prediction mode is supplied to the controller 125. The controller 125 selects the smallest prediction error of forward prediction, backward prediction, and bidirectional prediction as the prediction error of inter prediction. Further, the prediction error of the inter prediction and the prediction error of the intra-picture prediction are compared, the smaller one is selected, and the mode corresponding to the selected prediction error is selected as the prediction mode. That is, if the prediction error of intra prediction is smaller, the intra prediction mode is set. If the prediction error of the prediction is smaller, the mode in which the corresponding prediction error is the smallest of the forward prediction, backward prediction or bidirectional prediction mode is set. The controller 125 controls the calculator 114 and the motion compensation circuit 124 so as to correspond to the selected prediction mode. Thereby, the image data of the I picture is input to the DCT mode switching circuit.

  The DCT mode switching circuit 115 selects the frame DCT mode or the field DCT mode by using the signal format (frame DCT mode) in which the data of the four luminance blocks are mixed in the odd and even field lines. ) And a signal form (field DCT mode) in which the odd and even field lines are separated, and each signal is supplied to the DCT circuit 116. The DCT circuit 116 calculates the coding efficiency when the odd-numbered field and the even-numbered field are mixed and DCT processing is performed, and the coding efficiency when the odd-numbered field and the even-numbered field are separated and the DCT processing is performed. 125. The controller 125 compares the coding efficiencies supplied from the DCT circuit 116, selects the DCT mode with the better coding efficiency, and controls the DCT mode switching circuit 115 so as to be in the selected DCT mode. .

  The DCT mode switching circuit 115 outputs data having a configuration corresponding to the selected DCT mode to the DCT circuit 116, and outputs a DCT flag indicating the selected DCT mode to the variable length encoding circuit 118 and the motion compensation circuit 124. Output to.

  As is apparent from a comparison between the prediction mode in the prediction mode switching circuit 113 (that is, the field prediction mode or the frame prediction mode) and the DCT mode in the DCT mode switching circuit 115 (that is, the field DCT mode or the frame DCT mode), Regarding the luminance block, the data structure in each mode is substantially the same.

  When the frame prediction mode (a mode in which odd lines and even lines are mixed) is selected in the prediction mode switching circuit 113, the frame DCT mode (a mode in which odd lines and even lines are mixed) also in the DCT mode switching circuit 115. When the prediction mode switching circuit 113 selects the field prediction mode (the mode in which the data of the odd field and the even field are separated), the DCT mode switching circuit 115 selects the field DCT mode ( There is a high possibility that the mode in which the data of the odd field and the even field are separated is selected.

  However, the mode is not always selected in this manner. In the prediction mode switching circuit 113, the mode is determined so that the absolute value sum of the prediction errors becomes small. In the DCT mode switching circuit 115, the encoding is performed. The mode is determined so that the efficiency is good.

  The image data of the I picture output from the DCT mode switching circuit 115 is input to the DCT circuit 116, subjected to DCT processing, and converted into DCT coefficients. The DCT coefficient is input to the quantization circuit 117, quantized with a quantization scale corresponding to the data storage amount (buffer storage amount) of the transmission buffer 119, and then input to the variable length encoding circuit 118.

  That is, the controller 125 receives a target bit rate indicating the target bit rate supplied from the operator or the host computer, and a signal indicating the bit amount buffered in the transmission buffer 119, that is, a signal indicating the remaining buffer capacity, Based on this target bit rate and the remaining buffer capacity, feedback_q_scale_code for controlling the quantization step size of the quantization circuit 117 is generated. This feedback_q_scale_code is a control signal generated in accordance with the remaining buffer capacity of the transmission buffer 119 so that the transmission buffer 119 does not overflow or underflow, and the bits of the bit stream output from the transmission buffer 119 It is also a signal that controls the rate to be the target bit rate.

  Specifically, for example, when the bit amount buffered in the transmission buffer 119 has decreased, the quantization step size is reduced so that the generated bit amount of the picture to be encoded next increases. On the other hand, when the bit amount buffered in the transmission buffer 119 increases, the quantization step size is increased so that the generated bit amount of the next picture to be encoded is reduced. Note that feedback_q_scale_code and the quantization step size are proportional to each other. When feedback_q_scale_code is increased, the quantization step size is increased, and when feedback_q_scale_code is decreased, the quantization step size is decreased.

  Next, parameter reuse encoding processing, which is one of the features of the encoder 101, will be described focusing on differences from the above-described normal encoding processing. In order to explain this process more clearly, the reference picture is encoded as a P picture in the first generation encoding process, encoded as an I picture in the second generation encoding process, It is assumed that the B picture was encoded in the encoding process, and this reference picture must be encoded as a P picture in the current fourth generation encoding process.

  In this case, since this reference picture is the same picture type (P picture) as the picture type assigned as the fourth generation picture type and is encoded in the first generation encoding process, the controller 125 Instead of creating a new encoding parameter from the supplied video data, the encoding process is performed using the first generation encoding parameter. Typical encoding parameters to be reused in the fourth encoding process include quantizer_scale_code indicating the quantization scale step size, macroblock_type indicating the prediction direction mode, motion_vector indicating the motion vector, frame prediction mode or field. Frame / field_motion_type indicating the prediction mode, dct_type indicating the Frame DCT mode or Field DCT mode, and the like.

  That is, the controller 125 does not reuse all the encoding parameters transmitted as history information, but reuses and reuses the encoding parameters as described above that are expected to be reused. Encoding parameters that are considered to be preferable are not generated and are newly generated.

  The motion vector detection circuit 111 detects the motion vector of the reference picture in the normal encoding process described above, but does not perform the detection process of the motion vector motion_vector in the parameter reuse encoding process. The motion vector motion_vector supplied as history information of one generation is reused. The reason will be described.

  Since the baseband video data obtained by decoding the third generation encoded stream is subjected to at least three decoding and encoding processes, the image quality is clearly degraded as compared with the original video data. Even if a motion vector is detected from video data with degraded image quality, an accurate motion vector cannot be detected. That is, the motion vector supplied as the first generation history information is clearly a more accurate motion vector than the motion vector detected in the fourth generation encoding process. That is, by reusing the motion vector transmitted as the first generation encoding parameter, the image quality does not deteriorate even if the fourth generation encoding process is performed. The controller 125 uses the motion compensation circuit 124 and variable length coding as the motion vector information of this reference picture encoded in the fourth generation encoding process, using the motion vector motion_vector supplied as the first generation history information. Supply to circuit 118.

  Furthermore, the motion vector detection circuit 111 detects the prediction error in the frame prediction mode and the prediction error in the field prediction mode in order to determine whether the frame prediction mode or the field prediction mode is selected. In the use encoding process, the process of detecting the prediction error in the frame prediction mode and the prediction error in the field prediction mode is not performed, and whether the frame prediction mode or the field prediction mode supplied as the first generation history information is determined. Reuse the indicated frame / field_motion_type. This is because the prediction error in each prediction mode detected in the first generation is higher in accuracy than the prediction error in each prediction mode detected in the fourth generation encoding process. This is because a more optimal encoding process can be performed when the selected prediction mode is selected.

  Specifically, the controller 125 supplies a control signal corresponding to the frame / field_motion_type supplied as the first generation history information to the prediction mode switching circuit 113, and the prediction mode switching circuit 113 is reused. Signal processing corresponding to frame / field_motion_type is performed.

  Furthermore, in the normal encoding process, the motion vector detection circuit 111 uses any prediction mode (hereinafter referred to as prediction direction) of the intra-picture prediction mode, the forward prediction mode, the backward prediction mode, or the bidirectional prediction mode. The prediction error in each prediction direction mode is calculated in order to determine whether to select a mode), but in this parameter reuse encoding process, the prediction error in each prediction direction mode is not calculated. The prediction direction mode is determined based on the macroblock_type supplied as the first generation history information. This is because the prediction error in each prediction direction mode in the first generation encoding process is more accurate than the prediction error in each prediction direction mode in the fourth generation encoding process. This is because a more efficient encoding process can be performed by selecting the prediction direction mode determined by the above. Specifically, the controller 125 selects the prediction direction mode indicated by the macroblock_type included in the first generation history information, and the calculator 114 and the motion compensation circuit so as to correspond to the selected prediction direction mode. 124 is controlled.

  In the normal encoding process, the DCT mode switching circuit 115 compares the frame DCT mode encoding efficiency with the field DCT mode encoding efficiency, the field DCT mode signal format, and the field DCT mode Both of the signals converted to the signal format of the mode were supplied to the DCT circuit 116, but in this parameter reuse encoding process, the signal converted to the signal format of the frame DCT mode and the signal format of the field DCT mode are converted. The processing for generating both signals is not performed, and only the processing corresponding to the DCT mode indicated by dct_type included in the history information of the first generation is performed. Specifically, the controller 125 reuses the dct_type included in the first generation history information, so that the DCT mode switching circuit 115 performs signal processing corresponding to the DCT mode indicated by the dct_type. The mode switching circuit 115 is controlled.

  In the normal encoding process, the controller 125 controls the quantization step size of the quantization circuit 117 based on the target bit rate specified by the operator and the remaining amount of transmission buffer. This parameter reuse encoding process Then, the quantization step size of the quantization circuit 117 is controlled based on the target bit rate, the remaining transmission buffer capacity, and the past quantization scale included in the history information. In the following description, the past quantization scale included in the history information is described as history_q_scale_code. Further, in the history stream described later, this quantization scale is described as quantizer_scale_code.

  First, the controller 125 generates the current quantization scale feedback_q_scale_code as in the normal encoding process. The feedback_q_scale_code is a value determined according to the remaining buffer capacity of the transmission buffer 119 so that the transmission buffer 119 does not overflow and underflow. Subsequently, the previous quantization scale history_q_scale_code value included in the first generation history stream is compared with the current quantization scale feedback_q_scale_code value to determine which quantization scale is larger. . A large quantization scale means a large quantization step. If the current quantization scale feedback_q_scale_code is larger than the past quantization scale history_q_scale_code, the controller 125 supplies the current quantization scale feedback_q_scale_code to the quantization circuit 117. On the other hand, if the past quantization scale history_q_scale_code is larger than the current quantization scale feedback_q_scale_code, the controller 125 supplies the past quantization scale history_q_scale_code to the quantization circuit 117.

  That is, the controller 125 selects the largest quantization scale code among the plurality of past quantization scales included in the history information and the current quantization scale calculated from the remaining amount of the transmission buffer 119. . In other words, the controller 125 is used in the quantization step in the past (first, second, and third generation) encoding process or in the current (fourth generation) encoding process. The quantization circuit 117 is controlled to perform quantization using the largest quantization step among the quantization steps. The reason for this will be described below.

  For example, the bit rate of the stream generated in the third generation encoding process is 4 [Mbps], and the target bit rate set for the encoder 101 that performs the fourth generation encoding process is 15 [ Mbps]. At this time, since the target bit rate is increased, it is not actually the case that the quantization step should be simply reduced. Even if a picture encoded with a large quantization step in the past encoding process is encoded with a smaller quantization step in the current encoding process, the picture quality of this picture is improved. There is no. That is, encoding with a quantization step smaller than the quantization step in the past encoding process simply increases the bit amount and does not improve the image quality. Therefore, the largest quantization step in the quantization step used in the past (first, second, and third generation) encoding process or the quantization step used in the current (fourth generation) encoding process is selected. When used for quantization, the most efficient encoding process can be performed.

  As described above, the variable length encoding circuit 118 to which the data that has been subjected to the normal encoding process or the parameter reuse encoding process is input corresponds to the quantization scale supplied from the quantization circuit 117, The image data (in this case, I picture data) supplied from the conversion circuit 117 is converted into a variable-length code such as a Huffman code and output to the transmission buffer 119.

  In addition, the variable length coding circuit 118 has a quantization scale from the quantization circuit 117, and a prediction mode (a mode indicating whether intra prediction, forward prediction, backward prediction, or bidirectional prediction is set) from the controller 125, A motion vector is detected by the motion vector detection circuit 111, a prediction flag (a flag indicating whether the frame prediction mode or the field prediction mode is set) is set by the prediction mode switching circuit 113, and a DCT flag (frame DCT) output by the DCT mode switching circuit 115. A flag indicating whether the mode or the field DCT mode is set) is input, and these are also variable-length encoded.

  The transmission buffer 119 temporarily stores the input data and outputs buffer feedback to the controller 125. The controller 125 outputs data corresponding to the accumulation amount to the quantization circuit 117. When the remaining amount of data in the transmission buffer 119 is increased to the allowable upper limit value, the quantization scale of the quantization circuit 117 is increased by the quantization control signal from the controller 125, and the data amount of the quantized data is reduced. On the contrary, when the remaining data amount of the transmission buffer 119 decreases to the allowable lower limit value, the quantization scale of the quantization circuit 117 is reduced by the quantization control signal, thereby increasing the amount of quantized data. Let In this way, overflow or underflow of the transmission buffer 119 is prevented.

  Then, the data accumulated in the transmission buffer 119 is read and output at a predetermined timing.

  On the other hand, the I picture data output from the quantization circuit 117 is input to the inverse quantization circuit 120 and inversely quantized in accordance with the quantization scale supplied from the quantization circuit 117. The output of the inverse quantization circuit 120 is input to an IDCT (Inverse Discrete Cosine Transform) circuit 121, subjected to inverse discrete cosine transform processing, and then supplied to and stored in the forward prediction image unit 123a of the frame memory 123 via the arithmetic unit 122. Is done.

  When the motion vector detection circuit 111 processes the image data of each frame sequentially input as, for example, pictures of I, B, P, B, P, B,..., The motion vector detection circuit 111 After the image data is processed as an I picture, the image data of the next input frame is further processed as a P picture before the image of the next input frame is processed as a B picture. This is because a B picture is accompanied by backward prediction, and therefore cannot be decoded unless a P picture as a backward predicted image is prepared first.

  Therefore, the motion vector detection circuit 111 starts processing the image data of the P picture stored in the rear original image unit 112c after the processing of the I picture. As in the case described above, the absolute value sum of the inter-frame difference (prediction error) in units of macroblocks is supplied from the motion vector detection circuit 111 to the prediction mode switching circuit 113 and the controller 125. The prediction mode switching circuit 113 and the controller 125 correspond to the prediction of the frame / field prediction mode or intra-picture prediction, forward prediction, backward prediction, or bidirectional prediction in accordance with the absolute value sum of the prediction errors of the macroblock of the P picture. Set the mode.

  The computing unit 114 switches the switch 114d to the contact a side as described above when the intra-picture prediction mode is set. Accordingly, this data is transmitted to the transmission line via the DCT mode switching circuit 115, the DCT circuit 116, the quantization circuit 117, the variable length coding circuit 118, and the transmission buffer 119, similarly to the I picture data. Further, this data is supplied to and stored in the backward predicted image unit 123b of the frame memory 123 via the inverse quantization circuit 120, the IDCT circuit 121, and the calculator 122.

  When the forward prediction mode is set, the switch 114d is switched to the contact point b, and image (in this case, an I picture image) data stored in the forward prediction image portion 123a of the frame memory 123 is read. Then, the motion compensation circuit 124 performs motion compensation corresponding to the motion vector output from the motion vector detection circuit 111. That is, when the controller 125 instructs the motion compensation circuit 124 to set the forward prediction mode, the motion compensation circuit 124 sets the read address of the forward prediction image unit 123a to the position of the macroblock currently output by the motion vector detection circuit 111. Data is read out from the corresponding position by the amount corresponding to the motion vector, and predicted image data is generated.

  The predicted image data output from the motion compensation circuit 124 is supplied to the calculator 114a. The computing unit 114a subtracts the prediction image data corresponding to the macroblock supplied from the motion compensation circuit 124 from the macroblock data of the reference image supplied from the prediction mode switching circuit 113, and the difference (prediction error). ) Is output. The difference data is transmitted to the transmission line via the DCT mode switching circuit 115, the DCT circuit 116, the quantization circuit 117, the variable length coding circuit 118, and the transmission buffer 119. The difference data is locally decoded by the inverse quantization circuit 120 and the IDCT circuit 121 and input to the computing unit 122.

  The calculator 122 is also supplied with the same data as the predicted image data supplied to the calculator 114a. The calculator 122 adds the predicted image data output from the motion compensation circuit 124 to the difference data output from the IDCT circuit 121. As a result, image data of the original (decoded) P picture is obtained. The image data of the P picture is supplied to and stored in the backward predicted image unit 123b of the frame memory 123.

  In this way, the motion vector detection circuit 111 stores the data of the I picture and the P picture in the forward predicted image unit 123a and the backward predicted image unit 123b, respectively, and then executes the process of the B picture. The prediction mode switching circuit 113 and the controller 125 set the frame / field mode corresponding to the magnitude of the absolute value of the inter-frame difference in units of macroblocks, and the prediction mode is set to the intra prediction mode and the forward prediction. Set to either mode, backward prediction mode, or bidirectional prediction mode.

  As described above, in the intra prediction mode or the forward prediction mode, the switch 114d is switched to the contact point a or b. At this time, the same processing as in the case of the P picture is performed, and data is transmitted.

  On the other hand, when the backward prediction mode or the bidirectional prediction mode is set, the switch 114d is switched to the contact c or d, respectively.

  In the backward prediction mode in which the switch 114d is switched to the contact c, image data (in this case, a P picture image) data stored in the backward prediction image unit 123b is read out, and the motion compensation circuit 124 Motion compensation is performed corresponding to the motion vector output from the vector detection circuit 111. That is, when the controller 125 is instructed to set the backward prediction mode, the motion compensation circuit 124 sets the readout address of the backward prediction image unit 123b to the position of the macroblock currently being output by the motion vector detection circuit 111. Data is read out from the corresponding position by the amount corresponding to the motion vector, and predicted image data is generated.

  The predicted image data output from the motion compensation circuit 124 is supplied to the calculator 114b. The computing unit 114b subtracts the predicted image data supplied from the motion compensation circuit 124 from the macroblock data of the reference image supplied from the prediction mode switching circuit 113, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 115, the DCT circuit 116, the quantization circuit 117, the variable length coding circuit 118, and the transmission buffer 119.

  In the bidirectional prediction mode in which the switch 114d is switched to the contact point d, image data (in this case, an I picture image) data stored in the forward predicted image unit 123a and stored in the backward predicted image unit 123b. Image (in this case, an image of a P picture) data is read out, and motion compensation is performed by the motion compensation circuit 124 corresponding to the motion vector output from the motion vector detection circuit 111.

  That is, the motion compensation circuit 124, when the controller 125 is instructed to set the bidirectional prediction mode, the macro that the motion vector detection circuit 111 is currently outputting the read addresses of the forward prediction image unit 123a and the backward prediction image unit 123b. Data is read out from the position corresponding to the position of the block by the amount corresponding to the motion vector (the motion vectors in this case are two for the forward prediction image and the backward prediction image), and prediction image data is generated.

  The predicted image data output from the motion compensation circuit 124 is supplied to the calculator 114c. The computing unit 114c subtracts the average value of the predicted image data supplied from the motion compensation circuit 124 from the macroblock data of the reference image supplied from the motion vector detection circuit 111, and outputs the difference. The difference data is transmitted to the transmission line via the DCT mode switching circuit 115, the DCT circuit 116, the quantization circuit 117, the variable length coding circuit 118, and the transmission buffer 119.

  The B picture image is not stored in the frame memory 123 because it is not a predicted image of another image.

  Note that, in the frame memory 123, the forward predicted image unit 123a and the backward predicted image unit 123b are subjected to bank switching as necessary, and those stored in one or the other with respect to a predetermined reference image It can be switched and output as a predicted image or a backward predicted image.

  In the above description, the luminance block is mainly described, but the color difference block is also processed and transmitted in units of macroblocks shown in FIG. Note that the motion vector when processing the color difference block is obtained by halving the motion vector of the corresponding luminance block in the vertical direction and the horizontal direction, respectively.

  FIG. 13 shows a state where a plurality of transcoding systems are connected in series and used in a video editing studio, for example. The history information multiplexing device 63-i of each video decoding system 51-i (i = 1 to N) is set in the section in which the oldest encoding parameter in the above-described encoding parameter area is recorded. Overwrite the latest encoding parameters used. As a result, as described with reference to FIG. 11, encoding parameters (generation history information) for the four most recent generations corresponding to the same macroblock are recorded in the baseband image data. .

  In FIG. 13, the video decoding system 51 and the video encoding system 52 of the transcoding system described with reference to FIG. 4 are described. However, instead of the video decoding system 51 and the video encoding system 52, FIG. Needless to say, the video decoding system 1 and the video encoding system 2 described with reference to FIGS. 2 and 3 may be used.

  In the transcoding system in the present embodiment, as described above, the decoding (video decoding system 1 or video decoding system 51) side and the code side (video encoding system 2 or video encoding system 52) are provided. Coarse coupling is performed and the encoding parameters are multiplexed with the image data and transmitted. However, as shown in FIG. 14, the decoding apparatus and the encoding apparatus may be directly connected (tightly coupled).

  That is, the variable length decoding circuit 92 of the decoding device 61 extracts the third generation encoding parameter from the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the third generation encoded stream, Is supplied to the history encoding device 65, and the baseband video signal in which the encoding parameter of the picture layer or higher is inserted in the V Blanking region and the encoding parameter of the slice layer or lower is inserted in the H Blanking region is converted into the encoding device 66. The controller 125 is supplied. The history encoding device 65 converts the received third-generation encoding parameter into converted_history_stream () so that it can be described in the user data area of the picture layer, and converts the converted_history_stream () as user data into the variable length code of the encoding device 66. To the circuit 118.

  Further, the variable length decoding circuit 92 extracts user data user_data including the first generation encoding parameter and the second generation encoding parameter from the user data area of the picture layer of the third generation encoded stream, This is supplied to the history decoding device 62. The history decoding device 62 extracts the first generation encoding parameter and the second generation encoding parameter from the history stream described as converted_history_stream () in the user data area, and extracts it from the controller 125 of the encoding device 66. To supply.

  The controller 125 of the encoding device 66 includes the first generation and second generation encoding parameters received from the history decoding device 62 and the codes described in the V Blanking region and the H blanking region received from the decoding device 61. The encoding process of the encoding device 66 is controlled based on the encoding parameter.

  The variable length encoding circuit 118 of the encoding device 66 receives the encoding parameters included in the data input from the decoding device 61 and receives user data including the third generation encoding parameters from the history encoding device 65. The received user data is described as history information in the user data area of the picture layer of the fourth generation encoded stream and output.

  FIG. 15 is a diagram showing a syntax for decoding an MPEG video stream. The decoder extracts a plurality of meaningful data items (data elements) from the bit stream by decoding the MPEG bit stream according to this syntax. In the drawing, the syntax described below has functions and conditional statements expressed in small letters, and data elements are shown in bold letters. The data item is described in mnemonic (Mnemonic) indicating its name, bit length, its type and transmission order.

  First, functions used in the syntax shown in FIG. 15 will be described.

  The next_start_code () function is a function for searching for a start code described in the bitstream. In the syntax shown in FIG. 15, the sequence_header () function and the sequence_extension () function are arranged in this order after the next_start_code () function. Data elements defined by the sequence_extension () function are described. Therefore, when decoding the bitstream, the next_start_code () function finds the start code (a type of data element) described at the beginning of the sequence_header () function and sequence_extension () function from the bitstream, and uses it as a reference Thus, the sequence_header () function and the sequence_extension () function are further found, and each data element defined by them is decoded.

  The sequence_header () function is a function for defining header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is a function for defining extension data of the sequence layer of the MPEG bit stream. .

  The do {} while syntax placed next to the sequence_extension () function is a data element written based on the function in {} of the do statement while the condition defined by the while statement is true. This is a syntax for extracting from the stream. That is, with the do {} while syntax, while the condition defined by the while statement is true, a decoding process is performed to extract the data element described based on the function in the do statement from the bit stream.

  The nextbits () function used in the while statement is a function for comparing a bit or a bit string appearing in the bit stream with a data element to be decoded next. In the syntax example of FIG. 15, the nextbits () function compares the bit string in the bitstream with sequence_end_code indicating the end of the video sequence, and when the bit string in the bitstream does not match sequence_end_code, The sentence condition is true. Therefore, the do {} while syntax arranged next to the sequence_extension () function indicates that the data element defined by the function in the do statement is not included in the bitstream while the sequence_end_code indicating the end of the video sequence does not appear in the bitstream. It shows that it is described in.

  In the bitstream, data elements defined by the extension_and_user_data (0) function are described after each data element defined by the sequence_extension () function. The extension_and_user_data (0) function is a function for defining extension data and user data of the sequence layer of the MPEG bit stream.

  The do {} while syntax placed next to this extension_and_user_data (0) function is a data element written based on the function in {} of the do statement while the condition defined by the while statement is true. Is a function for extracting from the bitstream. The nextbits () function used in the while statement is a function for determining a match between a bit or a bit string appearing in the bit stream and picture_start_code or group_start_code, and the bit or bit string appearing in the bit stream, If picture_start_code or group_start_code matches, the condition defined by the while statement is true. Therefore, in this do {} while syntax, when picture_start_code or group_start_code appears in the bitstream, the code of the data element defined by the function in the do statement is described next to the start code. By searching for the start code indicated by this picture_start_code or group_start_code, the data element defined in the do statement can be extracted from the bitstream.

  The if statement described at the beginning of the do statement indicates a condition that group_start_code appears in the bitstream. When the condition by this if statement is true, the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function are described in order in the bitstream after the group_start_code.

  The group_of_picture_header (1) function is a function for defining the header data of the GOP layer of the MPEG bit stream. The extension_and_user_data (1) function is the extension data (extension_data) and user data (extension_data) of the MPEG bit stream. This is a function for defining (user_data).

  Furthermore, in this bitstream, data elements defined by the picture_header () function and the picture_coding_extension () function are described after the data elements defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function. Yes. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header (1) function and the extension_and_user_data (1) function is not described, so it is defined by the extension_and_user_data (0) function. The data element defined by the picture_header () function and the picture_coding_extension () function is described after the data element.

  The picture_header () function is a function for defining header data of the picture layer of the MPEG bit stream, and the picture_coding_extension () function is a function for defining first extension data of the picture layer of the MPEG bit stream It is.

  The next while statement is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement is a function for determining whether the bit string appearing in the bit stream matches extension_start_code or user_data_start_code, and the bit string appearing in the bit stream, extension_start_code or user_data_start_code, If they match, the condition defined by this while statement is true.

  The first if statement is a function for determining whether the bit string appearing in the bitstream matches extension_start_code. When the bit string appearing in the bitstream matches the 32-bit extension_start_code, the data element defined by the extension_data (2) function is described next to the extension_start_code in the bitstream.

  The second if statement is a syntax for determining a match between the bit string appearing in the bitstream and user_data_start_code. If the bit string appearing in the bitstream matches the 32-bit user_data_start_code, the third if statement Condition judgment of if statement is performed. This user_data_start_code is a start code for indicating the start of the user data area of the picture layer of the MPEG bit stream.

  The third if statement is a syntax for determining whether the bit string appearing in the bitstream matches History_Data_ID. If the bit string appearing in the bitstream matches this 32-bit History_Data_ID, then in the user data area of the picture layer of this MPEG bitstream, the converted_history_stream () is next to the code indicated by this 32-bit History_Data_ID. Describes a data element defined by a function.

  The converted_history_stream () function is a function for describing history information and history data for transmitting all the encoding parameters used at the time of MPEG encoding. Details of the data element defined by the converted_history_stream () function will be described later as history_stream () with reference to FIGS. 17 to 24. The History_Data_ID is a start code indicating the history information and history data described in the user data area of the picture layer of the MPEG bit stream.

  The else statement is a syntax for indicating that the condition is not true in the third if statement. Therefore, when the data element defined by the converted_history_stream () function is not described in the user data area of the picture layer of the MPEG bit stream, the data element defined by the user_data () function is described.

  In FIG. 15, the history information is described in converted_history_stream () and not described in user_data (), but this converted_history_stream () is described as a kind of user_data in the MPEG standard. Therefore, in this specification, it is also described that history information is described in user_data depending on the case, but this means that it is described as a kind of user_data of the MPEG standard.

  The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer next to the user data of the picture layer of the MPEG bit stream. Normally, the data element indicated by the picture_data () function is a data element defined by the converted_history_stream () function described in the user data area of the picture layer of the bitstream, or a data element defined by the user_data () function If there is no extension_start_code or user_data_start_code in the bitstream indicating the data element of the picture layer, the data element indicated by this picture_data () function is defined by the picture_coding_extension () function. Is described next to the data element to be processed.

  Next to the data element indicated by the picture_data () function, data elements defined by the sequence_header () function and the sequence_extension () function are arranged in order. The data elements described by the sequence_header () function and the sequence_extension () function are exactly the same as the data elements described by the sequence_header () function and the sequence_extension () function described at the beginning of the video stream sequence. The reason why the same data is described in the stream in this way is that the data of the sequence layer is received when the reception starts from the middle of the data stream (for example, the bit stream portion corresponding to the picture layer) on the bit stream receiver side. This is to prevent the stream from being able to be decoded and the stream from being decoded.

  Following the data element defined by the last sequence_header () function and sequence_extension () function, that is, at the end of the data stream, 32-bit sequence_end_code indicating the end of the sequence is described.

  An outline of the basic structure of the above syntax is as shown in FIG.

  Next, a history stream defined by the converted_history_stream () function will be described.

  This converted_history_stream () is a function for inserting a history stream indicating history information into the user data area of the MPEG picture layer. The meaning of “converted” is a conversion process that inserts a marker bit (1 bit) at least every 22 bits of a history stream composed of history data to be inserted into the user area in order to prevent start emulation. It means that it is a stream.

  The converted_history_stream () is described in either a fixed-length history stream (FIGS. 17 to 23) or a variable-length history stream (FIG. 24) described below. When a fixed-length history stream is selected on the encoder side, there is an advantage that a circuit and software for decoding each data element from the history stream on the decoder side are simplified. On the other hand, when a variable length history stream is selected on the encoder side, history information (data elements) described in the user area of the picture layer can be arbitrarily selected in the encoder as needed. Can be reduced, and as a result, the data rate of the entire encoded bitstream can be reduced.

  The “history stream”, “history stream”, “history information”, “history information”, “history data”, “history data”, “history parameter”, and “history parameter” described in the present invention are past codes. It means the encoding parameter (or data element) used in the encoding process, and does not mean the encoding parameter used in the current (final stage) encoding process. For example, in the first generation encoding process, a picture is encoded and transmitted with an I picture, and in the next second generation encoding process, this picture is encoded and transmitted as a P picture. In the third generation encoding process, an example will be described in which this picture is encoded with a B picture and transmitted.

  The encoding parameters used in the third generation encoding process are set at predetermined positions in the sequence layer, GOP layer, picture layer, slice layer, and macroblock layer of the encoded bitstream generated in the third generation encoding process. is described. On the other hand, the encoding parameters used in the first generation and second generation encoding processes, which are past encoding processes, are sequence layers and GOP layers in which the encoding parameters used in the third generation encoding process are described. Is described in the user data area of the picture layer as the history information of the encoding parameter according to the syntax described above.

  First, the fixed-length history stream syntax will be described with reference to FIGS.

  The user data area in the picture layer of the bitstream generated in the final stage (for example, third generation) encoding process is first used in the past (for example, first generation and second generation) encoding processes. The encoding parameter included in the sequence header of the sequence layer that has been stored is inserted as a history stream. Note that history information such as the sequence header of the sequence layer of the bit stream generated in the past encoding process is not inserted into the sequence header of the sequence layer of the bit stream generated in the encoding process of the final stage. It should be noted that.

  The data elements included in the sequence header (sequence_header) used in the past encoding process are sequence_header_code, sequence_header_present_flag, horizontal_size_value, marker_bit, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, VBV_buffer_size_value, constant_in_traiter, Composed.

  The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. The marker_bit is bit data inserted to prevent start code emulation. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle.

  bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantiser_matrix is data indicating the presence of intra MB quantization matrix data. load_non_intra_quantiser_matrix is data indicating the presence of non-intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the intra MB quantization matrix. non_intra_quantiser_matrix is data representing a value of a non-intra MB quantization matrix.

  In the user data area of the picture layer of the bit stream generated in the encoding process at the final stage, a data element representing a sequence extension of the sequence layer used in the past encoding process is described as a history stream.

  The data elements representing the sequence extensions (sequence_extension) used in the past encoding process are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_ext_, extension_delay_frame_extension_, It is.

  extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data.

  The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data to be added to the vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

  Subsequently, in the user area of the picture layer of the bit stream, a data element representing a sequence layer sequence display extension used in the past encoding process is described as a history stream.

  A data element described as this sequence display extension (sequence_display_extension) is composed of extension_start_code, extension_start_code_identifier, sequence_display_extension_present_flag, video_format, colour_description, colour_primaries, transfer_characteristics, matrix_coeffients, display_horizontal_size, and display_vertical_size.

  extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is a code indicating which extension data is sent. The sequence_display_extension_present_flag is data indicating whether the data element in the sequence display extension is valid or invalid. video_format is data representing the video format of the original signal. color_description is data indicating that there is detailed data of the color space. color_primaries is data indicating details of the color characteristics of the original signal. transfer_characteristics is data indicating details of how photoelectric conversion is performed. Matrix_coeffients is data indicating details of how the original signal is converted from the three primary colors of light. display_horizontal_size is data representing the active area (horizontal size) of the intended display. display_vertical_size is data representing the active area (vertical size) of the intended display.

  Subsequently, macroblock assignment data (macroblock_assignment_in_user_data) indicating the phase information of the macroblock generated in the past encoding process is stored in the user area of the picture layer of the bitstream generated in the final stage encoding process. It is described as a history stream.

  Macroblock_assignment_in_user_data indicating the phase information of the macroblock is composed of data elements such as macroblock_assignment_present_flag, v_phase, and h_phase.

  This macroblock_assignment_present_flag is data indicating whether the data element in macroblock_assignment_in_user_data is valid or invalid. v_phase is data indicating vertical phase information when a macroblock is cut out from image data. h_phase is data indicating horizontal phase information when a macroblock is cut out from image data.

  Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the GOP header of the GOP layer used in the past encoding process is described as a history stream. Yes.

  A data element representing this GOP header (group_of_picture_header) is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.

  group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. Broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.

  Subsequently, in the user area of the picture layer of the bitstream generated by the encoding process at the final stage, a data element representing the picture header of the picture layer used in the past encoding process is described as a history stream. Yes.

  Data elements relating to this picture header (picture_header) are composed of picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, and backward_f_code.

  Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range.

  Subsequently, in the user area of the picture layer of the bit stream generated by the encoding process at the final stage, the picture coding extension of the picture layer used in the past encoding process is described as a history stream.

  The data elements for this picture coding extension (picture_coding_extension) are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_carrier, burst_amplitude, burst_amplitude

  extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction.

  intra_dc_precision is data representing the precision of the DC coefficient. Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, the data indicates whether the upper field or the lower field. top_field_first is data indicating whether the first field is upper or lower in the case of a frame structure. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock.

  q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock. The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal. v_axis, field_sequence, sub_carrier, burst_amplitude, and sub_carrier_phase are data used when the source signal is PAL.

  Subsequently, the quantization matrix extension used in the past encoding process is described as a history stream in the user area of the picture layer of the bit stream generated by the encoding process at the final stage.

  Data elements related to the quantization matrix extension (quant_matrix_extension) are, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantiser_matrix, intra_quantiser_matrix [64], load_non_intra_quantiser_matrix, non_intra_quantiser_matrix [64], load_chroma_intra_quantiser_matrix, chroma_intra_quantiser_matrix [64], is composed of Load_chroma_non_intra_quantiser_matrix, and chroma_non_intra_quantiser_matrix [64] The

  extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantiser_matrix is data indicating the value of a quantization matrix for an intra macroblock.

  load_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantiser_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantiser_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantiser_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for color difference non-intra macroblocks. chroma_non_intra_quantiser_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.

  Subsequently, the copyright extension used in the past encoding process is described as the history stream in the user area of the picture layer of the bit stream generated by the encoding process in the final stage.

  Data elements related to this copyright extension (copyright_extension) are composed of extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.

  extension_start_code is a start code indicating the start of the copyright extension. This code indicates which extension data of extension_start_code_itentifier is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid. copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end.

  The copyright_identifier is data for identifying the registration organization of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.

  Subsequently, the picture display extension (picture_display_extension) used in the past encoding process is described as a history stream in the user area of the picture layer of the bitstream generated by the encoding process at the final stage.

  Data elements representing this picture display extension are composed of extension_start_code, extension_start_code_identifier, picture_display_extension_present_flag, frame_center_horizontal_offset_1, frame_center_vertical_offset_1, frame_center_horizontal_offset_2, frame_center_vertical_offset_2, frame_center_horizontal_offset_3, and _offcenter_3.

  extension_start_code is a start code for indicating the start of the picture display extension. extension_start_code_identifier is a code indicating which extension data is sent. picture_display_extension_present_flag is data indicating whether a data element in the picture display extension is valid or invalid. The frame_center_horizontal_offset is data indicating a horizontal offset of the display area, and can be defined up to three offset values. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and can be defined up to three offset values.

  In the user area of the picture layer of the bitstream generated in the encoding process of the final stage, user data (user_data) used in the past encoding process is next to the history information indicating the picture display extension described above. , Described as a history stream.

  Next to the user data, information on the macroblock layer used in the past encoding process is described as a history stream.

  Information about the macroblock layer includes data elements related to macroblock (macroblock) positions such as macroblock_address_h, macroblock_address_v, slice_header_present_flag, skipped_macroblock_flag, macroblock_quant, macroblock_motion_forward, macroblock_motion_backward, macroblock_pattern, macro_block_frame, (Macroblock_modes []), data elements related to quantization step control such as quantizer_scale_code, PMV [0] [0] [0], PMV [0] [0] [1], motion_vertical_field_select [0] [0 ], PMV [0] [1] [0], PMV [0] [1] [1], motion_vertical_field_select [0] [1], PMV [1] [0] [0], PMV [1] [0] Motion compensation data elements such as [1], motion_vertical_field_select [1] [0], PMV [1] [1] [0], PMV [1] [1] [1], motion_vertical_field_select [1] [1] Macro block such as coded_block_pattern And data elements related to the pattern, num_mv_bits, is configured Num_coef_bits, and the data elements relating to the generated code amount of such Num_other_bits.

  Hereinafter, data elements related to the macroblock layer will be described in detail.

  macroblock_address_h is data for defining the absolute position of the current macroblock in the horizontal direction. macroblock_address_v is data for defining the absolute position of the current macroblock in the vertical direction. The slice_header_present_flag is data indicating whether or not this macroblock is the head of the slice layer and is accompanied by a slice header. skipped_macroblock_flag is data indicating whether or not to skip this macroblock in the decoding process.

  The macroblock_quant is data derived from a macroblock type (macroblock_type) shown in FIGS. 38 and 39, which will be described later, and indicates whether quantizer_scale_code appears in the bitstream. Macroblock_motion_forward and macroblock_motion_backward are data derived from the macroblock types shown in FIGS. 38 and 39, and are used in the decoding process. mocroblock_pattern is data derived from the macroblock type shown in FIGS. 38 and 39, and indicates whether coded_block_pattern appears in the bitstream.

  The macroblock_intra is data derived from the macroblock type shown in FIGS. 38 and 39, and is used in the decoding process. spatial_temporal_weight_code_flag is data derived from the macroblock type shown in FIG. 38 and FIG. 39, and spatial_temporal_weight_code indicating the upsampling method of the lower layer image with temporal scalability is data indicating whether or not the bitstream exists. It is.

  frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”. field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime. dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode. quantiser_scale_code is data indicating the quantization step size of the macroblock.

  Next, data elements relating to motion vectors will be described. The motion vector is encoded as a difference with respect to the previously encoded vector in order to reduce the motion vector required during decoding. In order to perform motion vector decoding, the decoder must maintain four motion vector prediction values (with horizontal and vertical components, respectively). This predicted motion vector is expressed as PMV [r] [s] [v]. [r] is a flag indicating whether the motion vector in the macroblock is the first vector or the second vector, and “0” when the vector in the macroblock is the first vector. When the vector in the macroblock is the second vector, “1” is set. [s] is a flag indicating whether the direction of the motion vector in the macroblock is the forward direction or the backward direction, and is “0” in the case of the forward motion vector, and in the case of the backward motion vector. Becomes “1”. [v] is a flag indicating whether the vector component in the macroblock is in the horizontal direction or the vertical direction, and is “0” for the horizontal component and “1” for the vertical component. "

  Therefore, PMV [0] [0] [0] represents the horizontal component data of the forward motion vector of the first vector, and PMV [0] [0] [1] represents the first vector. PMV [0] [1] [0] represents the vertical component data of the forward motion vector, PMV [0] [1] [0] represents the horizontal component data of the backward motion vector of the first vector, and PMV [0] [ 1] [1] represents the data of the vertical component of the backward motion vector of the first vector, and PMV [1] [0] [0] represents the horizontal of the forward motion vector of the second vector. Represents the direction component data, PMV [1] [0] [1] represents the vertical component data of the forward motion vector of the second vector, and PMV [1] [1] [0] PMV [1] [1] [1] represents the data of the vertical component of the backward motion vector of the second vector, and represents the data of the horizontal component of the backward motion vector of the second vector. It represents.

  motion_vertical_field_select [r] [s] is data indicating which reference field is used for the prediction format. When the motion_vertical_field_select [r] [s] is “0”, the top reference field is used, and when it is “1”, the bottom reference field is used.

  Therefore, motion_vertical_field_select [0] [0] indicates a reference field for generating a forward motion vector of the first vector, and motion_vertical_field_select [0] [1] indicates a backward motion vector of the first vector. , Motion_vertical_field_select [1] [0] indicates a reference field for generating a forward motion vector of the second vector, and motion_vertical_field_select [1] [1] A reference field for generating a backward motion vector of the second vector is shown.

  The coded_block_pattern is variable length data indicating which DCT block has a significant coefficient (non-zero coefficient) among a plurality of DCT blocks storing DCT coefficients. num_mv_bits is data indicating the code amount of the motion vector in the macroblock. num_coef_bits is data indicating the code amount of the DCT coefficient in the macroblock. num_other_bits is data indicating the code amount of the macroblock and the code amount other than the motion vector and the DCT coefficient.

  Next, a syntax for decoding each data element from a variable-length history stream will be described with reference to FIGS.

  The variable-length history stream shown in FIG. 24 includes a next_start_code () function, a sequence_header () function, a sequence_extension () function, an extension_and_user_data (0) function, a group_of_picture_header () function, an extension_and_user_data (1) function, a picture_header () function, and a picture_coding_extension The data element is defined by the () function, the re_coding_stream_info () function, the extension_and_user_data (2) function, and the picture_data () function.

  Since the next_start_code () function is a function for searching for a start code existing in the bit stream, the top of the history stream is a data element used in the past encoding process as shown in FIG. A data element defined by the sequence_header () function is described.

  The data elements defined by the sequence_header () function shown in Fig. 25 are sequence_header_code, sequence_header_present_flag, horizontal_size_value, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, marker_bit, VBV_buffer_size_value, constrained_parameter_trait_intra

  The sequence_header_code is data representing the start synchronization code of the sequence layer. The sequence_header_present_flag is data indicating whether the data in the sequence_header is valid or invalid. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle. bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits.

  The marker_bit is bit data inserted to prevent start code emulation. VBV_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantiser_matrix is data indicating the presence of intra MB quantization matrix data. intra_quantiser_matrix is data indicating the value of the intra MB quantization matrix. load_non_intra_quantiser_matrix is data indicating the presence of non-intra MB quantization matrix data. non_intra_quantiser_matrix is data representing a value of a non-intra MB quantization matrix.

  Next to the data element defined by the sequence_header () function, the data element defined by the sequence_extension () function as shown in FIG. 26 is described as a history stream.

  Data elements defined by the sequence_extension () function are extension_start_code, extension_start_code_identifier, sequence_extension_present_flag, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_extension, bit_rate_extension, _delay_rate_rate_delay_rate_delay_rate

  extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. The sequence_extension_present_flag is data indicating whether the data in the sequence extension is valid or invalid. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data. The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header.

  low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

  Next to the data element defined by the sequence_extension () function, the data element defined by the extension_and_user_data (0) function as shown in FIG. 27 is described as a history stream. The extension_and_user_data (i) function describes only the data element defined by the user_data () function as a history stream without describing the data element defined by the extension_data () function when “i” is other than 1. . Therefore, the extension_and_user_data (0) function describes only the data element defined by the user_data () function as a history stream.

  The user_data () function describes user data as a history stream based on the syntax as shown in FIG.

  Next to the data element defined by the extension_and_user_data (0) function, the data element defined by the group_of_picture_header () function as shown in FIG. 29 and the data element defined by the extension_and_user_data (1) function are used as a history stream. is described. However, the data element defined by the group_of_picture_header () function and the data element defined by the extension_and_user_data (1) function are described only when group_start_code indicating the GOP layer start code is described in the history stream. ing.

  The data element defined by the group_of_picture_header () function is composed of group_start_code, group_of_picture_header_present_flag, time_code, closed_gop, and broken_link.

  group_start_code is data indicating the start synchronization code of the GOP layer. group_of_picture_header_present_flag is data indicating whether a data element in group_of_picture_header is valid or invalid. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. Broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.

  Similar to the extension_and_user_data (0) function, the extension_and_user_data (1) function describes only the data element defined by the user_data () function as a history stream.

  If group_start_code indicating the GOP layer start code does not exist in the history stream, the data elements defined by these group_of_picture_header () and extension_and_user_data (1) functions are described in the history stream. Absent. In that case, after the data element defined by the extension_and_user_data (0) function, the data element defined by the picture_headr () function is described as a history stream.

  Data elements defined by the picture_headr () function are picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, backward_f_code, extra_bit_picture, and extra_information_picture, as shown in FIG.

  Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type. vbv_delay is data indicating the initial state of the virtual buffer at the time of random access. full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range. extra_bit_picture is a flag indicating the presence of subsequent additional information. When this extra_bit_picture is “1”, there is next extra_information_picture, and when extra_bit_picture is “0”, it indicates that there is no subsequent data. extra_information_picture is information reserved in the standard.

  Next to the data element defined by the picture_headr () function, the data element defined by the picture_coding_extension () function as shown in FIG. 31 is described as a history stream.

  Data elements defined by this picture_coding_extension () function include extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision, picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_firt_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_mplitude, phase_st, sub_mpl,

  extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction. intra_dc_precision is data representing the precision of the DC coefficient.

  The picture_structure indicates whether it is a frame structure or a field structure, and in the case of a field structure, it is data indicating whether it is an upper field or a lower field. In the case of a frame structure, top_field_first is data indicating whether the first field is upper or lower. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC is used for the intra macroblock.

  The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan. repeat_firt_field is data used for 2: 3 pull-down. The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. progressive_frame is data indicating whether or not this picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal. v_axis, field_sequence, sub_carrier, burst_amplitude, and sub_carrier_phase are data used when the source signal is PAL.

  Next to the data element defined by the picture_coding_extension () function, the data element defined by the re_coding_stream_info () function is described as a history stream. The re_coding_stream_info () function is mainly used when describing a combination of history information, and details thereof will be described later with reference to FIG.

  Next to the data element defined by the re_coding_stream_info () function, the data element defined by extensions_and_user_data (2) is described as a history stream. As shown in FIG. 27, the extension_and_user_data (2) function describes data elements defined by the extension_data () function when an extension start code (extension_start_code) exists in the bitstream. Next to this data element, when a user data start code (user_data_start_code) exists in the bit stream, a data element defined by the user_data () function is described. However, when the extension start code and the user data start code do not exist in the bit stream, the data elements defined by the extension_data () function and the user_data () function are not described in the bit stream.

  As shown in FIG. 32, the extension_data () function records a data element indicating extension_start_code and data elements defined by the quant_matrix_extension () function, copyright_extension () function, and picture_display_extension () function in the bitstream. This is a function to describe as a stream.

  Data elements defined by the quant_matrix_extension () function, as shown in FIG. 33, extension_start_code, extension_start_code_identifier, quant_matrix_extension_present_flag, load_intra_quantiser_matrix, intra_quantiser_matrix [64], load_non_intra_quantiser_matrix, non_intra_quantiser_matrix [64], load_chroma_intra_quantiser_matrix, chroma_intra_quantiser_matrix [64], load_chroma_non_intra_quantiser_matrix, and chroma_non_intra_quantiser_matrix [64].

  extension_start_code is a start code indicating the start of the quantization matrix extension. extension_start_code_identifier is a code indicating which extension data is sent. quant_matrix_extension_present_flag is data for indicating whether the data element in the quantization matrix extension is valid or invalid. load_intra_quantiser_matrix is data indicating the presence of quantization matrix data for intra macroblocks. Intra_quantiser_matrix is data indicating the value of a quantization matrix for an intra macroblock.

  load_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for non-intra macroblocks. non_intra_quantiser_matrix is data representing the value of a quantization matrix for a non-intra macroblock. load_chroma_intra_quantiser_matrix is data indicating the presence of quantization matrix data for the color difference intra macroblock. chroma_intra_quantiser_matrix is data indicating the value of the quantization matrix for the color difference intra macroblock. load_chroma_non_intra_quantiser_matrix is data indicating the presence of quantization matrix data for color difference non-intra macroblocks. chroma_non_intra_quantiser_matrix is data indicating the value of the quantization matrix for the chrominance non-intra macroblock.

  As shown in FIG. 34, the data element defined by the copyright_extension () function includes extension_start_code, extension_start_code_itentifier, copyright_extension_present_flag, copyright_flag, copyright_identifier, original_or_copy, copyright_number_1, copyright_number_2, and copyright_number_3.

  extension_start_code is a start code indicating the start of the copyright extension. extension_start_code_itentifier This code indicates which extension data is sent. The copyright_extension_present_flag is data for indicating whether the data element in this copyright extension is valid or invalid.

  copyright_flag indicates whether or not a copy right is given to the encoded video data until the next copyright extension or sequence end. The copyright_identifier is data for identifying the registration organization of the copy right specified by ISO / IEC JTC / SC29. original_or_copy is data indicating whether the data in the bitstream is original data or copy data. copyright_number_1 is data representing bits 44 to 63 of the copyright number. copyright_number_2 is data representing bits 22 to 43 of the copyright number. copyright_number_3 is data representing bits 0 to 21 of the copyright number.

  Data elements defined by the picture_display_extension () function are extension_start_code_identifier, frame_center_horizontal_offset, frame_center_vertical_offset, etc., as shown in FIG.

  extension_start_code_identifier is a code indicating which extension data is sent. The frame_center_horizontal_offset is data indicating the horizontal offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined. The frame_center_vertical_offset is data indicating the vertical offset of the display area, and the number of offset values defined by number_of_frame_center_offsets can be defined.

  Returning to FIG. 24 again, after the data element defined by the extension_and_user_data (2) function, the data element defined by the picture_data () function is described as a history stream. However, this picture_data () function exists when red_bw_flag is not 1 or red_bw_indicator is 2 or less. The red_bw_flag and red_bw_indicator are described in the re_coding_stream_info () function, which will be described later with reference to FIGS. 48 and 49.

  The data element defined by the picture_data () function is a data element defined by the slice () function as shown in FIG. At least one data element defined by the slice () function is described in the bit stream.

  As shown in FIG. 37, the slice () function includes data elements such as slice_start_code, slice_quantiser_scale_code, intra_slice_flag, intra_slice, reserved_bits, extra_bit_slice, extra_information_slice, and extra_bit_slice, as well as data elements defined by the macroblock () function. It is a function to describe as.

  slice_start_code is a start code indicating the start of the data element defined by the slice () function. The slice_quantiser_scale_code is data indicating the quantization step size set for the macroblock existing in this slice layer. However, when quantiser_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used preferentially.

  intra_slice_flag is a flag indicating whether or not intra_slice and reserved_bits exist in the bitstream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_slice is “0”, and if all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. Become. reserved_bits is 7-bit data and takes a value of “0”. extra_bit_slice is a flag indicating that additional information exists as a history stream, and is set to “1” when extra_information_slice exists next. If there is no additional information, it is set to “0”.

  Next to these data elements, data elements defined by the macroblock () function are described as a history stream.

  As shown in FIG. 38, the macroblock () function includes data elements such as macroblock_escape, macroblock_address_increment, macroblock_quantiser_scale_code, and marker_bit, and data elements defined by the macroblock_modes () function, motion_vectors (s) function, and code_block_pattern () function. Is a function for describing

  macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, 33 is added to the value of macroblock_address_increment. The macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If there is one macroblock_escape before this macroblock_address_increment, the value obtained by adding 33 to the value of the macroblock_address_increment is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock. .

  The macroblock_quantiser_scale_code is data indicating the quantization step size set for each macroblock, and exists only when the macroblock_quant is “1”. In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set. When macroblock_quantiser_scale_code is set for the reference macroblock, this quantization step size is selected.

  Next to macroblock_address_increment, a data element defined by a macroblock_modes () function is described. As shown in FIG. 39, the macroblock_modes () function is a function for describing data elements such as macroblock_type, frame_motion_type, field_motion_type, and dct_type as a history stream.

  macroblock_type is data indicating the coding type of the macroblock. Details thereof will be described later with reference to FIGS.

  If macroblock_motion_forward or macroblock_motion_backward is “1”, the picture structure is a frame, and frame_pred_frame_dct is “0”, a data element representing frame_motion_type is described next to the data element representing macroblock_type. This frame_pred_frame_dct is a flag indicating whether or not frame_motion_type exists in the bitstream.

  frame_motion_type is a 2-bit code indicating the prediction type of the macroblock of the frame. If the number of prediction vectors is two and the field-based prediction type is “00”, if the number of prediction vectors is one and the field-based prediction type is “01”, the number of prediction vectors is one and the frame base The prediction type is “10”, and if the prediction type is one and the prime prediction type is “11”.

  If the condition describing the frame_motion_type is not satisfied, the data element representing the field_motion_type is described next to the data element representing the macroblock_type.

  field_motion_type is a 2-bit code indicating motion prediction of a macroblock in a field. If the prediction vector is one and the field-based prediction type is “01”, if the prediction vector is two and the 18 × 8 macroblock-based prediction type is “10”, the prediction vector is 1 It is “11” if the prediction type is individual and prime prime.

  If the picture structure is a frame, frame_pred_frame_dct indicates that frame_motion_type is present in the bitstream, and frame_pred_frame_dct indicates that dct_type is present in the bitstream, then the data element representing the macroblock_type Describes a data element representing dct_type. Dct_type is data indicating whether the DCT is a frame DCT mode or a field DCT mode.

  Returning again to FIG. 38, if the reference macroblock is a forward prediction macroblock or the reference macroblock is an intra macroblock and is a concealing macroblock, motion_vectors (0) A data element defined by a function is described. If the reference macroblock is a backward prediction macroblock, a data element defined by the motion_vectors (1) function is described. The motion_vectors (0) function is a function for describing the data elements related to the first motion vector, and the motion_vectors (1) function is a function for describing the data elements related to the second motion vector. It is.

  The motion_vectors (s) function is a function for describing data elements relating to motion vectors, as shown in FIG.

  If there is one motion vector and the dial prime prediction mode is not used, a data element defined by motion_vertical_field_select [0] [s] and motion_vector (0, s) is described.

  In this motion_vertical_field_select [r] [s], the first motion vector (which may be either forward or backward vector) is a vector created by referring to the bottom field, or refers to the top field It is a flag indicating whether the vector is created as a result. The index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is the prediction direction of forward or backward prediction. It is an indicator that shows.

  As shown in FIG. 41, the motion_vector (r, s) function includes a data string related to motion_code [r] [s] [t], a data string related to motion_residual [r] [s] [t], and dmvector [t ] Is a function for describing the data representing [].

  motion_code [r] [s] [t] is variable-length data representing the magnitude of the motion vector in the range of −16 to +16. motion_residual [r] [s] [t] is variable-length data representing the residual of the motion vector. Therefore, a detailed motion vector can be described by the values of motion_code [r] [s] [t] and motion_residual [r] [s] [t]. dmvector [t] is an existing motion depending on the time distance to generate a motion vector in one field (for example, the top field is one field with respect to the bottom field) in the dial prime prediction mode. This is data for correcting the vertical direction in order to reflect the vertical shift between the lines of the top field and the bottom field as the vector is scaled. The index “r” is an index indicating whether the vector is the first vector or the second vector, and “s” is the prediction direction of forward or backward prediction. It is an indicator that shows. “S” is data indicating whether the motion vector is a vertical component or a horizontal component.

  With the motion_vector (r, s) function shown in FIG. 41, first, a data string representing horizontal motion_coder [r] [s] [0] is described as a history stream. The number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], so f_code [s] [t] is 1 If not, it indicates that motion_residual [r] [s] [t] is present in the bitstream. The motion_residual [r] [s] [0] of the horizontal component is not “1” and the motion_code [r] [s] [0] of the horizontal component is not “0”. This means that there is a data element representing motion_residual [r] [s] [0] and there is a horizontal component of the motion vector. In this case, motion_residual [r] [ A data element representing s] [0] is described.

  Subsequently, a data string representing vertical motion_coder [r] [s] [1] is described as a history stream. Similarly, the number of bits of both motion_residual [0] [s] [t] and motion_residual [1] [s] [t] is indicated by f_code [s] [t], so f_code [s] [t] Is not 1, it indicates that motion_residual [r] [s] [t] is present in the bitstream. motion_residual [r] [s] [1] is not “1” and motion_code [r] [s] [1] is not “0”. This means that motion_residual [r] [s] [1] [ This means that there is a data element representing 1] and that there is a vertical component of the motion vector. In this case, data representing motion_residual [r] [s] [1] of the vertical component The element is described.

  Next, macroblock_type will be described with reference to FIGS. The macroblock_type is variable length data generated from flags such as macroblock_quant, dct_type_flag, macroblock_motion_forward, and macroblock_motion_backward. macroblock_quant is a flag indicating whether or not macroblock_quantiser_scale_code for setting the quantization step size for the macroblock is set. When macroblock_quantiser_scale_code exists in the bitstream, macroblock_quant is a value of “1”. I take the.

  dct_type_flag is a flag for indicating whether or not dct_type indicating whether the reference macroblock is encoded by the frame DCT or the field DCT exists (in other words, a flag indicating whether or not the DCT is DCT), and is a bit. When dct_type exists in the stream, this dct_type_flag takes a value of “1”. The macroblock_motion_forward is a flag indicating whether or not the reference macroblock is predicted forward, and takes a value of “1” when the reference macroblock is predicted forward. macroblock_motion_backward is a flag indicating whether or not the reference macroblock is predicted backward, and takes a value of “1” when backward prediction is performed.

  In the variable length format, history information can be reduced in order to reduce the transmission bit rate.

  That is, when macroblock_type and motion_vectors () are transferred but quantizer_scale_code is not transferred, the bit rate can be reduced by setting slice_quantiser_scale_code to “00000”.

  Further, when only macroblock_type is transferred and motion_vectors (), quantizer_scale_code, and dct_type are not transferred, the bit rate can be reduced by using “not coded” as macroblock_type.

  Furthermore, when only picture_coding_type is transferred and all information below slice () is not transferred, the bit rate can be reduced by using picture_data () without slice_start_code.

  In the above, “1” is inserted every 22 bits in order to prevent the continuous “0” of 23 bits in user_data from being output. However, it may not be every 22 bits. Further, instead of counting the number of consecutive “0” s and inserting “1”, it is possible to check and insert Byte_allign.

  Furthermore, in MPEG, the generation of 23 bits of continuous “0” is prohibited, but in practice, only the case where 23 bits are continued from the beginning of the byte is a problem. If 0 continues for 23 bits, this is not a problem. Therefore, for example, “1” may be inserted at a position other than the LSB every 24 bits.

  In the above description, the history information is in a format close to a video elementary stream, but may be in a format close to a packetized elementary stream or a transport stream. In addition, although the location of user_data in Elementary Stream is in front of picture_data, it can be other locations.

  In the transcoding system of FIG. 4, encoding parameters for four generations are output to the subsequent stage as history information. However, not all of the history information is actually required, and it is necessary for each application. History information will be different. In addition, the actual transmission path or recording medium (transmission medium) has a limited capacity, and although it is compressed, if all history information is transmitted, it will be a burden on capacity, and as a result The bit rate of the image bit stream is suppressed, and the effectiveness of history information transmission is impaired.

  Therefore, a descriptor describing a combination of items to be transmitted as history information is incorporated into the history information and transmitted to the subsequent stage so that information corresponding to various applications is transmitted instead of transmitting all history information. Can be. FIG. 45 shows a configuration example of the transcoding system in such a case.

  In FIG. 45, portions corresponding to those in FIG. 4 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. In the configuration example of FIG. 45, an encoding parameter selection circuit 151 is inserted between the history information separation device 64 and the encoding device 66 and between the history encoding device 65 and the encoding device 66.

  The encoding parameter selection circuit 151 includes an encoding parameter calculation unit 162 that calculates an encoding parameter from the baseband video signal output from the history information separation device 64, and an encoding code in the video encoding system 141 output from the history information separation device 64. Coding parameters and descriptors (red_bw_flag, red_bw_indicator) (described later with reference to FIG. 49) are separated from information on coding parameters determined to be optimal (for example, second generation coding parameters) The combination descriptor separation unit 161 selects one of the encoding parameter output from the combination descriptor separation unit 161 and the encoding parameter calculation unit 162 and the encoding parameter output from the combination descriptor separation unit 161. Select and encode according to the separated descriptor A switch 163 to be output to 66. Other configurations are the same as those in FIG.

  Here, combinations of items to be transmitted as history information will be described. The history information can be divided into information in picture units and information in macroblock units. Information in units of slices can be obtained by collecting information on macroblocks included therein, and information in units of GOPs can be obtained by collecting information in units of pictures included therein.

  Since information in picture units (that is, information described in the V Blanking area in FIG. 7) is transmitted only once per frame, the bit rate occupied in information transmission is not so large. On the other hand, since information in macroblock units (that is, information described in the H Blanking area in FIG. 7) is transmitted for each macroblock, for example, the number of scanning lines in one frame is 525, and the field rate is In the case of a video system of 60 fields / second, if the number of pixels in one frame is 720 × 480, information in macroblock units is transmitted 1350 (= (720/16) × (480/16)) times per frame. Is required. For this reason, a considerable part of the history information is occupied by information for each macroblock. Therefore, as history information, at least information in units of pictures is always transmitted, but information in units of macroblocks can be selected and transmitted according to the application, thereby suppressing the amount of information to be transmitted.

  The macroblock unit information transferred as history information includes, for example, num_coef_bits, num_mv_bits, num_other_bits, q_scale_code, q_scale_type, motion_type, mv_vert_field_sel [] [], mv [] [] [], mb_mfwd, mb_mbwd, mb_pattern, _coded_block_pattern, There are slice_start, dct_type, mb_quant, skipped_mb, etc. These are expressed using the element of macroblock rate information.

  num_coef_bits represents the code amount required for the DCT coefficient among the code amounts of the macroblock. num_mv_bits represents the code amount required for the motion vector among the code amounts of the macroblock. num_other_bits represents a code amount other than num_coef_bits and num_mv_bits among the code amounts of the macroblock.

  q_scale_code represents q_scale_code applied to the macroblock. motion_type represents the type of motion vector applied to the macroblock. mv_vert_field_sel [] [] represents a field select of a motion vector applied to a macroblock.

  mv [] [] [] represents a motion vector applied to a macroblock. mb_mfwd is a flag indicating that the prediction mode of the macroblock is forward prediction. mb_mbwd is a flag indicating that the prediction mode of the macroblock is backward prediction. The mb_pattern is a flag indicating whether or not there is a non-zero DCT coefficient of the macroblock.

  The coded_block_pattern is a flag indicating the presence or absence of non-zero macroblock DCT coefficients for each DCT block. mb_intra is a flag indicating whether the macroblock is intra_macro or not. slice_start is a flag indicating whether or not the macroblock is the head of the slice. dct_type is a flag indicating whether the macroblock is field_dct or flame_dct.

  mb_quant is a flag indicating whether or not the macroblock transmits quantizer_scale_code. skipped_mb is a flag indicating whether or not the macroblock is a skipped macroblock.

  All of these items are not always necessary, and the necessary items vary depending on the application. For example, items such as num_coef_bits and slice_start are necessary in an application having a request for transparent to restore the bitstream at the time of re-encoding as much as possible. In other words, these items are not necessary in an application that changes the bit rate. In addition, there are applications in which it is only necessary to know the coding type of each picture when the transmission path is extremely limited. From such a situation, for example, a combination shown in FIG. 46 can be considered as an example of a combination of items for transmitting history information.

  In FIG. 46, the value “2” corresponding to the item in each combination means that the information exists and can be used, and “0” means that the information does not exist. “1” indicates that the information itself has no meaning, for example, for the purpose of assisting the existence of other information, or syntactically, but not related to the original bitstream information. . For example, slice_start is “1” in the first macroblock of the slice when transmitting history information, but if the slice is not necessarily in the same positional relationship with the original bitstream, It becomes meaningless as information.

  In the example of FIG. 46, (num_coef_bits, num_mv_bits, num_other_bits), (q_scale_code, q_scale_type), (motion_type, mv_vert_field_sel [] [], mv [] [] []), (mb_mfwd, mb_mbwd), (mb_pattern), (mb_pattern), Depending on the presence or absence of each item of coded_block_pattern), (mb_intra), (slice_start), (dct_type), (mb_quant), and (skipped_mb), five combinations of combinations 1 to 5 are prepared.

  Combination 1 is a combination intended to reconstruct a completely transparent bit stream. According to this combination, highly accurate transcoding can be realized by using the generated code amount information. Combination 2 is also a combination intended to reconstruct a completely transparent bit stream. The combination 3 is a combination for allowing a completely transparent bit stream to be visually reconstructed although a completely transparent bit stream cannot be reconstructed. Combination 4 is inferior to combination 3 from the viewpoint of transparent, but is a combination that can reconstruct a bitstream with no visual problem. The combination 5 is inferior to the combination 4 from the viewpoint of transparent, but is a combination capable of incomplete reconstruction of the bitstream with a small amount of history information.

  Of these combinations, the smaller the combination number, the higher the function, but the larger the capacity required to transfer the history. Therefore, it is necessary to determine the combination to be transmitted by considering the assumed application and the capacity available for the history.

  Next, the operation of the transcoding system of FIG. 45 will be described with reference to the flowchart of FIG.

  In step S1, the decoding device 61 of the video decoding system 51 decodes the input bit stream, extracts the encoding parameter (fourth generation) used when encoding the bit stream, and extracts the code And the decoded video data are also output to the history information multiplexer 63. In step S <b> 2, the decoding device 61 extracts user_data from the input bitstream and outputs the user_data to the history decoding device 62. In step S3, the history decoding device 62 extracts combination information (descriptor) from the input user_data, and further uses it to extract encoding parameters (second generation to third generation) as history information. To the history information multiplexer 63.

  In step S4, the history information multiplexing device 63 uses the current encoding parameter (fourth generation) supplied from the decoding device 51 extracted in step S1 and the past output from the history decoding device 62 in step S3. The encoding parameters (second generation to third generation) are multiplexed with the baseband video data supplied from the decoding device 61 according to the format shown in FIGS. 7 and 11, and output to the history information separation device 64. To do.

  In step S5, the history information separating device 64 extracts coding parameters from the baseband video data supplied from the history information multiplexing device 63, and among them, the coding parameters most suitable for the current coding are extracted. (For example, the second generation encoding parameter) is selected and output to the combination descriptor separating unit 161 together with the descriptor. In addition, the history information separation device 64 determines an encoding parameter other than the encoding parameter determined to be optimal for the current encoding (for example, when the optimal encoding parameter is determined to be the second generation encoding parameter). Are output to the history encoding device 65 (the other first generation, third generation, and fourth generation encoding parameters).

  In step S <b> 6, the history encoding device 65 describes the encoding parameters input from the history information separation device 54 in user_data, and outputs the user_data (converted_history_stream ()) to the encoding device 66.

  In step S 7, the combination descriptor separation unit 161 separates the encoding parameter and the descriptor from the data supplied from the history information separation device 64, and sets the encoding parameter (second generation) as one contact point of the switch 163. To supply. The encoding parameter calculation unit 162 calculates and supplies encoding parameters from the baseband video data output by the history information separation device 64 to the other contact of the switch 163.

  In step S8, the switch 163 corresponds to the descriptor output from the combination descriptor separation unit 161, or the encoding parameter output from the combination descriptor separation unit 161 or the encoding parameter output from the coding parameter calculation unit 162. Is selected and output to the encoding device 66. That is, in the switch 163, when the encoding parameter supplied from the combination descriptor separation unit 161 is valid, the encoding parameter output from the combination descriptor separation unit 161 is selected. When the encoding parameter output from 161 is invalid, the encoding parameter calculated by the encoding parameter calculation unit 162 processing the baseband video is selected. This selection is performed according to the capacity of the transmission medium.

  In step S <b> 9, the encoding device 66 encodes the baseband video signal supplied from the history information separation device 105 based on the encoding parameter supplied from the switch 163.

  In step S10, the encoding device 66 multiplexes and outputs the user_data supplied from the history encoding device 65 to the encoded bit stream, and the process ends.

  In this way, even when the combination of encoding parameters obtained by each history is different, transcoding can be performed without any trouble.

  As described above, the history information is transmitted by history_stream () (more precisely, converted_history_stream ()) as a kind of user_data () function of the video stream, as shown in FIG. The syntax of the history_stream () is as shown in FIG. A descriptor (red_bw_flag, red_bw_indicator) representing a combination of items of history information and items (num_other_bits, num_mv_bits, num_coef_bits) not supported in the MPEG stream are transmitted by the re_coding_stream_info () function in FIG.

  As shown in FIG. 48, the re_coding_stream_info () function includes data elements such as user_data_start_code, re_coding_stream_info_ID, red_bw_flag, red_bw_indicator, marker_bit, num_other_bits, num_mv_bits, and num_coef_bits.

  user_data_start_code is a start code indicating that user_data starts. The re_coding_stream_info_ID is a 16-bit integer and is used for identifying the re_coding_stream_info () function. Specifically, the value is “1001 0001 1110 1100” (0x91ec).

  The red_bw_flag is a 1-bit flag, and is set to 0 when the history information transmits all items. When the value of this flag is 1, the red_bw_indicator following this flag is checked, as shown in FIG. Of the five combinations, it is possible to determine which combination is sending the item.

  red_bw_indicator is a 2-bit integer and describes a combination of items as shown in FIG.

  That is, among the five combinations shown in FIG. 46, red_bw_flag is 0 in the case of combination 1, and red_bw_flag is 1 in the case of combinations 2 to 5. On the other hand, red_bw_indicator is 0 for combination 2, 1 for combination 3, 2 for combination 4, and 3 for combination 5.

  Therefore, red_bw_indicator is defined when red_bw_flag is 1 (in the case of combination 2 to combination 5).

  Further, in the case of the combination 1 in FIG. 49 (when red_bw_flag is 0), marker_bit, num_other_bits, num_mv_bits, and num_coef_bits are described for each macroblock. These four data elements are not defined for combinations 2 to 5 (when red_bw_flag is 1).

  As shown in FIG. 36, the picture_data () function is composed of one or more slice () functions. However, in the case of the combination 5, the syntax elements below it including the picture_data () function are not transmitted. In this case, the history information is intended to transmit information in units of picture such as picture_type.

  In the case of the combination 1 to the combination 4, the slice () function shown in FIG. 37 exists. However, the position information of the slice determined by the slice () function and the position information of the slice of the original bitstream depend on the combination of items of history information. In the case of the combination 1 or the combination 2, the position information of the slice of the bit stream that is the source of the history information and the position information of the slice determined by the slice () function need to be the same.

  The syntax element of the macroblock () function shown in FIG. 38 depends on the combination of history information items. The macroblock_escape, macroblock_address_increment, and macroblock_modes () functions are always present. However, the effectiveness of macroblock_escape and macroblock_address_increment as information is determined by the combination. When the combination of history information items is combination 1 or combination 2, the same information as the skipped_mb information of the original bitstream needs to be transmitted.

  In the case of the combination 4, there is no motion_vectors () function. In the case of the combination 1 to the combination 3, the presence of the motion_vectors () function is determined by the macroblock_type of the macroblock_modes () function. In the case of combination 3 or combination 4, there is no coded_block_pattern () function. In the case of the combination 1 and the combination 2, the presence of the coded_block_pattern () function is determined by the macroblock_type of the macroblock_modes () function.

  The syntax elements of the macroblock_modes () function shown in FIG. 39 depend on the combination of history information items. macroblock_type is always present. When the combination is combination 4, flame_motion_type, field_motion_type, and dct_type do not exist.

  The effectiveness of the parameter obtained from macroblock_type as information is determined by a combination of items of history information.

  When the combination of history information items is combination 1 or combination 2, macroblock_quant needs to be the same as the original bitstream. For combination 3 or combination 4, macroblock_quant represents the presence of quantizer_scale_code in the macroblock () function and need not be the same as the original bitstream.

  When the combination is combination 1 to combination 3, macroblock_motion_forward and macroblock_motion_backward need to be the same as the original bitstream. If the combination is combination 4 or combination 5, this is not necessary.

  When the combination is combination 1 or combination 2, macroblock_pattern needs to be the same as the original bitstream. For combination 3, macroblock_pattern is used to indicate the presence of dct_type. When the combination is combination 4, the relationship as in combination 1 to combination 3 is not established.

  When the combination of history information items is combination 1 to combination 3, macroblock_intra needs to be the same as the original bitstream. In the case of combination 4, this is not the case.

  The history_stream () in FIG. 24 has a syntax when the history information has a variable length. However, as shown in FIGS. 17 to 23, when the history information has a fixed length, Descriptors (red_bw_flag and red_bw_indicator) as information indicating which of the items to be transmitted are valid are superimposed on the baseband image and transmitted. As a result, by examining this descriptor, it is possible to determine that it exists as a field but its contents are invalid.

  For this reason, as shown in FIG. 21, user_data_start_code, re_coding_stream_info_ID, red_bw_flag, red_bw_indicator, and marker_bit are arranged as re_coding_stream_information. Each meaning is the same as in FIG.

  Thus, by transmitting the elements of the encoding parameter transmitted as the history in a combination according to the application, it is possible to transmit the history according to the application with an appropriate amount of data.

  As described above, when history information is transmitted as a variable length code, the re_coding_stream_info () function is configured as shown in FIG. 48 and is transmitted as part of the history_stream () function as shown in FIG. On the other hand, when history information is transmitted as a fixed-length code, re_coding_stream_information () is transmitted as part of the history_stream () function, as shown in FIG. In the example of FIG. 21, user_data_start_code, re_coding_stream_info_ID, red_bw_flag, and red_bw_indicator are transmitted as re_coding_stream_information.

  In addition, a Re_Coding information Bus macroblock format as shown in FIG. 50 is defined for transmission of history information in a baseband signal output from the history information multiplexing apparatus 63 in FIG. This macro block is composed of 16 × 16 (= 256) bits. Then, 32 bits shown in the third and fourth rows from the top in FIG. 50 are picrate_element. In this picrate_element, picture rate elements shown in FIGS. 51 to 53 are described. In the second line from the top of FIG. 51, 1-bit red_bw_flag is defined, and in the third line, 3-bit red_bw_indicator is defined. That is, these flags red_bw_flag and red_bw_indicator are transmitted as picrate_element in FIG.

  The other data in FIG. 50 will be described. SRIB_sync_code is a code indicating that the first row of the macro block of this format is aligned left-justified, and is specifically set to “11111”. fr_fl_SRIB is set to 1 when picture_structure has a frame picture structure (when its value is “11”), indicating that Re_Coding Information Bus macroblock is transmitted over 16 lines, and picture_structure is not a frame structure In this case, it is set to 0, which means that Re_Coding Information Bus is transmitted over 16 lines. This mechanism locks the Re_Coding Information Bus to the corresponding pixel of the video frame or field decoded spatially and temporally.

  SRIB_top_field_first is set to the same value as top_field_first held in the original bitstream, and represents the temporal alignment of Re_Coding Information Bus of the related video together with repeat_first_field. SRIB_repeat_first_field is set to the same value as repeat_first_field held in the original bitstream. The content of Re_Coding Information Bus in the first field needs to be repeated as indicated by this flag.

  422_420_chroma represents whether the original bit stream is 4: 2: 2 or 4: 2: 0. The value of 0 indicates that the bitstream is 4: 2: 0 and that the upsampling of the color difference signal is performed so that 4: 2: 2 video is output. The value 0 indicates that the color difference signal filtering process is not executed.

  rolling_SRIB_mb_ref represents a 16-bit modulo 65521, and this value is incremented for each macroblock. This value must be continuous across frames of the frame picture structure. Otherwise, this value must be continuous across the field. This value is initialized to a predetermined value between 0 and 65520. This allows the incorporation of a unique Re_Coding Information Bus identifier into the recorder system.

  The meaning of the other data of the Re_Coding Information Bus macroblock is as described above, and is omitted here.

  As shown in FIG. 54, the 256-bit Re_Coding Information Bus data in FIG. 50 is Cb [0] [0], Cr [0] [0], Cb [1] which are LSBs of color difference data bit by bit. [0], Cr [1] [0]. Since the 4-bit data can be sent with the format shown in FIG. 54, the 256-bit data in FIG. 50 can be transmitted by sending 64 (= 256/4) formats in FIG.

  In this way, in the transcoding system of the present invention that can reduce the accumulation of image quality deterioration due to repetition of decoding processing and encoding processing, the encoding parameters generated in the past encoding processing are used as information usage frequency. Accordingly, the information is described in the V Blanking area and the H Blanking area, so that it is possible to easily extract information as necessary. Further, it is possible to select hardware as needed according to information desired to be used (or to delete hardware as necessary).

  Furthermore, the transcoding system of the present invention is a video decoding system in which the synchronization code (start_code) is always written at the head of the packet, so that the video encoding system to which the baseband video signal is input is described. Can easily perform recovery when an input packet is missing.

  In the above description, the example in which the encoding parameter is compressed and transmitted has been described. However, the present invention can be applied even if the encoding parameter is transmitted without being compressed.

  The computer program for performing each of the above processes is provided by being recorded on a recording medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. It can be provided by recording on a medium.

  Further, in this specification, the steps for describing the program provided by the medium may be performed in parallel or individually even if not necessarily performed in time series, as well as processes performed in time series in the order described. This includes the processing to be executed.

In the present specification, the term “system” represents the entire apparatus constituted by a plurality of apparatuses.

It is a block diagram which shows the structure of the transcoding system in this invention. It is a block diagram which shows the detailed structure of the video decoding system of FIG. It is a block diagram which shows the detailed structure of the video encoding system of FIG. It is a block diagram which shows the other structure of a transcoding system. It is a block diagram which shows the detailed structure of the decoder incorporated in the decoding apparatus of FIG. It is a figure for demonstrating the pixel of a macroblock. It is a figure for demonstrating V Blanking and H Blanking. It is a figure for demonstrating the position of start_cord. It is a figure for demonstrating a field structure and a frame structure. It is a figure for demonstrating the position of start_cord. It is a figure explaining the area | region where an encoding parameter is recorded. It is a block diagram which shows the detailed structure of the encoder incorporated in the encoding apparatus of FIG. FIG. 5 is a diagram illustrating a state in which the transcoding system of FIG. 4 is actually used. It is a block diagram which shows the structure of the tightly coupled transcoding system. It is a figure explaining the syntax of the stream of a video sequence. It is a figure explaining the structure of the syntax of FIG. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records history information of fixed length. It is a figure explaining the syntax of history_stream () which records variable length history information. It is a figure explaining the syntax of sequence_header (). It is a figure explaining the syntax of sequence_extension (). It is a figure explaining the syntax of extension_and_user_data (). It is a figure explaining the syntax of user_data (). It is a figure explaining the syntax of group_of_pictures_header (). It is a figure explaining the syntax of picture_header (). It is a figure explaining the syntax of picture_coding_extension (). It is a figure explaining the syntax of extension_data (). It is a figure explaining the syntax of quant_matrix_extension (). It is a figure explaining the syntax of copyright_extension (). It is a figure explaining the syntax of picture_display_extension (). It is a figure explaining the syntax of picture_data (). It is a figure explaining the syntax of slice (). It is a figure explaining the syntax of macroblock (). It is a figure explaining the syntax of macroblock_modes (). It is a figure explaining the syntax of motion_vectors (s). It is a figure explaining the syntax of motion_vector (r, s). It is a figure explaining the variable length code of macroblock_type with respect to I picture. It is a figure explaining the variable length code of macroblock_type with respect to P picture. It is a figure explaining the variable length code of macroblock_type with respect to B picture. It is a block diagram which shows the other structure of a transcoding system. It is a figure explaining the combination of the item of history information. It is a flowchart explaining operation | movement of the transcoding system of FIG. It is a figure explaining the syntax of re_coding_stream_info (). It is a figure explaining red_bw_flag and red_bw_indicator. It is a figure explaining Re_Coding Information Bus macroblock formation. It is a figure explaining Picture rate elements. It is a figure explaining Picture rate elements. It is a figure explaining Picture rate elements. It is a figure explaining the area | region where Re_Coding Information Bus is recorded.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Video decoding system, 2 Video encoding system, 11 Decoding apparatus, 12 History information multiplexing apparatus, 13 Data classification apparatus, 14 History decoding apparatus, 15 Format converter, 16 V Blanking insertion apparatus, 17 Format converter, 18 H Blanking Insertion device, 21 User data decoder, 22 Converter, 23 History VLD, 31 H Blanking extraction device, 32 Format converter 33 Multiplexer, 34 V Blanking extraction device, 35 Format converter, 36 Parameter playback unit, 37 History information separation device, 38 History encoding device, 39 encoding device, 41 user data formatter, 42 converter, 43 history VLC, 51 video Odecoding system, 52 video encoding system, 61 decoding device, 62 history decoding device, 63 history information multiplexing device, 64 history information separating device, 65 history encoding device, 66 encoding device, 71 user data decoder, 72 converter, 73 history VLD, 64 history information separation device, 65 history encoding device, 66 encoding device, 74 user data formatter, 75 converter, 76 history VLC, 81 decoder, 101 encoder, 151 encoding parameter selection circuit, 161 combination descriptor Separator, 162 Coding parameter calculator, 163 switch

Claims (8)

In an information processing apparatus that inserts an encoding parameter into image data obtained by decoding an encoded stream,
Obtaining means for obtaining an encoding parameter of the encoded stream together with the encoded stream;
In order that the encoding parameter of the picture layer or more and the encoding parameter of the slice layer or less included in the encoding parameter acquired by the acquisition unit are selectively used in the re-encoding process of the encoded stream, Whether the encoding parameter of the picture layer or higher is inserted into the vertical blanking region of the image data for transmission or whether the encoding parameter of the slice layer or lower is inserted into the horizontal blanking region of the image data for transmission An information processing apparatus comprising: a selectively performing transmission means. The information processing apparatus according to claim 1, further comprising decoding means for decoding the encoded stream to generate the image data. The information processing apparatus according to claim 1, wherein the encoding parameter includes a current encoding parameter used when generating the encoded stream. The information processing apparatus according to claim 1, wherein the encoding parameter includes a history encoding parameter used in a past encoding process or decoding process for the encoded stream. The encoding parameters are multiplexed into the encoded stream;
The information processing apparatus according to claim 1, wherein the acquisition unit acquires the encoding parameter from the encoded stream. The information processing apparatus according to claim 1, wherein the encoded stream is encoded according to an MPEG standard. The information processing apparatus according to claim 1, wherein the re-encoding process conforms to an MPEG standard. In an information processing method of an information processing apparatus for inserting an encoding parameter into image data obtained by decoding an encoded stream,
Obtaining the encoding parameters of the encoded stream together with the encoded stream;
More than the picture layer so that the encoding parameter of the picture layer or higher and the encoding parameter of the slice layer or lower included in the acquired encoding parameter are selectively used in the re-encoding process of the encoded stream. Is selectively inserted into the vertical blanking region of the image data for transmission, or the encoding parameter below the slice layer is inserted into the horizontal blanking region of the image data for transmission. An information processing method including executing steps.

Images