JP2006109513A - Encoding device and method, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To quickly process image data both in a predetermined studio and in another studio. <P>SOLUTION: An MPEG video encoder 42A superimposes, on the elementary stream of encoded video data, Picture Order Information representing an encoding order and display order of a picture and transmits the elementary stream to an SDTI-CP network within the same studio via an SDTI-CP interface 46. When the elementary stream is received via SDTI-CP interfaces 46, 51, a TS MUX/DEMUX 61 extracts the Picture Order Information contained therein, generates a time stamp on the basis of the extracted information, superimposes the time stamp on a transport stream and transmits it to the other studio. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a digital signal transmission apparatus and method, and a providing medium, and in particular, when an image is transmitted and received between different studios and processed, the amount of delay in the entire system of the image signal to be transmitted and received is minimized. The present invention relates to a digital signal transmission apparatus and method, and a providing medium.

  When an image signal or an audio signal is encoded and transmitted, an encoding method shown in ISO / IEC11172 (MPEG-1) or ISO / IEC13818 (MPEG-2) is often used. As a technique for encoding an image signal by the MPEG (Moving Picture Experts Group) method, there are a coding phase and bidirectional prediction.

  As shown in FIG. 1, the coding phase is a code for defining an effective pixel area 2 as a range to be encoded in the pixel area 1 of one image. The first line in the effective pixel area 2 (the line indicated by the arrow of the V-phase [line] in FIG. 1) and the first line in the effective pixel area 2 among all the samples (pixels). It shows the first sample of the line (sample (pixel) indicated by the arrow of H-phase [sample] in FIG. 1). When the last line of the pixel area 1 is V-Phase [Lmax] and the last sample (pixel) of the line of the pixel area 1 is H-Phase [Smax], the encoding is performed, for example, in the vertical direction. , V-Phase [line] to (V-Phase [Lmax]-V-Phase [line] +1). Horizontally, from H-Phase [sample] to (H-Phase [ Smax] -H-Phase [sample] +1).

  In addition, in a general image signal of, for example, an NTSC television receiver, the frame of the image signal in FIG. 1 is composed of two fields (field 1 and field 2 in FIG. 2) as shown in FIG. It is configured. In field 1, for example, odd line data is displayed, and in field 2, even line data is displayed. Each of the two fields is composed of auxiliary data (ancillary data) and image data.

  The auxiliary data is used for teletext data for teletext for a hearing impaired person, time code, or closed caption data of a caption such as a movie, and is used for a blanking interval (for example, the tenth line of each field to 22nd line) (in FIG. 2, each field is inserted in a predetermined number from the top of the field). In general, a line portion (actually displayed as an image) located below the 23rd line of each field in the drawing is encoded as image data by the MPEG method or the like.

  The above-described MPEG encoding method is applied only to image data, and the coding phase and auxiliary data are not clearly described (defined) in the MPEG standard. Therefore, the coding phase has a degree of freedom and varies depending on various applications.

  A system configuration when encoding or decoding such an image signal will be described with reference to FIG. The image data of application A is encoded by the MPEG encoder 11 for application A, output as an elementary stream (ES), and decoded by the MPEG decoder 12 for application A. The application B image data is encoded by the application B MPEG encoder 13, output as an elementary stream (ES), and decoded by the application B MPEG decoder 14.

  In other words, image data of a certain application is encoded by an encoder dedicated to the application (for example, the MPEG encoder 11 for application A), and as an elementary stream (ES), a dedicated decoder (for example, the application A corresponding to the application). MPEG decoder 12). The elementary stream (ES) input to the dedicated decoder is decoded based on the information regarding the coding phase of the application that the decoder has.

  Next, an auxiliary data transmission method that is currently mainstream will be described with reference to FIG. The image signal is input to the MPEG encoder 21 and is separated into image data and auxiliary data by the separation unit 21. The MPEG encoding unit 23 performs MPEG encoding on the image data. The auxiliary data is inserted by the variable length coding unit 24 into user data in the MPEG transport stream output from the MPEG encoding unit 23. Since user data in this transport stream can be inserted (description) in units of encoded image data (pictures), auxiliary data is inserted for each user data of a corresponding frame of encoded image data.

  A transport stream including user data in which auxiliary data is inserted is transmitted via a predetermined transmission path and input to the MPEG decoder 25. A variable length decoding unit 26 in the MPEG decoder 25 separates auxiliary data and image data. The image data is decoded by the MPEG decoding unit 27, synthesized with the auxiliary data by the synthesis unit 28, and output as an image signal to a display device (not shown).

  Next, MPEG-based prediction will be described. A picture of encoded image data generated based on bidirectional prediction is called a B picture. The B picture is predicted and generated from two pieces of reference image data positioned before or after in time. A picture of encoded image data generated based on forward prediction is referred to as a P picture. The P picture is predicted and generated from one piece of reference image data positioned in front of time. A picture of encoded image data in which image data is encoded (intra-encoded) without being predicted is called an I picture. That is, the input image data is encoded into encoded image data of any one of B picture, P picture, and I picture.

  Bidirectional prediction and forward prediction will be described with reference to FIG. In the example of FIG. 5, one GOP (Group of Picture) is composed of nine pictures. The upper part of FIG. 5 represents a prediction structure indicating the direction (dependency) of prediction when encoded image data is generated, and the lower part of FIG. 5 shows the order in which the image data is actually encoded. The encoding structure shown is shown. Since a B picture or P picture is predicted and generated from temporally previous or subsequent image data, it is not possible to perform coding only with a B picture or P picture. In other words, since the B picture or P picture is encoded image data that uses the difference from the reference image data as the data, the image data cannot be decoded only by the B picture or P picture.

  The prediction dependency will be described in detail. For example, in GOP (N-1), the first B picture in the display order is the third I picture from the top, and GOP (N-2) (GOP not shown). (GOP immediately before (N-1)) is predicted and encoded from the last P picture. Similarly, the second B picture from the beginning is predicted and encoded from the third I picture from the beginning and the last P picture of GOP (N-2) not shown. The third I picture from the beginning is encoded as it is (intra coding). The fourth and fifth B pictures from the beginning are predicted and encoded from the third I picture from the beginning and the sixth P picture from the beginning. The sixth P picture from the beginning is predicted and encoded from the third I picture from the beginning.

  That is, in the prediction structure, in order to encode a B picture (for example, the first B picture and the second B picture from the beginning), prediction reference image data (for example, the third I picture and GOP (N− The last P picture in 2) needs to be encoded first. That is, encoding must be performed in the order of the encoding structure as shown in the lower part of FIG. For this reason, the MPEG encoder encodes (encodes) reference image data (third I picture from the beginning) necessary for encoding two consecutive B pictures existing between an I picture and a P picture. ) Is input, it is necessary to buffer two B pictures (delay the start of encoding until an I picture is input). Due to this buffering, when the input image signal is encoded in the MPEG encoder, a delay time of the number of B pictures (= 2) +1 (total 3) sandwiched between reference image data occurs.

  The delay generated in the MPEG encoder will be described in more detail with reference to FIG. The upper part of FIG. 6 shows the input order (display order) of input images (image data) and their types in the MPEG encoder 31, and the middle part of FIG. 6 shows the encoding in which the input image (image data) is encoded. This represents the order of image data.

  The MPEG encoder 31 delays the encoding of the I picture input at time t1 until the P picture at time t4 is input (for the time corresponding to the number of B pictures (= 2) +1) (the aforementioned buffer). Do ring). That is, the MPEG encoder 31 encodes the I picture input at time t1 at time t4, encodes the P picture input at time t4 at time t5, and encodes the B picture input at time t2 at time t6. The B picture input at time t3 is encoded at time t7. Thereafter, the MPEG encoder 31 sequentially encodes the input image in the order of the encoded image data in the middle stage of FIG.

  As described above, the MPEG encoder 31 sequentially encodes the input image from the reference image data necessary for prediction. That is, in the MPEG encoder 31, the input images (image data) are rearranged in the order of the images to be encoded from the order of the input images, and the order in which the input images are encoded as shown in the middle of FIG. Are output to the MPEG decoder 32 as a bit stream.

  Thus, since the order in which the input images are encoded and the order in which they are displayed do not match, in MPEG, a DTS (Decoding Time Stamp) indicating the encoding order and a PTS (Presentation Time Stamp) indicating the display order are included. And inserted into the transport stream. The relationship between the encoding order of the input images and the display order will be further described. Now, assuming that the encoding order is expressed in units of frames, the input images are numbered in the order of encoding as shown in FIG. The encoding order value of the I picture encoded at time t4 is “1”, the encoding order value of the P picture encoded at time t5 is “2”, and the B picture encoded at time t6 is B. The value of the coding order of pictures is “3”. Hereinafter, the encoding order is assigned in the same encoding order. DTS does not represent the encoding order in units of frames, but almost corresponds to this encoding order. The DTS is also used as a decoding order in the MPEG decoder 32 when a bit stream is multiplexed with an audio signal and output.

  The display order is the order in which the image data is decoded and displayed (the same as the order of the input image data). Specifically, the display order of the I picture (encoding order = 1) encoded at time t4 must be displayed when the P picture (encoding order = 2) is encoded at time t5. Therefore, the value is the same as the encoding order of the P picture “2”. The display order of the P picture (encoding order = 2) encoded at time t5 must be displayed when the P picture (encoding order = 5) is encoded at time t8. It becomes the same value “5” as the encoding order. Since the B picture (encoding order = 3) encoded at time t6 must be decoded and displayed immediately after being encoded, the encoding order and the display order are the same, and the value of the display order is “3”. Similarly, the B picture (encoding order = 4) encoded at time t7 is set to display order = encoding order = 4. PTS does not represent the display order in units of frames, but almost corresponds to this display order. The PTS is also used as an output (display) order after decoding in the MPEG decoder 32.

  In the MPEG decoder 32, the first one of the two I pictures or P pictures in which the coding order is continuous is displayed when the second picture is decoded. For example, the first I picture (encoding order = 1, display order = 2) of the encoded image data in the bitstream output from the MPEG encoder 31 and the second P picture (encoding order = 2, In the display order = 5), the encoding order is continuous, and when the second P picture is encoded (at time t5), the first I picture is decoded and displayed.

  As described above, since the MPEG encoder 31 knows the prediction structure of the input image (image data) (the number of B pictures sandwiched between the reference image data), the encoding order (DTS) and the display order are included in the image data. (PTS) can be numbered.

  In a studio such as a broadcasting station, it is considered to transmit an MPEG-encoded bit stream with the system configuration shown in FIG. The MPEG encoder 42, the MPEG encoder 43, the MPEG decoder 44, and the MPEG decoder 45 in the studio 41 are respectively connected to an SDTI-CP network (for example, an SDTI-CP (Serial Data Transfer Interface-Content Package) interface 46 to 49). , A network constituted by coaxial cables) 50. The SDTI-CP network 50 has a transmission speed of 270 Mbps based on SDI (Serial Data Interface), and can transmit an MPEG elementary stream (ES) as it is. Suitable for other networks.

  In the studio 41, for example, the MPEG encoder 42 can transmit an MPEG-encoded elementary stream (ES) to the MPEG decoder 44 and the MPEG decoder 45 via the SDTI-CP network 50.

  The elementary stream (ES) transmitted in the SDTI-CP network 50 has the structure shown in FIG. 8, and image data (the part with a thin shadow in FIG. 8) and audio in frame units of the image signal. Data (the dark shaded portion in FIG. 8) is packed, and editing can be easily performed at frame boundaries delimited by a frame sync (dotted line in FIG. 8). The image data and audio data in the elementary stream (ES) are data subjected to intra (intra-frame coding) processing.

  In the system of the studio 41 shown in FIG. 7, the MPEG encoders 42 and 43 and the MPEG decoders 44 and 45 are connected to the SDTI-CP network 50 via the SDTI-CP interfaces 46 to 49, as shown in FIG. The configuration differs from that of a system in which a dedicated encoder (for example, the application A MPEG encoder 11) and a decoder (for example, the application A MPEG decoder 12) correspond one-to-one for each application. That is, a decoder (for example, MPEG decoder 44 in FIG. 7) is an elementary stream (ES) in which image signals of various applications are encoded by an encoder (for example, MPEG encoder 42 or MPEG encoder 43 in FIG. 7). Can receive.

  The system configuration of FIG. 7 will be described with reference to FIG. 9 corresponding to the system configuration of FIG. 3. The elementary stream (ES) encoded by the MPEG encoder for application A 42 and the MPEG encoder 43 for application B are described. The elementary streams (ES) encoded in the above are input to the SDTI-CP interfaces 46 and 47, respectively. The SDTI-CP interfaces 46 and 47 each convert an MPEG-encoded elementary stream (ES) into an SDTI-format elementary stream (ES) and transmit it through the SDTI-CP network 50. The SDTI-CP interface 48 converts the elementary stream (ES) in the SDTI format into an MPEG-encoded elementary stream (ES) and outputs it to the MPEG decoder 44.

  The MPEG decoder 44 decodes the input elementary stream (ES) for application A and the elementary stream (ES) for application B, respectively.

  By the way, when the auxiliary data is inserted into the user data in the transport stream, the auxiliary data is inserted in units of frames of the corresponding encoded image, so that the insertion is performed every frame (2 fields).

  When the signal to be encoded is a signal that has been subjected to 3-2 pull-down processing (for example, processing for converting a movie image signal having a frame rate of 24 Hz into an NTSC image signal having a frame rate of 30 Hz), As shown in FIG. 10, the signal is 30 Hz by alternately converting each frame of 24 Hz into a two-field frame in which a repeat field is not created or a three-field frame in which a repeat field is created. It is said that.

  For example, the MPEG encoder 21 in FIG. 4 detects field repetition when an image signal converted to a 30 Hz frame rate by 3-2 pull-down processing is input, and encodes the original 24 Hz encoded frame unit. And flags (Repeat_first_field, Top_field_first) are generated corresponding to the processing.

  The Repeat_first_field flag “1” means that a repeat field has been created, and the Repeat_first_field flag “0” means that a repeat field has not been created. The Top_field_first flag indicates whether the first field among the fields constituting the frame is a top field or a bottom field. The top_field_first flag “1” indicates that the top field has a frame structure earlier in time than the bottom field, and the Top_field_first flag “0” indicates that the bottom field has a frame structure earlier in time than the top field. It represents that.

  As shown in FIG. 10, when the frame of the original signal is a three-field frame, the corresponding one encoded frame includes two fields having the same phase (a field generated by copying by 3-2 pull-down processing). And the field of the copy source). However, when the three fields constituting one original frame are encoded in units of encoded frames, the auxiliary data for each encoded frame is stored in user data as one auxiliary data in units of encoded frames. Therefore, even if different auxiliary data is described in the same phase field of the original signal, the different auxiliary data cannot be distinguished.

  For example, when an MPEG-encoded image is processed in a predetermined studio, transmitted to another studio as a transport stream, and processed in another studio, the image data is transmitted in the encoded order. Although it is necessary to perform this, an apparatus that transmits image data as a transport stream normally does not know the encoding order. Therefore, it is necessary to buffer input image data and know the encoding order. As a result, there is a problem that it takes time from receiving image data supplied for buffering to transmitting the image data, making it difficult to perform rapid processing, particularly real-time processing.

  The present invention has been made in view of such a situation, and enables image data to be processed quickly.

  The digital signal transmission device according to one aspect of the present invention receives a first bitstream in which an encoding order of an image signal and order information regarding the display order of the image signal are inserted in units of access units of the image signal. First receiving means for extracting the order information from the first bit stream received by the first receiving means, generating a time stamp based on the extracted order information, and generating a second bit And a first output means for inserting and outputting the stream.

  The digital signal transmission method or the providing medium according to one aspect of the present invention includes a first bit in which order information relating to an encoding order of image signals and display order of the image signals is inserted in units of access units of the image signals. A receiving step of receiving a stream; and extracting the order information from the first bit stream received in the processing of the receiving step, generating a time stamp based on the extracted order information, and a second bit stream And an output step of inserting and outputting.

  In one aspect of the present invention, the first bitstream in which the encoding order of the image signal and the order information regarding the display order of the image signal are inserted in units of access units of the image signal is received and received. The order information is extracted from the first bit stream, a time stamp is generated based on the extracted order information, inserted into the second bit stream, and output.

  According to one aspect of the present invention, a bit stream can be output. In addition, according to one aspect of the present invention, image data can be processed quickly.

  FIG. 11 shows an example of a system that transmits an MPEG-encoded bit stream between studios 41 and 71 that are located apart from each other, and a portion corresponding to the case in FIG. The same reference numerals are given. In this example, the network in each studio is connected to a public network such as a satellite or ATM (Asynchronous Transfer Mode) via a multiplexing device (hereinafter referred to as TS MUX / DEMUX). Transmission can be performed. The MPEG encoder 72 through the SDTI-CP interface 79 of the studio 71 correspond to the MPEG encoder 42 through the SDTI-CP interface 49 of the studio 41 in FIG. 7, so that the description thereof is omitted here.

  As shown in FIG. 12, an elementary stream (ES) in the SDTI-CP network 50 is converted into a transport stream (TS) in units of 188 bytes by a TS MUX / DEMUX 61 and transmitted via a predetermined transmission medium. The transport stream (TS) transmitted is converted into an elementary stream (ES) in the SDTI format by the TS MUX / DEMUX 62. In FIG. 12, the lightly shaded portion indicates the image data packet, the darkly shaded portion indicates the audio data packet, and the unshaded portion indicates the empty data packet. Show.

  In FIG. 11, a process until the elementary stream (ES) output from the MPEG encoder 42 in the studio 41 is converted into a transport stream (TS) by the TS MUX / DEMUX 61 will be described with reference to FIG. 13. The MPEG video encoder 42A in the MPEG encoder 42 outputs an MPEG-encoded image data elementary stream (ES) to the SDTI-CP interface 46, and the audio encoder 42B provides an audio data elementary stream (ES). Is output to the SDTI-CP interface 46. The SDTI-CP interface 46 converts the input elementary stream (ES) into an SDTI-based format elementary stream (ES) and outputs it to the SDTI-CP interface 51 via the SDTI-CP network 50. The SDTI-CP interface 51 converts the SDTI format ES into an MPEG-encoded elementary stream (ES) and outputs it to the TS MUX / DEMUX 61. The TS MUX / DEMUX 61 converts an MPEG-encoded elementary stream (ES) into a transport stream (TS) in units of 188 bytes and outputs it to a transmission medium.

  In FIG. 11, a transport stream (TS) transmitted from TS MUX / DEMUX 61 is input to TS MUX / DEMUX 62 via a public network such as ATM, and converted to an MPEG encoded elementary stream (ES). The In the SDTI-CP interface 81, the MPEG-encoded elementary stream (ES) is converted into an SDTI format elementary stream (ES) and output to the SDTI-CP network 80 of the studio 71. The MPEG decoder 74 can receive an elementary stream (ES) transmitted from the studio 21 via the SDTI-CP interface 78.

  In the system shown in FIG. 11, when an image signal is transmitted between the studio 41 and the studio 71, an elementary stream (for example, TS MUX / DEMUX 61 shown in FIG. 11) in which image data is encoded is transmitted to the multiplexing device. Since only ES) is input, the multiplexing device does not know the information on the rearrangement of the encoding order performed in the encoder (for example, the MPEG encoder 42 in FIG. 11). For this reason, the multiplexing apparatus must interpret the input elementary stream (ES) and perform processing for converting the elementary stream (ES) into a transport stream (TS).

  The process of interpreting this elementary stream is a process of grasping the coding structure of the image signal. That is, the same processing as when the input image is buffered and encoded by the encoder described above and converted into a bit stream is also required in the multiplexing apparatus. Due to the buffering process in the multiplexing apparatus, a delay occurs until the image data is decoded in the subsequent stage, making real-time processing difficult.

  A delay which is a problem in the system configuration and occurs in the multiplexing apparatus will be described with reference to FIG. When a bit stream encoded by the encoder is input to the multiplexing device in the encoding order shown in FIG. 14, the multiplexing device receives two consecutive I pictures (time t4) and P pictures (time t5). ) Is input, and until the P picture (time t8) is input following the two B pictures (time t6 and time t7) sandwiched between them, the display order cannot be determined. That is, since the multiplexing apparatus does not know the coding structure as shown in FIG. 5 (a structure in which two B pictures are sandwiched between an I picture and a P picture), two B pictures are input. When the next P picture (time t8) is input (when it is confirmed that the input of the B picture has been completed), the display order (PTS) of the I picture input at time t4 can be determined. it can.

  That is, in this multiplexing apparatus, it is possible to determine the display order = 2 of the I picture with the encoding order = 1 only at time t8. For this reason, in the multiplexing apparatus, a delay of the appearance cycle (= 4) of two consecutive I pictures or P pictures + 2 B pictures occurs. Since this delay is newly generated separately from the delay at the encoder, a large delay of the encoder delay (= 3) + multiplexer delay (= 4) = 7 occurs in the entire system.

  Therefore, in the present invention, the order information including the encoding order and the display order is superimposed on the elementary stream as Picture Order Info. This will be described in detail later.

  FIG. 15 shows a configuration example of the MPEG encoder 42 to which the present invention is applied. The separation unit 101 of the MPEG encoder 42 separates image data and auxiliary data from the input image signal, outputs the image data to the MPEG encoding unit 103, and outputs the auxiliary data to the encoding controller 102. The MPEG encoding unit 103 encodes the input image data by the MPEG method, and outputs a POI (Picture Order Information) including a DTS_counter indicating the encoding order and a PTS_counter indicating the display order to the encoding controller 102. In addition, the MPEG encoding unit 103 encodes a coding phase (Coding Phase (V-Phase, H-Phase)) representing the upper left position of the encoded range (the position of the upper left pixel in the effective pixel area 2 in FIG. 1). Supply to the controller 102.

  The encoding controller 102 adds Field ID for identifying the field to which the field belongs to the auxiliary data supplied from the separation unit 101 and Line_number indicating the line into which the auxiliary data is inserted, and the auxiliary data and the MPEG encoding unit 103. The supplied POI and Coding Phase Information (CPI) are appropriately processed and output to the variable length encoding unit 104 as data in the user data format for multiplexing.

  The variable length encoding unit 102 performs variable length encoding on the encoded image data supplied from the MPEG encoding unit 103 and inserts user data supplied from the encoding controller 102 into an elementary stream of image data. The elementary stream output from the variable length encoding unit 104 is output via the transmission buffer 105.

  FIG. 16 shows a configuration example of the MPEG decoder 44 to which the present invention is applied. The reception buffer 111 once buffers input data, and then outputs the buffered data to the variable length decoding unit 112. The variable length decoding unit 112 separates image data and user data from the input data, outputs the image data to the MPEG decoding unit 114, and outputs the user data to the decoding controller 113. The decode controller 113 separates the POI and CPI from the user data and outputs them to the MPEG decoding unit 114. Further, the decode controller 113 outputs auxiliary data separated from the user data to the synthesis unit 115. The MPEG decoding unit 114 decodes the image data input from the variable length decoding unit 112 with reference to the POI and CPI input from the decoding controller 113, and outputs the decoded result to the synthesis unit 115. The synthesizer 115 synthesizes the auxiliary data supplied from the decode controller 113 with the image data supplied from the MPEG decoder 114, and outputs it.

  Hereinafter, operations of the MPEG encoder 42 and the MPEG decoder 44 will be described.

  The operation of describing the CPI in the user data and superimposing it on the elementary stream (ES) will be described with reference to FIG. 17. The MPEG encoder 42 for application A encoded the image signal of application A In the elementary stream (ES), CPI (V-Phase and H-Phase, which are data indicating the effective pixel area in the image signal format) is described in the user data (the variable length encoding unit 104 in FIG. 15 encodes it). The CPI data output from the controller 102 is described in user data) and output to the SDTI-CP interface 46. The MPEG encoder 43 for application B is also configured in the same manner as the MPEG encoder 42, and the CTI is described as user data in the elementary stream (ES) obtained by encoding the image signal of application B, and the SDTI-CP interface 47 Output to.

  A configuration example of the SDTI-CP interface 46 will be described with reference to FIG. 18 (other SDTI-CP interfaces are configured in the same manner). The decoding unit 121 separates the MPEG-encoded elementary stream (ES) input from the MPEG encoder 42 into encoding parameters and image data, decodes the image data, and encodes the encoding parameters together with the encoding parameters. To the unit 122. The encoding parameter multiplexing unit 122 generates and outputs an SDTI-CP based elementary stream (ES) from the image signal and the encoding parameter.

  When an SDTI-CP-based elementary stream (ES) is input to the SDTI-CP interface 46, the encoding parameter separation unit 123 separates image data and encoding parameters from the elementary stream (ES), and Each is output to the encoding unit 124. The encoding unit 124 encodes the image data using the encoding parameter and outputs the encoded image data as an MPEG-encoded elementary stream (ES), or the MPEG-encoded transport stream ( TS).

  The SDTI-CP interfaces 46 and 47 convert an MPEG-encoded elementary stream (ES) into an SDTI-format elementary stream (ES) and transmit the converted stream via the SDTI-CP network 50. The SDTI-CP interface 48 converts an SDTI format elementary stream (ES) into an MPEG-encoded elementary stream (ES) and outputs the converted MPEG stream 44 to the MPEG decoder 44.

  The MPEG decoder 44 decodes the input elementary stream (ES) of the application A or the elementary stream (ES) of the application B, and is described in each elementary stream (ES) (decoding controller in FIG. 16). The image signal is decoded so as to be arranged in the effective pixel area on the basis of the CPI (output from 113).

  The MPEG encoders 42 and 43 can transmit the CPI together with the image data by describing the CPI of the image signal of the encoded application in the user data in the MPEG-encoded elementary stream (ES). it can. Further, the MPEG encoders 42 and 43 have a function of transmitting CPI, so that various applications can be encoded and transmitted.

  The MPEG decoder 44 appropriately arranges the images of applications having various Coding Phases in the effective pixel area by separating and interpreting the CPI inserted in the MPEG-encoded elementary stream (ES). Thus, the decryption process can be performed.

  In the embodiment of the present invention, the CPI is inserted into the user data of the MPEG-encoded elementary stream, but may be inserted into the bit stream by another method.

  Next, an operation of identifying a plurality of auxiliary data for each field in the MPEG encoder 42 will be described with reference to FIG. It is assumed that auxiliary data is inserted in each field of the original signal (30 Hz) of the encoded frame on which 3-2 pull-down processing is performed. When the original signal (30 Hz) having the auxiliary data is encoded into the encoded frame (24 Hz) of the original frame rate, the auxiliary data described in two or three fields included in each encoded frame In addition, an identifier corresponding to the encoded field is added as field_ID (a counter value of 0 to 2) (field_ID is added to the auxiliary data by the encode controller 102 in FIG. 15) and transmitted together with the auxiliary data. The By adding this field_ID to the auxiliary data, it is identified which field in which encoded frame the auxiliary data corresponds to.

  Specifically, since the number of fields of the first encoded frame in FIG. 19 is two, there are two auxiliary data. The encoded frame is generated from two fields, and “0” or “1” is added as field_ID to each auxiliary data of the two fields corresponding to the encoded frame.

  Since the number of fields in the second encoded frame from the top is three, there are three auxiliary data. The encoded frame is generated from three fields, and “0”, “1”, or “2” is added as field_ID to each auxiliary data of the three fields corresponding to the encoded frame.

  In other words, one auxiliary data is not generated for one encoded frame, but the same number of auxiliary data as the number of fields included in the encoded frame is generated, and field_ID is added to each. Is done. As a result, a plurality of auxiliary data included in the same encoded frame is identified in the encoded frame by the added field_ID, so that even if different information is included in each auxiliary data, it can be identified. It will not disappear.

  In the MPEG encoder 42, the image data is encoded by the MPEG encoding unit 103 and variable-length encoded by the variable-length encoding unit 104. In addition, field_ID and the line number (Line_number) in which the auxiliary data has been inserted are added to the auxiliary data by the encoding controller 102, and are inserted as ancillary data in the user data in the elementary stream. Thereby, a plurality of auxiliary data can be identified and transmitted.

  In the MPEG decoder 44, user data in the MPEG-encoded elementary stream is separated by the variable length decoding unit 112 and supplied to the decoding controller 113. The decode controller 113 identifies and separates a plurality of auxiliary data based on the field_ID and Line_number of the ancillary data inserted in the user data, and outputs the auxiliary data to the combining unit 115. The synthesizing unit 115 synthesizes the image data decoded by the MPEG decoding unit 114 and the auxiliary data (text data) corresponding thereto, and outputs the synthesized data.

  The MPEG encoder 42 generates a POI for managing the encoding order and the display order as shown in FIG. 20 in order to reduce the delay of the entire system.

  For example, when the image data input to the MPEG encoding unit 103 of the MPEG encoder 42 is an image signal converted into a frame rate of 24 Hz by 3-2 pull-down processing, a flag (as shown in FIG. Each frame is managed by Repeat_first_field, Top_field_first).

  “1” of the flag of Repeat_first_field means that it is necessary to create a repeat field, and “0” of the flag of Repeat_first_field means that it is not necessary to create a repeat field. The Top_field_first flag indicates whether the first field among the fields constituting the frame is a top field or a bottom field. “1” of the Top_field_first flag indicates that the top field has a frame structure earlier in time than the bottom field, and “0” in the Top_field_first flag has a frame structure earlier in time than the top field. Represents that.

  Specifically, FIG. 20A will be described. The encoded image data type of an encoded frame of Frame No1 input to the MPEG encoding unit 103 first is an I picture, and two fields (top) of this I picture Field and bottom field), it is necessary to convert the top field to 3 fields by creating a repeat field, so the corresponding Repeat_first_field flag is "1" and the Top_field_first flag is "1" Become.

  The encoded image data type of the encoded frame of Frame No. 2 is a B picture, and since it is not necessary to generate a repeat field in this B picture, the flag of Repeat_first_field is set to “0” and the bottom field is the top field. Since the frame is earlier in time, the Top_field_first flag is set to “0”. The value of the Top_field_first flag at this time is not related to the 3-2 pull-down process.

  The encoded image data type of the encoded frame of Frame No3 is a B picture, and in the B picture of Frame No3, a repeat field is created by copying the bottom field, and the encoded frame is converted into 3 fields. . Therefore, the flag of Repeat_first_field is “1”, and the flag of Top_field_first is “0”.

  The encoded image data type of the encoded frame of Frame No4 is a P picture, no repeat field is created for this P picture, the flag of Repeat_first_field is “0”, and the flag of Top_field_first is It is set to 1.

  When image data that has been subjected to 3-2 pull-down processing as shown in FIG. 20A is input, the MPEG encoding unit 103 counts the number of fields with a built-in counter PTS_counter and displays the value PTS_counter It outputs to the encoding controller 102 as an order. The counter PTS_counter performs a counting operation that increases from 0 to 127 and then returns to 0 again. Therefore, the value of the counter PTS_counter changes as shown in FIG.

  More specifically, the value of PTS_counter of the I picture of Frame No. 1 input first is the value “0”. The PTS_counter value of the B picture of Frame No. 2 input second from the top is a value “3” (= 0 + 3) obtained by adding the number of P picture fields to 3 to the PTS_counter value “0” of the I picture of Frame No. 1 It becomes.

  The value of PTS_counter of the B picture of Frame No. 3 input from the beginning is the value “5” (= 3 + 2) obtained by adding the number of fields of B picture 2 to the value “3” of PTS_counter of the B picture of Frame No 2 It becomes. The PTS_counter value of the P picture of Frame No. 4 that is input fourth from the beginning is the value “8” (= 5 + 3) obtained by adding the number of fields 3 of the B picture to the value “5” of the PTS_counter of the B picture of Frame No 3 It becomes. The value of PTS_counter after the B picture of Frame No. 5 is calculated similarly.

  Further, the MPEG encoding unit 103 counts the frames encoded by the built-in counter DTS_counter, and outputs the counted result to the end controller 102.

Specifically, referring to FIG. 20C, the value 125 of the DTS_counter of the I picture of Frame No1 is one frame when the display order PTS_counter = 0 in which the I picture of Frame No1 is displayed. Must be encoded before the appearance cycle of minutes (corresponding to FIG. 14, the encoding order value of the first I picture is “1”, the display order value is “2”, The order value must be one frame earlier than the display order value). In other words, since the I picture has three fields, the value of DTS_counter is represented by a value “125” that is 3 earlier than 0 (DTS_counter is represented by a modulo of 2 ⁷ (= 128), so the value is 0. Cycle through values between 1 and 127).

  The DTS_counter value of the P picture of Frame No. 4 encoded next to the I picture of Frame No. 1 is 0 (= 128 = 125 + 3) obtained by adding the number of I picture fields to 3 to the DTS_counter value 125 of the I picture of Frame No. 1 )

  The DTS_counter value of the B picture of Frame No2 encoded next to the P picture of Frame No4 is PTS_counter = DTS_counter for the B picture, and is the same as the value of PTS_counter, and the value is “3”. Become. Similarly, the value of DTS_counter of the B picture of Frame No. 3 encoded next to the B picture of Frame No. 2 is also set to be the same as the value of PTS_counter, and the value is set to “5”. Hereinafter, the value of the DTS_counter after the P picture of Frame No. 7 is also calculated in the same manner, so that the description thereof is omitted here.

  The MPEG encoding unit 103 outputs the flags Repeat_first_field and Top_field_first and counters PTS_counter and DTS_counter as shown in FIG. 20 to the encoding controller 102 as POIs.

  Here, the SDTI-CP networks 50 and 80, the TS MUX / DEMUX 61 and 62, and the ATM network are connected from the MPEG encoder 42 of the studio 41 at a remote location to the MPEG decoder 74 of the studio 71 as shown in FIG. The encoding order and display order of the system that uses and transmits the image signal will be described with reference to FIG.

  An MPEG video encoder 42A in the MPEG encoder 42 outputs an elementary stream (ES) of MPEG-encoded image data, and inserts POI into user data in the elementary stream (ES) (FIG. 15). The encoding controller 102 describes POI data in user data, and outputs it to the variable-length encoding unit 104 for multiplexing). The audio encoder 42B in the MPEG encoder 42 encodes the audio data and outputs it as an elementary stream (ES). The SDTI-CP interface 46 converts the elementary stream (ES) (stream including POI) from the MPEG video encoder 42A and the elementary stream (ES) from the audio encoder 42B into an SDTI format elementary stream. To the SDTI-CP network 50.

  The SDTI-CP interface 51 converts the input elementary stream (ES) in the SDTI format into an MPEG-encoded elementary stream (ES) (described as an operation of the SDTI-CP interface 46 with reference to FIG. 18). Output to TS MUX / DEMUX 61. TS MUX / DEMUX 61 refers to the POI inserted in the elementary stream (ES), sets the value of PTS_counter to PTS (Presentation Time Stamp), and sets the value of DTS_counter to DTS (Decoding Time Stamp). Conversion, multiplexing processing is performed, and a transport stream (TS) composed of 188-byte packets is generated and output (therefore, the transport stream (TS) is not PTS_counter and DTS_counter but PTS and DTS included).

  The TS MUX / DEMUX 61 can immediately perform the multiplexing process without interpreting the buffering process described above by interpreting the POI inserted in the elementary stream (ES). In the TS MUX / DEMUX 61, There is no new delay. Also, since the POI is inserted in the elementary stream (ES), the TS MUX / DEMUX 61 does not need to perform the process of including the POI in the bitstream in order to transmit the POI to the subsequent stage.

  The TS MUX / DEMUX 62 in FIG. 11 separates image data and audio data from a transport stream (TS) input via a public network such as ATM, and converts it into an MPEG-encoded elementary stream (ES). . The TS MUX / DEMUX 62 also converts the PTS and DTS into PTS_counter and DTS_counter, inserts them into the user data of the elementary stream, and outputs them to the SDTI-CP interface 81. The SDTI-CP interface 81 converts an MPEG-encoded elementary stream (ES) into an SDTI format elementary stream (ES), and outputs the converted stream to the SDTI-CP network 80 of the studio 71. The MPEG decoder 74 (the same configuration as the MPEG decoder 44) receives and decodes the transmitted elementary stream (ES) of image data and audio data via the SDTI-CP interface 78.

  Similar to the TS MUX / DEMUX 61, the MPEG decoder 74 interprets the POI inserted in the elementary stream (ES) and immediately decodes it without performing the buffering process described above (decode controller 113 in FIG. 16). The MPEG decoding unit 114 can decode the image data based on the POI output by the MPEG decoder 74, and no new delay occurs in the MPEG decoder 74. That is, the MPEG video encoder 42A inserts and outputs the POI into the elementary stream together with the MPEG-encoded elementary stream (ES), so that the subsequent TS MUX / DEMUX 61 and the MPEG decoder 74 It is possible to immediately perform the multiplexing process or the decoding process after interpretation, and the delay of the entire system can be suppressed to only the delay of the number of B pictures (= 2) +1 sheets = 3 generated in the MPEG encoder 42. That is, in a system including such encoding, multiplexing, and decoding, the theoretically smallest delay can be achieved.

  Further, in the system configuration of FIG. 21, when an image signal is transmitted using the SDTI-CP network 50 by using the SDTI-CP interface 46 shown in FIG. It is easy to edit and can be transmitted in the form of elementary stream (ES) suitable for short-distance transmission. When transmitting image signals between remote studios using a public network such as ATM, a bit stream Can be transmitted in the form of a transport stream (TS) suitable for long-distance transmission.

  In the above, the POI is inserted into the elementary stream. However, when the distance between the TS MUX / DEMUX 61 and the MPEG encoder 42 is short, the POI is transferred from the MPEG encoder 42 to the TS as shown in FIG. You may make it supply directly to MUX / DEMUX61.

  However, in this case, wiring processing other than the SDTI-CP network 50 that transmits the elementary stream is required.

  As described above, the information possessed only by the encoder (Coding Phase) is described by describing the information possessed by the encoder in the user data in the elementary stream (ES) and outputting it to the multiplexing device or decoder. (V-Phase and H-Phase), Field_ID, encoding order DTS_counter, and display order PTS_counter) can be supplied to the multiplexing device and decoder subsequent to the encoder.

  Next, the bitstream syntax will be described with reference to FIGS.

  FIG. 23 is a diagram illustrating the syntax of an MPEG video stream. The MPEG encoder 42 generates an encoded elementary stream according to the syntax shown in FIG. In the syntax described below, functions and conditional statements are expressed in fine print, and data elements are expressed in bold print. The data item is described in mnemonic (Mnemonic) indicating its name, bit length, its type and transmission order.

  First, functions used in the syntax shown in FIG. 23 will be described. In practice, the syntax shown in FIG. 23 is a syntax used on the MPEG decoder 44 side to extract a predetermined meaningful data element from the transmitted encoded bit stream. . The syntax used on the MPEG encoder 42 side is a syntax in which conditional statements such as an if statement and a while statement are omitted from the syntax shown in FIG.

  The next_start_code () function described first in video_sequence () is a function for searching for a start code described in the bitstream. In the encoded stream generated according to the syntax shown in FIG. 23, first, data elements defined by the sequence_header () function and the sequence_extension () function are described. The sequence_header () function is a function for defining header data of the sequence layer of the MPEG bit stream, and the sequence_extension () function is a function for defining extension data of the sequence layer of the MPEG bit stream.

  The do {} while syntax placed next to the sequence_extension () function is a code where data elements described based on the functions in {} of the do statement are encoded while the condition defined by the while statement is true. Is a syntax indicating that it is described in the data stream. The nextbits () function used in the while statement is a function for comparing a bit or a bit string described in a bit stream with a referenced data element. In the syntax example shown in FIG. 23, the nextbits () function compares the bit string in the bit stream with sequence_end_code indicating the end of the video sequence, and the bit string in the bit stream does not match sequence_end_code. The condition of this while statement is true. Therefore, the do {} while syntax placed next to the sequence_extension () function encodes the data element defined by the function in the do statement while the sequence_end_code indicating the end of the video sequence does not appear in the bitstream. Indicates that it is described in the bitstream.

  In the encoded bitstream, data elements defined by the extension_and_user_data (0) function are described after each data element defined by the sequence_extension () function. The extension_and_user_data (0) function is a function for defining extension data and user data in the sequence layer of the MPEG bit stream.

  The do {} while syntax placed next to this extension_and_user_data (0) function is a data element described based on the function in {} of the do statement while the condition defined by the while statement is true. Is a function indicating that it is described in the bitstream. The nextbits () function used in the while statement is a function for determining a match between a bit or a bit string appearing in the bit stream and a picture_start_code or group_start_code, and the bit or the bit string appearing in the bit stream. If picture_start_code or group_start_code matches, the condition defined by the while statement is true. Therefore, in this do {} while syntax, when picture_start_code or group_start_code appears in the encoded bitstream, the code of the data element defined by the function in the do statement is described next to the start code. It shows that.

  The if statement described at the beginning of the do statement indicates a condition that group_start_code appears in the encoded bitstream. When the condition by this if statement is true, the data elements defined by the group_of_picture_header () function and the extension_and_user_data (1) function are described in order in the encoded bitstream after this group_start_code.

  This group_of_picture_header () function is a function for defining the GOP layer header data of the MPEG encoded bit stream, and the extension_and_user_data (1) function is the extension data and user data of the GOP layer of the MPEG encoded bit stream. It is a function for defining.

  Furthermore, in this encoded bitstream, the data elements defined by the picture_header () function and the picture_coding_extension () function are described next to the data elements defined by the group_of_picture_header () function and the extension_and_user_data (1) function. Yes. Of course, if the condition of the if statement described above is not true, the data element defined by the group_of_picture_header () function and the extension_and_user_data (1) function is not described, so it is defined by the extension_and_user_data (0) function. The data elements defined by the picture_header () function, the picture_coding_extension () function, and the extension_and_user_data (2) function are described next to the data element.

  The picture_header () function is a function for defining the header data of the picture layer of the MPEG encoded bit stream, and the picture_coding_extension () function is for defining the first extension data of the picture layer of the MPEG encoded bit stream It is a function to do. The extension_and_user_data (2) function is a function for defining extension data and user data of a picture layer of an MPEG encoded bit stream. The user data defined by the extension_and_user_data (2) function is data described in the picture layer and can be described for each picture.

  In the encoded bitstream, data elements defined by the picture_data () function are described after the user data of the picture layer. The picture_data () function is a function for describing data elements related to the slice layer and the macroblock layer.

  The while statement described next to the picture_data () function is a function for determining the condition of the next if statement while the condition defined by the while statement is true. The nextbits () function used in this while statement is a function for determining whether picture_start_code or group_start_code is described in the encoded bitstream, and picture_start_code or group_start_code is described in the bitstream. If so, the condition defined by this while statement is true.

  The next if statement is a conditional statement for determining whether sequence_end_code is described in the encoded bitstream. If sequence_end_code is not described, sequence_header () function and sequence_extension () function Indicates that the data element defined by and is described. Since sequence_end_code is a code indicating the end of the sequence of the encoded video stream, unless the encoded stream ends, the data element defined by the sequence_header () function and the sequence_extension () function is described in the encoded stream. ing.

  The data elements described by the sequence_header () function and the sequence_extension () function are exactly the same as the data elements described by the sequence_header () function and the sequence_extension () function described at the beginning of the video stream sequence. The reason why the same data is described in the stream in this way is that the reception of the sequence layer is received when the reception starts from the middle of the data stream (for example, the bit stream portion corresponding to the picture layer) on the bit stream receiver side. This is to prevent the stream from being able to be decoded and being unable to decode the stream.

  Following the data element defined by the last sequence_header () function and sequence_extension () function, that is, at the end of the data stream, 2-bit sequence_end_code indicating the end of the sequence is described.

  The sequence_header () function, sequence_extension () function, extension_and_user_data (0) function, group_of_picture_header () function, picture_header () function, picture_coding_extension () function, and picture_data () function will be described in detail below.

  FIG. 24 is a diagram for explaining the syntax of the sequence_header () function. The data elements defined by this sequence_header () function are sequence_header_code, horizontal_size_value, vertical_size_value, aspect_ratio_information, frame_rate_code, bit_rate_value, marker_bit, vbv_buffer_size_value, constrained_parameter_flag, load_intra_quantizer_matrix, non64_intrix_intra_quantizer_

  sequence_header_code is data representing the start synchronization code of the sequence layer. horizontal_size_value is data consisting of the lower 12 bits of the number of pixels in the horizontal direction of the image. vertical_size_value is data consisting of the lower 12 bits of the number of vertical lines of the image. Aspect_ratio_information is data representing the pixel aspect ratio (aspect ratio) or display screen aspect ratio. The frame_rate_code is data representing an image display cycle. bit_rate_value is lower 18 bits (rounded up in units of 400 bsp) of the bit rate for limiting the amount of generated bits. The marker_bit is bit data inserted to prevent start code emulation. vbv_buffer_size_value is lower 10-bit data of a value that determines the size of the generated code amount control virtual buffer (video buffer verifier). constrained_parameter_flag is data indicating that each parameter is within the limit. The load_intra_quantizer_matrix is data indicating the presence of intra MB quantization matrix data. intra_quantizer_matrix [64] is data indicating the value of the intra MB quantization matrix. The load_non_intra_quantizer_matrix is data indicating the presence of non-intra MB quantization matrix data. non_intra_quantizer_matrix is data representing the value of the non-intra MB quantization matrix.

  FIG. 25 is a diagram for explaining the syntax of the sequence_extension () function. The data elements defined by this sequence_extension () function are extension_start_code, extension_start_code_identifier, profile_and_level_indication, progressive_sequence, chroma_format, horizontal_size_extension, vertical_size_extension, bit_rate_extension, vbv_buffer_size_extension, low_delay, frame_rate_ext_, extension_data_ext_

  extension_start_code is data representing a start synchronization code of extension data. extension_start_code_identifier is data indicating which extension data is sent. Profile_and_level_indication is data for designating the profile and level of video data. progressive_sequence is data indicating that the video data is sequentially scanned. chroma_format is data for designating the color difference format of the video data. The horizontal_size_extension is upper 2 bits data added to the horizntal_size_value of the sequence header. vertical_size_extension is upper 2 bits of data added to vertical_size_value of the sequence header. bit_rate_extension is upper 12-bit data added to bit_rate_value of the sequence header. vbv_buffer_size_extension is upper 8-bit data to be added to vbv_buffer_size_value of the sequence header. low_delay is data indicating that a B picture is not included. Frame_rate_extension_n is data for obtaining a frame rate in combination with frame_rate_code of the sequence header. Frame_rate_extension_d is data for obtaining a frame rate in combination with frame_rate_code of the sequence header.

  FIG. 26 is a diagram for explaining the syntax of the extension_and_user_data (i) function. The extension_and_user_data (i) function describes only the data element defined by the user_data () function without describing the data element defined by the extension_data () function when “i” is other than 1. Therefore, the extension_and_user_data (0) function describes only the data element defined by the user_data () function.

  First, functions used in the syntax shown in FIG. 26 will be described. The nextbits () function is a function for comparing a bit or a bit string appearing in a bit stream with a data element to be decoded next.

  As shown in FIG. 27, the user_data () function includes user_data_start_code, V-phase () function, H-phase () function, Time_code () function, Picture-order () function, Ancillary_data () function, history_data () function. , And a function for describing the data element of user_data.

  user_data_start_code is a start code for indicating the start of the user data area of the picture layer of the MPEG bit stream. The if statement described next to the user_data_start_code executes the while syntax described next when i of the user_data (i) function is “0”. This while syntax is true unless 24-bit data composed of 23 “0” s and subsequent “1” appears in the bitstream.

  The 24-bit data composed of these 23 “0” s followed by “1” is data added to the head of all the start codes, and all the start codes are after the 24 bits. By being provided, the nextbits () function can find the position of each start code in the bitstream.

  When the while syntax is true, if the nextbits () function of the if statement described next detects a bit string (Data_ID) indicating V-Phase, V-Phase () is calculated from the next bit of that bit string (Data_ID). Know that the V-Phase data element indicated by the function is described. When the nextbits () function of the next Else if statement detects a bit string (Data_ID) indicating an H-Phase, the data element of the H-Phase indicated by the H-Phase () function is determined from the next bit of the bit string (Data_ID). Know what is written.

  Here, as shown in FIG. 28, the V-Phase Data_ID is a bit string representing “01”, and the H-Phase Data_ID is a bit string representing “02”.

  The syntax of the V-Phase () function described in the bitstream will be described with reference to FIG. First, as described above, Data_ID is 8-bit data indicating that the data element of the bit string next to Data_ID is V-Phase, and is the value “01” shown in FIG. V-Phase is 16-bit data indicating the first line to be encoded in the frame of the image signal.

  The syntax of the H-Phase () function described in the bitstream will be described with reference to FIG. First, as described above, Data_ID is 8-bit data indicating that the data element of the bit string next to Data_ID is H-Phase, and is the value “02” shown in FIG. H-Phase is 8-bit data indicating the first sample to be encoded in the image signal frame.

  Returning to FIG. 27, the next Else if statement executes the while syntax described next when i of the user_data (i) function is 2. Since the contents of the while syntax are the same as those described above, the description thereof is omitted here.

  When the while syntax is true, in the next if statement, the nextbits () function detects a bit string indicating Time code 1 or when a bit string indicating Time code 2 is detected, the Time_code () function is calculated from the next bit of the bit string. Knows that the time code data element indicated by is described.

  As shown in FIG. 28, Data_ID of Time code1 is a bit string representing “03”, and the data of Time code1 indicates a time code inserted in the vertical blanking period of the image, VITC (Vertical Interval Time Code) It is. As shown in FIG. 28, Data_ID of Time code 2 is a bit string representing “04”, and the data of Time code 2 is LTC (Longitudinal Time Code) indicating the time code recorded on the time code track of the recording medium. is there.

  Next, in the Else if statement, when the nextbits () function detects a bit string indicating Picture Order, it knows that the data element of Picture Order indicated by the Picture_Order () function is described from the next bit of the bit string. Here, Data_ID of the Picture_Order () function is a bit string representing “05” as shown in FIG.

  The syntax of the Picture_Order () function that is actually inserted into the elementary stream (ES) by the encoder will be described with reference to FIG. First, as described above, Data_ID is 8-bit data indicating that the data after the Data_ID is POI data, and its value is “05”. DTS_presence is 1-bit data indicating the presence / absence of the coding order DTS_counter. For example, when DTS_counter = PTS_counter as in a B picture, only the display order PTS_counter exists, and the bit of DTS_presence is “0”. On the contrary, in the case of P picture and I picture, since the encoding order DTS_counter and the display order PTS_counter are not the same, both the display order PTS_counter and the encoding order DTS_counter exist, and the bit of DTS_presence is 1.

  PTS_counter is 7-bit data representing the display order that is counted up every time one field in the encoded frame is input to the encoder. The 7-bit data is a modulo value ranging from 0 to 127. After the if statement, when the bit of DTS_presence is 1, that is, when it is a P picture and an I picture, DTS_counter is counted up.

  Marker_bits is a bit inserted every 16 bits to prevent start code emulation, in which the bit string described in user data coincides with the start code described above by chance and is likely to cause image lockup. is there.

  DTS_counter is 7-bit data representing an encoding order that is counted up every time encoded image data for one field is encoded by the encoder. The 7-bit data is a modulo value ranging from 0 to 127.

  As described above, since the display order PTS_counter is numbered in units of fields, for example, when encoding image data converted from a frame rate of 24 Hz to 30 Hz, 3-2 pull-down processing is performed. It will need to be numbered later.

  Returning to FIG. 27, the contents of the “while” syntax described next are the same as those in the case described above, and the description thereof is omitted here. When the while syntax is true, in the next if statement, when the nextbits () function detects a bit string indicating Ancillary data, the data element of Ancillary data indicated by the Ancillary_data () function is described from the next bit of the bit string. Know that you are. Data_ID of the Ancillary_data () function is a bit string representing “07” as shown in FIG.

  The syntax of ancillary data for adding an identifier to the auxiliary data will be described with reference to FIG. The Ancillary_data () function is transmitted as user data in the picture layer, and a field identifier (Field_ID), a line number (Line_number), and supplementary data (ancillary data) are inserted as data.

  Data_ID is 8-bit data indicating ancillary data in the user data area, and its value is “07” as shown in FIG.

  Field_ID is 2-bit data, and when the value of progressive_sequence flag (FIG. 25) is “0”, Field_ID is added for each field in the encoded frame. When repeat_first_field is set to “0”, there are two fields in this encoded frame, and Field_ID is “0” in the first field and the next field as shown in FIG. When “1” is set and repeat_first_field is set to “1”, there are three fields in this encoded frame, and Field_ID is set to “0” in the first field, and thereafter “1” and “2” are set in the field.

  Field_ID is added for each encoded frame when the value of progressive_sequence flag is “1”. In Field_ID, when repeat_first_field and Top_field_first are both set to “0”, since the encoded frame has one progressive frame, the value “0” is set, and repeat_first_field is set to value “1” and Top_field_first. When the value is set to “0”, since there are two progressive frames in the encoded frame, the values “0” and “1” are set, and both repeat_first_field and Top_field_first are set to “1”. Since there are three progressive frames in the encoded frame, values “0” to “2” are set.

  Line_number is 14-bit data, and indicates a line number defined in ITU-R BT.656-3, SMPTE274M, SMPTE293M, and SMPTE296M in which auxiliary data in each frame is described.

  Ancillary_data_length is 16-bit data and indicates the data length of ancillary_data_payload. Ancillary_data_payload represents the content of auxiliary data consisting of 22-bit data. When the value of Ancillary_data_length of Ancillary_data_payload is larger than the value of j (initial value 0), the value j (data length of Ancillary_data_length) is incremented by 1. , And is described from the bit string of the value of j.

  The following While syntax represents the syntax for the bytealigned () function. When the next data is not a bytealigned () function (when the While syntax is true), Zero_bit (1-bit data “0”) is used. Describe.

  Returning to FIG. 27, in the next Else if statement, when the nextbits () function detects a bit string indicating History data, the data element of History data indicated by the History_data () function is described from the next bit of the bit string. Know that you are. As shown in FIG. 28, Data_ID of the History_data () function is a bit string representing “08”, and data represented by Data_ID “08” represents History data including history information of encoding parameters.

  In the last if statement, when the nextbits () function detects a bit string indicating user data, it knows that the data element of user_data indicated by the user_data () function is described from the next bit of the bit string.

  A bit string that the nextbits () function in FIG. 27 knows that each data element is described is described as Data_ID shown in FIG. However, using “00” as Data_ID is prohibited. Data indicated by Data_ID “80” represents a control flag, and data indicated by Data_ID “FF” represents user data.

  FIG. 33 is a diagram for explaining the syntax of the group_of_picture_header () function. The data element defined by the group_of_picture_header () function is composed of group_start_code, time_code, closed_gop, and broken_link.

  group_start_code is data indicating the start synchronization code of the GOP layer. time_code is a time code indicating the time from the beginning of the sequence of the first picture of the GOP. closed_gop is flag data indicating that an image in a GOP can be reproduced independently from other GOPs. Broken_link is flag data indicating that the first B picture in the GOP cannot be accurately reproduced for editing or the like.

  The extension_and_user_data (1) function is a function for describing only the data element defined by the user_data () function, like the extension_and_user_data (0) function.

  Next, a picture_headr () function, a picture_coding_extension () function, and picture_data () for describing data elements related to the picture layer of the encoded stream will be described with reference to FIGS. 34 to 36.

  FIG. 34 is a diagram for explaining the syntax of the picture_headr () function. The data elements defined by this picture_headr () function are picture_start_code, temporal_reference, picture_coding_type, vbv_delay, full_pel_forward_vector, forward_f_code, full_pel_backward_vector, backward_f_code, extra_bit_picture, and extra_information_picture.

  Specifically, picture_start_code is data representing the start synchronization code of the picture layer. temporal_reference is a number indicating the display order of pictures and is data to be reset at the top of the GOP. picture_coding_type is data indicating a picture type.

  vbv_delay is data indicating the initial state of the VBV buffer, and is set for each picture. The picture of the encoded elementary stream transmitted from the transmission side system to the reception side system is buffered in a VBV buffer provided in the reception side system, and this VBV buffer is stored at the time specified by DTS (Decoding Time Stamp). Is extracted (read) and supplied to the decoder. The time defined by vbv_delay means the time from when the decoding target picture starts to be buffered in the VBV buffer until the encoding target picture is pulled out of the VBV buffer, that is, the time specified by the DTS. To do. By using vbv_delay stored in the picture header, seamless splicing can be realized in which the data occupation amount of the VBV buffer is not discontinuous.

  full_pel_forward_vector is data indicating whether the accuracy of the forward motion vector is an integer unit or a half pixel unit. forward_f_code is data representing the forward motion vector search range. full_pel_backward_vector is data indicating whether the accuracy of the backward motion vector is an integer unit or a half pixel unit. backward_f_code is data representing the backward motion vector search range. extra_bit_picture is a flag indicating the presence of subsequent additional information. When this extra_bit_picture is “1”, there is next extra_information_picture, and when extra_bit_picture is “0”, it indicates that there is no subsequent data. extra_information_picture is information reserved in the standard.

  FIG. 35 is a diagram for explaining the syntax of the picture_coding_extension () function. The data elements defined by this picture_coding_extension () function are extension_start_code, extension_start_code_identifier, f_code [0] [0], f_code [0] [1], f_code [1] [0], f_code [1] [1], intra_dc_precision , Picture_structure, top_field_first, frame_predictive_frame_dct, concealment_motion_vectors, q_scale_type, intra_vlc_format, alternate_scan, repeat_first_field, chroma_420_type, progressive_frame, composite_display_flag, v_axis, field_sequence, sub_carrier, burst_amplitude

  extension_start_code is a start code indicating the start of extension data of the picture layer. extension_start_code_identifier is a code indicating which extension data is sent. f_code [0] [0] is data representing the horizontal motion vector search range in the forward direction. f_code [0] [1] is data representing a vertical motion vector search range in the forward direction. f_code [1] [0] is data representing the horizontal motion vector search range in the backward direction. f_code [1] [1] is data representing a vertical motion vector search range in the backward direction. intra_dc_precision is data representing the precision of the DC coefficient. Picture_structure is data indicating a frame structure or a field structure. In the case of a field structure, this indicates whether it is an upper field or a lower field.

  In the case of a frame structure, top_field_first is a flag indicating whether the first field is a top field or a bottom field. In the case of a frame structure, frame_predictive_frame_dct is data indicating that the prediction of the frame mode DCT is only the frame mode. concealment_motion_vectors is data indicating that a motion vector for concealing a transmission error is attached to an intra macroblock. q_scale_type is data indicating whether to use a linear quantization scale or a nonlinear quantization scale. The intra_vlc_format is data indicating whether another two-dimensional VLC (variable length code) is used for the intra macroblock. The alternate_scan is data representing a selection between using a zigzag scan or an alternate scan.

  repeat_first_field is a flag indicating whether or not a repeat field is to be generated at the time of decoding. When repeat_first_field is “1” in the process at the time of decoding, a repeat field is generated and when repeat_first_field is “0” Is processed so as not to generate a repeat field.

  The chroma_420_type is data representing the same value as the next progressive_frame when the signal format is 4: 2: 0, and 0 otherwise. Progressive_frame is data indicating whether or not the picture can be sequentially scanned. composite_display_flag is data indicating whether the source signal is a composite signal. v_axis is data used when the source signal is PAL. The field_sequence is data used when the source signal is PAL. sub_carrier is data used when the source signal is PAL. burst_amplitude is data used when the source signal is PAL. sub_carrier_phase is data used when the source signal is PAL.

  FIG. 36 is a diagram for explaining the syntax of the picture_data () function. The data element defined by the picture_data () function is a data element defined by the slice () function. However, when slice_start_code indicating the start code of the slice () function does not exist in the bitstream, the data element defined by this slice () function is not described in the bitstream.

  The slice () function is a function for describing data elements related to a slice layer. ) A function for describing a data element defined by a function.

  The slice_start_code is a start code indicating the start of the data element defined by the slice () function. The slice_quantiser_scale_code is data indicating the quantization step size set for the macroblock existing in this slice layer. However, when quantiser_scale_code is set for each macroblock, the macroblock_quantiser_scale_code data set for each macroblock is used preferentially. intra_slice_flag is a flag indicating whether or not intra_slice and reserved_bits exist in the bitstream. intra_slice is data indicating whether or not a non-intra macroblock exists in the slice layer. If any of the macroblocks in the slice layer is a non-intra macroblock, intra_slice is “0”, and if all of the macroblocks in the slice layer are non-intra macroblocks, intra_slice is “1”. Become. reserved_bits is 7-bit data and takes a value of “0”. extra_bit_slice is a flag indicating that additional information exists as an encoded stream, and is set to “1” when extra_information_slice exists next. If there is no additional information, it is set to “0”.

  The macroblock () function is a function for describing data elements related to the macroblock layer. Specifically, data elements such as macroblock_escape, macroblock_address_increment, and macroblock_quantiser_scale_code, macroblock_modes () function, and macroblock_vecters (s) function Is a function for describing data elements defined by.

  macroblock_escape is a fixed bit string indicating whether or not the horizontal difference between the reference macroblock and the previous macroblock is 34 or more. If the horizontal difference between the reference macroblock and the previous macroblock is 34 or more, 33 is added to the value of macroblock_address_increment. The macroblock_address_increment is data indicating a horizontal difference between the reference macroblock and the previous macroblock. If there is one macroblock_escape before this macroblock_address_increment, the value obtained by adding 33 to the value of this macroblock_address_increment is the data indicating the horizontal difference between the actual reference macroblock and the previous macroblock. . macroblock_quantiser_scale_code is a quantization step size set for each macroblock. In each slice layer, slice_quantiser_scale_code indicating the quantization step size of the slice layer is set, but when macroblock_quantiser_scale_code is set for the reference macroblock, this quantization step size is selected.

  FIG. 37 is an explanatory diagram showing the data structure of an MPEG encoded stream. As shown in this figure, the data structure of the video elementary stream includes at least a sequence layer, a GOP layer, and a picture layer.

  The sequence layer includes data elements defined by a next_start_code () function 201, a sequence_header () function 202, an extension_start_code () 203, a sequence_extension () function 204, and an extension_and_user_data (0) function 205. The GOP layer includes data elements defined by a group_start_code 206, a group_of_picture_header () function 207, and an extension_and_user_data (1) function 208. The picture layer includes data elements defined by a picture_header () function 209, a picture_coding_extension () function 210, an extension_and_user_data (2) function 211, and a picture_data () function 212. At the end of the video sequence, sequence_end_code 213 is described.

  The extension_and_user_data (2) function 211 includes data elements defined by user_data_start_code 214, user_data () function 215, and next_start_code 216, as can be understood from the syntax described with reference to FIG.

  The user_data () function 215 includes data elements defined by the time_code () function 217 and user_data 218 as can be understood from the syntax already described with reference to FIG.

  In the present specification, the term “system” represents the entire apparatus including a plurality of apparatuses.

  In addition, in the present specification, as a providing medium for providing a user with a computer program for executing the above processing, an information recording medium such as a magnetic disk and a CD-ROM, and a transmission medium via a network such as the Internet and a digital satellite are also included. included.

  As described above, according to the digital signal transmission device according to claim 1, the digital signal transmission method according to claim 4, and the providing medium according to claim 5, the order information is extracted from the first bit stream. Since the time stamp is generated based on the extracted order information and inserted into the second bit stream, the image data can be quickly processed at a predetermined location, In this case, image data can be processed quickly.

It is a figure explaining Coding Phase. It is a figure explaining auxiliary data. It is a figure which shows the structure of the system which transmits an elementary stream. It is a figure which shows the structure of the system which transmits auxiliary data. It is a figure explaining the prediction structure and encoding structure in MPEG system. It is a figure explaining the delay which generate | occur | produces in an MPEG encoder. It is a block diagram which shows the structure inside a studio. It is a figure explaining the data transmitted inside the studio of FIG. It is a figure which shows the structure of the system which transmits data in the system of FIG. It is a figure explaining the auxiliary data of the image by which 3-2 pull-down process was carried out. It is a block diagram showing a system configuration when transmitting a bit stream between two studios. It is a figure showing the structure of the stream transmitted with the system of FIG. FIG. 12 is a block diagram illustrating a configuration for transmitting a transport stream in the system of FIG. 11. It is a figure explaining the delay of an encoding process. It is a block diagram showing the structure of the MPEG encoder to which this invention is applied. It is a block diagram showing the structure of the MPEG decoder to which this invention is applied. It is a figure showing the structure of the system in the case of transmitting CPI. FIG. 12 is a block diagram illustrating a configuration of an SDTI-CP interface 46 in FIG. 11. It is a figure explaining transmission of auxiliary data. It is a figure explaining PTS_counter and DTS_counter in 3-2 pulldown processing. It is a figure showing the structure of the system in the case of transmitting POI. It is a figure showing the structure of the other system at the time of transmitting POI. It is a figure explaining the syntax of a video_sequence function. It is a figure explaining the syntax of a sequence_header () function. It is a figure explaining the syntax of sequence_extension () function. It is a figure explaining the syntax of extension_and_user_data (i) function. It is a figure showing the syntax of a user data () function. It is a figure explaining Data_ID of the function described by user data. It is a figure explaining the syntax of a V-Phase () function. It is a figure explaining the syntax of an H-Phase () function. It is a figure explaining the syntax of a Picture_Order () function. It is a figure explaining the syntax of Ancillary_data () function. It is a figure explaining the syntax of group_of_picture_header () function. It is a figure explaining the syntax of a picture_header () function. It is a figure explaining the syntax of a picture_coding_extension () function. It is a figure explaining the syntax of a picture_data () function. It is a figure explaining the layer of an MPEG system.

Explanation of symbols

42, 43 MPEG encoder, 44, 45 MPEG decoder, 46 to 49 SDTI-CP interface, 50 SDTI-CP network, 51 SDTI-CP interface, 61, 62 TS MUX / DEMUX, 72, 73 MPEG encoder, 74, 75 MPEG Decoder, 76 to 81 SDTI-CP interface, 8
0 SDTI-CP network

Claims (5)

A first receiving means for receiving a first bitstream in which order information relating to an encoding order of image signals and a display order of the image signals is inserted in units of access units of the image signals;
The sequence information is extracted from the first bit stream received by the first receiving means, a time stamp is generated based on the extracted sequence information, inserted into a second bit stream, and output. And a digital signal transmission device. Second receiving means for receiving the second bitstream;
And a second output means for generating the time stamp from the second bit stream received by the second receiving means, inserting the time stamp into the first bit stream, and outputting the generated time stamp. The digital signal transmission device according to claim 1. The digital signal transmission apparatus according to claim 1, wherein the encoding order is based on a display time of a field of the image signal. A reception step of receiving a first bitstream in which the encoding order of the image signals and the order information regarding the display order of the image signals are inserted in units of access units of the image signals;
An output step of extracting the order information from the first bit stream received in the process of the receiving step, generating a time stamp based on the extracted order information, inserting the time stamp into the second bit stream, and outputting the generated time stamp And a digital signal transmission method comprising: A reception step of receiving a first bitstream in which the encoding order of the image signals and the order information regarding the display order of the image signals are inserted in units of access units of the image signals;
An output step of extracting the order information from the first bit stream received in the process of the receiving step, generating a time stamp based on the extracted order information, inserting the time stamp into the second bit stream, and outputting the generated time stamp A computer-readable program for causing a digital signal transmission apparatus to execute a process including: and a computer-readable program.

Images