KR20010050811A - Apparatus and Method for Processing Timing - Google Patents

Abstract

PURPOSE: A linkage system, an independent timing processing apparatus and its method are provided to simplify interfacing with an element system by calculating timing information through the use of a fixed value of parameter that is set prior to operation of the element system and a timing parameter contained in an element stream in a multiplexer, and by controlling an output timing of a video and an audio stream to an element decoder for synchronization in a demultiplexer. CONSTITUTION: A video buffer(131) buffers video information inputted thereto according to a multiplexing plan. A delay buffer(132) delays input audio information for a predetermined time period to synchronize the audio information with the video information. An audio buffer(133) buffers the audio information inputted thereto from the delay buffer(132) according to the multiplexing plan. A multiplexing part(134) receives the video information from the video buffer(131) and the audio information from the audio buffer(132), and obtains timing information based on a parameter, timing information in an element bit stream and itself information, thereby performing a multiplexing process.

Description

Timing processing device independent of an associated system and its method {Apparatus and Method for Processing Timing}

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a timing processing device and a method thereof independent of an associative system. In particular, the present invention relates to a timing processing device independent of an associative system that simplifies the interface between element systems and processes timing synchronization independently of the element system in a multimedia data processing system. And to a method thereof.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to timing processing in a multiplexer and demultiplexer for video / audio timing synchronization of a multimedia data processing system such as MPEG (Motion Picture Expert Group, MPEG-1 and MPEG-2) system. The multiplexer controls the video and audio encoders in calculating timing information and accepts parameters that change during operation through separate signal lines. In addition, the demultiplexer transmits timing information to the video decoder and the audio decoder for timing synchronization, and these element decoders require separate hardware for timing synchronization. As a result, the interface between the timing processing system (multiplexer and demultiplexer) and the element system is complicated, and there is a problem such that the requirements of the timing processing system must be reflected in the specification of the element system.

Disclosure of Invention The present invention has been made to solve the above problems, and an object thereof is to provide a timing processing apparatus and a method independent of an associated system which processes timing synchronization of data independently of other elements of the system in a multimedia data processing system. .

That is, in the present invention, in order to simplify the interface with the element system and allow the element system to be independently implemented, the multiplexer uses a fixed value parameter set before the operation of the element system and a timing parameter included in the element stream. It is an object of the present invention to provide a timing processing apparatus and a method independent of an associated system that calculate timing information by using the demultiplexer and synchronize the timing of output of the video stream and the audio stream to the element decoder.

1 is an exemplary view of an MPEG codec delay model for explaining the present invention.

2 is a block diagram illustrating a timing processing device and method using a multiplexer and a demultiplexer according to the present invention.

* Explanation of symbols for the main parts of the drawings

110: video encoder 120: audio encoder

130: multiplexer 140: demultiplexer

150: video decoder 160: audio decoder

An apparatus of the present invention for achieving the above object comprises: a timing processing apparatus independent of an associating system, comprising: a video buffer for buffering input video information according to a multiplexing strategy; A delay buffer for delaying the input audio information for a predetermined time for the synchronization processing of the audio information and the video information; An audio buffer for buffering audio information from the delay buffer according to a multiplexing strategy; And receiving video information and audio information from the video buffer and the audio buffer to obtain timing information by using parameters determined before operation, timing information in an element bitstream, and information that can be determined by itself. It characterized in that it comprises multiple processing means.

Another apparatus of the present invention is a timing processing apparatus independent of an associated system, comprising: a jitter buffer for processing compensation of jitter included when a bitstream from a multiplexer is passed through a communication medium; Demultiplex processing means for calculating and controlling an output time of the element stream to synchronize output of video and audio; A video buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and allowing demultiplexing of video information regardless of timing synchronization; An audio buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and allowing demultiplexing of audio information regardless of timing synchronization; And a delay buffer for selectively delaying the video information or the audio information for a predetermined time according to the control of the demultiplexing means in order to synchronize the output of the video and the audio.

In addition, another apparatus of the present invention is a timing processing apparatus that is independent of an associated system, wherein the parameters determined before operation, timing information in the element bitstream, and information that can be determined on its own are controlled without controlling the element encoder. Multiplexing processing means for calculating timing information (DTS, PTS) by using the multiplication processing means; And calculates the output time of the element stream based on the obtained decoding delay time from the element decoding to the output, timing information, and vbv delay time to synchronize video and audio and control the output time of the element stream. It is characterized by including the multiplexing processing means.

On the other hand, the method of the present invention is a timing processing method applied to a timing processing device independent of the associated system, the parameters (M, T _video , d _VE , d _AE , d _VB ) determined before operation and the input element bits A first step of obtaining timing information (t ^I _video (j), t ^I _audio (j), d _vbv (j)) in the _stream , and information that can be determined by itself (d _sys ); A second step of calculating decoding time information (DTS: Decoding Time Stamp) for the video based on the obtained information t ⁱ _video (j), d _vbv (j), d _sys ; M * T _video obtained by multiplying the maximum frame interval (M) and the frame period (T _video ) by the video decoding time information (DTS _video ) or the video decoding time information (DTS _video ) calculated according to the type of processed video. A third step of determining the added value as the video presentation time information (PTS); And for the obtained information (t ^I _audio (j), d _VE , d _AE , d _VB , M * T _video , d _sys ), adding the value of the information excluding the audio encoding delay time (d _AE ) and the audio encoding delay time. and d) calculating a timing information (DTS, PTS) for audio by _subtracting (d _AE ).

In addition, another method of the present invention is a timing processing method applied to a timing processing apparatus independent of an associated system, wherein the video decoding delay time d _VD is controlled to control an output time of an element stream in a demultiplexing process. Determining whether the decoding delay time is greater than d _AD ; As a result of the determination of the first step, if the video decoding delay time d _VD is greater, the video output time t is subtracted from the obtained video decoding time information DTS _{video by} subtracting the VBV delay time d _VBV . second calculating a ^O _video), and calculating the audio decoding time information (DTS _audeo) video decode delay time (d _VD) audio output time (t ^O _audeo) plus limit value for an audio decoding delay (d _AD) in the step; And when the video decoding delay time d _VD is not large as a result of the determination of the first step, the VBV delay time d _VBV is subtracted from the obtained video decoding time information DTS _video and the video decoding delay time is A video output time t ^O _video is calculated by adding the limit value of the audio decoding delay time d _AD at (d _VD ), and the audio output time t ^o _audeo is calculated from the audio decoding time information DTS _audeo . Characterized in that it comprises three steps.

As described above, the present invention synchronizes the decoder final output of video and audio in an MPEG decoder including an MPEG (MPEG-1 and MPEG-2) encoder including a video encoder, an audio encoder, and a multiplexer, a demultiplexer, a video decoder, and an audio decoder. The multiplexer and demultiplexer, which allows Lip Sync, to simplify the interface between element encoders (video encoders, audio encoders) and element decoders (video decoders and audio decoders), and allow the element system to be implemented independently of the timing processor. That's the point.

In real-time multimedia communication, data should be output from the decoder at the same speed as the input of the encoder, and sync between the output media. MPEG-2 transmits timing information such as Program Clock Reference (PCR), Decoding Time Stamp (DTS), and Presentation Time Stamp (PTS). The present invention analyzes the timing of MPEG-2 system and presents the timing synchronization implementation of video / audio (calculation of DTS / PTS value in the multiplexer, timing processing in the demultiplexer, and buffer configuration of the multiplexer and demultiplexer for synchronization). do. Its implementation is timed using only the variable parameters extracted from the input data and the constant parameters set before operation, without requiring any control or communication to the element system for the video / audio synchronization by the multiplexer and demultiplexer. Accordingly, the present invention simplifies the interface between the element system and the multiplexer / demultiplexer, and the element encoder and the element decoder can be independently implemented without regard to the multiplexer and the demultiplexer.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The present invention will be briefly described taking MPEG-2 as an example.

MPEG-2 is an international standard that defines coding and multiplexing for video and audio. The compression layer of MPEG-2 defines compression encoding for video and audio, and the system layer defines multiplexing of video data and audio data. Multiplexing includes timing information for encoder / decoder synchronization and video / audio synchronization.

An embodiment of the present invention implements timing in a system layer for synchronizing and outputting video and audio in an implementation of an MPEG-2 system (calculating timing parameters in a multiplexer, processing these timing information in a demultiplexer, and synchronizing). Multiplexer and demultiplexer buffers).

Although methods have been proposed for synchronizing video and audio in practical implementations where the encoder and decoder have delays, these methods allow multiplexers to control the operation of element encoders (video encoders, audio encoders), and demultiplexers control element decoders (video). Decoders and audio decoders use only constant parameters determined before operation without any control or communication with the element system for timing processing, simplifying the modularity and interface of the MPEG-2 system and allowing the element system to be implemented independently. .

The timing of the MPEG system is as follows.

1 is an exemplary view of an MPEG codec delay model for explaining the present invention.

The MPEG encoder is divided into a video encoder 110 for compressing video data, an audio encoder 120 for compressing audio data, and a multiplexer 130 for multiplexing these data with timing information. A demultiplexer 140 for demultiplexing and timing data, a video decoder 150 for decoding video data, and an audio decoder 160 for decoding audio data are divided.

Here, the video encoder 110 is an encoder rearrange buffer 111 for delaying a B picture for adjusting the picture order of encoding, a video encoder 112 for encoding at a variable rate, and a channel for outputting a variable rate of encoded bits at a fixed rate. It is divided into a buffer 113, the video decoder 150 is a VBV (VBV: Video Buffering Verifier) buffer for outputting the input bit at a fixed rate, a video decoder for reading and decoding at a variable rate from the VBV buffer, I It is divided into a decoder reordering buffer that delays the picture and the P picture by the B picture in the encoder.

The video encoder 112 encodes an input picture using an I picture (picture encoding mode for encoding using only intra-frame information) and a P picture (forward prediction picture encoding mode) with reference to a previous frame in the input order. Picture mode) or B picture (Bi-direction predictive picture encoding mode), which is encoded by referring to a previous frame and a next frame in the input order. A picture encoded as a picture is encoded as input to a video encoder, but in the case of a B picture, encoding is possible after encoding a P picture that is subsequently input. The encoder rearrangement buffer 111 shown in FIG. 1 is a buffer for storing such a B picture.

In the video decoder 150, in order to output a picture in the same order as the input sequence of the video encoder, the B picture is presented as it is decoded, and the I or P picture is delayed and output by the decoder as much as the B picture is delayed in the encoder.

The video encoder 110 changes the encoding bit generation rate of the picture according to the coding modes (I, P, and B pictures) of the picture and the characteristics of the input video. However, the output of video encoder 110 is a constant bit rate. The channel buffer 113 located in the video encoder 110 is a buffer for this. The output bitstream of the video encoder 110 includes timing information such as the output bit rate (bit_rate), frame rate (frame_rate), VBV buffer size (vbv_buffer_size), and VBV delay (vbv_delay), and the video encoder 110 includes these parameters. When a virtual decoder (VBV: Video Buffering Verifier) operating in accordance with the decoding should generate such a bitstream that operates normally without overflowing or exhausting the VBV buffer.

Here, vbv_delay transmitted for each picture represents a time to wait in the VBV buffer from the time when the first byte of the corresponding picture data is input to the VBV decoder until the VBV decodes. In this case, the VBV buffer input rate of the data is included in the bitstream. Bit_rate, and the buffer size is vbv_buffer_size. On the other hand, the video decoder should have a buffer of such a size that it can decode to VBV and decode any problem-free bitstream, as shown in FIG.

Unlike video in which the input order and encoding order are not the same and the encoding bit rate of the picture is not constant, the audio is encoded in the order of the input frames, and the number of encoding bits per frame is also input data sampling rate (sampling_frequency) and output data rate (bit_rate). Is a fixed value determined by Accordingly, the audio is not distinguished in the delay model of FIG. 1.

In the real-time multimedia service, all data in the media must have the same delay from the input point of the encoder to the output point of the decoder, and the timing synchronization (Lip Sync) must be performed between the media. In the system section of the MPEG-1 and MPEG-2 standards defined for the multiplexer 130 and the demultiplexer 140 of FIG. 1, SCR (SCR) is used for timing synchronization with the encoder 110 in the decoder 150. Decoding Time Stamp (DTS) and PTS (TTS) for timing information called System Clock Reference (PCR) or Program Clock Reference (PCR) and timing synchronization of video and audio at decoder 150. It defines the timing information called Presentation Time Stamp.

SCR is used in the MPEG-1 and MPEG-2 PS (Program Stream) standards specified for digital storage media, and PCR is defined in TS (Transport) for applications that may cause transmission errors such as broadcasting and communication. Stream) is used in the specification. DTS and PTS are timing variables that are equally applied to PS and TS of MPEG-1 and MPEG-2, and the decoding time of an access unit (AU), which is the unit data of timing, and the PU, which is the decoding data of AU, The presentation time of a unit is shown. The MPEG decoder restores the system clock of the encoder through PCR and synchronizes the output of video and audio through DTS and PTS.

MPEG defines a virtual decoder called STD (System Target Decoder) and access unit (AU), which is unit data of timing of video and audio, for reference of these timings. The video is the picture, the audio is the AU, and the STD decodes the AU data instantaneously at the DTS time point and outputs it instantaneously at the PTS time point. Also, the AU output immediately after decoding, such as B picture or audio frame of video, should have the same DTS and PTS value when decoding, and the I or P picture of the video undergoing picture rearrangement delay in the decoder should have the DTS and PTS values of the picture period. It should be a drain. The multiplexer, which is the timing processor of the encoder, must calculate and transmit the DTS and PTS for the video AU and the audio AU so that the video and audio are synchronized when the virtual decoder decodes. The demultiplexer, which is the timing processor of the MPEG decoder, The PTS information should be used for synchronization processing. The STD also includes provisions for buffer size for multiplexing video, audio, and system information, buffering data of each buffer, input / output ratio of each buffer, system clock frequency precision of the encoder, STD input rate, and the like.

In the present invention, a DTS and PTS calculation scheme in a multiplexer, a timing processing scheme of a demultiplexer in an actual decoder having a delay, and a buffer configuration of a multiplexer and a demultiplexer for such timing processing are described.

First, the timing information (DTS, PTS) calculation method of the multiplexer will be described as follows.

The processing for timing synchronization of video and audio in the multiplexer 130 is to obtain the values of DTS and PTS for the AU of the video and the AU of the audio, and plant the values in the packetized elementary stream (PES) header of the bitstream. .

The DTS and PTS values in the multiplexer 130 should set values such that video and audio are synchronized and output while satisfying buffering conditions when the STD decodes them. The following is a formula of DTS and PTS values for AU of video and audio of the present invention.

PTS _video (j) = DTS _video (j) for B picture

= DTS _video (j) + M * T _video I / P picture

PTS _audio (j) = DTS _audio (j)

The video DTS is based on the point in time at which the first byte of the video AU is input to the multiplexer 130 (t ^I _video (j)) and the vbv_delay (d _vbv (j)) and d _sys (multiplexer delay (d _mux ) of the video AU. ) Is the sum of the demultiplexer delay (d _demux ) of the STD, and the video PTS is the maximum frame interval between the picture encoding mode (I, P, B picture) and the reference frame (I or P frame) based on the DTS. (M) is determined according to the frame period (T _video ). Equation [2], which is the equation for the video PTS, indicates that the video encoder 110 delays the B picture by M * T _video and encodes the I and P pictures by the decoder. Occurs.

Equations 3 and 4 are equations for DTS and PTS of audio. Where d _VB is the sum of d _Vbuf and d _vbv (j) and is a fixed value. For synchronization, the total delay from the encoder input point of the audio to the output point PTS of the STD decoder should be equal to the total delay of the video. Therefore, the DTS value of the audio becomes the value obtained by adding (Video Total Delay-d _AE ) based on the first byte input time of the AU as shown in [Equation 3]. Where D _VE is the delay in the video encoder and D _AE is the delay in the audio encoder 120.

The above DTS / PTS operation includes the parameters (M, T _video , d _VE , d _AE , d _VB ) determined before the operation and the input data (t ^I _video (j), t ^I _audio (j), d We can see that only parameters that can be either _vbv (j)) or self-determined (d _sys ) are used. Here, the multiplexer delay (d _mux ) can be known by itself, and the demultiplexer delay (d _demux ) can be derived by the STD specification. Accordingly, the DTS and PTS operations according to the present invention do not need to accept parameters that are changed during operation through separate signal lines, and the multiplexer 130 does not need to control the video encoder 110 and the audio encoder 120.

Next, the method of calculating the element stream output time point in the demultiplexer 140 is as follows.

This is a description of the video / audio timing synchronization processing in the demultiplexer 140. The present invention does not deal with the system clock recovery of the demultiplexer 140. In the description herein, the demultiplexer 140 uses the SCR in the case of the MPEG-1 and MPEG-2 program streams (PS). In the case of TS (Transport Stream), it is assumed that the system clock has already been restored by PCR. The system clock recovery of the decoder refers to the restoration of the timing clock used in the encoder and the setting of the reference clock for DTS and PTS processing. In the input data of the demultiplexer 140, a DTS and a PTS are planted so that video and audio are synchronized and output when the STD is decoded. Timing processing of the demultiplexer 140 allows the final outputs of the video decoder 150 and the audio decoder 160 having different delay values to be synchronized from the timing information of the STD from the video and audio decoder operating rules. The present invention synchronizes the final output of video and audio in such a way that the demultiplexer 140 controls the output timing of the element stream (video stream and audio stream). The output times t ⁰ _video (j) and t ^O _audio (j) of the demultiplexer 140 of the video and audio AU according to the delay model of FIG. 1 are as follows.

If d _VD 〉 d _AD

t ^O _video (j) = DTS _video (j)-d _vbv (j)

d _AD 〉 d _VD

t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD )

If d _VD 〉 d _AD ,

t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD )

If d _AD > d _VD ,

t ^O _audio (j) = DTS _audio (j)

Here, d _VD and d _AD are the time from the decoding of the element decoders (video decoder and audio decoder) to the output as the decoding delay time and do not include the picture rearrangement delay of the video.

d _VD and d _AD are constants provided in the implementation specifications of the video decoder 150 and the audio decoder 160 connected to the demultiplexer 140, and d _vbv (j) is a variable that varies depending on the AU of the video. Decode and extract the picture header of.

According to Equations 5 and 6, the demultiplexer 140 supplies the element stream to the element decoder, and the element decoder decodes and outputs the AU as if it were connected to the element encoder. Video and audio are synchronized. Of course, the video decoder 150 may output the rearranged pictures by itself.

Next, a buffer configuration of the multiplexer 130 and the demultiplexer 140 for timing processing according to the present invention will be described with reference to FIG. 2.

2 is a block diagram illustrating a timing processing apparatus using a multiplexer and a demultiplexer according to the present invention.

The multiplexer 130 and the demultiplexer 140 need a buffer for multiplexing and demultiplexing video, audio, and system information regardless of the timing synchronization of video and audio. In FIG. 2, the video buffers 131 and 143 and the audio buffers 133 and 144 of the multiplexer 130 and the demultiplexer 140 are the same. The size of these buffers is determined by their multiplexing strategy in the multiplexer 130, and the demultiplexer 140 should be a size capable of demultiplexing any bitstreams satisfying the STD.

The jitter buffer in the demultiplexer 140 is a buffer for compensation of jitter included when the bitstream output from the multiplexer 130 is transferred to the demultiplexer 140 through a medium such as an ATM network. Maximum jitter time * TS bit rate * 2). In FIG. 2, the remaining buffers (delay buffers 132 and 146 in the multiplexer and demultiplexer) are buffers for implementing the present invention for synchronizing video and audio. Multiple processing of the multiplexer 130 and demultiplexing of the demultiplexer 140 are provided with constant parameters for video / audio timing synchronization.

The delay buffer 132 in the multiplexer 130 is described as follows.

The output bitstream of the multiplexer 130 should be decoded without buffer overflow in the STD. As shown in Equation 3, audio includes a large time delay for synchronization with video. The delay of the video is accepted in the buffer in the video system but in the case of audio there is no buffer for the delay in the audio system and there is no buffer in the STD for this. Therefore, audio must be delayed by putting a buffer in the multiplexer 130 after all. The size of the buffer is determined by specifying the maximum allowable value of these variables as [(d _VE -d _AE + d _VB + M * T _video + d _sys ) * audio encoding bit rate].

The delay buffer 146 in the demultiplexer 140 is a buffer for the synchronization processing of Equations 5 and 6. d _VD is on the side is larger audio than _{d AD [(d VD - d} AD) * The audio encoding bit rate; a buffer size, a d _AD is larger than the d _VD video path _{[(d AD - d VD)} * video encoding bit rate; A buffer of size is required. The decoding delay d _VD and d _AD may be within 5 ms in the field picture mode for video and 24 ms (frame period) at a sample rate of 48 kHz for audio. These values are the minimum delays of video and audio, respectively. The minimum delay is large audio, but video may be large depending on the application. Figure 2 is to be configured as a video delay buffer or an audio delay buffer in one buffer to enable any case.

The present invention described above is not limited to the above-described embodiments and the accompanying drawings, and various substitutions, modifications, and changes can be made in the art without departing from the technical spirit of the present invention. It will be apparent to those of ordinary knowledge.

The present invention as described above is a method that does not require information exchange for separate timing with element encoders (video encoders and audio encoders) and element decoders (video decoders and audio decoders) during timing processing of the multiplexer and demultiplexer. The interface between the processor (multiplexer and demultiplexer) and the element system is simplified, and the element system can be developed independently of the timing processor. In addition, the element decoder does not require any hardware for video / audio timing synchronization.

Claims (18)

A timing processing device independent of an associated system, A video buffer for buffering input video information according to a multiplexing strategy; A delay buffer for delaying the input audio information for a predetermined time for the synchronization processing of the audio information and the video information; An audio buffer for buffering audio information from the delay buffer according to a multiplexing strategy; And Video information and audio information are received from the video buffer and the audio buffer, and the timing information is obtained using parameters determined before operation, timing information in an element bitstream, and information that can be determined by itself. Treatment A timing processing device independent of an associated system comprising a. The method of claim 1, The delay buffer is, Delay time in channel buffer (d _Vbuf ), VBV (Video Buffering Verifier) delay time (d _VBV ), video encoder delay time (d _VE ), maximum frame interval (M) and frame period (T _video ) Independent of the associating system, which determines the size of the buffer by summing the time obtained by multiplying the M * T _video and the system delay time d _sys and multiplying the audio encoding bit rate by subtracting the audio encoder delay time d _AE . Timing processing unit. The method according to claim 1 or 2, The multi-processing means, Without controlling the element encoder, parameters determined before operation (M, T _video , d _VE , d _AE , d _VB ) and timing information in the input element bitstream (t ^I _video (j), t ^I _audio (j) ), d _vbv (j)), and the timing information (DTS (Decoding Time Stamp), PTS (Presentation Time Stamp)) is calculated using the information (d _sys ) that can be determined by itself according to the following equation. And a timing processing device independent of the associating system. DTS _video (j) = t ^I _video (j) + d _vbv (j) + d _sys PTS _video (j) = DTS _video (j) for B picture For DTS _video (j) + M * T _video I / P pictures DTS _audio (j) = t ^I _audio (j) + (d _VE -d _AE ) + d _VB + M * T _video + d _sys PTS _audio (j) = DTS _audio (j) A timing processing device independent of an associated system, A jitter buffer for processing the compensation of jitter included when the bitstream from the multiplexer is passed through the communication medium; Demultiplex processing means for calculating and controlling an output time of the element stream to synchronize output of video and audio; A video buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and allowing demultiplexing of video information regardless of timing synchronization; An audio buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and allowing demultiplexing of audio information regardless of timing synchronization; And Delay buffer for selectively delaying video information or audio information for a predetermined time under the control of the demultiplexing means to synchronize output of video and audio A timing processing device independent of an associated system comprising a. The method of claim 4, wherein The jitter buffer is, A jitter buffer having a buffer size of a value doubled by multiplying a maximum delay time by a transport stream (TS) bit rate. The method according to claim 4 or 5, The demultiplex means, Based on the obtained decoding time (d _VD , d _AD ), timing information (DTS _video , DTS _audio ), and vbv delay time (d _vbv ) from the obtained element decoding to the output, the output time of the element stream according to the following equation. A timing processing device independent of an associative system for synchronizing video and audio and controlling the output time of the element stream by calculating the " If d _VD 〉 d _AD t ^O _video (j) = DTS _video (j)-d _vbv (j) t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD ) d _AD 〉 d _VD t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD ) t ^O _audio (j) = DTS _audio (j) The method of claim 6, The delay buffer, Check the decoding delay time (d _VD , d _AD ) from the decoding to the output. If the video decoding delay time (d _VD ) is large, the difference between the video and audio decoding delay time for the audio stream from the audio buffer is converted into the audio encoding bit rate. It acts as a buffer of the size obtained by multiplying, and if the audio decoding delay time (d _AD ) is large, it acts as a buffer of the size obtained by multiplying the difference between audio and video decoding delay times by the video encoding bit rate for the video stream from the video buffer. And a timing processing device independent of the associating system. A timing processing device independent of an associated system, Multiplexing processing means for processing timing by calculating timing information (DTS, PTS) using parameters determined before operation, timing information in the element bitstream, and information that can be determined by itself, without controlling the element encoder; And Demultiplexing to synchronize the video and audio and to control the output time of the element stream by calculating the output time of the element stream based on the obtained decoding delay time, timing information, and VBV delay time from the obtained element decoding to the output. Treatment A timing processing device independent of the associated system comprising a. The method of claim 8, The multiplexing processing means, A first video buffer for buffering input video information according to a multiplexing strategy; A first delay buffer for delaying the input audio information for a predetermined time for the synchronization processing of the audio information and the video information; A first audio buffer for buffering audio information from the first delay buffer according to a multiplexing strategy; And Video information and audio information are received from the first video buffer and the first audio buffer, and timing information is obtained by using parameters determined before operation, timing information in an element bitstream, and information that can be determined by itself. Multiple processing means to carry out A timing processing device independent of an associated system comprising a. The method according to claim 8 or 9, The demultiplexing means, Jitter buffer for processing the compensation of jitter included when the bitstream from the multiplexing means is conveyed through a communication medium; Demultiplex processing means for calculating and controlling an output time of the element stream to synchronize output of video and audio; A second video buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and enabling demultiplexing of video information regardless of timing synchronization; A second audio buffer having a size capable of demultiplexing any bitstream satisfying the virtual decoder STD and enabling to demultiplex audio information regardless of timing synchronization; And A second delay buffer for selectively delaying the video information or the audio information for a predetermined time according to the control of the demultiplexing means to synchronize the output of the video and the audio; A timing processing device independent of an associated system comprising a. A timing processing device independent of an associated system, Without controlling the element encoder in the multiplexer, timing information (DTS, PTS) is calculated according to the following equation using only parameters determined before operation, timing information in the element bitstream, and information that can be determined by itself. A timing processing device independent of an associated system that handles timing. DTS _video (j) = t ^I _video (j) + d _vbv (j) + d _sys PTS _video (j) = DTS _video (j) for B picture For DTS _video (j) + M * T _video I / P pictures DTS _audio (j) = t ^I _audio (j) + (d _VE -d _AE ) + d _VB + M * T _video + d _sys PTS _audio (j) = DTS _audio (j) A timing processing device independent of an associated system, In the case of controlling the output time of the element stream in the demultiplexer, a timing processing apparatus independent of an associated system for synchronizing video and audio by calculating an output time of the element stream according to the following equation. If d _VD 〉 d _AD t ^O _video (j) = DTS _video (j)-d _vbv (j) t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD ) d _AD 〉 d _VD t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD ) t ^O _audio (j) = DTS _audio (j) A timing processing device independent of an associated system, Without controlling the element encoder in the multiplexer, timing information (DTS, PTS) is obtained using only parameters determined before operation, timing information in the element bitstream, and information that can be determined by itself. DTS _video (j) = t ^I _video (j) + d _vbv (j) + d _sys PTS _video (j) = DTS _video (j) for B picture For DTS _video (j) + M * T _video I / P pictures DTS _audio (j) = t ^I _audio (j) + (d _VE -d _AE ) + d _VB + M * T _video + d _sys PTS _audio (j) = DTS _audio (j) The timing is calculated according to the following equation, When the demultiplexer controls the output time of the element stream If d _VD 〉 d _AD t ^O _video (j) = DTS _video (j)-d _vbv (j) t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD ) d _AD 〉 d _VD t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD ) t ^O _audio (j) = DTS _audio (j) A timing processing device independent of an associative system for synchronizing video and audio by calculating an output time point of an element stream according to the following equation. A timing processing method applied to a timing processing device independent of an associated system, Parameters determined before operation (M, T _video , d _VE , d _AE , d _VB ) and timing information in the input element bitstream (t ^I _video (j), t ^I _audio (j), d _vbv (j)) And a first step of obtaining information d _sys which can be determined by itself; A second step of calculating decoding time information (DTS: Decoding Time Stamp) for the video based on the obtained information t ⁱ _video (j), d _vbv (j), d _sys ; M * T _video obtained by multiplying the maximum frame interval (M) and the frame period (T _video ) by the video decoding time information (DTS _video ) or the video decoding time information (DTS _video ) calculated according to the type of processed video. A third step of determining the added value as the video presentation time information (PTS); And For the information obtained (t ^I _audio (j), d _VE , d _AE , d _VB , M * T _video , d _sys ), the value of the information minus the audio encoding delay time (d _AE ) is added and the audio encoding delay time ( a fourth step of calculating timing information (DTS, PTS) for audio by _subtracting d _AE ) Timing processing method comprising a. A timing processing method applied to a timing processing device independent of an associated system, A first step of determining whether the video decoding delay time d _VD is greater than the audio decoding delay time d _{AD in} order to control the output time of the element stream in the demultiplexing process; As a result of the determination of the first step, if the video decoding delay time d _VD is greater, the video output time t is subtracted from the obtained video decoding time information DTS _{video by} subtracting the VBV delay time d _VBV . second calculating a ^O _video), and calculating the audio decoding time information (DTS _audeo) video decode delay time (d _VD) audio output time (t ^O _audeo) plus limit value for an audio decoding delay (d _AD) in the step; And As a result of the determination in the first step, if the video decoding delay time d _VD is not large, the VBV delay time d _VBV is subtracted from the obtained video decoding time information DTS _video and the video decoding delay time ( d _VD ) by adding the limit value of the audio decoding delay time d _AD to calculate a video output time t ^O _video and a third audio output time t ^O _audeo to calculate the audio decoding time information DTS _audeo . step Timing processing method comprising a. A timing processing method applied to a timing processing device independent of an associated system, Without controlling the element encoder in the multiplexer, timing information (DTS, PTS) is calculated according to the following equation using only parameters determined before operation, timing information in the element bitstream, and information that can be determined by itself. A timing processing method that is independent of an associated system that handles timing. DTS _video (j) = t ^I _video (j) + d _vbv (j) + d _sys PTS _video (j) = DTS _video (j) for B picture For DTS _video (j) + M * T _video I / P pictures DTS _audio (j) = t ^I _audio (j) + (d _VE -d _AE ) + d _VB + M * T _video + d _sys PTS _audio (j) = DTS _audio (j) A timing processing method applied to a timing processing device independent of an associated system, When the demultiplexer controls the output time of the element stream, a timing processing method independent of an associated system for synchronizing video and audio by calculating the output time of the element stream according to the following equation. If d _VD 〉 d _AD t ^O _video (j) = DTS _video (j)-d _vbv (j) t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD ) d _AD 〉 d _VD t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD ) t ^O _audio (j) = DTS _audio (j) A timing processing method applied to a timing processing device independent of an associated system, Without controlling the element encoder in the multiplexer, timing information (DTS, PTS) is obtained using only parameters determined before operation, timing information in the element bitstream, and information that can be determined by itself. DTS _video (j) = t ^I _video (j) + d _vbv (j) + d _sys PTS _video (j) = DTS _video (j) for B picture For DTS _video (j) + M * T _video I / P pictures DTS _audio (j) = t ^I _audio (j) + (d _VE -d _AE ) + d _VB + M * T _video + d _sys PTS _audio (j) = DTS _audio (j) The timing is calculated according to the following equation, When the demultiplexer controls the output time of the element stream If d _VD 〉 d _AD t ^O _video (j) = DTS _video (j)-d _vbv (j) t ^O _audio (j) = DTS _audio (j) + (d _VD -d _AD ) d _AD 〉 d _VD t ^O _video (j) = DTS _video (j)-d _vbv (j) + (d _AD -d _VD ) t ^O _audio (j) = DTS _audio (j) A timing processing method independent of an associated system for synchronizing video and audio by calculating an output time point of an element stream according to the equation