KR960013217B1 - Video and audio signal synchronization apparatus for mpeg decoder system - Google Patents

Abstract

The apparatus uses a display time value and an audio generating time value every time difference corresponding to the least common multiple for a video frame difference. The MPEG system including the apparatus comprises a demultiplexor(60) for extracting the least common multiple and drawing a video and audio frame; a buffering devices(70,80) to store temporarily the video and audio frame drawn; a system time clock(STC)(90) initialized with a system clock reference; a video and audio present time stamp register(100) for storing the video displaying time and the audio generating time; a comparator(110) for generating a pulse at a synchronizing point; a video and audio time interval generator(140) for generating strobe signals; and a video and audio decoder(160) for outputting the video and audio frame with the pulse according to the certain frame difference.

Description

Video and Audio Signal Synchronizer in MPEG Coding System

1 is a diagram illustrating a display and decoding order of frames in an MPEG system.

2 is a schematic block diagram of a conventional MPEG decoding system.

3 (a) and (b) show video and audio decoding and display procedures of a conventional MPEG decoding system.

4 illustrates an MPEG decoding system according to the present invention.

FIG. 5 is a diagram showing a video and audio decoding order and a display order realized through the preferred embodiment of the present invention of FIG.

* Explanation of symbols for main parts of the drawings

10,40,70,80: Buffer 20,140: Video decoder

50,150: voice decoder 60: DMUX

90: system time clock 100: video and audio performance time stamp register

110: comparator 120: video time interval generator

130: voice time interval generator

The present invention relates to an apparatus for synchronizing video and audio signals in an MPEG decoding system. In particular, the present invention relates to a video and audio signal of an MPEG decoding system that receives a bit stream of video and audio signals transmitted in an irregular frame order and synchronizes the video and audio signals. It relates to a synchronizer.

Moving picture experts group (MPEG) includes compact disks (CDs), digital audio tapes (DAT) and CD-ROMs (CD read only memory) with continuous transmission rates up to 1.5M bits / sec; CD-ROM) or a method of compressing video and audio onto digital recording media such as magnetic hard disks.

Referring to FIG. 1, it can be seen that in the system using the MPEG compression method, transmission is performed while the order of the frames is changed. For example, the order of the transmitted frames (decoding order) is intra frame (I) frame, predicted frame (P) frame, bidirectionally predicted frame (B frame), frame B, frame P, frame B, frame B, frame P. Like a frame, these frames are grouped together to form a group of pictures (GOP). In addition, the display order is indicated by the frame order such as I, B, B, P, B, B, P, and the like. In the above, the I frame represents a screen to which only intraframe coding is applied, and the P frame is a frame that is predictively coded by forward motion estimation / compensation from a previous I frame P frame. Indicates.

In addition, the B frame represents a frame that is predictively encoded by bidirectional motion compensation from two frames, i and P frames before and after. As such, if the MPEG system that changes the frame order does not have any external device or control device, video and audio are not synchronized.

2 shows a schematic block diagram of a typical MPEG decoding system. As shown, a video section of a typical MPEG video decoding system includes a buffer 10, a decoder 20, and a RAM 21, and an audio section includes a buffer 40, a decoder 50, and a RAM 51. do.

Conventional image and speech processing techniques configured as described above are performed by compressing or stretching an image in real time and displaying the same, and simultaneously compressing or stretching an audio in real time.

However, since motion compensation is performed bidirectionally for a very high compression ratio, the transmission order (decoding order) is different as described above, and thus real-time compression is difficult. In addition, even though real-time compression is performed, frame delay is inevitably generated, and in order to synchronize the audio with the audio, the audio must be synchronized with the video in such a manner that the audio is delayed in accordance with the video.

As shown in Fig. 3A, the video signal starts decoding one frame or displays the display in accordance with the vertical synchronization signal Vsync, which is a reference signal. The IO frame starts decoding in accordance with the vertical synchronization signal, ends decoding within one frame interval, and is stored in the frame rearrangement memory 21. Decode the next P3 frame transmitted in accordance with the next vertical synchronization. At this time, the P3 frame performs motion compensation based on the IO frame. The decoded P3 frame is also stored in the frame rearrangement memory 21. Display the IO frame while decoding the B1 frame in accordance with the next vertical sync. At this time, the B1 frame is compensated for motion by using the IO frame and the P3 frame stored in the rearrangement memory. Also, the B1 frame is displayed while decoding the B2 frame in accordance with the next vertical synchronization. In this way, the B2 frame is displayed while decoding P6 at the next vertical sync, and the P3 frame is displayed only after the next vertical sync. In MPEG, the instantaneous value of the time when this vertical sync pulse occurs can be used as a complex image and a display timing signal.

Referring to Fig. 3 (b), the MPEG voice decoder is implemented to decode one frame during one voice frame period and simultaneously play the previous frame. For example, AO will play when the AO frame is decoded and the next frame A1 is decoded.

The MPEG video and audio system as described above is designed to multiplex and transmit or store data of video and audio in packets. The video and audio are separated by a demultiplexer and transmitted to the video decoder and the audio decoder, respectively. Accordingly, the video and audio decoder receives each bitstream and starts decoding.

Therefore, in the conventional technology operating as described above, there is a problem in that the display of the image and the generation of audio do not coincide due to the decoding delay of the video frame at the moment when the IO and the AO are displayed, as shown in the above-described drawings.

Accordingly, an object of the present invention is to solve the above-described problem, and an object of the present invention is to synchronize the video (screen) and audio signals transmitted by changing the order of frames in the MPEG decoding system to a minimum common multiple of the video frame interval and the audio frame interval. The present invention provides an apparatus for synchronizing video and audio signals of an MPEG decoding system in which a display time value of an image and an audio generation time value of an audio are matched at corresponding time intervals.

The apparatus for synchronizing video and audio signals of an MPEG decoding system according to the present invention receives a bit stream of video and audio signals and synchronizes the video and audio signals in a MEPE system in which the order of video frames is changed. And a demultiplexer (DMUX) for extracting a video and audio frame by receiving the bit stream and extracting a least common multiple of time information and a frame interval value at which the video and audio are to be displayed. A buffer means for temporarily receiving video and audio frame data extracted from DMUX for a predetermined time, and a system time clock initialized by a system clock reference extracted from the DMUX to provide a reference time of the MPEG system. (STC), the display time value of the image extracted from the DMUX and the sound of the voice A video and audio presentation time stamp register (PTSR) for storing time values when the inter-values match, a system time clock value output from the system time clock, and a video and audio frame output from the PTSR. Comparison means for generating a predetermined pulse by comparing a common display value, and initializing when a predetermined pulse is generated at the output of the comparing means, and counting continues from this initializing time. Video time interval generater (VTIG) means for generating a strobe signal when generating a pulse by the frame interval value provided from the DMUX, and a predetermined pulse is generated at the output of the comparison means. Is initialized, and counting continues from the initializing time by one frame interval value provided from the DMUX. An audio time interval generator generating a strobe when issuing a pulse once; ATIG) and audio time interval generator (ATIG) for generating a strobe when a pulse is issued once by a predetermined frame interval value from the generator after decoding the video and audio frames passing through the buffer means. Means for decoding the video and audio frames that have passed through the buffer means and for outputting video and audio frames by means of pulses provided by the frame interval values from the generator. Characterized in that it comprises a.

Hereinafter, described in detail with reference to the accompanying drawings of the present invention.

4 is a detailed block diagram illustrating an MPEG decoding system according to the present invention. Such MPEG decoding systems include demultiplexer (DMUX) 60, buffers 70 and 80, system time clock (STC) 90, video and audio performance time stamp registers, and audio presentation time stamp register (VAPTSR) 100, comparator 110, video time interval generator (VTIG) 120, voice time interval generator (ATIG) 130, video decoder ( 140, voice decoder 150, RAM 141, and 151.

The DMUX 60 receives the bitstream transmitted through the channel by MUX through a multiplexer (MUX) of an encoder (not shown) and has its own time to be displayed through each video and audio frame. The time information is extracted and provided to the blocks 90, 100, 120, and 130, respectively. Also, the video and audio frames are stored in the buffers 70 and 80 for a predetermined time, respectively.

For example, by separating the video and audio bit frames, the value L between the display instant of the video frame and the playing instant of the audio frame is matched at predetermined time intervals corresponding to the least common multiple of the video frame time interval and the audio frame time interval. The value is stored in the VAPTSR (100). Here, the time corresponding to the minimum common multiple of the video frame interval and the audio frame interval is a moment at which the start time of the video frame and the audio frame coincide. For example, if the video frame interval is 40 msec and the audio frame interval is 24 msec, it occurs once every 120 msec. A system clock reference (SCR) signal, which is reference time information, is extracted from the system bit stream received by the DMUX to adjust the system time clock. A counter for providing a reference time of the system with the system time clock. It is initialized by the SCR value and operates while continuously increasing the value by receiving a system clock of 90 KHz.

The VAPSTR 100 receives and stores time information provided from the DMUX 60, respectively. The time information stored for the predetermined time in the above-described register 100 is provided by the DMUX 60 and updated.

Comparator 110 is a system time value 91 and VAPTS output from the STC (90)

Compare the time value 101 at the instant when the strobe signal of the video and audio frame output from the R 100 is matched. For example, when the values 91 and 101 to be compared coincide with each other, a pulse ( pulse) and simultaneously transmits to the VTIG 120 and the TIG (130).

The VTIG 140 is initialized when a pulse is generated in the comparator 110. In other words, it operates only when the system time clock value 91 and the performance time value 101 coincide. The VTIG 140 consists of a programmable counter, which counts continuously from the moment it is initialized, pulses once every 40ms for a given interval, i.e. 25 frames per second, for a video frame interval. Continue to occur.

In addition, the time interval, that is, the frame interval value, is transmitted from the DMUX 60 at system startup. In addition, when the VTIG 140 receives a PTS-O from the DMUX 60 and is initialized in accordance with the STC, the VTIG 140 continuously generates a pulse corresponding to the instant of vertical synchronization. Generate.

In operation, after passing a corresponding value to the block 100 again after a special function such as freezing, the comparator 120 again returns the system time clock value 91 from the STCR 90 and the VAPTSR 100 and the minimum. The VTIG 140 is reinitialized by comparing the common multiple time value 101. As described above, the time value corresponding to the least common multiple of the video and audio frame intervals (PTS-1, PTS-) once each time corresponds to the least common multiple of the video frame time interval and the audio frame time interval. 2,) may be delivered to the DMUX 60 by the receiver.

The image decoder 140 decodes the image frame stored in the above-described buffer 70 for a predetermined time and then sequentially frames by the time interval control signal provided from the VTIG 120, that is, the display strobe signal. Output the data and display it. The RAM 141 stores the decoded frame in the video decoder 140.

Audio frame processing is the same as the above-described image frame processing. That is, the comparator 110 compares the system time clock value 91 output from the STC 90 with the least common multiple value 101 of the time intervals of the video and audio frames output from the VAPTSR 100. When the values 91 and 101 coincide with each other, the pulse is transmitted to the ATIG 130 to be initialized, and the ATIG 130 continuously generates the pulse at the voice frame time interval.

ATIG 130 operates only when a pulse is generated from the comparator 110 as described above, i.e., only when the system time clock signal 91 and the minimum gap time value 101 coincide. The VTIG 120 and the ATIG 130 are composed of a means for initializing and a programmable counter, and the counting interval continues for a given interval, i.e., a time interval of, for example, 24 ms, from the moment of initializing. Then, a voice generating strobe signal is generated and transmitted to the voice decoder 150.

The voice decoder 150 receives and decodes the voice bit stream stored for a predetermined time in the transmitted buffer 80 and stores the decoded voice bit stream in the RAM 151. At this time, the DMUX 60 adjusts the transmission rate such that the decoded voice frame is depleted or overflowed in the RAM 151 as described above.

FIG. 5 is a diagram showing that the start points of the vertical synchronization signal Vsync and the image display frame and the voice generation frame are realized according to another embodiment of the present invention of FIG. 4 described above. Referring to FIG. 4 again, the image and audio decoders 140 and 150 display or display the voice generating strobe signal at the time intervals set in the DMUX 60, that is, in the frame order shown in FIG. Provides video and audio frames to display and generate audio through monitor or speaker.

As described above, according to the present invention, the MPEG decoding system has a great advantage in that video and audio of higher quality can be viewed by synchronizing the video (screen) and audio signal transmitted by changing the frame order.

Claims (4)

Receives and decodes a bit stream of video and audio signals in a moving picture expert group (MPEG) system in which video frame order is changed, and synchronizes the decoded video and audio signals. A demultiplexer (DMUX) 60 which receives and extracts a video and audio frame, and extracts a minimum common multiple of time information and frame interval value at which the video and audio are to be displayed; Buffer means (70) and (80) for temporarily receiving the video and audio frames extracted from the DMUX (60) for a predetermined time; A system time clock (STC) 90 initialized by a system clock reference extracted from the DMUX 60 to provide a reference time of the MPEG system; A video and audio present time stamp register (VAPTSR) 100 for storing a display time value of an image extracted from the DMUX 60 and a voice generation time value of an audio; Comparing means 110 for comparing the system time clock value 91 output from the system time clock 90 and the register 100 with the video and audio frame display time value 101 to generate a predetermined pulse at the same time. and; It is initialized when a predetermined pulse is generated at the output of the comparing means 110, and counting is performed once per frame interval value provided from the DMUX 60 by continuing counting from this initializing moment. Video time interval henerator (VTIG) means 140 for generating a strobe signal when generated; It is initialized when a predetermined pulse is generated at the output of the comparing means 110, and the count is continuously generated from this initializing moment to generate a strobe signal when generating a pulse once per frame interval value provided from the DMUX 60. Audio time interval generator (ATIG) means 150; After decoding the video and audio frames passing through the buffer means 70 and 80, the video and audio frames are output by the pulses provided by the predetermined frame interval values from the generators 120 and 130. A video and audio signal synchronizing apparatus of an MPEG decoding system comprising video and audio decoder means (160) and (170). The method according to claim 1, wherein the decoding system stores the display time values of the first video and audio in the VAPTSR 100, and then adds the time values corresponding to the least common multiple of the video frame interval and the audio frame interval. And a video and audio time synchronization device of an MPEG decoding system, characterized in that: The apparatus of claim 1, wherein the VTIG (140) and the ATIG (150) comprise a programmable counter. According to claim 1, The DMUX (60) re-initializes the VTIG 140 and ATIG 150 by sending a video and audio display time value to the register 100 after a special effect such as screen freeze during operation Video and audio signal synchronization device of the MPEG decoding system, characterized in that the.