JPH0898160A - Data multiplexing device - Google Patents

Abstract

PURPOSE: To provide a multiplexing device which can multiplex data without inserting invalid data for transfer rate adjustment and can effectively use a recording medium without the occurrence of the overflow or underflow of a decoding buffer. CONSTITUTION: When the free storage area of a virtual decoding buffer, which are previously set in ROM 75 for respective encoding streams, become more than the data part of a pack, CPU 74 outputs a packet request signal to a packet generator 76. When the free storage areas exist in the virtual decoding buffer and more than two kinds of encoding streams can be made into packets, it is decided from which encoding stream the packet is made in accordance with previously set priority, and the packet request signal including the decision signal is transmitted to the packet generator 76. The packet generated by the packet generator 76 is transmitted to a pack generator 77, is made into the pack and a multiplex data string is transmitted through an output buffer 79.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data multiplexer for adding a variety of information to a plurality of time-synchronized digital audio information data and image information data to form one time series data. .

[0002]

2. Description of the Related Art A plurality of digital audio and image data
In order to transmit in a system, it is necessary to previously multiplex data according to a predetermined format to create a data string. Also, the receiving side needs to separate (demultiplex) the data string into audio and image data again according to the format. As an example of the format of this multiplexing, ISO / IEC DIS 11172 (MPEG-
1) There is Part 1. The multiplexed data string ISO11172 stream defined here is composed of a pack and a packet, the pack is composed of a pack header and one or more packets, and the packet is composed of a packet header and a plurality of streams. The plurality of streams may be compressed data of voice or image or character data. Also,
In order to record this multiplexed data string ISO11172 stream on an actual recording medium, a format depending on the recording medium (compact disc, DAT, etc.) is required, and it has already been standardized as CD-1, video CD, CD-α. Has been done.

FIG. 4 is a block diagram showing a multiplexing device according to a conventional example of this type. In the multiplexing apparatus shown in FIG. 4, when decoding each coded stream, the transfer rate is known (fixed transfer rate), and the transition of the data occupancy of the decoding buffer can be calculated in advance as voice. It defines the ratio of image packets and multiplexes the data accordingly. The multiplexing device shown in FIG.
It has encoders 11 to 13 that encode audio and image data and outputs the encoded streams as encoded streams, and input buffers 21 to 23 corresponding to each encoded stream, and data via the input buffers 21 to 23. Is the multiplexing controller 30
Are multiplexed and transmitted as a multiplexed data string to the receiving side based on a synchronization signal given from the outside.

That is, in the multiplexing controller 30, when the input buffers 21 to 23 store a predetermined amount of voice or image data, the packet generator 31 generates a packet to form a pack. When a number of image and audio packets are generated, the pack generator 32 combines the packets generated by the additional data packet generator 33 and the invalid data packet generator 34 to generate a multiplexed data sequence, and the multiplexed data sequence is generated. Output the data string 35
The multiplexed data string stored in the output buffer 35 is transmitted to the receiving side in accordance with the synchronizing signal.

FIG. 5 is a diagram schematically showing an example of the multiplexed data sequence thus obtained. That is, the multiplexed data string shown in FIG. 5 is composed of a system header, an additional data packet, an audio packet, an invalid data packet, an audio packet, an invalid data packet, and an image packet, which are sequentially packed with a pack header at the beginning. A packet consists of four packet headers and an image stream.

Next, the configuration of the demultiplexer for receiving the multiplexed data string transmitted from the multiplexer side is shown in FIG.
Shown in. The multiplexed data string transmitted from the multiplexer is
It is input to the demultiplex controller 42 via the demultiplex buffer 41, where each control data, additional data, etc. are separated and then returned to each encoded stream,
It is sent to each of the decoding buffers 51 to 53. Data is sent from each of the decoding buffers 51 to 53 in response to a request from the decoders 61 to 63.

By the way, as described above, each coded stream is decoded by each of the decoders 61 to 63 and output as voice or image data. At this time, the coded data is variable length coded. If so, the amount of data sent from the decoding buffers 51 to 53 to the decoders 61 to 63 differs for each decoding unit, for example, for each picture in the case of an image. If the encoding is performed so as to be constant for each unit time, the fluctuations of the decoding buffers 51 to 53 are absorbed within this time.

FIG. 7 shows how the data amount in the decoding buffers 51 to 53 changes. For example, the decoding buffer 51 stores the data of the voice packet P1 of the multiplexed data string, and the decoding buffer 52 stores the voice packet P2.
When the data of the image packet P3 is stored by the decoding buffer 53, the data amount of each decoding buffer is as follows. That is, for example, if the audio coded stream of the audio packet P1 is input to the decoding buffer 51, there is no data input to the decoding buffer 51 at first, and the data is only sent to the decoder 61, and the decoding is performed. The data amount of the buffer 51 gradually decreases until time t1. Next, after the voice packet P1 in the multiplexed data string is input and demultiplexed,
When input to the decoding buffer 51, the decoding buffer 5
Since the input data amount of 1 exceeds the output data amount, the data amount in the decoding buffer 51 gradually increases until time t2. After that, the voice packet P1 is not input and the data amount of the decoding buffer 51 decreases again. This cycle is the pack cycle.

The same applies to the other decoding buffers 52 and 53.
The data amount of the decoding buffer 52 gradually decreases until time t3, conversely increases until time t4 after time t3, and then decreases again. Further, the data amount of the decoding buffer 53 gradually decreases until time t5, conversely increases after the time t5, and then decreases again. Here, if each coded stream has a fixed transfer rate, the transition of the amount of data stored in the decoding buffer is repeated for each pack as shown in FIG. 7, and overflow or underflow of the decoding buffer does not occur. .

[0010]

The conventional multiplexing device is
As described above, it is premised that the coded stream has a fixed transfer rate, but the following points become a problem when the coded stream has a variable transfer rate. here,
An example will be described in which the audio coded stream has a fixed rate and the image coded stream has a variable transfer rate.

Image coded stream (eg ISO /
IEC DIS 11172 (MPEG-1) Part 2
Etc. perform variable length coding, and the data amount for each decoding unit (picture unit) is significantly different. Therefore, the amount of data fluctuation in the buffer is large for each decoding, and the image decoding buffer needs to be larger than the fixed length coded stream. In addition, the image coded stream has a large time difference from decoding to playback compared with the audio coded stream, and the purpose is to reduce the decoding buffer for audio. It was necessary to store the image coded stream in the decoding buffer before the audio coded stream.

FIG. 8 shows the amount of data in the audio decoding buffer and the image decoding buffer and the decoding start timing at that time. First, the image code stream is stored in the image decoding buffer until a predetermined amount is reached and the decoding is not started. Therefore, the amount of data in the image decoding buffer gradually increases until time T1. Since the audio coded stream does not start decoding until the image signal is reproduced, a predetermined amount is stored in the audio decoding buffer and waiting for decoding start.

Next, the decoding of the image coded stream is started at time T1, and the decoding of the audio coded stream is started at time T2, which is the reproduction time, so that the audio can be reproduced in synchronization with the image. . Therefore, as shown in FIG. 8, the audio coded stream stops the multiplexing when a predetermined amount is stored in the audio decoding buffer, and waits for the reproduction of the image coded stream to start. . for that reason,
When the multiplexed stream is configured in the single format shown in FIG. 5, the multiplexed data sequence starts reproduction of the image encoded stream after a predetermined amount of data is stored in the audio decoding buffer. Until I was told, I had to replace voice packets with invalid data packets.

Further, in the format of the conventional multiplexed data string shown in FIG. 5, when the recording medium is, for example, a video CD, the management unit of the recording medium is a sector unit so that each encoded stream can be easily demultiplexed. Since the size of the packet of voice, image, and other data is determined, it is necessary to insert invalid data for transfer rate adjustment due to the coding rate. Finally, when the image transfer rate changes, it is difficult to fix the audio data and image data formats in advance.

The present invention has been made to solve the above problems, and when a digitized moving image and audio signal is multiplexed for simultaneous recording on the same recording medium, the decoding buffer overflows or underscores. It is an object of the present invention to obtain a multiplexing device that can be multiplexed without causing a flow and without inserting invalid data for transfer rate adjustment and the like, and can effectively use a recording medium.

[0016]

In order to achieve the above-mentioned object, a multiplexing device according to the present invention comprises a plurality of encoders for encoding input digital audio or moving images and outputting them as encoded streams. In a data multiplexing device that multiplexes the plurality of coded streams so that they can be reproduced synchronously, a virtual decoding buffer simulation is performed for each of the coded streams, and a multiplexing request is made based on the data occupation amount in the buffer. It is characterized in that it is provided with a control means for outputting a signal and a multiplexing means for multiplexing each coded stream based on the multiplexing request signal.

Further, means for setting a priority to each of the coded streams is provided, and the control means outputs a multiplexing request signal to be multiplexed according to the priority, and the multiplexing means each according to the priority. It can be configured to multiplex encoded streams.

[0018]

According to the present invention, each coded stream is multiplexed on the basis of the occupancy of the virtual decoding buffer, so that the variation of the decoding buffer is reduced and the overflow or underflow of the decoding buffer is prevented. Disappear. In addition, since the pack structure is not specified in advance, it is not necessary to generate a pack with transfer rate adjustment invalid data or the like, which enables effective use of the data range of the recording medium.

Further, since the coded streams are multiplexed in accordance with the preset priority, the data string can be rearranged by setting the priority.

[0020]

Embodiments In the embodiments of the present invention, two audio encoded streams and one image encoded stream are multiplexed and a multiplexed data string is recorded on a digital disc (compact disc CD, hard disk drive HDD, etc.). In this case, an example in which one pack is composed of one packet will be described. Further, the frame used in this embodiment is not a general single image referred to as an image signal but a data unit to be decoded.

FIG. 1 is a block diagram showing a multiplexer according to an embodiment of the present invention. In FIG. 1, encoders 11 to 1
The plurality of audio and image encoded streams encoded in 3 are first read into the input buffers 21 to 23. The time information generator 71 reads the decoding time corresponding to the head position information of the frame of each encoded stream in the input buffers 21 to 23 from the frame timer 72, calculates the decoding time and the address in the input buffer, and The number is recorded in the frame pointer register 73 for each number.

FIG. 2 is an explanatory diagram showing the contents recorded in the frame pointer register 73. Frame timer 7
In 2, the data is sent to the time information generator 71 based on the reference clock according to each encoded stream. If the reference clock is 90,000 Hz, the value of the clock of one image frame is 30 × 1000/100 for the NTSC signal.
Since it is 1 second, 90000 / (30 × 1000/1001) = 3003, and the value for each clock is sent to the time information generator 71.

The time information generator 71 has input buffers 21 to 21.
The decoding time corresponding to the head position information of the frame of each coded stream in 23 is read from the frame timer 72, the decoding time and the input buffer address are calculated,
Each frame number is recorded in the frame pointer register 73. In this way, the frame pointer register 73
Is, for example, 3 as the decoding time of frame number 100.
003, and 120 is stored as the address in the input buffer.

In the CPU (Central Processing Unit) 74, the virtual decoding buffer is simulated for each encoded stream according to the program in the ROM (Read Only Memory) 75, and the data occupancy in the buffer is simulated. Control to generate a packet based on That is, in the CPU 74, RO is previously set for each encoded stream.
Of the virtual decoding buffer capacity set in M75,
When the free storage area becomes equal to or larger than the data of the pack, that is, the data portion of the encoded stream that is the portion excluding the header information in the pack, a packet request signal is output to the packet generator 76, so that the packet generator 76 Control to generate packets. At this time, the CPU
No. 74 has a free storage area in the virtual decoding buffer, and 2
If it is possible to packetize more than one type of coded stream, it is determined from which coded stream to packetize according to the preset priority, and the packet request signal including this decision signal is sent to the packet generator. Send to 76.

FIG. 3 is a flow chart showing the specific contents of the control operation of the CPU 74 which operates according to the program in the ROM 75. Each coded stream is operated according to this procedure in accordance with a preset priority. That is,
When the interrupt signal is supplied from the reference clock, it is checked whether the virtual system clock generated from the reference clock and the decoding time in the frame pointer register 73 are the same (steps S31 and S32). If YES, the virtual decoding buffer The one frame data is removed from (step S33).

If the determination in step S32 is NO, and after step S33, the process shifts to step S34 to calculate the free storage area of the virtual decoding buffer and check whether a packet can be generated (step S34). ,
S35). If YES, the packet request signal is output to the packet generator 76 (step S36), and if NO,
Wait for the next reference clock. Incidentally, the above-mentioned step S3
If the decoding time in the frame pointer register 73 matches the clock in 2, the pointer is moved to the input buffer address of the next frame data.

The packet generator 76 receives the packet request signal from the CPU 74, generates a packet, and the generated packet is sent to the pack generator 77.
The pack generator 77 packs and sends the multiplexed data sequence through the output buffer 79. In addition, when the additional data is multiplexed, the packet is sent from the additional data packet generator 78 to the pack generator 77 according to the reference clock, and therefore the pack generator 77 also multiplexes the additional data and packs the additional data. The data string is transmitted via the output buffer 79.

Therefore, according to this embodiment, when the digitized moving image and audio signals are multiplexed and recorded on the same recording medium, the state of the virtual decoding buffer is constantly monitored and the encoded streams are multiplexed. Therefore, the decoding buffer does not overflow or underflow. Even if the transfer rate of the coded stream is indefinite and the size of the packet and the pack cannot be set for each coded stream in advance, there is no problem because the multiplexing is performed according to the decoding buffer amount. Further, even if the transfer rate of the coded stream is constant, it is not necessary to set the size of the packet and the pack for each coded stream in advance, and the same decoder can be used and can be generalized.

In the above embodiment, it is of course possible to generate the multiplexed data string with N (N is an integer) coded streams or with N packets forming a pack. In addition, ISO / IEC DIS 111
In addition to the multiplexed data string of 72 (MPEG-1) Part 1, if the header of the packet includes at least type information (voice, image, character, etc.) of each information and time information for each information, It is possible for the device to generate a multiplexed data sequence.

[0030]

As described in detail above, according to the present invention,
By multiplexing each coded stream based on the size of the virtual decoding buffer occupancy, the fluctuation amount of the decoding buffer is reduced, and the overflow or underflow of the decoding buffer is eliminated. Also, since the pack structure is not specified in advance, it is not necessary to generate a pack with transfer rate adjustment invalid data or the like, that is, it is not necessary to insert transfer rate adjustment invalid data or the like at the time of multiplexing,
Therefore, the data range of the recording medium can be effectively used.

Since the coded streams are multiplexed according to the preset priority, the data of the multiplexed data string can be rearranged by setting the priority.

[Brief description of drawings]

FIG. 1 is a block diagram showing a multiplexer according to an embodiment of the present invention.

2 is an explanatory diagram showing contents recorded in a frame pointer register of FIG. 1. FIG.

FIG. 3 is a flowchart showing a control operation of the CPU of FIG.

FIG. 4 is a block diagram showing an example of a conventional multiplexing device.

FIG. 5 is a format diagram showing an example of a conventional multiplexed data sequence.

FIG. 6 is a block diagram showing a conventional demultiplexer.

7 is an explanatory diagram showing a transition of the data amount of the decoding buffer of FIG.

FIG. 8 is an explanatory diagram showing the relationship between the decoding start time and the data amount in the decoding buffer in FIG.

[Explanation of symbols]

11-13 Encoder 21-23 Input buffer 71 Time information generator 72 Frame timer 73 Frame pointer register 74 CPU (Operating as a control means, RO
M75 constitutes a means for setting a priority) 75 ROM 76 packet generator (comprises a pack generator and a multiplexing means together with an additional data packet generator) 77 pack generator 78 additional data packet generator 79 output buffer

Claims (2)

[Claims] 1. A data multiplexing apparatus comprising a plurality of encoders for encoding input digital audio or moving images and outputting them as encoded streams, and for multiplexing the plurality of encoded streams so that they can be synchronously reproduced. , A virtual decoding buffer is simulated for each of the coded streams, a control means for outputting a multiplexing request signal based on the amount of data occupied in the buffer, and each coding based on the multiplexing request signal. A data multiplexing apparatus comprising: a multiplexing unit for multiplexing streams. 2. A means for setting a priority to each of the coded streams is further provided, the control means outputs a multiplexing request signal to be multiplexed in accordance with the priority, and the multiplexing means has the priority. 2. The data multiplexing device according to claim 1, wherein the data multiplexing device is configured to multiplex each coded stream according to the degree.