JP2000031929A - Multimedia multiplex system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To allow the system to demultiplex packet data consisting of divisions of multimedia data in the unit of a prescribed number by respective information sources comprising a plurality of different kinds of media and a plurality of same kinds of media and to take synchronization for reproduction among the media. SOLUTION: A media ID 4 is used to identify information sent in a form of a packet and consists of a media type 41 (bit length: b 41-bit) and of same media identification information 42 (bit length: b 42-bit). The media type 41 is represented by media identification codes assigned corresponding to each medium such as audio (monaural, stereo), still image (low resolution, high resolution), and video (low resolution, high resolution). The same media identification information 42 is used to identify packets whose media type 41 is the same but whose information sources differ. Furthermore, the system ensures synchronization among the media by using a time difference 523 from other media packet.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a multimedia multiplexing system for separating data of various multiplexed media.

[0002]

2. Description of the Related Art FIG. 6 is, for example, "International Standards for Multimedia" edited by Hiroshi Yasuda (Maruzen Co., Ltd., pp. 230-232, 1991)
FIG. 1 shows an embodiment of the multimedia multiplexing method described in US Pat. The multiplexing is basically performed in units of time length equivalent to one frame of video, and a data stream (data stream) of each medium corresponding to this time length is referred to as a packet here. Since audio data and video data have a larger data amount per unit time than other media data, packetization is performed on a data sequence after encoding and removing redundancy.
In the encoded data sequence, a unique word is inserted every time corresponding to one frame of video, and packetization is performed using this as a mark.

Each packet is composed of a header and an information field as shown in FIG. The information field stores a data string to be multiplexed. The header is composed of the following information. Start code: A code for indicating a media type and detecting a boundary for demultiplexing. System clock reference: time information (time stamp) for synchronizing each medium. Data length: information giving the information field length.

[0004] On the other hand, in demultiplexing (demultiplexing), individual packets are extracted from a series of packet sequences generated by the above multiplexing, and the media type of each packet is determined. After synchronization, the data stream is reproduced for each medium. Packet sequence 102
Is extracted by detecting a header. A start code is used for the media type.

FIG. 8 shows an embodiment for ensuring synchronization between media. 8, reference numeral 200 denotes a demultiplexer (demultiplexer); 205, a video decoder;
06 is an audio decoder, 207 is a synchronization information comparator, 2
08 is a delay memory. In order to synchronize the respective media, the respective media data are temporarily stored in the delay memory 208, respectively. When the timing pulse 304 from the synchronization information comparator 207 is input to the respective delay memories 208, the respective media data are stored in the respective delay memories 208. The media data 301 is read all at once. The synchronization information comparator 207 compares the time stamp 303 of each media packet and generates a timing pulse 304 that synchronizes with the media signal having the latest time stamp.

When information is recorded on a storage medium such as a CD-ROM, high-speed reproduction, reverse reproduction, and reproduction by selecting only a part of media information according to the purpose of use of the user (for example, Special playback functions such as video only and audio only) are required. These functions are realized by performing random access to the multiplexed series of packets. With random access,
The target packet is accessed by repeating the operation of reading the data length information in the header and jumping to the next header.

[0007]

Since the conventional multimedia multiplexing system is configured as described above, when simultaneously reproducing a plurality of media from multiplexed data,
Processing such as detecting the header and determining the type of the medium must be sequentially repeated, and high-speed processing is required. In addition, since packetization is performed at a time interval corresponding to one frame of video, when performing a series of communication processing such as multiplexing, transmission, reception, and multiplexing / demultiplexing, the delay increases as the number of media to be multiplexed increases. There was a problem that it could not be ignored.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-described problems. Multimedia data having a plurality of different types of media and a plurality of the same types of media as information sources are divided into predetermined units for each information source. It is an object of the present invention to provide a multimedia multiplexing system that can separate each divided packet data and synchronize reproduction between media.

[0009]

In order to solve the above-mentioned problems, a multimedia multiplexing system according to the present invention uses a plurality of different types of media and a plurality of same types of media as information sources. Information source identification information for identifying the information sources of the plurality of different types of media and a plurality of the same type media of each packet data, and time information for synchronizing the reproduction of the respective media, into each packet data divided in a predetermined unit. As time difference information from the reproduction time of the other packet data, and from the multiplexed multimedia data, based on the information source identification information, the packet data of each information source is separated for each information source, and the time difference It is characterized in that the reproduction between the media is synchronized based on the information.

[0010]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiment 1 Hereinafter, an embodiment of the present invention will be described with reference to FIG.

FIG. 1 shows a multiplexing method according to an embodiment of the present invention, in which data of each medium is divided into equal lengths for each information source, each medium and each media attribute, and various identification information is added. Multiplex.

Each packet has call identification information 1 (bit length:
b1 bit), ground identification information 2 (bit length: b2 bits), packet number 3 (bit length: b3 bits), media ID 4 (bit length: b4 bits), link information 5
(Bit length: b5 bits), a header composed of a system clock reference 6 (bit length: b6 bits), and an information field (bit length: b7 bits). Each of these packet components has a fixed bit length.

The call identification information 1 and the ground identification information 2 are used when transmitting and receiving a packet via a transmission line. The call identification information 1 is information for identifying the transmitting side, and a calling number is used. The ground identification information 2 is information for identifying a receiving side, and an incoming call number is used.

The packet number 3 specifies the packet itself, and is a modulo M of M (M is represented by a power of 2 and log2 M = b3) and is numbered in chronological order in the transmission order of the packet. Attached.

The media ID 4 is for identifying information transmitted in a packet, and has a media type 41.
(Bit length: b41 bits) and the same media identification information 42
(Bit length: b42 bits). The media type 41 is a media identification code assigned to each media such as audio (monaural, stereo), still image (low resolution, high resolution), and video (low resolution, high resolution) as shown in Table 1. It is represented by The same media identification information 42 is for identifying packets having the same media type 41 but different information sources.

[0016]

[Table 1]

The link information 5 indicates the correspondence between the current packet and other packets in the multiplexed packet sequence.
It is composed of intra-media link information 51 (bit length: b51 bits) and inter-media link information 52 (b52 bits).

The intra-media link information 51 indicates the correspondence between packets having the same media ID 4.
It is composed of a past packet number 511 (bit length: b511 bits) and a sequence number 512 (bit length: b512 bits). The past packet number 511 stores the packet number 3 of the packet having the same media ID 4 as the current packet and preceding in time. The sequence number 512 is N for each packet having the same media ID 4 (N is represented by a power of 2, and log2 N = b512
Are assigned in chronological order in the packet transmission order with the remainder (modulo N). Note that b511 = b3.

The inter-media link information 52 indicates the correspondence between the current packet and a packet having a media ID 4 different from the current packet, and includes the number 521 of other media links (bit length: b521 bits), and a plurality of other media packet numbers. 522 (bit length: b522 bits each). The number 521 of other media links indicates the total number K of packets that temporally correspond to the current packet and have a different media ID 4. The other media packet number 522 temporally corresponds to the current packet and has a different media ID 4
The number of other media links 521 is followed by K other media packet numbers 522. Remaining bits in the inter-media link information 52 (bit length: b523 bits, b523 = b52−b521
-K * b522) are stuffing bits.

FIG. 2 is a block diagram showing a configuration of a multiplexer (multiplexer) according to the first embodiment. In FIG. 2, reference numeral 100 denotes a multiplexer, 101 denotes a video encoder,
2 is an audio encoder, 103 is an information field generation unit, 104 is a clock generation unit, 105 is a synchronization information addition unit, 106 is a packet number / media ID / link information addition unit, and 107 is a call identification / ground identification information addition unit. is there.

The information field generation unit 103 has a function as a division unit for dividing multimedia data into data for each information source. The clock generation unit 10
4. Synchronization information adding unit 105, packet number / media I
The D / link information addition unit 106 and the call identification / ground identification information addition unit 107 include various information for the divided data, for example, a media ID 4 as information source identification information indicating an information source of the data, link information 5, and a system clock reference. 6 has a function as generating means for generating various information such as 6 and multiplexing the various information with the divided data. Next, the operation will be described. In the multiplexer 100, the information field generation unit 103 first divides each piece of media data 301 having different input information sources to generate information field data. Note that the video data and the audio data are encoded by the video encoder 101 and the audio encoder 102, respectively,
Input to 0. Next, the synchronization information adding unit 105 adds the system clock reference 6 to the information field data. In adding the system clock reference 6, a time stamp is created using the clock signal generated by the clock generation unit 104. The information field data for each information source to which the system clock reference 6 is added is added with the packet number 3, the media ID 4, and the link information 5 by the packet number / media ID / link information adding unit 106, and is output in the order of the added packet number. Is done. Further, a call identification / ground identification information adding unit 10
At 7, the call identification information and the ground identification information are added and output from the multiplexer 100 as a series of packet strings 302.

FIG. 3 is a block diagram showing the configuration of the demultiplexer (demultiplexer) according to the first embodiment.
In the figure, 200 is a demultiplexer, 201 is a call identification /
Ground identification information separating section, 202 is a packet number / media ID / link information separating section, 203 is a synchronization information separating section, 2
04 is an information field synthesis unit, 205 is a video decoder,
206 is an audio decoder, 207 is a synchronization information comparator,
208 is a delay memory.

Next, the operation will be described. In the demultiplexer 200, first, a call identification / ground identification information separation unit 201
Then, separation processing of call identification information 1 and ground identification information 2 is performed from each packet in the series of packet strings 302. Next, the packet number 3, media ID 4, and link information 5 are separated by the packet number / media ID / link information separation unit 202, and data composed of the system clock reference 6 and the information field is output for each information source. Further, the system clock reference 6 is separated in the synchronization information separation unit 203, and information field data is output. As for the information field data for each information source, the information fields are combined by the information field combining unit 204 to generate respective media data. Video data and audio data among various types of media data having different information sources are supplied to a video decoder 205 and an audio decoder 206.
Respectively. In order to ensure synchronization between media, the synchronization information separating unit 2 is used as in the conventional embodiment.
03, the time stamp 303 for each medium separated for each medium is input to the synchronization information comparator 207 and compared, and a delay process is performed so as to synchronize with the media signal having the latest time stamp. As described above, according to the present embodiment, multimedia data is multiplexed for each attribute (information source, media, coding scheme, quality, etc.) on a packet basis, and information indicating the attribute of each packet or the like Since the information indicating the correspondence is multiplexed, a part or a plurality of data can be easily extracted and reproduced from the multiplexed data. For example, extraction of transmission data from a specific user from a series of data strings, or extraction and decoding of specific video data from encoded video data can be easily realized. Also, when multimedia data is recorded on a storage medium using this method, special reproduction such as high-speed reproduction, reverse reproduction, or reproduction in which only some media are selected can be easily realized.

Embodiment 2 FIG. In the first embodiment, the intra-media link information 51 is composed of the past packet number 511 and the sequence number 512. However, as shown in FIG. 4, it has the same media ID 4 as the current packet,
And a future packet number 510 in which the packet number 3 of the temporally succeeding packet is stored, and a past packet number 5
11 and a sequence number 512, which makes it easier to access the multimedia data recorded on the storage medium and enables high-speed reproduction.

Embodiment 3 In the first and second embodiments, the synchronization between the media is achieved by using the time stamp written in the system clock reference 6. However, as shown in FIG. The synchronization between the media may be ensured using the time difference 523.

Embodiment 4 In the first, second and third embodiments, the call identification information 1 and the ground identification information 2 are added for each packet. However, when the present multiplexing method is used in a field other than communication, the call identification information 1 is added. 1 and the ground identification information 2 may be omitted. Depending on the application, the addition of other identification information may be omitted.

Embodiment 5 FIG. Of course, some kind of identification information may be put in the empty bits in the first, second and third embodiments.

[0028]

As described above, according to the present invention, each piece of multimedia data having a plurality of different types of media and a plurality of same types of media as information sources is divided into predetermined units for each information source. Information source identification information for identifying the information sources of the plurality of different types of media and the plurality of the same types of media of each packet data, and from the playback time of the other packet data as time information for synchronizing the playback of each media. And the packet data of each information source is separated for each information source from the multiplexed multimedia data based on the information source identification information, and reproduction synchronization between media is performed based on the time difference information. So that a plurality of different types of media and a plurality of same types of media are used as information sources based on the information source identification information. With the multimedia data can be separated each packet data divided separately in predetermined units each information source, it may take the reproducing synchronization between media based on time difference information.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram of a multiplexing method according to Embodiment 1 of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a multiplexer according to Embodiment 1 of the present invention.

FIG. 3 is a block diagram showing a configuration of a demultiplexer according to Embodiment 1 of the present invention.

FIG. 4 is an explanatory diagram of a multiplexing method according to a second embodiment of the present invention.

FIG. 5 is an explanatory diagram of a multiplexing method according to a third embodiment of the present invention.

FIG. 6 is an explanatory diagram of a conventional multimedia multiplexing method.

FIG. 7 is an explanatory diagram of a packet structure in a conventional multimedia multiplexing method.

FIG. 8 is an example of a device configuration for ensuring synchronization between media.

[Explanation of symbols]

1 call identification information, 2 ground identification information, 3 packet number, 4 media ID, 5 link information, 6 system clock reference, 41 media type, 42
Same media identification information, 51 intra-media link information,
52 inter-media link information, 100 multiplexer, 10
1 video encoder, 102 audio encoder, 103
Information field generator, 104 clock generator, 1
05 Synchronization information addition unit, 106 packet number / media ID / link information addition unit, 107 call identification / ground identification information addition unit, 200 multiplex / separator, 201 call identification / ground identification information separation unit, 202 packet number / media I
D / link information separation unit, 203 synchronization information separation unit, 20
4 information field synthesis unit, 205 video decoder, 2
06 audio decoder, 207 synchronization information comparator, 2
08 delay memory, 301 media data, 302
Packet sequence, 303 time stamp, 304 timing pulse, 510 future packet number, 511 past packet number, 512 sequence number, 521 other media link number, 522 other media packet number, 5
23 hours difference.

Claims (1)

[Claims] 1. A method according to claim 1, wherein each of the plurality of different types of media and a plurality of pieces of multimedia data using the same type of media as information sources are divided into predetermined packet units for each information source. A multimedia in which information source identification information for identifying information sources of a medium and a plurality of media of the same type, and time difference information from the reproduction time of other packet data as time information for synchronizing reproduction of each medium are multiplexed. A multimedia multiplexing method, wherein packet data of each information source is separated for each information source from data based on the information source identification information, and reproduction synchronization between media is established based on the time difference information. .