JPH0974547A - Data multiplexer - Google Patents

Abstract

PROBLEM TO BE SOLVED: To output the multiplexed data at a fixed rate and to reduce the transmission delay of the private data. SOLUTION: A data multiplexing device is provided with a basic packet part generation means B, an additional packet part generation means C, and a pack generation means D. The means B selects the video data, the audio data or the padding data and generates a basic packet part where the selected data are inserted. The means C selects one or both of the private data and true padding data and generates an additional packet part where the selected data are inserted. Then the means D generates the packs via both basic and additional packet parts. The multiplexed data include the continuous packs which are generated by the means D.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a data multiplexer, and more particularly to a data multiplexer suitable for multiplexing private data with video data and audio data. Here, as the private data, for example, data that is not synchronized with the system clock, such as still image data, FAX data and text data, or closed caption data.
It is data generated in synchronization with a video signal such as data.

[0002]

2. Description of the Related Art Conventionally, various kinds of information (for example, signals such as images and sounds) have been used by using various communication devices and computers.
In the technical field of processing or processing, there is a demand for exchanging a large amount of information in a short time.

Therefore, in recent years, a technique has been developed in which the amount of information is reduced by converting the exchanged information into a digital signal and further compressing and multiplexing this digital signal by an original method. There is an MPEG system as an international standard using a technique for compressing and multiplexing such digital signals.

In the MPEG system, for example, image signals and audio signals are compressed at a high density and recorded on a recording medium such as a CD-ROM (compact disc-ROM) or a DVD (digital video disc), or It can be used for exchanging information by transmitting a compressed signal such as digital satellite broadcasting or a high quality video conference system.

By the way, according to the MPEG system, data-compressed video data and audio data private data are multiplexed and transmitted by a packet-based multiplexing system. That is, data-compressed video data, audio data, and private data are divided into streams of appropriate length called packets, and then the packets of video data, audio data packets, and private data. Each pack, which is one or a plurality of sets of data packets, is transmitted in a time division manner.

[0006]

However, in realizing such a multiplexing system, there are the following problems. (1) The data to be inserted in the packet is usually selected from video data, audio data and private data.
Depending on the frequency of data generation, private data that is not synchronized with the system clock, such as still image data, FAX data, and text data, does not occur evenly, so it is multiplexed. There is a problem that the output rate of the entire data becomes unstable.

As one means for solving such a problem, there is a method in which the packet length of video data, the packet length of audio data, and the packet length of private data are made substantially the same. However, in such a method, the frequency of generation of private data is very low compared to that of video data and audio data, so that there is a problem that the multiplexing delay becomes large.

(2) In addition, closed captions
In the case of data such as data that is synchronized with a picture, if the packet lengths are multiplexed in the same size, it becomes difficult to maintain synchronization with video data.

(3) The same problem as in the above (1) and (2) has occurred also in the multiplexing corresponding to the multi-program for generating a plurality of programs using the packet.

In view of the above problems, it is a first object of the present invention to provide a data multiplexer capable of outputting multiplexed data at a constant rate and reducing the transmission delay of private data. To do.

A second object of the present invention is to reduce the transmission delay of private data and to enable multiplexing while maintaining the synchronization between private data and video data. To do.

Further, in the multiplexing compatible with the multi-program, the transmission delay of the private data can be reduced, and the multiplexing can be performed while maintaining the synchronization between the private data and the video data. The third purpose is to

A fourth object of the present invention is to enable multiplexed data to be output at a constant rate and to reduce transmission delay of private data in multiplexing compatible with multiple programs.

[0014]

A first data multiplexing apparatus of the present invention divides input data into a fixed size to generate a packet part and adds a pack header to the packet part. In the data multiplexing device for generating a pack, the packet part generating means for generating the packet part is composed of basic packet part generating means and additional packet part generating means, and the basic packet part generating means is Input video data, audio
One data is selected from the data and the padding data to generate the basic packet part, and the additional packet part generating means includes either one of the input private data or the padding data, or both of them. Data is selected and an additional packet part is generated.

Further, the first data multiplexing apparatus of the present invention is such that video data input as basic packet data to be multiplexed in the basic packet portion forming the pack,
Basic packet source selecting means for selecting one data from audio data and padding data, basic packet data selected by the basic packet source selecting means is divided into a size of a predetermined length, and Basic packet part generation means for generating a basic packet part by adding a basic packet header to the basic packet data divided into sizes, and additional packet data to be multiplexed in the additional packet part forming the pack,
Either the private data or the padding data to be input, or both data are selected according to the pack length, and an additional packet header is added to the selected additional packet data to add an additional packet section. It is provided with additional packet part generating means for generating, and pack generating means for generating a pack by adding a pack header to each packet part given respectively from the basic packet part generating means and the additional packet part generating means.

Further, the first data multiplexing apparatus of the present invention is characterized in that the length of the additional packet portion is obtained from a pack length, a multiplexing output rate and a maximum input rate of private data.

Since the first data multiplexing device of the present invention comprises the above technical means, the pack is composed of two types of the basic packet part and the additional packet part, and the private data packet length is the maximum input rate. Therefore, even if the private data is asynchronous, the multiplexing delay can be effectively suppressed, and the private packet and padding data are selectively used to add the additional packet part. Is configured, so
It becomes possible to perform constant rate output.

The second data multiplexing apparatus of the present invention divides the input data into a fixed size to generate a packet part, and adds a pack header to the packet part to generate a pack. In the above data multiplexing device, the packet part generating means for generating the packet part is constituted by the basic packet part generating means and the additional packet part generating means, and the basic packet part generating means is inputted. First basic packet part generating means for generating a first basic packet part in which video data is inserted;
And a second basic packet part generating means for generating a second basic packet part in which either one of the input audio data or padding data is inserted, and the additional packet part generating means. Is
Either the private data or the padding data to be input, or both data are selected to generate the additional packet portion.

The second data multiplexing apparatus of the present invention further comprises a picture information detecting means for detecting picture information from the input video data, and the input video data, audio data or padding data. Source selecting means for selecting a source to be multiplexed into a pack, basic packet part generating means for generating a basic packet part from the source selected by the source selecting means, and unmultiplexing of private data synchronized with a video signal A counting means for counting the number of blocks, and a source to be multiplexed in the pack from the private data or padding data,
Additional packet part generation means for cutting out the selected source according to the pack length to generate an additional packet part, and a pack header added to each packet part generated by the basic packet part generation means and the additional packet part generation means. And a pack generating means for generating a pack.

In the second data multiplexing device of the present invention, the private data synchronized with the video signal is closed caption data.

Since the second data multiplexing apparatus of the present invention comprises the above technical means, the packet part of the pack is composed of the basic packet part alone or the basic packet part and the additional packet part. Therefore, even in the case of private data synchronized with the video signal, the multiplexing delay is suppressed, and the additional packet part is added only to the video packet during multiplexing, so the private data can be maintained while synchronizing with the video signal. Will be able to be multiplexed.

A third data multiplexing apparatus of the present invention is a multi-program-compatible data multiplexing apparatus using PES packets, in which private data PES packets are generated from private data corresponding to one picture. A video containing a picture start code for the data PES packet generation means and the private data PES packet generated by the private data PES packet generation means
And multiplexing means for multiplexing after the data.

The third data multiplexing apparatus of the present invention is a data multiplexing apparatus compatible with a multi-program using PES packets, which cuts input video data by a predetermined size and cuts it out. Video PES packet generation means for generating a video PES packet by adding header information to the video data, and a video PE output from the video PES packet generation unit.
Picture information detecting means for detecting whether or not there is a picture start code indicating the beginning of a picture from an S packet, and audio data to be input are cut by a predetermined size, and the cut audio data is Audio PES packet generation means for generating an audio PES packet by adding header information,
Private data for generating private data PES packets by adding header information to private data corresponding to the picture in which the picture start code is detected, based on the detection result information provided from the picture information detecting means. PES packet generation means, means for selecting a PES packet to be output from the video PES packet, audio PES packet, and private data PES packet, and the video to be output based on the detection result of the picture information detection means. And a multiplexing means for outputting a private data PES packet when a picture start code exists in the data.

Further, in the third data multiplexing apparatus of the present invention, the multiplexing means sets a flag when the detection result information output from the picture information detection means indicates that there is a picture. If the flag is set, it is determined that the private data PES packet is to be output, and if the determination is made, the process of lowering the flag is performed.

Since the third data multiplexing apparatus of the present invention comprises the above technical means, the private data PES packet is configured to correspond to one picture.
The PES packet of the private data has a short structure, and the multiplexing delay can be suppressed even in the case of the private data synchronized with the video data. Further, since the private data is always multiplexed after the video data including the picture start code, it is possible to multiplex the private data while maintaining the synchronization with the video data.

A fourth data multiplexer of the present invention is a multi-program-compatible data multiplexer using PES packets, in which PES packets of private data to be multiplexed with video data and audio data are multiplexed. Private data PE to generate
S packet generation means and the private data PE
Private generated by S packet generation means
PES that adaptively changes the PES packet length of data
Packet length changing means and padding data filling means for filling padding data adaptively according to the amount of private data multiplexed with the video data and audio data are provided. .

The fourth data multiplexing device of the present invention is a timing notifying means for notifying the timing of multiplexing the private data together with the video data and the audio data, and the timing of the multiplexing from the timing notifying means. The PES packet length determining means for determining the PES packet length of the private data in accordance with the amount of data existing at the time of being notified, and the PES.
A private data PES packet is generated with the length determined by the packet length determining means, and when the private data PES packet is not notified when the timing for multiplexing the private data is notified, the private data PES is generated. The private data PES packet generation means for not generating a packet, the payload determination means for determining the payload of the transport packet as private data at the timing when the notification from the timing notification means is received, and the payload determination means. When the payload is determined to be private data, the PE of the private data
If the S packet length is shorter than the predetermined payload length, a length adjusting means for adjusting the length by the stuffing byte of the adaptation field,
The payload determining means determines that the payload is private
When the data is decided, padding data packing means for packing the padding data in the payload when private data does not exist is provided.

Since the fourth data multiplexer of the present invention comprises the above technical means, the PES packet length of the private data is adaptively changed, so that the multiplexing delay is suppressed even in the case of asynchronous private data. It is. Also, since the padding data is adaptively packed depending on the amount of private data, it is possible to multiplex the private data together with the video data and audio data without any inconvenience while keeping the output rate constant.

[0029]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of a data multiplexing apparatus of the present invention will be described below with reference to the drawings. FIG. 1 shows a first embodiment of the present invention and is a configuration diagram showing a main configuration of a data multiplexing apparatus. As shown in FIG. 1, the data multiplexing apparatus of the present embodiment has an input terminal E to which video data, audio data, private data and padding data are input, a basic packet part generating means B and A pack generation means D including an additional packet part generation means C and an output terminal F are provided.

The basic packet part generating means B selects one data from the input video data, audio data and padding data to generate a basic packet part. Further, the additional packet part generation means C selects either one or both of the input private data and padding data to generate the additional packet part.

The pack generating means D adds a pack header to the basic packet portion and the additional packet portion respectively input from the basic packet portion generating means B and the additional packet portion generating means C to generate a pack.

According to the data multiplexing apparatus of the present embodiment configured as described above, since the packet of private data is generated separately from the packets of video data and audio data, the packet length is set to the maximum input rate. Correspondingly, a packet of private data can be generated.

As a result, the multiplexing delay can be suppressed even for private data such as still image data, FAX data and text data which are not synchronized with the system clock.

Further, in the additional packet section generating means C, the additional packet section is generated by selectively using the private data and the padding data, so that the multiplexed data can be output at a constant rate.

Next, the video data / audio data according to the second embodiment of the first data multiplexing apparatus of the present invention will be described.
The data multiplexing device will be described with reference to FIGS. As shown in FIG. 2, the video data / audio data multiplexing device of the present embodiment has a video data input terminal 1, an audio data input terminal 2, and
Private data input terminal 3, basic packet selection unit 4, basic packet unit generation unit 5, and pack generation unit 6
An additional packet unit generation unit 7 and a multiplexing buffer unit 8
And a multiplexed data output terminal 9.

Video data 26 encoded (compressed) by a video data encoding device (not shown) is input to the video data input terminal 1. Audio data 27 encoded (data compressed) by an audio data encoding device (not shown) is input to the audio data input terminal 2. These video data 26 and audio data 27 are given to the basic packet selection unit 4, respectively.

Private data 28 is input to the private data input terminal 3, and the private data 28 is synchronized with a system clock such as still image data, FAX data and text data. There is data that does not exist or data that occurs in synchronization with the video signal, such as closed caption data. The private data 28 is given to the additional packet unit generation unit 7.

The basic packet selection unit 4 assumes which of the video data 26 and the audio data 27 is to be used in constructing a pack in consideration of the data generation status and the buffer status in the virtual separation / decoding device. It is determined whether the packet section 14 (see FIG. 3A) can be smoothly decoded.

Then, based on the result of this judgment, either one of the video data 26 and the audio data 27 is selected. If the basic packet selection unit 4 determines that neither the video data 26 nor the audio data 27 is appropriate, it generates padding data 29 (redundant data).

From the basic packet selection section 4, the selected video data 26 or audio data 27 is selected.
Alternatively, the generated padding data 29 is a packet.
The data length is cut out and output as basic packet data S4.

The basic packet part generator 5 uses the basic packet header 17 (see FIG. 3B) as the basic packet data S4.
In addition to the above, the basic packet unit 14 is generated. Here, the basic packet header 17 is composed of a packet start code indicating the start of a packet, information on a packet length, video data, audio data, and a time stamp indicating the decoding time of private data.

The additional packet section generator 7 has the following (1)
Private data packet calculated by the formula
The private data 28 is cut out by the data length PL, and the additional packet header 18 (see FIG. 3) is added to the cut out private data 28 to generate the additional packet unit 15 in which the private data 28 is stored (FIG. See 3B).

PL = PA × PR / MR + 1 (1) In the equation (1), PA is the packet length, PR is the private data bit rate, MR is the multiplexing rate, and the private data packet data length PL is , Rounded down to the nearest whole number.

If the private data 28 does not exist for the packet data length, the additional packet is added using the existing private data 28 and the padding data 29 generated corresponding to the surplus data length. The part 15 is generated (see (c) of FIG. 3).

If the private data 28 is not input to the additional packet unit generator 7, the additional packet unit 15 is generated using only the padding data 29 (see (d) in FIG. 3). .

The pack generation unit 6 includes the basic packet unit 14 output from the basic packet unit generation unit 5 and / or the additional packet unit 1 output from the additional packet unit generation unit 7.
The pack header 16 is added to 5 to generate the pack 12 (see FIG. 3A). Here, the pack header 16 is
It includes information such as a pack identification code and a system clock reference that serves as a reference for the above-mentioned time stamp.

The multiplexing buffer unit 8 temporarily stores the multiplexed data output from the pack generation unit 6 and adjusts the burst property. The burst data adjusted multiplexed data is output to the outside via the multiplexed data output terminal 9, recorded on a recording medium, or transmitted via a transmission path.

Next, this video data / audio
The data structure of the multiplexed data multiplexed by the data multiplexing device will be described with reference to FIGS. The multiplexed data is composed of a plurality of consecutive packs 12 as shown in FIGS. 3 (a), 4 (a) and 5 (a). Here, each pack 12 is composed of the pack header 16 indicating information such as time reference information, the basic packet part 14 and the additional packet part 15, as described above. A system header showing the overall outline is further added (the system header may also be added to the second and subsequent packs).

The basic packet section 14 is made up of one type of packet data. As shown in FIG. 3B, the basic packet header 17 and video data 26, and FIG. 4B.
The basic packet header 17 and audio
The data 27, or as shown in FIG. 5B, is composed of a basic packet header 17 and padding data 29.

The structure of the additional packet unit 15 is classified into three types, first to third types. The additional packet unit 15 of the first type is shown in FIG. 3 (b), FIG. 4 (b), and FIG.
(B), it is composed of private data 28 to which the additional packet header 18 is added.

The second type of additional packet unit 15 has a private packet added with an additional packet header 18 as shown in FIGS. 3C, 4C and 5C. It is composed of data 28 and padding data 29 to which the additional packet header 18 is added.

The third type of additional packet section 15 has a padding / adding section to which an additional packet header 18 is added, as shown in FIGS. 3 (d), 4 (d) and 5 (d). And data 29.

Additional packet unit 1 of the first to third types
Which of 5 is selected depends on the amount of private data 28 existing. That is, when there is a predetermined amount or more of private data 28, the first type additional packet unit 15 is selected.

When the private data 28 does not exist at all, the third type additional packet unit 15
Is selected. Further, when the private data 28 is less than a predetermined amount, the private data 28 having a data length that can be cut out and padding data for adjusting the remaining data length are inserted into the second type. The additional packet unit 15 is selected.

As described above, in the data multiplexing apparatus of the present embodiment, since the pack is composed of the basic packet part and the additional packet part, all packs have private packets.
Since data can be inserted, the private data packet length can be reduced. Therefore, in the data multiplexing apparatus of the present embodiment, the multiplexing delay can be suppressed, and by switching between the private packet and the padding packet,
The multiplexed data can be output at a constant rate.

That is, for example, the frequency of occurrence of video data (ie, bit rate) is 1,200,000 bits /
Seconds, audio data 192,000 bits / second, maximum frequency of private data generation is 9,600 bits / second, and total multiplexed data output rate is 1,500,000 bits / second. If the pack length is set to 2,324 bytes according to the sector of the CD-ROM and each packet length is set to 2,286 bytes, there is a probability that the pack for selecting the private data packet is about once in 153 packs.

In this case, even if the private data is input at the maximum rate, a delay of a maximum of about 1.9 seconds occurs due to the multiplexing alone. Further, since asynchronous private data such as still images, FAX, and text data are generally generated in bursts rather than always generated, the delay may be further increased. Further, since the output rate fluctuates due to the burst property, it is not suitable for an output medium such as a communication line that requires a constant rate output.

On the other hand, in the data multiplexer of the present embodiment, video data and audio data that are input at a constant rate and have a large data amount are inserted into the basic packet unit 14. Also, by inserting private data having a small amount of data and having a burst property into the additional packet unit 15 to form one pack, the multiplexing delay can be reduced.

That is, in the above specific example, the private data packet data length PL is (1)
Since it is 15 bytes from the formula, if the packet header is 9 bytes, the length of the additional packet portion 15 is 24 bytes, and the length of the basic packet portion 14 is 2,262 bytes. Therefore, assuming that private data is always input at the maximum rate, the multiplexing delay becomes about 12 milliseconds at the maximum, which can be greatly reduced compared to the conventional maximum of about 1.9 seconds.

Further, in the data multiplexing apparatus of the present embodiment, either one or both of the private data and the padding data are used when the basic packet unit 14 is generated, so that constant rate output is possible. Is.

That is, when the private data is not generated, the additional packet unit 15 is configured by using the padding data, so that the output rate of the multiplexed data can be made constant.

The embodiment of the first data multiplexing apparatus of the present invention has been described above by taking the case of multiplexing video data, audio data and private data based on the MPEG standard as an example. The data to be multiplexed by the first multiplexer according to the invention includes video data, audio data and private data.
It may be data other than data. The data compression / multiplexing technique may be other than the MPEG system.

As described above, according to the first data multiplexing apparatus of the present invention, the pack is configured with two types of the basic packet portion and the additional packet portion, and the private packet
Since the data packet length can be configured according to the maximum input rate, asynchronous private data can be output, the multiplexing delay can be dramatically reduced, and the multiplexed data can be transmitted at a constant rate. Can be output with.

A video data / audio data multiplexer according to an embodiment of the second data multiplexer of the present invention will be described below. As shown in FIG. 6, this multiplexer includes an input terminal E10 to which video data, audio data private data and padding data are input, a basic packet part generating means B10 and an additional packet part generating means C10. Packet part generating means A10 including the pack part generating means D10 and output terminal F1
It has 0 and.

The basic packet part generating means B10 is a first basic packet part generating means B10 ₁ for generating a first basic packet part into which video data inputted via the input terminal E10 is inserted, and the input terminal E10. Second basic packet part generating means B10 ₂ for selecting either one of the audio data and the padding data input via the above to generate the second basic packet part.

The additional packet part generating means C10 selects either one or both of the private data and the padding data input via the input terminal E10 to generate the additional packet part, or the private data and padding data. Both the padding data are selected to generate the additional packet part.

The pack generating means D10 adds a pack header to the first and second basic packet portions and the additional packet portion respectively input from the basic packet portion generating means B10 and the additional packet portion generating means C10. Generate a pack.

In the video / audio data multiplexing apparatus of the present embodiment having such a configuration, the packet portion is composed of two types, that is, the basic packet portion and the additional packet portion. Further, since the packet part is composed of the basic packet part alone or the basic packet part and the additional packet part, the multiplexing delay can be suppressed well even in the case of private data synchronized with the video signal. You can

Furthermore, since the additional packet portion is added only to the video packet when multiplexing,
Private data can be multiplexed while maintaining synchronization with the video signal.

Next, the specific structure and operation of the video / audio data multiplexing apparatus of the present invention will be described with reference to FIGS. In this embodiment, as shown in FIG. 7, an audio data input terminal 101 and a video
A data input terminal 102, a private data input terminal 103, a picture information detection unit 104, a picture
Counter 105, basic packet selection unit 107, basic packet unit generation unit 108, and additional packet unit generation unit 10
6, the pack generation unit 109, and the multiplexing buffer unit 110
And a multiplexed data output terminal 111.

At the audio / data input terminal 101,
Audio data 127 encoded (data compressed) by an audio data encoding device (not shown) is input.

To the video data input terminal 102, the video data 122 coded (data compressed) by a video data coding device (not shown) is input. Private data 123 such as closed caption data synchronized with a picture is input to the private data input terminal 103.

The video data 122 input from the video data input terminal 102 is the picture information detection unit 10
4 is input. The picture information detection unit 104 detects whether or not a picture start code exists in the packet data to be generated next time, and detects the number of pictures existing in the packet (that is, the number of closed frames to be multiplexed). -Add the contents of the register of the picture counter 105 by the number of blocks of caption data).

Audio data 127 input from the audio data input terminal 101 and video data 1 whose picture information is detected by the picture information detection unit 104.
22 is given to the basic packet selection unit 107, respectively.

In the basic packet selection unit 107, whichever of the video data 122 and the audio data 127 is used for constructing a pack, the basic packet unit 107 assumes the data generation condition and the buffer condition in the virtual separation / decoding device. It is judged whether or not it is possible to smoothly perform the decoding.

Then, based on this result, one of the video data 122 and the audio data 127 is selected. The basic packet selection unit 107
Video data 122 and audio data 127
If it is determined that neither of the above is appropriate, padding data 124 (redundant data) is generated.

From the basic packet selection unit 107, the selected video data 122 or audio data 127 or the generated padding data 124 is cut out by the packet data length to generate a basic packet unit as basic packet data S104. It is output to the unit 108. In addition, the basic packet selection unit 107 outputs packet information S101 indicating whether the selected data is the video data 122 to the additional packet unit generation unit 106.

The basic packet part generator 108 adds the basic packet header 125 (see FIG. 8B) to the basic packet data S104 to generate the basic packet part 121. Here, the basic packet header 125 is composed of a packet start code indicating the start of a packet, packet length, time stamp information indicating the decoding time of video data, audio data, and private data, and the like.

The private data 123 is given to the additional packet section generator 106. The picture counter 105 counts the number of blocks of the private data 123 (closed caption data) given via the additional packet unit generation unit 106, and adds the number of pictures detected by the picture information detection unit 104. At the same time, when the additional packet unit generation unit 106 outputs the additional packet unit 128 corresponding to the picture, the number of pictures is subtracted.

In the additional packet portion generation unit 106, the private data 123 is cut out by the length of the closed caption data, and the additional packet header 126 is added to the cut out private data 123 (see FIG. 8B). Is added to the private data 12
An additional packet unit 128 containing 3 is generated.

In this case, the condition is that the packet information S101 output from the basic packet selection unit 107 indicates that the additional packet header 126 has been selected, and that the content of the register of the picture counter 105 is not "0". As a result, the private data 123 is cut out and the additional packet header 126 is added.

The private data 123 is a packet.
If the data length does not exist, the additional packet unit 128 is generated using the existing private data 123 and the padding data 124 generated corresponding to the surplus data length ((c in FIG. 8). )reference).

Private data 12 to be sent
When the length of 3 (closed caption data) is longer than the length of the predetermined additional packet portion 128 (for example, when one video packet includes a plurality of pictures, or closed caption for one picture) -If the data is large), the remaining length is output at the timing of the next video packet.

When the content of the register of the picture counter 105 is "0", the additional packet portion 128 is generated using only the padding data 124 (see (d) in FIG. 8).

The additional packet unit generator 106 outputs the private data 1 output as the additional packet unit 128.
The unit of 23 is output to the picture counter 105, and the picture counter 105 subtracts the contents of the register by this number.

When the basic packet unit 121 output from the basic packet unit generation unit 108 includes the video data 122, the pack generation unit 109 and the additional packet unit 128 output from the additional packet unit generation unit 106 and the pack. A header 120 is added to generate a pack (see FIG. 8A). Here, the pack header 120 is composed of information such as a pack identification code and a system clock reference serving as a reference for the above-mentioned time stamp.

The multiplexing buffer unit 110 temporarily stores the multiplexed data output from the pack generation unit 109 and adjusts the burst property. The burst data adjusted multiplexed data is output to the outside via the multiplexed data output terminal 111, recorded on a recording medium, or transmitted via a transmission path.

Next, this video data / audio
The data structure of the multiplexed data multiplexed by the data multiplexing device will be described with reference to FIGS. 8 and 9. The multiplexed data is shown in (a) of FIG. 8 and (a) of FIG.
It consists of a continuous pack, as shown in. Here, as described above, each pack includes a pack header 120 indicating information such as time reference information and a first basic packet portion 121a.
And the additional packet part 128 or the pack header 1
20 and the second basic packet unit 121b.

Here, the first basic packet portion 121a
Is composed of video data 122 to which a basic packet header 125 is added, as shown in (a) to (d) of FIG. 8 and (a) to (d) of FIG.

Alternatively, the second basic packet part 121b is
As shown on the right side of FIGS. 8A to 8D, the audio data 12 to which the basic packet header 125 is added is added.
7, or as shown on the right side of (a) to (d) of FIG.
It is composed of padding data 124 to which a basic packet header 125 is added.

The structure of the additional packet unit 128 is divided into three types, first to third types. The first type additional packet unit 128 is composed of private data 123 to which an additional packet header 126 is added, as shown in FIGS. 8B and 9B.

The second type additional packet unit 128 is
As shown in (c) of FIG. 8 and (c) of FIG. 9, it is composed of private data 123 to which the additional packet header 126 is added and padding data 124 to which the additional packet header 126 is added. .

The third type additional packet unit 128 is
As shown in (d) of FIG. 8 and (d) of FIG. 9, a basic packet header 125 to which the video data 122 is added.
And the additional packet header 126 is added to the padding data 124.

The first and second types of additional packet parts 128 are selected by the first basic packet part 121a.
This is the case where there is a picture start code in it, or there is private data 123 (closed caption data) that could not be sent by the additional packet unit 128 last time.

In neither case, the third type additional packet unit 128 is selected. Private data 123 (closed caption
Data) is present in a predetermined amount or more, the private data 123 (closed caption data) is constituted by only the first type additional packet part 128.
Is less than a predetermined amount, it is composed of private data 123 having a data length that can be cut out and padding data 124 for adjusting the remaining length.

As described above, in the video data / audio data multiplexing device of the present embodiment, the pack containing the video data 122 is stored in the first basic packet unit 12 as a pack.
Since the additional packet unit 128 includes the 1a and the additional packet unit 128 including the private data 123, the additional packet length can be reduced, and the multiplexing delay can be suppressed.

When the first basic packet portion 121a includes the picture start code, the additional packet portion 128 including the private data 123 is inserted, so that the video data 122 and the private data 123 are combined. It can be completely synchronized.

That is, for example, the video data 122
Occurrence frequency (that is, bit rate) is 1,200,000 bits / sec, and the number of pictures per second of the video data 122 is 3
0, the frequency of audio data 127 is 192,000 bits / second, and the private data 123 is 2 bytes of closed caption data per picture, the frequency of private data 123 (bit rate) is 480 bits / second. Becomes

The output rate of the entire multiplexed data is 1,500,000 bits / second, and the pack length is CD-R.
2,324 bytes for each OM sector, each packet length
With 2,286 bytes, there is a probability that the pack that selects the private data 123 is about once for every 3,074 packs.

In this case, in the conventional video data / audio data multiplexer, a delay of up to about 38.1 seconds occurs due to the multiplexing alone. Also, since closed caption data for 1,143 pictures is multiplexed at once in the additional packet portion, it is difficult to perform decoding in synchronization with the video data 122 on the decoding device side.

On the other hand, in the video data / audio data multiplexer of the present embodiment, the video data 122 and the audio data 12 having a large data amount are recorded.
7 and padding data 12 for rate adjustment
4 is used as the basic packet portion 121, and the private data 123 having a small data amount and the padding data 124 packet for adjusting the pack length are
The additional packet unit 128 having a short packet length is multiplexed.

That is, in the video data / audio data multiplexing apparatus of the present embodiment, in the case of the packet of the video data 122, the first basic packet unit 1
21a and the additional packet unit 128 constitute a pack, and audio data 127 or padding data 12
In the case of a packet of 4, the second basic packet part 121
By configuring the pack with only b, the additional packet length can be shortened and the multiplexing delay can be suppressed.

When the picture start code is present in the packet of the video data 122, the additional packet portion 128 including the private data 123 is added to synchronize the picture with the closed caption data. Can be taken.

In the above specific example, the additional packet unit 12
If 8 is generated for each picture, the packet data length is 2 bytes, and if the packet header is 8 bytes, the length of the additional packet part 128 is 10 bytes,
First basic packet portion 12 containing video data 122
The packet length of 1a is 2,276 bytes, and the length of the other basic packet portion 121 is 2,286 bytes.

The delay caused by the multiplexing at this time is about 33 milliseconds at maximum. Also, picture start
Closed caption data can be multiplexed at approximately the same time as the code so that synchronization between the picture and the closed caption data is maintained.

In the video data / audio data multiplexing device of the present embodiment, since the synchronization between the video signal and the private data can be maintained well,
For example, in karaoke communication, it can be favorably used when changing the character color in accordance with the progress of a song.

The embodiment of the second multiplexer of the present invention has been described above by taking the case of multiplexing video data, audio data and private data based on the MPEG standard as an example. By the second
The data to be multiplexed by the multiplexing device may be other than video data, audio data and private data. The data compression / multiplexing technique may be other than the MPEG system.

As described above, according to the second data multiplexing apparatus of the present invention, when multiplexing including private data synchronized with a video signal, private data is added in pairs with a video packet. Since the basic packet portion is configured by this, the multiplexing delay can be dramatically reduced, and the private data can be multiplexed while always synchronizing with the video signal.

Next, a third embodiment of the data multiplexing apparatus of the present invention will be described with reference to FIGS. 10 and 11. As shown in FIG. 10, in the data multiplexing apparatus of the present embodiment, a video data input terminal 201, an audio data input terminal 202, a private data input terminal 203, and a video PES packet generation unit 204.
An audio PES packet generator 205, a private PES packet generator 206, a picture information detector 207, and a transport packet selector 208.
A transport packet generator 209, a multiplexing buffer 210, and a multiplexed data output terminal 211.

Video data 224 coded (data compressed) by the video data coding apparatus is input to the video data input terminal 201. Audio data 225 encoded (data compressed) by an audio data encoder (not shown) is input to the audio data input terminal 202. Further, private data 226 synchronized with a picture, such as closed caption data, is input to the private data input terminal 203.

The video PES packet generator 204 cuts out the video data 224 input from the video data input terminal 201 by a predetermined size, and then adds header information to the cut out video data 224 to add video information. A PES packet (see FIG. 11B) is generated. The header information added by the video PES packet generation unit 204 is packet start code indicating the start of the PES packet, packet length, time stamp information indicating the decoding time of video data, audio data, and private data. And so on.

The picture information detecting unit 207 receives the video PE supplied from the video PES packet generating unit 204.
Next transport packet 221 among S packets
A picture start code 228 indicating the beginning of a picture is displayed in the portion selected as (see (a) of FIG. 11).
(See FIG. 11B), the detection result information S201 is detected as private data PES.
The data is output to the packet generation unit 206 and the transport packet selection unit 208.

Audio PES packet generator 205
11 cuts the audio data 225 input from the audio data input terminal 202 by a predetermined size, adds header information to the cut audio data 225, and adds the audio PES packet (see FIG. 11).
(See (d)).

When the detection result information S201 provided from the picture information detection unit 207 indicates the presence of a picture, the private data PES packet generation unit 206 outputs the private data 226 corresponding to the detected picture. Private data input terminal 2
03, and adds header information to this to generate a private data PES packet (see (c) of FIG. 11).

Transport packet selection unit 208
Selects the payload of the transport packet 221 to be output this time from the video data 224, the audio data 225, the private data 226 and the padding data 227 to determine the PES packet data, and the determined PES packet The data is cut out by the data length of the transport packet 221 and output. In this case, the transport packet selector 2
08, first determines whether or not to select the private data 226. When the private data 226 is not selected, the video data 224, the audio data 225, and the padding data 227 are appropriately selected.

That is, the transport packet selecting section 208 determines that the payload of the immediately previous transport packet 221 is the picture start
If it was video data 224 containing code 228, select private data 226. Specifically, the transport packet selection unit 208 selects the detection result information S2 provided from the picture information detection unit 207 when the video data 224 is selected in the second determination.
When 01 indicates the existence of a picture, a flag is set. And the next transport packet 2
If the flag is set in the determination of No. 21, the transport packet selection unit 208 determines that the private data 2
After it is determined that 26 is selected, the flag is lowered.

In the second determination, the transport packet selection unit 208 assumes that the video data 224 and the audio data 225 will be stored in consideration of the data generation status and the buffer status in the virtual separation / decoding device. Which of the PES packets 230 is put in is selected based on whether decoding can be performed smoothly. The transport / packet selection unit 208
Is video data 224 and audio data 2
If it is determined that neither of the 25 is suitable, the padding data 227 (redundant data) is selected.

The transport packet generator 209
Header information and an adaptation field 223 are added to the data cut out by the transport packet selection unit 208 to generate a transport packet 221. The header information added here is a synchronization byte, a packet identification code, and the like. Further, the adaptation field 223 is composed of information such as a program clock reference serving as a time reference for the above-mentioned time stamp.

By the way, when the transport packet 221 is generated as described above when the type of the PES packet 230 is the private data 226, the size of the PES packet 230 is the transport packet.
It is often smaller than the data area of the packet 221. In this case, the adaptation field 22
The length of the transport packet 221 is adjusted so that the size of the transport packet 221 becomes constant by using the optional stuffing byte of 3.

The multiplexing buffer unit 210 temporarily stores the multiplexed data output from the transport packet generation unit 209 and adjusts the burst property. The burst-adjusted multiplexed data is output to the outside via the multiplexed data output terminal 211, recorded on a recording medium, or transmitted via a transmission path.

Next, this video data / audio
The data structure of the multiplexed data multiplexed by the data multiplexing device will be described with reference to FIG. The multiplexed data is composed of a plurality of consecutive transport packets 221 as shown in FIG. here,
The transport packet 221 is a transport packet.
It is composed of a packet header 222, a payload and an adaptation field 223. Here, the adaptation field 223 is used for information such as a time reference and for adjusting the length of the transport packet 221.

The payload includes video data 224, audio data 225 and private data 2.
A fragment of 26 PES packets, padding,
It consists of data 227. The criterion for selecting the private data 226 as the payload is that the payload of the previous transport packet 221 is the picture
It is the video data 224 including the start code 228.

As described above, in the video data / audio data multiplexing device of the present embodiment, in the case of private data 226 synchronized with video data 224, private data 2 generated in one picture is
Depending on the length of 26, P of private data 226
By determining the size of the ES packet 230, multiplexing delay is possible.

Transport packet 22 of video data 224 containing picture start code 228
By making the transport packet 221 next to 1 the transport packet 221 of the private data 226, the video data 224 and the private data 226 can be completely synchronized.

For example, the frequency of occurrence of the video data 224 (that is, the bit rate) is 1,200,000 bits / second,
The number of pictures per second of the video data 224 is 30, and the frequency of occurrence of the audio data 225 is 192,000 bits / second. When the private data 226 is closed caption data of 2 bytes per picture, the occurrence frequency (bit rate) of the private data 226 is 480 bits / second.

Then, assuming that the output rate of the entire multiplexed data is 1,500,000 bits / sec, the transport packet length is 188 bytes matched to the cells of the asynchronous transfer mode (ATM), and each PES packet length is 2,256 bytes.
If the header length of the PES packet 230 of the private data 226 is set to 9 bytes, the conventional video data /
In the audio data multiplexer, the multiplexing delay of the private data 226 is about 6 seconds.

Since the closed caption data for 88 to 92 pictures is multiplexed in the transport packet 221 of the private data 226 at one time, the video data 224 on the decoding device side.
It becomes difficult to perform synchronized decoding with.

On the other hand, in the video data / audio data multiplexing apparatus of this embodiment, the PES packet length of the private data 226 is equal to the PES packet length of the video data 224 and the audio data 225. Not as an arbitrary length, but as a unit of the amount of private data 226 corresponding to one picture,
Multiplex.

That is, the transport packet 221
The internal private data 226 is generated in short units so that the multiplexing delay can be suppressed. Also, when selecting the transport packet 221,
When the payload of the previous transport packet 221 is the video data 224 including the picture start code 228, the payload of the current transport packet 221 is set to the private data 22.
By setting the number to 6, the video data 224 and the private data 226 can be perfectly synchronized, and the picture and the closed caption data can be synchronized.

In the above specific example, if the PES packet 230 of the private data 226 is generated for one picture, the PES packet length is 2 bytes, and P
If the ES packet header 230a is 9 bytes, PE
The S packet length is 11 bytes.

Therefore, the length of the adaptation field 223 is 173 bytes, and the delay caused by the multiplexing at this time is about 1 millisecond. Further, since the closed caption data can be multiplexed almost at the same time as the picture start code 228, the synchronization between the picture and the closed caption data is maintained.

The third embodiment of the multiplexer of the present invention has been described above by taking the case of multiplexing video data, audio data and private data based on the MPEG standard as an example. By the third
The data to be multiplexed by the multiplexing device may be data other than video data, audio data and private data. The data compression / multiplexing technique may be other than the MPEG system.

As described above, according to the third data multiplexing apparatus of the present invention, when multiplexing including private data synchronized with video data is performed, private data can be output in units of one picture. With such a configuration, it is possible to dramatically reduce the multiplexing delay, and it is possible to multiplex the private data while always synchronizing with the video data.

Next, a first embodiment of the fourth data multiplexing apparatus of the present invention will be described. The video data / audio data multiplexer of the present embodiment is a PE
It is used as a multi-program compatible video data / audio data multiplexing device using S (Packed Elementary Stream) packets.

As shown in FIG. 12, the video data / audio data multiplexer of this embodiment is used as a video / audio data multiplexer compatible with multi-program using PES packets. An input terminal D30 to which private data 326 multiplexed with video data and audio data is input, a PES packet generating means A30, a PES packet length changing means B30, a padding data filling means C30, and a private. A data PES packet output terminal E30 is provided.

The PES packet generating means A30 generates a PES packet of private data 326 input via the input terminal D30. The PES packet length changing means B30 adaptively changes the length of the PES packet of the private data 326 generated by the PES packet generating means A30. The padding data packing means C30 uses the private data 32 to be multiplexed.
The padding data (redundant data) is adaptively packed according to the amount of 6.

According to the video data / audio data multiplexing apparatus of this embodiment having the above-mentioned configuration,
Upon receiving the notification from the PES packet length changing unit B30, the PES packet generating unit A30 inputs the private data 326 via the input terminal D30 and adds the header information to the input private data 326. Then, a PES packet is generated.

At this time, when the data length of the existing private data 326 is smaller than the preset predetermined data length, the PES packet is generated with the inputtable data length, and the remaining portion is padded. The data is packed by the padding / data packing means C30. If the private data 326 does not exist, no PES packet is generated and the padding data is packed by the padding data packing means C30.

Therefore, according to the video / audio data multiplexer of the present embodiment, the length of the PES packet of the private data 326 can be adaptively changed, so that the private data asynchronous to the video data can be changed. -When multiplexing including the data 326 is performed, the delay due to the multiplexing can be suppressed low.

When the data length of the private data 326 is smaller than a predetermined data length or when the private data 326 does not exist, the padding data packing means C30 packs the padding data. Therefore, the output rate can be kept constant. Therefore, the video / audio data multiplexer of the present embodiment can be effectively used in a communication system.

Next, the specific structure and operation of the video data / audio data multiplexing apparatus according to the fourth embodiment of the present invention will be described in detail with reference to FIGS.

As shown in FIG. 13, the video data / audio data multiplexing apparatus of the present embodiment has a video data input terminal 301, an audio data input terminal 302, and a private data input terminal 303. ,
Video PES packet generator 304 and audio PE
S packet generation unit 305 and PES packet length determination unit 3
06 ₁ and private data PES packet generation section 306 having the private data timer 307
A transport packet selecting unit 308 having a payload determining unit 308 ₁ , a stuffing byte length adjusting unit 309 ₁ and a padding data filling unit 30.
9 ₂ transport packet generation unit 309 with
And a multiplexed buffer unit 310 and a multiplexed data output terminal 311.

Video data encoded (data compression) by a video data encoding device (not shown) is input to the video data input terminal 301. Audio data encoded (data compressed) by an audio data encoding device (not shown) is input to the audio data input terminal 302. In addition, the private data input terminal 303 has still image data and FAX.
Asynchronous private data 326 (see (a) of FIG. 14) is input to video data such as data.

The video PES packet generator 304 cuts out the video data input from the video data input terminal 301 by a predetermined size and adds header information to the cut out video data. Then, a video PES packet (see (e) in FIG. 14) is generated. The header information added by the video PES packet generation unit 304 includes a packet start code indicating the start of a PES packet, a packet length, time stamp information indicating the decoding time of video data and audio data, and the like.

Audio PES packet generator 305
Cut out audio data input from the audio data input terminal 302 by a predetermined size, add header information to the cut out audio data, and add an audio PES packet (see (f) in FIG. 14). To generate.

The private data timer 307 is
This corresponds to the PES packet length changing unit B30 shown in FIG. 12, calculates the timing at which the private data PES packet should be generated, and when a predetermined time has elapsed, the private data PES packet generation unit 306.
Also, the generation timing signal S301 is output to the transport packet selection unit 308.

The time interval T for generating a private data PES packet is the transport packet 320.
The maximum payload length (see FIG. 14A), the minimum length AL of the adaptation field 324 (see FIG. 14B), and the private data 326.
Using the maximum bit rate PR of, the following equation (2) is used. T = (TP-AL) / PR (2)

The private data PES packet generator 306 is the PES packet generator A3 shown in FIG.
It corresponds to 0. This private data P
The ES packet generation unit 306 uses the private data
Upon receiving the generation timing signal S301 from the timer 307, the PES packet length determination unit 306 _1, the maximum payload of the transport packet 320 length TP, the minimum length AL and private data PES adaptation field 324 Using the header length HL of the packet, the maximum data length DL of the private data PES packet is obtained from the following equation (3).

Then, after that, the maximum data length DL obtained
The private data 326 is input via the private data input terminal 303 by the amount, and header information is added to the input private data 326 to generate a private data PES packet. DL = TP-AL-HL (3)

Here, the private data PES packet generator 306 receives the input private data 3
When the data length of 26 is smaller than the obtained maximum data length DL, the private data PES packet is generated with the inputtable data length.

When the private data PES packet generator 306 receives the generation timing signal S301 and the private data 326 does not exist, the private data PES packet generator 306 does not generate the private data PES packet.

Transport packet selector 308
When receiving the generation timing signal S301 from the private data timer 307, the payload determination unit 30
8 ₁ , the payload of the transport packet 320 output this time is set to video data, audio data, private data 326 and padding.
After the data (redundant data) 325 is selected and determined, the determined data is transferred to the transport packet 32.
The data length of 0 is cut and output.

That is, when the transport packet selector 308 receives the generation timing signal S301 from the private data timer 307, it first selects the private data 326. And
At this time, when the private data PES packet is not output from the private data PES packet generation unit 306, the padding data 325 is selected. In other cases, the video data, the audio data, and the padding data 325 are appropriately selected.

Selection from video data, audio data and padding data is performed as follows. First, data generation status and virtual separation /
Based on whether the PES packet of the video data or the audio data can be smoothly decoded, assuming the buffer situation in the decoding device, etc. Select one of the data. At this time, if neither the video data nor the audio data is appropriate, the padding data 325 is selected.

Transport packet generator 309
Adds the header information and the adaptation field 324 to the data cut out by the transport packet selecting unit 308 and adds the transport packet 320.
Generate The header information added here is a synchronization byte, a packet identification code, and the like.

In addition, the adaptation field 32
4 includes information such as a program clock reference serving as a time reference for the above-mentioned time stamp. The transport packet generator 309
Has a PES packet type of private data 3
If 26, adaptation field 3
Be sure to insert 24.

Also, the transport packet generator 3
If the data length of the PES packet is smaller than the maximum data length DL obtained by the equation (3), the stuffing byte length adjusting unit 309 ₁ stores an optional stuffing byte (of the adaptation field 324) The length is adjusted so that the size of the transport packet 320 becomes constant by using (c) of FIG.

[0158] In addition, the transport packet generation unit 309, padding when padding data 325 from the transport packet selector 308 is sent, the padding data packing unit 309 _2, sent The data 325 is packed in the payload to generate the transport packet 320 (see FIG. 1).
4 (c)).

The transport packet selector 308 and transport packet generator 309 are shown in FIG.
This corresponds to the padding / data stuffing means C30 shown in FIG.

The multiplexing buffer unit 310 temporarily stores the multiplexed data output from the transport packet generation unit 309 and adjusts the burst property. The burst data adjusted multiplexed data is output to the outside via the multiplexed data output terminal 311, recorded on a recording medium, or transmitted via a transmission path.

Next, this video data / audio
Data structure of multiplexed data multiplexed by data multiplexer and transport of private data
The data structure of the packet will be described with reference to FIG. The multiplexed data is composed of a series of transport packets 320 as shown in FIG.

Here, the transport packet 320
Is composed of a transport packet header 321, a payload and an adaptation field 323. Where the adaptation field is
Information such as time base and transport packet 320
Used to adjust the length of.

The payload is a fragment of PES packet 322 of video data, audio data and private data, and padding data 325.
Consists of

As shown in FIGS. 14A and 14B, the private data timer 30 is used as the criterion for selecting the private data 326 for the payload.
The private data 326 existing at this time is determined by the generation timing signal S301 output from
There are three types depending on the size of.

The first type is an existing private
This is a case where the data length L of the data 326 is equal to the maximum data length DL obtained by the equation (3). In this case, FIG.
4 (a), only the adaptation field 323 and the private data PES packet 322 are used, and the adaptation field 32
Transport packet 320 is constructed without the use of three stuffing bytes 324. Reference numeral 327 indicates a PES packet header.

The second type is an existing private
This is a case where the data length L of the data 326 is not "0" and is smaller than the maximum data length DL obtained by the equation (3). In this case, as shown in FIG. 14B, the stuffing bytes 324 of the adaptation field 323 are packed by the difference (DL-L) to make the length of the transport packet 320 constant and A packet 320 is constructed

The third type is an existing private
This is the case where the data length L of the data 326 is “0”. In this case, as shown in (c) of FIG. 14, the payload is all padded data 325 to form the transport packet 320. An example of the multiplexed data is shown in (d) of FIG. 14, an example of the video PES packet is shown in (e) of FIG. 14, and an example of the audio PES packet is shown in (f) of FIG.

Details of the transport stream in "MPEG2" will be described with reference to FIG.
explain. The transport stream in "MPEG2" is composed of a series of transport packets, as shown in FIG. The transport packet is, as shown in FIG.
Sync byte, error indication, unit start indication, transport packet priority, PID (PacketIden
tification), scramble control, adaptation
It consists of field control, cyclic counter (continuity counter) and adaptation field.

The contents of the adaptation field are
As shown in FIG. 15C, adaptation field length, discontinuous display, random access display, stream priority display, 5 flags, conditional coding (optional field), stuffing
For example, a byte.

The contents of conditional coding (optional field) are as shown in FIG.
PCR (Program Clock Reference), OPCR (Origin
al Program Clock Reference), sprite count down, transport private data length and data and adaptation field extension.

As described above, in the video data / audio data multiplexing apparatus of this embodiment, in the case of private data asynchronous with the video data, the size of the PES packet of the private data (data length) )
Is varied by the size of the payload of the transport packet or less to suppress the multiplexing delay. When private data is not input, padding data is packed to keep the output rate constant.

For example, the transport packet length is 188 bytes matched to the cells of the asynchronous transfer mode (ATM), each PES packet length is 2,256 bytes, the private data PES packet header length is 9 bytes, and the entire multiplexing is performed. Data output rate of 1,500,000 bits / second,
If you set the maximum frequency of private data (that is, the maximum input bit rate) to 9,600 bits / second and the multiplexing time is 60 seconds to generate 30,000 bytes of private data, the average rate will be 400 bits / second. In the conventional video / audio data multiplexer, the multiplexing delay is about 3.3 seconds on average.

The total multiplexed data is reduced by 69,000 bytes, and the actual output rate of the multiplexed data is 1,490,80.
It becomes 0 bits / second, and the target output rate is not met.

On the other hand, in the video data / audio data multiplexing apparatus of this embodiment, the private data
The PES packet length of data is set as the payload length of the maximum transport packet, and the PE of private data is determined by the amount of private data that exists at the time when private data should be sent.
The S packet length is adaptively changed.

That is, the private data in the transport packet is generated in short units so that the multiplexing delay can be suppressed. Further, when private data is selected, if the private data does not exist, the padding data is packed, so that the output rate can be made constant.

In the above specific example, since the PES packet length of private data is adaptively changed, assuming that 400 bits / sec are generated evenly, the header length of the PES packet is 9 bytes, and the data length is about 7 bytes. Next to
The delay due to multiplexing is about 140 ms.

Also, in the conventional video data / audio data multiplexer, the output multiplexed data is reduced by 69,000 bytes.
In the audio data multiplexer, stuffing bytes and padding data are packed in this amount, so the output rate is fixed at 1,500,000 bits / sec.

The embodiment of the fourth data multiplexer of the present invention has been described above by taking the case of multiplexing video data, audio data and private data based on the MPEG standard as an example. The data to be multiplexed by the fourth data multiplexing device according to
It may be other than video data, audio data and private data. The data compression / multiplexing technique may be other than the MPEG system.

[0179]

As described above, according to the first data multiplexing apparatus of the present invention, a pack is constructed with two types of basic packet part and additional packet part, and padding
Since the data packet length can be configured according to the maximum input rate, asynchronous private data can be output, the multiplexing delay can be dramatically reduced, and the multiplexed data can be transmitted at a constant rate. Can be output with.

According to the second data multiplexing apparatus of the present invention, when multiplexing including private data synchronized with a video signal is performed, private data is added in pairs with a video packet to form a basic packet. Since it is configured, it is possible to dramatically reduce the multiplexing delay, and it is possible to multiplex the private data while always synchronizing with the video signal.

According to the third data multiplexing apparatus of the present invention, the private data can be output in units of one picture when the multiplexing including the private data synchronized with the video data is performed. Therefore, the multiplexing delay can be dramatically reduced, and the private data can be multiplexed while always synchronizing with the video data.

According to the fourth data multiplexer of the present invention, when the PES packet of the private data to be multiplexed with the video data and the audio data is generated, the PES packet length of the private data is adaptively adjusted. Since the padding data is adaptively packed according to the amount of private data, the delay due to the multiplexing including the video data and the asynchronous private data can be kept low.

If there is no private data or the amount of private data is small, the stuffing bytes and padding data are packed, so that the output rate can be kept constant and the communication system It is possible to provide an effective video data / audio data multiplexing device.

[Brief description of drawings]

FIG. 1 is a block diagram showing main components of a video data / audio data multiplexer according to a first embodiment of a first data multiplexer of the present invention.

FIG. 2 is a block diagram showing a video data / audio data multiplexer according to a second embodiment of the first data multiplexer of the present invention.

3 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

4 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

5 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

FIG. 6 is a block diagram showing main components of a video data / audio data multiplexer according to a first embodiment of a second data multiplexer of the present invention.

FIG. 7 is a block diagram showing a video data / audio data multiplexer according to a second embodiment of the second data multiplexer of the present invention.

8 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

9 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

FIG. 10 is a block diagram showing a packet-based multi-channel video data / audio data multiplexer according to an embodiment of a third data multiplexer of the present invention.

11 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

FIG. 12 is a block diagram showing main components of a video data / audio data multiplexer according to a first embodiment of a fourth data multiplexer of the present invention.

FIG. 13 is a block diagram showing a packet-based multi-channel video data / audio data multiplexer according to a second embodiment of the fourth data multiplexer of the present invention.

14 is a diagram showing an example of a data structure of multiplexed data output from the video data / audio data multiplexing device shown in FIG.

[Fig. 15] Fig. 15 is a diagram illustrating details of a transport stream in "MPEG2".

[Explanation of symbols]

 A packet part generating means B basic packet part generating means C additional packet part generating means D pack generating means 1 video data input terminal 2 audio data input terminal 3 private data input terminal 4 basic packet selecting section 5 basic packet generating section 6 Pack generation unit 7 Private data packet generation unit 8 Multiplexing buffer unit 9 Multiplexed data output terminal

 ─────────────────────────────────────────────────── ─── Continuation of the front page (31) Priority claim number Japanese Patent Application No. 7-188250 (32) Priority date Hei 7 (1995) June 30 (33) Priority claim country Japan (JP)

Claims (20)

[Claims] 1. A first aggregation unit including first data, a second aggregation unit including second data, a third aggregation unit asynchronous with at least one of the first data and the second data.
Is a data multiplexing device that multiplexes the first to third data by generating a plurality of aggregates by using a third aggregation part including the data of Generating means, an additional aggregate part generating means for generating an additional aggregate part, and an aggregate generating means for generating the aggregate from the basic aggregate part and the additional aggregate part, the basic aggregate part generating means, One data is selected from the first data, the second data, and the first redundant data, and when the first data is selected, the first aggregation unit is generated, When the second data is selected, the second aggregation unit is generated, and when the first redundant data is selected, the fourth aggregation unit including the first redundant data is generated, The generated first, second or fourth
Of the third data and the second redundant data, or both of the third data and the second redundant data. If the third data is selected, a third aggregation part is generated, and if the second redundant data is selected, a fifth aggregation part including the second redundant data is generated. However, when both the third data and the second redundant data are selected, a sixth aggregation part including both the third data and the second redundant data is generated and the generated A data multiplexing apparatus, wherein a third, fifth or sixth aggregation unit is output as the additional aggregation unit. 2. The data multiplexing apparatus according to claim 1, wherein the data multiplexing apparatus is an MPEG (Moving Picture E).
xperts Group) data multiplexing device, wherein the first data is video data, the second data is audio data, the third data is private data, and the third data is A data multiplexing device, wherein the first and second redundant data are padding data. 3. A data multiplexer that divides input data into a fixed size to generate a packet part, and adds a pack header to the packet part to generate a pack. And a packet part generating means for generating a packet part generating means for generating the packet part generating means and the additional packet part generating means. One of the data is selected to generate a basic packet part, and the additional packet part generation means selects and adds either one of the input private data or padding data, or both data. A data multiplexing device characterized by generating a packet part. 4. The data multiplexing apparatus according to claim 3, wherein the length of the additional packet portion is obtained from a pack length, a multiplexing output rate, and a maximum input rate of private data. 5. A basic packet source selection means for selecting one data from input video data, audio data and padding data as basic packet data to be multiplexed in a basic packet portion forming a pack. While dividing the basic packet data selected by the basic packet source selecting means into a predetermined length size,
Basic packet part generation means for generating a basic packet part by adding a basic packet header to the basic packet data divided into the predetermined length size, and input as packet data to be multiplexed in the additional packet part forming the pack. Select either private data or padding data, or both data according to the pack length, and
Additional packet part generation means for adding an additional packet header to the selected additional packet data to generate an additional packet part, and a pack header for each packet part given from the basic packet part generation means and the additional packet part generation means, respectively. And a pack generating means for generating a pack by adding the. 6. The data multiplexing apparatus according to claim 5, wherein the length of the additional packet portion is obtained from a pack length, a multiplexing output rate, and a maximum input rate of private data. 7. A first aggregation part including first data, a second aggregation part including second data, and a third aggregation part including third data synchronized with the first data. A data multiplexing device that multiplexes the first to third data by generating a plurality of aggregates by using a first aggregation unit that includes the first data. Selecting a data of either one of the second data and the first redundant data, and generating the second aggregation part when the second data is selected, Basic set part generation means having a second basic set part generation means for generating a fourth set part including the first redundant data when the first redundant data is selected; and an additional set part. Means for generating an additional aggregation part, and the aggregate from the basic aggregation part and the additional aggregation part And an aggregate generating unit that generates a body, wherein the additional aggregate unit generating unit is one of the third data and the second redundant data, or the third data and the second redundant data. If both are selected and the third data is selected, the third aggregation part is generated, and if the second redundant data is selected, a fifth data including the second redundant data is selected. When the aggregate part of is generated and both the third data and the second redundant data are selected,
Generating a sixth aggregation part including both the third data and the second redundant data, and outputting the generated third, fifth or sixth aggregation part as the additional aggregation part, When the first aggregation unit received from the first basic aggregation unit generation unit includes a predetermined code, the field generation unit generates the received first aggregation unit and the additional aggregation unit. A data multiplexing apparatus, wherein one aggregate is generated from the additional aggregation unit received from the means. 8. The data multiplexing apparatus according to claim 7, wherein the data multiplexing apparatus is an MPEG (Moving Picture E)
xperts Group) data multiplexing device, wherein the first data is video data, the second data is audio data, the third data is private data, and the third data is A data multiplexing device, wherein the first and second redundant data are padding data. 9. A data multiplexing apparatus, wherein input data is divided into a certain size to generate a packet part, and a pack header is added to the packet part to generate a pack. And a packet unit generating means for generating a packet packet generating unit for generating the packet data, the first packet unit generating unit including the basic packet unit generating unit and the additional packet unit generating unit. And a second basic packet part generating means for generating a second basic packet part in which either one of the input audio data or padding data is inserted.
Of the basic packet part generating means, and the additional packet part generating means generates the additional packet part by selecting either one of the input private data or padding data, or both data. A data multiplexing device characterized by: 10. A picture information detecting means for detecting picture information from input video data, and a source for selecting a source to be multiplexed into a pack from the input video data, audio data or padding data. Selecting means, basic packet part generating means for generating a basic packet part from the source selected by the source selecting means, counting means for counting the number of unmultiplexed blocks of private data synchronized with a video signal, and the private An additional packet part generating means for selecting a source to be multiplexed in the pack from data or padding data, and cutting out the selected source according to the pack length to generate an additional packet part; and the basic packet part. Generating means and the additional packet part generating means Data multiplexing apparatus characterized by comprising a pack generating means for generating a pack by adding a pack header to each packet unit which is generated by the. 11. The data multiplexing apparatus according to claim 10, wherein the private data synchronized with the video signal is closed caption data. 12. A PES (Packetized Elementary Stre)
am) In a multi-program compatible data multiplexer using packets, input video data is cut out by a predetermined size, and header information is added to the cut out video data to generate a video PES packet. Video PE
An S packet generating means, and a picture indicating the beginning of a picture from the video PES packet output from the video PES packet generating unit.
Picture information detecting means for detecting whether or not a start code is present, and cut the input audio data by a predetermined size, add header information to the cut audio data, and output an audio PES packet. The header information is added to the private data corresponding to the picture in which the picture start code is detected, based on the detection result information provided from the audio PES packet generation means to be generated and the picture information detection means, Private data PES packet generation means for generating a data PES packet, means for selecting a PES packet to be output from the video PES packet, audio PES packet and private data PES packet, and the picture And a multiplexing means for outputting a private data PES packet when a picture start code is present in the output video data based on the detection result of the information detection means. Multiplexer. 13. The multiplexing means sets a flag when the detection result information output from the picture information detection means indicates that there is a picture, and when the flag is set, a private data PES packet. 13. The data multiplexing apparatus according to claim 12, wherein it is determined to output, and if the determination is performed, processing for lowering the flag is performed. 14. A multi-program compatible data multiplexing device for multiplexing first data and second data synchronized with the first data, the first data including the first data. Included in the first aggregation part, a first aggregation part generation means for generating an aggregation part of, a second aggregation part generation means for generating a second aggregation part including the second data, Determining means for determining whether or not the first data includes a predetermined code; and the determining means for determining that the first data included in the first collecting portion includes the predetermined code. If it is determined that the second data is collected, the second data collection unit is added after the first data collection unit to multiplex the first data and the second data. A data multiplexing device characterized by: 15. The data multiplexing apparatus according to claim 14, wherein the data multiplexing apparatus is an MPEG (Moving Picture E).
xperts Group) data multiplexing device, wherein the first data is video data and the second data is private data corresponding to one picture. 16. A data multiplexing apparatus compatible with multi-program using PES packets, private data PES packet generating means for generating private data PES packets from private data corresponding to one picture, and said private data. A data multiplexing apparatus comprising: a multiplexing unit that multiplexes private data PES packets generated by the PES packet generation unit after video data including a picture start code. 17. A data multiplexing apparatus compatible with a multi-program, which multiplexes first data and second data asynchronous with the first data, the first data including the first data. And a second aggregation part generation means for generating a second aggregation part including the second data, and a second aggregation part generation means. Length changing means for adaptively changing the length of the second collecting portion, and redundant data filling means for filling the second collecting portion with redundant data according to the data amount of the second data. And the second aggregation output from the first aggregation part generated by the first aggregation part generation means and the redundant data packing means.
And a multiplexing unit that multiplexes the first data and the second data by using the collecting unit. 18. The data multiplexing apparatus according to claim 17, wherein the data multiplexing apparatus is an MPEG (Moving Picture E).
xperts Group) data multiplexing device, wherein the first data is video data and the second data is private data. 19. A PES (Packetized Elementary Stre)
am) in a multi-program compatible data multiplexing device using packets, private data PES packet generation means for generating PES packets of private data to be multiplexed together with video data and audio data, and the private data PES PES packet length changing means for adaptively changing the PES packet length of private data generated by the packet generating means, and adaptively according to the amount of private data to be multiplexed with the video data and audio data. A data multiplexing apparatus comprising padding data packing means for packing padding data. 20. A timing notifying unit for notifying a timing of multiplexing the private data together with the video data and the audio data, and a data amount existing at the time of the timing of the multiplexing being notified from the timing notifying unit. PES packet length determining means for determining the PES packet length of the private data, and generating private data PES packets with the length determined by the PES packet length determining means and multiplexing the private data. Private data PES packet generation means for not generating a private data PES packet when the private data does not exist when the timing is notified, and the timing which receives the notification from the timing notification means. Payload determining means for determining the payload of the transport packet as private data, and the payload determining means determines the payload as private data.
When deciding on data, if the PES packet length of the private data is shorter than the payload length specified in advance, length adjusting means for adjusting the length with the stuffing byte of the adaptation field, Payload determination means that the payload is private
A data multiplexing device comprising padding data packing means for packing the padding data into the payload when private data does not exist when the data is decided.