JP2006186674A - Mpeg stream processing method and processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To output one and the same stream at different timing and to multiplex and output different streams. <P>SOLUTION: An MPEG stream processor includes packet generation sections 9-1 to 9-3 which input video/audio information, add time stamps corresponding to output timing, and generate MPEG packets where null packets are inserted for differences between an output rate and effective packets, storage sections 22 to 24 which stores the MPEG packets which have generated time stamps and the inserted null packets, and packet output sections 10-1 and 10-2 which read each MPEG packet reaching output timing based upon the time stamp added to the stored MPEG packet, delete the time stamp added to the MPEG packet, and judge that a null packet is outputted to the outside or discarded, or another packet is outputted replacing the null packet to output an MPEG packet corresponding to the judgment result. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an MPEG stream processing method and processing apparatus for processing a video stream, an audio stream, and the like.

  In recent years, MPEG systems represented by MPEG-2 and MPEG-4 have attracted attention as stream processing systems for processing video streams, audio streams, and the like.

  In these MPEG systems, two types of schemes are defined: a scheme for multiplexing individual streams such as video information, audio information, and additional data, and a scheme for reproducing each stream while synchronizing them.

  One of them is a program stream (PS: Program Stream) system, and only one program can be included in one stream as in the case of MPEG-1 that has been conventionally used. Since this method can be applied to data transmission and data storage in an error-free environment and can reduce redundancy, it can be used in digital storage media using powerful error correction codes such as DVD. Often used.

The other is a transport stream (TS) system, in which a plurality of programs can be included in one stream. Assuming that this method is applied to an environment where a data transmission error occurs, the redundancy is set to be large correspondingly, so that it is often used in broadcasting, communication networks, and the like.
JP-A-8-339630 Japanese Patent No. 3466417

  By the way, in such an MPEG system, when video, audio, additional data, etc. are encoded and output to the outside in a program stream format, a program stream including a system clock reference (SCR) is included. Must be a packet. In addition, when video, audio, additional data, etc. are encoded and output to the outside in the transport stream format, it is necessary to form a transport stream packet including a program clock reference (PCR). .

  At that time, immediately before outputting video, audio, additional data, etc. in the program stream format or transport stream format, the system clock reference or program clock reference is calculated and the program stream packet or transport is calculated. Since stream packets are generated and output to the outside, it is difficult to time shift from encoding to output, and the system load at the time of output increases. It was.

  Also, in such an MPEG system, when a single program stream or a single transport stream is output, it is output from a single output port at a single output timing. When there are a large number of packets of a single program stream or a single transport stream, there is a problem that not only does it take a long time from the start of output to the completion of output, but also the degree of freedom of output is limited.

  On the other hand, conventionally, when there is a difference between a set output rate and a valid packet, a technique of inserting a NULL (null) packet to fill the difference is known. Alternatively, as described in Patent Document 2 described above, a method of inserting PSI (Program Specific Information) instead of a null packet is also known.

  However, the conventional method of inserting null packets and the method of inserting PSI instead of null packets have a problem that transmission at different timings for each packet output unit for the same stream is not possible. there were. Another problem is that different streams cannot be multiplexed for each packet output unit for the same stream.

  In view of the above circumstances, the present invention enables a time shift from encoding to output when outputting a program stream packet and a transport stream packet, thereby reducing a system load at the time of output. Even when a single program stream or a single transport stream is output, even if there are many packets using multiple output ports and multiple output timings, the time required from output start to output completion should be shortened It is an object of the present invention to provide an MPEG stream processing method and a processing apparatus that can increase the degree of freedom of output.

  In order to achieve the above object, according to the first aspect of the present invention, in the MPEG stream processing method for outputting video information and audio information to the outside in the MPEG packet format, the video information and audio information are converted into MPEG packets. A time stamp corresponding to the output timing is added, a null packet is inserted into the difference between the output rate and the valid packet and stored in the storage area, and then the output timing is determined based on the time stamp added to each MPEG packet. If there is an MPEG packet that is in output timing, the MPEG packet is read from the storage area, the time stamp added to the MPEG packet is deleted, and the null packet Output to the outside, discard, etc. After determining whether to output the replace packet, it is characterized by outputting the MPEG packets in accordance with the determination result to the outside.

  According to a second aspect of the present invention, in the MPEG stream processing apparatus for outputting video information and audio information to the outside in the MPEG packet format, the video information and the audio information are captured, and a time stamp corresponding to the output timing is added. A packet generation unit for generating an MPEG packet in which a null packet is inserted into the difference between the output rate and the effective packet, and a time stamp generated by the packet generation unit and the MPEG packet in which the null packet is inserted is stored Based on the content of the time stamp added to each MPEG packet stored in the storage unit and the storage unit, the MPEG packet at the output timing is read, and the time stamp added to the MPEG packet is deleted. And the null packet Or output to the outside, or discard, is characterized in that it is determined whether to output replaced with other packets, and a packet output unit for outputting the MPEG packets in accordance with the determination result to the outside.

  According to the first and second aspects of the invention, it is possible to send the same stream at different timings for each packet output unit, and for the same stream, different for each packet output unit Multiplexing of streams is possible.

《Reference example》
FIG. 1 is a block diagram showing a reference example of the present invention.

  The MPEG stream processing apparatus 1 shown in this figure has a video stream area 2, an audio stream area 3, and a packetized stream area 4, and a video stream, audio stream, program stream packet, transport stream, A memory 5 that captures and temporarily stores packets and the like, a video encoding unit 7 that captures and encodes video information to be output via the memory bus 6, and stores the encoded video information in the video stream area 2 of the memory 5, and a memory bus 6 After the audio information to be output is encoded via the audio encoding unit 8, the audio encoding unit 8 stores the audio information in the audio stream region 3 of the memory 5, and the video stream region 2 and the audio stream region 3 of the memory 5 via the memory bus 6. Video and audio streams temporarily stored in Using a program clock reference (or system clock reference) output from the CPU while reading a stream, a time stamp, etc., a transport stream packet with a time stamp, or a program stream with a time stamp A packet generator 9 for generating a packet and temporarily storing the packet in the packetized stream area 4 of the memory 5; and each transport stream temporarily stored in the packetized stream area 4 of the memory 5 via the memory bus 6. The time stamp added to the packet (or each program stream packet) is checked, and when it is output, the time attached to each transport stream packet (or each program stream packet) And a packet output unit 10 for outputting to the outside to remove the stamp.

  As shown in FIG. 2, the packet generator 9 includes four synthesizers 12, 13, 14, and 15, to which a program clock reference (PCR) is added and a time stamp is inserted. Generate transport stream packets with

  As shown in FIG. 4, the packet output unit 10 includes an output timing counter 16 that counts output timing, a buffer 17 that temporarily holds a packet read from the memory 5, a time stamp inserted in the packet, and an output timing And a time stamp comparison unit 18 that determines whether or not the output timing has arrived. The transport stream packet can be output to the outside at different timings.

  In the above configuration, when an instruction for encoding video information and an instruction for encoding audio information is issued from the CPU, video information and audio information are captured via the memory bus 6 by the video encoding unit 7 and the audio encoding unit 8, Encoded. The video stream and audio stream thus obtained are temporarily stored in the video stream area 2 and audio stream area 3 of the memory 5, respectively.

  After that, as shown in FIG. 2, the output timing prediction calculation unit 11 formed in the CPU converts the video stream and the audio stream temporarily stored in the video stream area 2 and the audio stream area 3 of the memory 5, respectively. On the other hand, it is checked whether a program stream instruction or a transport stream instruction is instructed. At this time, if a transport stream instruction is given, the output timing prediction calculation unit 11 predicts and calculates a program clock reference necessary for outputting the transport stream packet to the outside.

  Next, it is necessary for the output timing prediction calculation unit 11 to packetize the video stream and the audio stream temporarily stored in the video stream area 2 and the audio stream area 3 on the memory 5 and to output the packet at a specified time. A time stamp that is a continuous value for each transport stream packet and a prediction result of the program clock reference obtained by the above-described calculation operation are output on the memory bus 6.

  Thereby, the packet generation unit 9 starts to capture the video stream and the audio stream temporarily stored in the video stream area 2 and the audio stream area 3 of the memory 5, and the video stream and the audio stream are set for each packet unit length. And a divided stream is generated. On each synthesizer 12-15, a transport stream packet is generated by adding the program clock reference output from the output timing prediction calculation unit 11 to each divided stream, as shown in FIG. Further, the time stamp output from the output timing prediction calculation unit 11 is added to each transport stream packet, and is temporarily stored in the packetized stream area 4 on the memory 5.

  In parallel with this processing, each transport stream packet temporarily stored in the packetized stream area 4 on the memory 5 is read out by the packet output unit 10, temporarily stored in the buffer 17, and compared with the time stamp. The unit 18 compares the value of the time stamp added to each transport stream packet with the output timing value corresponding to the current time output from the output timing counter 16. When the time stamp value matches the output timing value, the time stamp is deleted from the transport stream packet in the buffer 17 and output to the outside.

  Thus, in this reference example, before the actual output timing for the video stream and the audio stream temporarily stored in the video stream area 2 and the audio stream area 3 formed on the memory 5, respectively, the CPU Thus, the program clock reference when the transport stream is output to the outside or the system clock reference when the program stream is output to the outside is predicted and calculated. Then, using this prediction calculation result and the time stamp necessary for output at the specified time, as shown in the schematic diagram of FIG. 5, the packet generation unit 9 packetizes the video stream and the audio stream. , Temporarily stored in the packetized stream area 4 on the memory 5. Thereafter, when the designated time comes, the packet output unit 10 detects this and adds it to each transport stream packet or each program stream packet temporarily stored in the packetized stream area 4. Deleted the time stamp that is present and output it to the outside. Therefore, when transport stream packets or program stream packets are output to the outside, it is possible to shift the time from encoding to output, thereby reducing the system load during output.

《Other reference examples》
In the reference example described above, each transport stream packet with a time stamp generated by the packet generation unit 9 or each program stream packet is temporarily stored in the packetized stream area 4 of the memory 5 at a specified time. The packet output unit 10 detects this and deletes each transport stream packet temporarily stored in the packetized stream area 4 or a time stamp added to each program stream packet to externally Output to.

  However, when a plurality of video streams and audio streams are continuously packetized, as shown in the schematic diagram of FIG. 6, each transport stream packet with a time stamp generated by the packet generator 9, a program stream, The packet is temporarily stored in the packetized stream area 4 of the memory 5 and then moved to the external storage device 19 such as a hard disk.

  Then, after transporting each transport stream packet with a time stamp or each program stream packet stored in the external storage device 19 to the memory 5 again, the packet output unit 10 externally transmits the transport stream packet at a designated output timing. May be output.

  With this configuration, even when the capacity of the memory 5 is small, a plurality of video streams and audio streams can be continuously packetized and output to the outside.

  Further, in the above-described reference example, each transport stream packet with a time stamp or each program stream packet temporarily stored in the packetized stream area 4 of the memory 5 is transmitted using one packet output unit 10. The data is read and output to the outside at a specified output timing. However, as shown in FIG. Each transport stream packet with a time stamp or program stream packet temporarily stored in the stream area 4 may be read and output to the outside at a specified output timing.

  With this configuration, a plurality of transport stream packets or program stream packets can be read simultaneously and in parallel and output to the outside. Also, when outputting a single transport stream or a single program stream, multiple output ports and multiple output timings can be used. Even when the number of packets is large, from the start of output to the end of output. Can be shortened and the degree of freedom of output can be increased.

<Embodiment>
FIG. 8 is a block diagram showing an embodiment of the present invention.

  In this embodiment, the memory 5 includes a plurality of other stream areas 21, a first packet area 22, a second packet area 23, and a third packet area 24 in addition to the video stream area 2 and the audio stream area 3. It is prepared for.

In this embodiment, in addition to the video encoding unit 7 and the audio encoding unit 8, other information different from video information and audio information is captured and encoded, and then stored in the other stream area 21 of the memory 5. The video stream temporarily stored in the video stream area 2 and the audio stream area 3 of the memory 5 and the other stream temporarily stored in the other stream area 21 via the memory bus 6 and the unit 20; A null packet corresponding to the difference from the output rate is inserted, and a time stamped transformer is used by using a program clock reference (PCR) (or system clock reference) output from the CPU, a time stamp, and the like. Port stream packet or time Generates a program stream packet with flop, which first packet region 22 of the memory 5, a second packet region, first packet generating unit _9-1 is temporarily stored in the third packet _region, the second packet generating unit 9 _-2 and a third packet generator _9-3 .

Further, the time stamp added to each program stream packet (or each transport stream packet) temporarily stored in the first packet area 22, the second packet area 23, and the third packet area 24 of the memory 5 is checked. At the time of output, the time stamp added to each program stream packet (or each transport stream packet) is deleted, and the null packet is output as it is, discarded, or others The first packet output unit 10 _-1 and the second packet output unit 10 _-2 are provided to select and replace each packet.

As shown in FIG. 9, each packet generation unit 9 ₋₁ , 9 ₋₂ , 9 ₋₃ includes four synthesizers 12, 13, 14, and 15, and outputs video, audio, and other streams from the memory. While reading, a null packet is appropriately inserted, and a transport stream packet with a time stamp to which a PCR and a time stamp are added is generated.

Each of the packet output units 10 _-1 and 10 _-2 compares the output timing counter 16 in which different output timings are set with the time stamp inserted in the packet and the output timing to determine whether the output timing has arrived. And a time stamp comparing unit 18 for judging, and configured to be able to output a transport stream packet to the outside at different timings.

  In the above configuration, when an instruction for encoding video information, an instruction for encoding audio information, and an instruction for encoding other information are issued from the CPU, the video encoder 7, the audio encoder 8, and the other encoder 20 connect the memory bus 6. Video information, audio information, and other information are taken in through and encoded. The video stream, audio stream, and other stream obtained in this way are temporarily stored in the video stream area 2, audio stream area 3, and other stream area 21 of the memory 5, respectively.

  Thereafter, as shown in FIG. 9, the video temporarily stored in the video stream area 2, the audio stream area 3, and the other stream area 21 of the memory 5 by the output timing prediction calculation unit 11 formed in the CPU, respectively. It is checked whether a program stream instruction or a transport stream instruction is instructed for a stream, an audio stream, and another stream. At this time, if a transport stream instruction is given, the output timing prediction calculation unit 11 predicts and calculates a program clock reference (PCR) necessary for outputting a transport stream packet to the outside.

  Next, it is necessary for the output timing prediction calculation unit 11 to packetize the video stream and the audio stream temporarily stored in the video stream area 2 and the audio stream area 3 on the memory 5 and to output the packet at a specified time. A time stamp that is a continuous value for each transport stream packet and a prediction result of the program clock reference (PCR) obtained by the above-described calculation operation are output on the memory bus 6.

Accordingly, the video stream, the audio stream, and the video stream temporarily stored in the video stream area 2, the audio stream area 3, and the other stream area 21 of the memory 5 by the packet generators 9 ₋₁ , 9 ₋₂ , and 9 ₋₃ . The capture of the other stream is started, and the video stream, the audio stream, and the other stream are divided for each packet unit length, and a divided stream is generated. At this time, the difference between the output rate and the valid packet is obtained. On each synthesizer 12 to 15, a program clock reference output from the output timing prediction calculation unit 11 is added to each divided stream, and a null packet is inserted into the difference to generate a transport stream packet. Is generated, and the time stamp output from the output timing prediction calculation unit 11 is added to each transport stream packet, and is temporarily stored in each packet area 22, 23, 24 on the memory 5. .

In parallel with this processing, the transport stream packets temporarily stored in the packet area 4 on the memory 5 are read out by the packet output units 10 ₋₁ and 10 ₋₂ and temporarily stored in the buffer 17. The time stamp comparator 18 compares the value of the time stamp added to each transport stream packet with the output timing value corresponding to the current time output from the output timing counter 16. When the time stamp value matches the output timing value, the time stamp is deleted from the transport stream packet in the buffer 17 and output to the outside.

In particular, in this embodiment, it is possible to select whether to output a null packet as it is, discard the null packet, or replace it with a packet temporarily stored in the buffer 25 and output it. For example, in packet 1 in FIG. 11A, two null packets are output as they are, but the other two null packets are output by being replaced with packet P1 (0) and packet P1 (1). In the same way as packet 2 in (b), the two null packets are output as they are, but the other two null packets are replaced with the packet P2 (0) and the packet P2 (1) and output. Different stream packets can be multiplexed for each of the units 10 ₋₁ and 10 ₋₂ .

Further, as shown in FIG. 11, the output timing of the first packet output unit _10-1 that outputs the packet _{1 and} the output timing of the second packet output unit _10-2 that outputs the packet 2 are different from each other. Packets can be output at different timings with a time difference.

  As described above, according to this embodiment, the same stream can be output at different timings. Also, different streams can be multiplexed and output for the same stream.

<Modification>
As shown in FIG. 12, the second packet generation unit 9 ₋₁ and the third packet generation unit _9-2 do not have to be connected to the memory bus 6 and are directly connected to the packet output units 10 ₋₁ and 10 _{−. Even} if it is configured to be connected to ₂ , the same effect can be obtained.

It is a block diagram which shows the reference example of an MPEG stream processing method and its apparatus. It is a schematic diagram which shows the operation example of the packet generation part shown in FIG. It is a schematic diagram which shows an example of the transport stream packet with a time stamp produced | generated by the packet production | generation part shown in FIG. It is a schematic diagram which shows the operation example of the packet output part shown in FIG. FIG. 2 is a schematic diagram illustrating an outline of an operation of the MPEG stream processing apparatus illustrated in FIG. 1. It is a schematic diagram which shows the other reference example of an MPEG stream processing method and its apparatus. It is a schematic diagram which shows the further another reference example of an MPEG stream processing method and its apparatus. 1 is a block diagram illustrating an embodiment of an MPEG stream processing method and apparatus according to the present invention. FIG. It is a schematic diagram which shows the operation example of the packet generation part shown in FIG. It is a schematic diagram which shows the operation example of the packet output part shown in FIG. It is a schematic diagram which shows the operation example of a packet output part. FIG. 6 is a schematic diagram showing a modification of the embodiment of the MPEG stream processing method and apparatus according to the present invention.

Explanation of symbols

1: MPEG stream processing apparatus 2: a video stream area 3: audio stream area 4: packetized stream area 5: Memory 6: memory bus 7: image encoding section 8: audio encoding unit 9, 9 _{-1, 9} -2, 9 _-3: packet generating unit 10, 10 _{-1, 10 -2:} packet output unit 11: output timing prediction calculating unit 12-15: combiner 16: output timing counter 17, 25: buffer 18: timestamp comparison 19: External storage device 20: Other encoding unit 21: Other stream area 22: First packet area 22: Second packet area 23: Third packet area

Claims (2)

In an MPEG stream processing method for outputting video information and audio information to the outside in MPEG packet format,
When the video information and the audio information are converted into MPEG packets, a time stamp corresponding to the output timing is added, a null packet is inserted into the difference between the output rate and the effective packet, and stored in the storage area. Based on the time stamp added to the packet, check whether it is the output timing,
When there is an MPEG packet at the output timing, the MPEG packet is read from the storage area, the time stamp added to the MPEG packet is deleted, and the null packet is output to the outside or discarded. Or after deciding whether to output by replacing with other packets, output the MPEG packet according to the judgment result to the outside,
An MPEG stream processing method. In an MPEG stream processing apparatus that outputs video information and audio information to the outside in MPEG packet format,
A packet generation unit that captures the video information and the audio information, adds a time stamp corresponding to the output timing, and generates an MPEG packet in which a null packet is inserted in the difference between the output rate and the valid packet;
A storage unit for storing the MPEG packet with a time stamp generated by the packet generation unit and a null packet inserted;
Based on the content of the time stamp added to each MPEG packet stored in the storage unit, the MPEG packet at the output timing is read, the time stamp added to the MPEG packet is deleted, and the A packet output unit that determines whether to output a null packet to the outside, discard it, or replace it with another packet and output the MPEG packet according to the determination result,
An MPEG stream processing apparatus comprising:

Images