JP2013192031A - Packet transmitter, packet receiver, and packet transmission system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a packet transmitter which suppresses start-up delay.SOLUTION: A packet transmitter 1 includes: an AV encoder 11 which encodes an input signal and generates an access unit to which a CTS and a DTS are added; multiplying means 12 of multiplying access units for video and audio; block formation means 13 of generating an AL-FEC block as a set of source packets; AL-FEC processing means 14 of generating redundant packets by performing AL-FEC processing on the AL-FEC blocks; timestamp adding means 15 of extracting a CTS or DTS of the earliest time from the access unit stored in a payload of source packets included in the AL-FEC block and adding it as a timestamp to the redundant packet of the AL-FEC block; and transmission means 16 of transmitting the source packets and the redundant packet.

Description

  The present invention relates to a packet transmission device, a packet reception device, and a packet transmission system that use AL-FEC technology for restoring packet loss that occurs during transmission.

  In the IP packet transmission path, packet loss may occur due to router buffer overflow or the like. Therefore, conventionally, in order to restore a source packet lost on the receiving side, an AL-FEC (Application Layer-Forward Error Correction) technique is known in which a redundant packet is generated in advance and transmitted together with the original source packet. (For example, Non-Patent Documents 1 and 2). By using this AL-FEC technology, it is possible to improve the packet loss tolerance.

IETF RFC 6363 “Forward Error Correction (FEC) Framework” IETF RFC 5109 “RTP Payload Format for Generic Forward Error Correction”

  By using the conventional AL-FEC technology, the reception side increases the startup delay in streaming playback, that is, the time from when a source packet is received to start playback to when video or audio is first output. There is a problem.

Here, it is considered that the startup delay is caused by the blocking delay, the AL-FEC restoration processing delay, and the buffering delay.
The blocking delay is a delay when a certain number of source packets are collected to form an AL-FEC block.
The delay of the AL-FEC restoration process is a delay when the processor generates a redundant packet by operation and restores the lost source packet.
The buffering delay is caused by a delay when connecting the AL-FEC restoration processing and media processing (decoding) such as video and audio, that is, waiting time until the media processing is performed after the source packet is extracted. It is a delay.

  However, since the blocking delay is determined by the configuration of the AL-FEC block, that is, the type of code to be used and the block length (number of packets in the block), it is difficult to suppress the blocking delay. Further, the delay of the AL-FEC restoration process is a finite time, but since it varies depending on the processing speed of the processor, it is generally treated as zero and is difficult to suppress.

  From the above, we focus on the buffering delay that can be suppressed among the startup delays. On the receiving side, the AL-FEC restoration processing is performed at an appropriate timing regardless of the timing of the media processing of the video and audio included in the source packet constituting the AL-FEC block. For this reason, on the receiving side, there is a problem that the time from the AL-FEC restoration processing to the media processing becomes longer, and the buffering delay increases.

  Accordingly, an object of the present invention is to provide a packet transmission device, a packet reception device, and a packet transmission system that solve the above-described problems and suppress startup delay.

  In view of the above-described problems, the packet transmission device according to the first invention of the present application includes an input including at least one of a video signal and an audio signal according to an encoding method using a CTS (Composition Time Stamp) indicating a presentation or playback time. A packet transmission apparatus that encodes a signal and performs transmission by performing AL-FEC block processing, and includes an encoder, source packet generation means, blocking means, AL-FEC processing means, time stamp addition means, And a transmission means.

  According to such a configuration, the packet transmission device generates an access unit (AU: Access Unit) to which the CTS is added by encoding the input signal by the encoder using the encoding method. Further, in the packet transmission device, the access unit generates a source packet stored in the payload by the source packet generation means. Then, the packet transmitting apparatus generates an AL-FEC block that is a set of source packets by the blocking unit. Further, the packet transmission device generates a redundant packet for restoring the source packet by performing AL-FEC processing on the generated AL-FEC block by the AL-FEC processing means.

  Further, the packet transmission device extracts the CTS that is the earliest time from the access unit stored in the payload of the source packet included in the same AL-FEC block by the time stamp adding means, and based on the extracted CTS A time stamp indicating the start time of the AL-FEC restoration process is added to the redundant packet generated from the same AL-FEC block. Then, the packet transmission device transmits the source packet and the redundant packet with the start time added by the transmission means. As described above, the packet transmission apparatus instructs the packet reception apparatus when to perform the AL-FEC restoration process by the time stamp of the redundant packet.

  In the packet transmission device according to the second aspect of the present invention, the encoder generates an access unit to which the CTS and the DTS are added, using an encoding method in which the DTS indicating the decoding time is used together with the CTS. Here, when the decoding order and the presentation order of the access units are different as in the video signal, both CTS and DTS (Decoding Time Stamp) are added to the access unit. On the other hand, when the decoding order and presentation order of the access units are the same as in the case of an audio signal, only the CTS is added to the access unit.

  For this reason, in the packet transmission apparatus, the time stamp adding means extracts the CTS or DTS that is the earliest time from the access unit stored in the payload of the source packet included in the same AL-FEC block. Alternatively, a time stamp is added based on the DTS. Thus, even when both CTS and DTS are added, the packet transmission device can appropriately instruct the packet reception device when to perform the AL-FEC restoration processing.

  Further, in the packet transmitting apparatus according to the third invention of the present application, the time stamp adding means is configured to use the time indicated by the extracted CTS or DTS as a time stamp, assuming that there is no delay in the AL-FEC restoration process in the packet receiving apparatus. You may add to a packet as it is. Further, the time stamp adding means adds a time that is a preset value from the extracted CTS or DTS to the redundant packet as a time stamp in anticipation of the delay of the AL-FEC restoration processing in the packet receiving device. Also good. As described above, the packet transmission apparatus instructs the packet reception apparatus of the timing in consideration of the delay of the AL-FEC restoration processing in the packet reception apparatus.

  Moreover, in view of the above-described problem, the packet receiving device according to the fourth invention of the present application is provided for restoring the source packet and the source packet whose access unit is stored in the payload from the packet transmitting device according to the first invention of the present application. A packet receiving apparatus that receives a redundant packet, and includes a receiving unit, a blocking unit, a packet loss detecting unit, an AL-FEC restoration processing unit, and a decoder.

  According to this configuration, the packet receiving apparatus receives the source packet and the redundant packet to which the time stamp indicating the start time of the AL-FEC restoration process is added by the receiving unit. Further, the packet reception device generates an AL-FEC block that is a set of redundant packets having the same time stamp and source packets by the blocking unit. Then, the packet reception device detects a packet loss for each AL-FEC block by the packet loss detection means.

  In addition, the packet reception device performs AL-FEC restoration processing on the AL-FEC block at the start time indicated by the time stamp of the redundant packet when the packet loss is detected by the AL-FEC restoration processing unit. Restore the lost source packet. As described above, the packet reception device performs the AL-FEC restoration process at the timing instructed by the packet transmission device.

  Further, the packet receiving apparatus decodes the access unit included in the source packet received by the receiving unit and the source packet restored by the AL-FEC restoration processing unit, by a decoder.

  In view of the above-described problems, a packet transmission system according to a fifth invention of the present application includes the packet transmission device according to the first invention of the present application and the packet reception device according to the fourth invention of the present application.

  According to such a configuration, the packet transmission system instructs the packet receiving device when to perform the AL-FEC restoration process by the time stamp of the redundant packet, and the packet receiving device performs the AL-FEC restoration processing at that timing. I do.

According to the present invention, the following excellent effects can be obtained.
According to the first, fourth, and fifth inventions of the present application, the packet transmitting apparatus instructs the packet receiving apparatus when to perform the AL-FEC restoration process, and the packet receiving apparatus performs the AL-FEC restoration process at that timing. Thus, according to the first, fourth, and fifth inventions of the present application, the timing of the AL-FEC restoration processing and the decoding processing of the video signal and the audio signal can be linked to suppress the startup delay.

According to the second invention of this application, even when both CTS and DTS are added, the timing for performing the AL-FEC restoration process can be appropriately instructed to the packet reception apparatus, and the startup delay can be suppressed.
According to the third aspect of the present invention, it is possible to instruct the packet reception apparatus of the timing in consideration of the delay of the AL-FEC restoration processing in the packet reception apparatus, and to minimize the startup delay.

It is a block diagram which shows the structure of the packet transmission system which concerns on embodiment of this invention. It is a figure explaining the 1st example of the redundant packet production | generation by the AL-FEC process means of FIG. It is a figure explaining the 2nd example of the redundant packet production | generation by the AL-FEC process means of FIG. It is a figure explaining the addition of the time stamp by the time stamp addition means of FIG. It is a figure explaining the buffer arrangement | positioning in the packet receiver of FIG. It is a flowchart which shows operation | movement of the packet transmission apparatus of FIG. 3 is a flowchart showing an operation of the packet reception device of FIG. 1.

[Configuration of packet transmission system]
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
As shown in FIG. 1, a packet transmission system 100 according to an embodiment of the present invention performs streaming distribution using the AL-FEC technology, and includes a packet transmission device 1 and a packet reception device 2.

In the packet transmission system 100, a packet transmission device 1 and a packet reception device 2 are connected via a network N. The network N is a transmission path for IP packets such as the Internet and an intranet.
In the present embodiment, the packet transmission system 100 includes one packet transmission device 1 and one packet reception device 2.

[Configuration of packet transmitter]
Hereinafter, the configuration of the packet transmission device 1 will be described.
The packet transmission device 1 encodes an input signal and performs AL-FEC processing to transmit it to the packet reception device 2. For this reason, the packet transmitting apparatus 1 includes an AV encoder (encoder) 11, a multiplexing unit (source packet generating unit) 12, a blocking unit 13, an AL-FEC processing unit 14, a time stamp adding unit 15, Transmission means 16.

  The AV encoder 11 encodes an input signal by an encoding method using at least CTS, thereby generating an access unit to which at least CTS is added. Here, the AV encoder 11 receives both a video signal and an audio signal as input signals from the outside.

  The AV encoder 11 encodes an input video signal using a video encoding method such as MPEG (Moving Picture Experts Group) -2 Video, and generates a video access unit. Here, when the decoding order and the presentation order of the access units are different as in the video signal, both CTS and DTS are added for each access unit.

  The AV encoder 11 encodes the input audio signal using an audio encoding method such as MPEG-2 AAC (Advanced Audio Coding), and generates an audio access unit. Here, when the decoding order and the presentation order of the access units are the same as in the case of an audio signal, only CTS is added for each access unit. The AV encoder 11 outputs the generated video access unit and audio access unit to the multiplexing means 12.

CTS is information indicating the time of presentation or reproduction, and may be used synonymously with PTS (Presentation Time Stamp).
The DTS is information indicating the time for decoding.
An access unit is a set of codes (input signals) having the same DTS.

The multiplexing means 12 receives the video and audio access units from the AV encoder 11 and multiplexes the input video and audio access units.
Specifically, the multiplexing means 12 packetizes the input video and audio access units in a media unit (MU) format for each access unit. Also, the multiplexing means 12 multiplexes the video and audio access units that have been converted into MU packets into one media unit sequence using a multiplexing scheme such as ATS (Advanced Transport Scheme).

  The multiplexing unit 12 divides the multiplexed access unit into a size that does not exceed a preset maximum packetization size. Then, the multiplexing unit 12 converts the divided access unit into an ATS packet so that it is stored in the payload, and generates a source packet (ATS packet). Further, the multiplexing means 12 adds a sequence number to the generated source packet and outputs it to the blocking means 13.

  This source packet is a packet in which a part of the access unit is stored in the payload. In this embodiment, since video and audio are multiplexed, a part of an access unit of either video or audio is stored in the payload of each source packet.

  The media unit and ATS multiplexing method are described in, for example, the reference document “Media Transport Method in Hybrid Broadcasting, Research Report of Information Processing Society of Japan, Vol.2011-AVN-72 No.1 2011/3/11”. Has been.

The blocking unit 13 receives the source packet from the multiplexing unit 12 and generates an AL-FEC block that is a set of the input source packets.
This AL-FEC block is a block that is generated in order to perform an AL-FEC process and an AL-FEC restoration process to be described later.

  Specifically, in order to generate AL-FEC redundant packets, the blocking unit 13 collects source packets in order of sequence numbers according to a preset block generation rule, and generates an AL-FEC block. For example, the blocking unit 13 arranges source packets in a one-dimensional direction to generate an AL-FEC block (FIG. 2). For example, the blocking unit 13 arranges the source packets in the two-dimensional direction to generate an AL-FEC block (FIG. 3). Then, the blocking unit 13 outputs the generated AL-FEC block to the AL-FEC processing unit 14.

  Note that information such as the number of source packets stored in the AL-FEC block and the direction in which the source packets are arranged in the AL-FEC block (for example, one-dimensional direction or two-dimensional direction) is set in the block generation rule.

The AL-FEC processing unit 14 generates a redundant packet for restoring the source packet by performing an AL-FEC process on the AL-FEC block input from the blocking unit 13.
This AL-FEC process is a process for generating a redundant packet from an AL-FEC block, and is, for example, an XOR operation.

<Redundant Packet Generation: First Example>
Two specific examples of the generation of the redundant packet 92 by the AL-FEC processing unit 14 will be described with reference to FIGS. 2 and 3 (see FIG. 1 as appropriate).
In FIGS. 2 and 3, illustrations other than the payload Py of the source packet 91 and the redundant packet 92 are omitted for the sake of simplicity.
In this first example, as shown in FIG. 2, an AL-FEC block is composed of M source packets 91 arranged in the column direction (one-dimensional direction) (however, an arbitrary value satisfying M ≧ 2) integer).

  Here, the AL-FEC processing unit 14 delimits the bit string included in the payload Py of each source packet 91 in units of bytes and performs an XOR operation. For example, if the payload Py is a fixed length of 1300 bytes, the AL-FEC processing unit 14 repeats the XOR operation for the payload Py 1300 times.

  First, the AL-FEC processing means 14 performs an XOR operation on the payload Py of the first source packet 91 and the payload Py of the second source packet 91. Next, the AL-FEC processing means 14 XORs the operation results (bit strings having the same length as the payload Py) of the first and second source packets 91 and the payload Py of the third source packet 91. Perform the operation. The same procedure is repeated, and the AL-FEC processing means 14 performs an XOR operation between the calculation results of the first to M−1 source packets 91 and the payload Py of the Mth source packet 91. Do. Further, the AL-FEC processing unit 14 stores the calculation results from the first to Mth source packets 91 in the payload Py, and generates a redundant packet 92. Further, the AL-FEC processing unit 14 adds a sequence number to the generated redundant packet 92 and adds it to the AL-FEC block 90.

  When the payload Py has a variable length, the AL-FEC processing unit 14 obtains the maximum size (number of bytes) of the payload Py among all the source packets 91 of the AL-FEC block 90. Then, the AL-FEC processing unit 14 performs the XOR operation after performing zero padding so that the payload Py less than the maximum size becomes equal to the maximum size. In this case, the payload Py of the redundant packet 92 is equal to the determined maximum size.

<Redundant Packet Generation: Second Example>
In this second example, as shown in FIG. 3, an AL-FEC block 90 is configured by arranging M source packets 91 in the column direction and N source packets 91 in the row direction in a two-dimensional direction (however, N Any integer satisfying ≧ 2).

  As in the first example, the AL-FEC processing unit 14 performs an XOR operation on the AL-FEC block 90 in the column direction. Then, the AL-FEC processing unit 14 stores the operation result in the column direction in the payload Py, and generates N redundant packets 92.

  Also, the AL-FEC processing means 14 performs an XOR operation on the AL-FEC block 90 in the row direction, as in the first example. Then, the AL-FEC processing unit 14 stores the operation result in the row direction in the payload Py, and generates M redundant packets 92. Further, the AL-FEC processing unit 14 adds the generated M + N redundant packets 92 to the AL-FEC block 90.

Thereafter, the AL-FEC processing unit 14 outputs the AL-FEC block 90 generated by the technique of the first example or the second example to the time stamp adding unit 15.
The AL-FEC processing means 14 is set in advance as to which of the first example and the second example is used.

Returning to FIG. 1, the description of the configuration of the packet transmission device 1 will be continued.
The time stamp adding means 15 receives the AL-FEC block 90 from the AL-FEC processing means 14 and receives the earliest time from the access unit stored in the payload Py of the source packet 91 included in the same AL-FEC block 90. CTS or DTS to be extracted. Then, the time stamp adding means 15 adds a time stamp indicating the start time of the AL-FEC restoration process to the redundant packet 92 generated from the same AL-FEC block 90 based on the extracted CTS or DTS.

<Add time stamp>
With reference to FIG. 4, the addition of the time stamp Ts by the time stamp adding means 15 will be specifically described (see FIG. 1 as appropriate).
In FIG. 4, two AL-FEC blocks 90 are shown. Each AL-FEC block 90 is composed of 50 source packets 91 and 10 redundant packets 92.

Note that a sequence number Sn of “1” to “100” is added to the source packet 91, and a sequence number Sn of “201” to “220” is added to the redundant packet 92.
Further, the sequence number Sn of the source packet 91 and the sequence number Sn of the redundant packet 92 are not continuous because the number system is different.

  For example, in the first AL-FEC block 90, among the CTS and DTS added to the access units of 50 source packets 91, the sequence number Sn is stored in the payload Py of the source packet 91 with “3”. It is assumed that the CTS added to the access unit is the earliest time.

In this case, the time stamp adding means 15 extracts the CTS of the access unit stored in the payload Py from the source packet 91 whose sequence number Sn is “3”. The time stamp adding means 15 adds the time indicated by the extracted CTS as it is as the time stamp Ts of the redundant packet 92 having the sequence numbers Sn of “201” to “210”. That is, the same time stamp Ts is added to the redundant packet 92 included in the same AL-FEC block 90.
Since the second AL-FEC block 90 is also added with the time stamp Ts as in the first, the description is omitted.

  At this time, the time stamp adding means 15 may describe the absolute time of the clock as the time stamp Ts to be added to the redundant packet 92, and the difference between the start times of the AL-FEC restoration processes in the continuous AL-FEC blocks 90. May be described.

  Then, the time stamp adding means 15 outputs the source packet 91 included in the AL-FEC block 90 and the redundant packet 92 added with the time stamp Ts to the transmitting means 16.

Returning to FIG. 1, the description of the configuration of the packet transmission device 1 will be continued.
The transmission unit 16 receives the source packet 91 and the redundant packet 92 from the time stamp adding unit 15 and transmits the source packet 91 and the redundant packet 92 to the packet receiving device 2 via the network N.

  Specifically, the transmission means 16 adds an IP (Internet Protocol) header and a UDP (User Datagram Protocol) header to the source packet 91 and the redundant packet 92 to generate an IP packet. Then, the transmission unit 16 performs transmission path encoding processing and modulation processing according to the network N, and transmits the generated IP packet.

  Here, the transmission means 16 may transmit an IP packet as an IP data flow. This IP data flow is a set of a series of IP packets in which five pieces of information of the IP packet source IP address, destination IP address, next header type, source port number, and destination port number are the same combination. is there.

[Configuration of packet receiver]
The configuration of the packet reception device 2 will be described with reference to FIGS.
The packet receiving device 2 receives the source packet 91 and the redundant packet 92 from the packet transmitting device 1, restores the lost source packet 91 by the AL-FEC restoration processing, and decodes the access unit included in the source packet 91. is there.

  For this reason, the packet receiver 2 includes a receiving unit 21, a blocking unit 22, a packet loss detecting unit 23, an AL-FEC restoration processing unit 24, a demultiplexing unit 25, and an AV decoder (decoder) 26. Provide (FIG. 1).

  In the packet receiver 2, the receiving means 21 includes a buffer (IP pkt buffer) 21a, the blocking means 22 includes a buffer (ATS pkt buffer 1) 22a and a buffer (Reorder buffer) 22b, and the demultiplexing means 25 includes A buffer (ATS pkt buffer 2) 25a and buffers (Media unit buffers) 25b and 25c are provided (FIG. 5).

The receiving means 21 receives the source packet 91 and the redundant packet 92 to which the time stamp Ts is added from the packet transmitting apparatus 1 via the network N.
Specifically, the receiving means 21 performs demodulation processing and transmission path decoding processing according to the network N, receives IP packets, and stores them in a buffer (IP pkt buffer) 21a.

The blocking unit 22 generates an AL-FEC block 90 that is a set of the source packet 91 and the redundant packet 92 received by the receiving unit 21.
Specifically, the blocking unit 22 removes the IP header and the UDP header from the IP packet stored in the buffer 21a, extracts the source packet 91 and the redundant packet 92, and stores them in the buffer 22a.

  The source packet 91 and the redundant packet 92 may change the arrival order of IP packets in the network N, and may include jitter in the arrival time. For this reason, the blocking unit 22 rearranges the source packet 91 and the redundant packet 92 accumulated in the buffer 22a in the transmission order (that is, the order of the sequence number Sn) and accumulates them in the buffer 22b.

  Further, the blocking unit 22 uses the same AL-FEC block 90 as that at the time of transmission from the source packet 91 and the redundant packet 92 stored in the buffer 22b according to the block generation rule set with the same contents as the blocking unit 13. Generate. Then, the blocking unit 22 outputs the generated AL-FEC block 90 to the packet loss detection unit 23.

The packet loss detecting unit 23 detects a packet loss for each AL-FEC block 90 generated by the blocking unit 22.
Specifically, the packet loss detection unit 23 detects a packet loss by the missing number of the sequence number Sn of the source packet 91 included in each AL-FEC block 90. That is, the packet loss detecting means 23 detects that the source packet 91 having the missing sequence number Sn has been lost.

  Then, the packet loss detection unit 23 generates packet loss detection information that stores the sequence number Sn of the source packet 91 in which the packet loss is detected. Further, the packet loss detection unit 23 outputs the input AL-FEC block 90 and the generated packet loss detection information to the AL-FEC restoration processing unit 24.

The AL-FEC restoration processing unit 24 refers to the packet loss detection information input from the packet loss detection unit 23, and restores the AL-FEC block 90 to the AL-FEC block 90 at the start time indicated by the time stamp Ts of the redundant packet 92. By performing the processing, the source packet 91 in which the packet loss has occurred is restored (D _{FEC in} FIG. 5).
This AL-FEC process is a process for restoring a lost source packet from the AL-FEC block 90, and is, for example, an XOR operation.

  Specifically, the AL-FEC restoration processing unit 24 performs an AL-FEC restoration process for each AL-FEC block 90. First, the AL-FEC restoration processing means 24 extracts any time stamp Ts from the redundant packet 92 of the AL-FEC block 90. Next, the AL-FEC restoration processing unit 24 waits until the start time of the time stamp Ts matches the clock. Then, when the start time coincides with the clock, the AL-FEC restoration processing unit 24 performs an XOR operation on the non-lost source packet 91 and the redundant packet 92 to restore the lost source packet 91.

Then, the AL-FEC restoration processing unit 24 accumulates the source packet 91 included in the AL-FEC block 90 and the restored source packet 91 in the buffer 25a.
Note that the AL-FEC restoration processing unit 24 accumulates only the source packet 91 included in the AL-FEC block 90 in the buffer 25a when no packet loss is detected in the AL-FEC block 90.

  The demultiplexing unit 25 separates the video access unit and the audio access unit from the payload Py of the source packet 91 received by the receiving unit 21 and the source packet 91 restored by the AL-FEC restoration processing unit 24. Is.

  Specifically, the demultiplexing means 25 extracts the payload Py of the source packet 91 stored in the buffer 25a, and combines the extracted payload Py to take out an access unit. The demultiplexing means 25 separates the extracted access unit into a video access unit and an audio access unit. Here, the access unit is packetized in the media unit format in the packet transmission device 1. Accordingly, the demultiplexing means 25 stores the video access unit separated from the source packet 91 in the buffer 25b in the media unit format. The demultiplexing unit 25 stores the audio access unit separated from the source packet 91 in the buffer 25c in the media unit format.

  The AV decoder 26 decodes the video access unit and the audio access unit separated by the demultiplexing means 25 by the same decoding method as the AV encoder 11.

Specifically, the AV decoder 26 reads and decodes the video access unit from the buffer 25b using MPEG-2 Video (D _{V in} FIG. 5). The AV decoder 26 reads and decodes the audio access unit from the buffer 25c using MPEG-2 AAC (D _{A in} FIG. 5). Both CTS and DTS or only CTS is added to this access unit. Therefore, the AV decoder 26 performs video and audio decoding and presentation processing at the times indicated by them, and reproduces the video and audio.
The AV decoder 26 treats DTS and CTS as the same time when only the CTS is added to the access unit.

[Operation of packet transmitting device]
The operation of the packet transmission device 1 will be described with reference to FIG. 6 (see FIG. 1 as appropriate).
The packet transmitter 1 encodes the input signal by the AV encoder 11 to generate a video access unit and an audio access unit (step S11).

The packet transmission device 1 multiplexes the video and audio access units by the multiplexing means 12, and generates a source packet in which the multiplexed access units are stored in the payload Py (step S12).
The packet transmission device 1 generates an AL-FEC block that is a set of source packets by the blocking unit 13 (step S13).
The packet transmission device 1 generates a redundant packet by performing AL-FEC processing on the AL-FEC block by the AL-FEC processing unit 14 (step S14).

  The packet transmission device 1 extracts the CTS or DTS that is the earliest time from the access unit stored in the payload Py of the source packet 91 included in the same AL-FEC block 90 by the time stamp adding unit 15. Then, the packet transmitter 1 adds the time indicated by the extracted CTS or DTS as the time stamp Ts to the redundant packet 92 of the same AL-FEC block 90 by the time stamp adding means 15 (step S15).

  The packet transmission device 1 transmits the source packet 91 and the redundant packet 92 to which the time stamp Ts is added to the packet reception device 2 via the network N by the transmission unit 16 (step S16). .

[Operation of packet receiving device]
With reference to FIG. 7, the operation of the packet receiver 2 will be described (see FIG. 1 as appropriate).
The packet receiver 2 receives the source packet 91 and the redundant packet 92 to which the time stamp Ts is added as an IP packet from the packet transmitter 1 via the network N by the receiving unit 21 (step S21). .

  The packet receiving device 2 generates the AL-FEC block 90 that is a set of the source packet 91 and the redundant packet 92 received by the receiving unit 21 by the blocking unit 22 (step S22).

The packet receiving device 2 detects the packet loss for each AL-FEC block 90 generated by the blocking unit 22 by the packet loss detecting unit 23 (step S23).
Here, when a packet loss is detected (Yes in step S23), the packet reception device 2 proceeds to the process of step S24.
On the other hand, when no packet loss is detected (No in step S23), the packet reception device 2 proceeds to the process of step S25.

  The packet receiving apparatus 2 performs the AL-FEC restoration processing on the AL-FEC block 90 at the time indicated by the time stamp Ts of the redundant packet 92 by the AL-FEC restoration processing unit 24, thereby causing the source packet in which the packet loss has occurred. 91 is restored (step S24).

The packet receiving apparatus 2 separates the video access unit and the audio access unit from the payload Py of the source packet 91 received by the receiving unit 21 and the source packet 91 restored by the AL-FEC restoration processing unit 24. (Step S25).
The packet receiver 2 decodes the video access unit and the audio access unit separated by the demultiplexing means 25 by the AV decoder 26 (step S26).

  As described above, in the packet transmission system 100 according to the embodiment of the present invention, the packet transmission device 1 instructs the packet reception device 2 when to perform the AL-FEC restoration process, and the packet reception device 2 performs the AL- Perform FEC restoration processing. In the packet transmission system 100, the timing for performing the AL-FEC restoration process (time stamp Ts) is set to the earliest time (decoding time) of the CTS and DTS. Thereby, the packet transmission system 100 shortens the time from the AL-FEC processing to the decoding, so that the buffering delay can be minimized, and as a result, the startup delay of the packet receiving device 2 can be suppressed. .

  That is, it can be said that the packet transmission system 100 considers a media coding layer (video and audio encoding layer) and a transmission layer (IP transmission layer) that have been handled separately in the past.

(Modification)
As mentioned above, although embodiment of this invention was described, this invention is not limited to this, It can implement in the range which does not change the meaning. The modification of embodiment is shown below.

  The packet receiving device 2 may include buffer deletion means (not shown) that deletes data stored in the buffers 21a to 25b. In this case, the buffer deletion means can immediately reduce the data stored in the buffers 21a to 25b when the time stamp Ts of the redundant packet 92 included in the AL-FEC block 90 has passed the time of media processing. The buffer capacity of the device 2 can be saved.

  The time stamp adding means 15 may set the time Ts before the extracted CTS or DTS by a preset value (for example, 10 milliseconds). In this case, the packet transmission device 1 can instruct the packet reception device 2 of the timing in consideration of the delay of the AL-FEC restoration process in the packet reception device 2, and thus the startup delay can be minimized.

  The packet transmission system 100 may use a block correction code such as a Reed-Solomon code or an LDPC code in addition to the XOR operation as the AL-FEC process and the AL-FEC restoration process. In this case, the packet transmission system 100 determines in advance according to which block correction code is used for the AL-FEC processing and the AL-FEC restoration processing, the packet loss rate in the network N, and the allowable range of blocking delay. You can choose.

  Further, the packet transmission system 100 may use MPEG-4 AVC (Advanced Video Coding) as a video encoding method and MPEG-2 System as a multiplexing method.

  Further, the packet transmission device 1 may transmit the video signal and the audio signal to the packet reception device 2 without multiplexing. Specifically, the packet transmission device 1 includes source packet generation means (not shown) instead of the multiplexing means 12. The packet transmission device 1 generates ATS packets from each media unit sequence after the source packet generation unit generates separate media unit sequences for video and audio. Then, the packet transmission device 1 transmits the IP packet including the video and audio access units by the transmission unit 16 in separate IP data flows.

  Further, the packet transmission device 1 may add a time stamp to the source packet 91 included in the same AL-FEC block 90 by the time stamp adding means 15. In this case, the source packet 91 and the redundant packet 92 have the same data structure, which is convenient for system management.

Further, only one of the video signal and the audio signal may be input as an input signal to the packet transmission device 1.
Further, the packet transmission device 1 may add consecutive sequence numbers Sn between the source packet 91 and the redundant packet 92 included in the same AL-FEC block 90.
In addition, the packet transmission system 100 may include two or more packet transmission devices 1 or two or more packet reception devices 2.

1 Packet transmitter 11 AV encoder (encoder)
12 Multiplexing means (source packet generating means)
13 Blocking means 14 AL-FEC processing means 15 Time stamp adding means 16 Transmitting means 2 Packet receiving device 21 Receiving means 22 Blocking means 23 Packet loss detecting means 24 AL-FEC restoration processing means 25 Demultiplexing means 26 AV decoder (decoder) )
100 packet transmission system

Claims (5)

A packet transmission apparatus that encodes an input signal including at least one of a video signal and an audio signal by an encoding method using a CTS that indicates a time of presentation or reproduction, and performs AL-FEC block processing to transmit the encoded signal.
An encoder for generating an access unit to which the CTS is added by encoding the input signal by the encoding method;
Source packet generating means for generating a source packet stored in a payload by the access unit;
Blocking means for generating an AL-FEC block which is a set of the source packets;
AL-FEC processing means for generating a redundant packet for restoring the source packet by performing AL-FEC processing on the generated AL-FEC block;
The CTS that is the earliest time is extracted from the access unit stored in the payload of the source packet included in the same AL-FEC block, and is generated from the same AL-FEC block based on the extracted CTS. A time stamp adding means for adding a time stamp indicating the start time of the AL-FEC restoration process to the redundant packet;
Transmitting means for transmitting the source packet and the redundant packet to which the start time is added;
A packet transmission device comprising: The encoder generates, together with the CTS, an access unit to which the CTS and the DTS are added, using the encoding method in which a DTS indicating a decoding time is used.
The time stamp adding means extracts the CTS or DTS that is the earliest time from the access unit stored in the payload of the source packet included in the same AL-FEC block, and based on the extracted CTS or DTS The packet transmission apparatus according to claim 1, wherein the time stamp is added.   The time stamp adding means adds the time indicated by the extracted CTS or DTS, or the time preceding the extracted CTS or DTS by a preset value to the redundant packet as the time stamp. The packet transmission device according to claim 2. A packet receiving device, wherein the access unit receives a source packet stored in a payload and a redundant packet for restoring the source packet from the packet transmitting device according to claim 1,
Receiving means for receiving the source packet and the redundant packet to which a time stamp indicating the start time of the AL-FEC restoration process is added;
Blocking means for generating an AL-FEC block that is a set of redundant packets having the same time stamp and the source packets;
Packet loss detection means for detecting packet loss for each AL-FEC block;
When the packet loss is detected, the AL-FEC restoration process is performed on the AL-FEC block at the time indicated by the time stamp of the redundant packet to restore the source packet in which the packet loss has occurred. FEC restoration processing means;
A decoder for decoding the access unit stored in the payload of the source packet received by the receiving means and the source packet restored by the AL-FEC restoration processing means;
A packet receiving apparatus comprising: A packet transmission device according to claim 1;
A packet receiving device according to claim 4;
A packet transmission system comprising:

Images