JP2012114569A - Data communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data communication system which uses an error correction code as a countermeasure against packet loss as well as suppresses increase in packet loss and jitters newly caused consequently, without being accompanied by increase in the number of transmission packets which burdens a network.SOLUTION: In a data communication system, a video transmission side transmission device 10 divides a plurality of packets generated by dividing transmission object data into subblocks having a predetermined length, and generates error correction codes by performing error correction to the subblocks allocated to blocks that are calculation units of the error correction. A video reception side transmission device 11 allocates subblocks having the predetermined length generated by dividing received packets to blocks that are the calculation units of the error correction, detects missing packets on a communication path based on the received packets, and restores the missing packets by using the error correction code belonging to a specific block to which the subblock is allocated and the subblock allocated to the specific block, for each of the subblocks forming the missing packets.

Description

  The present invention is a data communication system for transmitting moving images and voices in real time via a data communication network such as the Internet or an intranet, and particularly for correcting transmission errors during transmission from the transmission side to the reception side. The present invention relates to a data communication system that transmits an error correction code.

  In a conventional data communication system, a combination of error correction code technology in the physical layer (layer 1), retransmission control technology in the data link layer (layer 2), etc., for transmission errors or information loss (packet loss) during transmission As a result, transmission quality is ensured. However, since these technologies are link quality assurance technologies, they do not deal with data loss due to transmission errors exceeding a certain rate, congestion in a system that communicates through multiple networks, etc. It is powerless. Therefore, in order to ensure the quality between communication end points, many retransmission control techniques and the like in the transport layer (layer 4) and application layer (layer 7) have been devised and implemented in combination.

  On the other hand, the quality assurance means in these data communication systems is often not suitable for maintaining the quality as an application because of an increase in delay in applications that require real-time performance such as video and audio. Therefore, for such an application, a quality ensuring method using an error correction code technique in a session layer (layer 5) is common.

  For example, a video transmission system in which a video encoding unit and a video decoding unit are combined with a conventional data communication system is mainly composed of a video transmission side transmission device 60, a video reception side transmission device 61, and a network as shown in FIG. It is configured. The video transmission side transmission device 60 includes a video encoding unit 601, a packet generation unit 602, a packet multiplexing unit 603, a packet transmission control unit 604, an error correction code generation unit 605, and an error correction code packet generation unit. 606 and the like. The video reception side transmission device 61 includes a packet reception control unit 611, a packet demultiplexing unit 612, a received packet storage unit 613, a packet alignment unit 614, a video code decoding unit 615, and an error correction code packet storage unit. 616, a packet loss detection unit 617, and an error correction loss packet regeneration unit 618.

  First, in the video transmission side transmission device 60, the video encoding unit 601 encodes an input video generated by an imaging element (camera) or the like based on a moving image encoding algorithm such as MPEG. The packet generation unit 602 arranges the code generated by the video encoding unit 601 into a transmission packet format, and outputs it to the packet transmission control unit 604 via the packet multiplexing unit 603. The packet transmission control unit 604 temporarily accumulates the packets and then sends the packets to the network based on the network communication protocol.

  In the video reception side transmission device 61, the packet reception control unit 611 receives a packet sent to the video reception side transmission device 61 via the network. The received packet is stored in the received packet storage unit 613 via the packet demultiplexing unit 612. The packet aligning unit 614 takes out the packet storing the video code from the received packet accumulating unit 613 and sends it to the video code decoding unit 615 in consideration of the variation in the delay time of packet transmission by the network. The video code decoding unit 615 decodes video based on the moving image encoding algorithm used in the video transmission side transmission device 60 and sends the decoded data to a display device or the like.

  Here, in the data communication network, there is a risk of packet loss during transmission. Therefore, the video transmission apparatus 60 shown in FIG. 1 outputs the code generated by the video encoding unit 601 to the error correction code generation unit 605 as well. Then, the error correction code generation unit 605 generates an error correction code based on an error correction algorithm typified by an exclusive OR (XOR) and a Reed-Solomon code. The error correction code packet generation unit 606 arranges the generated error correction code into a transmission packet format. The packet multiplexing unit 603 packet-multiplexes the generated error correction code packet and the data packet generated by the packet generation unit 602, and transmits the packet to the network via the packet transmission control unit 604.

  On the other hand, in the video receiving side transmission device 61, the packet received by the packet reception control unit 611 is identified by the packet demultiplexing unit 612, whereby the data packet is accumulated by the received packet accumulation unit 613, and the error correction code packet is error corrected. The code packet is stored in the code packet storage unit 616. Next, the packet loss detection unit 617 checks the order of the data packets stored in the received packet storage unit 613, and detects a packet that is lost during the transmission of the network in consideration of the dispersion of the transmission delay time in the network. . The packet loss detection unit 617 then notifies the error correction loss packet regeneration unit 618 of the loss. The error correction loss packet regeneration unit 618 that has received the missing notification is necessary to regenerate the missing data packet based on the error correction code algorithm used by the error correction code generation unit 605 of the video transmission side transmission device 60. Are extracted from the received packet storage unit 613, and the error correction code packet is extracted from the error correction code packet storage unit 616. Then, the error correction loss packet regenerator 618 regenerates the missing packet by an operation based on the error correction code algorithm, and sends it to the packet aligner 614. The packet alignment unit 614 receives the received data packet and the regenerated data packet, arranges the order of the data packets necessary for decoding the video code, and sends the data packet to the video code decoding unit 615. (For example, see Patent Document 1 and Non-Patent Document 1).

  The detailed operation of the video transmission side transmission device 60 will be described with reference to FIG. The video code generated by the video encoding unit 601 becomes user data 700, which is divided into fixed lengths for packetization and becomes payloads 701a to 705a of each packet. In the packet generation unit 602, transmission headers 701b to 705b are added to the respective payloads, and packets 701 to 705 are generated. The generated packets 701 to 705 are sent to the transmission line of the network via the packet multiplexing unit 603 and the packet transmission control unit 604. Further, the error correction code generation unit 605 generates a parity 711a based on an error correction code algorithm using payloads 701a to 705a (for five payloads in FIG. 2) obtained by dividing a certain amount of user data 700. . Next, the error correction code packet generation unit 606 adds a transmission header 711b similar to the payloads 701a to 705a based on user data to the parity 711a to generate a packet 711. The generated packet 711 is sent to the transmission line of the network via the packet multiplexing unit 603 and the packet transmission control unit 604.

  The packet 711 of the error correction code stores identification information in the transmission header 711b so that it can be easily identified by the packet demultiplexing unit 612 of the video receiving side transmission device 61. In general, the port number of the UDP (User Datagram Protocol) protocol is set to a value different from that of the data packet.

  Similarly, in the video transmission side transmission device 60, the generated video code is packetized and transmitted to the transmission line as 706 to 709.

  In the video receiving side transmission device 61, the packet demultiplexing unit 612 identifies and separates the user data packet and the error correction code packet. Then, the packet loss detection unit 617 detects packet loss using the packet order information stored in the transmission header, and the error correction loss packet regeneration unit 618 uses the error correction code to lose the packet loss. Regenerate the video code (packet). Further, the packet sorting unit 614 inserts the video code regenerated using the packet order information or the like at an appropriate position.

Japanese Patent No. 3934073

IETF RFC5109 RTP Payload Format for Generic Forward Error Correction

  However, in a video transmission system using a conventional data communication system, a data packet storing a video code and an error correction code packet storing an error correction code are transmitted by being individually transmitted over the network. The number of packets increases as compared with the case where error correction is not performed, resulting in problems described later.

  First, in a network using a contention method typified by a wireless local area network (LAN) and a conventional Ethernet (registered trademark), packets are generated due to collision between packets whose occurrence probability increases with an increase in the number of packets. Will be lost. That is, as the number of packets increases, the frequency of collisions increases rapidly and the number of lost packets increases. Or, as the packet is lost, retransmission control in the lower layer occurs, so that the delay increases and the effective transmission band decreases.

  Next, in a network relay device represented by an IP router, a LAN switch, and the like, multiplexing jitter and jitter due to traffic shaping for ensuring transmission quality increase.

  Here, generation of multiplexed jitter will be described with reference to FIG. 3, and generation of jitter due to traffic shaping will be described with reference to FIG.

  FIG. 3A shows a network configuration in which multiplexing jitter occurs. FIG. 3A is a model in which a plurality of input traffics 811, 812, and 813 are input to the network 801 and output as a plurality of outputs 821, 822, and 823. The network 801 includes a relay device 802 that accommodates inputs, a relay device 804 that accommodates outputs, and a relay line 803 that connects them. The relay device 802 sends the input traffic 811 to 813 to the relay line 803 using time division multiplexing. On the trunk line 803, the packets of the traffics 831 to 833 are transmitted in a mixed form (packet string). The relay device 804 refers to the packet stream transmission control information received from the relay line 803, performs demultiplexing processing on each traffic, and outputs it as output traffic 821 to 823.

  FIG. 3B shows a time period obtained by arranging the packet movements in FIG. 3A in time series. In FIG. 3B, the input traffic 811 is composed of packets 811a to 811d, the input traffic 812 is composed of packets 812a to 812e, and the input traffic 813 is composed of packets 813a to 813c.

  Since a plurality of packets cannot be transmitted at the same time on the transmission line of the relay line 803, the relay device 802 arranges and sends out each packet in a line by the time division multiplexing technique. FIG. 3B shows an example of multiplexing based on the same priority and arrival order, which is the most basic multiplexing method. As shown in FIG. 3B, the packet 811a and the packet 812a are transmitted in the order of 831a and 832a on the transmission line of the relay line 803 because the packet 811a arrives at the relay device 802 first with a slight time difference. Further, since the packet 811a arrives at the relay device 802 at the timing when the transmission line of the relay line 803 is free, the packet 811a is immediately transmitted as the packet 831a. On the other hand, the packet 812a is delayed by waiting until the packet 811a is completely transmitted as the packet 831a, and is transmitted as the packet 832a after completion of the transmission. Similarly to 811a, the packet 813a is transmitted as it is as the packet 833a because the transmission path is free.

  Similarly, packets 811b to 811d are sent as 831b to 831d, 812b to 812e are sent as 832b to 832e, and 813b and 813c are sent as 833b and 833c, respectively. In FIG. 3B, 832b, 832c, 831c, 833c, and 831d are delayed due to waiting.

  In other words, depending on the arrival timing of packets, any of the input traffic 811 to 813 shown in FIG. As shown in FIG. 3A, each traffic demultiplexed by the relay device 803 is output as output traffic 821 to 823, but the delay time of the packet delayed by the multiplexing processing by the relay device 802 is the relay device 803. Is not compensated for and is output with a delay. As a result, in FIG. 3B, output packets 821a, 821b, 823a, 823b, 822d, and 822e are output only with a fixed transmission delay and processing delay, whereas output packets 822a, 822b, 822c, 821c, 823c, and 821d are output. The delay time increases by the delay time associated with multiplexing, and is observed as an increase in jitter.

  FIG. 4A is a system configuration for explaining a problem that a traffic shaping function mounted on a relay apparatus or the like in order to stabilize transmission quality increases jitter on the contrary.

  The traffic shaping function is used for the purpose of absorbing the jitter of packet sequences that arrive at regular intervals, stabilizing the packet interval, and preventing congestion due to temporary packet concentration. As shown in the upper side of FIG. 4A, a packet sequence 904 is input from the input line 901 to the relay device 902 and output to the output line 903 to the packet sequence 905. In the network configuration, the input packet sequence 904 is spaced at a certain interval. Assuming that At this time, even when the input packet sequence 904 includes jitter, the relay apparatus 902 adjusts and aligns the timing of the output packet sequence 905 using the traffic shaping function.

  A case where a loss packet compensation function by a conventional error correction code (for example, Forward Error Correction: FEC) is further combined with the upper side of FIG. 4A is shown on the lower side of FIG. 4A. An error correction code packet 906 is inserted in the middle of the input packet sequence 904, and an error correction code packet 907 is inserted in the middle of the output packet sequence 905 and output. In addition, the relay apparatus 902 adjusts the packet interval by using the traffic shaping function so that the input packet sequence 904 and the error correction code packet 906 are integrated, and the output packet sequence 905 and the error correction code packet 907 are integrated. Output as stuff.

  Next, the operation timing of the traffic shaping function will be described with reference to FIG. 4B. As shown in the upper side of FIG. 4B, the input packet sequence 904 arrives at the relay device 902 at substantially equal intervals. An error correction code packet 906 is inserted between the input packet sequences 904. In the traffic shaping function of the relay apparatus 902, the minimum packet interval 910 to be adjusted is obtained based on the set upper limit packet rate, and individual packets are output with an interval equal to or greater than the time corresponding to the minimum packet interval 910 being secured. For packets arriving at intervals less than the minimum packet interval 910, transmission is intentionally delayed until a time corresponding to the packet interval 910. Therefore, in the upper side of FIG. 4B, the error correction code packet 906 that has arrived does not have a sufficient arrival interval (interval greater than or equal to the minimum packet interval 910) with respect to the packet of the previous input traffic 904. As shown on the side, the output error correction code packet 907 is output with a waiting time so that the interval with the immediately preceding output packet 905 is more than the minimum packet interval 910. In FIG. 4B, due to the packet interval adjustment with respect to the error correction code packet 907, the subsequent four packets of the error correction code packet 907 also do not satisfy the minimum packet interval 910, so a delay time is added.

  In the video receiving side transmission apparatus, only the user data packet excluding the error correction packet is extracted, the video code is extracted, and the video is reproduced. However, the packet following the error correction code packet (four in FIG. 4B) has a delay. Since it is added, it is observed that the jitter is increased.

  Also, in a conventional video transmission system that supports lost packets using error correction codes, the amount of packets required for packet regeneration processing in the video receiving side transmission apparatus when a packet is lost varies depending on the position of the lost packet. . For this reason, there is a problem that the processing delay time fluctuates and hardware / software resources for buffer management and the like increase. This problem will be described with reference to FIG.

  FIG. 5 is a diagram showing that a target packet required for regeneration differs depending on a packet sequence including a series of user data packets and error correction code packets and a packet lost during transmission in the network. FIG. 5A shows a case where the user data packet 1002a immediately after the error correction code packet 1011 for the user data packets 1001a to 1001e is lost. Similarly, FIG. 5B shows a case where the third packet 1002c of the user data packets 1002a to 1002e is lost. FIG. 5C shows a case where the last packet 1002e of the user data packets 1002a to 1002e is lost.

  When any one of the user data packet sequences 1002a to 1002e is lost, the video receiving side transmission apparatus regenerates a lost packet using the remaining normally received user data packet and the error correction code packet 1012. . Therefore, in FIG. 5A, the received packet before the lost packet (1002a) is not used, and the subsequent five packets received after the lost packet (user data packets 1002b to 1002e, error correction code packet) 1012). For this reason, the video receiving side transmission apparatus waits for the arrival of five user data packets and executes the regeneration process. Similarly, in FIG. 5B, reproduction is performed using the two packets before the lost packet (1002c) (user data packets 1002a and 1002b) and the following three packets (user data packets 1002d and 1002e, error correction code packet 1012). Execute the creation process. In FIG. 5C, the regeneration process is executed using the preceding four (user data packets 1002a to 1002d) and the subsequent one (error correction code packet 1012) of the lost packet (1002e).

  For this reason, the video receiving side transmission apparatus always requires an operation in consideration of the position of the lost packet in the error correction code block, and requires hardware / software resources such as buffer management.

  Furthermore, by making the error correction code packet an individual packet, overhead such as a transmission control header is added and the transmission efficiency is lowered, so that the frequency of occurrence of network congestion that causes transmission jitter and packet loss is increased.

  Due to these phenomena, the above-described conventional technique has a problem that the video quality as a real-time moving image is deteriorated unless a network having a high quality and sufficient bandwidth is always used.

  An object of the present invention is to suppress packet loss and jitter increase as a result of the increase in the number of transmission packets that become a load on the network while addressing packet loss using an error correction code. A data communication system is provided.

  One aspect of the data communication system of the present invention is a data communication system that performs error correction of data transmitted from a transmission device to a reception device via a communication path using a block code, and the transmission device includes: Packet generating means for generating a plurality of packets by dividing data to be transmitted; first sub-block dividing means for dividing the packet generated by the packet generating means into sub-blocks of a predetermined length; A first sub-block allocating unit for allocating the sub-blocks divided by the first sub-block dividing unit to a block which is an error correction calculation unit; and the first sub-block allocating unit allocated to the block by the first sub-block allocating unit. Error correction code generation means for generating an error correction code by performing error correction on the sub-block, and the error correction Error correction code multiplexing means for multiplexing the error correction code generated by the signal generation means to the packet, and a packet in which the error correction code is multiplexed by the error correction code multiplexing means to the receiving device Packet transmitting means for transmitting the packet, wherein the receiving device receives the packet transmitted from the transmitting device, and packet storage for temporarily storing the packet received by the packet receiving device. Means, an error correction code separating means for separating the error correction code multiplexed in the packet stored by the packet storage means from the packet, and the packet stored by the packet storage means having a predetermined length Second sub-block dividing means for dividing into sub-blocks, and second sub-block dividing means Therefore, a second sub-block allocating unit for allocating the divided sub-blocks to a block which is an error correction calculation unit, and a packet loss transmitted from the transmitting device on the communication path based on the received packet When packet loss is detected by the packet loss detection means, and for each subblock forming the lost packet, a specific block necessary for restoring each subblock is detected. And using the error correction code belonging to the specific block to which the sub-block is allocated and the sub-block allocated to the specific block by the second sub-block allocation means, And a lost packet restoring means for restoring.

  According to this configuration, the number of transmitted packets can be maintained when a transmission system with high error tolerance is constructed by adding an error correction code.

  In one aspect of the data communication system of the present invention, in the transmission device, a subblock is based on at least one of communication quality of the communication path, transmission data amount, and traffic amount that is a transmission data amount per unit time. Sub-block division length control means for determining the division length of the sub-block, wherein the first sub-block division means divides the packet by the sub-block division length determined by the sub-block division length control means, and The sub-block allocating means allocates each sub-block to a block which is a unit of error correction with the sub-block division length determined by the sub-block division length control means, and the error correction code multiplexing means Information on the division length of the sub-block determined by the block division length control means is multiplexed into the packet together with the error correction code. In the receiving apparatus, the error correction code separation means separates the error correction code multiplexed in the received packet from the packet, separates the information from the packet, and separates the error correction code separation. Means further comprising sub-block length detection means for detecting a division length of the sub-block based on the information, wherein the second sub-block division means is a sub-block detected by the sub-block length detection means. The packet accumulated in the packet accumulating means is divided by the division length of the block, and the second sub-block allocating means converts each sub-block to a block that is an arithmetic unit for error correction with the division length of the sub-block. It has a configuration to allocate.

  According to this configuration, it is possible to calculate an appropriate subblock length according to the traffic amount and the network state, and to control the user information dividing unit and the subblock allocating unit during transmission. It is possible to dynamically give an error correction capability suitable for the state at a low cost.

  In the transmission apparatus according to an aspect of the data communication system of the present invention, the error correction code generation means corresponds to a packet generated by the packet generation means, and is an intermediate value of a sequential calculation process of error correction codes. Are sequentially stored, a read pointer for the memory, and a write pointer having an offset corresponding to the division length of the sub-block from the read pointer, the first sub-block allocating means includes the read pointer and By sequentially moving the write pointer for each packet, the divided sub-blocks are allocated to blocks that are calculation units for error correction, and the error correction code generation means is indicated by the write pointer. A sub-block and a support corresponding to the packet following the packet. An intermediate value in an error correction calculation process is calculated from the block, the intermediate value is written at the position of the write pointer in the memory, and belongs to the block allocated by the first sub-block allocation means When the calculation for all the sub-blocks is completed, the sub-block indicated by the read pointer is read from the memory as an error correction code and output.

  This configuration eliminates the need for a memory for storing all packets required for error correction code calculation, and corresponds to an error correction code that multiplexes one packet of user data and one packet for each interleaving unit. Therefore, it is possible to realize a data communication system that can perform error correction at a low hardware cost.

  In one aspect of the data communication system of the present invention, the transmitting device transmits video or audio streaming data to the receiving device using RTP (Real-time Transport Protocol), and the error correction code multiplexing The means is configured to multiplex the error correction code in an RTP extension header.

  This configuration facilitates connection via a Network Address Port Translation (NAPT) device and allows the same packet to be transmitted even in an environment where receivers not equipped with an error correction function coexist. It becomes possible to reduce the consumption of resources.

  According to the present invention, while addressing packet loss using an error correction code, it is possible to suppress an increase in newly generated packet loss and jitter without increasing the number of transmission packets that are a load on the network. it can.

Block diagram of video transmission system Data flow diagram of the transmission device on the transmission side for explaining the operation of the video transmission system Illustration for explaining the mechanism of increase in multiplexed jitter, which is a problem of the prior art Illustration for explaining the mechanism of increase in multiplexed jitter, which is a problem of the prior art Illustration for explaining the mechanism of jitter increase during traffic shaping, a problem of the prior art Illustration for explaining the mechanism of jitter increase during traffic shaping, a problem of the prior art Illustration for explaining fluctuation of accumulated amount required for loss data regeneration of the problem of the prior art Block diagram of a video transmission system according to Embodiment 1 of the present invention. The figure which shows the data flow in the video transmission side transmission apparatus which concerns on Embodiment 1 of this invention. The figure which shows the data flow in the video receiving side transmission apparatus which concerns on Embodiment 1 of this invention. The figure which shows the flow of the data of the interleaving for multiple packet loss which concerns on Embodiment 1 of this invention Buffer memory configuration diagram of video transmission side transmission apparatus according to Embodiment 1 of the present invention Illustration for explaining problems that occur when the error correction code amount in the prior art is dynamically changed Illustration for explaining problems that occur when the error correction code amount in the prior art is dynamically changed The figure for operation | movement description when the error correction code amount in Embodiment 2 of this invention is changed dynamically. The figure for operation | movement description when the error correction code amount in Embodiment 2 of this invention is changed dynamically. The figure for demonstrating operation | movement when the interleaving space | interval in Embodiment 2 of this invention is reduced dynamically. Structure diagram of RTP packet in Embodiment 3 of the present invention The figure which shows the network at the time of the multicast delivery in Embodiment 3 of this invention

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 6 is a block diagram showing a main configuration of the data communication system according to the present embodiment.

  In FIG. 6, the data communication system performs error correction of data transmitted from the video transmission side transmission apparatus 10 to the video reception side transmission apparatus 11 via the communication path using a block code. The video transmission apparatus 10 according to the present embodiment encodes an input video signal and sends it as a packet to the network, and the video reception transmission apparatus 11 extracts the video code from the packet received from the network and decodes it. Output as a video signal. This is because a video encoding unit 101 that encodes a video signal is connected to the input side of the data communication system according to the present embodiment, and a video code that decodes the video signal from the video code to the output side of the data communication system. The decoding unit 114 is connected.

  The video transmission side transmission apparatus 10 includes a video encoding unit 101, a packet generation unit 102, a redundant code multiplexing unit 103, a packet transmission control unit 104, a sub-block length control unit 105, a user information division unit 106, It is mainly composed of a sub-block allocation unit 107 and an error correction code generation unit 108. The video reception side transmission device 11 includes a packet reception control unit 111, a received packet storage unit 112, a packet alignment unit 113, a video code decoding unit 114, a packet loss detection unit 115, and a redundant code separation unit 116. The sub-block length detection unit 117, the user information division unit 118, the sub-block allocation unit 119, and the error correction loss packet regeneration unit 120 are mainly configured.

  In the video transmission side transmission apparatus 10 of FIG. 6, the video encoding unit 101 encodes a video signal using a standard algorithm such as MPEG, and outputs the generated video code to the packet generation unit 102 as user data. Also, the video encoding unit 101 outputs the output amount (output code amount) of the video code that is the encoding result to the sub-block length control unit 105.

  When the packet generation unit 102 uses user data (video code, that is, data to be transmitted) received from the video encoding unit 101 with an appropriate length (for example, Ethernet (registered trademark)) suitable for the transmission method. And a plurality of transmission packets are generated by adding a transmission header based on a communication protocol (for example, protocols such as RTP, UDP, and IP). Then, the packet generation unit 102 outputs the generated transmission packet to the redundant code multiplexing unit 103 and the user information division unit 106.

  The redundant code multiplexing unit 103 multiplexes the transmission packet input from the packet generation unit 102 and the error correction code input from the error correction code generation unit 108, and sends the multiplexed signal (packet) to the packet transmission control unit 104. Output. The redundant code multiplexing unit 103 also relates to information (subblock length information) related to a subblock length (subblock division length) determined by a subblock length control unit 105, which will be described later, and a combination (block) of subblocks. Information (subblock combination pattern information) is multiplexed into a packet together with an error correction code.

  The packet transmission control unit 104 sends a packet to the transmission path of the network according to a procedure based on the network communication protocol. Further, the packet transmission control unit 104 detects the line quality of the network, and outputs line quality information indicating the detected line quality to the sub-block length control unit 105.

  The subblock length control unit 105 calculates an appropriate subblock length (subblock division length) based on the output code amount of the video encoding unit 101 and the channel quality information detected by the packet transmission control unit 104. To do. For example, the sub-block length control unit 105 improves the transmission efficiency by reducing the error correction code amount when the line state is good (high quality), and corrects the error when the line state is poor (low quality). In order to increase the code amount and improve the correction capability, the sub-block length instructed to the user information dividing unit 106 is increased or decreased. In addition, the sub-block length control unit 105 instructs the sub-block allocation unit 107 to combine sub-blocks to be extracted from a plurality of packets. The sub-block length control unit 105 determines the sub-block length (sub-block length) based on at least one of the communication quality (channel quality information) of the communication path, the transmission data amount (output code amount), and the transmission data traffic amount. What is necessary is just to determine the division length of a block.

  The user information dividing unit 106 sets user data (video code), which is a payload of a transmission packet input from the packet generation unit 102, to an appropriate length (predetermined length) according to the subblock length specified by the subblock length control unit 105. The sub-block is output to the sub-block allocation unit 107.

  The sub-block allocating unit 107 accumulates sub-blocks obtained by dividing the payload of the transmission packet received from the user information dividing unit 106. Then, the sub-block allocating unit 107 allocates the sub-block to a block (combination of sub-blocks) that is an error correction calculation unit. Specifically, subblock allocating section 107 extracts subblocks one by one from a plurality of transmission packets in accordance with an instruction from subblock length control section 105, and combines the extracted subblocks with error correction code generation section 108. Output to.

  The error correction code generation unit 108 generates and generates an error correction code by using an error correction code algorithm for a plurality of subblocks (subblocks allocated to each block) received from the subblock allocation unit 107. The error correction code is output to the redundant code multiplexer 103.

  In the video reception side transmission device 11 of FIG. 6, the packet reception control unit 111 receives a packet (a packet transmitted from the video transmission side transmission device 10) from the transmission line of the network in a procedure based on the communication protocol of the network. The received packet is output to the received packet storage unit 112.

  The reception packet accumulation unit 112 accumulates (holds) the packets received by the packet reception control unit 111 for a certain period of time or a certain number.

  The packet alignment unit 113 takes out the packets stored in the received packet storage unit 112 and arranges them in the order generated on the transmission side. Then, the packet sorting unit 113 removes the transmission header from the sorted packets, reassembles the video code, and outputs the video code to the video code decoding unit 114.

  The video code decoding unit 114 decodes the video code received from the packet sorting unit 113 in accordance with a standard algorithm such as MPEG and outputs a video signal.

  On the other hand, in parallel with the packet sorting unit 113 taking out the transmission packet from the received packet storage unit 112, the packet loss detection unit 115 monitors the packet sequence stored in the received packet storage unit 112 and detects the lost packet. Is detected. That is, the packet loss detection unit 115 detects the loss of the packet transmitted from the video transmission side transmission device 10 on the network (communication path) based on the received packet. When the packet loss detection unit 115 detects the packet loss, the packet loss detection unit 115 instructs the error correction loss packet regeneration unit 120 to regenerate the missing packet.

  In the packet regeneration process, the redundant code separation unit 116, the error correction code multiplexed in the packet stored by the received packet storage unit 112, and management information for error correction (sub-block length information, Sub-block combination pattern information) is separated from the packet. Specifically, the redundant code separation unit 116 separates the error correction code in the transmission packet and the management information for error correction (sub-block length information, sub-block combination pattern information), and sets the error correction code as an error. The data is output to the correction loss packet regeneration unit 120, and the management information is output to the sub-block length detection unit 117.

  The subblock length detection unit 117 receives the subblock length information and the subblock length information from the management information for error correction (management information sequence stored in successive packets) received from the redundant code separation unit 116. Combination pattern information is detected, sub-block length information is output to user information division section 118, and sub-block combination pattern information is output to sub-block allocation section 119.

  The user information dividing unit 118 divides user data (video code) in the payload of the transmission packet stored in the received packet storage unit 112 based on the subblock length information instructed from the subblock length detection unit 117. A plurality of sub-blocks are generated. Then, the user information dividing unit 118 outputs the generated sub block to the sub block allocating unit 119.

  The sub-block allocation unit 119 receives the sub-block (a plurality of sub-blocks generated from a plurality of transmission packets) received from the user information division unit 118 based on the sub-block combination pattern information instructed from the sub-block length detection unit 117. Are combined to reconstruct a block which is an error correction operation unit. That is, the sub-block allocating unit 119 allocates the sub-block divided by the user information dividing unit 118 to a block that is an error correction calculation unit. Then, the sub-block allocation unit 119 outputs the reconfigured block to the error correction loss packet regeneration unit 120.

  The error correction loss packet regeneration unit 120 uses a set of the error correction code received from the redundant code separation unit 116 and the sub-block of user information included in the error correction code block received from the sub-block allocation unit 119. Based on the algorithm of the error correction code, the sub-block not received is regenerated. Specifically, the error correction loss packet regeneration unit 120 restores each sub-block for each sub-block forming the lost packet when the packet loss detection unit 115 detects the packet loss. A specific packet that is required and is a missing packet using an error correction code belonging to the specific block to which the subblock is allocated and a subblock allocated to the specific block by the subblock allocation unit 119 To restore. The error correction loss packet regenerator 120 repeats this process until the entire sub-block constituting the payload of the lost packet is regenerated. The regenerated payload of the transmission packet is output to the packet alignment unit 113 and inserted at an appropriate position in the packet sequence.

  The packet generation unit 102 of the video transmission side transmission device 10 functions as a packet generation unit, the redundant code multiplexing unit 103 functions as an error correction code multiplexing unit, the packet transmission control unit 104 functions as a packet transmission unit, The subblock length control unit 105 functions as a subblock division length control unit, the user information division unit 106 functions as a first subblock division unit, and the subblock allocation unit 107 functions as a first subblock allocation unit. The error correction code generation unit 108 functions as error correction code generation means. In addition, the packet reception control unit 111 of the video reception side transmission device 11 functions as a packet reception unit, the received packet storage unit 112 functions as a packet storage unit, the packet loss detection unit 115 functions as a packet loss detection unit, and redundancy. The code separation unit 116 functions as an error correction code separation unit, the subblock length detection unit 117 functions as a subblock length detection unit, the user information division unit 118 functions as a second subblock division unit, and subblock allocation The unit 119 functions as a second sub-block allocation unit, and the error correction loss packet regeneration unit 120 functions as a lost packet restoration unit.

  About the data communication system comprised as mentioned above, the operation | movement is demonstrated using FIGS. FIG. 7 is a diagram for explaining the basic operation of the video transmission side transmission device 10, FIG. 8 is a diagram for explaining the basic operation of the video reception side transmission device 11 when a packet is lost, and FIG. FIG. 2 is a diagram illustrating the operation of the data communication system according to the present invention when packets are interleaved in correspondence with a state in which a plurality of consecutive packets are lost as the line quality of the network.

  In FIG. 7, a video encoding unit 101 generates user data 201 (video code). The packet generation unit 102 divides the user data 201 into an appropriate length (fixed length) (converts it into a transmission packet), generates payloads 211 to 216, and adds transmission headers 221 to 226 to the transmission packets. Is generated.

  The redundant code multiplexing unit 103 multiplexes the error correction code information 231 to 235 immediately after the transmission header in the generated transmission packet. Here, the error correction code information is an information element including an error correction code and its management / control information. One method of multiplexing error correction code information is to use an extension header of the transmission protocol RTP. In this method, the error correction code information appears as a header information element integrated with the transmission headers 222 to 226 in the transmission apparatus and the relay apparatus in the network.

  For example, the payload 211 is divided into a certain length by the user information dividing unit 106 and becomes sub-blocks 211a to 211e. Similarly, payloads 212 to 216 are each divided into five sub-blocks (sub-blocks 212a to 216e). In FIG. 7, the case where the payload is divided into five sub-blocks will be described as an example, but the number of payload divisions (number of generated sub-blocks) is not limited to five.

  The sub-block allocating unit 107 extracts a combination in which the position in each packet is shifted one by one from these sub-blocks and outputs it to the error correction code generating unit 108. For example, by combining the sub-blocks 211e, 212d, 213c, 214b, and 215a shown in FIG. 7, a block that is an error correction calculation unit is formed. Then, an error correction code is generated by performing error correction on the combination (block) of sub-blocks. Thereby, the error correction code information 235 shown in FIG. 7 is generated. Similarly, other error correction code information 231 to 234 and subsequent error correction code information (not shown) are extracted by shifting the position in the packet one by one from the sub-block obtained by dividing the payload of each packet. Generate using a combination.

  FIG. 8 illustrates an operation when the packet 304 is lost in any of the received packets 300 to 309 received by the video reception side transmission apparatus 11 on the network.

  The packet reception control unit 111 stores the packets 300 to 309 received from the network in the received packet storage unit 112. Note that the delay time of packets varies in the network, and the arrival order of packets may be switched. Therefore, in the received packet storage unit 112, the packets 300 to 309 are not necessarily arranged in the order shown in FIG. Therefore, the packet sorting unit 113 sorts these packets in the order of generation on the transmission side, and delivers them to the video code decoding unit 114.

  In FIG. 8, the packet loss detection unit 115 monitors the received packets 300 to 309 stored in the reception packet accumulation unit 112, and a packet 304 that does not arrive even after a delay time allowed for the system is lost. Judge that

  The error correction loss packet regeneration unit 120 collects sub-blocks necessary for regeneration when the packet loss detection unit 115 detects packet loss and is instructed to regenerate lost packets (304). At that time, first, the redundant code separation unit 116 extracts the error correction code information 330 to 339 from each packet, outputs it to the error correction loss packet regeneration unit 120, and extracts the error correction code management / control information from each packet. And output to the sub-block length detector 117.

  The sub-block length detection unit 117 determines the sub-block combination pattern and sub-block length based on the management / control information, outputs the combination pattern to the sub-block allocation unit 119, and the sub-block length is the user information division unit 118. Output to. For example, in FIG. 8, the lost packet (304) due to the combination pattern information of the sub-blocks calculated based on the error correction code management / control information multiplexed in the error correction code information 330 to 339. The received packets necessary for regeneration of the packets are determined as packets 300 to 303 and 305 to 309.

  As shown in FIG. 8, when the payload of the user information is divided into five sub-blocks, four near-side packets belonging to the same error correction code group (block) as the lost packets ( (5-1)) and five subsequent packets are extracted from the received packet storage section 112. That is, when the video receiving side transmission apparatus 11 is divided into N sub-blocks, (N−1) packets on the near side of the lost packet and N packets on the following side are lost. Just take out (collect). That is, the packets to be collected to regenerate lost packets are uniquely determined only by the payload length and subblock length. Therefore, information for identifying the sub-block length and the group is sufficient as the error correction code management / control information. In other words, the video receiving side transmission device 11 does not need to be aware of the position of the lost packet in the packet sequence constituting the error correction code block. The purpose of using the group will be described in detail later with reference to FIG.

  Next, division processing into sub-blocks and combinations of sub-blocks will be described.

  The payload of the received packet 300 shown in FIG. 8, that is, the user information 310 is divided by the user information dividing unit 118 into lengths (subblock division lengths) designated by the subblock length detecting unit 117, and subblocks 310 a to 310- 310e. Similarly, the payload of the received packet 301 is also divided into sub-blocks 311a to 311e. The other packets 302 to 303 and 305 to 308 are similarly divided into sub-blocks (302a to 308e). For the received packet 309, only the error correction code information 339 is used.

  In the video transmission side transmission device 10, the error correction code block is configured with sub-blocks shifted by one for each packet. Even in the video reception side transmission device 11, when regenerating a lost packet, An error correction code block is generated by combining sub-blocks whose positions in the packet are shifted one by one. For example, the subblock allocation unit 119 performs error correction in the subblocks 310e, 311d, 312c, and 313b and the error correction code information 335 in order to regenerate (restore) the first subblock 314a of the payload of the lost packet 304. The codes are combined as one error correction code block. Then, the sub-block allocation unit 119 outputs the generated error correction code block to the error correction loss packet regeneration unit 120.

  Similarly, the subblock allocation unit 119 performs error correction in the subblocks 311e, 312d, 313c, and 315a and the error correction code information 336 in order to regenerate the second subblock 314b of the payload of the lost packet 304. Combine the signs. Further, for example, the sub-block length control unit 105 generates errors in the sub-blocks 315d, 316c, 317b, and 318a and the error correction code information 339 in order to regenerate the sub-block 314e at the end of the payload of the lost packet 304. Combine correction codes. In this way, the sub-block length control unit 105 sequentially regenerates the sub-blocks constituting the payload of the lost packet 304. Thus, the payload of the lost packet 304, that is, the sub blocks 314a to 314e necessary for regenerating the user information 314 are regenerated.

  In the present embodiment, since one piece of code information is originally divided and mounted on the payloads of a plurality of packets, the redundancy of the information stored in the transmission header is high. However, since the video receiving side transmission device 11 regenerates only the payload portion of the lost packet, it is possible to output user information without loss.

  The error correction loss packet regeneration unit 120 outputs the regenerated payload 314 to the packet alignment unit 113. Then, the packet aligning unit 113 takes out the payloads 310 to 313 and 315 to 319 of the normally received packet from the received packet storage unit 112, arranges them together with the regenerated payload 314 in order of generation, and combines them. The data is output to the code decoding unit 114. The video code decoding unit 114 decodes the video code to obtain a video signal.

  FIG. 9 illustrates an operation in a poor network state where a plurality of consecutive packets are missing.

  In FIG. 9, consecutive packets 401 to 420 are generated in packet generation order (for example, RTP sequence number order) 431, 432, and 433, and these packets are classified into four groups. As shown in FIG. 9, the first group 441 includes packets 401, 405, 409, 413, and 417. The second group 442 includes packets 402, 406, 410, 414, and 418. The third group 443 includes packets 403, 407, 411, 415, and 419. The fourth group 444 includes packets 404, 408, 412, 416 and 420. Note that a packet (not shown) that continues after the packet shown in FIG. 9 also belongs to any of the groups 441 to 444. As a result, when the number of consecutive lost packets is 4 (that is, the number of groups) or less, the number of lost packets in each group is 1 or less (that is, at most 1).

  For example, in FIG. 9, it is assumed that packets 407, 408, 409, and 410 (four packets) indicated by the packet generation order 432 are continuously lost somewhere on the network during transmission. That is, the packet 409 is lost in the group 441, the packet 410 is lost in the group 442, the packet 407 is lost in the group 443, the packet 408 is lost in the group 444, and one packet is included in each group. Observed as a missing packet.

  In other words, even when an error correction code system having an error correction capability for one payload of a packet is used, it is already common that a continuous burst loss can be dealt with by combining interleaving techniques as shown in FIG. It is done as a technology. In the present embodiment, as shown in FIG. 8, by storing information indicating the group to which each packet belongs at the time of interleaving as an identifier in the error correction code information, the video receiving side transmission apparatus 11 can easily perform grouping. Can be identified, and it is possible to determine whether to collect packets necessary for correction.

  Next, an efficient implementation example of the main part of the video transmission side transmission apparatus 10 in the data communication system (FIG. 6) of the present embodiment will be described with reference to FIG.

  In FIG. 10, two software tasks (transmission task 501 and error correction code (FEC) generation task 502) are defined. The transmission task 501 stores the user data (video code) generated by the video encoding unit 101 in the transmission waiting code buffer 541. Further, in the transmission waiting code buffer 541, user data is divided into lengths corresponding to the payload length of the packet, and payloads 511, 512, and 513 are generated. The transmission task 501 adds a transmission header 521 to the first payload 511 to generate a transmission packet, calls the FEC generation task 502, and then transmits the transmission packet 56 to the network via the packet transmission control unit 104.

  The FEC generation task 502 (mainly corresponding to the error correction code generation unit 108) is provided with work memories 551 to 554 corresponding to the number of interleave groups, and provided with a read pointer 561a and a write pointer 561b for each work memory. .

  Further, the read pointer 561a and the write pointer 561b are provided with an offset corresponding to the sub-block length (an offset corresponding to the sub-block division length). That is, the write pointer 561b is obtained by adding the sub-block length to the read pointer 561a.

  To the error correction code generation task 502 called from the transmission task 501, the transmission header 521 of the packet waiting for transmission and the address on the memory of the payload 511 are passed as arguments. The error correction code generation task 502 accesses each piece of information of the transmission waiting packet using the received argument.

  In this embodiment, as described above, the payload 511 of the transmission packet is divided into sub-blocks 511a to 511e having a certain length, and error correction codes are combined by combining the sub-blocks at corresponding positions in the preceding and succeeding packets. Is generated.

  For example, in FIG. 10, the error correction code generation task 502 selects the work memory 552 of #l and determines the error correction code generation for each sub-block when the interleave group to which the first packet belongs is determined to be # 1. Are sequentially executed from the position indicated by the write pointer 561b. Whether the exclusive OR (XOR) is used as the error correction code method or the remainder of division by the generator polynomial is used, the work memory 552 stores the operation result (intermediate value of the sequential operation process of the error correction code). ) Only. In some cases, the subblock boundary may be recognized depending on the method, but since the subblock boundary is obtained by simple addition from the write pointer 561b, no special hardware is required. Further, since there is an uncalculated area 563 at the end of the work memory 552, it is cleared to zero (initial value “0”) before the error correction code calculation. In addition, since the subblocks are assigned to consecutive addresses on the memory in which the payload 511 is stored and on the work memory 552, pointers indicating the boundaries of the individual subblocks are not necessary. In order, operations are performed with the efficient length of the hardware (8 bits, 16 bits, 32 bits, etc.).

  Further, the interleave group identifier “# 1” indicating the classification of the work memory is multiplexed behind the transmission header 521 of the transmission packet as the error correction code management / control information 531a together with the sub-block length. After completion of the calculation, information on the work memory from the read pointer 561a to immediately before the write pointer 561b is multiplexed as an error correction code 531b (FEC parity) behind the transmission header 521 of the packet. The read pointer 561a and the write pointer 561b are updated to new read pointers 562a and write pointers 562b by adding the subblock lengths, respectively. By updating the write pointer 561b with a new write pointer 562b, it is possible to calculate the payloads of packets that are consecutive in the same group while shifting them by sub-block lengths.

  As described above, in the FEC task 502, the sub-block allocating unit 107 sequentially stores the intermediate value of the sequential calculation process of the error correction code corresponding to the packet generated by the packet generating unit 102 in the transmission task 501. It has a memory, a read point for the memory, and a write point. Then, the sub-block allocating unit 107 performs allocation to a block serving as an error correction calculation unit for each divided sub-block by sequentially moving the read pointer and the write pointer for each packet. Then, the error correction code generation unit 108 calculates an intermediate value of the error correction calculation process from the sub block indicated by the write pointer and the sub block corresponding to the packet subsequent to the target packet, and calculates the intermediate value. Write to the location of the memory write pointer. When the calculation for all the subblocks belonging to the block assigned by the subblock allocating unit 107 is completed, the error correction code generation unit 108 reads out the subblock indicated by the read pointer from the memory as an error correction code and outputs it.

  In addition, the interleaving group determination is sufficient if the groups are assigned in order each time it is called from the transmission task 501, and can also be realized by a counter that is simply a residue system based on the number of groups.

  In FIG. 10, the error correction code generation task 502 only refers to the payload 511 of the first packet stored on the memory in the transmission task 501, does not write information, and transmits the transmission task 501 ( Packet transmission control unit 104). For this reason, resource consumption can be minimized and high-speed processing can be performed. Further, when the error correction function is set to be invalid, the control can be performed only by stopping the call of the error correction code generation task 502.

  As described above, in the present embodiment, assuming that the division length of the sub-block is N, when the lost packet is regenerated on the receiving side, (N−1) pieces of data before the lost packet are generated. It is only necessary to take out (collect) the packets and N packets on the subsequent side, and the amount of accumulated packets required on the receiving side when the lost packet is regenerated can be made constant. That is, any video code packet can be regenerated at least when a packet after N sub-block divisions is received. Thereby, on the receiving side, the delay time due to the regeneration process can be made constant. In other words, variation in processing delay time due to regeneration processing between packets can be suppressed.

  In the present embodiment, since the error correction code and information about the error correction code (subblock length information, combination pattern information, etc.) are multiplexed in the header of each packet, the amount of data increases in the parity part of each packet. However, the total number of packets does not increase. That is, as compared with the case where the data packet and the error correction code are individually transmitted over the network, the total number of packets is not increased due to the transmission of the error correction code, and The sending interval is not changed.

  Therefore, according to the present embodiment, it is possible to suppress an increase in the packet loss rate due to the load of the error correction code, prevent an increase in jitter due to the transmission delay time and the transmission, and a jitter due to the loss compensation process, and to perform a high quality transmission. It can be performed.

(Embodiment 2)
The error correction code increases the overhead, which is a part other than user data, with respect to the network, and causes a significant decrease in transmission efficiency. Therefore, if the error correction capability is strengthened, the load on the network, that is, the total amount of traffic increases, which may even cause congestion. On the other hand, when the amount of traffic is limited in a certain effective band, the substantial transmission capacity of user data is reduced. Therefore, a technique for dynamically controlling the error correction capability based on the transmission quality of the network has been considered.

  FIG. 11 shows a problem in controlling the error correction capability based on the transmission quality of the network in the prior art.

  The upper side of FIG. 11A shows a state in which one error correction code packet is transmitted for every six data packets. Specifically, the data packet group 11a1 is one block that generates an error correction code, and an error correction code is calculated for the data packet group 11a1. The generated error correction code packet 11a2 is sent to the network following the data packet (data packet group 11a1).

  Here, when it is determined that the channel quality of the network is deteriorated and the error correction capability is to be improved, the ratio between the data packet and the error correction code packet is controlled, and the ratio of the error correction code packet is increased. For example, the lower part of FIG. 11A is changed from the process of generating one error correction code packet for six data packets (upper part of FIG. 11A) to the process of generating one error correction code packet for five data packets. The operation is shown. Specifically, the data packet group 11a3 forms one block for generating an error correction code with five data packets, and the error correction code is calculated for the data packet group 11a3. The generated error correction code packet 11a4 is sent to the network following the data packet group 11a3.

  Here, considering the operation on the receiving side, since one error correction code packet has arrived after 6 data packets before the change of the processing, the receiving side receives the packet immediately after the data packet 11a6 shown in FIG. 11A. The error correction code packet is expected to arrive at the position 11a5. However, since the error correction code packet 11a4 actually arrives immediately after the five data packets in the data packet group 11a3, the receiving side changes the order of the error correction code packet that should reach the packet position 11a5 to the data packet 11a6. It cannot be determined whether the packet has arrived first or a packet corresponding to the data packet 11a6 has been lost in the middle of the network.

  In particular, in the case of a system that does not allow the order of packets to be changed, it is erroneously recognized that one data packet is missing. Further, in the case of a system on the premise that the order of the packets is changed, when the data packet 11a6 arrives immediately after the error correction code packet 11a4, the error correction code packet 11a4 includes the data packet group 11a3 and the data packet 11a6. There is a risk of erroneous recognition as an error correction code packet for the combined six packets. Therefore, the error correction code packet 11a4 is synchronized between the transmission side and the reception side with respect to information for identifying the target data packet group 11a3 or a change in the generation rate of the error correction code packet. It is necessary to add control information.

  However, if this special control information (synchronization information) is multiplexed only in the error correction code packet 11a4, when the error correction code packet 11a4 is lost on the network, the receiving side can receive the target data packet of the error correction code block. May be misrecognized as being six, and the combination of blocks cannot be recognized correctly, which may lead to a frustration in subsequent error correction processing.

  Next, FIG. 11B shows an example of control for reducing the error correction capability and improving the transmission efficiency.

  The upper side of FIG. 11B shows a state in which one error correction code packet is transmitted for every six data packets. Specifically, the packet group 11b1 is one block that generates an error correction code, and an error correction code is calculated for the data packet group 11b1. The generated error correction code packet 11b2 is sent to the network following the data packet.

  Here, if it is determined that the transmission efficiency is to be improved due to factors such as improvement of the channel quality of the network, the ratio of data packets and error correction code packets is controlled and the ratio of error correction code packets is decreased. . For example, the lower part of FIG. 11B is changed from the process of generating one error correction code packet for six data packets (upper part of FIG. 11B) to the process of generating one error correction code packet for seven data packets. The operation is shown. Specifically, the data packet group 11b3 forms one block for generating an error correction code packet with seven data packets, and an error correction code is calculated for the data packet group 11b3. The generated error correction code packet 11b4 is sent to the network following the data packet group 11b3.

  Considering the operation on the receiving side, since one error correction code packet has arrived after 6 data packets before the change of processing, the receiving side performs error correction at the packet position 11b5 before the data packet 11b6. I expect code packets to arrive. However, since the error correction code packet 11b4 arrives after the seven data packets of the data packet group 11b3 in practice, the receiving side changes the order of the error correction code packet that should reach the packet position 11b5 to the data packet 11b6. It cannot be determined whether it arrived later or whether a packet corresponding to the error correction code packet 11b4 was lost in the middle of the network.

  In particular, in the case of a system that does not allow the order of packets to be changed, it is erroneously recognized that one error correction code packet has been lost. Further, in the case of a system on the premise that the packet order is changed, when the error correction code packet 11b4 arrives immediately after the arrival of the data packet 11b6, the error correction code packet 11b4 is changed from the data packet group 11b3 to the data packet 11b6. There is a risk of erroneous recognition as an error correction code packet for six packets excluding. Therefore, the error correction code packet 11b4 is synchronized between the transmission side and the reception side with respect to information for identifying the target data packet group 11b3 or a change in the generation rate of the error correction code packet. It is necessary to add control information.

  However, if this special control information (synchronization information) is multiplexed only in the error correction code packet 11b4, when the error correction code packet 11b4 is lost on the network, the receiving side can receive the target data packet of the error correction code block. May remain erroneously recognized, and the combination of blocks cannot be recognized correctly, which may lead to errors in subsequent error correction processing.

  To avoid the error correction processing described with reference to FIGS. 11A and 11B, some control information (synchronization information) is multiplexed not only on the error correction code packet but also on the data packet. There is a need. However, in consideration of packet loss or change of order, a complicated protection processing flow is required, and RFC 5109 (IETF RTP Payload Format for Generic Forward Error Correction) and RFC 3550 (IETF RTP: A Transport Protocol for Real). -time Applications) and other industry standard specifications.

  On the other hand, FIG. 12 and FIG. 13 show the operation of the data communication system according to the present embodiment, that is, the operation when the error correction capability is dynamically controlled during transmission. FIG. 12 shows the operation when the error correction capability and overhead are changed by controlling the sub-block length, and FIG. 13 shows the operation when the interleaving interval, that is, the number of groups due to interleaving is decreased. Indicates.

  In FIG. 12A, as in the example of FIG. 11A, in order to improve the error correction capability due to factors such as detection of deterioration in the transmission path quality of the network, an error is generated at a ratio of one error correction code to six user data. An operation in a case where a state in which a correction code is generated and multiplexed is shifted to a state in which an error correction code is generated and multiplexed at a ratio of one error correction code to five user data is shown.

  In FIG. 12A, transmission packets 12a1 to 12a8 are sent in order from the transmission side. Further, for example, the transmission packet 12a1 includes a transmission header 12a11, an error correction code control header 12a12, an error correction code 12a13, and a payload 12a14, and the payload 12a14 is divided into six equal-length sub-blocks.

  The transmission packets 12a2 to 12a8 are also configured by the same information elements as the transmission packet 12a1. However, there are differences in the configuration of sub-blocks generated by dividing the payload between transmission packets.

  For example, in the transmission packet 12a2 shown in FIG. 12A, the subblock length from the top to the fifth subblock 12a24e is the length when the payload is divided into six. On the other hand, the last (sixth) sub-block is formed of a payload tail portion 12a24f and a virtual zero portion 12a25 obtained by dividing the payload into six parts. The sub-block length of the last sub-block (the sum of 12a24f and 12a25) is equal to the sub-block length when the payload is divided into five.

  In the transmission packet 12a3 shown in FIG. 12A, the subblock length from the top to the fourth subblock 12a34d is the length when the payload is divided into six, and the subblock length of the fifth subblock 12a34e. Is the length when the payload is divided into five. The last (sixth) sub-block is formed of a payload tail portion 12a34f and a virtual zero portion 12a35. The sub-block length of the last sub-block (the sum of 12a34f and 12a35) is equal to the sub-block length when the payload is divided into five.

  Hereinafter, in the order of the transmission packets 12a4, 12a5, and 12a6 shown in FIG. 12A, the number of subblocks each having a length obtained by dividing the payload into six is reduced by 1, and the number of subblocks having a length obtained by dividing the payload into five is obtained. A sixth sub-block is formed by incrementing by 1 and using a virtual zero portion 12a45, 12a55, 12a65 at the end.

  On the other hand, in the transmission packet 12a7, the payload is composed only of sub-blocks having a length when the payload is divided into five, and the sixth sub-block disappears.

  The error correction code 12a73 multiplexed in the transmission packet 12a7 is an error correction code calculated from the six sub-blocks 12a14f, 12a24e, 12a34d, 12a44c, 12a54b, 12a64a, and the length of the error correction code 12a73 is payload. Is the length when the is divided into six. The error correction code 12a83 multiplexed in the transmission packet 12a8 includes a sub-block formed from the tail 12a24f of the payload divided by the transmission packet 12a2 and the virtual zero portion 12a25, and five sub-blocks 12a34e and 12a44d. , 12a54c, 12a64b, and 12a74a, the length of which is the length when the payload is divided into five.

  By this procedure, as shown in FIG. 12A, the proportion of the virtual zero portion in the last sub-block increases in the order of the transmission packets 12a2 to 12a6, and the number of sub-blocks becomes five after the transmission packet 12a7. . For this reason, in FIG. 12A, when the last sub-block of the transmission packet 12a8 is used for the calculation of the error correction code, the number of sub-blocks used for the calculation of the error correction code is five.

  Here, a case will be described in which the transmission apparatus on the transmission side uses a vertical (intersubblock) exclusive OR (vertical parity method) as an error correction code for the reception side transmission apparatus. In this case, there is no correlation in error correction code generation calculation between adjacent calculation units (bits, bytes, etc.) in the sub-block, and the error correction code itself generated wherever the sub-block boundary is provided is the same. Will be. Therefore, the boundary between subblocks in the payload has no meaning other than the boundary between the first subblock multiplexed in the next packet and the second and subsequent subblocks. For this reason, it is possible to easily change the length of the sub-block at an arbitrary timing. For example, in FIG. 12A, even when it is the timing when the transmission packet 12a7 is generated that determines the error correction capability change based on the state of the transmission path, the transmission side can generate the transmission packet 12a8. That is, after the determination of the control, the error correction capability can be changed immediately without waiting for the next error correction code block as in the prior art.

  In FIG. 12B, as in the example of FIG. 11B, error correction is performed at a rate of one error correction code for five pieces of user data in order to improve transmission efficiency due to factors such as detecting an improvement in network transmission path quality. An operation in a case where a state where a code is generated and multiplexed is shifted to a state where an error correction code is generated and multiplexed at a ratio of one error correction code to six user data is shown.

  In FIG. 12B, transmission packets 12b1 to 12b8 are sequentially sent from the transmission side. Further, for example, the transmission packet 12b1 includes a transmission header 12b11, an error correction code control header 12b12, an error correction code 12b13, and a payload 12b14, and the payload 12b14 is divided into five equal-length subblocks.

  The transmission packets 12b2 to 12b8 are also configured by the same information elements as the transmission packet 12b1. However, there are differences in the configuration of sub-blocks generated by dividing the payload between transmission packets.

  For example, in the transmission packet 12b2 shown in FIG. 12B, the subblock length from the top to the fourth subblock 12b24d is the length when the payload is divided into five, and the subblock length of the fifth subblock 12b24e Is the length when the payload is divided into six. Further, the tail (sixth) sub-block is formed by a divided payload tail portion 12b24f and a virtual zero portion 12b25. The sub-block length of the last sub-block is equal to the length when the payload is divided into six.

  In the transmission packet 12b3 shown in FIG. 12B, the subblock length from the top to the third subblock 12b34c is the length when the payload is divided into five, and the fourth subblock 12b34d and the fifth subblock The sub-block length of the sub-block 12b34e is the length when the payload is divided into six. The last (sixth) sub-block is formed by a payload tail part 12b34f and a virtual zero part 12b35. The sub-block length of the last sub-block is the length when the payload is divided into six.

  Hereinafter, in the order of the transmission packets 12b4 and 12b5 shown in FIG. 12B, the number of sub-blocks each having a length obtained by dividing the payload into 5 is reduced by 1, and the number of sub-blocks having a length obtained by dividing the payload into 6 is respectively incremented by 1. The sixth sub-block is formed using a virtual zero portion 12b45, 12b55 at the end.

  On the other hand, in the transmission packet 12b6, the payload is composed only of sub-blocks having a length when the payload is divided into six, and the virtual zero portion disappears.

  The error correction code 12b73 multiplexed in the transmission packet 12b7 is an error correction code calculated from the five sub-blocks 12b24e, 12b34d, 12b44c, 12b54b, and 12b64a. The length of the error correction code 12b73 is 6 payloads. It becomes the length when divided into pieces. The error correction code 12b83 multiplexed in the transmission packet 12b8 includes a sub-block formed from the tail 12b24f of the payload divided by the transmission packet 12b2 and a virtual zero portion 12b25, and five sub-blocks 12b34e and 12b44d. 12b54c, 12b64b, and 12b74a, and the length of the error correction code 12b83 is the length when the payload is divided into six.

  By this procedure, as shown in FIG. 12B, the proportion of the virtual zero portion in the last sub-block decreases in the order of the transmission packets 12b2 to 12b5, and the transmission packet 12b6 and subsequent payloads have exactly six sub-blocks. It is divided into. For this reason, in FIG. 12B, when the last sub-block 12b64f of the transmission packet 12b6 is used for the calculation of the error correction code, the change control to the number of sub-blocks used for the calculation of the error correction code is completed.

  Here, a case will be described in which the transmission apparatus on the transmission side uses a vertical (intersubblock) exclusive OR (vertical parity method) as an error correction code for the reception side transmission apparatus. In this case, there is no correlation in error correction code generation calculation between adjacent calculation units (bits, bytes, etc.) in the sub-block, and the error correction code itself generated wherever the sub-block boundary is provided is the same. Will be. Therefore, the boundary between subblocks in the payload has no meaning other than the boundary between the first subblock multiplexed in the next packet and the second and subsequent subblocks. For this reason, it is possible to easily change the length of the sub-block at an arbitrary timing. For example, in FIG. 12B, even when it is the timing when the transmission packet 12b6 is generated that determines the change in the error correction capability based on the state of the transmission path, the transmission packet 12b7 can be generated on the transmission side. . That is, after the determination of the control, as in the prior art. The error correction capability can be changed immediately without waiting for the next error correction code block.

  In FIG. 12A and FIG. 12B, error correction code (FEC) headers 12a12 and 12b12 have error correction code control for the length of error correction code 12a13 and error correction code 12b13, and the length of the subblock at the beginning of the payload. Store it as information. As a result, even if packet loss occurs before and after switching, the receiving side can correctly recognize the error correction code.

  Next, FIG. 13 shows the operation of the present embodiment when the interleaving interval is reduced by one. A block formed for error correction code generation is divided into a number of groups equal to the interleaving interval by interleaving, and a block formed for error correction code generation is composed of sub-blocks of transmission packets in each group. Increasing and decreasing the interleaving interval is nothing but increasing or decreasing the number of groups of transmission packets. FIG. 13 shows details of the operation when one transmission packet group is reduced.

  In FIG. 13, transmission packets 1301 to 1313 are sent in order from the transmission side. For example, the transmission packet 1301 includes a transmission header 13011, an error correction code control header 13012, an error correction code 13013, and a payload 13014. The payload 13014 is divided into five equal-length subblocks. Also, the transmission packets 1302 to 1313 are configured by similar information elements.

  In FIG. 13, transmission packets 1301, 1303, 1305, and 1307 belong to the first interleave group (group #n), and transmission packets 1302, 1304, and 1306 belong to the second interleave group (group #m). The transmission packets 1308 to 1313 belong to an interleave group in which the first interleave group and the second interleave group are integrated.

  Here, when the seventh transmission packet 1307 shown in FIG. 13 is transmitted, it is determined that the reduction of the interleaving interval (reduction of the number of groups) has been determined, and it becomes necessary to eliminate the second interleaving group (that is, , Group #m is abolished and group #n is left).

  In this case, first, an error correction code 13083a generated from the sub-blocks 13014d, 13034c, 13054b, and 13074a of the first interleave group is multiplexed in the eighth transmission packet 1308.

  A transmission packet after the sixth transmission packet 1306 belonging to the group of the second interleave to be canceled is not generated. However, the error correction code generated from the sub-blocks 13024c, 13044b, and 13064a of the second interleave group is multiplexed as the error correction code 13083b on the eighth transmission packet 1308 belonging to the integrated interleave group.

  Further, in multiplexing the error correction code in the eighth transmission packet 1308, the information for identifying the group of interleaving and the information indicating the length of each error correction code are stored in error correction code control information 13082a (related to group #n). Information), 13082b (information on group #m). As a result, the individual error correction codes 13083a and 13083b can be correctly separated on the receiving side.

  In FIG. 13, since the payload of the transmission packet is divided into five sub-blocks, five transmission packets to be mounted (multiplexed) with the error correction code of the second interleave group are also required. Therefore, as with the transmission packet 1308, the error correction codes 13093a, 13103a, 13113a, 13112a and the error correction code control information 13092a, 13102a, 13112a, and the first interleaved group are transmitted to the transmission packets 1309, 1310, 1311, and 1312. 13122a, error correction codes 13093b, 13103b, 13113b, 13112b and error correction code control information 13092b, 13102b, 13112b, 13122b of the second interleave group are multiplexed. As a result, all the error correction codes relating to the second group of interleaves can be delivered to the receiving side.

  In the subsequent transmission packet 1313, since the transmission of the error correction code relating to the second interleave group has been completed, only one set of the error correction code 13133 and the error correction code control information 13132 is multiplexed.

  As described above, according to the present embodiment, the increase / decrease in error correction capability during communication can be easily and dynamically controlled by providing the subblock division length control means on the transmission side. In other words, by changing the sub-block division length according to the channel quality, error correction capability and error correction can be achieved without installing special functions such as a synchronization function between transmission and reception on both the transmission and reception side transmission devices. It becomes possible to dynamically control the relationship with the overhead of the transmission amount by the code.

  Further, according to the present embodiment, as in the first embodiment, an increase in the packet loss rate due to the load of the error correction code is suppressed, and the transmission delay time, the jitter accompanying the transmission, and the jitter due to the loss compensation process are increased. And high-quality transmission can be performed.

(Embodiment 3)
FIG. 14A shows the structure of an RTP (Real-time Transport Protocol) packet defined as an industry standard (see, for example, RFC3550 RTP (Real-time Transport Protocol)). As shown in FIG. 14, the packet is composed of an RTP header 141 and a payload 143, and user data such as video and audio is stored in the payload 143.

  For example, when “1” is set in the 4th bit x bit 144 from the top of the RTP header 141, the RTP extension header 142 is added to the end of the basic information. As shown in FIG. 14, the RTP extension header 142 is composed of a field indicating the type of extension header, a field indicating the length of the extension header, and an RTP extension header information unit 145 according to an application using RTP. By giving an appropriate type, it is possible to multiplex arbitrary information. It is stipulated that a receiving terminal that does not support the RTP extension header or a receiving terminal that does not support the multiplexed extension header type ignores the RTP extension header information part 145 (for example, RFC3550 RTP (Real- time Transport Protocol).

  FIG. 14B shows an example of a method for multiplexing information to the RTP extension header information unit 145 according to the present embodiment. As shown in FIG. 14B, the RTP extension header information part 145 is an information type indicating that the information is for a functional part that handles an error correction code (FEC) so that a plurality of pieces of information can be multiplexed. 146a, an information element length 146b indicating the length of the information element, a payload identifier 146c indicating the type of packet loss compensation information as an error correction code function unit, and packet loss compensation information 147 and 148. The two pieces of packet loss compensation information (packet loss compensation information 1 and 2) are stored, for example, in the situation of FIG. 13 of the second embodiment (a situation where error correction codes of two groups are multiplexed). In view of this, normally only one is stored.

  FIG. 14C shows an example of the internal structure of the packet loss compensation information 147 and 148. As shown in FIG. 14 (c), the packet loss compensation information 147 includes an interleave group identifier 147a, a sub-block length 147b indicating the length of the error correction code sub-block, and an error correction code 147c. The The packet loss compensation information 148 has a similar structure.

  FIG. 15 is an example of a form of distribution of video and audio to a plurality of terminals simultaneously. In FIG. 15, the transmission side device 151 is connected to the public network 155 and transmits a transmission packet sequence 158. The receiving side devices 152, 153, and 154 receive the transmission packet sequence 158 sent via the public network 155, and reproduce the original video and audio.

  In FIG. 15, the transmission side device 151, the reception side device 152, and the reception side device 154 are devices that support loss packet compensation by error correction, and the reception device 153 is a device that does not support loss packet compensation by error correction. . The receiving side devices 152 and 153 are directly connected to the public network 155, and the receiving side device 154 is connected to a local network 156 having a management system different from that of the public network 155. The public network 155 and the local network 156 are connected via, for example, a NAPT device 157 (NAPT router). Most of the general corporate network and home network have the same management / operation form as the local network 156 in FIG.

  In FIG. 15, the packet sequence 158 is duplicated at each point where the route branches depending on the destination in the public network 155, and packet sequences 158a, 158b, 158c are generated. As described above, the multicast distribution function of the public network 155 (IP multicast network) is used to save resources of the transmission side device 151 and the public network 155.

  In FIG. 15, when the conventional technique shown in Patent Document 1 and Non-Patent Document 1 is used as a lost packet compensation function by error correction, the data packet and the error correction code packet constitute separate packet sequences. It becomes easier to consume resources in the network. Furthermore, the standard or the like does not guarantee the relationship between two packet sequences each composed of a data packet and an error correction code packet. For this reason, if the packet sequence 158c becomes the packet sequence 159 in which the packet identification information (UDP port number, etc.) is converted through the NAPT device 157, the receiving side device 154 cannot understand the relationship. Therefore, it becomes difficult to regenerate a packet lost by chance in the public network 155 using an error correction code. Therefore, in this case, a special NAPT device (see, for example, JP 2009-188688 A) is required.

  Further, when a conventional technique (for example, Japanese Patent Laid-Open No. 2005-210219) is used as a lost packet compensation function by error correction, the error correction code is multiplexed together with user information in the packet payload. For this reason, the packet sequence 158 shown in FIG. 15 becomes one type, and distribution to the receiving side device 154 via the NAPT device 157 is performed without any problem. However, if the packet delivery destination is a receiving terminal (receiving side device 153 in FIG. 15) that does not support loss packet compensation using an error correction code, the receiving side terminal extracts user information from the received transmission packet in the first place. Therefore, the presence of an incompatible receiving terminal (receiving device 153) is not allowed.

  On the other hand, in the present embodiment, as shown in FIG. 14, the error correction code packet (147c) is multiplexed into the RTP extension header. Thus, in FIG. 15, by using the transmission side device 151 and the reception side devices 152 and 154 (devices corresponding to loss packet compensation by error correction codes), the transmission packet sequence 158 becomes one type and is delivered through the NAPT device 157. The receiving side device 154 can correctly separate the error correction code. The receiving side device 153 that does not support the RTP extension header or does not support the error correction code multiplexing function of the RTP extension header performs an operation of ignoring the packet loss compensation information by the error correction code. For this reason, the user information in the payload of the RTP packet is correctly recognized, and it becomes possible to reproduce moving images, sounds, and the like.

  As described above, according to the present embodiment, information (in this case, an error correction code) for compensating for a lost packet by an error correction code is multiplexed and transmitted in an RTP extension header. As a result, while using a multicast distribution function to save resources in the network, a mixture of receiving terminals that do not support the lost packet compensation function and a receiving terminal according to the present invention are accommodated in a local network via a NAPT device. be able to.

  The embodiments of the present invention have been described above.

  INDUSTRIAL APPLICABILITY The present invention has an effect that high-quality transmission can be performed even for an application that requires real-time performance, and is useful as a data communication system that transmits moving images and audio in real time via a data communication network. .

10, 60 Video transmission side transmission apparatus 101, 601 Video encoding unit 102, 602 Packet generation unit 103 Redundant code multiplexing unit 104, 604 Packet transmission control unit 105 Sub-block length control unit 106, 118 User information division unit 107, 119 Sub Block allocation unit 108,605 Error correction code generation unit 11,61 Video reception side transmission device 111,611 Packet reception control unit 112,613 Received packet storage unit 113,614 Packet alignment unit 114,615 Video code decoding unit 115,617 packet Missing detector 116 Redundant code separator 117 Subblock length detector 120, 618 Error correction loss packet regenerator 603 Packet multiplexer 606 Error correction code packet generator 612 Packet demultiplexer 616 Error correction code packet accumulator

Claims (6)

A data communication system that performs error correction of data transmitted from a transmission device to a reception device via a communication path using a block code,
The transmitter is
Packet generation means for generating a plurality of packets by dividing data to be transmitted;
First sub-block dividing means for dividing the packet generated by the packet generating means into sub-blocks of a predetermined length;
First sub-block allocating means for allocating the sub-block divided by the first sub-block dividing means to a block which is an error correction calculation unit;
Error correction code generation means for generating an error correction code by performing error correction on the sub-block assigned to the block by the first sub-block assignment means;
Error correction code multiplexing means for multiplexing the error correction code generated by the error correction code generation means into the packet;
Packet transmitting means for transmitting the packet in which the error correcting code is multiplexed by the error correcting code multiplexing means to the receiving device;
With
The receiving device is:
A packet receiving means for receiving a packet transmitted from the transmitting device;
Packet storing means for temporarily storing the packet received by the packet receiving means;
Error correction code separating means for separating the error correction code multiplexed in the packet stored by the packet storage means from the packet;
Second sub-block dividing means for dividing the packet stored by the packet storage means into sub-blocks of a predetermined length;
Second sub-block allocating means for allocating the sub-blocks divided by the second sub-block dividing means to blocks serving as error correction calculation units;
Based on the received packet, packet loss detection means for detecting a loss of a packet transmitted from the transmission device on the communication path;
When packet loss is detected by the packet loss detection means, each subblock forming the lost packet is a specific block required for restoration of each subblock, and the subblock is assigned to the subblock. Using the error correction code belonging to the specific block and the sub-block allocated to the specific block by the second sub-block allocation unit, the lost packet recovery unit for recovering the lost packet;
Comprising
Data communication system. In the transmitter,
Sub-block division length control means for determining a division length of the sub-block based on at least one of the communication quality of the communication path, the transmission data amount, and the traffic amount that is the transmission data amount per unit time,
The first sub-block dividing unit divides the packet by the sub-block division length determined by the sub-block division length control unit;
The first sub-block allocating unit allocates each sub-block to a block serving as an error correction calculation unit with the sub-block division length determined by the sub-block division length control unit,
The error correction code multiplexing means multiplexes information on the subblock division length determined by the subblock division length control means together with an error correction code into a packet,
In the receiving device,
The error correction code separation means separates the error correction code multiplexed in the received packet from the packet and separates the information from the packet,
Sub-block length detection means for detecting a division length of the sub-block based on the information separated by the error correction code separation means,
The second sub-block dividing unit divides the packet accumulated in the packet accumulating unit by the sub-block division length detected by the sub-block length detecting unit,
The second sub-block allocating means allocates each sub-block to a block that is an arithmetic unit for error correction with the division length of the sub-block.
The data communication system according to claim 1. In the transmitter,
The error correction code generation means includes
Corresponding to the packet generated by the packet generating means, a memory for sequentially storing intermediate values of sequential calculation processes of error correction codes;
A read pointer to the memory;
A write pointer having an offset corresponding to the division length of the sub-block from the read pointer,
The first sub-block allocating means includes
By sequentially moving the read pointer and the write pointer for each packet, allocation to a block that is an error correction calculation unit for each divided sub-block is performed,
The error correction code generation means calculates an intermediate value in an error correction calculation process from a sub-block indicated by the write pointer and a sub-block corresponding to a packet subsequent to the packet, and the intermediate value is stored in the memory. When the calculation for all the sub-blocks belonging to the block assigned by the first sub-block allocating means is completed, the sub-block indicated by the read pointer is corrected from the memory. Read and output as a code,
The data communication system according to claim 1 or 2. The transmitter transmits video or audio streaming data to the receiver using RTP (Real-time Transport Protocol),
The error correction code multiplexing means multiplexes the error correction code in an RTP extension header;
4. A data communication system according to claim 1.   5. A transmission apparatus used in the data communication system according to claim 1.   5. A receiving apparatus used in the data communication system according to claim 1.

Images