JP2011211616A - Moving picture transmission apparatus, moving picture transmission system, moving picture transmission method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture transmission apparatus, a moving picture transmission system, a moving picture transmission method, and a program, for transmitting a moving picture in which all data corresponding to one picture is required for display with little delay while suppressing the consumption of communication resources.SOLUTION: Data for one picture are captured as the block data of a line block, and whether or not to utilize an FEC coding scheme as an error correction scheme of the respective block data is dynamically determined according to the situation of a transmission line NW and the point of time of the capture of the respective block data, FEC redundant coding of respective packets is performed for the packets utilizing the FEC coding scheme, and the respective packets are transmitted to a reception apparatus 20 through the transmission line without performing the FEC redundant coding for the packets not utilizing the FEC coding scheme. Also, when packet loss occurs, a lost packet is retransmitted to the reception apparatus by an ARQ scheme, and a series of processes including the capture and transmission of the respective block data is processed in parallel while shifting only the time required for the capture for the plurality of line blocks.

Description

  The present invention relates to a moving image transmission apparatus, a moving image transmission system, a moving image transmission method, and a program.

  In recent years, it has been required to transmit data such as moving images with low delay through a transmission path such as the Internet. For example, when transmitting a moving image in real time, it is required to transmit moving image data with a delay equal to or less than the frame interval of the moving image.

  For this reason, the following Patent Document 1 discloses a technique (line block unit method) in which data for one picture is divided into line blocks including a predetermined number of pixel lines and compressed as block data.

  According to this technique, the transmission side starts subsequent processing such as forward error correction (FEC) coding for each block data without completing the capture of all data for one picture. Can do. Similarly, the receiving side can start subsequent processing such as FEC decoding for each block data without completing reception of all data for one picture. Therefore, if the propagation delay of the transmission path is sufficiently small, the moving image can be transmitted with a delay equal to or shorter than the frame interval.

  However, some display devices that display moving images may require all data for one picture for display. In this case, even if the subsequent processing can be started for each block data, the moving image cannot be displayed until the subsequent processing of all block data for one picture is completed. For this reason, the display time point of the moving image depends on the time point when the subsequent processing of the block data captured last among the block data for one picture is completed.

  By the way, in the transmission of moving images, the FEC code method is frequently used as an error correction method at the time of transmission. In the FEC encoding method, on the transmission side, original data is redundantly encoded, and a plurality of packets including redundant packets are transmitted through a transmission path. On the receiving side, the original data can be decoded by receiving a predetermined number of packets among the plurality of packets.

JP 2007-311924 A

  However, in moving picture transmission using the FEC coding scheme, a lot of communication resources such as the bandwidth of the transmission path and the processing time required for FEC processing (encoding / decoding) are consumed to transmit redundant packets. Become.

  Accordingly, the present invention provides a moving image transmission apparatus, a moving image transmission system, and a moving image that can transmit a moving image that requires all data for one picture for display with low delay while suppressing consumption of communication resources. A transmission method and program are to be provided.

  According to an aspect of the present invention, for a moving image that requires all data for one picture for display, the data for one picture is divided into line blocks including a predetermined number of pixel lines and captured as block data. Whether to use a forward error correction (FEC) coding method as an error correction method for each block data packet is dynamically determined according to the capture unit, the condition of the transmission path, and the capture time of each block data. FEC usage determination unit, and a device that receives FEC redundancy-encoded packets for packets that use the FEC encoding scheme, and that does not use FEC redundancy encoding for packets that do not use the FEC encoding scheme, via a transmission path Packet loss occurs for packets that are transmitted to and that do not use the FEC encoding method And a transmission unit that retransmits the lost packet to the reception device by an automatic retransmission (ARQ) method, and the capture unit and the transmission unit perform a series of processes including capture and transmission of each block data for a plurality of block data. In addition, a moving image transmission apparatus that performs parallel processing while shifting the time required for capture is provided.

  The FEC usage determining unit performs predetermined processing after error correction by retransmitting lost packets within a margin time from a scheduled completion time point where the processing necessary to display each block data is expected to be completed to a moving image display time point. When the lost packet cannot be recovered while achieving the target loss rate, the use of the FEC coding method may be determined.

  The FEC usage determining unit may determine the usage of the FEC coding method in consideration of the round-trip transmission time of the transmission path and the packet loss rate.

  The FEC usage determining unit may determine the redundancy of the FEC encoding so that a predetermined target loss rate can be achieved when the FEC encoding method is used.

  The FEC usage determining unit has a short margin time from the completion of the processing necessary for displaying the block data with the late capture start order to the display time of the moving image. The use of a coding scheme may be determined.

  According to another aspect of the present invention, there is provided a moving image transmission system for transmitting a moving image from the moving image transmission apparatus to the receiving apparatus through a transmission path.

  According to another aspect of the present invention, block data is obtained by dividing data for one picture into line blocks each including a predetermined number of pixel lines for a moving image that requires all data for one picture for display. And whether to use a forward error correction (FEC) coding method as an error correction method for each block data packet, depending on the state of the transmission path and the capture time of each block data. For the packet that uses the FEC encoding method, each packet is FEC redundantly encoded, and for the packet that does not use the FEC encoding method, each packet is transmitted to the receiving apparatus through the transmission line without FEC redundant encoding. Steps and packets that do not use the FEC encoding method are lost when packet loss occurs A step of retransmitting a packet to a receiving apparatus by an automatic retransmission (ARQ) method, and a step of parallelly processing a plurality of block data including a capture step and a transmission step of each block data by shifting the capture time. Is provided.

  Moreover, according to another viewpoint of this invention, the program for making a computer perform the said moving image transmission method is provided. Here, the program may be provided using a computer-readable recording medium or may be provided via communication means.

  As described above, according to the present invention, a moving image transmission apparatus and a moving image that can transmit a moving image that requires all data for one picture for display with low delay while suppressing consumption of communication resources. A transmission system, a moving image transmission method, and a program can be provided.

1 is a diagram illustrating an overall configuration of a transmission system according to an embodiment of the present invention. It is a figure which compares and shows the moving image transmission process of a picture unit system and a line block unit system. It is a figure (1/2) which shows the processing condition of the moving image transmission in a line block unit system. It is a figure (2/2) which shows the processing condition of the moving image transmission by a line block unit system. It is a block diagram which shows the main function structures of a transmitter and a receiver. It is a figure for demonstrating the determination conditions for determining utilization of a FEC code system. It is a flowchart (1/2) which shows the operation | movement procedure of a moving image transmission system. It is a flowchart (2/2) which shows the operation | movement procedure of a moving image transmission system.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

In the present specification, preferred embodiments of the present invention will be described along the following items.
1. 1. Overall configuration of moving image transmission system 2. Line block unit type moving image transmission processing 3. Configuration of transmitting device and receiving device 4. Use determination of FEC coding method 5. Operation of moving image transmission system Summary

[1. Overall configuration of moving image transmission system]
First, an overall configuration of a moving image transmission system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 shows the overall configuration of a moving image transmission system.

  As shown in FIG. 1, the moving image transmission system includes a transmitting device 10 and a receiving device 20 capable of transmitting moving image data through a transmission line NW such as the Internet. The moving image data is transmitted by a stream transmission method such as RTP (Realtime Transport Protocol) defined in IETF RFC3550.

  The transmission device 10 is connected to a moving image input device such as a video camera V, and the receiving device 20 is connected to a moving image output device such as a display D. The transmitting device 10 may be configured integrally with the video camera V, and the receiving device 20 may be configured integrally with the display D.

  The transmission apparatus 10 captures (captures) the moving image signal input from the video camera V as data (a), compresses and encodes (b), packetizes (c), and forward error correction (FEC) as necessary. It is encoded (f) and transmitted to the receiving device 20 through the transmission line NW (e).

  On the other hand, the receiving device 20 receives data from the transmitting device 10 through the transmission line NW (e ′), performs FEC decoding (f), depacketizes (g), and compresses and decodes (h) as necessary. When the receiving device 20 completes the processing (e ′) to (h) for all the data for one picture, the receiving device 20 performs display processing and outputs the data to the display D as data for one picture. indicate.

[2. Line block unit video transmission processing]
Next, with reference to FIG. 2 and FIG. 3, the moving image transmission process of the line block unit method will be described.

  FIG. 2 shows a comparison between moving picture transmission processing of the picture unit method and the line block unit method. In the following description, it is assumed that data for one picture is divided into line blocks with line block numbers k = 1 to N.

  As shown in FIG. 2, in the picture unit system, capture (A), compression encoding (B), packetizing (C), FEC encoding (D), transmission (E), FEC decoding (F), depacketizing ( Each process (A) to (H) of G) and compression decoding (H) is executed in units of pictures. For this reason, the transmission side cannot start the subsequent processes (B) to (E) until the data capture (A) for one picture is completed, and the transmission side transmits data for one picture. Subsequent processes (F) to (H) cannot be started until (E) is completed.

  On the other hand, in the line block unit method, capture (a), compression coding (b), packetization (c), FEC coding (d), transmission (e), FEC decoding (f), depacketization (g), and compression Each process (a) to (h) of decoding (h) is executed in units of line blocks. That is, even if the data capture (A) for one picture is not completed on the transmission side, if the data capture (a) for each line block is completed, the subsequent processes (b) to (e) are sequentially performed. If the transmission (e) of the data in units of line blocks is completed, the subsequent processes (f) to (h) can be started in sequence.

  On the transmission side, for example, when the block data capture (a) of the line block number k = 1 is completed, the subsequent processes (b) to (e) are sequentially started and the block data capture of k = 2 ( Each processing (a) to (e) is performed in parallel in units of line blocks, such as starting a). Note that the processing (f) to (h) is performed in parallel on the reception side in units of line blocks as in the transmission side.

  For this reason, in the picture unit method, the delay caused by at least one of the processes (A) to (H) is stored as it is as a process delay for one picture, and the process from capture (A) to display is performed. Overall delay increases. On the other hand, in the line block unit method, a delay caused by at least one of the processes (a) to (h) is absorbed between the processes of the line blocks and stored as it is as a delay of one picture. Therefore, the delay of the entire process from the first capture start (a) to the display is suppressed.

  3A and 3B show the processing status of moving image transmission in the line block unit method. 3A and 3B show the processes (a) to (h) executed in units of line blocks.

  In FIG. 3A, the horizontal axis indicates the capture start time and the moving image display time for a moving picture of one picture. Also, processing blocks in line block units are shown in parallel in the vertical axis direction according to the processing start order.

As shown in FIG. 3A, each processing block is processed in parallel with an offset T _capture corresponding to the processing time required for capture (a). The time required for each processing (a) to (d), (f) to (h) except for the transmission (e) is almost the same between the processing blocks, but the processing time required for the transmission (e) is the transmission time. It fluctuates according to the situation of road NW.

Since each processing block is started with an offset T _capture , a processing block with a fast capture start order has a sufficient margin _Tremain from the completion of processing (a) to (h) to the display time point. In a processing block whose capture start order is late, there is almost no margin time _Tremain .

  For this reason, as shown in FIG. 3B, in a processing block with a fast capture start order, even if a packet loss of block data occurs during transmission, it is relatively easy to recover lost packets by the time of display by automatic packet retransmission. Easy. On the other hand, in a processing block whose capture start order is late, it is difficult to recover lost packets by the time of display by automatic packet retransmission. Therefore, in the moving image transmission process of the line block unit method, a processing block with a late capture start order, particularly the last processing block, becomes critical with respect to transmission delay.

  On the other hand, in the FEC process (encoding / decoding), the block data packet is redundantly encoded and the redundant packet is transmitted. Therefore, the bandwidth of the transmission line NW and the processing time required for the FEC process (encoding / decoding), etc. Many communication resources are consumed.

Therefore, in the present embodiment, either an FEC code method or an automatic repeat (ARQ) method is used as an error correction method at the time of transmission according to the capture time point and the state of the transmission line NW. Although details will be described later, the ARQ scheme is used for block data that can be recovered by retransmission, and the FEC coding scheme is used for block data that cannot be recovered by retransmission. The ARQ scheme is used when the target loss rate P _arg after error correction can be achieved according to the state of the transmission line NW.

  As a result, block data with an earlier capture start order is recovered by resending lost packets, and block data with a later capture start order is recovered by receiving a predetermined number of packets that have been redundantly encoded and transmitted. Is done. Therefore, by using the FEC coding method only for block data that is critical with respect to transmission delay, a moving image that requires all data for one picture for display is transmitted with low delay while suppressing the consumption of communication resources. can do.

[3. Configuration of transmitter and receiver]
Next, with reference to FIG. 4, the configuration of the transmission device 10 and the reception device 20 configuring the moving image transmission system will be described. In FIG. 4, the transmission path of block data is indicated by solid arrows, and the transmission path of control commands and the like is indicated by broken arrows.

  FIG. 4 shows main functional configurations of the transmission device 10 and the reception device 20. As illustrated in FIG. 4, the transmission device 10 includes an input interface (IF) 11, a capture unit 12, a compression encoding unit 13, a packetizing unit 14, an FEC encoding unit 15, an RTP transmission unit 16, and an RTCP (RTP Control Protocol). A unit 17, an ARQ control unit 18, and an FEC usage determination unit 19 are configured.

  The input IF 11 inputs a moving image signal supplied from the video camera V as data. The capture unit 12 captures data supplied from the input IF 11 as block data in units of line blocks. In addition, the capture unit 12 notifies the FEC usage determination unit 19 of the line block number k of the captured line block. The line block number k is assigned as k = 1,..., N to the first to Nth line blocks at the capture time point for the data for one picture.

  The compression encoding unit 13 compresses and encodes the block data supplied from the capture unit 12. The packetizing unit 14 packetizes the compressed data supplied from the compression encoding unit 13 into RTP packets.

  The FEC encoding unit 15 is supplied with the packet from the packetizing unit 14 and performs FEC encoding according to the FEC instruction from the FEC usage determining unit 19. In FEC encoding, a packet (original data) is redundantly encoded with an erasure error correction code such as a Reed-Solomon code.

  The RTP transmission unit 16 transmits a packet that has not been FEC encoded or a packet that has been FEC encoded to the reception device 20 through the transmission line NW. In addition, in response to the ARQ instruction from the ARQ control unit 18, the RTP transmission unit 16 retransmits, to the reception device 20, packets that have been lost due to packet loss at the time of transmission, among the packets that are not FEC encoded. Note that packets that are not FEC encoded are stored in a buffer (not shown) until reception is confirmed.

The RTCP unit 17 controls a session for transmitting and receiving data to and from the receiving device 20 by RTP. The RTCP unit 17 transmits / receives, for example, an RTCP transmission report (SR) packet and an RTCP reception report (RR) packet defined in IETF RFC3550 to / from the RTCP unit 29 of the receiving device 20, so that the status of the transmission line NW Is obtained and notified to the FEC usage determining unit 19. As the transmission path information, various parameters such as a round-trip transmission time (RTT) and an average packet loss rate _Pl are used.

  In addition, when the RTCP unit 17 receives a retransmission request (NACK) packet from the RTCP unit 29 of the receiving device 20, the RTCP unit 17 notifies the ARQ control unit 18 as an ARQ instruction. The ARQ control unit 18 controls retransmission of lost packets by the RTP transmission unit 16 in response to an ARQ instruction from the RTCP unit 17.

  Based on the transmission path information and the line block number k, the FEC usage determining unit 19 dynamically determines whether to use the FEC coding scheme as an error correction scheme at the time of transmission. The FEC usage determining unit 19 notifies the FEC encoding unit 15 of an FEC instruction when using the FEC encoding method. Note that the determination conditions for determining the use of the FEC encoding method will be described later.

  Further, the FEC usage determining unit 19 may change the redundancy of FEC encoding based on the transmission path information, the amount of ARQ retransmission packet, the transmission rate of block data, and the like. In this case, for example, the target loss rate after error correction can be achieved by reducing the redundancy when the variation in RTT is small and increasing the redundancy when the variation in RTT is large.

  Thereby, in the moving image transmission system, the lost data is recovered by the FEC encoding method for the block data to be used in the FEC encoding method, and the lost packet is recovered by the ARQ method for the block data that is not to be used. .

  As described with reference to FIGS. 3A and 3B, in the transmission apparatus 10, each process of capture (a), compression encoding (b), packetizing (c), FEC encoding (d), and RTP transmission (e) is performed. A plurality of block data are processed in parallel.

  The functional configuration of the transmission device 10 may be realized as hardware, or at least partially realized as software. In the latter case, a control unit (not shown) provided in the transmission apparatus 10 executes a program for executing the moving image transmission method.

  The receiving device 20 includes an RTP receiving unit 21, an FEC decoding unit 22, a depacketizing unit 23, a compression decoding unit 24, a display processing unit 25, an output IF 26, a loss detection unit 27, an ARQ control unit 28, and an RTCP unit 29. The

  The RTP receiver 21 receives a packet from the transmission device 10 through the transmission line NW. The RTP receiver 21 receives a packet that has not been FEC encoded or a packet that has been FEC encoded. The FEC decoding unit 22 is supplied with the FEC-encoded packet from the RTP receiving unit 21 and performs FEC decoding.

  The depacketizing unit 23 depacketizes the packet supplied from the RTP receiving unit 21 into compressed data. Note that the FEC-encoded packet is supplied to the depacketizing unit 23 after FEC decoding, and the non-FEC-encoded packet is supplied without FEC decoding.

  The compression decoding unit 24 is supplied with the compressed data from the depacketizing unit 23, decodes the compressed data into block data, and stores it in a buffer (not shown). The display processing unit 25 is supplied with block data for one picture from the compression decoding unit 24, and performs display processing to generate a moving image signal. The output IF 26 is supplied with the moving image signal from the display processing unit 25 and outputs it to the display D.

  The loss detection unit 27 detects a packet loss based on the reception information supplied from the RTP reception unit 21 and notifies the ARQ control unit 28 of the detection result. For example, the packet loss is detected when the sequence number described in the packet header is inspected and the sequence number of the received packet has discontinuity.

  Here, detection of packet loss is performed only for packets that are not FEC encoded, and is not performed for packets that are FEC encoded. This is because an FEC-encoded packet recovers a lost packet not by the ARQ scheme but by the FEC encoding scheme.

  Based on the detection result supplied from the loss detection unit 27, the ARQ control unit 28 adds retransmission request (NACK) packet information for specifying a lost packet to the NACK list. The ARQ control unit 28 reads the NACK packet information from the NACK list at a predetermined time, and transmits the NACK packet to the transmission device 10 through the RTCP unit 29.

  The NACK packet conforms to a format described in, for example, IETF Internet Draft “Extended RTP Profile for RTCP-based Feedback”.

  In the NACK list, time point information of “NACK timeout” and “NACK deadline” is set for each NACK packet. “NACK timeout” is set, for example, as the time when RTT elapses from transmission of a NACK packet. “NACKdeadline” is set as a time point before the RTT rather than a scheduled arrival time of the packet.

  When detecting a packet loss, the ARQ control unit 28 transmits a NACK packet to the transmission device 10 through the RTCP unit 29. Then, when the retransmission packet is not received even when the time point of “NACK timeout” arrives, the NACK packet is retransmitted. The NACK packet is retransmitted repeatedly until the time point of “NACK deadline” elapses.

  As described with reference to FIGS. 3A and 3B, in the receiving device 20, RTP reception (e ′), FEC decoding (f), depacketization (g), and compression decoding (h) are performed in parallel for a plurality of block data. It is processed.

  Further, the functional configuration of the receiving device 20 may be realized as hardware, or at least partially realized as software. In the latter case, a control unit (not shown) provided in the receiving device 20 executes a program for executing the moving image transmission method.

[4. Use determination of FEC coding method]
FIG. 5 schematically illustrates determination conditions for determining the use of the FEC code method as an error correction method.

First, a target loss rate _Parq after error correction is set. The target loss rate P _arq is set as a value such as P _arq = 10 ⁻⁶ in moving image transmission for IPTV, for example. Next, assuming that error correction is performed by the ARQ method, the number of packet retransmissions N _net required to achieve the target loss rate P _arq is estimated. The number of retransmissions N _net is estimated by Equation 1 in consideration of the average packet loss rate P ₁ obtained as transmission path information. Then, a retransmission time t _arq required to retransmit the packet over the number of retransmissions N _net is calculated. The retransmission time t _arq is calculated by Equation 2 based on the RTT obtained as transmission path information.

  In addition, Formula 1 is "examination of" Intelligent QoS "control system for real-time communication system" by Kuri et al., IEICE technical report, vol. 107, no. 19, CQ2007-1, pp. 1-6, April 2007 ”.

Where P _l : average packet loss rate (random packet loss rate)
P _arg : Target loss rate after error correction (set value)
N _net : Number of packet retransmissions required to achieve the target loss rate
t _arg : Retransmission time required to retransmit a packet over the number of retransmissions
RTT: Round trip transmission time

On the other hand, the total processing time T _total required for the entire processing of the block data is calculated by Equation 3. The total processing time T _total is the processing time T _process and transmission (e) required for capture (a), compression processing (b), (h), packet processing (c), and (g) performed on the transmission side and reception side. It is calculated as the sum of the transmission time T _trans required for. Note that the processing time required for the processing (d) and (f) of the FEC processing (encoding / decoding) is not included in the processing time T _process because it is assumed that error correction is performed by the ARQ method.

Here, the processing time T _process is set in advance according to the moving image to be transmitted, and the transmission time T _trans is set as ½ of RTT. Therefore, the total processing time _Ttotal varies depending on the state of the transmission line NW.

Next, the margin time _Tremain from the completion time point of all the block data processing (processing (a) to (c), (e), (g), (h)) to the display time point is calculated by Equation 4. The margin time _Tremain is calculated as a difference between the usable time _Tdelay from the start of the capture of data for one picture to the display time and the end time of all the processes for each block data. Therefore, the margin time _Tremain is longer as the block data is earlier in the capture start order and shorter as the block data is later.

Here, T _process : block data processing time (processing (a) to (c), (g), (h))
T _trans : Block data transmission time (process (e) (= RTT / 2))
T _total : Total processing time of block data
_Tremain : time margin from completion of all block data processing to display
T _delay : Available time from the start of capturing data for one picture to the point of display
T _capture : capture time of block data
k: line block number k (k = 1, 2,..., N)

  Then, it is determined whether each block data satisfies the determination condition according to Expression 5 using Expression 2 and Expression 4.

Here, when the determination condition is satisfied, since the lost packet cannot be recovered while achieving the target loss rate P _arq by the ARQ method, the use of the FEC code method is determined. On the other hand, if the determination condition is not satisfied, the lost packet can be recovered while achieving the target loss rate P _arq by the ARQ method, so the ARQ method is used without using the FEC code method.

[5. Operation of moving image transmission system]
Next, the operation of the moving image transmission system will be described with reference to FIGS. 6A and 6B. FIGS. 6A and 6B respectively show operation procedures of the transmission device 10 and the reception device 20 from the start of capture to the display of a moving image for one picture data.

  In FIGS. 6A and 6B, the processing is shown to be completed in units of line blocks. However, in practice, as described above, when the capture (a) of the line block is completed, the compression encoding (b) of the line block is started and the capture (a) of another line block is started. As described above, processing of a plurality of line blocks is executed in parallel.

  First, the operation of the transmission device 10 will be described. As shown in FIG. 6A, in the transmission apparatus 10, the capture unit 12 captures data in units of line blocks as block data (step S101). In the process of step S101, each time the process is executed, block data of line block numbers k = 1 to N are captured sequentially.

  The compression encoding unit 13 compresses and encodes the block data, and the packetizing unit 14 packetizes the compressed data into RTP packets (S103).

Based on the line block number k from the capture unit 12, the FEC usage determining unit 19 determines whether to use the FEC code method as an error correction method at the time of transmission according to the above-described determination condition (S105). Specifically, FEC utilization determination unit 19, based on the transmission path information and the line block number k, and calculates a retransmission time _{t arq} and margin time _{T Remain,} determines whether the determination condition is satisfied in Equation 5. When the determination condition is satisfied, the use of the FEC code method is determined as the error correction method, and the FEC instruction is given to the FEC encoder 15, and when the determination condition is not satisfied, the ARQ method is used.

  The FEC encoding unit 15 determines whether an FEC instruction has been issued (S107). When the FEC instruction is given, the packet is FEC encoded (S109), and transmitted to the receiving device 20 as an FEC encoded packet (S111). On the other hand, when the FEC instruction is not performed, the packet is not FEC encoded, and is transmitted to the receiving device 20 as a packet not FEC encoded (S111).

  On the other hand, when the RTCP unit 17 receives the NACK packet from the RTCP unit 29 of the receiving device 20, the RTCP unit 17 notifies the ARQ control unit 18 of an ARQ instruction. The ARQ control unit 18 determines whether an ARQ instruction has been issued (S113). When the ARQ instruction is given, the ARQ control unit 18 controls retransmission of the lost packet, and the lost packet is retransmitted to the receiving device 20 (S115). Note that the processes of steps S113 and S115 may be performed in parallel with the processes of steps S101 to S109 instead of being performed as a subsequent process of the process of step S111.

  Further, the capture unit 12 determines whether the capture of data for one picture has been completed based on the line block number k, that is, whether the block data of the line block number k = N has been captured (S117). When the capture is completed, the determination of the ARQ instruction and the retransmission of the packet are repeated until the transmission of the packet for one picture is completed (S119 to S123).

  On the other hand, if the capture has not been completed, the capture unit 12 increments the line block number k by 1 and captures the data of the next line block as block data (S101).

  Next, the operation of the receiving device 20 will be described. As shown in FIG. 6B, in the receiving device 20, the RTP receiving unit 21 receives a packet from the transmitting device 10 through the transmission line NW (S151). The packet is received as an FEC encoded packet or a packet that is not FEC encoded.

  The RTP reception unit 21 determines whether the packet is FEC encoded (S153). When the FEC encoding is performed, the FEC decoding unit 22 performs FEC decoding on the packet (S155). Here, when a packet loss occurs, the loss is recovered by the redundant packet. The depacketizing unit 23 depackets the packet, and the compression decoding unit 24 compresses and decodes the compressed data (S157).

  On the other hand, when the FEC encoding is not performed, the loss detection unit 27 detects whether a packet loss has occurred in the received packet (S159). The loss detection unit 27 notifies the ARQ control unit 28 of the detection result. If a loss has occurred, the ARQ control unit 28 transmits a NACK packet to the transmission device 10 through the RTCP unit 29 (S161). On the other hand, if no loss has occurred, packet depacketization and compression decoding of the compressed data are performed (S157).

  When the process of step S157 or S161 is completed, it is determined whether the process for one picture is completed (S163). When the process is completed, a display process (S165) is performed. When the process is not completed, a packet of the next line block is received (S151).

[6. Summary]
As described above, the moving image transmission system according to the embodiment of the present invention transmits a moving image that requires all data for one picture for display. In the system, data for one picture is captured as block data in units of line blocks. Also, depending on the state of the transmission line NW and the capture time of each block data, either the FEC code method or the ARQ method is used as an error correction method when transmitting each block data packet. Here, the ARQ method is used for block data that can be recovered by retransmission, and the FEC code method is used for block data that cannot be recovered by retransmission.

  As a result, block data with an earlier capture start order is recovered by resending lost packets, and block data with a later capture start order is recovered by receiving a predetermined number of packets that have been redundantly encoded and transmitted. Is done. Therefore, by using the FEC coding method only for block data that is critical with respect to transmission delay, a moving image that requires all data for one picture for display is transmitted with low delay while suppressing the consumption of communication resources. can do.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

For example, in the above description, a case where parallel processing is performed in units of line blocks has been described. However, a processing unit composed of a predetermined number of block data may be pipelined and processed in parallel. In this case, the use of the FEC coding method calculates the number of retransmissions N _net , the retransmission time t _arq , the total processing time T _total , the capture time T _capture and the margin time T _remain for a predetermined number of block data packets. It is determined based on the determination condition.

10 Transmitter 11 Input IF
DESCRIPTION OF SYMBOLS 12 Capture part 13 Compression encoding part 14 Packetization part 15 FEC encoding part 16 RTP transmission part 17 RTCP part 18 ARQ control part 19 FEC utilization determination part 20 Receiving device 21 RTP receiving part 22 FEC decoding part 23 Depacketizing part 24 Compression decoding part 25 Display processing unit 26 Output IF
27 Loss detection unit 28 ARQ control unit 29 RTCP unit NW transmission path V Video camera D Display

Claims (8)

For a moving image that requires all data for one picture for display, a capture unit that divides the data for one picture into line blocks composed of a predetermined number of pixel lines and captures the data as block data;
FEC usage decision that dynamically determines whether or not to use the forward error correction (FEC) coding method as the error correction method of the packet of each block data according to the condition of the transmission path and the capture time of each block data And
For each packet that uses the FEC encoding method, each packet that has been FEC redundantly encoded is transmitted, and for each packet that does not use the FEC encoding method, each packet that has not been FEC redundantly encoded is transmitted to the receiving device through the transmission path. And a transmission unit that retransmits a lost packet to the receiving device by an automatic retransmission (ARQ) method when a packet loss occurs with respect to a packet that does not use the FEC coding method,
The moving image transmission apparatus, wherein the capture unit and the transmission unit perform a parallel processing on a plurality of block data by shifting a series of processes including capture and transmission of each block data by a capture required time.   The FEC usage determining unit performs error correction by retransmitting the lost packet within a margin time from a scheduled completion time point where the processing necessary for displaying each block data is expected to be completed to a moving image display time point. The moving picture transmission apparatus according to claim 1, wherein when the lost packet cannot be recovered while achieving the predetermined target loss rate, the use of the FEC coding method is determined.   The moving image transmission apparatus according to claim 1, wherein the FEC usage determining unit determines the usage of the FEC code method in consideration of a round-trip transmission time and a packet loss rate of the transmission path.   The said FEC utilization determination part determines the redundancy of the said FEC encoding so that a predetermined target loss rate may be achieved, when using the said FEC encoding system, The any one of Claims 1-3 Video transmission device.   The FEC utilization determining unit has a short margin time from the completion time of processing necessary for displaying the block data whose capture start order is late to the display time of the moving image, and therefore the block data whose capture start order is late The moving image transmission apparatus according to claim 1, wherein use of the FEC encoding method is determined. A moving image transmission system for transmitting a moving image from a transmitting device to a receiving device through a transmission path,
The transmitter is
For a moving image that requires all data for one picture for display, a capture unit that divides the data of the one picture into line blocks including a predetermined number of pixel lines and captures the data as block data;
FEC usage that dynamically determines whether or not to use a forward error correction (FEC) coding scheme as an error correction scheme for each block data packet according to the condition of the transmission path and the capture time of each block data A decision unit;
For each packet that uses the FEC encoding method, each packet that has been FEC redundantly encoded is transmitted, and for each packet that does not use the FEC encoding method, each packet that has not been FEC redundantly encoded is transmitted to the receiving device through the transmission path. A transmission unit that retransmits a lost packet to the reception device by an automatic retransmission (ARQ) method when a packet loss occurs with respect to a packet that does not use the FEC coding method,
The capture unit and the transmission unit, for a plurality of block data, a series of processes including capture and transmission of each block data are shifted in parallel by a capture time,
The receiving device is:
For each packet that uses the FEC coding method, each packet that has been FEC redundantly coded, and for each packet that does not use the FEC coding method, each packet that has not been FEC redundantly coded is sent through the transmission path. When the packet loss occurs for a packet that is received from the FEC code and does not use the FEC coding method, a receiving unit that re-receives the lost packet from the transmission device by an automatic retransmission (ARQ) method and all of the one picture When the processing necessary for receiving and displaying the block data is completed, an output interface that outputs the block data to the display device as data for one picture is provided. For a moving image that requires all data for one picture for display, dividing the data for one picture into line blocks made up of a predetermined number of pixel lines and capturing it as block data;
Dynamically determining whether or not to use a forward error correction (FEC) coding method as an error correction method for each block data packet according to the condition of the transmission path and the capture time of each block data;
For each packet that uses the FEC encoding method, each packet is FEC redundantly encoded, and for each packet that does not use the FEC encoding method, each packet is not FEC redundantly encoded and transmitted to the receiving device through the transmission path; ,
Retransmitting lost packets to the receiving device in an automatic retransmission (ARQ) scheme when a packet loss occurs for a packet that does not use the FEC coding scheme;
A method of transmitting a plurality of block data, wherein a series of processing including a capture step and a transmission step of each block data is shifted in parallel by a capture time and parallel processing. A program for causing a computer to execute the moving image transmission method according to claim 7.

Images