JP2011211617A - Moving picture transmission apparatus, moving picture transmission system, moving picture transmission method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving picture transmission apparatus, a moving picture transmission system, a moving picture transmission method, and a program, capable of transmitting a moving picture in which all data corresponding to one picture is required for display, with little delay.SOLUTION: In a moving picture in which data corresponding to one picture is required for display, data corresponding to one picture is divided into line blocks including a predetermined number of pixel lines and captured as block data. A compression rate e of each block data is dynamically set according to a capture point in time of each block data. Each block data is compressed at the set compression rate and transmitted to a reception apparatus 20 through a transmission path. On a plurality of block data, a series of processes including the capture and the transmission of each block data is performed in parallel while shifting only a time Trequired for the capture. Block data that is late in start order of the capture, among the data corresponding to one picture, is set to be lower in compression rate than block data that is early in start order of the capture.

Description

  The present invention relates to a moving image transmission apparatus, a moving image transmission system, a moving image transmission method, and a program.

  In recent years, it has been required to transmit data such as moving images with low delay through a transmission path such as the Internet. For example, when transmitting a moving image in real time, it is required to transmit moving image data with a delay equal to or less than the frame interval of the moving image.

  For this reason, the following Patent Document 1 discloses a technique (line block unit method) in which data for one picture is divided into line blocks including a predetermined number of pixel lines and compressed as block data.

  According to this technique, the transmission side can start subsequent processing for each block data without completing the capture of all data for one picture. Similarly, the receiving side can start subsequent processing for each block data without completing the reception of all the data for one picture. Therefore, if the propagation delay of the transmission path is sufficiently small, the moving image can be transmitted with a delay equal to or shorter than the frame interval.

  However, some display devices that display moving images may require all data for one picture for display. In this case, even if the subsequent processing can be started for each block data, the moving image cannot be displayed until the subsequent processing of all block data for one picture is completed. For this reason, the display time point of the moving image depends on the time point when the subsequent processing of the block data captured last among the block data for one picture is completed.

  By the way, in moving picture transmission, block data is usually compressed and encoded at a predetermined compression rate on the transmission side, transmitted through a transmission path, and compressed and decoded on the reception side. In line block unit moving image transmission, among the block data for one picture, block data with a late capture start order often has no time margin from the capture start time to the display time.

JP 2007-311924 A

  For this reason, if block data with a slow capture start order is compressed at a high compression rate, a long time is required for compression processing (encoding / decoding), and all data for one picture can be prepared by the time of display. It may disappear.

  Therefore, the present invention intends to provide a moving image transmission apparatus, a moving image transmission system, a moving image transmission method, and a program capable of transmitting a moving image that requires all data for one picture for display with low delay. Is.

  According to an aspect of the present invention, for a moving image that requires data for one picture for display, a capture unit that divides data for one picture into line blocks including a predetermined number of pixel lines and captures the data as block data And a compression rate setting unit that dynamically sets the compression rate of each block data according to the capture time of each block data, and compresses each block data at the set compression rate and transmits it to the receiving device through the transmission path There is provided a moving image transmission apparatus including a transmitting unit. In the moving image transmission apparatus, the capture unit and the transmission unit perform a series of processes including capture and transmission of each block data on a plurality of block data in parallel by shifting the capture time, and the compression rate setting unit is 1 Of the data for the picture, the compression rate of the block data with the later capture start order is set lower than the block data with the earlier capture start order.

  In addition, the compression rate setting unit may set the compression rate of a predetermined number of block data from the last in the capture order in the data for one picture lower than the reference compression rate.

  The compression rate setting unit may set the compression rate of a predetermined number of block data from the end of the capture order of data of one picture to the minimum compression rate.

  In addition, when block data can be transmitted without compression, the compression rate setting unit sets the compression rate of a predetermined number of block data from the last among the data for one picture to 1.0. May be.

  The compression rate setting unit may set the compression rate of a predetermined number of block data whose capture start order is early among the data for one picture higher than the reference compression rate.

  The compression rate setting unit may set the compression rate of each block data so that the data amount after compression for one picture becomes a predetermined value.

  The compression rate setting unit may further set the compression rate of each block data in consideration of the condition of the transmission path.

  In addition, when the transmission path condition is stable, the compression rate setting unit lowers the compression rate of a predetermined number of block data from the end of the data for one picture, or 1.0 May be set.

  In addition, when the transmission path condition is unstable, the compression ratio setting unit sets a compression ratio of a larger number of block data out of one picture data than when the transmission path condition is stable. It may be set low or 1.0.

  The compression rate setting unit compresses block data whose capture start order is slow because the margin time from the completion of processing for displaying block data whose capture start order is slow to the display time of moving images is short. The rate may be set low.

  According to another aspect of the present invention, there is provided a moving image transmission including a transmitting device that transmits a moving image and a receiving device that receives the moving image from the transmitting device through a transmission path, and the transmitting device is the moving image transmission device. A system is provided.

  According to another aspect of the present invention, for a moving image that requires data for one picture for display, the step of dividing the data for one picture into line blocks composed of a predetermined number of pixel lines and capturing it as block data And dynamically setting the compression rate of each block data according to the capture time of each block data, compressing each block data at the set compression rate, and transmitting to the receiving device through the transmission path; A moving image transmission method is provided that includes, for a plurality of block data, a series of processes including a capture step and a transmission step for each block data, shifted in parallel by a capture time, and parallel processing. In the moving image transmission method, in the setting step, the compression rate of the block data whose capture start order is late is set lower than the block data whose capture start order is fast among the data for one picture.

  According to another aspect of the present invention, a program for causing a computer to execute the moving image transmission method is provided. Here, the program may be provided using a computer-readable recording medium or may be provided via communication means.

  As described above, according to the present invention, a moving image transmission apparatus, a moving image transmission system, a moving image transmission method, and a program capable of transmitting a moving image that requires all data for one picture for display with low delay. Can be provided.

1 is a diagram illustrating an overall configuration of a transmission system according to an embodiment of the present invention. It is a figure which compares and shows the moving image transmission process of a picture unit system and a line block unit system. It is a figure (1/2) which shows the processing condition of the moving image transmission in a line block unit system. It is a figure (2/2) which shows the processing condition of the moving image transmission by a line block unit system. It is a block diagram which shows the main function structures of a transmitter and a receiver. It is a figure which shows the scenario A of the dynamic setting of a compression rate. It is a figure which shows scenario B of the dynamic setting of a compression rate. It is a figure which shows the scenario C of the dynamic setting of a compression rate. It is a figure (1/2) which shows the scenario D of the dynamic setting of a compression rate. It is a figure (2/2) which shows scenario D of the dynamic setting of a compression rate. It is a flowchart (1/2) which shows the operation | movement procedure of a moving image transmission system. It is a flowchart (2/2) which shows the operation | movement procedure of a moving image transmission system.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

In the present specification, preferred embodiments of the present invention will be described along the following items.
1. 1. Overall configuration of moving image transmission system 2. Line block unit type moving image transmission processing 3. Configuration of transmitting device and receiving device 4. Dynamic setting of compression rate 5. Operation of moving image transmission system Summary

[1. Overall configuration of moving image transmission system]
First, an overall configuration of a moving image transmission system according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 shows the overall configuration of a moving image transmission system.

  As shown in FIG. 1, the moving image transmission system includes a transmitting device 10 and a receiving device 20 capable of transmitting moving image data through a transmission line NW such as the Internet. The moving image data is transmitted by a stream transmission method such as RTP (Realtime Transport Protocol) defined in IETF RFC3550.

  The transmission device 10 is connected to a moving image input device such as a video camera V, and the receiving device 20 is connected to a moving image output device such as a display D. The transmitting device 10 may be configured integrally with the video camera V, and the receiving device 20 may be configured integrally with the display D.

  The transmission apparatus 10 captures (captures) the moving image signal input from the video camera V as data (a), compresses and encodes (b), packetizes (c), and forward error correction (FEC) encodes (f ) And transmitted to the receiver 20 through the transmission line NW (e).

  On the other hand, the receiving device 20 receives data from the transmitting device 10 through the transmission line NW (e ′), performs FEC decoding (f), depacketizes (g), and compresses and decodes (h). When the receiving device 20 completes the processing (e ′) to (h) for all the data for one picture, the receiving device 20 performs display processing and outputs the data to the display D as data for one picture. indicate.

[2. Line block unit video transmission processing]
Next, with reference to FIG. 2 and FIG. 3, the moving image transmission process of the line block unit method will be described.

  FIG. 2 shows a comparison between moving picture transmission processing of the picture unit method and the line block unit method. In the following description, it is assumed that data for one picture is divided into line blocks with line block numbers k = 1 to N.

  As shown in FIG. 2, in the picture unit system, capture (A), compression encoding (B), packetizing (C), FEC encoding (D), transmission (E), FEC decoding (F), depacketizing ( Each process (A) to (H) of G) and compression decoding (H) is executed in units of pictures. For this reason, the transmission side cannot start the subsequent processes (B) to (E) until the data capture (A) for one picture is completed, and the transmission side transmits data for one picture. Subsequent processes (F) to (H) cannot be started until (E) is completed.

  On the other hand, in the line block unit method, capture (a), compression coding (b), packetization (c), FEC coding (d), transmission (e), FEC decoding (f), depacketization (g), and compression Each process (a) to (h) of decoding (h) is executed in units of line blocks. That is, even if the data capture (A) for one picture is not completed on the transmission side, if the data capture (a) for each line block is completed, the subsequent processes (b) to (e) are sequentially performed. If the transmission (e) of the data in units of line blocks is completed, the subsequent processes (f) to (h) can be started in sequence.

  On the transmission side, for example, when the block data capture (a) of the line block number k = 1 is completed, the subsequent processes (b) to (e) are sequentially started and the block data capture of k = 2 ( Each processing (a) to (e) is performed in parallel in units of line blocks, such as starting a). Note that the processing (f) to (h) is performed in parallel on the reception side in units of line blocks as in the transmission side.

  For this reason, in the picture unit method, the delay caused by at least one of the processes (A) to (H) is stored as it is as a process delay for one picture, and the process from capture (A) to display is performed. Overall delay increases. On the other hand, in the line block unit method, a delay caused by at least one of the processes (a) to (h) is absorbed between the processes of the line blocks and stored as it is as a delay of one picture. Therefore, the delay of the entire process from the first capture start (a) to the display is suppressed.

  3A and 3B show the processing status of moving image transmission in the line block unit method. 3A and 3B show the processes (a) to (h) executed in units of line blocks.

  In FIG. 3A, the horizontal axis indicates the capture start time and the moving image display time for a moving picture of one picture. Also, processing blocks in line block units are shown in parallel in the vertical axis direction according to the processing start order.

As shown in FIG. 3A, each processing block is processed in parallel with an offset T _capture corresponding to the processing time required for capture (a). Here, the processing time required for compression encoding (b) and compression decoding (h) varies according to the compression rate e of the block data, and the processing time required for transmission (e) depends on the situation of the transmission line NW. Fluctuate.

Since each processing block is started with an offset T _capture , a processing block with a fast capture start order has a sufficient margin _Tremain from the completion of processing (a) to (h) to the display time point. In a processing block whose capture start order is late, there is almost no margin time _Tremain .

  For this reason, as shown in FIG. 3B, in a processing block whose capture start order is early, even if the compression rate e of the block data is high, the compression processing (encoding / decoding) can be completed relatively easily by the time of display. Can do. On the other hand, in a processing block whose capture start order is slow, if the compression rate e of the block data is high, especially when the state of the transmission line NW is unstable, the compression processing cannot be completed by the time of display, and transmission is performed. There may be a delay. Therefore, in the moving image transmission process of the line block unit method, block data with a late capture start order, particularly the last block data, becomes critical with respect to transmission delay.

  For this reason, in this embodiment, the compression rate e of the block data is dynamically set according to the block data capture time. Here, the compression rate e of the block data is set to be lower for the block data with the later start order than the block data with the earlier capture start order. In the present embodiment, it is assumed that the bandwidth of the transmission line NW is sufficiently large and the time required for block data transmission does not change according to the compression rate e.

  As a result, the processing time required for the compression processing is shortened for block data whose capture start order is late, and the compression processing can be completed by the time of display. Therefore, by compressing block data that is critical with respect to transmission delay at a low compression rate e, a moving image that requires all data for one picture for display can be transmitted with low delay.

[3. Configuration of transmitter and receiver]
Next, with reference to FIG. 4, the configuration of the transmission device 10 and the reception device 20 configuring the moving image transmission system will be described. In FIG. 4, the transmission path of block data is indicated by solid arrows, and the transmission path of control commands and the like is indicated by broken arrows.

  FIG. 4 shows main functional configurations of the transmission device 10 and the reception device 20. As illustrated in FIG. 4, the transmission device 10 includes an input interface (IF) 11, a capture unit 12, a compression encoding unit 13, a packetizing unit 14, an FEC encoding unit 15, an RTP transmission unit 16, and an RTCP (RTP Control Protocol). Section 17 and compression rate setting section 18.

  The input IF 11 inputs a moving image signal supplied from the video camera V as data. The capture unit 12 captures data supplied from the input IF 11 as block data in units of line blocks. In addition, the capture unit 12 notifies the compression rate setting unit 18 of the line block number k of the captured line block. The line block number k is assigned as k = 1,..., N to the first to Nth line blocks at the capture time point for the data for one picture.

  The compression encoding unit 13 compresses and encodes the block data supplied from the capture unit 12. The compression encoding unit 13 compresses and encodes the block data at the compression rate e notified from the compression rate setting unit 18. The compression encoding requires a longer processing time as the compression rate e of the block data is higher. In some cases, compression encoding can be omitted depending on system conditions. In this case, the compression encoding unit 13 supplies the block data to the packetizing unit 14 in an uncompressed state without performing compression encoding.

  The packetizing unit 14 packetizes the compressed data supplied from the compression encoding unit 13 into RTP packets. The FEC encoding unit 15 is supplied with the packet from the packetizing unit 14 and performs FEC encoding. In FEC encoding, a packet is redundantly encoded with an erasure error correction code such as a Reed-Solomon code. The redundancy of FEC encoding may be changed based on transmission path information, block data transmission rate, and the like. The RTP transmission unit 16 transmits the FEC-encoded packet to the reception device 20 through the transmission path NW.

  The RTCP unit 17 controls a session for transmitting and receiving data to and from the receiving device 20 by RTP. The RTCP unit 17 transmits / receives, for example, an RTCP transmission report (SR) packet and an RTCP reception report (RR) packet defined in IETF RFC3550 to / from the RTCP unit 27 of the receiving device 20, so that the status of the transmission line NW Is sent to the compression rate setting unit 18. As the transmission path information, various parameters such as a round-trip transmission time (RTT) and an average packet loss rate are used.

  The compression rate setting unit 18 dynamically sets the compression rate e of the data block based on the line block number k. Further, the compression rate setting unit 18 may dynamically set the compression rate e of the data block in consideration of the transmission path information. The compression rate e is set in the range from the minimum compression rate emin to the maximum compression rate emax. Further, 1.0 (non-compressed) may be set as the compression rate e depending on the system conditions. The compression rate setting unit 18 notifies the compression encoding unit 13 of the set value of the compression rate e. Thereby, in the moving image transmission system, each block data can be compressed and transmitted at a different compression rate e based on the capture start order (and the state of the transmission path NW). The dynamic setting of the compression rate e will be described later.

  As described with reference to FIGS. 3A and 3B, in the transmission apparatus 10, each process of capture (a), compression encoding (b), packetizing (c), FEC encoding (d), and RTP transmission (e) is performed. A plurality of block data are processed in parallel.

  The functional configuration of the transmission device 10 may be realized as hardware, or at least partially realized as software. In the latter case, a control unit (not shown) provided in the transmission apparatus 10 executes a program for executing the moving image transmission method.

  The receiving device 20 includes an RTP receiving unit 21, an FEC decoding unit 22, a depacketizing unit 23, a compression decoding unit 24, a display processing unit 25, an output IF 26, and an RTCP unit 27.

  The RTP receiver 21 receives a packet from the transmission device 10 through the transmission line NW. The RTP receiver 21 receives an FEC encoded packet. The FEC decoding unit 22 is supplied with the FEC-encoded packet from the RTP receiving unit 21 and performs FEC decoding. The depacketizing unit 23 depacketizes the packet supplied from the RTP receiving unit 21 into compressed data (or uncompressed data).

  The compression decoding unit 24 is supplied with the compressed data from the depacketizing unit 23 and decodes the compressed data into block data. The compression decoding unit 24 compresses and decodes the compressed data compressed at a predetermined compression rate e and stores it in a buffer (not shown). In compression decoding, the higher the block data compression rate e, the more processing time is required. In some cases, compression decoding can be omitted depending on system conditions.

  The display processing unit 25 is supplied with block data for one picture from the compression decoding unit 24, and performs display processing to generate a moving image signal. The output IF 26 is supplied with the moving image signal from the display processing unit 25 and outputs it to the display D. The RTCP 26 controls a session for transmitting and receiving data to and from the transmission device 10 by RTP. The RTCP 26 is configured similarly to the RTCP unit 17 of the transmission device 10.

  As described with reference to FIGS. 3A and 3B, in the receiving device 20, RTP reception (e ′), FEC decoding (f), depacketization (g), and compression decoding (h) are performed in parallel for a plurality of block data. It is processed.

  Further, the functional configuration of the receiving device 20 may be realized as hardware, or at least partially realized as software. In the latter case, a control unit (not shown) provided in the receiving device 20 executes a program for executing the moving image transmission method.

[4. Dynamic setting of compression ratio]
5 to 8A and 8B schematically show scenarios A to D in which the compression rate e is dynamically set. In the following, it is assumed that the reference compression rate e0 = 2.0, the maximum compression rate emax = 4.0, and the minimum compression rate emin = 1.2 when dynamic setting of the compression rate e is not performed. In addition, these compression rates are only examples, and are variously set according to a system. Further, it is assumed that the reference bit rate when data for one picture is transmitted at the reference compression rate e0 is R0.

  Scenario A shown in FIG. 5 shows a case where the compression rate e of a predetermined number of block data whose capture start order is late is set low. FIG. 5 shows a block processing status, a compression rate e setting status, and a bit rate R for a line block for one picture in scenario A. Note that only the capture (a), compression encoding (b), transmission (e), and compression decoding (h) processes are (a), (b), (e), (h) ) Shown in order.

In scenario A, it is assumed that the processing of k = N−1, N processing blocks cannot be completed by the time of display even if transmission at an average RTT is assumed. In this case, the compression rate e of the block data of k = N−1 and N is set to e = 1.5 and 1.2 (= emin), respectively, which is lower than the reference compression rate e0. The bit rate R of data for one picture is higher than the reference bit rate R0. Here, the compression rate e is set based on the block data capture start time and the processing time T _total of the processing block so that the processing of the processing block can be completed by the display time.

  Note that the compression rate e may be set such that a processing time of the processing block is completed with an allowance time until the display time. Further, the compression rate e may be set as the same value (for example, e = 1.2) without being set as different values for a plurality of block data.

Specifically, the processing time T _total of the processing block is defined as capture (a), compression encoding (b), packetizing (c), FEC encoding (d), transmission (e), FEC decoding (f), depacketizing ( g) and the total processing time of compression decoding (h). Here, processing times other than transmission (e), compression processing (b), and (h) are set as predetermined values. The processing time for transmission (e) is obtained from the average RTT, and the processing times for compression processing (b) and (h) are obtained using the compression rate e as a parameter.

And the compression rate e of each block data is calculated | required so that the processing time _Ttotal of each process block may be less than the processable time _Tremain . Here, the processable time _Tremain is set as the time from the completion of the processing of each block data to the display time, as shown in FIG. 3A.

In Scenario A, k = N-1, the processing time _{T total} processing block of N such that less than processible time _{T Remain,} compression ratio e = 1.5,1.2 is set to each. Thereby, the processing time of the compression processing (b) and (h) can be shortened for the block data of k = N−1 and N, and the processing of the processing block can be completed by the time of display. When the compression processes (b) and (h) are simultaneously performed in a plurality of processing blocks, the processing can be implemented by dividing into a plurality of threads.

Here, depending on the system conditions, the compression rate e may be set to e = 1.0 (non-compression) in order to further reduce the processing time of the compression processes (b) and (h). When the compression rate e is e = 1.0, the processing times of the compression processes (b) and (h) are substantially zero, and the processing time T _total of the processing block can be further shortened.

  Note that by reducing the compression rate e of some block data (eg, k = N−1, N), the processing block of block data (eg, k = N−2) whose capture start order is earlier than that block data May become critical with respect to transmission delay. In this case, a compression rate e lower than the reference compression rate e0 is also set for critical block data.

  In scenario B shown in FIG. 6, the compression rate e of a predetermined number of block data whose capture start order is slow is set low, and the compression rate e of a predetermined number of block data whose capture start order is fast is set high. ing. FIG. 6 shows the block processing status, the compression rate e setting status, and the bit rate R for a line block for one picture in scenario B.

  In this case, the compression rate e of the block data of k = N−1 and N is set to e = 1.5 and 1.2 (= emin) lower than the reference compression rate e0, and k = 1 to n1. The compression rate e of the block data is set to e = 4.0 (= emax). The compression rate e of each block data is set so that the processing of the processing block can be completed by the time of display. Further, the compression rate e may be set as a different value for the block data of k = 1 to n1, instead of being set as the same value.

  Here, as shown in scenario A, if the compression rate e of a predetermined number of block data is set low, the bit rate R of data for one picture increases. Therefore, in order to suppress an increase in the bit rate R, the compression rate e of a predetermined number of block data whose capture start order is early is set high. It should be noted that, as shown in FIG. 3A, the block data whose capture start order is early does not cause a transmission delay because there is a sufficient margin time from the completion of processing to the display time point. As a result, the processing time of the compression processing (b) and (h) can be shortened while suppressing the increase in the bit rate R of the data for one picture, and the processing of the processing block can be completed by the time of display.

  In the scenario C shown in FIG. 7, a case is shown in which the compression rate e of each block data is set so that the bit rate R of data for one picture becomes a predetermined value. FIG. 7 shows the block processing status, the compression rate e setting status, and the bit rate R for a line block for one picture in scenario C.

  In scenario C, a case will be described in which the bit rate R of data for one picture becomes the reference bit rate R0. However, the compression rate e may be set so as to have a bit rate other than the reference bit rate R0.

  In this case, the compression rate e of the block data is e1 = 3.0 when k = 1 to n1 (number of blocks n1), and e2 = 2.0 (= e0) when k = n1 + 1 to N-2 (number of blocks n2). , K = N−1, N (number of blocks n3), and e3 = 1.2 (= emin). Here, the combination of the compression rate e and the number of blocks and the number of sections of the compression rate e are merely examples. Further, the compression rate e of each block data is set so that the processing of the processing block can be completed by the time of display.

  Here, the compression ratios e1 to e3 are set so as to satisfy the relationship of e1 · n1 + e2 · n2 + e3 · n3 = e0 · (n1 + n2 + n3). As a result, the processing time of the compression processing (b) and (h) can be shortened while maintaining the bit rate of the data for one picture at a predetermined value, and the processing of the processing block can be completed by the time of display.

  Scenario D shown in FIGS. 8A and 8B shows a case where the compression rate e of each block data is set according to the state of the transmission line NW. 8A and 8B show the RTT fluctuation status, the block processing status, and the compression rate e of the line block for one picture in scenario D when the transmission line NW is stable and unstable. Each setting is shown. In addition, as a case where the transmission line NW is unstable, for example, a case where a variation in RTT is large, a case where packet reordering or the like can occur on the transmission line NW, and the like are assumed.

  Even in scenario D, it is assumed that even if transmission at an average RTT (RTTavg) is assumed, the processing of the processing blocks of k = N−1 and N cannot be completed by the time of display. Note that the state of the transmission line NW is, for example, that the RTT achieved at the time of transmitting a picture of the previous frame and the rate of change of the RTT achieved at the time of transmitting a picture of the same frame (transmitted block data) are a predetermined threshold value. Judgment is made according to whether or not the above is satisfied.

As shown in FIG. 8A, when the state of the transmission line NW is stable, the processing time of transmission (e) is substantially constant at RTTavg / 2, so that the processing time T _total of the processing block is the compression processing (b ) And (h) will vary depending on the processing time. For this reason, the compression rate e of the block data of k = N−1 and N is set to e = 1.5 and 1.2, respectively, which is lower than the reference compression rate e0.

Here, in the setting of the compression rate e of the block data of k = N−1 and N, the processing time T _total of each processing block is obtained by setting the processing time of transmission (e) to RTTavg / 2. Then, the compression rate e of each block data is determined so that the margin time _Tremain from the completion of the processing of each processing block to the display time is 0 or more.

On the other hand, as shown in FIG. 8B, when the state of the transmission line NW is unstable, the processing time of transmission (e) varies, so that the processing time T _total of the processing block is equal to the compression processing (b), (h ) In accordance with the processing time of transmission (e). For this reason, the compression rate e of a predetermined number of block data whose capture start order is late is set low according to the state of the transmission line NW.

  In setting the compression rate e of each block data, the processing time (expected transmission time) of transmission (e) expected to be achieved with a predetermined probability is obtained. For example, the expected transmission time achieved with a probability of 80% is obtained as an 80% value when ½ of the latest RTTs are arranged in ascending order (ascending order). As the most recent RTT, a predetermined number of data from the most recent RTT used for determining the state of the transmission line NW is used. The expected transmission time may be obtained as a value other than the 80% value.

Next, the processing time T _total of each processing block is obtained using the processing time of transmission (e) as the expected transmission time. Then, the compression rate e of each block data is determined so that the margin time _Tremain from the completion of the processing of each processing block to the display time is 0 or more.

  For example, in scenario D, the compression rate e of each processing block is obtained by setting the expected transmission time as 80% of 1/2 of the latest RTT. In scenario D, assuming that non-compression transmission is possible, the compression rate e is e = 1.5, 1.2 for block data of k = N−2, N−1, and N. , 1.0, respectively. In block data of k = N, the processing time of compression encoding (b) and compression decoding (h) is almost zero. As a result, the processing time of the compression processes (b) and (h) can be shortened while considering the state of the transmission line NW, and the processing of the processing block can be completed by the display time.

  Note that the block data for which the compression rate e is lowered according to the condition of the transmission line NW is not limited to a predetermined number of block data from the end in the capture start order. Therefore, the compression rate e of the block data having the middle capture start order may be lowered. In any case, by reducing the compression rate e of the block data to shorten the time required for the compression process, it is possible to suppress the transmission delay of the moving image caused by the fluctuation of the transmission path NW.

[5. Operation of moving image transmission system]
Next, the operation of the moving image transmission system will be described with reference to FIGS. 9A and 9B. 9A and 9B show the operation procedures of the transmission device 10 and the reception device 20 from the start of capture to the display of a moving image for one picture of data.

  9A and 9B show that the processing is completed in units of line blocks. However, in practice, as described above, when the capture (a) of the line block is completed, the compression encoding (b) of the line block is started and the capture (a) of another line block is started. As described above, processing of a plurality of line blocks is executed in parallel.

  First, the operation of the transmission device 10 will be described. As shown in FIG. 6A, in the transmission apparatus 10, the capture unit 12 captures data in units of line blocks as block data (step S101). In the process of step S101, each time the process is executed, block data of line block numbers k = 1 to N are captured sequentially.

  Based on the line block number k from the capture unit 12, the compression rate setting unit 18 dynamically sets the compression rate e of each block data according to the setting method described above (S103). When block data can be transmitted without compression, the compression rate e may be set to e = 1.0, that is, uncompressed.

  The compression encoding unit 13 confirms whether the compression rate e is set to other than e = 1.0 (S105). When the compression rate e is set to a value other than e = 1.0, the compression encoding unit 13 compresses and encodes the block data at the set compression rate e (S107). The packetizing unit 14 is supplied with block data that has been compression-encoded or block data that has not been compression-encoded.

  The packetizing unit 14 packetizes compressed data or non-compressed data into RTP packets, and the FEC encoding unit 15 performs FEC encoding of the packets (S109). The RTP transmitter 16 transmits the FEC encoded packet to the receiving device 20 (S111).

  Further, the transmitting apparatus 10 determines whether the processing of data for one picture has been completed based on the line block number k, that is, whether the block data of the line block number k = N has been processed (S113). When the processing is completed, the processing on the transmission side is completed for one picture. On the other hand, if not completed, the line block number k is incremented by 1, and the data of the next line block is captured (S101).

  Next, the operation of the receiving device 20 will be described. As illustrated in FIG. 9B, in the receiving device 20, the RTP receiving unit 21 receives a packet from the transmitting device 10 through the transmission line NW (S151). The packet is received as an FEC encoded packet.

  The FEC decoding unit 22 performs FEC decoding on the FEC-encoded packet, and the depacketizing unit 23 depacketizes the packet (S153). The compression / decryption unit 24 confirms whether the depacketized data is compressed (S155). When the data is compressed, the compression decoding unit 24 compresses and decodes the depacketized data and stores the data in a buffer (not shown) or the like (S157).

  When the process of step S157 is completed, it is determined whether the process for one picture is completed (S159). When the process is completed, the display process (S161) is performed on the data for one picture. When the process is not completed, the packet of the next line block is received (S151).

[6. Summary]
As described above, the moving image transmission system according to the embodiment of the present invention transmits a moving image that requires all data for one picture for display. In the system, data for one picture is captured as block data in units of line blocks. Further, the compression rate e of the block data is dynamically set according to the capture time of the block data. Here, the compression rate e of the block data is set to be lower for the block data with the later start order than the block data with the earlier capture start order. In the present embodiment, it is assumed that the bandwidth of the transmission line NW is sufficiently large and the time required for block data transmission does not change according to the compression rate e.

  As a result, the processing time required for the compression processing is shortened for block data whose capture start order is late, and the compression processing can be completed by the time of display. Therefore, by compressing block data that is critical with respect to transmission delay at a low compression rate e, a moving image that requires all data for one picture for display can be transmitted with low delay.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, in the above description, a case where parallel processing is performed in units of line blocks has been described. However, a processing unit composed of a predetermined number of block data may be pipelined and processed in parallel.

  In the above description, the case where the FEC code method is used as an error correction method at the time of transmission has been described. However, as an error correction method at the time of transmission, for example, the FEC code method may not be used, a combination of the FEC code method and the automatic reproduction (ARQ) method may be used, and other methods may be used. Of course, error correction need not be performed.

10 Transmitter 11 Input IF
DESCRIPTION OF SYMBOLS 12 Capture part 13 Compression encoding part 14 Packetization part 15 FEC encoding part 16 RTP transmission part 17 RTCP part 18 Compression rate setting part 20 Receiving device 21 RTP receiving part 22 FEC decoding part 23 Depacketizing part 24 Compression decoding part 25 Display processing part 26 Output IF
27 RTCP part NW transmission line V Video camera D Display

Claims (15)

For a moving image that requires data for one picture for display, a capture unit that divides the data for one picture into line blocks including a predetermined number of pixel lines and captures the data as block data;
According to the capture time of each block data, a compression rate setting unit that dynamically sets the compression rate of each block data,
A transmission unit that compresses each block data at the set compression rate and transmits the compressed data to a receiving device through a transmission path;
The capture unit and the transmission unit, for a plurality of block data, a series of processes including the capture and transmission of the block data are shifted in parallel by a capture time, and are processed in parallel.
The moving picture transmission apparatus, wherein the compression rate setting unit sets a compression rate of block data whose capture start order is later than block data whose capture start order is early among the data for one picture.   The moving image according to claim 1, wherein the compression rate setting unit sets a compression rate of a predetermined number of block data from the end of the capture order in the data for one picture lower than a reference compression rate. Transmission equipment.   The moving image transmission apparatus according to claim 2, wherein the compression rate setting unit sets a compression rate of a predetermined number of block data from the end of the capture order of the data for one picture to a minimum compression rate. . In the case where the block data can be transmitted without compression,
The moving image transmission apparatus according to claim 3, wherein the compression rate setting unit sets the compression rate of a predetermined number of block data from the last in the capture start order to 1.0 among the data for one picture. .   5. The compression ratio setting unit sets a compression ratio of a predetermined number of block data whose capture start order is early among the data for one picture higher than a reference compression ratio. The moving image transmission apparatus according to the item.   The moving image according to any one of claims 1 to 5, wherein the compression rate setting unit sets the compression rate of each block data so that the amount of data after compression of the one picture becomes a predetermined value. Image transmission device.   The moving image transmission apparatus according to claim 1, wherein the compression rate setting unit further sets a compression rate of each block data in consideration of a state of the transmission path.   The compression rate setting unit lowers the compression rate of a predetermined number of block data from the end of the capture start order of the data for one picture when the transmission path condition is stable, or The moving image transmission apparatus according to claim 7, wherein the moving image transmission apparatus is set to zero.   The compression rate setting unit compresses a larger number of block data of the data for one picture when the transmission path condition is unstable than when the transmission path condition is stable. The moving image transmission apparatus according to claim 7, wherein the value is set to be low or 1.0.   The compression rate setting unit has a short margin time from the completion time of processing for displaying the block data whose capture start order is slow to the display time of the moving image, and therefore the block data whose capture start order is slow The moving image transmission apparatus according to claim 1, wherein the compression rate is set low. A transmitting device that transmits a moving image; and a receiving device that receives the moving image from the transmitting device through a transmission path;
A moving image transmission system, wherein the transmission device is the moving image transmission device according to claim 1. For a moving image that requires data for one picture for display, the data for one picture is divided into line blocks including a predetermined number of pixel lines and captured as block data;
Dynamically setting the compression rate of each block data according to the capture time of each block data;
Compressing each block data at the set compression rate and transmitting it to a receiving device through a transmission path;
For a plurality of block data, a series of processes including the capture step and the transmission step of each block data are shifted in parallel by a capture required time and processed in parallel.
In the setting step, the compression rate of the block data whose capture start order is later than the block data whose capture start order is early among the data for one picture is set lower.   A program for causing a computer to execute the moving image transmission method according to claim 12. For a moving image that requires data for one picture for display, a capture unit that divides the data for one picture into line blocks including a predetermined number of pixel lines and captures the data as block data;
According to the capture time of each block data, a compression rate setting unit that dynamically sets the compression rate of each block data,
A transmission unit that compresses each block data at the set compression rate and transmits the compressed data to a receiving device through a transmission path;
The capture unit and the transmission unit, for a plurality of block data, a series of processes including the capture and transmission of the block data are shifted in parallel by a capture time, and are processed in parallel.
The compression rate setting unit sets the compression rate of the predetermined number of block data from the beginning of the data for one picture higher than the reference compression rate from the beginning, and sets the predetermined number of block data from the end. The compression rate is lower than the reference compression rate or set to 1.0,
Video transmission device. For a moving image that requires data for one picture for display, the data for one picture is divided into line blocks including a predetermined number of pixel lines and captured as block data;
Dynamically setting the compression rate of each block data according to the capture time of each block data;
Compressing each block data at the set compression rate and transmitting it to a receiving device through a transmission path;
For a plurality of block data, a series of processing including the capture and transmission of each block data is shifted by a capture time, and a parallel processing step,
In the setting step, the compression rate of the predetermined number of block data is set higher than the reference compression rate from the beginning in the capture start order of the data for one picture, and the compression rate of the predetermined number of block data from the end Is set lower than the reference compression ratio or 1.0,
Video transmission method.

Images