JP2012134758A - Video distribution device and video distribution method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To retransmit video data efficiently.SOLUTION: An encoder 10 transmits a video stream to which a multicast address is set. The encoder 10 acquires a reciprocation delay time between its own device and a plurality of reception terminals included by the multicast address. The encoder 10 also accepts a retransmission request of a data packet included in the video stream from the plurality of terminals. The encoder 10 further transmits a packet required for reproducing the video stream at the terminal at least after the maximum reciprocation delay time among the reciprocation delay times acquired previously passes from transmission time of the data packet whose retransmission request has been accepted.

Description

  The present invention relates to a video distribution apparatus and a video distribution method.

  When video data is distributed in real time, an error correction method that suppresses retransmission due to packet loss is used. As an example of an error correction method, forward error correction, so-called FEC (Forward Error Correction) in which data that has not arrived on the receiving side is restored from redundant data by adding redundant data at the time of transmission and transmitting the data. Is used.

  As an example of a technique using such FEC, there is a data transmission server that increases FEC added to data to be transmitted to a terminal via a network as the transmission rate increases. As another example, there is a media transmission device that adaptively controls each use band so that the transmission band of the sum of the use band by the retransmission method, the use band by the redundant code method, and the use band of the media data is constant. Can be mentioned.

  In addition to the technique of adding an FEC packet to a data packet and distributing the data packet, there is a technique of transmitting the data packet in advance and then transmitting the FEC packet. As an example, there is a data broadcast delivery method used when a data delivery terminal executes one-way broadcast delivery without return to a plurality of receiving terminals. In this data broadcast delivery method, the data delivery terminal broadcasts the error correction check data following the delivery data. On the other hand, the receiving terminal performs error correction after combining the received data received from the data distribution terminal with the error correction check data.

JP 2001-333061 A JP 2004-289621 A Japanese Patent Laying-Open No. 2005-064648

  However, the above-described prior art has a problem that video data cannot be efficiently retransmitted when multicast is used for real-time video data distribution.

  For example, in the case of the data transmission server and the media transmission device described above, every time a retransmission request is received from the receiving terminal, the video data is retransmitted to all receiving terminals included in the multicast group. For this reason, in the data transmission server and the media transmission apparatus, the video data is retransmitted redundantly to the receiving terminal in which no error has occurred. Therefore, in the above-described data transmission server and media transmission apparatus, resources of the transmission terminal and the network are wasted due to redundant retransmission of video data.

  In the case of the data broadcast delivery method described above, the error correction check data generated by the data delivery terminal is broadcast to the receiving terminal regardless of whether there is an error in the receiving terminal. For this reason, in the data broadcast delivery method described above, even when no error has occurred at the receiving terminal, the delivery data is duplicated and delivered to the receiving terminal. Further, in the data broadcast delivery method described above, when the video data cannot be restored only with the error correction check data, the delivery data must be retransmitted in addition to the error correction check data. Therefore, also in the data broadcast delivery method, the resources of the transmission terminal and the network are wasted as in the case of the data transmission server and the media transmission device.

  The disclosed technology has been made in view of the above, and an object thereof is to provide a video distribution apparatus and a video distribution method capable of efficiently retransmitting video data.

  The video distribution apparatus disclosed in the present application includes a first transmission unit that transmits a video stream in which a multicast address is set. Furthermore, the video distribution apparatus has an acquisition unit that acquires round-trip delay times between a plurality of receiving terminals included in the multicast address and the own apparatus. Furthermore, the video distribution apparatus has a reception unit that receives a retransmission request for a data packet included in the video stream from the plurality of terminals. Further, the video distribution device is necessary for reproducing the video stream at the terminal after at least the maximum round-trip delay time of the round-trip delay time has elapsed from the transmission time of the data packet for which the retransmission request has been accepted. A second transmission unit configured to transmit the packet;

  According to one aspect of the video distribution device disclosed in the present application, there is an effect that video data can be efficiently retransmitted.

FIG. 1 is a diagram illustrating the configuration of the video distribution system according to the first embodiment. FIG. 2 is a block diagram illustrating the configuration of the encoder according to the first embodiment. FIG. 3 is a diagram illustrating a configuration example of information stored in the FEC information storage unit. FIG. 4 is a diagram illustrating a configuration example of information stored in the FEC information storage unit. FIG. 5 is a diagram illustrating a configuration example of information stored in the RTT storage unit. FIG. 6 is a diagram illustrating a configuration example of information stored in the RTT storage unit. FIG. 7 is a diagram illustrating a configuration example of information stored in the retransmission request storage unit. FIG. 8 is a diagram illustrating a configuration example of information stored in the reception status storage unit. FIG. 9 is a diagram for explaining the decrement of the retransmission request total value. FIG. 10 is a flowchart illustrating the procedure of the first transmission process according to the first embodiment. FIG. 11 is a flowchart illustrating the procedure of the second transmission process according to the first embodiment. FIG. 12 is a diagram illustrating an example of a packet reception order in the decoder. FIG. 13 is a diagram illustrating an example of a packet reception order in the decoder. FIG. 14 is a diagram illustrating an example of a packet reception order in the decoder. FIG. 15 is a schematic diagram illustrating an example of a computer that executes a video distribution program according to the second embodiment.

  Embodiments of a video distribution apparatus and a video distribution method disclosed in the present application will be described below in detail with reference to the drawings. Note that this embodiment does not limit the disclosed technology. Each embodiment can be appropriately combined within a range in which processing contents are not contradictory.

[System configuration]
FIG. 1 is a diagram illustrating the configuration of the video distribution system according to the first embodiment. As shown in FIG. 1, the video delivery system 1 accommodates an encoder 10, a camera 20, decoders 30A to 30C, and display devices 40A to 40C. In the example of FIG. 1, it is assumed that the video data captured by the camera 20 is distributed from the encoder 10 to the decoders 30A to 30C in real time.

  The encoder 10 and the decoders 30 </ b> A to 30 </ b> C are connected to each other via the network 3 so as to communicate with each other. As one aspect of the network 3, a communication network such as the Internet, a LAN (Local Area Network), and a VPN (Virtual Private Network) can be cited. In the example of FIG. 1, the case where three decoders 30 </ b> A to 30 </ b> C are accommodated has been illustrated, but it is sufficient that at least one decoder is accommodated, and the present invention can be applied when an arbitrary number of decoders are accommodated.

  Among these, the camera 20 is an imaging device that captures an image. An example of the camera 20 is an imaging device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). Video data captured by the camera 20 is output to the encoder 10.

  The encoder 10 is a device that encodes video data output by the camera 20. The encoder 10 distributes the encoded video data to the decoders 30A to 30C. At this time, the encoder 10 multicasts video data to the decoders 30A to 30C by transmitting to the network 3 a video stream in which a multicast address assigned to the multicast group in which the decoders 30A to 30C participate is set. In the example of FIG. 1, a case where RTP (Realtime Transport Protocol) is adopted as a media data transmission method will be described, but other transmission methods may be adopted.

  The video distributed by the encoder 10 in this way may be a monitoring video imaged to monitor a predesignated area, or may be a broadcast issued to the viewer. In the example of FIG. 1, the case where the camera 20 supplies the video data to the encoder 10 is illustrated, but the video data does not necessarily have to be supplied by the camera 20. For example, video data may be acquired from a recording medium such as a flexible disk, so-called FD, CD-ROM, DVD disk, magneto-optical disk, IC card, or the like via a predetermined reader.

  The decoders 30 </ b> A to 30 </ b> C are devices that decode the video data distributed by the encoder 10. The decoders 30A to 30C reproduce the decoded video data on the display devices 40A to 40C. Hereinafter, when the decoders 30 </ b> A to 30 </ b> C are collectively referred to without being distinguished, they are collectively referred to as “decoder 30”.

  The encoding function of the encoder 10 and the decoding functions of the decoders 30A to 30C may be realized by software or may be realized by an integrated circuit such as an LSI (Large Scale Integration).

  The display devices 40A to 40C are display devices that display video supplied from the decoder. As an example of the display devices 40A to 40C, display devices such as a monitor, a display, and a touch panel can be applied. Hereinafter, when the display devices 40 </ b> A to 40 </ b> C are collectively referred to without distinction, they are collectively referred to as “display device 40”.

  Here, the encoder 10 according to the present embodiment transmits a video stream in which a multicast address is set. Also, the encoder 10 according to the present embodiment acquires the round-trip delay time between the plurality of decoders 30A to 30C included in the multicast address and the own apparatus. In addition, the encoder 10 according to the present embodiment accepts retransmission requests for data packets included in the video stream from the plurality of decoders 30A to 30C. Furthermore, the encoder 10 according to the present embodiment transmits a packet necessary for reproducing the video stream by the decoders 30A to 30C after at least the maximum round-trip delay time has elapsed from the transmission time of the data packet for which the retransmission request has been accepted. Send.

  As described above, the encoder 10 according to the present embodiment distributes only the data packet in advance when the video stream is multicast, and retransmits the packet necessary for reproducing the video stream. For this reason, the encoder 10 according to the present embodiment does not transmit the packet to the distribution destination decoders 30A to 30C when the packet retransmitted from the encoder 10 is not necessary for reproducing the video stream. Therefore, according to the encoder 10 according to the present embodiment, it is possible to efficiently retransmit the video data.

  Furthermore, the encoder 10 according to the present embodiment waits for the maximum round-trip delay time with the distribution destination decoders 30A to 30C, and then retransmits a packet necessary for reproducing the video stream. For this reason, the encoder 10 according to the present embodiment can retransmit when it is estimated that all the retransmission requests of the decoders 30A to 30C have arrived. Therefore, according to the encoder 10 according to the present embodiment, it is possible to prevent a situation in which some of the decoders 30 are not retransmitted by retransmitting a packet without receiving a retransmission request from the decoders 30A to 30C. In addition, according to the encoder 10 according to the present embodiment, it is possible to prevent a delay in packet retransmission by excessively extending the time from when a retransmission request is received from the decoders 30A to 30C until the packet is retransmitted. it can.

[Configuration of Encoder 10]
Subsequently, the configuration of the encoder 10 according to the present embodiment will be described. FIG. 2 is a block diagram illustrating the configuration of the encoder according to the first embodiment. As shown in FIG. 2, the encoder 10 includes a video buffer 11a, an encoding unit 11, an FEC buffer 12a, an FEC information storage unit 12b, a generation unit 12, an RTT storage unit 13a, an acquisition unit 13, A retransmission request storage unit 14 a and a reception unit 14 are included. Furthermore, the encoder 10 includes an acceptance status storage unit 15 a, a determination unit 15, a first transmission unit 16, and a second transmission unit 17.

  Among these, the video buffer 11a is a buffer for storing video data encoded by the encoding unit 11 described later. The video data stored in the video buffer 11a in this way is multicast to the decoders 30A to 30C by the first transmission unit 16 described later. Note that the video data stored in the video buffer 11a can be deleted at any time. For example, it may be deleted when it is confirmed that the video data has been received by all the decoders 30A to 30C, or may be deleted when a predetermined time is measured by a timer.

  As an example of the video buffer 11a, a semiconductor memory device such as a video random access memory (VRAM), a random access memory (RAM), a read only memory (ROM), or a flash memory can be employed. As another example, the video buffer 11a may employ a storage device such as a hard disk or an optical disk.

  The encoding unit 11 is a processing unit that encodes video data output by the camera 20. The encoding unit 11 stores the encoded video data in the video buffer 11a. As an example of such a compression encoding system, any compression system such as MPEG (Moving Picture Experts Group phase) 2 or MPEG 4 can be adopted.

  The FEC buffer 12a is a buffer that accumulates FEC (Forward Error Correction) data generated from encoded data by the generation unit 12 described later. The FEC data accumulated in the FEC buffer 12a in this way is multicast to the decoders 30A to 30C by the second transmission unit 17 described later under an instruction from the determination unit 15 described later.

  As an example of the FEC buffer 12a, a semiconductor memory device such as a video random access memory (VRAM), a random access memory (RAM), a read only memory (ROM), or a flash memory can be employed. As another example, the FEC buffer 12a may employ a storage device such as a hard disk or an optical disk.

  The FEC information storage unit 12b is a storage unit that stores management information related to FEC data generated by the generation unit 12 described later. As an example, the FEC information storage unit 12b is referred to by the determination unit 15 described later in order to determine whether or not the timing for determining whether retransmission is necessary is appropriate.

  As an aspect of such management information, data in which the FEC number, the start video number, the end video number, the generation time of the FEC data, and the determination time of the FEC packet are associated can be employed. The “FEC number” here refers to a sequence number assigned to the FEC data. The “video number” refers to a sequence number assigned to one frame of video data included in the video stream. The “starting video number” refers to the smallest video number among the video numbers of video data used to generate FEC data. The “end video number” indicates the highest video number among the video numbers of video data used to generate FEC data. The “generation time” refers to the time when the FEC data is generated. The “determination time” refers to a time at which a later-described determination unit 15 determines whether or not the second transmission unit 17 described later needs to transmit FEC data to the decoders 30A to 30C.

  3 and 4 are diagrams illustrating a configuration example of information stored in the FEC information storage unit. In the example of FIG. 3, the management information regarding the FEC data of FEC number 1 generated using the video data of video number 1 to video number 4 is registered. Further, the example of FIG. 4 shows a case where management information related to FEC data of FEC number 2 generated using video data of video numbers 5 to 8 is additionally registered. In the examples of FIGS. 3 and 4, it is assumed that one FEC data is generated for four pieces of video data, and that one piece of video data can be restored. 3 and 4 exemplify a case where one piece of FEC data is generated for four pieces of video data, the number of pieces of video data used for generating FEC data may be arbitrary.

  The example shown in FIG. 3 indicates that the FEC data of FEC number 1 is generated from the video data of video number 1 to video number 4 at the generation time “2010/06/01 10: 00: 00.123”. Further, in the case of FEC data with FEC number 1, it is determined whether or not it is necessary to transmit FEC data with FEC number 1 to the decoders 30A to 30C by a second transmission unit 17 to be described later. 01 10: 00: 00.243 "indicates that the determination unit 15 described later makes a determination. The example shown in FIG. 4 indicates that the FEC data of FEC number 2 is generated from the video data of video number 5 to video number 8 at the generation time “2010/06/01 10: 00: 00.678”. Further, in the case of FEC number 2 FEC data, it is determined whether or not it is necessary to transmit the FEC number 2 FEC data to the decoders 30 </ b> A to 30 </ b> C by a second transmission unit 17 described later. “01 10: 00: 00.751” indicates that the determination unit 15 described later makes a determination. Note that the value of the record for which the determination of whether or not transmission is necessary by the determination unit 15 described later is initialized to zero. Further, the management information is not necessarily stored in a table format, and may be stored in another format.

  The generation unit 12 is a processing unit that generates FEC data from video data continuous over a predetermined number of frames in a video stream. As an example, the generation unit 12 generates parity information of FEC data by calculating an exclusive OR (XOR) in units of bits of video data of four consecutive frames in the video stream.

  For example, assume that FEC data of FEC number 1 is generated using video data of video number 1 to video number 4. In this case, the first bit of the FEC data of FEC number 1 includes the first bit of the video data of video number 1, the first bit of the video data of video number 2, the first bit of the video data of video number 3, and the video number 4. It is calculated by adding the first bits of the video data. Similarly, the Nth bit is calculated from the second bit of the FEC data. In this manner, the generation unit 12 generates FEC data with FEC number 1 from the video data with video numbers 1 to 4.

  The RTT storage unit 13a is a storage unit that stores a round-trip delay time, so-called RTT (Round Trip Time). As an example, the RTT storage unit 13a is referred to by the generation unit 12 to determine the determination time of the FEC data from the generation time of the FEC data. Note that the round-trip delay time acquired by the acquisition unit 13 described later is registered in the RTT storage unit 13a.

  As one aspect, the RTT storage unit 13a stores the round-trip delay time between the own device and each decoder 30. 5 and 6 are diagrams illustrating an example of a configuration of information stored in the RTT storage unit. In the example of FIG. 5, the round trip delay time at the time when the FEC data of FEC number 1 is generated is shown. In the example of FIG. 6, it indicates the round trip delay time at the time when the FEC data of FEC number 2 is generated. 5 and FIG. 6, the IP address of the decoder 30A is “10.0.0.1”, the IP address of the decoder 30B is “10.0.0.2”, and the IP address of the decoder 30C is “10.0.0.1”. It shall be 0.3 ”.

  In the example shown in FIG. 5, the round-trip delay time between the encoder 10 and the decoder 30 </ b> A is “98 (ms)”. Furthermore, it shows that the round-trip delay time between the encoder 10 and the decoder 30B is “71 (ms)”. Furthermore, it indicates that the round-trip delay time between the encoder 10 and the decoder 30C is “120 (ms)”. In the example shown in FIG. 6, the round-trip delay time between the encoder 10 and the decoder 30 </ b> A is “66 (ms)”. Furthermore, it shows that the round-trip delay time between the encoder 10 and the decoder 30B is “73 (ms)”. Furthermore, it shows that the round-trip delay time between the encoder 10 and the decoder 30C is “69 (ms)”.

  The acquisition unit 13 is a processing unit that acquires a round-trip delay time between the own device and each decoder 30. As an example, the acquisition unit 13 transmits a packet instructing measurement of the round-trip delay time to each decoder 30 every time a predetermined period, for example, 5 minutes elapses. In addition, the acquisition unit 13 receives the round trip delay times measured by the decoders 30A to 30C, and overwrites the round trip delay times of the decoders 30A to 30C in the corresponding fields of the RTT storage unit 13a. Thereby, the acquisition unit 13 updates the round-trip delay time of each decoder 30 stored in the RTT storage unit 13a. Although the case where the acquisition unit 13 mainly collects the round-trip delay time is illustrated here, the decoder 30 may measure the round-trip delay time of the packet retransmitted from the encoder 10 and upload it to the encoder 10. Good.

  Here, the determination method of the determination time of the FEC data by the generation unit 12 will be described with reference to FIGS. Each time the generation unit 12 generates FEC data from video data, the generation unit 12 determines the determination time of the FEC data using the generation time of the FEC data and the round-trip delay time stored in the RTT storage unit 13a. 5 and 6 indicate the maximum round-trip delay time among the decoders 30A to 30C.

  As shown in FIG. 3, when the FEC data of FEC number 1 is generated at the generation time “2010/06/01 10: 00: 00.123”, the generation unit 12 includes the decoder 30 shown in FIG. The round trip delay time “120 (ms)” of the decoder 30C having the maximum value among the round trip delay times is added to the generation time. Accordingly, the generation unit 12 calculates the determination time “2010/06/01 10: 00: 00.243” illustrated in FIG. As shown in FIG. 4, when the FEC data of FEC number 2 is generated at the generation time “2010/06/01 10: 00: 00.678”, the generation unit 12 displays each decoder shown in FIG. The round trip delay time “73 (ms)” of the decoder 30B having the maximum value among the 30 round trip delay times is added to the generation time. Accordingly, the generation unit 12 calculates the determination time “2010/06/01 10: 00: 00.751” illustrated in FIG.

  As described above, the generation unit 12 determines the determination time of the FEC data by adding the maximum round-trip delay time among the round-trip delay times of each decoder 30 stored in the RTT storage unit 13a to the FEC data generation time. To do. Accordingly, the generation unit 12 prevents a packet from being retransmitted by the second transmission unit 17 (to be described later) when a retransmission request cannot be received from the decoders 30A to 30C. Furthermore, the generation unit 12 prevents the time from when a retransmission request is received from the decoders 30A to 30C until the packet is retransmitted from being excessively extended.

  Returning to the description of FIG. 2, the retransmission request storage unit 14 a is a storage unit that stores, for each decoder 30, whether or not there is a retransmission request regarding video data whose FEC data is to be restored. As an example, the retransmission request storage unit 14a generates a later-described reception status storage unit 15a for determining the presence / absence of a packet to be transmitted to a second transmission unit 17, which will be described later, and a breakdown of the packet when the packet is transmitted. Therefore, it is referred to by the determination unit 15 described later. In the retransmission request storage unit 14a, information indicating that when a retransmission request is received from the decoder 30, is registered by the receiving unit 14 described later.

  As one aspect, the retransmission request storage unit 14a stores data in which the IP address of the decoder 30 is associated with the presence / absence of a retransmission request for each video number. FIG. 7 is a diagram illustrating a configuration example of information stored in the retransmission request storage unit. In the example of FIG. 7, “0” stored in the video number field indicates that a retransmission request has not been received from the decoder 30, and “1” stored in the video number field represents the decoder 30. Indicates that a retransmission request has been accepted. In the example of FIG. 7, as in FIGS. 5 and 6, the IP address of the decoder 30A is “10.0.0.1”, the IP address of the decoder 30B is “10.0.0.2”, and the decoder 30C It is assumed that the IP address is “10.0.0.3”.

  In the example shown in FIG. 7, this indicates that a request for retransmission of video data of video number 1 and video number 3 has been received from the decoder 30A. Furthermore, it indicates that a request for retransmission of video data of video number 2 has been received from the decoder 30B. Furthermore, it indicates that a request for retransmission of video data of video number 1 has been received from the decoder 30C. Note that the initial value of each video number field is “0”, and “0” is updated to “1” when a retransmission request is received by the receiving unit 14 described later.

  The receiving unit 14 is a processing unit that receives video data retransmission requests from the decoders 30A to 30C. As an example, when receiving a retransmission request from the decoder 30, the reception unit 14 out of the fields stored in the retransmission request storage unit 14a and the video 30 of the video data for which the retransmission request was made and the decoder 30 that made the retransmission request. The field corresponding to the number is updated from “0” to “1”.

  The reception status storage unit 15a is a storage unit that stores, for each FEC data, a reception status of a retransmission request from the decoder 30 regarding video data that is to be restored by the FEC data. As an example, the reception status storage unit 15a is referred to by the determination unit 15 described later in order to determine the presence / absence of a packet to be transmitted to the second transmission unit 17 described later, and the breakdown of the packet when transmitting. . The reception status storage unit 15a is generated by the determination unit 15 described later using the presence / absence of a retransmission request for each video number stored for each decoder 30 by the retransmission request storage unit 14a.

  As an aspect, the reception status storage unit 15a stores data in which the FEC number of the FEC data is associated with the retransmission request total value for each video data to be restored. The “retransmission request total value” here refers to the total value of the retransmission requests received from the decoders 30A to 30C for one video data.

  FIG. 8 is a diagram illustrating a configuration example of information stored in the reception status storage unit. The value stored in the video number field shown in FIG. 8 indicates the total retransmission request value. In the example shown in FIG. 8, among the video data of video numbers 1 to 4 to be restored by the FEC data of FEC number 1, the encoder 10 receives the retransmission request of video number 1 from the two decoders 30. Indicates. Furthermore, it shows that the encoder 10 has received a retransmission request for the video number 2 from one decoder 30. Furthermore, it shows that the encoder 10 has received a retransmission request for the video number 3 from one decoder 30.

  The determination unit 15 determines the presence / absence of a packet to be retransmitted by a second transmission unit 17 (to be described later) from the video data multicast by the first transmission unit 16 (to be described later). Is a processing unit.

  Explaining this, the determination unit 15 monitors whether or not the current time obtained from a time stamp (not shown) has reached the determination time of the FEC data stored in the FEC information storage unit 12b. In the example illustrated in FIG. 3, the determination unit 15 waits until the determination time “2010/06/01 10: 00: 00.243” of the FEC data with the FEC number 1 is reached, and the current time is determined as the determination time “2010/06. The following processing is started when / 01 10: 00: 00.243 "is reached.

  That is, the determination unit 15 uses the retransmission request storage unit 14a to retransmit each video number in the reception status storage unit 15a when the current time obtained from a time stamp (not shown) is the FEC data determination time. Generate the requested total value.

  Here, the case where the FEC data of FEC number 1 generates retransmission request total values of video numbers 1 to 4 to be restored will be described with reference to FIGS. 7 and 8. Among these, for the video data of video number 1, while the retransmission request is received from the decoder 30A and the decoder 30C, the retransmission request is not received from the decoder 30B. Therefore, the determination unit 15 writes the retransmission request total value “2” of the video number 1 calculated by totaling the retransmission requests of the decoders 30A to 30C in the reception status storage unit 15a. For the video data of video number 2, while the retransmission request is received from the decoder 30B, the retransmission request is not received from the decoder 30A and the decoder 30C. Therefore, the determination unit 15 writes the retransmission request total value “1” of the video number 2 calculated by totaling the retransmission requests of the decoders 30A to 30C in the reception status storage unit 15a. For the video data of video number 3, a retransmission request is received from the decoder 30A, but no retransmission request is received from the decoder 30B and the decoder 30C. Therefore, the determination unit 15 writes the retransmission request total value “1” of the video number 3 calculated by totaling the retransmission requests of the decoders 30A to 30C in the reception status storage unit 15a. For the video data of video number 4, no retransmission request is received from any of the decoders 30A to 30C. Therefore, the determination unit 15 writes the retransmission request total value “0” of the video number 4 calculated by totaling the retransmission requests of the decoders 30A to 30C in the reception status storage unit 15a.

  Then, the determination unit 15 determines whether or not the logical sum of the retransmission request total values for each video number is greater than zero. That is, the determination unit 15 determines whether or not at least one of the retransmission request total values for each video number is greater than zero. In the example of FIG. 8, since the total retransmission request values for video number 1, video number 2 and video number 3 are all greater than zero, the logical sum of the retransmission request total values for the respective video numbers is determined by the determination unit 15 to be zero. Is also determined to be large.

  At this time, when the logical sum of the retransmission request total values for each video number is larger than zero, the determination unit 15 decrements the retransmission request total value for each video number one by one. FIG. 9 is a diagram for explaining the decrement of the retransmission request total value. As illustrated in FIG. 9, the determination unit 15 decrements the retransmission request total value “2” of video number 1 to be restored by the FEC data of FEC number 1 and updates it to “1”. Further, the determination unit 15 decrements the retransmission request total value “1” of the video number 2 to be restored by the FEC data of the FEC number 1 and updates it to “0”. Further, the determination unit 15 decrements the retransmission request total value “1” of the video number 3 to be restored by the FEC data of the FEC number 1 and updates it to “0”. Further, the determination unit 15 decrements the retransmission request total value “0” of the video number 4 to be restored by the FEC data of the FEC number 1 and updates it to “−1”.

  Thereafter, the determination unit 15 determines whether or not the logical sum of the retransmission request total value of each video number after decrement is zero. That is, the determination unit 15 determines whether all the retransmission request total values of the video numbers after the decrement are zero. In the example of FIG. 9, since the retransmission request total value for video number 1 is greater than zero, the determination unit 15 determines that the logical sum of the retransmission request total values for each video number after decrementing is greater than zero. .

  The first transmission unit 16 is a processing unit that transmits the encoded data stored in the video buffer 11a to the decoders 30A to 30C. The first transmission unit 16 sets the multicast address assigned to the multicast group in which the decoders 30 </ b> A to 30 </ b> C participate in the header of the video data, and transmits the video data to the network 3. Thereby, the first transmission unit 16 multicasts the video data to the decoders 30A to 30C.

  The second transmission unit 17 is a processing unit that transmits packets necessary for reproducing a video stream by the decoder 30 using the video buffer 11a and the FEC buffer 12a.

  For example, when the logical sum of the retransmission request total values of each video number after decrement is zero, video data in which an error is detected on the decoder 30 side can be restored by transmitting only FEC data to the decoders 30A to 30C. Therefore, the second transmission unit 17 takes out only the FEC data from the FEC buffer 12 a and multicasts it to the decoder 30.

  Further, when the logical sum of the retransmission request total values of the respective video numbers after the decrement is larger than zero, even if only the FEC data is transmitted to the decoders 30A to 30C, all the video data in which an error is detected on the decoder 30 side. It cannot be restored. Therefore, the second transmission unit 17 multicasts the FEC data to the decoder 30 and multicasts the encoded data to the decoder 30. At this time, the second transmission unit 17 may extract the encoded data corresponding to the video number whose retransmission request total value after decrement is larger than zero from the video buffer 11a and transmit it.

  When the logical sum of the retransmission request total values of the video numbers before decrement is zero, there is no video data in which an error is detected on the decoder 30 side. Therefore, the second transmission unit 17 does not transmit both FEC data and encoded data.

  As described above, even when the video stream is multicast, only the data packet is distributed in advance, and even when the packet necessary for reproducing the video stream is subsequently retransmitted, the FEC data and the encoded data for retransmission are decoded 30. It is possible to prevent the situation that is not in time for the re-creation. As an example, parameters such as the interval at which the encoder 10 inserts the FEC packet into the data packet and the buffering time from when the decoders 30A to 30C receive the video data to playback are appropriately set on the decoder 30 side or the encoder 10 side. You only have to set it.

  Note that various types of integrated circuits and electronic circuits can be employed for the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, the first transmission unit 16, and the second transmission unit 17. For example, an ASIC (Application Specific Integrated Circuit) is an example of the integrated circuit. Examples of the electronic circuit include a central processing unit (CPU) and a micro processing unit (MPU).

  Further, a semiconductor memory element or a storage device can be adopted for the FEC information storage unit 12b, the RTT storage unit 13a, the retransmission request storage unit 14a, and the reception status storage unit 15a. For example, examples of the semiconductor memory element include a video random access memory (VRAM), a random access memory (RAM), a read only memory (ROM), and a flash memory. Examples of the storage device include storage devices such as a hard disk and an optical disk.

[Process flow]
Next, a processing flow of the encoder 10 according to the present embodiment will be described. Here, after (1) the first transmission process executed by the first transmission unit 16 is described, (2) the second transmission process executed by the second transmission unit 17 is described. And

(1) First Transmission Processing FIG. 10 is a flowchart illustrating a procedure of first transmission processing according to the first embodiment. This first transmission process is a process that is repeatedly executed when the power supply of the encoder 10 is ON, and the process is started each time video data is input from the camera 20.

  As shown in FIG. 10, when video data is input from the camera 20 (Yes at Step S101), the encoding unit 11 encodes the video data input from the camera 20 (Step S102). Subsequently, the encoding unit 11 stores the encoded data obtained by encoding the video data in the video buffer 11a (step S103).

  At this time, when encoded data of a predetermined number of frames is accumulated in the video buffer 11a (Yes in step S104), the generation unit 12 takes out video data continuous over a predetermined number of frames from the video buffer 11a and obtains FEC data. Generate (step S105). If encoded data of a predetermined number of frames is not stored in the video buffer 11a (No at Step S104), the process proceeds to Step S108 without generating FEC data.

  Then, the generation unit 12 stores the FEC data generation time in the FEC information storage unit 12b together with the FEC number of the FEC data generated in step S105, the start video number of the video data used to generate the FEC data, and the end video number. It sets (step S106).

  Subsequently, the generation unit 12 sets the FEC data determination time determined based on the generation time of the FEC data and the maximum round-trip delay time stored in the RTT storage unit 13a in the FEC information storage unit 12b ( Step S107).

  Then, the first transmission unit 16 multicasts the encoded data stored in the video buffer 11a to the decoders 30A to 30C (step S108), and proceeds to the above step S101. After that, every time video data is input from the camera 20, the encoder 10 recursively repeats the processes in steps S102 to S108.

(2) Second Transmission Processing FIG. 11 is a flowchart illustrating a procedure of second transmission processing according to the first embodiment. This second transmission process is a process that is repeatedly executed when the power supply of the encoder 10 is in the ON state, and is activated whenever the time reaches the determination time of the FEC data stored in the FEC information storage unit 12b. To do.

  As shown in FIG. 11, when the time becomes the determination time of the FEC data (Yes in step S301), the determination unit 15 uses the retransmission request storage unit 14a to request retransmission for each video number in the reception status storage unit 15a. A total value is generated (step S302).

  Then, the determination unit 15 determines whether or not the logical sum of the retransmission request total values for each video number is greater than zero (step S303). At this time, when the logical sum of the retransmission request total values of the respective video numbers is zero (No in step S303), there is no video data in which an error is detected on the decoder 30 side. Accordingly, neither the FEC data nor the encoded data is transmitted, and the process proceeds to step S301.

  On the other hand, when the logical sum of the retransmission request total values for each video number is greater than zero (Yes at Step S303), the determination unit 15 decrements the retransmission request total value for each video number one by one (Step S304). . Subsequently, the determination unit 15 determines whether or not the logical sum of the retransmission request total value of each video number after decrement is zero (step S305).

  Here, when the logical sum of the retransmission request total values of each video number after decrement is zero (Yes in step S305), an error is detected on the decoder 30 side as long as FEC data is transmitted to the decoders 30A to 30C. Video data can be restored. Therefore, the second transmission unit 17 extracts only the FEC data from the FEC buffer 12a and multicasts it to the decoder 30 (step S306), and initializes the retransmission request storage unit 14a and the reception status storage unit 15a (step S307). ) The process proceeds to step S301.

  On the other hand, when the logical sum of the retransmission request total values of each video number after decrement is larger than zero (No in step S305), an error is detected on the decoder 30 side even if only the FEC data is transmitted to the decoders 30A to 30C. Cannot restore all the video data. Therefore, the second transmission unit 17 multicasts the FEC data to the decoder 30 and multicasts the encoded data to the decoder 30 (step S308). Thereafter, the second transmission unit 17 initializes the retransmission request storage unit 14a and the reception status storage unit 15a (step S307), and proceeds to step S301.

[Effect of Example 1]
As described above, when the video stream is multicast, the encoder 10 according to the present embodiment distributes only the data packet in advance and retransmits the packet necessary for reproducing the video stream. For this reason, the encoder 10 according to the present embodiment does not transmit the packet to the distribution destination decoders 30A to 30C when the packet retransmitted from the encoder 10 is not necessary for reproducing the video stream. Therefore, according to the encoder 10 according to the present embodiment, it is possible to efficiently retransmit the video data.

  Furthermore, the encoder 10 according to the present embodiment waits for the maximum round-trip delay time with the distribution destination decoders 30A to 30C, and then retransmits a packet necessary for reproducing the video stream. For this reason, in the encoder 10 according to the present embodiment, it is possible to retransmit when it is estimated that all the retransmission requests of the decoders 30A to 30C have arrived. Therefore, according to the encoder 10 according to the present embodiment, it is possible to prevent a situation in which some of the decoders 30 are not retransmitted by retransmitting a packet without receiving a retransmission request from the decoders 30A to 30C. In addition, according to the encoder 10 according to the present embodiment, it is possible to prevent a delay in packet retransmission by excessively extending the time from when a retransmission request is received from the decoders 30A to 30C until the packet is retransmitted. it can.

  Also, the encoder 10 according to the present embodiment does not transmit a packet when a retransmission request for a data packet to be restored by the FEC packet is not accepted. FIG. 12 is a diagram illustrating an example of a packet reception order in the decoder. In the example of FIG. 12, it is assumed that the encoder 10 transmits data packets of video data in the order of video number 1 to video number N. As illustrated in FIG. 12, the decoders 30 </ b> A to 30 </ b> C normally receive all the data packets of video number 1 to video number 4 that are to be restored by the FEC packet of FEC number 1. In this case, neither the FEC packet nor the data packet is retransmitted by the second transmission unit 17 of the encoder 10. Therefore, it is possible to prevent the resources of the network 3 and the resources of the encoder 10 and the decoders 30A to 30C from being wasted due to the FEC packet and the data packet.

  Further, the encoder 10 according to the present embodiment transmits the FEC packet when the number of data packets for which a retransmission request has been received is a number that can be restored using the FEC packet. FIG. 13 is a diagram illustrating an example of a packet reception order in the decoder. In the example of FIG. 13, as in the example of FIG. 12, it is assumed that the encoder 10 transmits data packets of video data in the order of video number 1 to video number N. In the example of FIG. 13, a packet in which an error is detected by the decoder 30 is represented by hatching, and an FEC packet transmitted by the second transmission unit 17 is represented by black. As shown in FIG. 13, in the decoder 30 </ b> A, an error is detected in the data packet of the video number 3 among the data packets of the video number 1 to the video number 4 to be restored by the FEC packet of the FEC number 1. In the decoder 30B, an error is detected in the data packet of video number 2. In the decoder 30C, an error is detected in the data packet of video number 1. In this case, an error in the data packet is detected as many times as the number of FEC packets of FEC number 1 that can be restored by each decoder 30. Therefore, only the FEC packet with the FEC number 1 is multicast to each decoder 30 by the second transmission unit 17 of the encoder 10. Therefore, the number of packets sent to the network 3 can be kept to a minimum.

  In addition, when the number of data packets that have received a retransmission request is an unrecoverable number using FEC packets, the encoder 10 according to the present embodiment transmits the FEC packets and retransmits the data packets. FIG. 14 is a diagram illustrating an example of a packet reception order in the decoder. In the example of FIG. 14, as in the example of FIG. 13, it is assumed that the encoder 10 transmits data packets of video data in the order of video number 1 to video number N. In the example of FIG. 14, a packet in which an error is detected by the decoder 30 is represented by hatching, the FEC packet transmitted by the second transmission unit 17 is represented by black, and the second transmission unit The data packet retransmitted by 17 is represented by a dot fill. As shown in FIG. 14, in the decoder 30A, an error is detected in the data packets of the video number 1 and the video number 3 among the data packets of the video number 1 to the video number 4 to be restored by the FEC packet of the FEC number 1. . In the decoder 30B, an error is detected in the data packet of video number 2. In the decoder 30C, an error is detected in the data packet of video number 1. In this case, the decoder 30A detects a number of errors in which the FEC packet with FEC number 1 cannot be restored. For this reason, the second transmission unit 17 of the encoder 10 multicasts the data packet of video number 1 together with the FEC packet of FEC number 1 to each decoder 30. Therefore, the data packets of the video number 1 and the video number 3 in which the error is detected by the decoder 30A can be reproduced.

  Therefore, according to the encoder 10 according to the present embodiment, packet retransmission can be appropriately performed according to the above three situations.

  Although the embodiments related to the disclosed apparatus have been described above, the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, another embodiment included in the present invention will be described below.

[Distribution and integration of functions]
In addition, each component of each illustrated apparatus does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, the first transmission unit 16, or the second transmission unit 17 is connected as an external device of the encoder 10 via a network. May be. In addition, the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, the first transmission unit 16, or the second transmission unit 17 are respectively provided by different devices and connected to the network for cooperation. The function of the encoder 10 may be realized by operating.

[Video distribution program]
The various processes described in the above embodiments can be realized by executing a prepared program on a computer such as a personal computer or a workstation. In the following, an example of a computer that executes a video distribution program having the same function as that of the above embodiment will be described with reference to FIG.

  FIG. 15 is a schematic diagram illustrating an example of a computer that executes a video distribution program according to the second embodiment. As illustrated in FIG. 15, the computer 100 according to the second embodiment includes an operation unit 110 a, a microphone 110 b, a camera 110 c, a display 120, and a communication unit 130. The computer 100 further includes a CPU 150, a ROM 160, an HDD (Hard Disk Drive) 170, and a RAM (Random Access Memory) 180. These units 110 to 180 are connected via a bus 140.

  As illustrated in FIG. 15, the HDD 170 includes the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, and the first transmission unit illustrated in the first embodiment. 16 and a video distribution program 170a that performs the same function as the second transmission unit 17 is stored in advance. Regarding the video distribution program 170a, the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, the first transmission unit 16, and the second transmission unit 17 illustrated in FIG. Like each component, you may integrate or isolate | separate suitably. In other words, all data stored in the HDD 170 need not always be stored in the HDD 170, and only data necessary for processing may be stored in the HDD 170.

  Then, the CPU 150 reads the video distribution program 170 a from the HDD 170 and develops it in the RAM 180. Thereby, as shown in FIG. 15, the video distribution program 170a functions as a video distribution process 180a. The video distribution process 180a expands various data read from the HDD 170 in an area allocated to itself on the RAM 180 as appropriate, and executes various processes based on the expanded various data. The video distribution process 180a is performed by, for example, the encoding unit 11, the generation unit 12, the acquisition unit 13, the reception unit 14, the determination unit 15, the first transmission unit 16, and the second transmission unit 17 illustrated in FIG. For example, the process shown in FIG. It should be noted that all the processing units virtually realized on the CPU 150 do not always have to operate on the CPU 150, and only the processing units necessary for the processing need only be virtually realized.

  Note that the video distribution program 170a is not necessarily stored in the HDD 170 or the ROM 160 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk inserted into the computer 100, so-called FD, CD-ROM, DVD disk, magneto-optical disk, or IC card. Then, the computer 100 may acquire and execute each program from these portable physical media. Each program is stored in another computer or server device connected to the computer 100 via a public line, the Internet, a LAN, a WAN, etc., and the computer 100 acquires and executes each program from these. It may be.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Supplementary note 1) a first transmission unit that transmits a video stream in which a multicast address is set;
An acquisition unit that acquires a round trip delay time between a plurality of receiving terminals included in the multicast address and the own device;
A reception unit that receives a retransmission request of a data packet included in the video stream from the plurality of terminals;
Necessary to reproduce the video stream at the terminal after at least the maximum round-trip delay time of the round-trip delay time acquired by the acquisition unit from the transmission time of the data packet for which the retransmission request has been received by the reception unit. And a second transmitter for transmitting a packet.

(Additional remark 2) It further has the production | generation part which produces | generates the packet for forward error correction from the data packet which continues over the predetermined number of packets among the said video streams,
The second transmission unit does not transmit a packet when a retransmission request for a data packet to be restored by the forward error correction packet is not received, and the number of data packets for which the retransmission request is received Is a number that can be restored using the forward error correction packet, the forward error correction packet is transmitted, and the number of data packets for which the retransmission request has been accepted is the forward error correction packet. The video distribution device according to appendix 1, wherein when the number is unrecoverable, the forward error correction packet is transmitted and the data packet is retransmitted.

(Supplementary note 3)
Send a video stream with a multicast address,
Obtaining a round trip delay time between a plurality of receiving terminals included in the multicast address and the own device;
Receiving a retransmission request for a data packet included in the video stream from the plurality of terminals,
A process of transmitting a packet necessary for reproducing a video stream at the terminal after at least the maximum round-trip delay time of the round-trip delay time has elapsed from the transmission time of the data packet for which the retransmission request has been accepted. A video distribution method characterized by the above.

(Appendix 4) The computer
Further executing a process of generating a forward error correction packet from data packets continuous over a predetermined number of packets in the video stream;
As a process of transmitting a packet necessary for reproducing a video stream at the terminal, a packet is not transmitted when a retransmission request for a data packet to be restored by the forward error correction packet is not accepted. When the number of data packets for which the retransmission request has been accepted is a number that can be restored using the forward error correction packet, the forward error correction packet is transmitted, and the retransmission request has been accepted. When the number of data packets is an unrecoverable number using the forward error correction packet, the forward error correction packet is transmitted and a process of resending the data packet is executed. 4. The video distribution method according to 3.

(Appendix 5)
Send a video stream with a multicast address,
Obtaining a round trip delay time between a plurality of receiving terminals included in the multicast address and the own device;
Receiving a retransmission request for a data packet included in the video stream from the plurality of terminals,
Causing the terminal to transmit a packet necessary for reproducing the video stream after at least the maximum round-trip delay time of the round-trip delay time has elapsed from the transmission time of the data packet for which the retransmission request has been accepted. A video distribution program characterized by this.

1 video distribution system 3 network 10 encoder 11a video buffer 11 encoding unit 12a FEC buffer 12b FEC information storage unit 12 generation unit 13a RTT storage unit 13 acquisition unit 14a retransmission request storage unit 14 reception unit 15a reception status storage unit 15 determination unit 16 1st transmission part 17 2nd transmission part 20 Camera 30A, 30B, 30C Decoder 40A, 40B, 40C Display apparatus

Claims (3)

A first transmitter that transmits a video stream in which a multicast address is set;
An acquisition unit that acquires a round trip delay time between a plurality of receiving terminals included in the multicast address and the own device;
A reception unit that receives a retransmission request of a data packet included in the video stream from the plurality of terminals;
Necessary to reproduce the video stream at the terminal after at least the maximum round-trip delay time of the round-trip delay time acquired by the acquisition unit from the transmission time of the data packet for which the retransmission request has been received by the reception unit. And a second transmitter for transmitting a packet. A generation unit for generating a packet for forward error correction from data packets continuous over a predetermined number of packets in the video stream;
The second transmission unit does not transmit a packet when a retransmission request for a data packet to be reproduced by the forward error correction packet is not received, and the number of data packets for which the retransmission request is received Is a number that can be restored using the forward error correction packet, the forward error correction packet is transmitted, and the number of data packets for which the retransmission request has been accepted is the forward error correction packet. The video distribution apparatus according to claim 1, wherein when the number is unrecoverable, the forward error correction packet is transmitted and the data packet is retransmitted. Computer
Send a video stream with a multicast address,
Obtaining a round trip delay time between a plurality of receiving terminals included in the multicast address and the own device;
Receiving a retransmission request for a data packet included in the video stream from the plurality of terminals,
A process of transmitting a packet necessary for reproducing a video stream at the terminal after at least the maximum round-trip delay time of the round-trip delay time has elapsed from the transmission time of the data packet for which the retransmission request has been accepted. A video distribution method characterized by the above.

Images