JP2005051299A - Packet transmission apparatus, packet reception apparatus, packet transmission method and packet reception method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide packet transmission apparatus for efficiently transmitting multimedia contents through a network wherein it is possible to intrude errors to packets, and to provide packet reception apparatus, a packet transmission method, a packet reception method and a program. <P>SOLUTION: When a retransmission request reception section 107 receives a retransmission request from a receiver, a retransmission control section 108 determines the importance of the request in the light of whether or not data of the retransmitted packet are in time for the utilization of the data by the receiver when the packet is retransmitted, the importance being related to retransmission of the packet with respect to the request. The request is stored in a retransmission request queue section 110 together with the importance. The retransmission control section 109 selects the packet to be retransmitted on the basis of the importance when a plurality of packets to be retransmitted exist. A retransmission packetizing section 111 assembles the selected packet into the retransmission packet, which is transmitted to the receiver from a retransmission packet transmission section 113 via a retransmission packet queue section 112. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a packet transmission device that transmits a packet related to an encoded moving image / still image using a wired communication network such as ISDN or the Internet or a wireless communication network such as a mobile phone or a wireless LAN, and receives the packet. The present invention relates to a packet reception device, a packet transmission method, and a packet reception method.
[0002]
[Prior art]
In recent years, with the development of digital coding technology for various information such as images and broadband network technology, development of applications using these has become active, and a system for transmitting compression-coded images etc. using a communication network Has been developed.
[0003]
In particular, with the widespread use of the Internet and Intranet, applications and systems that transmit and receive data in packets are increasing. Packetization is an effective means for efficiently sharing the bandwidth of a communication path among a plurality of users. TCP / IP, UDP / IP, and the like exist as protocols for transmitting and receiving packet data. TCP / IP is resistant to errors because it incorporates a framework such as retransmission, and is effective for download-type applications that want to receive data correctly even if it takes some time. UDP / IP does not have a retransmission framework, but has no delay for retransmission and is effective for applications that require real-time performance.
[0004]
In moving image communication, image data usually has a very large amount of data and often does not fit in the network bandwidth. In this case, a method is used in which image data is encoded and transmitted with a reduced amount of data. As compression coding techniques for moving image signals, techniques such as motion compensation, discrete cosine transform (DCT), subband coding, pyramid coding, variable length coding, etc., and methods combining these techniques have been developed. International standards for video coding include ISO MPEG-1, MPEG-2, ITU-TH. 261, H. 262, H.M. H.263, and ISO MPEG system, ITU-T H.264, and the like as international standard systems for multiplexing moving images, code sequences compressed voice / audio signals, and other data. 221, H.M. 223 exists.
[0005]
The Internet and the like are connected through a myriad of networks, and it is normal not to know which network is in what state. In addition, since the amount of data flowing there fluctuates from moment to moment, a mechanism for determining how much data can be communicated in real time is necessary. Therefore, an application that uses a packet format called RTP (Real-time Transport Protocol), which advances one step from a real-time application using UDP / IP and adds time information or the like to a packet for transmission, is increasing. By using RTP, time information and packet number are added to the packet, and the receiving side can display the voice and image using the correct time information, or determine the packet whose order has been changed on the network. By looking at the packet number, it can be detected that the packet is lost. In addition, RTP also has a mechanism (RTCP (RTP Control Protocol)) for notifying the jitter, the packet loss rate, and the like as supplementary information from the transmission side and the reception side. Further, a mechanism for retransmitting RTP packets is currently under discussion. For example, a function is known in which priority is added to a packet and traffic can be controlled so as not to retransmit everything. However, there is no provision for this usage.
[0006]
In addition, in the IETF (The Internet Engineering Task Force), RTSP is defined as a session management protocol between a transmission terminal and a reception terminal for establishing the image communication as described above, and for describing the contents of the image content. SDP, which is the language specification of the Thereby, between the server and the client, it is possible to control the start of the session, the start of transmission of the multimedia content, the pause, the end of the session, etc. by RTSP.
[0007]
Here, when an image is transmitted over the Internet or the like, a band for raw image transmission cannot be secured over the Internet or the like, and therefore, an image signal is compressed and sent using an encoding method such as MPEG. This is effective in reducing the amount of data, but it becomes very vulnerable to packet loss and error mixing caused by data flowing over the unstable Internet. In the moving image coding system, only the difference from the previous frame is transmitted, so that a part of the data is lost. Since UDP and RTP basically do not retransmit data, it is necessary to take measures against this problem. For this reason, standards for RTP retransmission are established by IETF. In simple terms, the packet number that could not be received is notified to the transmission side by a command called NACK using a protocol called AVPF. On the transmission side, a packet lost from NACK is specified, a header is constructed according to the RTP retransmission standard, and the packet is transmitted to the reception side. Furthermore, since RTP is mainly used for real-time communication, a technique has been proposed for controlling packets that are not in time for display even if they are retransmitted (see, for example, Patent Document 1). In addition, since there is a delay until the retransmission packet arrives and there is a possibility that it may not be in time for real-time reproduction, it has been proposed to use retransmission for recording purposes. At that time, a contrivance is made such that the retransmission packet is not transmitted with the same low-reliability UDP as in real time but is transmitted with reliable TCP (for example, see Patent Document 2).
[0008]
[Patent Document 1]
Daiji Ido, et al., “Data Transmitting Device, Data Receiving Device, and Data Communication System”, P2002-84338, Mar. 2002.
[0009]
[Patent Document 2]
Makoto Kobayashi, “Communication control device, communication control method, and recording medium on which communication control program is recorded”, P2002-199019, July 2002.
[0010]
[Problems to be solved by the invention]
In the prior art, RTP feedback information (RTCP), retransmission control (RTP Retransmission), and the like have been proposed and standardized as network protocols, but how to use RTCP and retransmission is presented. Absent. There are known techniques for re-transmission using TCP only for recording purposes and not transmitting retransmission packets that are not in time for real-time playback. However, these are single functions, and playback quality and recording can be performed during recording. There is no method that is effective for both quality. In addition, regarding transmission bandwidth consumption during retransmission, there is a known technology that compensates for the amount of retransmission by not transmitting normal transmissions that are less important, but this can cause loss of original data. As a result, problems such as a decrease in reproduction quality and an increase in the quality of recorded content occur.
[0011]
The present invention has been made in view of the above circumstances, and provides a packet transmission device, a packet reception device, a packet transmission method, and a packet reception method for efficiently transmitting multimedia contents in a network in which errors may be mixed. The purpose is to do.
[0012]
[Means for Solving the Problems]
The packet transmission device according to the present invention includes a first transmission means for sequentially transmitting a plurality of packets generated from multimedia content data to a reception device via a network, and retransmitting the packet from the reception device via the network. Receiving means for receiving a retransmission request including information for specifying a packet to be transmitted, and the importance level regarding retransmission of the packet related to the retransmission request, a first time when data related to the packet is scheduled to be used by the receiving device, and Determining means for determining based on a second time at which data related to a retransmission packet corresponding to the packet is expected to be available in the receiving apparatus; and selecting means for selecting a packet to be retransmitted based on the importance level And a second transmitter that transmits a retransmission packet corresponding to the selected packet to the receiving device via the network. Characterized by comprising and.
[0013]
The packet reception device according to the present invention includes: a first reception unit that sequentially receives a plurality of packets generated from multimedia content data or a retransmission packet corresponding to the plurality of packets from the transmission device via a network; Analyzing a packet received by one receiving means, specifying a packet to be retransmitted based on the result, and a retransmission request including information specifying a packet to be retransmitted to the transmitting apparatus via the network It is characterized by comprising transmitting means for transmitting, data using means for using data relating to the packet, and recording means for recording data relating to the packet.
[0014]
The present invention relating to the apparatus is also established as an invention relating to a method, and the present invention relating to a method is also established as an invention relating to an apparatus.
Further, the present invention relating to an apparatus or a method has a function for causing a computer to execute a procedure corresponding to the invention (or for causing a computer to function as a means corresponding to the invention, or for a computer to have a function corresponding to the invention. It is also established as a program (for realizing) and also as a computer-readable recording medium on which the program is recorded.
[0015]
According to the present invention, it is possible to efficiently achieve both real-time reproduction and record storage in a system that restores data using retransmission for errors such as packet loss.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the invention will be described with reference to the drawings.
[0017]
In the present embodiment, a case where an RTP (Real Time Transport Protocol) that is a protocol for transferring image data and audio data in real time is used as an example will be described (details of RTP are described in, for example, “Schulzrinne, H., etc.”). Casner, S., Frederick, R. and V. Jacobson "RTP: A Transport Protocol for Real Time Applications", RFC 1889, January 1996.).
[0018]
(First embodiment)
FIG. 1 shows a configuration example of a multimedia content distribution apparatus according to the first embodiment of the present invention.
[0019]
As shown in FIG. 1, this multimedia content distribution apparatus includes a multimedia content storage unit 101, an RTP packetization unit 102, a transmission queue unit 103, a transmission unit 104, a communication path state information reception unit 105, and a round trip time calculation unit. 106, a retransmission request reception unit 107, a packet buffer unit 108, a retransmission control unit 109, a retransmission request queue unit 110, a retransmission packetization unit 111, a retransmission packet queue unit 112, and a retransmission packet transmission unit 113.
[0020]
In FIG. 1, the range indicated by a is a part related to packet transmission, the range indicated by b is a part related to retransmission control, and the range indicated by c is a part related to retransmission packet transmission. (In the configuration diagrams referred to later, a, b, and c are used in the same meaning).
[0021]
Hereinafter, when the multimedia content distribution apparatus (also referred to as a transmission-side apparatus) transmits multimedia content to a multimedia content reception apparatus (also referred to as a reception-side apparatus) (not illustrated) via a network (not illustrated). Will be described. The multimedia content receiving apparatus is not particularly limited, and may be, for example, an existing one or may be described in the sixth embodiment.
[0022]
The content data (131) output from the multimedia content storage unit 101 that holds the multimedia content is RTP packetized by the RTP packetization unit 102. The RTP packet data (132) is temporarily stored in the transmission queue unit 103, and then sequentially transmitted from the transmission unit 104 to the reception side along the network bandwidth. On the other hand, the RTP packet data (132) is stored in the packet buffer unit 108, and is stored for retransmission when the transmitted packet does not reach the receiving apparatus.
[0023]
FIG. 2 shows the configuration of the header portion of the RTP packet. As shown in FIG. 2, the header includes RTP version information (V), padding information (P), presence / absence of extension header (X), number of CSRC (CC), data type (PT), and transmission source identifier. A packet number (Sequence Number) and packet time information (timestamp) are included together with (SSRC) and an associated transmission source identifier (CRSC).
[0024]
Information indicating the state of the communication path (hereinafter referred to as communication path state information) is sent from the receiving side apparatus to the transmitting side apparatus using RTCP or the like. The communication path state information receiving unit 105 receives this communication path state information (134) and sends it to the round trip time calculation unit 106. The time required for data transfer between the transmission side device and the reception side device (hereinafter referred to as round trip time) (138) is determined by the round trip time calculation unit 106 and input to the retransmission control unit 109. In addition to the communication path state information, a retransmission request including information notifying the number of a packet that has not arrived is sent from the receiving side device. The retransmission request receiving unit 107 receives this retransmission request (135) and sends it to the retransmission control unit 109.
[0025]
The retransmission control unit 109 specifies the packet number to be retransmitted from the retransmission request (135), and acquires the corresponding packet data (137) from the packet buffer unit. The retransmission control unit 109 processes a highly important one among the current time, the round trip time (138) determined by the round trip time calculation unit 106, and the unprocessed retransmission requests stored in the retransmission request queue unit 110. In consideration of the time required for processing (hereinafter referred to as required processing time) (140), it is determined whether or not the packet specified in the retransmission request (135) arrives at the receiving side device in time for the reproduction time. . The result determined here is set as the importance of the packet designated by the retransmission request (135) and is temporarily stored in the retransmission request queue unit 110. In addition to the above determination, the retransmission control unit 109 also selects a retransmission request from the retransmission request queue unit 110 in descending order of importance and instructs the packet buffer unit 108 to retransmit.
[0026]
The packet buffer unit 108 sends the instructed packet data (141) to the retransmission packetization unit 111, and the retransmission packetization unit 111 uses the packet data (141) as a retransmission packet ( 142).
[0027]
The generated retransmission packet (142) is temporarily stored in the retransmission packet queue unit 112 and transmitted from the retransmission packet transmission unit 113 to the reception side.
[0028]
FIG. 3 shows an example of a schematic processing procedure for determining the importance of a packet requested to be retransmitted by the retransmission control unit 109.
[0029]
The retransmission control unit 109 determines whether or not the retransmission packet is in time (step S201), and when it is determined that the packet is in time, sets the importance to “high” (step S202) and determines that the packet is not in time. In this case, the importance level is set to “low” (step S203). Based on the determination result, the retransmission request (139) is stored in the retransmission request queue unit 110.
[0030]
FIG. 4 shows an example of the data structure of a retransmission request stored in the retransmission request queue unit 110. In this example, the number of packets to be retransmitted for each retransmission request and their importance are stored in pairs, and the retransmission control unit 109 is configured to be able to process from the most important one. Of course, FIG. 4 is an example, and various other variations are possible. For example, it is possible to hold information such as packet time information and position information in the packet buffer unit 108 and the multimedia content storage unit 101. It is also possible to store the data efficiently by using a method of dividing the list according to the importance or using the pointer to point to the top data of each importance. Also, importance information can be recorded not only in binary values of “high” and “low” but also in multiple values.
[0031]
FIG. 5 shows an example of a more detailed processing procedure for determining the importance of a packet requested to be retransmitted by the retransmission control unit 109.
[0032]
The retransmission control unit 109 identifies the packet number to be retransmitted from the retransmission request (135) (step S401). Note that, for the retransmission request (135), NACK information or the like for notifying packet non-delivery can be used.
[0033]
Using the packet number specified in step S401, packet time information (scheduled reproduction time) is acquired from the packet buffer unit 108 (step S402).
[0034]
In addition to round-trip time information (138) calculated from packet time information and communication path state information, initial buffer amount information indicating the amount of initial buffer prepared in the receiving side device, information such as transmission start time, current time, etc. Obtained from the system, the estimated arrival time when the packet is retransmitted is calculated (step S403).
[0035]
If the scheduled reproduction time of the packet acquired in step S402 is later than the estimated arrival time calculated in step S403 (because it is predicted to be in time for reproduction if it is retransmitted quickly), it is determined that the packet is worth retransmitting immediately. If the estimated arrival time is later than the scheduled reproduction time (because it is predicted that it will not be in time for reproduction even if it is retransmitted immediately), it is determined that the packet need not be retransmitted immediately (step S404).
[0036]
If it is determined that the packet should be retransmitted promptly, the retransmission request for the packet is set to “high” importance, stored in the retransmission request queue unit 110 (step S405), and may not be retransmitted immediately. If determined, the retransmission request for the packet is set to the importance level “low” and stored in the retransmission request queue unit 110 (step S406).
[0037]
Here, FIG. 6 and FIG. 7 show the relationship between the estimated arrival time and the scheduled reproduction time in steps S403 and S404 in FIG. FIG. 6 shows an example in which the estimated arrival time is before the scheduled reproduction time and the importance is set to “high”. On the contrary, FIG. 7 shows that the estimated arrival time is later than the scheduled reproduction time and the importance is set. This is an example when “low” is set.
[0038]
Here, the transmission start time is set to T for the transmission side device. _START , T _SEND , P for the packet time stamp _TS , The time when the retransmission request from the receiving device is received is T _RREQ , The time taken to process a resend request with high importance waiting for processing stored in the resend request queue unit 110 or the like is D _QUE In addition, for the receiving side device, the initial response delay from the start of transmission from the transmitting side device until the receiving side device starts the reproduction operation is α, and the time when the reproduction operation is started is TC _START , D is the initial delay time until the first data is displayed by the reception buffer. _INIT , RTT is the round-trip time for round-trip packet transmission, and T is the time when the retransmission packet arrives based on the retransmission request. _RRCV , T _PLAY The time taken for packet processing is defined as DLSR.
[0039]
As can be seen from FIGS. 6 and 7, the retransmission packet arrival time (that is, the time at which the retransmission packet can be reproduced) T _RRCV Is the scheduled packet playback time T _PLAY If it is earlier than this, it means that the packet should be retransmitted immediately. This can be expressed as a mathematical expression (1). If the formula (1) becomes true, the retransmission packet will be in time for reproduction, and the importance will be set high. If the formula (1) becomes false, the retransmission packet will not be in time. And set the importance level low.
(T _RREQ -T _START ) + D _QUE + RTT / 2 ≦ D _INIT + P _TS ... (1)
The determination formula is determined not only by the above formula (1), but also by a method such as the following formula (2), formula (3), and formula (4), or by other methods. It is also possible.
(T _RREQ -T _START ) + D _QUE + RTT / 2 ≦ D _INIT + P _TS + Α (2)
(3/2) × RTT ≦ D _INIT ... (3)
(3/2) × RTT + DLSR ≦ D _INIT ... (4)
Formula (2) is an example in which the determination is performed in consideration of the initial response delay α, and Formula (3) ignores the effects of the initial delay and cumulative delay, and simply calculates the RTT and the buffer amount on the receiving side. D _INIT This is an example in which the determination is performed only from. Formula (4) is a determination condition formula that takes into account the packet processing time DLSR until the packet that arrives at the receiving side apparatus is processed and issues a retransmission request. To calculate DLSR, Formula (5) An expression can be used.
DLSR = T _RREQ -T _SEND -RTT (5)
Since the retransmission control unit 109 of the transmission side apparatus performs the determination, it is necessary to acquire time information, buffer amount information, and the like of the reception side apparatus in order to use the above formula. Initial delay time D _INIT SDP at the start of the session (see, for example, “M. Handley, et al,“ SDP: Session Description Protocol ”, IETF RFC2327, April 1998.)) or terminal capability exchange, etc. By determining the initial values in step S3, the transmitting side device can know them. The round trip time can be known by using RTCP. Information that cannot be notified by a standard method may be taken by using an application-specific format using an RTCP APP packet (Application Specific Packet).
[0040]
The present embodiment can be implemented with various modifications. Some examples are shown below.
[0041]
Next, FIG. 8 shows another configuration example of the multimedia content distribution apparatus of the present embodiment.
[0042]
In the configuration example of FIG. 1, the retransmission control unit 109 gives an instruction to send the retransmission packet data (141) to the retransmission packetizing unit 111. However, in the configuration example of FIG. Is what I do. In this case, the retransmission control unit 109 only determines the importance level of the retransmission request, and the retransmission request queue unit 110 executes transmission interval control according to the retransmission request. Thereby, it is possible to prevent the configuration of the retransmission control unit 109 from becoming complicated.
[0043]
Next, FIG. 9 shows still another configuration example of the multimedia content distribution apparatus of the present embodiment.
[0044]
In FIG. 9, RTP packetization is not performed in real time, but RTP data is stored in the RTP multimedia content storage unit 801 in advance. The RTP multimedia content storage unit 801 stores the content in a form preliminarily RTP packeted or added with additional information for RTP packetization, and the RTP multimedia content storage unit 801 stores the RTP packet. The converted data (831) is output. This provides the advantage that packet generation need not be performed in real time.
[0045]
Further, the packet buffer unit 108 can be omitted by the retransmission control unit 109 accessing the RTP multimedia content storage unit 801 directly. In addition, the packet buffer unit 108 may not store actual data, but may store only a pointer to data in the RTP multimedia content storage unit 801. By performing such a modification, it becomes possible to omit the packet buffer unit and simplify the distribution device and the distribution program, or to reduce the memory amount required for the packet buffer unit by using a pointer. .
[0046]
In these cases, the RTP packetizing unit may not be provided.
[0047]
In addition, the multimedia content storage unit 101, the RTP packetization unit 102, and the RTP multimedia content storage unit 801 are all provided, and a function of performing RTP packetization in real time and transmitting the RTP packet stored in advance is transmitted. Both functions may be used as appropriate.
[0048]
By the way, in this embodiment, the estimated arrival time is calculated from the delay time of the network, packet time information, reception side buffer information, transmission start time, initial delay time, and the like.
[0049]
When the estimated arrival time is calculated in the transmission side apparatus, the packet time information can be obtained from the packet data stored in the packet buffer unit 108. Further, the information on the reception-side buffer amount and the initial delay time are values that are known in advance by the transmission-side device, for example, as system prescribed values. The transmission side device also has a transmission start time. The network delay time information is a value obtained from the exchange of packet data with the receiving side device.
[0050]
This network delay time information can be calculated using, for example, a control packet called RTCP and using time information included therein.
[0051]
10 and 11 show the configuration of the RTCP packet. FIG. 10 shows a transmission side report sent from the transmission side apparatus to the reception side apparatus, and FIG. 11 shows a reception side report sent from the reception side apparatus to the transmission side apparatus.
[0052]
Here, a method for obtaining the delay time of the network using RTCP will be described.
[0053]
The transmitting side device that has received the receiving side report receives the NTP time T at which the receiving side report has been received. _r-recv "NTP timestamp of last sender report" T in the receiver report _LSR And “time from the arrival of the last sender report to sending the receiver report” T _DLSR And is calculated as shown in Equation (5).
T _delay = T _r-recv -T _LSR -T _DLSR ... (5)
The value obtained here is the round-trip network delay time T _delay It is.
[0054]
In addition to using the prescribed values as described above for the information on the reception side buffer amount and the initial delay time, there is a method of notifying the reception side device by negotiation when establishing a session.
[0055]
On the other hand, when the estimated arrival time is determined on the receiving device side, the same determination method can be used. In the packet time information, the exact time cannot be determined because the packet is lost. In this case, the packet lost due to the method of using the time information of the packet closest to the target packet lost packet or the closest packet in one direction before or after, the time information of the nearby packet and the packet number, etc. It is possible to use a method for estimating packet time information. The buffer amount and initial delay time of the receiving side device are values held in the receiving side device, and can be used as they are.
[0056]
The network delay time can be calculated by receiving the transmission side report shown in FIG. The NTP time when the sender report was received is T _s-recv And "NTP timestamp" T included in the sender report _s-send And One-way network delay time T _{delay_oneway} Can be obtained by Equation (6).
T _{delay_oneway} = T _s-recv -T _s-send (6)
When calculating the round-trip network delay time, the method of simply doubling, the delay in the direction from the transmission side device to the reception side device, and the delay in the direction from the reception side device to the transmission side device are uniform. In consideration of the case where there is not, there is a method of responding by multiplying a coefficient or adding a value.
[0057]
Further, since the transmission start time cannot be notified by the existing RTCP, it is only necessary to take measures such as replacing it with an alternative value such as 0. In addition to the RTP / RTCP, it is also possible to use a method of notifying the receiving side apparatus from the transmitting side apparatus by negotiation at the time of session establishment.
[0058]
(Second Embodiment)
FIG. 12 shows a configuration example of a multimedia content distribution apparatus according to the second embodiment of the present invention.
[0059]
As shown in FIG. 12, the multimedia content distribution apparatus includes a multimedia content storage unit 101, an RTP packetization unit 102, a transmission queue unit 103, a transmission unit 104, a communication path state information reception unit 105, and a round trip time calculation unit. 106, a retransmission request receiving unit 107, a packet buffer unit 108, a retransmission control unit 109, a retransmission packetizing unit 111, a retransmission packet transmitting unit 113, and a retransmission waiting packet buffer unit 901.
[0060]
In the present embodiment, the retransmission request queue unit 110 and the retransmission packet queue unit 112 of the first embodiment are replaced with a retransmission waiting packet buffer unit 901.
[0061]
The operation when the multimedia content distribution apparatus transmits multimedia content to the multimedia content receiving apparatus via the network will be described below. Below, it demonstrates centering on the point which is different from what was demonstrated in 1st Embodiment.
[0062]
The content data (131) output from the multimedia content storage unit 101 is converted into RTP packets by the RTP packetizing unit 102. The RTP packet data (132) is temporarily stored in the transmission queue unit 103, then sequentially transmitted from the transmission unit 104 to the reception side along the network band, and stored in the packet buffer unit 108 for retransmission. Is done.
[0063]
The communication path state information receiving unit 105 receives the communication path state information (134) sent from the receiving side device and sends it to the round trip time calculation unit 106. The round trip time calculation unit 106 determines the round trip time (138) and inputs it to the retransmission control unit 109.
[0064]
The retransmission request receiving unit 107 receives the retransmission request (135) sent from the receiving side device and sends it to the retransmission control unit 109.
[0065]
The retransmission control unit 109 specifies the packet number to be retransmitted from the retransmission request (135), and acquires the corresponding packet data (137) from the packet buffer unit. In the retransmission control unit 109, the required time for processing the current time, the round-trip time (138), and unprocessed retransmission requests stored in the retransmission-waiting packet buffer unit 901 with high importance ( 932), it is determined whether or not the packet specified in the retransmission request (135) arrives at the receiving apparatus in time for the reproduction time. The result determined here is set as the importance level of the packet designated by the retransmission request (135), and the retransmission packet designation information (136) in which the importance level is added to the designation information of the packet to be retransmitted to the packet buffer unit. ) Is sent.
[0066]
The packet buffer unit 108 sends packet data for retransmission (933) to the retransmission wait packet buffer unit 901 based on the retransmission packet designation information (136). At this time, the importance level information added to the retransmission packet designation information (136) is also sent.
[0067]
In the retransmission wait packet buffer unit 901, packet data to be retransmitted is ordered and temporarily stored in accordance with the importance, and packet data (934) is sequentially output according to an instruction from the retransmission control unit 109.
[0068]
Based on packet data (934), retransmission packetizing section 111 generates retransmission packet (143) according to the packet format for retransmission.
[0069]
The retransmission packet (143) is transmitted from the retransmission packet transmission unit 113 to the reception side device.
[0070]
FIG. 13 shows an example of a storage processing procedure corresponding to the importance level information in the retransmission wait packet buffer unit 901. Here, a case where the degree of importance is indicated by binary values will be described as an example.
[0071]
It is determined whether or not the added importance information is high for the input packet data (933) (step S1001). When the importance is “high”, the packet data (933) is inserted in the order position according to the time stamp of the packet in the packet data list of the importance “high” (step S1002). If the importance is “low”, the packet data (933) is inserted in the order position according to the packet time stamp in the packet data list of the importance “low” (step S1003).
[0072]
It should be noted that even when the importance has a multivalue other than binary, the packet data may be inserted at the sequential position according to each value.
[0073]
Next, a configuration example of the retransmission wait packet buffer unit 901 will be described.
[0074]
FIG. 14 shows an example of a list method in which importance, packet number, and retransmission wait packet data are stored in the retransmission wait packet buffer unit 901 as a set. In this example, when a packet having a high importance level is received, the packet data is inserted so that the packet numbers are in order according to the high priority level. In the example of FIG. 14, the packet number is used as information for order management, but other information such as a packet time stamp may be used. Since this method directly arranges data in order, the configuration can be simplified. In addition, when it becomes a system load to rewrite the subsequent data with the insertion of data, for example, the list of the ones with importance “high” and “low” are divided, and each is followed by It is possible to improve efficiency by taking the form of adding data.
[0075]
FIG. 15 shows an example of the configuration of a retransmission-waiting packet buffer unit 901 that can be arranged in a line from the higher importance level to the lower importance level without insertion or rewriting of data in the middle. In this example, each packet data has a structure having pointers to the previous one and the next one with respect to each packet data in addition to the above-mentioned data such as importance, packet number, packet data, etc. It has become. By having this, the pointer to the head data, and the pointer to the tail data, the data is searched in order from the head, and when the position to be inserted is found, the packet data to which the pointer of the data before and after it is inserted is indicated. And set its own pointer to indicate the previous and next packets. By using the pointer in this way, it is possible to store the data in consideration of the order only by additional writing of data. In the specific example of FIG. 15, when packet data of importance “high” and packet number “# 42” is inserted in the state of (a), the result is as shown in (b).
[0076]
These are merely examples, and any method can be used as long as the packet data can be output to the retransmission packetization unit 111 according to the importance. Further, as described as a modification of the first embodiment, by using the RTP multimedia content storage unit 801 as shown in FIG. 9, the packet data itself is not stored in the buffer, but the content storage unit. It is also possible to store only the pointer to the data in 801 and extract it when packetizing by the retransmission packetizing unit 111.
[0077]
In the first embodiment, data that has been re-packetized is temporarily stored in the re-transmission packet queue unit 112 and then transmitted by the re-transmission packet transmission unit 113. However, the same configuration is adopted in this embodiment. (In the configuration example of FIG. 12, it is considered that the retransmission control unit 109 and the retransmission wait packet buffer unit 901 have a function of outputting data in consideration of the transmission band, and thus the retransmission packet queue unit is omitted. It was explained in the form).
[0078]
(Third embodiment)
FIG. 16 shows a configuration example of a multimedia content distribution apparatus according to the third embodiment of the present invention.
[0079]
As shown in FIG. 16, the multimedia content distribution apparatus includes a multimedia content storage unit 101, an RTP packetization unit 102, a transmission queue unit 103, a transmission unit 104, a communication path state information reception unit 105, and a round trip time calculation unit. 106, a retransmission request reception unit 107, a packet buffer unit 108, a retransmission control unit 109, a retransmission request queue unit 110, and a retransmission packetization unit 111.
[0080]
In FIG. 16, the range indicated by b is a part related to retransmission control as described above, and the range indicated by d is a part related to packet transmission and retransmission packet transmission (each configuration to be referred to later). In the figure, d is used in the same meaning).
[0081]
In the present embodiment, the configuration related to packet transmission and the configuration related to retransmission packet transmission of the first embodiment are integrated.
[0082]
The operation when the multimedia content distribution apparatus transmits multimedia content to the multimedia content receiving apparatus via the network will be described below. Below, it demonstrates centering on the point which is different from what was demonstrated in 1st Embodiment.
[0083]
The content data (131) output from the multimedia content storage unit 101 is converted into RTP packets by the RTP packetizing unit 102. The RTP packet data (132) is temporarily stored in the transmission queue unit 103, then sequentially transmitted from the transmission unit 104 to the reception side along the network band, and stored in the packet buffer unit 108 for retransmission. Is done.
[0084]
The communication path state information receiving unit 105 receives the communication path state information (134) from the receiving side device. The round trip time calculation unit 106 determines the round trip time (138). The retransmission request receiving unit 107 receives a retransmission request (135) from the receiving side device.
[0085]
The retransmission control unit 109 specifies the packet number to be retransmitted from the retransmission request (135), and acquires the corresponding packet data (137) from the packet buffer unit. Whether or not the packet specified in the retransmission request (135) arrives at the receiving apparatus in time for the reproduction time in consideration of the current time, the round trip time (138), and the required processing time (140). Judgment is made. This determination result is set as the importance of the packet designated by the retransmission request (135), and is temporarily stored in the retransmission request queue unit 110. Further, retransmission requests are selected from the retransmission request queue unit 110 in descending order of importance, and the packet buffer unit 108 is instructed to retransmit.
[0086]
The packet buffer unit 108 sends the instructed packet data (141) to the retransmission packetizing unit 111, and generates a retransmission packet (142) according to the retransmission packet format.
[0087]
The retransmission packet (142) is temporarily stored in the transmission queue unit 103 in the form of coexistence with the original packet data, and is transmitted to the reception side device by the transmission unit 104. The transmission queue unit 103 centrally manages the original packet data and the packet data for retransmission, and adjusts the transmission timing.
[0088]
According to the present embodiment, it is possible to make one transmission part, and it is possible to simplify the configuration of the apparatus. In addition, this method is useful for realizing this method with few resources such as a portable terminal.
[0089]
Further, in this embodiment, it is of course possible to use the configuration examples and the modification examples shown in the first embodiment and the second embodiment.
[0090]
As an example, FIG. 17 shows a configuration example when the retransmission wait packet buffer unit 901 (see FIG. 12) is used. In this case, the retransmission packet data (933) to which the importance level is output from the packet buffer unit 108 is input to the retransmission wait packet buffer unit 901. In the retransmission wait packet buffer unit 901, the packet data to be retransmitted is important. The packets are ordered and temporarily stored in accordance with the degree, and packet data 934 is sequentially output in accordance with an instruction from the retransmission control unit 109. Based on the packet data (934), retransmission packetizing section 111 generates retransmission packet (142) according to the retransmission packet format. The retransmitted packet (142) is temporarily stored in the transmission queue unit 103 so as to coexist with the original packet data, and is transmitted to the receiving side by the transmitting unit 104. Here, the retransmission control unit 109 extracts information indicating how many retransmission requests are waiting to be processed from the retransmission waiting packet buffer unit 901 and uses the information as determination material for retransmission control.
[0091]
Next, FIG. 18 shows another configuration example. FIG. 18 shows the same idea as that of the modified example of FIG. 17, but instead of using the retransmission waiting packet buffer unit 901, the transmission queue unit 103 is used instead. In this case, the retransmission packet data (141) output from the packet buffer unit 108 is input to the retransmission packetization unit 111, and the retransmission packet data (142) is generated and then input to the transmission queue unit 103. The retransmission packet (142) is temporarily stored in the transmission queue unit 103 in the form of coexistence with the original packet data, and is transmitted to the reception side device by the transmission unit 104. Here, the retransmission control unit 109 receives information (1532) such as the amount of packet data stored in the transmission queue unit 103, and uses it as a determination material for retransmission control. This makes it possible to implement with a very simple module configuration.
[0092]
(Fourth embodiment)
In the first to third embodiments, various configuration examples have been shown for the part called retransmission control. However, in the fourth embodiment, communication band control (rate control) that takes retransmission into consideration for retransmission control. The case where these are combined will be described.
[0093]
FIG. 19 shows a configuration example of the multimedia content distribution apparatus according to this embodiment.
[0094]
As shown in FIG. 19, the multimedia content distribution apparatus includes a multimedia content storage unit 101, an RTP packetization unit 102, a transmission queue unit 103, a transmission unit 104, a communication path state information reception unit 105, and a round trip time calculation unit. 106, a retransmission request receiving unit 107, a packet buffer unit 108, a retransmission control unit 109, a retransmission request queue unit 110, a retransmission packetization unit 111, a retransmission packet queue unit 112, a retransmission packet transmission unit 113, and a session control unit 1601. . That is, the configuration example of FIG. 19 is obtained by adding a session control unit 1601 to the configuration example of FIG.
[0095]
The operation when the multimedia content distribution apparatus transmits multimedia content to the multimedia content receiving apparatus via the network will be described below. Below, it demonstrates centering on the point which is different from what was demonstrated in 1st Embodiment.
[0096]
Each unit other than the session control unit 1601 is basically the same as in the first embodiment.
[0097]
The session control unit 1601 uses SDP or RTSP (for example, refer to “H. Schulzrinne, et al,“ Real Time Streaming Protocol (RTSP) ”, IETF RFC2326, April 1998.”) or the like to request a session with the receiving apparatus.・ We manage permission. Here, using the transmission band information determined as a result of negotiation between the reception side device and the transmission side device, the transmission queue unit 103, the transmission unit 104, the retransmission control unit 109, the retransmission packet queue unit 112, the retransmission packet transmission unit 113 is controlled. Specifically, each unit transmits data in a manner that protects a designated band. As a method for specifying the bandwidth of the transmission path, a method in which the transmission band used for retransmission is added in advance to the transmission band used by the original data, or the transmission side of the original data and the transmission band for retransmission are used. A method of notifying the transmission band of each of them separately is used.
[0098]
Next, FIG. 20 shows a configuration example in which it is possible to easily follow dynamic band fluctuation by having a stream switching function.
[0099]
In this configuration example, instead of the multimedia content storage unit 101 that holds multimedia content, a switching-compatible multiple multimedia content storage unit 1702 that stores the same content encoded with a plurality of parameters for content switching is provided. The rate control unit 1701 manages this. For the rate control unit 1701, information (1731 and 1733) indicating how many packets are waiting to be transmitted from the transmission queue unit 103, the retransmission packet queue unit 112, and the communication channel state information receiving unit 105, and communication channel errors And delay information (1734). Based on these pieces of information, the content that optimizes the transmission band and parameters of the original data is selected from the switching-compatible multi-media content storage unit 1702. The retransmission control unit 109 also notifies the current transmission status and content switching information 1732 from the rate control unit 1701 and uses it for determining the importance of the retransmission packet. By adopting such a configuration, it is possible to perform rate control corresponding to communication path bandwidth fluctuations and environmental fluctuations.
[0100]
(Fifth embodiment)
The first to fourth embodiments are distributed from content data created in advance. In the fifth embodiment, it is possible to deal with live video input in real time. It is.
[0101]
FIG. 21 shows a configuration example of a multimedia content distribution apparatus according to the fifth embodiment of the present invention.
[0102]
As shown in FIG. 21, the multimedia content distribution apparatus includes an RTP packetization unit 102, a transmission queue unit 103, a transmission unit 104, a channel state information reception unit 105, a round trip time calculation unit 106, and a retransmission request reception unit 107. A packet buffer unit 108, a retransmission control unit 109, a retransmission request queue unit 110, a retransmission packetization unit 111, a retransmission packet queue unit 112, a retransmission packet transmission unit 113, an encoding unit 1801, and a rate control unit 1802. That is, the configuration example of FIG. 21 includes an encoding unit 1801 and a rate control unit 1802 instead of the multimedia content storage unit 101 of the configuration example of FIG.
[0103]
The operation when the multimedia content distribution apparatus transmits multimedia content to the multimedia content receiving apparatus via the network will be described below. Below, it demonstrates centering on the point which is different from what was demonstrated in 1st Embodiment.
[0104]
In the configuration example of FIG. 1 of the first embodiment, the content data (131) output from the multimedia content storage unit 101 that holds the multimedia content is RTP packetized by the RTP packetization unit 102. In the configuration example of FIG. 21, the input real-time video data (1831) is encoded by the encoding unit 1801, and the encoded data (1832) is RTP packetized by the RTP packetizing unit 102.
[0105]
Further, the rate control unit 1802 acquires the data amount (1835 and 1833) of the transmission waiting packet and the communication path state information (1834) from the transmission queue unit 103, the retransmission packet queue unit 112, and the communication path state information reception unit 105. . Based on these pieces of information, the encoding parameter (1836) of the encoding unit 1801 is determined and notified to the encoding unit 1801. For example, when there are a lot of packet data waiting for transmission in the transmission queue unit 103 and the retransmission packet queue unit 112, the encoding parameter is controlled by increasing the quantization width so as to reduce the generated code amount, Control to lower the rate. Also, considering the state of the communication path, it is possible to perform the same control when the delay becomes large. In contrast, when the amount of data waiting for transmission decreases, control can be performed so that the amount of generated code may be increased.
[0106]
Next, FIG. 22 shows another configuration example of the multimedia content distribution apparatus of the present embodiment. As shown in FIG. 22, by inputting information (1931) such as the determination result and the number of retransmission requests temporarily stored in the retransmission request queue unit 110 from the retransmission control unit 109 to the rate control unit 1802, more detailed information can be obtained. It is also possible to perform rate control. Thereby, it is possible to carry out distribution using the transmission band effectively.
[0107]
Note that a multimedia content storage unit 101, an encoding unit 1801, and a rate control unit 1802 are all provided, and a function for distributing pre-created content data and a function for distributing live video input in real time are provided. Both functions may be used as appropriate.
[0108]
Next, an example in which the concept of sharing the transmission part shown in the third embodiment is applied to the case of real-time data input will be shown. FIG. 23 shows an example of the configuration in this case.
[0109]
In the configuration example of FIG. 16 of the third embodiment, the content data (131) output from the multimedia content storage unit 101 that holds the multimedia content is RTP packetized by the RTP packetization unit 102. 23, the input real-time video data (1831) is encoded by the encoding unit 1801, and the encoded data (1832) is RTP packetized by the RTP packetizing unit 102.
[0110]
Further, the rate control unit 1802 obtains the data amount (1835) and communication channel state information (1834) of the transmission waiting packet from the transmission queue unit 103 and the communication channel state information receiving unit 105. Based on these pieces of information, the encoding parameter (1836) of the encoding unit 1801 is determined and notified to the encoding unit 1801.
[0111]
Of the various configuration examples described in the first to fourth embodiments, configurations corresponding to live video input in real time according to the present embodiment are applicable in addition to those described here.
[0112]
(Sixth embodiment)
FIG. 24 shows a configuration example of a multimedia content receiving apparatus according to the sixth embodiment of the present invention.
[0113]
As shown in FIG. 24, the multimedia content receiving apparatus includes a receiving unit 2101, a packet analyzing unit 2102, a communication channel state calculating unit 2103, a communication channel state information transmitting unit 2104, a retransmission control unit 2105, and a retransmission request transmitting unit 2106. A packet buffer unit 2107, a decoding unit 2108, a display unit 2109, a recording buffer unit 2110, and a recording unit 2111.
[0114]
In FIG. 24, the range indicated by e is a portion related to RTP, and the range indicated by f is a portion related to RTCP (e. Used for meaning).
[0115]
Hereinafter, the multimedia content receiving device (also referred to as a receiving device) receives multimedia content from a multimedia content distribution device (also referred to as a transmitting device) (not shown) via a network (not shown). Will be described. The multimedia content transmitting apparatus is not particularly limited, and may be, for example, an existing apparatus or the one described in the first to fifth embodiments.
[0116]
The packet data transmitted from the transmission side device is received by the reception unit 2101, and the packet analysis unit 2102 reads the packet header and the like, and obtains the packet number, time stamp, and the like. In the packet analysis information (2132), a packet loss, a delay amount, and the like are calculated by the communication path state calculation unit 2103, and are transmitted to the transmission side device by the communication path state information transmission unit 2104 using RTCP or the like.
[0117]
Further, the packet analysis information (2132) and the communication path state information (2134) are input to the retransmission control unit 2105, and it is determined whether or not to request retransmission. A retransmission request (2135) using, for example, NACK information or the like is generated from the packet number or the like where the packet loss has occurred, and the retransmission request transmission unit 2106 notifies the transmission side device.
[0118]
On the other hand, the received packet data (2136) is temporarily stored in the packet buffer unit 2107, and after adjusting the order together with the packet sent by retransmission or the like, only the data in time for reproduction is selected, and the decoding unit 2108.
[0119]
The decryption unit 2108 performs decryption processing according to the input content data (2137), and the display unit 2109 reproduces the data. On the other hand, content data 2139 is output from the packet buffer unit 2107 to the recording buffer unit 2110 in a form including data that is not in time for reproduction. The re-transmission data that is not in time for reproduction in the recording buffer unit 2110 is also adjusted in order and is recorded in the recording unit 2111.
[0120]
In the recording buffer unit 2110, the portion where no data is missing from the head is sequentially output to the recording unit 2111, and the data area that has been output is released and used as a new data writing area, thereby making the memory more efficient. It is possible to use it.
[0121]
As described above, the packet buffer unit 2107 performs processing of selecting and reproducing data that is in time for the reproduction time, and recording data that is not in time for reproduction as recording data. Thereby, at the time of reproduction, multimedia data can be reproduced in real time (although there may be an error), and error-free data can be recorded as recording data. At that time, the order is controlled so that the retransmission packet works effectively at the time of real-time reproduction by using any one of various configuration examples and modifications of the first to fifth embodiments. Is possible.
[0122]
Next, FIG. 25 shows another configuration example of the multimedia content distribution apparatus of the present embodiment.
[0123]
When the entire size of the content is very large, the data is temporarily stored in the recording buffer unit 2110 as in the configuration example of FIG. 24. Even if the data is output to 2111, there is a possibility that a large amount of memory is required. In particular, when retransmission of data that is not in time for reproduction is postponed on the distribution side, the data cannot be collected and cannot be output to the recording unit 2111 indefinitely. Therefore, in the configuration example of FIG. 25, it is also possible to use a method in which data is directly written from the packet buffer unit 2107 to the recording unit 2111 and the data inside the recording unit 2111 is rearranged by the recording control unit 2201 later.
[0124]
At this time, it is possible to extend the data rewriting in the recording unit 2111 to a minimum by recording the data to be written in the recording unit 2111 by using the pointer described with reference to FIG.
[0125]
Furthermore, in the first to fifth embodiments, the case has been described where the importance of retransmission requests is determined and the order is controlled in the transmission side device, but these methods can also be applied to the reception side device. .
[0126]
As an example, FIG. 26 shows a configuration example of a multimedia content distribution apparatus when the method adopted in the first embodiment is applied.
[0127]
In this case, the retransmission control unit 2105 also determines the importance level of the retransmission request, and the retransmission request queue unit 2301 temporarily stores the retransmission request (2331) to which the importance level information generated by the retransmission control unit 2105 is added. . The retransmission request information (2332) ordered using the importance information is sent to the retransmission request transmission unit 2106 by the retransmission control unit 2105 in consideration of the request transmission timing, and is transmitted to the transmission side apparatus.
[0128]
Of course, also in this case, as shown in FIG. 25, it is also possible to use a method of directly writing data from the packet buffer unit 2107 to the recording unit 2111 and rearranging the data in the recording unit 2111 by the recording control unit 2201 later. It is.
[0129]
In addition, among the various configuration examples described in the first to fifth embodiments, the configuration examples other than those described here can be similarly applied.
[0130]
In the first to sixth embodiments, the case where RTP is used has been described as an example. However, the protocol is not limited to this, and various protocols that can be retransmitted can be used. It is. In the first to sixth embodiments, video signals such as moving images and still images have been mainly described as content data. However, the types of data are not limited, and audio data, text data, or the like can be used. It can be implemented.
[0131]
Each of the above functions can be realized even if it is described as software and processed by a computer having an appropriate mechanism.
The present embodiment can also be implemented as a program for causing a computer to execute predetermined means, causing a computer to function as predetermined means, or causing a computer to realize predetermined functions. In addition, the present invention can be implemented as a computer-readable recording medium on which the program is recorded.
[0132]
Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.
[0133]
【The invention's effect】
According to the present invention, it is possible to efficiently transmit multimedia contents in a network in which errors may be mixed.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration example of a multimedia content distribution apparatus according to a first embodiment of the present invention.
FIG. 2 is a diagram showing a configuration of a header portion of an RTP packet
FIG. 3 is a flowchart showing an example of a schematic procedure of retransmission request importance determination processing in the embodiment;
FIG. 4 is a view showing a configuration example of a retransmission request queue in the embodiment.
FIG. 5 is a flowchart showing an example of a detailed procedure of retransmission request importance determination processing in the embodiment;
FIG. 6 is a diagram for explaining a transmission / reception time when a retransmission packet is in time for the scheduled reproduction time in the embodiment;
FIG. 7 is a diagram for explaining a transmission / reception time when a retransmission packet is not in time for the scheduled reproduction time in the embodiment;
FIG. 8 is a view showing another configuration example of the multimedia content distribution apparatus in the embodiment.
FIG. 9 is a diagram showing still another configuration example of the multimedia content distribution apparatus according to the embodiment.
FIG. 10 is a diagram illustrating a packet configuration of an RTCP transmission side report sent from the transmission side to the reception side;
FIG. 11 is a diagram showing a configuration of an RTCP transmission side report packet sent from the reception side to the transmission side;
FIG. 12 is a diagram showing a configuration example of a multimedia content distribution apparatus according to the second embodiment of the present invention.
FIG. 13 is a flowchart showing an example of a processing procedure according to the importance in the retransmission packet buffer unit in the embodiment;
FIG. 14 is a diagram showing a configuration example of a retransmission wait packet buffer unit in the embodiment;
FIG. 15 is a view showing another configuration example of the retransmission waiting packet buffer unit in the embodiment;
FIG. 16 is a diagram showing a configuration example of a multimedia content distribution apparatus according to the third embodiment of the present invention.
FIG. 17 is a diagram showing another configuration example of the multimedia content distribution apparatus in the embodiment.
FIG. 18 is a diagram showing still another configuration example of the multimedia content distribution apparatus according to the embodiment.
FIG. 19 is a diagram showing a configuration example of a multimedia content distribution apparatus according to the fourth embodiment of the present invention.
FIG. 20 is a diagram showing another configuration example of the multimedia content distribution apparatus in the embodiment.
FIG. 21 is a diagram showing a configuration example of a multimedia content distribution apparatus according to a fifth embodiment of the present invention.
FIG. 22 is a diagram showing another configuration example of the multimedia content distribution apparatus in the embodiment.
FIG. 23 is a diagram showing still another configuration example of the multimedia content distribution apparatus according to the embodiment.
FIG. 24 is a diagram showing a configuration example of a multimedia content receiving apparatus according to a sixth embodiment of the present invention.
FIG. 25 is a view showing another configuration example of the multimedia content receiving apparatus according to the embodiment;
FIG. 26 is a diagram showing still another configuration example of the multimedia content receiving apparatus in the embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 101 ... Multimedia content preservation | save part, 102 ... RTP packetization part, 103 ... Transmission queue part, 104 ... Transmission part, 105 ... Communication path state information reception part, 106 ... Round-trip time calculation part, 107 ... Retransmission request reception part, 108 ... Packet buffer unit, 109 ... Retransmission control unit, 110 ... Retransmission request queue unit, 111 ... Retransmission packetization unit, 112 ... Retransmission packet queue unit, 113 ... Retransmission packet transmission unit, 801 ... RTP multimedia content storage unit, 901 ... Retransmission waiting packet buffer unit, 1601... Session control unit, 1701... Rate control unit, 1702 .. switching compatible multi-media content storage unit, 1801 .. encoding unit, 1802. Part, 2103 ... communication path state calculation part, 2104 ... communication path state Report transmission unit, 2105 ... retransmission control unit, 2106 ... retransmission request transmission unit, 2107 ... packet buffer unit, 2108 ... decoding unit, 2109 ... display unit, 2110 ... recording buffer unit, 2111 ... recording unit, 2201 ... recording control Part 2301... Retransmission request queue part

Claims (15)

First transmitting means for sequentially transmitting a plurality of packets generated from the multimedia content data to the receiving device via the network;
Receiving means for receiving a retransmission request including information specifying a packet to be retransmitted from the receiving device via the network;
The importance related to retransmission of the packet related to the retransmission request, the first time when the data related to the packet is scheduled to be used by the receiving device, and the data related to the retransmission packet corresponding to the packet can be used by the receiving device Determining means for determining based on a second time expected to be;
Selecting means for selecting a packet to be retransmitted based on the importance;
A packet transmission apparatus comprising: a second transmission unit configured to transmit a retransmission packet corresponding to the selected packet to the reception apparatus via the network. The processing means sets the importance level to a high value when the first time is later than the second time, and the first time is a time before the second time. The packet transmission device according to claim 1, wherein the importance is set to a low value in some cases. The processing means includes a first time required for processing until a retransmission packet related to the retransmission request is transmitted by the second transmission means, and a retransmission packet transmitted by the second transmission means to the receiving apparatus. The packet transmission device according to claim 2, wherein the second time is predicted based on a second time required for arrival. The processing means obtains the first time based on a time required to complete transmission of a retransmission packet related to another retransmission request to be transmitted before a retransmission packet related to the retransmission request, and the receiving apparatus The packet transmission device according to claim 3, wherein the second time is obtained based on a time required for a packet to reciprocate between the first and second packets. The determining means, when the first time is a time after the second time, the importance of making the retransmission packet related to the retransmission request available at the first time in the receiving device And when the first time is a time before the second time, the importance that the packet related to the retransmission request is subordinately selected by the selection means is set. The packet transmission device according to claim 1. Storage means for storing multimedia content data;
The packet transmission device according to claim 1, further comprising a generation unit configured to generate the plurality of packets from the multimedia content data. The packet transmission apparatus according to claim 1, further comprising storage means for storing data relating to a plurality of packets generated from the multimedia content data. First receiving means for sequentially receiving a plurality of packets generated from multimedia content data or retransmission packets corresponding to the packets from a transmitting device via a network;
Analyzing means for analyzing the packet received by the first receiving means, and specifying means for specifying a packet to be retransmitted based on the result;
Transmitting means for transmitting a retransmission request including information specifying a packet to be retransmitted to the transmitting device via the network;
Data utilization means for utilizing data relating to the packet;
A packet receiving apparatus comprising recording means for recording data relating to the packet. Further comprising holding means for rearranging and holding all the data relating to the packet and the data relating to the retransmission packet in the order in which they are used,
9. The packet receiving apparatus according to claim 8, wherein the recording unit records the rearranged data held in the holding unit. Accumulating all the data related to the packet and the data related to the retransmitted packet, and storing the data, and directly writing to the recording means,
9. The packet receiving apparatus according to claim 8, further comprising means for rearranging data recorded in the recording means in an order in which they are used. 9. The data utilization unit uses only data related to the packet and data related to the retransmission packet that are available before the time when the data should be used. The packet receiving device described. Storage means for storing the retransmission request;
The apparatus further comprises means for selecting a high-priority request among the retransmission request related to the packet specified by the specifying means and the retransmission request stored in the accumulating means, and passing the selected request to the transmitting means. Item 9. The packet receiving device according to Item 8. 9. The packet receiving apparatus according to claim 8, wherein the data using means includes means for decoding the data and means for displaying the decoded data. Sequentially transmitting a plurality of packets generated from the multimedia content data to a receiving device via a network;
Receiving a retransmission request including information identifying a packet to be retransmitted from the receiving device via the network;
The importance related to retransmission of the packet related to the retransmission request, the first time when the data related to the packet is scheduled to be used by the receiving device, and the data related to the retransmission packet corresponding to the packet can be used by the receiving device Determining based on a second time expected to be;
Selecting a packet to be retransmitted based on the importance;
And a step of transmitting a retransmission packet corresponding to the selected packet to the receiving device via the network. Receiving a plurality of packets generated from multimedia content data or a retransmission packet corresponding to the packets sequentially from a transmitting device via a network;
Analyzing received packets and identifying packets to be retransmitted based on the results;
Transmitting a retransmission request including information specifying a packet to be retransmitted to the transmission device via the network;
Utilizing the data associated with the packet;
A packet receiving method comprising: recording data relating to the packet.

Images