JP4127843B2 - Data relay apparatus and data relay method - Google Patents

Description

  The present invention relates to a data relay apparatus that receives packet data transmitted from a streaming server and distributes the received data to a mobile device, and a data relay method in the data relay apparatus.

  2. Description of the Related Art In recent years, services that distribute video streaming data in real time as internet communication speeds up are provided. As an application of this service, streaming data from a streaming server on the Internet is converted into circuit-switched data in real time in a video conversion device that is a data relay device on a mobile communication network, and is converted into a mobile device equipped with a videophone function. A distribution system is provided. As a result, streaming data on the Internet can be viewed on a mobile device.

  As described above, the streaming server and the video conversion device are connected by the Internet. However, in a communication environment having a plurality of communication paths such as the Internet, the order of packet data is changed between the streaming server and the video conversion device (jitter). Ring) and packet data may be discarded. Therefore, in order for the streaming data to be correctly reproduced on the mobile device, the video conversion apparatus needs to consider events such as jittering and packet data discarding.

In the existing system, as a countermeasure against such an event, the video conversion device has a function of monitoring the order of packet data from the streaming server. As a specific function to manage the order of packet data, it has a dedicated buffer and timer, and if the sequence number of the received packet data is missing, the missing packet data arrives for a fixed time determined by the timer. Until then, there is a function of waiting for transmission of subsequent packet data to the mobile device (for example, see Non-Patent Document 1 below). In this method, packet data corresponding to video and audio for a predetermined time received from a streaming server is buffered on a memory (transmission buffer) as transmission data for a mobile device by a video conversion device.
Development of video distribution gateway for FOMA V Live Service, “NTT DoCoMo Technical Journal” Vol. 11 No. 3 P.M. 42-47, October 2003

  However, in the above method, when packet data is lost, the lost packet data is always waited for a certain time. For this reason, there is a problem that when there is little remaining packet data buffered on the transmission buffer for the mobile device, there is no packet data to be delivered to the mobile device, and video and audio on the mobile device are stopped.

  Here, if the transmission timing for the mobile device is delayed and the amount of packet data buffered in the transmission buffer for the mobile device is increased, the jitter of the packet data occurs and the subsequent packet data transmission is continued for a certain period of time. Even when waiting, the packet data remaining in the transmission buffer for the mobile device can be transmitted. Therefore, the influence on the video and audio on the mobile device side is reduced. However, increasing the buffering amount of packet data corresponding to video and audio causes a large delay between the reception of the packet data from the streaming server and the reproduction of the packet data on the mobile device, so that the real-time property is lost. It will be.

  The present invention has been made to solve the above-described problems, and prevents the real-time property from being lost in response to events such as jittering of packet data and discarding of packet data, and at the same time, high-quality streaming data. It is an object of the present invention to provide a data relay apparatus and a data relay method capable of providing a mobile station with a mobile station.

  In order to achieve the above object, a data relay apparatus according to the present invention includes a receiving means for receiving a plurality of packet data related to streaming data transmitted from a streaming server, and the packet data received by the receiving means inside the apparatus. Storage means for storing in the memory, sequence number acquisition means for acquiring a sequence number assigned to the packet data stored by the storage means, and continuous packet data based on the sequence number acquired by the sequence number acquisition means Continuity determining means for determining the continuity, and if the result of determination by the continuity determining means indicates that there is continuity, the packet data to be determined is stored in a data transmission buffer for mobile devices inside the apparatus. As a result of judgment by credit data storage means and continuity judgment means, there is no continuity If it is disconnected, the remaining data confirmation means in the buffer for confirming the data amount of the packet data remaining in the data transmission buffer for the mobile device and the continuity according to the data amount confirmed by the remaining data confirmation means in the buffer Transmission data control means for controlling the time until the packet data to be determined by the determination means is stored in the data transmission buffer for mobile devices, and packet data stored in the data transmission buffer for mobile devices is transmitted to the mobile device And a transmission means.

  In the data relay device according to the present invention, when it is determined that the packet data transmitted from the streaming server is not continuous, the time until the packet data is stored in the data transmission buffer for the mobile device is determined. Control based on the amount of packet data remaining in the trust buffer. Therefore, when packet data is lost due to jittering or discarding, if there is a lot of packet data to be transmitted to the mobile device, it is possible to wait for the lost packet, and there is packet data to be transmitted to the mobile device. If there is not much remaining, subsequent packet data can be transmitted. As a result, it is possible to provide streaming data that is as complete as possible to the mobile device while preventing occurrence of a large delay.

  That is, according to the data relay device of the present invention, it is possible to prevent the real-time property from being impaired in response to an event such as packet data jittering or packet data discarding, and at the same time, send high-quality streaming data to the mobile device. Can be provided.

  Incidentally, the present invention can be described as an invention of a data relay apparatus as described above, and can also be described as an invention of a data relay method in the data relay apparatus as follows. This is substantially the same invention only in different categories, and has the same operations and effects.

  That is, the data relay method according to the present invention is a data relay method in a data relay device that relays data between a streaming server and a mobile device, and a plurality of packet data related to streaming data transmitted from the streaming server. A reception step for receiving the packet data, a storage step for storing the packet data received in the reception step in a memory inside the device, and a sequence number acquisition step for acquiring a sequence number assigned to the packet data stored in the storage step; The continuity determination step for determining the continuity of the packet data based on the sequence number acquired in the sequence number acquisition step, and the determination target when it is determined that there is continuity as a result of the determination in the continuity determination step Packet data Packets remaining in the mobile device data transmission buffer when it is determined that there is no continuity as a result of determination in the transmission data storage step stored in the internal mobile device data transmission buffer and the continuity determination step According to the remaining data confirmation step in the buffer for confirming the data amount of the data and the data amount confirmed in the remaining data confirmation step in the buffer, the packet data to be judged in the continuity judgment step is stored in the data transmission buffer for the mobile device. It includes a transmission data control step for controlling the time until storage, and a transmission step for transmitting packet data stored in the data transmission buffer for mobile devices to the mobile device.

  According to the present invention, when packet data is lost, it is possible to prevent a large delay from occurring, so that real-time performance is impaired with respect to events such as packet data jittering and packet data discarding. It is possible to provide high quality streaming data to the mobile device at the same time.

  Hereinafter, preferred embodiments of a data relay device and a data relay method according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 shows a video conversion apparatus 10 that is a data relay apparatus according to the present embodiment, and a video distribution system 1 that includes the video conversion apparatus 10. As shown in FIG. 1, the video distribution system 1 includes a streaming server 30 and a mobile device 40 in addition to the video conversion device 10. The video distribution system 1 is a system in which content is provided as streaming data from the streaming server 30 to the mobile device 40.

  The video conversion apparatus 10 is included in the mobile communication network 20, receives packet data related to streaming data transmitted from the streaming server 30, and distributes the received data to the mobile device 40 by mobile communication. Device. That is, the video conversion apparatus 10 relays data between the streaming server 30 and the mobile device 40. Transmission of data from the video conversion device 10 to the mobile device 40 is performed by converting packet data into circuit-switched data and via a circuit-switched call established by mobile communication between the video conversion device 10 and the mobile device 40. It is done. Thus, by transmitting data as circuit-switched data, relatively large data such as streaming data can be transmitted more reliably in mobile communication.

  The streaming server 30 is a device that stores a plurality of content files 31 a to 31 c and distributes the content files 31 a to 31 c as streaming data to the mobile device 40 via the video conversion device 10. The streaming server 30 divides the streaming data related to the content files 31a to 31c via the Internet N and transmits the divided packet data to the video conversion apparatus 10. The content files 31a to 31c are reproduced as video in the mobile device 40. Further, what is distributed from the streaming server 30 is not limited to the content files 31a to 31c, but may be data that requires real-time properties, such as camera images such as news and concerts. In this case, the streaming server 30 receives input of data from a video photographing device such as a camera and transmits it as packet data related to the streaming data.

  The mobile device 40 is a communication device used by a user and can be connected to the mobile communication network 20. The mobile device 40 includes a display device that can receive the circuit-switched data received and converted by the video conversion device 10 from the streaming server 30 and reproduce the distributed circuit-switched data as video. Specifically, the mobile device 40 corresponds to, for example, a mobile phone having a videophone function.

  In FIG. 1, only one streaming server 30 and one mobile device 40 are depicted, but the video distribution system 1 normally includes a plurality of each.

  Subsequently, the configuration of each device will be described in more detail. As shown in FIG. 1, the video conversion apparatus 10 includes an inter-terminal communication control unit 11, an inter-server communication control unit 12, a jitter buffer 13, a received data order control unit 14, and a data transmission buffer 15 for mobile devices. And a video data control unit 16.

  The inter-terminal communication control unit 11 is a means for performing mobile communication with the mobile device 40. Specifically, the inter-terminal communication control unit 11 receives a request for content from the mobile device 40 and transmits packet data related to the content files 31 a to 31 c acquired from the streaming server 30 in response to the request to the mobile device 40. To do. That is, the inter-terminal communication control unit 11 is a transmission unit that transmits the packet data stored in the data transmission buffer 15 for the mobile device to the mobile device 40. This transmission is performed after establishing a circuit-switched call with the mobile device 40 and converting packet data into circuit-switched data. When the inter-terminal communication control unit 11 receives a request for content from the mobile device 40, the inter-terminal communication control unit 11 notifies the inter-server communication control unit 12 accordingly.

  The inter-server communication control unit 12 is means for communicating with the streaming server 30 via the Internet N. Specifically, the inter-server communication control unit 12 is a receiving unit that receives a plurality of packet data related to streaming data transmitted from the streaming server 30. The inter-server communication control unit 12 makes a content request to the streaming server 30 in response to the content request from the mobile device 40 notified by the inter-terminal communication control unit 11. The inter-server communication control unit 12 receives a plurality of packet data relating to streaming data transmitted in response to the request. The inter-server communication control unit 12 is also a storage unit that stores a plurality of received packet data in the jitter buffer 13.

  The plurality of packet data related to the streaming data has an order. This order is the order in which packet data is received and reproduced. This order can be determined by referring to the sequence number included in the packet data. The sequence number is a consecutive number assigned to each packet data according to the order of each packet data when streaming data is divided into packet data by the streaming server 30 and transmitted to the video conversion apparatus 10. By referring to the sequence number, the order of packet data, that is, the continuity of packet data can be determined.

  Packet data from the streaming server 30 is received via the Internet N. As described above, since the Internet N has a plurality of communication paths, the packet data is jittered at the time of reception and the reliability is low. When the route is passed, packet data may be discarded.

  The jitter buffer 13 is a memory inside the video conversion apparatus 10 and stores packet data received by the inter-server communication control unit 12. The jitter buffer 13 always stores at least one packet data when it starts to receive the packet data related to the streaming data. This is because the received data order control unit 14 determines the continuity of packet data as will be described later.

  The reception data order control unit 14 is continuity determination means for determining the continuity of a plurality of packet data received by the inter-server communication control unit 12 and stored in the jitter buffer 13. As described above, the continuity of packet data indicates that the packet data is in the order in which it should be reproduced (whether the packets in the order in which they are to be reproduced exist without omission).

  The reception data order control unit 14 determines continuity as follows. The reception data order control unit 14 reads the packet data stored in the jitter buffer 13 and acquires the sequence number assigned to the packet data. That is, the reception data order control unit 14 is also a sequence number acquisition unit as described above.

  The received data order control unit 14 determines the continuity of the packet data based on the acquired sequence number of each packet data. This determination is performed when new packet data is received and stored in the jitter buffer 13 and when packet data is stored in the data transmission buffer 15 for the mobile device from the jitter buffer 13. This determination is made for the packet data stored in the jitter buffer 13 having the smallest sequence number. The continuity of packet data is determined based on the sequence number of packet data stored in the jitter buffer 13. In this determination, the determination result of the continuity of the previous packet data may be held and referred to.

  The determination of the continuity of the packet data is performed by confirming whether there is a continuous sequence number with respect to the youngest sequence number among the sequence numbers of the packet data stored in the jitter buffer 13. When there is no continuous sequence number, it is determined that the packet data to be determined (packet data having the youngest sequence number) has no continuity. When there are consecutive sequence numbers, it is determined that the packet data to be determined (packet data having the smallest sequence number) has continuity.

  For example, when packet data with a sequence number “5” is stored in the jitter buffer 13 and new packet data with a sequence number “6” is stored in the jitter buffer 13, the smallest sequence number “5” is set. Since there is a continuous sequence number “6”, it is determined that the packet data with the sequence number “5” has continuity.

  Further, when packet data with a sequence number “5” is stored in the jitter buffer 13 and new packet data with a sequence number “7” is stored in the jitter buffer 13, the smallest sequence number “5” is set. Since there is no continuous sequence number “6” (since the packet data with sequence number “6” is missing), it is determined that the packet data with sequence number “5” has no continuity. Here, the sequence number “6” for determining that the packet data of the sequence number “5” has continuity is stored in the received data order control unit 14, and the information is used for the next continuity determination. It may be used. For example, by checking whether or not the sequence number stored in the jitter buffer 13 next is “6”, it can be determined that there is continuity of the packet data having the sequence number “5”.

  The reception data order control unit 14 notifies the video data control unit 16 of information indicating the determined continuity. This notification is performed, for example, by transmitting information that associates information that identifies packet data (for example, a sequence number) with information that indicates whether there is continuity.

  The mobile device data transmission buffer 15 is a memory inside the video conversion apparatus 10 and stores packet data to be transmitted to the mobile device 40. The packet data stored in the mobile device data transmission buffer 15 is acquired by the inter-terminal communication control unit 11 and transmitted sequentially. The transmitted packet data is erased from the mobile device data transmission buffer 15.

  The video data control unit 16 is a unit that controls packet data based on information indicating continuity notified from the reception data order control unit 14. Specifically, when the video data control unit 16 determines that there is continuity of packet data as a result of the determination by the reception data order control unit 14, the video data control unit 16 transfers the packet data to be determined from the jitter buffer 13 to the mobile device. This is transmission data storage means for storing in the data transmission buffer 15. The packet data stored in the mobile device data transmission buffer 15 is erased from the jitter buffer 13.

  If the video data control unit 16 determines that there is no continuity of the packet data as a result of the determination by the reception data order control unit 14, the video data control unit 16 sends the video data control unit 16 to the mobile device 40 remaining in the mobile device data transmission buffer 15. It is also a buffer remaining data confirmation means for confirming the data amount of packet data related to streaming data to be transmitted. The video data control unit 16 is also transmission data control means for controlling the time until the packet data to be determined is stored in the mobile device data transmission buffer 15 according to the confirmed data amount. When the video data control unit 16 determines that the amount of data is small, the video data control unit 16 sets the time until the packet data to be determined to be stored in the mobile device data transmission buffer 15 to zero. That is, the packet data to be determined is immediately stored in the mobile device data transmission buffer 15. When the video data control unit 16 determines that the amount of data is large, the video data control unit 16 sets a predetermined time (a certain length of time) until the packet data to be determined is stored in the mobile device data transmission buffer 15. Time). The determination of whether the amount of data is large or small is made based on the threshold value TH1 of the data amount stored in advance by the video data control unit 16. If the data amount is larger than the threshold value TH1, the data amount is assumed to be large. If the amount of data is smaller than the threshold value TH1, the amount of data is assumed to be small. Also, the video data control unit 16 stores in advance each set time (expiration value of the timer T1 described below) according to the amount of data.

  The video data control unit 16 measures the time from the determination time by the timer T1 for each packet data determined to have no continuity of the packet data, and when the set time expires, The packet data is stored from the jitter buffer 13 into the mobile device data transmission buffer 15. Note that the inter-server communication control unit 12 receives packet data from the streaming server 30 until the timer T1 expires (while waiting for storage of packet data in the mobile device data transmission buffer 15). And stored in the jitter buffer 13.

  Even when it is determined that there is no continuity, the missing packet data is stored in the jitter buffer 13, and it is received that the packet data determined to have no continuity has continuity. When notified from the data order control unit 14, the video data control unit 16 stores the packet data determined to have no continuity from the jitter buffer 13 to the mobile device data transmission buffer 15.

  The above control waits for missing packet data without sending discontinuous packet data to the mobile device 40 when there is (many) data to be transmitted to the mobile device 40. Is. Further, when there is little data to be transmitted to the mobile device 40, discontinuous packet data is transmitted to the mobile device 40 without waiting for the missing packet data. .

  Therefore, the values of the expiration values of the threshold value TH1 and the timer T1 are arbitrarily changed according to the speed of the communication path between the streaming server 30 and the video conversion device 10, the real-time property of the necessary streaming data, and the video quality. Is possible. For example, in order to ensure real-time properties, it is preferable to make the expiration value of the timer T1 as small as possible. In this case, however, the video quality is likely to deteriorate. On the other hand, in order to ensure video quality, the threshold value TH1 should be as large as possible. However, in that case, real-time property may be impaired. The functional configuration of the video conversion device 10 has been described above.

  Next, FIG. 2 shows a hardware configuration of the video conversion apparatus 10. As shown in FIG. 2, a video conversion apparatus 10 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102 and a ROM (Read Only Memory) 103, which are main storage devices, and a communication module 104 for performing communication. And a computer including hardware such as an auxiliary storage device 105 such as a hard disk. The functions described above are exhibited by the operation of these components.

  As shown in FIG. 1, the streaming server 30 includes a communication control unit 32 and a data management unit 33.

  The communication control unit 32 is means for performing communication with the video conversion device 10 via the Internet N. Specifically, the communication control unit 32 receives a content request from the video conversion device 10, acquires the content files 31 a to 31 c corresponding to the request from the data management unit 33, and transmits the content files 31 a to 31 c to the video conversion device 10. To do. This transmission / reception is performed via the Internet N. Further, the transmission of the content files 31a to 31c to the video conversion device 10 is performed by dividing it into packet data as streaming data. The data management unit 33 is means for storing a plurality of content files 31a to 31c.

  The streaming server 30 is configured as a computer including a CPU, a memory, a hard disk, and the like, similar to the video conversion apparatus 10 shown in FIG. The functions described above are exhibited by the operation of these components.

  As shown in FIG. 1, the mobile device 40 includes a communication control unit 41, an input control unit 42, an information processing unit 43, and a screen control unit 44.

  The communication control unit 41 is means for performing mobile communication with the video conversion device 10. Specifically, the communication control unit 41 transmits a content request to the video conversion device 10 in response to an input from the user, and is transmitted from the video conversion device 10 in response to the request. The packet data converted into circuit switching data according to 31a to 31c is received. The received packet data is output to the information processing unit 43.

  The input control unit 42 is means for receiving an input from the user. The input from the user is performed from an operation unit such as a button provided in the mobile device 40. The content request transmitted by the communication control unit 41 is input from the input control unit 42 by the user. The information processing unit 43 is means for checking the contents of the packet data related to the content files 31 a to 31 c received by the communication control unit 41 and outputting the packet data to the screen control unit 44. The screen control unit 44 outputs the packet data input from the information processing unit 43 to a display or a speaker (not shown) included in the mobile device 40, so that the user can view the content.

  The mobile device 40 is configured by hardware such as a CPU, a memory, and an antenna, and the above-described functions are exhibited by operating these components.

  Subsequently, processing (data relay method) executed by the video conversion apparatus 10 that is the data relay apparatus according to the present embodiment will be described. This processing is processing when the mobile device 40 receives packet data related to streaming data from the streaming server 30 via the video conversion device 10.

  First, in the mobile device 40, a request for content and a number specifying the content related to the request are input from the user to the input control unit 42, and a circuit-switched call with the video conversion device 10 is established. An input for making a call is made. As a result of the input, the communication control unit 41 of the mobile device 40 makes a call (connection request) to the video conversion device 10 and transmits a request for content and a number for specifying the content related to the request. The Note that the information transmitted to the video conversion device 10 may include, for example, information (for example, URL (Uniform Resource Location)) that identifies the streaming server 30 by the video conversion device 10.

  In the video conversion device 10, a signal related to the call is received by the inter-terminal communication control unit 11, and a process for establishing a circuit-switched call with the communication control unit 41 of the mobile device 40 is performed. In addition, the inter-terminal communication control unit 11 receives a request for content transmitted from the mobile device 40 and a number for specifying the content related to the request. The inter-terminal communication control unit 11 notifies the inter-server communication control unit 12 of the content request and the number for specifying the content.

  Subsequently, the inter-server communication control unit 12 establishes a connection with the streaming server 30 (the communication control unit 32) via the Internet N. When the connection is established, a request for content including a number specifying the content notified from the inter-terminal communication control unit 11 is made to the streaming server 30 from the inter-server communication control unit 12. In the streaming server 30 for which the request has been made, the communication control unit 32 receives the request for the content, and content files 31 a to 31 c corresponding to the number for specifying the content included in the request are received from the data management unit 33. To be acquired. Subsequently, the content files 31 a to 31 c are divided into packets as streaming data by the communication control unit 32 and transmitted (distributed) to the video conversion apparatus 10 via the Internet N.

  Processing in the video conversion apparatus 10 that receives the packet data transmitted from the streaming server 30 and transmits it to the mobile device 40 will be described with reference to the flowchart of FIG. In the following description, FIG. 4 schematically showing the state of the memory in the video conversion apparatus 10 is used. In FIG. 4, a block indicates each packet data. The numbers written in the blocks indicate the sequence numbers of the packet data. As shown in FIG. 4A, packet data is not stored in the jitter buffer 13 and the mobile device data transmission buffer 15 of the video conversion device 10.

  In the video conversion device 10, the inter-server communication control unit 12 receives packet data from the streaming server 30 (S01, reception step). The following processing is performed each time packet data is received by the video conversion apparatus 10 for a plurality of packet data. The packet data received by the inter-server communication control unit 12 is stored in the jitter buffer 13 (S02, storage step). Subsequently, the sequence number is acquired from the packet data stored in the jitter buffer 13 by the reception data order control unit 14 (S03, sequence number acquisition step). Subsequently, the continuity of the stored packet data is determined by the received data order control unit 14 based on the acquired sequence number (S04, continuity determination step).

  For example, as shown in FIG. 4B, when packet data having sequence numbers “1” and “2” are stored in the jitter buffer 13, the sequence number “2” that is consecutive to the youngest sequence number “1” is stored. Therefore, it is determined that there is continuity of packet data. Subsequently, the reception data sequence control unit 14 notifies the video data control unit 16 of the fact and information for specifying the packet data to be determined. When it is determined that the packet data has continuity, as shown in FIG. 4C, the packet data of the sequence number “1” determined to have continuity by the video data control unit 16 is stored in the jitter buffer. 13 is stored in the mobile device data transmission buffer 15 (S05, transmission data storage step). As described above, at least one packet data must be stored in the jitter buffer 13 (except when the packet data is determined as the last data related to the streaming data).

  Subsequently, as in the packet data of sequence number “1” shown in FIG. 4D, the packet data stored in the mobile device data transmission buffer 15 by the inter-terminal communication control unit 11 is changed in the order of the sequence numbers. It is converted into circuit-switched data and is sequentially transmitted (distributed) to the mobile device 40 (S06, transmission step). The packet data transmitted to the mobile device 40 is received by the communication control unit 41 of the mobile device 40, is output by the screen control unit 44 via the information processing unit 43, and is referred to by the user. If it is determined that the packet data stored in the jitter buffer 13 is the last packet data related to the streaming data, the continuity is not determined for the packet data, and data for the mobile device is used. It is stored in the transmission buffer 15 and transmitted to the mobile device 40.

  It is assumed that the above processing is sequentially repeated, and the jitter buffer 13 and the mobile device data transmission buffer 15 are in the state shown in FIG. That is, packet data with a sequence number “4” is stored in the jitter buffer 13, and packet data with a sequence number “3” is stored in the data transmission buffer 15 for mobile devices.

  Here, as shown in FIG. 4E, packet data with a sequence number of “6” is received (S01, reception step) and stored in the jitter buffer 13 (S02, storage step). As described above, the packet data with the sequence number “6” is received instead of the packet data with the sequence number “5” following the sequence number “4”. This is because destruction occurred.

  Subsequently, the sequence number is acquired from the packet data stored in the jitter buffer 13 by the reception data order control unit 14 (S03, sequence number acquisition step). Subsequently, the continuity of the stored packet data is determined by the received data order control unit 14 based on the acquired sequence number (S04, continuity determination step). Here, as shown in FIG. 4E, when packet data of sequence numbers “4” and “6” are stored in the jitter buffer 13, the sequence number “5” that is consecutive to the youngest sequence number “4” is stored. "Is not present, it is determined that there is no continuity of packet data. Subsequently, the reception data sequence control unit 14 notifies the video data control unit 16 of the fact and information for specifying the packet data to be determined.

  When it is determined that there is no continuity of packet data, the video data control unit 16 confirms the amount of packet data remaining in the mobile device data transmission buffer 15 (S07, remaining data in buffer confirmation step). ). In the case shown in FIG. 4E, the data amount for one packet is confirmed. Subsequently, the video data control unit 16 determines whether or not the confirmed data amount is smaller than the threshold (S08, transmission data control step). If the threshold value is not less than one packet and not more than two, in this case, it is determined that the data amount is smaller than the threshold value.

  If so, as shown in FIG. 4 (f), the packet data of sequence number “4” determined to be non-continuous by the video data control unit 16 is immediately sent to the mobile device data transmission buffer. 15 (S09, transmission data control step). This is to prevent packet data to be transmitted to the mobile device 40 from being cut even when packet data is lost or discarded. The packet data stored in the mobile device data transmission buffer 15 is sequentially transmitted (distributed) to the mobile device 40 (S06, transmission step).

  On the other hand, it is assumed that the jitter buffer 13 and the mobile data transmission buffer 15 are not in the state shown in FIG. 4D but in the state shown in FIG. In other words, the packet data with the sequence number “4” is stored in the jitter buffer 13, and the packet data with the sequence numbers “2” and “3” is stored in the data transmission buffer 15 for the mobile device. Here, the packet data with the sequence number “6” is received (S01, reception step) and stored in the jitter buffer 13 (S02, storage step).

  Subsequently, the sequence number is acquired from the packet data stored in the jitter buffer 13 by the reception data order control unit 14 (S03, sequence number acquisition step). Subsequently, the continuity of the stored packet data is determined by the received data order control unit 14 based on the acquired sequence number (S04, continuity determination step). Here, as shown in FIG. 4 (h), when packet data of sequence numbers “4” and “6” are stored in the jitter buffer 13, the sequence number “5” consecutive to the youngest sequence number “4” is stored. "Is not present, it is determined that there is no continuity of packet data. Subsequently, the reception data sequence control unit 14 notifies the video data control unit 16 of the fact and information for specifying the packet data to be determined.

  When it is determined that there is no continuity of packet data, the video data control unit 16 confirms the amount of packet data remaining in the mobile device data transmission buffer 15 (S07, remaining data in buffer confirmation step). ). In the case shown in FIG. 4H, the data amount for two packets is confirmed. Subsequently, the video data control unit 16 determines whether or not the confirmed data amount is smaller than the threshold (S08, transmission data control step). If the threshold value is not less than one packet and not more than two, in this case, it is determined that the data amount is larger than the threshold value.

  When such a determination is made, the time from the determination time is measured by the video data control unit 16 by the timer T1, and when the set time expires, the packet data of the sequence number “4” is jittered. The data is stored from the buffer 13 into the mobile device data transmission buffer 15. That is, it waits for reception of packet data until the timer T1 expires (S10, transmission data control step). At this time, when new packet data is received (S01, reception step), it is stored in the jitter buffer 13 as described above (S02, storage step). For example, when packet data with a sequence number “7” is received, the state shown in FIG.

  Also at this time, sequence number acquisition (S03, sequence number acquisition step) and sequence number continuity are determined (S04, continuity determination step). However, even in this case, it is determined that there is no continuity of the packet data. In this case, since the measurement by the timer T1 is once performed, the timer T1 is not reset.

  When the timer T1 expires (S11, transmission data control step), as shown in FIG. 4 (j), the packet data of the sequence number “4” determined not to be continuous by the video data control unit 16 is moved. The data is stored in the machine-oriented data transmission buffer 15 (S12, transmission data control step). If a plurality of packet data remains in the jitter buffer 13, further determination of the continuity of the packet data (S04, continuity determination step) is performed at this time. As shown in FIG. 4 (j), when packet data having sequence numbers “6” and “7” are stored in the jitter buffer 13, there is a sequence number “7” continuous to the youngest sequence number “6”. Therefore, it is determined that there is continuity of packet data. In this case, the packet data of the sequence number “6” determined to be continuous by the video data control unit 16 is stored from the jitter buffer 13 to the mobile device data transmission buffer 15 (S05, transmission data). Storage step).

  By the way, when the sequence number of the packet data received during the measurement by the timer T1 as described above is “7” instead of “7”, the state of the jitter buffer 13 is as shown in FIG. As shown in (k). In this case, in the determination of the continuity of the sequence numbers (S04, continuity determination step), since there is the sequence number “5” continuous to the youngest sequence number “4”, it is determined that there is continuity of the packet data. . In this case, as shown in FIG. 4 (l), the packet data of sequence number “4” determined to be continuous by the video data control unit 16 is transferred from the jitter buffer 13 to the data transmission buffer 15 for the mobile device. (S05, transmission data storage step). Similarly to the above, continuity is also determined for the packet data with the sequence numbers “5” and “6” (S04, continuity determination step), and the packet data with the sequence number “5”. Are stored from the jitter buffer 13 into the mobile device data transmission buffer 15 (S05, transmission data storage step).

  When packet data has no continuity as described above, the time stored in the mobile device data transmission buffer 15 according to the amount of packet data stored in the mobile device data transmission buffer 15 The reason is that the quality is improved without impairing the real-time property of the streaming data transmitted to the mobile device 40. Here, high quality means that there is little missing of packet data, and smooth video is obtained when streaming data is reproduced in the mobile device 40. That is, when there is a lot of data in the mobile device data transmission buffer 15 (when there is sufficient packet data to be transmitted to the mobile device 40), the quality can be improved by waiting for missing packet data. On the other hand, that is, when there is not much data in the mobile device data transmission buffer 15 (when there is not enough packet data to be transmitted to the mobile device 40), real-time performance is ensured without waiting for missing packet data. be able to.

  Therefore, in the video conversion device 10 which is a data relay device according to the present embodiment, when packet data is lost due to jittering or discarding, the packet data to be transmitted to the mobile device 40 is a data transmission buffer for the mobile device. If more than 15 remain, it is possible to wait for the missing packet, and if there is not much packet data to be transmitted to the mobile device 40, subsequent packet data can be transmitted. Thereby, it is possible to provide streaming data as little as possible to the mobile device 40 while preventing a large delay from occurring. That is, according to the video conversion apparatus 10 that is the data relay apparatus according to the present embodiment, it is possible to prevent the real-time property from being impaired in response to an event such as packet data jittering or packet data discarding, and at the same time to perform high quality streaming Data can be provided to the mobile device.

  In the above-described embodiment, the data relayed by the data relay device is packet data related to streaming data of content related to video. However, the data is not necessarily related to video. The data may be a plurality of packet data related to streaming data transmitted from the streaming server 30. For example, it may be related to audio instead of video.

It is a figure which shows the structure of the video converter which is a data relay apparatus which concerns on embodiment of this invention. It is a figure which shows the hardware constitutions of the video converter which is a data relay apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process (data relay method) performed with the video converter which is a data relay apparatus which concerns on embodiment of this invention. It is the figure which showed typically the state of the memory in a video converter.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Video delivery system, 10 ... Video conversion apparatus, 11 ... Inter-terminal communication control part, 12 ... Inter-server communication control part, 13 ... Jitter buffer, 14 ... Received data order control part, 15 ... Buffer for data transmission for mobile devices 16 ... Video data controller 101 ... CPU 102 ... RAM 103 ... ROM 104 ... Communication module 105 ... Auxiliary storage device 20 ... Mobile communication network 30 ... Streaming server 31 ... Content file 32 ... Communication control unit 33 ... Data management unit 40 ... Mobile device 41 ... Communication control unit 42 ... Input control unit 43 ... Information processing unit 44 ... Screen control unit N ... Internet.

Claims (2)

Receiving means for receiving a plurality of packet data related to streaming data transmitted from a streaming server;
Storage means for storing packet data received by the receiving means in a memory inside the apparatus;
Sequence number acquisition means for acquiring a sequence number assigned to the packet data stored by the storage means;
Continuity determining means for determining the continuity of the packet data based on the sequence number acquired by the sequence number acquiring means;
As a result of the determination by the continuity determination means, when it is determined that there is continuity, the transmission data storage means for storing the packet data to be determined in a data transmission buffer for mobile devices inside the device;
As a result of the determination by the continuity determination means, if it is determined that there is no continuity, the remaining data confirmation means in the buffer for confirming the data amount of the packet data remaining in the data transmission buffer for the mobile device;
Transmission data control for controlling the time until the packet data to be determined by the continuity determining unit is stored in the data transmission buffer for the mobile device according to the amount of data confirmed by the remaining data confirmation unit in the buffer Means,
Transmitting means for transmitting packet data stored in the data transmission buffer for the mobile device to the mobile device;
A data relay device comprising: A data relay method in a data relay device that relays data between a streaming server and a mobile device,
A receiving step of receiving a plurality of packet data related to the streaming data transmitted from the streaming server;
A storing step of storing the packet data received in the receiving step in a memory inside the device;
A sequence number acquisition step of acquiring a sequence number assigned to the packet data stored in the storage step;
A continuity determination step of determining the continuity of the packet data based on the sequence number acquired in the sequence number acquisition step;
As a result of the determination in the continuity determination step, when it is determined that there is continuity, a transmission data storage step of storing the packet data to be determined in a data transmission buffer for mobile devices inside the apparatus;
As a result of the determination in the continuity determination step, if it is determined that there is no continuity, a buffer residual data confirmation step for confirming the data amount of the packet data remaining in the data transmission buffer for the mobile device;
Transmission data control for controlling the time until packet data to be determined in the continuity determining step is stored in the data transmission buffer for the mobile device according to the data amount confirmed in the remaining data confirmation step in the buffer Steps,
Transmitting the packet data stored in the data transmission buffer for the mobile device to the mobile device;
Data relay method including

Images