JP5218995B2 - Video playback terminal, video playback method and program - Google Patents

Description

  The present invention relates to a moving image reproducing terminal, a moving image reproducing method, and a program for receiving moving image data transmitted from a moving image distribution server via a network and reproducing the same in real time.

  In streaming video playback, it is required to maintain a level of video quality that does not give the user a sense of incongruity. However, when streaming video playback is performed on a thin client with a limited buffer capacity, the network bandwidth changes. As soon as the arrival of moving image data is delayed, video and audio are interrupted or lost, which makes it difficult to perform normal reproduction.

  A technique for this problem is disclosed in Patent Document 1. This Patent Document 1 discloses a delay time until reproduction of media stream data on the receiving terminal side by dynamically changing the encoding bit amount, sampling frequency, etc. of media stream data in accordance with the network bandwidth. Is disclosed to be constant.

  However, in a situation where data transmission delay due to a change in the network band has progressed, it is difficult to accurately detect the network band and synchronize between the distribution side and the reception side. When an extreme data transmission delay occurs, there is a problem in that streaming is interrupted or lost until the quality-changed stream data packet becomes valid on the receiving side.

  On the other hand, Patent Documents 2 and 3 disclose techniques for adjusting the content playback speed at the receiving terminal. In Patent Document 2, in a terminal that receives stream data transmitted from a server and reproduces it while writing it to the buffer, when the data amount in the buffer falls below a threshold, the stream reproduction speed may be reproduced at a speed lower than the normal speed. Have been described.

  Patent Document 3 describes that when the packet reception interval becomes equal to or greater than a predetermined value after the handover is completed, the application playback speed is controlled based on the amount of packets stored in the jitter buffer.

JP 2004-214755 A JP 2009-159450 A JP 2004-214902 A

  However, in the techniques described in the cited references 2 and 3, the playback speed is determined according to the amount of data or the number of packets stored in the buffer on the terminal (client) side. In a system in which data quality adjustment is performed according to the network delay situation, if data with a playback time adjusted per unit data amount arrives at the client side and is played back when a data transmission delay occurs, an appropriate playback speed is set. There was a problem that it could not be decided.

  For example, when a sufficient amount of content data that has undergone quality adjustment on the streaming distribution server side arrives at the client side, the data in the buffer is not sufficient even if it is less than the normal standard buffer data amount. In this state, if the playback speed is adjusted using the buffer data amount as an index as in Patent Document 2 or 3, playback can be performed at the normal speed (same speed). In spite of this, the low speed playback is set. If low-speed playback continues while the data that has undergone quality adjustment on the server side continues to arrive at the client, the playback speed and the amount of buffer data cannot be matched, and as a result, they do not fit in the buffer. There was a risk of missing packets being dropped. There was also a similar problem in the determination (adjustment) of high-speed playback.

  Therefore, it is difficult to apply the techniques described in the cited documents 2 and 3 to a normal stream distribution system that adjusts the moving image quality on the server side.

[the purpose]
Therefore, the present invention improves the problems of the techniques described in Patent Documents 2 and 3 above, and in moving image reproduction by streaming, continuous moving image reproduction that is not interrupted even when data arrival delay occurs due to a decrease in the line bandwidth. It is an object of the present invention to provide a video playback terminal, a video playback method, and a program that can execute the above.

  In order to achieve the above object, a video playback terminal of the present invention includes a video receiving unit that receives video data divided and sent from a server for each fixed length, and temporarily receives the received fixed-length video data. A video playback terminal comprising a video data storage buffer to be stored and a video playback unit that extracts and plays back fixed-length video data stored in the video data storage buffer in the order of reception, wherein the video receiver receives an initial fixed The normal reproduction time of each fixed-length moving image data stored in the moving image data storage buffer at a predetermined timing from when the long moving image data is received until the last fixed-length moving image data is received Calculate based on the number of image frames included in fixed-length video data and its frame rate, and calculate the total playback time in the buffer, which is the sum of these normal playback times A buffer monitoring unit and a playback speed determination unit that determines the speed of the video playback speed according to the calculated total playback time in the buffer. In reproducing data, a function of reproducing at a moving image reproduction speed determined by the reproduction speed determining unit is provided.

  In addition, the moving image playback method of the present invention includes a moving image receiving unit that receives moving image data divided and sent from the server for each fixed length, and moving image data that temporarily stores the fixed length moving image data received in this order. A video playback terminal comprising a storage buffer and a video playback unit that extracts and plays back the fixed-length video data stored in the video data storage buffer in the order of reception, wherein the first fixed-length video data is stored in the video receiver. The normal playback time of each fixed-length moving image data stored in the moving image data storage buffer at a predetermined timing from when the last fixed-length moving image data is received until the last fixed-length moving image data is received. The buffer monitoring unit calculates the total playback time in the buffer that is calculated based on the number of frames included in the frame rate and the frame rate, and is the total of each normal playback time, The playback speed determination unit determines the video playback speed according to the calculated total playback time in the buffer, and the fixed-length video data retrieved from the buffer at the video playback speed determined by the playback speed determination unit The moving picture playback unit plays back the video.

  In addition, the moving image playback program of the present invention has a buffer function for temporarily storing fixed-length moving image data in a buffer when receiving moving image data divided and sent from a server over a network. , Each fixed-length moving image data stored in the buffer at a predetermined timing from when the first fixed-length moving image data is received to when the last fixed-length moving image data is received. A buffer monitoring function for calculating a normal playback time based on the number of frames included in each fixed-length moving image data and its frame rate, and calculating a total playback time in the buffer which is the sum of the normal playback times, and the calculated buffer A playback speed determination function that determines the speed of the video playback speed according to the size of the total playback time, and a fixed value stored in the buffer. Video playback function for reproducing moving image data at a video playback speed determined in the playback speed determining section is taken out in the order received, is characterized in that to realize the computer.

  Since the present invention is configured as described above, the terminal calculates the total playback time in the buffer, which is the sum of the normal playback times of each fixed-length video data stored in the buffer, in the terminal that performs streaming video playback. Since the video playback speed is determined according to the total playback time in the buffer, the video playback speed depends on whether or not the total playback time for the data in the buffer is sufficient when a data arrival delay occurs. Therefore, it is possible to suppress interruptions and omissions during moving image reproduction, and to complete streaming moving image reproduction without causing discomfort to the user.

It is a functional block diagram which shows the structure of the moving image reproduction terminal 1 of one Embodiment concerning this invention. It is a flowchart which shows operation | movement of the moving image reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows an example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows the other example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows the other example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows the other example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows the other example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG. It is a flowchart which shows an example of reproduction | regeneration time calculation confirmation operation | movement of the moving image reproduction terminal 1 of embodiment disclosed in FIG. It is a figure which shows the other example of the buffer storage condition in the video reproduction terminal 1 of embodiment disclosed in FIG.

  Hereinafter, an embodiment according to the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing the configuration of the video playback terminal 1 of the present embodiment. As shown in FIG. 1, the video playback terminal 1 of the present embodiment is connected to a video distribution server 2 via a network 3 so as to be communicable, and plays back video data sent from the video distribution server 2 in real time. It has a function to do.

The moving image transmission unit 21 of the moving image distribution server 2 has a function of dividing moving image data into fixed-length packet units and transmitting them to the moving image playback terminal 1 in order. The first packet of the moving image data transmitted from the moving image transmitting unit 21 includes information indicating the frame rate of the moving image.
The video distribution server 2 has a function of receiving network status information such as an execution bandwidth and a transmission delay time from the video playback terminal 1 and adjusting the quality of the video data according to the network status. Also good. For example, the network status information sent from the video playback terminal 1 is configured to perform video delivery to the video playback terminal 1 using RTP (Real-time Transport Protocol) and RTCP (Real-time Transport Control Protocol). The quality of the moving image data transmitted by RTP may be adjusted according to the above.

The video playback terminal 1 of the present embodiment temporarily stores the received fixed-length video data, and the video receiver 11 that receives the video data divided and sent from the video distribution server 2 for each fixed length. The moving image data storage buffer 12, the moving image reproduction unit 16 that extracts and reproduces the fixed-length moving image data stored in the buffer 12 in the order received, and the reception interval of the fixed-length moving image data by the moving image reception unit 11, and the reception interval , The moving image reception monitoring unit 13 for determining whether or not a predetermined time has been exceeded, and the fixed values stored in the buffer 12 when the moving image reception monitoring unit 13 determines that the reception interval has exceeded a predetermined time. Based on the number of frames included in the long moving image data and the frame rate, the normal reproduction time of each fixed-length moving image data is calculated, and the total reproduction time of each normal reproduction time is calculated. A buffer monitoring section 14 that calculates a file within a total playing time, and a playback speed determining section 15 that determines the speed of video playback speed in accordance with the magnitude of the calculated total playing time in the buffer.
According to the video playback terminal 1 of this embodiment, even when a data arrival delay occurs, the video playback speed is adjusted according to whether or not the playback time that is normally possible with the data in the buffer 12 is sufficient. Therefore, it is possible to suppress interruptions and omissions during moving image reproduction, and thus it is possible to complete streaming moving image reproduction without causing discomfort to the user.

  The moving image receiving unit 11 has a function of receiving moving image data transmitted from the moving image distribution server 2 in units of fixed-length packets and sending it to the buffer 12. Further, the moving image receiving unit 11 may be configured to transmit network status information such as an execution bandwidth and a transmission delay time to the moving image distribution server 2 every time a moving image packet is received. For example, communication with the video distribution server 2 may be performed using RTP and RTCP, and network status information may be transmitted using RTCP.

  The buffer 12 has a function of temporarily storing each moving image data packet received by the moving image receiving unit 11 in order to reproduce the moving image without interruption even if the transmission speed of the moving image data changes.

  The moving image reception monitoring unit 13 monitors the function of notifying the buffer monitoring unit 14 when the moving image receiving unit 11 receives moving image data packets (fixed length moving image data) and the reception interval of moving image data packets by the moving image receiving unit 11. When the reception interval exceeds a preset standard delay time, it is determined that a “packet waiting time-out” has occurred, the buffer monitoring unit 14 is notified, and the standard delay time until the moving image data packet is received thereafter. And a function of continuously notifying every time elapses.

  The buffer monitoring unit 14 is newly sent each time it receives a packet reception notification from the moving image reception monitoring unit 13 and a function for identifying the frame rate of the moving image from information in the moving image data packet transmitted first. A function for calculating the time required for reproducing a moving image data packet at a normal speed (normal reproduction time) based on the number of frames of the video included in the moving image data packet and the frame rate described above, and the moving image reception monitoring unit 13 And a function for calculating the sum of the normal playback times of the respective moving image data packets stored in the buffer 12 as the total playback time in the buffer when receiving a packet reception notification or a timeout notification. Here, the normal reproduction time of the moving image data packet received in the initial stage may be set to the standard delay time that is an index of the packet waiting interval used by the moving image reception monitoring unit 13.

  The playback speed determination unit 15 determines whether or not the total playback time in the buffer calculated by the buffer monitoring unit 14 is below a preset threshold C, and the total playback time in the buffer is below the threshold C. In this case, the first moving image data packet of the moving image data packets waiting for reproduction in the buffer 12 based on the preset standard reproduction time in the buffer and the total reproduction time in the buffer according to the equation [1] A function for calculating an auxiliary playback time and setting a moving image playback speed according to the auxiliary playback time is provided. When this function is activated, since the total playback time in the buffer is smaller than the standard playback time in the buffer, the moving image playback speed is set to a slower speed than usual.

  Here, the standard reproduction time in the buffer is a standard value of the total reproduction time in the buffer necessary for maintaining a normal moving image reproduction speed, and is larger than the threshold value C. The standard reproduction time in the buffer may be set based on the normal reproduction time of the moving image data packet received in the initial stage.

[Equation 1]
(Auxiliary playback time of packet) = [(Normal playback time of packet) × {(Standard playback time in buffer) / (Total playback time in buffer)}]

  Furthermore, each time the playback speed determination unit 15 calculates the auxiliary playback time of the first video data packet waiting for playback, the playback speed determination unit 15 sets the motion picture playback adjustment time representing the difference between the normal playback time of the video and the actual playback time [Equation 2]. A function of calculating according to the equation and registering in the reproduction adjustment time holding unit 17 is provided.

[Equation 2]
(Movie playback adjustment time) = [(Registered movie playback adjustment time) − {(Normal playback time of packet) − (Auxiliary playback time of packet)}]

  Further, when the total playback time in the buffer is equal to or greater than the threshold value C, the playback speed determination unit 15 determines whether or not the total playback time in the buffer exceeds the standard playback time in the buffer, and the total playback time in the buffer is buffered. When the internal standard playback time is exceeded, it is determined whether or not the video playback adjustment time recorded in the playback adjustment time storage unit 17 is a positive value, and the video playback adjustment time is a positive value. Similarly to the above-described function, a function of calculating the auxiliary reproduction time of the packet according to the equation [1] and setting the moving image reproduction speed according to the auxiliary reproduction time is provided. When this function is activated, since the total playback time in the buffer is larger than the standard playback time in the buffer, the moving image playback speed is set to a speed higher than normal.

  Furthermore, the playback speed determination unit 15 stores the playback adjustment time when the total playback time in the buffer is equal to or greater than the threshold value C and equal to or shorter than the standard playback time in the buffer, or the total playback time in the buffer exceeds the standard playback time in the buffer. When the moving image reproduction adjustment time recorded in the unit 17 is “0” or a negative value, a function for determining the moving image reproduction speed to a normal speed (equal speed) is provided.

  Here, when the auxiliary reproduction time of the packet calculated according to the equation [1] exceeds the preset upper limit value A, the reproduction speed determination unit 15 determines the upper limit value A as the auxiliary reproduction time and is preset. If the lower limit value B is below the lower limit value B, the lower limit value B may be determined as the auxiliary regeneration time. Thus, by providing the upper and lower limits of the playback time for one packet, it is possible to maintain a certain standard moving image playback quality.

  An arbitrary threshold can be set for each of the upper limit A and the lower limit B, and the above-described threshold C can also be set to an arbitrary value depending on the system environment.

  The moving image reproduction unit 16 has a function of taking out the moving image data packets stored in the buffer 12 in the order of reception and reproducing them at the moving image reproduction speed set by the reproduction speed determination unit 15. A well-known method is used for changing the moving image (video / audio) reproduction speed.

  The reproduction adjustment time holding unit 17 has a function of holding data indicating the moving image reproduction adjustment time calculated by the reproduction speed determination unit 15. The moving image reproduction adjustment time is updated every time the moving image reproduction speed is changed.

  Thus, in the video playback terminal 1 of the present embodiment, the buffer monitoring unit 14 has a function of calculating the total playback time in the buffer based on the normal playback time of each video data packet stored in the buffer 12, In order to compensate for the reproduction of the moving image for the packet whose arrival is delayed, the reproduction speed determination unit 15 determines that a data transmission delay has occurred when the total reproduction time in the buffer is less than the threshold value C. 1. Set a video playback time (auxiliary playback time) of one packet longer than normal, and subtract the difference between the packet normal playback time and auxiliary playback time from the registered video playback adjustment time to create a new video A function for calculating the reproduction adjustment time and recording it in the reproduction adjustment time holding unit 17 is provided.

  In addition, the playback speed determination unit 15 determines that the total playback time in the buffer is in the buffer standard with the adjustment of the video quality on the video distribution server 2 side enabled after the data arrival delay has occurred due to the change in the network bandwidth. When the playback time is exceeded, the video playback time (auxiliary playback time) of one packet is set to be shorter than normal so as to eliminate the video playback adjustment time, and the normal playback time and auxiliary playback time of the packet are A function of subtracting the difference from the registered moving image reproduction adjustment time to calculate a new moving image reproduction adjustment time and recording it in the reproduction adjustment time holding unit 17 is provided.

  As described above, according to the video playback terminal 1 of the present embodiment, in the stream video playback processing, it is possible to adjust the playback speed according to the network bandwidth situation, and thereby the video playback due to the arrival delay of the video packet Can be minimized, and smooth moving image reproduction without missing / interrupted can be executed. Also, the video playback terminal 1 of the present embodiment executes video quality control on the video distribution server 2 side in order to set the video playback speed based on the total normal playback time of each packet stored in the buffer 12. Even in an environment, the moving image playback speed can be adjusted effectively.

  Next, the operation of the video playback terminal 1 of this embodiment will be described. Here, the following description of the operation is an embodiment of the moving image reproducing method of the present invention.

  FIG. 2 is a flowchart showing the operation of the video playback terminal 1 of the present embodiment. As shown in FIG. 2, in the video playback terminal 1 of the present embodiment, first, the video reception monitoring unit 13 receives the video data packet when the video reception unit 11 receives the video data packet when the video reception unit 11 receives the video data packet. When the interval exceeds the standard delay time, the buffer monitoring unit 14 is called, and the called buffer monitoring unit 14 calculates the total reproduction time in the buffer.

  Subsequently, the playback speed determination unit 15 compares the total playback time in the buffer calculated by the buffer monitoring unit 14 with the threshold C (step S21 in FIG. 2), and when the total playback time in the buffer is less than the threshold C. The total reproduction time in the buffer is compared with the standard reproduction time in the buffer (step S22 in FIG. 2). If the total playback time in the buffer is less than or equal to the standard playback time in the buffer, or if the video playback adjustment time is “0” or a negative value even if the total playback time in the buffer is greater than the standard playback time in the buffer, the video playback speed Is set to a normal speed (equal speed) (step S29 in FIG. 2).

  When the total playback time in the buffer is larger than the standard playback time in the buffer and the video playback adjustment time is a positive value, or when the total playback time in the buffer is greater than or equal to the threshold value C, The auxiliary reproduction time of the first moving image packet waiting for reproduction stored in the buffer 12 is calculated (step S24 in FIG. 2).

  It is determined whether or not the calculated auxiliary reproduction time is larger than the upper limit value A (step S25 in FIG. 2). If it is larger, the upper limit value A is set as the auxiliary reproduction time (step S30 in FIG. 2). If it is smaller, it is determined whether or not the auxiliary reproduction time is smaller than the lower limit value B (step S26 in FIG. 2). If smaller, the lower limit value B is set as the auxiliary reproduction time (step S31 in FIG. 2).

  Then, the video playback speed of this packet is set based on the ratio of the auxiliary playback time of this video packet to the normal playback time (step S27 in FIG. 2), and the video playback adjustment time is calculated according to the equation [2]. And registered in the reproduction adjustment time holding unit 17.

As described above, the video playback terminal 1 according to the present embodiment has a video data packet in which the video data in the buffer 12 is equal to or lower than a certain standard due to a packet arrival delay, or a video data packet whose video quality is adjusted by the video distribution server 2. Is detected to be equal to or greater than a certain standard, and the moving image playback speed is adjusted for each packet of moving image data.
In addition, when the reception delay of the moving image packet is detected, the moving image is reproduced at a reduced speed so as to compensate for the reproduction time of the moving image data corresponding to the packet whose arrival is delayed.
Also, when the adjustment of the moving image quality in the moving image distribution server 2 after the occurrence of the data transmission delay becomes effective and it is detected that the moving image data having the number of frames exceeding a certain standard is stored in the buffer 12, the moving image is increased in speed. Can be kept consistent with the normal playback time of the entire moving image.
Further, when decelerating or accelerating moving image reproduction depending on the packet arrival status, it is possible to maintain a certain standard moving image reproduction quality by providing threshold values at the upper and lower limits of the moving image data reproduction time for one packet.
Therefore, according to the video playback terminal 1 of the present embodiment, in the video playback by streaming, the playback speed can be adjusted according to the arrival delay situation of the video data, and smooth video playback without missing / interrupted can be performed. Can be executed. Also, the video playback terminal 1 of the present embodiment executes video quality control on the video distribution server 2 side in order to set the video playback speed based on the total normal playback time of each packet stored in the buffer 12. Even in an environment, the moving image playback speed can be adjusted effectively.

  Next, a specific example of the present embodiment will be described with reference to the drawings.

  In the specific example described below, the buffer standard playback time is “600 ms (initial normal playback time 100 ms × 6 packets)”, and the threshold C for the buffer total playback time is “420 ms (70% of the buffer standard playback time)”. It is assumed that the upper limit value A of the auxiliary playback time for one packet is set to 200% of the normal playback time, and the lower limit value B of the auxiliary playback time is set to 50% of the normal playback time.

  FIG. 3 is a diagram illustrating an example of a data storage state of the buffer 12 when a transmission delay is detected in the initial stage. FIG. 3 shows a state in which the packets 31, 32, 33, and 34 are stored in the buffer 12. Here, it is assumed that the moving image reproduction adjustment time is registered as “0 ms” at this time.

  In this state, when the moving image reception monitoring unit 13 detects a packet waiting timeout, the buffer monitoring unit 14 sets the normal reproduction time of each packet 31, 32, 33, 34 stored in the buffer 12. It is calculated individually and the total is calculated as the total playback time in the buffer.

  Here, assuming that the normal reproduction times of the packets 31, 32, 33, and 34 are both “100 ms” and the total reproduction time in the buffer is “400 ms”, the reproduction speed determination unit 15 performs the total reproduction time in the buffer. It is detected that “400 ms” is lower than the threshold C “420 ms” and it is determined that a transmission delay has occurred, and the auxiliary reproduction time of the packet 31 is set to “150 ms = {100 ms × (600 ms) according to the equation [1]. / 400 ms)} ”, and it is determined that the auxiliary reproduction time“ 150 ms ”of the packet 31 is within the range of the lower limit value B“ 50 ms ”and the upper limit value A“ 200 ms ”. “150 ms” is set, and the reproduction speed of the packet 31 is determined.

  Further, in order to match the moving image playback time, a value “50 ms = {0 ms− (100 ms−150 ms)}” obtained by subtracting the difference between the playback time of the packet 31 and the normal playback time from the registered moving image playback adjustment time is obtained. Register as a new video playback adjustment time.

  Next, a case where the auxiliary reproduction time exceeds the upper limit value A will be described. FIG. 4 shows a state in which only the packet 41 having the normal reproduction time “200 ms” is stored in the buffer 12. Here, the moving image reproduction adjustment time recorded at this time is “200 ms”.

  When the moving image reception monitoring unit 13 detects a packet waiting timeout in such a state, the playback speed determining unit 15 sets the auxiliary playback time of the packet 41 to “600 ms = 200 ms × ( 600 ms / 200 ms) ”, and it is detected that“ 600 ms ”exceeds the upper limit value A“ 400 ms ”, so that the playback time of the packet 41 is the same as the upper limit value A in order to maintain the video quality. Is set to “400 ms” and the playback speed is calculated.

  Further, in order to match the moving image reproduction time, the difference between the normal reproduction time and the reproduction time of the packet 41 is subtracted from the previously calculated moving image reproduction adjustment time to calculate and register a new moving image reproduction adjustment time “400 ms”.

  Next, a case where a moving image data packet whose moving image quality has been adjusted on the moving image distribution server 2 side after the transmission delay is detected will be described. FIG. 5 shows a case where a packet 52 having a normal playback time “300 ms” is newly received and stored in a state where only the packet 51 having a normal playback time “200 ms” is stored in the buffer 12.

  When the packet 52 is received, the buffer monitoring unit 14 calculates the total reproduction time “500 ms” in the buffer. Then, the playback speed determination unit 15 determines that the total playback time “500” ms in the buffer exceeds the threshold C “420 ms” and is equal to or shorter than the standard playback time “600 ms” in the buffer, and adjustment of the playback time is unnecessary. Thus, the playback time of the packet 51 is set to the normal playback time “200 ms”, and the playback speed is calculated.

  Next, a case where a moving image data packet whose moving image quality has been adjusted on the moving image distribution server 2 side after the transmission delay is detected arrives and the total reproduction time in the buffer 12 reaches the standard reproduction time in the buffer will be described.

  FIG. 6 shows a case where the normal reproduction time “400 ms” packet 62 is newly received and stored in the state where only the packet 61 of the normal reproduction time “400 ms” is stored in the buffer 12. Here, the moving image reproduction adjustment time registered at this time is “600 ms”.

  In such a state, the total reproduction time in the buffer is “800 ms”. The playback speed determination unit 15 has a value in which the total playback time “800 ms” in the buffer exceeds the threshold C “420 ms” and the standard playback time “600 ms” in the buffer, and the registered video playback adjustment time is a positive value. If it is confirmed that it is necessary to match the reproduction times, the auxiliary reproduction time of the packet 61 is calculated as “300 ms = {400 ms × (600 ms / 800 ms)}” according to the formula [1], and this packet is calculated. 61 is determined that the auxiliary reproduction time “300 ms” is not less than the lower limit B “200 ms” and not more than the upper limit B “800 ms”, the reproduction time of the packet 61 is set to “300 ms”, and the reproduction speed is calculated. To do.

  In this case, since the playback speed is set to be faster than the standard in order to compensate for the video playback adjustment time, the new video playback adjustment time is “500 ms = {600 ms− (400 ms−300 ms)}”, compared to the previous time. Decrease.

  Next, after the transmission delay is detected, a video data packet whose video quality has been adjusted on the video distribution server 2 side arrives, and the auxiliary playback time calculated in a state where the total playback time in the buffer 12 has reached the standard playback time in the buffer Will be described below.

  FIG. 7 shows a case where a packet 73 having a normal playback time of “500 ms” is newly received and stored in a state where packets 71 and 72 having a normal playback time of “400 ms” are stored in the buffer 12. Here, it is assumed that the auxiliary moving image reproduction adjustment time registered at this time is “600 ms”.

  In this state, since the total playback time in the buffer is “1300 ms”, the playback speed determination unit 16 sets the total playback time in the buffer to “1300 ms” as the threshold C “420 ms” and the standard playback time in the buffer “600 ms”. When the registered video playback adjustment time is a positive value, it is determined that the video playback time needs to be matched, and the packet 71 The auxiliary playback time is calculated as “184 ms = {400 ms × (600 ms / 1300 ms)}”, and it is determined that the auxiliary playback time “184 ms” of the packet 71 is below the lower limit B “200 ms”. In order to keep it, the lower limit B “200 ms” is set as the playback time of the packet 71, and the playback speed is calculated.

  In this case, in order to compensate for the moving image reproduction adjustment time, the reproduction speed is set to be faster than the standard, so the new moving image reproduction adjustment time is “400 ms = {600 ms− (400 ms−200 ms)}”, which is compared with the previous time. Decrease.

  Thus, according to the present embodiment, in addition to the dynamic change of the video quality (encoding bit amount / sampling frequency) in the video distribution server 2 when the network bandwidth is insufficient, the video playback terminal 1 By adjusting the, continuous video reproduction without interruption can be realized.

  Next, the operation when the network delay is restored to the normal level and the moving image data packet starts to arrive in the normal state will be described with reference to FIGS.

  FIG. 8 is a flowchart showing an operation of determining whether the reproduction time calculation is necessary / unnecessary by the buffer monitoring unit 14 when a packet waiting timeout or packet arrival is notified from the moving image reception monitoring unit 13. FIG. 9 is an explanatory diagram showing an example of state transition of the buffer 12 in a state where the delay is recovered to the normal level communication state after the arrival delay of the moving image data occurs.

  First, it is assumed that the packet 91 is stored in the buffer 12 in a state where a moving image packet whose reproduction time is longer than the standard is reproduced because the moving image quality is adjusted on the moving image distribution server 2 side (state A in FIG. 9). ). When a packet with a long reproduction time for which moving image quality adjustment has been performed is reproduced with the data arrival delay restored to a normal state, a plurality of packets arrive while the packet 91 is waiting for reproduction processing.

  For example, when a packet waiting timeout occurs (state B1 in FIG. 9) in a state where the reproduction time of the packet 91 waiting for reproduction is not set (calculated), or when a packet 92 arrives (step in FIG. 8) S81, state B2 in FIG. 9), the reproduction time of the packet 91 waiting for reproduction is calculated as usual (step S82 in FIG. 8).

  Subsequently, when the packet 93 arrives while waiting for the reproduction process of the packet 91 (state C in FIG. 9), the reproduction time of the packet 91 waiting for the reproduction process is already set, so the packet 91 Is not calculated (Yes in step S81 in FIG. 8). Thereafter, when a new packet arrives while the packet 91 is waiting for reproduction processing, or when a packet wait timeout occurs, the reproduction time of the reproduction waiting packet is not calculated. When the packet 91 shifts to the reproduction process from this state and the packet 92 becomes the head of the buffer 12 waiting for the reproduction process, when the next packet arrives or when a packet wait timeout occurs (step of FIG. 8). No in S81), the reproduction time of the packet 92 is calculated (step S82 in FIG. 8, state D in FIG. 9).

  As described above, when the data arrival delay is restored to a normal level after the adjustment of the video quality and the adjustment of the video playback time occurs, by performing the execution determination of the playback time according to the playback wait state, The effect that the playback time is calculated at the optimum timing can be obtained.

Here, in the above description, when the moving image reception monitoring unit 13 receives the moving image data packet by the moving image receiving unit 11 and when the receiving interval of the moving image data packet by the moving image receiving unit 11 exceeds the standard delay time, A function of notifying the monitoring unit 14, and a function of calculating the normal playback time of the newly transmitted video data packet each time the buffer monitoring unit 14 receives a packet reception notification from the video reception monitoring unit 13, and a video A function of calculating the sum of the normal playback times of the respective video data packets stored in the buffer 12 as the total playback time in the buffer when receiving a packet reception notification from the reception monitoring unit 13 or receiving a timeout notification The buffer monitoring unit 14 calculates the normal playback time of each video data packet and the buffer. The timing for calculating the total reproduction time in the video is not limited to this, but is an appropriate timing from when the first fixed-length moving image data is received to when the moving image receiving unit 11 receives the last fixed-length moving image data. The timing may not necessarily be the timing corresponding to the reception status of the moving image data packet.
Therefore, the moving image playback terminal 1 of the present embodiment may be configured not to be equipped with the moving image reception monitoring unit 13 but so that the buffer monitoring unit 14 calculates the total playback time in the buffer at a predetermined timing. The predetermined timing may be, for example, a periodic timing, a timing each time the first moving image packet waiting for reproduction is taken out from the buffer 12, or the like.
Furthermore, the above-described moving image receiving unit 11, moving image reception monitoring unit 12, buffer monitoring unit 14, reproduction speed determining unit 15, and moving image reproduction unit 16 are configured so that their function contents are programmed and executed by a computer. May be.

  As described above, according to the video playback terminal 1 of the present embodiment, in real-time video playback, when the reception interval of video data packets exceeds the delay time, the video distribution server 2 prepares for the progress of network delay. As a mechanism for compensating for the time lag until the moving image data whose moving image quality has been adjusted actually arrives at the moving image playback terminal 1, the moving image playback speed can be changed according to the delay time of arrival of the packet. Can be played back at low speed in a manner that compensates for packets that have arrived late.

  In addition, when the adjustment of the moving image quality in the moving image distribution server 2 according to the data arrival delay state becomes effective and moving image data exceeding a certain standard is stored in the buffer 12, the moving image is reproduced at a higher speed. Thus, it is possible to maintain consistency with the standard playback time of the entire moving image.

  In addition, since the video playback terminal 1 according to the present embodiment determines the playback speed in units of packets, the video playback speed can be changed in stages, and changes in sound quality and video disturbance due to changes in the video playback speed can be achieved. Can be relaxed.

  The video playback speed adjustment processing executed by the video playback terminal 1 according to the present embodiment is the video delivery after the data transmission delay occurs when the network bandwidth adjustment by the video quality adjustment is executed on the video delivery server 2 side. It can be employed as a primary adjustment process until the relaxation of the delay by adjusting the moving image quality in the server 2 becomes effective.

  The main points of the technical contents in the above-described embodiment will be added below. In addition, although a part or all of this embodiment is put together in the content described below, this invention is not necessarily limited to this.

[Supplementary Note 1] A moving image receiving unit that receives moving image data divided and sent from the server for each fixed length, a buffer that temporarily stores the received fixed length moving image data, and a buffer that is stored in the buffer A video playback terminal including a video playback unit that extracts and plays back fixed-length video data in the order received;
Each fixed-length moving image data stored in the buffer at a predetermined timing from when the first fixed-length moving image data is received to the moving image receiving unit until the last fixed-length moving image data is received. A buffer monitoring unit that calculates the reproduction time based on the number of frames included in each fixed-length moving image data and the frame rate thereof, and calculates the total reproduction time in the buffer that is the sum of each normal reproduction time;
A playback speed determining unit that determines a video playback speed according to the calculated total playback time in the buffer;
A video playback terminal, wherein the video playback unit has a function of playing back at a video playback speed determined by the playback speed determination unit when the fixed-length video data retrieved from the buffer is played back.
[Supplementary Note 2] In the video playback terminal according to Supplementary Note 1, the playback speed determination unit may determine a standard that is defined in advance when the total playback time in the buffer calculated by the buffer monitoring unit is smaller than a preset threshold value. A video playback terminal comprising a function of calculating a video playback speed according to a ratio of the total playback time in the buffer to a value.
[Supplementary Note 3] In the video playback terminal described in Supplementary Note 2, the playback speed determination unit plays the video when the total playback time in the buffer calculated by the buffer monitoring unit is larger than the threshold and smaller than the standard value. A video playback terminal having a function of setting a speed to a normal speed.
[Appendix 4] In the video playback terminal according to Appendix 3, the playback speed determination unit may determine whether the playback speed determination unit stores the buffer in relation to the standard value when the total playback time in the buffer calculated by the buffer monitoring unit is greater than the standard value. A video playback terminal having a function of calculating a video playback speed in accordance with a ratio of a total playback time.
[Supplementary Note 5] In the video playback terminal described in Supplementary Note 4, the playback speed determination unit may determine that the video playback speed calculated according to the ratio of the total playback time in the buffer to the standard value is lower than a preset lower limit speed. In such a case, the video playback terminal determines the lower limit speed as the video playback speed.
[Appendix 6] In the video playback terminal according to Appendix 5, the playback speed determination unit has a video playback speed calculated according to a ratio of the total playback time in the buffer with respect to the standard value exceeds a preset upper limit speed. A moving image playback terminal that determines the upper limit speed as the moving image playback speed.
[Appendix 7] In the video playback terminal according to Appendix 6, the reception interval of the fixed-length video data by the video reception unit is monitored, and the buffer monitoring unit is called when the reception interval exceeds a predetermined time. A video playback terminal comprising a video reception monitoring unit, wherein the buffer monitoring unit employs a timing called from the video reception monitoring unit as the predetermined timing.
[Supplementary Note 8] In the video playback terminal described in Supplementary Note 7, the video reception monitoring unit includes a function of calling the buffer monitoring unit every time fixed-length video data is received by the video reception unit. To play video.
[Supplementary Note 9] A moving image receiving unit that receives moving image data divided and sent from the server for each fixed length, a buffer that temporarily stores the fixed length moving image data received in this order, and a buffer that stores the moving image data. A video playback terminal including a video playback unit that extracts and plays back the fixed-length video data in the order received,
Each fixed-length moving image data stored in the buffer at a predetermined timing from when the first fixed-length moving image data is received to the moving image receiving unit until the last fixed-length moving image data is received. The playback time is calculated based on the number of frames included in each fixed-length moving image data and its frame rate, and the buffer monitoring unit calculates the total playback time in the buffer, which is the sum of the normal playback times, and the calculated buffer The video playback speed is determined by the playback speed determination unit according to the size of the total playback time, and the video playback unit plays back the fixed-length video data retrieved from the buffer at the video playback speed determined by the playback speed determination unit. A method for reproducing a moving image.
[Supplementary Note 10] In the moving image reproduction method according to Supplementary Note 9, when the reproduction speed determination unit determines the moving image reproduction speed, a standard specified in advance when the total reproduction time in the buffer is smaller than a preset threshold value. A moving image reproduction method, wherein a moving image reproduction speed is calculated according to a ratio of the total reproduction time in the buffer to a value.
[Appendix 11] In the video playback method according to Appendix 10, when the video playback speed is determined by the playback speed determination unit, the video playback is performed when the total playback time in the buffer is larger than the threshold and smaller than the standard value. A method for reproducing a moving image, wherein the speed is set to a normal speed.
[Supplementary Note 12] In the moving picture reproduction method according to Supplementary Note 11, when the reproduction speed determination unit determines the moving picture reproduction speed, when the total reproduction time in the buffer is larger than the standard value, A moving image reproduction method, wherein a moving image reproduction speed is calculated according to a ratio of a total reproduction time.
[Supplementary Note 13] In the video playback method described in Supplementary Note 12, the playback speed determination unit is configured such that the video playback speed calculated according to the ratio of the total playback time in the buffer to the standard value falls below a preset lower limit speed. In such a case, the moving image reproduction method is characterized in that the lower limit speed is determined as the moving image reproduction speed.
[Supplementary Note 14] In the video playback method according to supplementary note 13, the playback speed determination unit has a video playback speed calculated according to a ratio of the total playback time in the buffer with respect to the standard value exceeds a preset upper limit speed. In this case, the upper limit speed is determined as the moving picture playback speed.
[Supplementary Note 15] In the video reproduction method described in Supplementary Note 14, the timing at which the reception interval of the fixed-length video data by the video reception unit reaches a predetermined time is adopted as the predetermined timing. A video playback method.
[Supplementary Note 16] The video reproduction method according to Supplementary Note 15, wherein a timing at which fixed-length video data is received by the video reception unit is adopted as the predetermined timing.
[Supplementary Note 17] A buffer function for temporarily storing the fixed-length moving image data in the buffer when receiving moving image data that is divided and sent in fixed lengths from the server via the network; Normal playback time of each fixed-length video data stored in the buffer at a predetermined timing from when the fixed-length video data is received until the last fixed-length video data is received. A buffer monitoring function that calculates the total playback time in the buffer, which is the sum of these normal playback times, calculated based on the number of frames included in the long video data and the frame rate, and the size of the calculated total playback time in the buffer. A playback speed determination function for determining the video playback speed in response to the above, and taking out the fixed-length video data stored in the buffer in the order of reception. Video playback program characterized by executing video playback function of reproducing at a serial playback speed determining unit video playback speed determined in, to the computer.
[Supplementary note 18] In the moving image reproduction program according to supplementary note 17, a standard defined in advance when the reproduction speed determination function is less than a preset threshold value calculated in the buffer monitoring function. A moving image reproduction program characterized by having a function of calculating the moving image reproduction speed in accordance with a ratio of the total reproduction time in the buffer to a value.
[Supplementary note 19] In the moving picture reproduction program according to supplementary note 18, when the reproduction speed determination function is set so that the total reproduction time in the buffer calculated by the buffer monitoring unit is larger than the threshold value and smaller than the standard value, A moving picture playback program characterized by having a function for setting a playback speed to a normal speed.
[Supplementary note 20] In the moving image reproduction program according to supplementary note 19, when the reproduction speed determination function is within the buffer for the standard value when the total reproduction time in the buffer calculated by the buffer monitoring function is greater than the standard value A moving image reproduction program characterized by having a function of calculating the moving image reproduction speed in accordance with a ratio of a total reproduction time.
[Supplementary note 21] In the video playback program according to supplementary note 20, the playback speed determination function sets a lower limit speed in which the video playback speed calculated according to the ratio of the total playback time in the buffer to the standard value is set in advance. A moving image reproduction program characterized by having a function of determining the lower limit speed as the moving image reproduction speed when the lower limit is exceeded.
[Supplementary Note 22] In the video playback program described in Supplementary Note 21, the video playback speed calculated by the playback speed determination function according to the ratio of the total playback time in the buffer to the standard value is a preset upper limit speed. A moving image reproduction program characterized by having a function of determining the upper limit speed as the moving image reproduction speed when the upper limit is exceeded.
[Supplementary Note 23] In the video playback program described in Supplementary Note 22, the buffer monitoring function employs a timing at which the reception interval of the fixed-length video data reaches a predetermined time as the predetermined timing. A video playback program characterized by being a function.
[Supplementary Note 24] In the video reproduction program according to Supplementary Note 23, the buffer monitoring function is a function that adopts a timing at which the fixed-length video data is received as the predetermined timing. Movie playback program.

  INDUSTRIAL APPLICABILITY The present invention can be applied to a moving image reproduction method when receiving and reproducing moving image stream data that is a resource on the server side in an environment where resources such as a thin client are limited.

DESCRIPTION OF SYMBOLS 1 Movie reproduction terminal 2 Movie distribution server 3 Network 11 Movie reception unit 12 Buffer 13 Movie reception monitoring unit 14 Buffer monitoring unit 15 Playback speed determination unit 16 Movie playback unit 17 Playback adjustment time storage unit 21 Movie transmission unit

Claims (10)

A moving image receiving unit that receives moving image data divided and sent from the server for each fixed length, a moving image data storage buffer that temporarily stores the received fixed length moving image data, and an accumulation in the moving image data storage buffer A video playback terminal including a video playback unit that extracts and plays back the fixed length video data in the order of reception,
Each fixed length moving image accumulated in the moving image data storage buffer at a predetermined timing from when the first fixed length moving image data is received to the moving image receiving unit until the last fixed length moving image data is received A buffer monitoring unit that calculates a normal reproduction time of data based on the number of image frames included in each fixed-length moving image data and its frame rate, and calculates a total reproduction time in the buffer that is a sum of each normal reproduction time;
A playback speed determination unit that determines the speed of the video playback speed according to the calculated total playback time in the buffer;
A video playback terminal, wherein the video playback unit has a function of playing back at a video playback speed determined by the playback speed determination unit when the fixed-length video data retrieved from the buffer is played back. In the video playback terminal according to claim 1,
The playback speed determination unit, when the total playback time in the buffer calculated by the buffer monitoring unit is smaller than a preset threshold value, the moving image according to the ratio of the total playback time in the buffer to the predetermined standard value A video playback terminal comprising a function for setting a playback speed. In the video playback terminal according to claim 2,
The playback speed determination unit has a function of setting the video playback speed to a normal speed when the total playback time in the buffer calculated by the buffer monitoring unit is larger than the threshold and smaller than the standard value. Feature video playback terminal. In the video playback terminal according to claim 3,
When the total playback time in the buffer calculated by the buffer monitoring unit is larger than the standard value, the playback speed determination unit sets the video playback speed from the normal value according to the ratio of the total playback time in the buffer to the standard value. A video playback terminal characterized by a high-speed setting function. In the video playback terminal according to claim 4,
The playback speed determination unit determines the lower limit speed as the video playback speed when the video playback speed set based on the ratio of the total playback time in the buffer to the standard value falls below a preset lower speed limit. A video playback terminal characterized by having a function to perform. In the video playback terminal according to claim 5,
The playback speed determination unit determines the upper limit speed as the video playback speed when the video playback speed set based on the ratio of the total playback time in the buffer to the standard value exceeds a preset upper limit speed. A video playback terminal characterized by having a function to perform. In the video playback terminal according to claim 6,
A video reception monitoring unit that monitors a reception interval of fixed-length video data by the video reception unit and calls the buffer monitoring unit when the reception interval reaches a predetermined time,
The video playback terminal characterized in that the buffer monitoring unit employs a timing called from the video reception monitoring unit as the predetermined timing. In the video playback terminal according to claim 6,
The video playback terminal, wherein the video reception monitoring unit has a function of calling the buffer monitoring unit every time fixed-length video data is received by the video reception unit. A moving image receiving unit that receives moving image data divided and sent from the server for each fixed length, a moving image data storage buffer that temporarily stores fixed-length moving image data received in this order, and a moving image data storage buffer A video playback terminal including a video playback unit that extracts and plays back the stored fixed-length video data in the order received;
Each fixed length moving image accumulated in the moving image data storage buffer at a predetermined timing from when the first fixed length moving image data is received to the moving image receiving unit until the last fixed length moving image data is received The normal monitoring time of the data is calculated based on the number of frames included in each fixed-length moving image data and the frame rate, and the buffer monitoring unit calculates the total playback time in the buffer that is the sum of each normal playback time,
The playback speed determination unit determines the video playback speed according to the calculated total playback time in the buffer,
The moving image reproducing method, wherein the moving image reproducing unit reproduces the fixed-length moving image data extracted from the buffer at a moving image reproducing speed determined by the reproduction speed determining unit. A buffer function that temporarily stores fixed-length video data in a buffer when it receives video data that is divided and sent from the server over the network.
Each fixed-length moving image data stored in the buffer at a predetermined timing from when the first fixed-length moving image data is received to the moving image receiving unit until the last fixed-length moving image data is received. A buffer monitoring function for calculating a reproduction time based on the number of frames included in each fixed-length moving image data and its frame rate, and calculating a total reproduction time in the buffer which is the sum of the respective normal reproduction times;
A playback speed determination function that determines the speed of the video playback speed according to the calculated total playback time in the buffer,
And a moving picture playback function for taking out the fixed length moving picture data stored in the buffer in the order of reception and playing it at the moving picture playback speed determined by the playback speed determination unit,
A program for reproducing a moving picture, characterized in that a computer is realized.

Images