JP2017204700A - Image reproduction apparatus, image reproduction method, and image reproduction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image reproduction device which is capable of expanding the range of a reproduction buffer period in which an unreproduced image is reproduced at double speed while preventing the exhaustion of the unreproduced image.SOLUTION: An image reproduction device 20 comprises: a measurement part 21 which measures a reception interval being a time interval between receptions of image frames; and a determination part 22 which determines, on the basis of the measured reproduction interval, the lower limit value of an imaging period in which the unreproduced image is reproduced in the same period as an imaging period being a period in which the unreproduced image is captured.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a video playback device, a video playback method, and a video playback program, and in particular, a video that prevents a video from being stopped and maintains a quality of experience for a user even when a change occurs in a communication network that distributes the video. The present invention relates to a playback device, a video playback method, and a video playback program.

  In video distribution via a communication network, for example, an adaptive rate control technique is used. Adaptive rate control technology prevents video playback from being stopped due to video data arrival delay by distributing video while dynamically changing the bit rate (image quality) of video to be distributed according to bandwidth fluctuations. Technology.

  However, when video is distributed via a communication network having a large bandwidth fluctuation, it is not sufficient to control the bit rate of the video distributed by the distribution terminal in order to prevent the reproduction of the video from being stopped.

  The reason for this is that when video is distributed via a communication network whose bandwidth varies greatly, for example, when the bandwidth is narrow, it is stored in a playback video buffer (hereinafter referred to as a playback buffer) of the video playback terminal. This is because the distributed video data may be exhausted. When the video data stored in the playback buffer is exhausted, video playback stops.

  Stopping video playback due to the depletion of video data stored in the playback buffer is a factor that greatly reduces the quality of experience (QoE), which is the service quality felt by video viewers for video distribution services.

  In order to solve the problem of video playback stoppage due to depletion of video data stored in the playback buffer, for example, video playback control that prevents playback stoppage by adjusting the playback speed of the video delivered by the video playback terminal, for example Technology is used. When the video reproduction control technology is used, the quality of experience for the video viewer is prevented from being lowered.

  As an example of the video reproduction control technique, there is a TCP quality measurement system that estimates the quality in data transfer by TCP (Transmission Control Protocol) described in Patent Document 1. The TCP quality measurement system described in Patent Document 1 sets an upper limit value and a lower limit value in advance in a playback buffer time that is a playback time of an unplayed video stored in the playback buffer. The playback buffer time corresponds to the time when an unplayed video is shot.

  The TCP quality measurement system described in Patent Document 1 plays back an unplayed video at a speed higher than the normal speed when the playback buffer time is greater than or equal to the upper limit, and does not play back when the playback buffer time is less than the lower limit. Is played at a lower speed than the normal speed. Further, the TCP quality measurement system described in Patent Document 1 reproduces an unreproduced video at the same speed when the reproduction buffer time is larger than the lower limit value and smaller than the upper limit value.

JP 2004-140596 A

  In the TCP quality measurement system described in Patent Document 1, the minimum unit of control is playback buffer time. The playback buffer time is the total display time per frame. Therefore, when the frame rate is small and the display time per frame is long, the control time for each frame is also long, so that the control interval by the TCP quality measurement system is long.

  In addition, the TCP quality measurement system described in Patent Document 1 adjusts the video playback speed according to the fluctuation of the playback buffer time with reference to a predetermined threshold. The TCP quality measurement system described in Patent Document 1 has a problem that it is difficult to finely adjust the video playback speed according to changes in the state of the communication network.

  For example, when the TCP quality measurement system described in Patent Document 1 is used for live video distribution or the like that requires no significant delay in video arrival, the TCP quality measurement system stores it in a playback buffer. Therefore, it is required to make the amount of unreproduced video relatively small. The reason is that if there is a large amount of unreproduced video stored in the playback buffer, a video with a large time difference from shooting to viewing may occur.

  That is, when the TCP quality measurement system described in Patent Document 1 is used for live video distribution or the like, the difference between the upper limit value and the lower limit value, which is a threshold value, is the playback buffer time for video playback at the same speed. The range is reduced.

  When the range in which the video is played back at the same speed is reduced, the playback buffer time frequently exceeds the threshold value, so that the playback speed of the video is frequently changed. That is, the reduction of the range in which video is reproduced at the same speed may cause a reduction in the quality of experience for the video viewer.

  Further, when the lower limit value is changed to a minimum value with respect to the upper limit value, the range in which the video is reproduced at the same speed is widened. However, if the range in which the video is reproduced at the same speed is expanded, the opportunity for the video to be reproduced at a low speed is reduced.

  That is, when video is distributed via a communication network with a large bandwidth fluctuation, there is a concern that video playback may be stopped due to depletion of video data stored in the playback buffer due to the above contents.

  As described above, when video playback technology that controls playback speed with reference to a predetermined threshold is used in video distribution over a communication network where bandwidth fluctuation is significant, video is used to prevent playback from stopping. Is reproduced at a normal speed. However, if the range in which the video is reproduced at the same speed is reduced, there is a problem that the quality of experience for the video viewer is lowered.

  In view of the above contents, there is a need for a technique that can expand the range in which video is played at the same speed so that unplayed video is not exhausted according to the state of the communication network used for video distribution.

  Therefore, an object of the present invention is to provide a video playback device, a video playback method, and a video playback program capable of extending the range of playback buffer time that is played back at the same speed while preventing depletion of unplayed video.

  A video playback apparatus according to the present invention includes a measuring unit that measures a reception interval that is an interval of time at which a video frame is received, and a shooting time that is played back at the same time as a shooting time that is a time when an unplayed video is shot. And a determining unit that determines a lower limit value based on the measured reception interval.

  The video playback method according to the present invention measures the reception interval, which is the time interval when video frames are received, and the lower limit value of the shooting time during which the playback time is the same as the shooting time when the unplayed video is shot. Is determined based on the measured reception interval.

  The video playback program according to the present invention is played back by a computer at the same time as a measurement process for measuring a reception interval, which is an interval of time when a video frame is received, and a shooting time, which is a time when an unplayed video is shot. And determining processing for determining a lower limit value of the photographing time based on the measured reception interval.

  According to the present invention, it is possible to extend the range of the playback buffer time that is played back at the same speed while preventing the unplayed video from being depleted.

1 is a block diagram showing a configuration example of a first embodiment of a video reproduction system 10 according to the present invention. 6 is a flowchart showing an operation of reception processing by the reception unit 201. 6 is a flowchart illustrating an operation of threshold determination processing by a second threshold determination unit 205. 7 is a flowchart showing an operation of display time determination processing by a frame display time determination unit 206. It is explanatory drawing which shows the correspondence of the receiving interval of a video frame when the video reproduction system 10 of this embodiment is used, and a 2nd threshold value. It is a block diagram which shows the outline | summary of the video reproduction apparatus by this invention.

<First Embodiment>
[Description of configuration]
DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, a first embodiment of the invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of a first embodiment of a video reproduction system 10 according to the present invention.

  The video reproduction system 10 shown in FIG. 1 is capable of reproducing video even if arrival delay or burst arrival of video data distributed from a distribution terminal occurs due to fluctuations in the communication network or the like during reproduction of video distributed via a communication network. This is a video playback system that prevents the stop of playback and maintains the quality of experience for the user. Note that burst arrival is an event in which a large amount of video data arrives at once.

  As shown in FIG. 1, the video playback system 10 of this embodiment includes a distribution terminal 100, a playback terminal 200, and a communication network 300. The distribution terminal 100 is communicably connected to the playback terminal 200 via the communication network 300.

  The distribution terminal 100 and the reproduction terminal 200 of this embodiment are personal computers provided with a communication device such as a network interface card. Further, the distribution terminal 100 and the playback terminal 200 may be terminal devices other than personal computers.

  In addition, the communication network 300 of the present embodiment is a communication network configured by a WAN (Wide Area Network) such as the Internet or a mobile network.

  In this embodiment, data transmitted from the distribution terminal 100 to the playback terminal 200 is, for example, stream information obtained by encoding video data captured by the distribution terminal 100.

  As illustrated in FIG. 1, the distribution terminal 100 includes an encoding unit 101, a transmission unit 102, a throughput measurement unit 103, and a parameter determination unit 104.

  The encoding unit 101 has a function of encoding a video frame to be distributed using an encoder (not shown) in which designated parameters are set.

  The transmission unit 102 has a function of transmitting the video frame encoded by the encoding unit 101 to the playback terminal 200 via the communication network 300. The transmission unit 102 transmits, for example, an encoded video frame by TCP.

  The throughput measuring unit 103 has a function of measuring the transmission throughput of the transmitting unit 102. Further, the parameter determination unit 104 has a function of determining a parameter set in the encoder that encodes the next video frame based on the measurement result by the throughput measurement unit 103.

  The distribution terminal 100 included in the video reproduction system 10 of the present embodiment may be a simple distribution terminal including only the transmission unit 102 that transmits the encoded video frame.

  Further, as shown in FIG. 1, the playback terminal 200 includes a receiving unit 201, a decoding unit 202, a video frame buffering unit 203, a reception interval measuring unit 204, a second threshold value determining unit 205, a frame display A time determination unit 206 and a frame display unit 207 are included.

  The receiving unit 201 has a function of receiving an encoded video frame distributed from the distribution terminal 100. The decoding unit 202 has a function of decoding the encoded video frame received by the receiving unit 201.

  The video frame buffering unit 203 has a function of buffering the video frame decoded by the decoding unit 202 in a reproduction buffer. The video frame buffering unit 203 has a playback buffer. A plurality of rectangles positioned below the video frame buffering unit 203 shown in FIG. 1 represent buffered video frames.

  The reception interval measurement unit 204 has a function of measuring the arrival interval of the video frames received by the reception unit 201 at the playback terminal 200. Further, the second threshold value determination unit 205 has a function of determining a second threshold value to be described later based on the arrival interval measurement value by the reception interval measurement unit 204.

  The frame display time determination unit 206 has a function of determining the display time of the video frame. The frame display time determining unit 206 includes a first threshold given in advance, a second threshold given from the second threshold determining unit 205, and a playback buffer time of the video frame buffered in the video frame buffering unit 203. Based on the above, the display time of the video frame is determined as will be described later. Note that the first threshold value in the present embodiment is a target delay time from when video shooting is started until the captured video becomes viewable.

  The frame display unit 207 has a function of extracting the buffered video frame from the video frame buffering unit 203 and displaying the video frame extracted for the display time of the video frame determined by the frame display time determination unit 206.

[Description of operation]
Hereinafter, the operation of the playback terminal 200 of the video playback system 10 of the present embodiment will be described with reference to FIGS.

  First, reception processing by the reception unit 201 of the playback terminal 200 will be described. FIG. 2 is a flowchart illustrating an operation of reception processing by the reception unit 201.

  The receiving unit 201 receives the i-th video frame (i = 1, 2,..., N) distributed from the distribution terminal 100 (step S101). Hereinafter, the i-th video frame is also referred to as “frame i”.

  Next, the receiving unit 201 holds the time when the frame i is received as t (i). Further, the receiving unit 201 acquires the time stamp pts_i included in the frame i (step S102).

  Next, the receiving unit 201 calculates the frame rate fps_i set for the frame i (step S103). As shown in FIG. 2, the receiving unit 201 calculates the frame rate fps_i using the following equation.

  fps_i = 1000 / (pts_i-pts_ (i-1)) [fps] Expression (1)

  Note that fps (frames per second) is a unit of fps_i calculated by Expression (1). The frame rate calculation method is changed according to the type of time stamp. The receiving unit 201 may calculate the frame rate fps_i using an expression other than the expression (1) in step S103.

  The receiving unit 201 holds the reception time t (i) and the frame rate fps_i. After the holding, the receiving unit 201 confirms whether or not the received i-th video frame is the n-th video frame (step S104).

  When i <n, that is, when the video frame is not the n-th video frame (True in Step S104), the receiving unit 201 adds 1 to i (Step S105). Next, the receiving unit 201 performs the process of step S101 again.

  When i <n is not true, that is, when the video frame is the n-th video frame (False in step S104), the reception unit 201 ends the reception process.

  As described above, the receiving unit 201 calculates the frame rate fps_i of the received i-th video frame based on the time stamp difference set in the video frame. The unit of the calculated frame rate is, for example, fps.

  As a specific calculation method, for example, when a time stamp corresponding to real time is set in a video frame, a calculation method represented by Expression (1) is used. However, when i = 1, fps_i = 1.

  The time stamp setting format depends on the encoder used by the encoding unit 101. That is, the calculation method of the frame rate fps_i of the i-th video frame also depends on the encoder used by the encoding unit 101.

  Next, threshold determination processing by the second threshold determination unit 205 of the playback terminal 200 will be described. FIG. 3 is a flowchart showing the operation of threshold value determination processing by the second threshold value determination unit 205.

  When the reception unit 201 of the playback terminal 200 receives the i-th video frame, as described above, the arrival time of the i-th video frame is held in the reception unit 201 as the reception time t (i).

  The second threshold value determination unit 205 calculates a reception interval interval_i for the received i-th video frame (step S201). As shown in FIG. 3, the second threshold value determination unit 205 calculates the reception interval interval_i using the following equation.

  interval_i = t (i)-t (i-1) [sec] ... Equation (2)

  Next, the reception interval measurement unit 204 receives the frame rate fps_i of the i-th video frame from the reception unit 201, and calculates the arrival delay Ad_i of the i-th video frame (step S202). As shown in FIG. 3, the reception interval measurement unit 204 calculates the arrival delay Ad_i using the following equation.

  Ad_i = interval_i-(1 / fps_i) [sec] ... Formula (3)

  However, the reception interval measurement unit 204 sets Ad_i = 0 when i = 1. Further, when the calculated Ad_i is a negative value, the reception interval measurement unit 204 sets Ad_i = 0.

  The second threshold value determination unit 205 receives from the reception interval measurement unit 204 the arrival delay Ad_i of the video frame calculated in step S202. Next, the second threshold value determination unit 205 confirms the magnitude relationship between the received arrival delay Ad_i and the second threshold value p_lower_ (i-1) regarding the (i-1) th video frame (step S203).

  When p_lower_ (i-1) <= Ad_i (False in step S203), the second threshold value determination unit 205 sets the second threshold value p_lower_i related to the i-th video frame as shown in FIG. Calculate (step S204).

  p_lower_i = coef_2 * p_lower_ (i-1) + (1.0-coef_2) * Ad_i Equation (4)

  When p_lower_ (i-1)> Ad_i (True in step S203), the second threshold value determination unit 205 calculates the second threshold value p_lower_i for the i-th video frame as shown in FIG. (Step S205).

  p_lower_i = coef_1 * p_lower_ (i-1) + (1.0-coef_1) * Ad_i (5)

  Note that coef_2 in equation (4) and coef_1 in equation (5) are arbitrary parameters. Specifically, for example, coef_1 = 0.95 and coef_2 = 0.2 are set.

  When i = 1, the second threshold value determination unit 205 sets p_lower_i = Ad_i. When the calculated second threshold value p_lower_i exceeds the first threshold value p_upper given in advance, the second threshold value determination unit 205 sets p_lower_i = p_upper.

  Next, the second threshold value determination unit 205 checks whether or not the i-th video frame for which the second threshold value p_lower_i has been calculated is the n-th video frame (step S206).

  If i <n, that is, if the video frame is not the n-th video frame (True in step S206), the second threshold value determination unit 205 adds 1 to i (step S207). Next, the second threshold value determination unit 205 performs the process of step S201 again.

  When i <n is not true, that is, when the video frame is the n-th video frame (False in step S206), the second threshold value determination unit 205 ends the threshold value determination process.

  Hereinafter, the meaning of the process of step S204 and the process of step S205 in the threshold value determination process by the second threshold value determination unit 205 will be described. When the playback buffer time of the unplayed video buffered in the video frame buffering unit 203 is less than the second threshold value p_lower, the display time coefficient d_coef is increased, and the video is played back slowly. That is, the quality of experience for the user is slightly lowered.

  When the second threshold value p_lower is reduced, the range to be played back at the same speed increases, but the risk that the video data stored in the playback buffer is exhausted also increases. When the second threshold value p_lower is increased, the risk that the video data stored in the playback buffer is depleted is reduced, but the range of playback at the same speed is narrowed. In the threshold value determination process, the second threshold value determination unit 205 dynamically controls the second threshold value p_lower, which is a factor of the correspondence relationship, according to fluctuations in arrival of the video frame.

  Specifically, when the arrival delay Ad_i is smaller than the second threshold p_lower_ (i−1), that is, when the arrival of the video frame is not delayed, the risk that the video data stored in the reproduction buffer is exhausted is small. . Therefore, the second threshold value determination unit 205 widens the range of the reproduction buffer time that is reproduced at the same speed.

  In order to widen the range that is reproduced at the normal speed, the second threshold value determination unit 205 decreases the second threshold value p_lower_i. However, since the fluctuation of the arrival delay Ad_i is large, the second threshold value determination unit 205 calculates the second threshold value p_lower_i using the smoothing coefficient coef_1 close to 1.0 in Step S205 as in Expression (5). To smooth.

  Further, when the arrival delay Ad_i is equal to or greater than the second threshold value p_lower_ (i−1), that is, when the arrival of the video frame is delayed, there is a high risk that the video data stored in the reproduction buffer will be exhausted. Therefore, the second threshold value determination unit 205 narrows the range of the reproduction buffer time that is reproduced at the same speed.

  In order to narrow the range reproduced at the same speed, the second threshold value determination unit 205 increases the second threshold value p_lower_i. However, in general, when playback stops due to the depletion of video data stored in the playback buffer, a small change in playback speed has a great influence on the quality of experience for the user.

  In the above case, in order to increase the second threshold value p_lower_i so as to follow the arrival delay Ad_i, the second threshold value determining unit 205 uses the smoothing coefficient coef_2 close to 0.0 in step S204. Smoothing by calculating as shown in equation (4). The reason is to make it easier to expand the range of the playback buffer time that is played back at the same speed.

  That is, the second threshold value determination unit 205 calculates the second threshold value p_lower_i using an exponential smoothing moving average method in which the latest data is more important.

  Next, display time determination processing by the frame display time determination unit 206 of the playback terminal 200 will be described. FIG. 4 is a flowchart showing the operation of display time determination processing by the frame display time determination unit 206.

  In the display time determination process, the frame display time determination unit 206 compares the first threshold value p_upper given in advance with the reproduction buffer time p_i that the video frame buffering unit 203 has. Further, the frame display time determination unit 206 compares the second buffer p_lower_i given from the second threshold determination unit 205 with the reproduction buffer time p_i. Based on the comparison result, the frame display time determination unit 206 determines the display time coefficient d_coef.

  The frame display time determination unit 206 is given a first threshold value p_upper as an input. Also, the second threshold value p_lower_i is given from the second threshold value determining unit 205 to the frame display time determining unit 206 (step S301). In this example, the unit of the first threshold value p_upper and the unit of the second threshold value p_lower_i are seconds (sec).

  Next, the video frame buffering unit 203 calculates a playback buffer time p_i (step S302). The reciprocal of the frame rate fps_i set for the i-th video frame calculated in the process of step S103 corresponds to the display time disp_i of the i-th video frame, that is, the playback buffer time p_i.

  Therefore, if the set of video frames stored in the video frame buffering unit 203 is B, the video frame buffering unit 203 calculates the playback buffer time p_i by the following equation.

  In this example, the unit of the display time disp_i and the unit of the reproduction buffer time p_i are seconds (sec). The frame display time determination unit 206 receives the playback buffer time p_i calculated in step S302 from the video frame buffering unit 203.

  Next, the frame display time determination unit 206 confirms the magnitude relationship between the received reproduction buffer time p_i and the first threshold value p_upper given in advance (step S303).

  When p_upper <p_i (True in step S303), the frame display time determination unit 206 determines the display time coefficient d_coef for the i-th video frame according to the following equation as shown in FIG. 4 (step S307).

  d_coef = fast_coef (7)

  When p_upper> = p_i (False in Step S303), the frame display time determination unit 206 confirms the magnitude relationship between the received reproduction buffer time p_i and the second threshold p_lower_i for the i-th video frame (Step S303). S304).

  When p_lower_i> p_i (true in step S304), the frame display time determination unit 206 determines the display time coefficient d_coef for the i-th video frame according to the following equation as shown in FIG. 4 (step S306).

  d_coef = slow_coef (8)

  If p_lower_i <= p_i (False in step S304), the frame display time determination unit 206 determines the display time coefficient d_coef for the i-th video frame according to the following equation as shown in FIG. 4 (step S305).

  d_coef = 1.0 (9)

  Each time a video frame arrives, the display time coefficient d_coef is updated. Note that fast_coef and slow_coef are coefficients given in advance. Specifically, fast_coef is a coefficient larger than 0 and smaller than 1.0, and slow_coef is a coefficient larger than 1.0.

  Next, the frame display time determination unit 206 checks whether or not the i-th video frame for which the display time coefficient d_coef has been calculated is the n-th video frame (step S308).

  When i <n, that is, when the video frame is not the n-th video frame (True in Step S308), the frame display time determination unit 206 adds 1 to i (Step S309). Next, the frame display time determination unit 206 performs the process of step S301 again.

  If i <n, that is, if the video frame is the n-th video frame (False in step S308), the frame display time determination unit 206 ends the display time determination process.

  The frame display unit 207 obtains the j-th frame that is the first frame of the set B of video frames stored in the video frame buffering unit 203. Note that j is an integer equal to or less than i. Also, the frame display unit 207 acquires the display time coefficient d_coef determined from the frame display time determination unit 206.

  The frame display unit 207 displays the j-th frame for the display time (disp_j * d_coef) based on the acquired information. The unit of the display time is, for example, second (sec).

  After displaying the jth frame, if the (j + 1) th frame is stored in the video frame buffering unit 203, the frame display unit 207 performs display processing of the (j + 1) th frame .

  Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, the operation of the distribution terminal 100 and the operation of the playback terminal 200 will be described using specific numerical values.

  In the present embodiment, it is assumed that the distribution terminal 100 continues video distribution at 10 fps toward the playback terminal 200. When video distribution is performed at a frame rate of 10 fps, the frame arrival interval is 100 msec.

  The receiving unit 201 of the playback terminal 200 receives the i-th video frame (i = 1, 2,..., N), and then inputs the i-th frame to the decoding unit 202. The decoding unit 202 decodes the input i-th frame and stores the decoded frame in the video frame buffering unit 203.

  Assume that the video frame buffering unit 203 buffers three frames including the i-th frame buffered immediately before. The reproduction buffer time p_i when 3 frames are buffered is calculated as follows according to the equation (6).

p_i = Σ (1 / fps _b ) = (1 / 0.01) * 3 = 300 [msec]

  The reception interval measurement unit 204 holds a difference between the reception time of the i-th frame and the reception time of the (i−1) -th frame as a reception interval interval_i. Next, the reception interval measurement unit 204 calculates the arrival delay Ad_i of the video frame according to Equation (3).

  When the reception interval interval_i is 130 msec, the reception interval measurement unit 204 calculates the arrival delay Ad_i of the video frame according to the equation (3) as follows.

  Ad_i = interval_i-(1 / fps_i) = 130-100 = 30 [msec]

  Next, the second threshold value determination unit 205 compares the second threshold value p_lower_ (i−1) for the (i−1) -th video frame with the calculated arrival delay Ad_i of the video frame. In this example, each value is set as follows.

  p_lower_ (i-1) = 60 [msec], coef_1 = 0.95, coef_2 = 0.2

  That is, since the second threshold value p_lower_ (i−1) is larger than the arrival delay Ad_i of the video frame, the second threshold value determination unit 205 performs the second operation on the i-th video frame as follows according to the equation (5). Threshold value p_lower_i is calculated.

p_lower_i = coef_1 * p_lower_ (i-1) + (1.0-coef_1) * Ad_i
= 0.95 * 60 + 0.05 * 30 = 58.5 [msec]

  Next, the frame display time determination unit 206 compares the current playback buffer time p_i with the first threshold p_upper and the second threshold p_lower_i, respectively. Based on the comparison result, the frame display time determination unit 206 updates the display time coefficient d_coef of the jth frame buffered at the head in the video frame buffering unit 203. Note that j is an integer equal to or less than i.

  Assume that the first threshold value p_upper, which is a delay target time given as an input, is 500 msec. Since the current reproduction buffer time p_i is 300 msec and the second threshold p_lower_i is 58.5 msec, the following relational expression is established.

  p_lower_i <= p_i <= p_upper

  Therefore, the frame display time determination unit 206 updates the display time coefficient d_coef to 1.0 according to the display time determination process shown in FIG.

  Next, the frame display unit 207 extracts the j-th frame stored in the video frame buffering unit 203. Also, the frame display unit 207 acquires the display time coefficient d_coef from the frame display time determination unit 206. Based on the acquired information, the frame display unit 207 calculates the display time of the j-th frame as follows.

  disp_j * d_coef = 100 * 1.0 = 100 [msec]

  The frame display unit 207 displays the j-th frame for the calculated display time, that is, 100 msec.

  Next, it is assumed that the reception unit 201 receives the (i + 1) th video frame and the reception interval interval_ (i + 1) is 220 msec. The reception interval interval_ (i + 1) being 220 msec means that 220 msec has elapsed since the i-th frame was received until the (i + 1) -th frame was received.

  Similarly to the above example, the receiving unit 201 inputs the (i + 1) -th frame to the decoding unit 202. The decoding unit 202 decodes the input (i + 1) th frame, and stores the decoded frame in the video frame buffering unit 203.

  The display time coefficient d_coef included in the frame display time determination unit 206 remains 1.0. Therefore, after displaying the jth frame, the frame display unit 207 extracts the (j + 1) th frame from the video frame buffering unit 203 and displays the (j + 1) th frame for 100 msec.

  Similarly, after displaying the (j + 1) -th frame, the frame display unit 207 extracts the (j + 2) -th frame from the video frame buffering unit 203, and the (j + 2) -th frame for 100 msec. Is displayed. That is, when the (i + 1) th frame is received, the (j + 2) th frame is being displayed. Specifically, the (i + 1) th frame is received when 20 msec has elapsed since the display of the (j + 2) th frame was started.

  At the stage when 20 msec has elapsed since the display of the (j + 2) th frame was started, the first three buffered frames have already been displayed. Therefore, one frame including the (i + 1) th frame buffered immediately before is buffered in the video frame buffering unit 203. That is, the reproduction buffer time p_ (i + 1) is calculated as follows according to the equation (6).

p_ (i + 1) = Σ (1 / fps _b ) = (1 / 0.01) * 1 = 100 [msec]

  When the reception interval interval_ (i + 1) is 220 msec, the reception interval measurement unit 204 calculates the arrival delay Ad_ (i + 1) of the video frame according to the equation (3) as follows.

  Ad_ (i + 1) = interval_ (i + 1)-(1 / fps_ (i + 1)) = 220-100 = 120 [msec]

  Next, the second threshold value determination unit 205 compares the second threshold value p_lower_i related to the i-th video frame with the calculated arrival delay Ad_ (i + 1) of the video frame. Since p_lower_i is determined to be 58.5 msec in the above calculation, the second threshold value p_lower_i is equal to or less than the arrival delay Ad_ (i + 1). Therefore, the second threshold value determination unit 205 calculates the second threshold value p_lower_ (i + 1) for the (i + 1) -th video frame as follows according to the equation (4).

p_lower_ (i + 1) = coef_2 * p_lower_i + (1.0-coef_2) * Ad_ (i + 1)
= 0.2 * 58.5 + 0.8 * 120 = 107.7 [msec]

  Next, the frame display time determination unit 206 updates the display time coefficient d_coef of the (j + 3) th frame buffered at the head in the video frame buffering unit 203. Since the current playback buffer time p_ (i + 1) is 100 msec, the first threshold p_upper is 500 msec, and the second threshold p_lower_ (i + 1) is 107.7 msec, the following relational expression is established.

  p_ (i + 1) <p_lower_ (i + 1)

  Therefore, the frame display time determination unit 206 updates the display time coefficient d_coef to slow_coef according to the display time determination process shown in FIG. When fast_coef = 0.8 and slow_coef = 1.2 are set, d_coef is updated to 1.2.

  After the display of the currently displayed (j + 2) th frame is completed, the frame display unit 207 extracts the (j + 3) th frame stored in the video frame buffering unit 203. Also, the frame display unit 207 acquires the display time coefficient d_coef from the frame display time determination unit 206. Based on the acquired information, the frame display unit 207 calculates the display time of the (j + 3) th frame as follows.

  disp_ (j + 3) * d_coef = 100 * 1.2 = 120 [msec]

  The frame display unit 207 displays the (j + 3) -th frame for the calculated display time, that is, 120 msec.

  As described above, the second threshold value determination unit 205 dynamically changes the threshold value at which the frame display time fluctuates according to the video frame reception interval. By dynamically changing the threshold value, it is possible to prevent the playback of the video from being stopped while extending the range of the playback buffer time during which the video is played at the same speed.

  FIG. 5 is an explanatory diagram showing the correspondence between the reception interval of video frames and the second threshold when the video playback system 10 of this embodiment is used. FIG. 5A is an explanatory diagram showing the arrival delay Ad with respect to the number of arrival frames. FIG. 5B is an explanatory diagram showing a second threshold value Plower with respect to the number of arrival frames.

  The unit of arrival delay Ad shown in FIG. 5A and the unit of second threshold value Plower shown in FIG. 5B are msec. As shown in FIG. 5, the second threshold value determination unit 205 changes the second threshold value in accordance with the fluctuation of the arrival delay when the arrival delay, that is, the reception interval of the video frame fluctuates.

[Description of effects]
A playback terminal according to the present embodiment and the video playback control system of the present example that solves the above problems, a receiving unit that receives encoded video data, a decoding unit that decodes encoded video data, and a decoding And a buffering unit for buffering the video frames.

  The playback terminal also includes a reception interval measurement unit that measures the arrival interval of the video frames received by the reception unit, and a second threshold value based on the first threshold value and the reception interval variation information measured by the reception interval measurement unit. And a frame display time determination unit that determines the display time.

  The playback terminal also includes a frame display unit that displays buffered video frames for the display time notified from the frame display time determination unit. That is, since the playback stop is prevented by adjusting the playback speed, when a playback terminal according to the video playback control system of the present embodiment is used, the quality of experience for the user is improved.

  The video playback system of the present embodiment can prevent video playback from being stopped due to the depletion of video data stored in the playback buffer without deteriorating the quality of experience for the user. The reason is that the second threshold value determination unit 205 of the playback terminal 200 dynamically changes the threshold value for adjusting the playback speed in accordance with the arrival interval of video frames.

  Specifically, when the arrival of the video frame is delayed, the second threshold value determination unit 205 expands the range of the playback buffer time that is played back at a low speed by increasing the second threshold value, and stores the playback buffer in the playback buffer. Prevents playback stoppage due to depletion of stored video data. In addition, when the arrival interval of the video frames is approximately a predetermined interval, the second threshold value determination unit 205 expands the range of the reproduction buffer time that is reproduced at the same speed by reducing the second threshold value, and is useful for the user. Maintain the quality of experience.

  Therefore, the video playback system of the present embodiment prevents the video data stored in the playback buffer from being depleted when a video distributed under an environment where the bandwidth fluctuation is severe such as a mobile network is viewed. The reproduction control is performed to extend the range of the reproduction buffer time reproduced at the same speed. The video playback system can also maximize the range of playback buffer time that is played back at the same speed. By performing the reproduction control, it is possible to prevent the quality of experience for the user from being deteriorated.

  Note that the playback terminal 200 of the present embodiment is realized by, for example, a CPU (Central Processing Unit) that executes processing according to a program stored in a storage medium. That is, the reception unit 201, the decoding unit 202, the video frame buffering unit 203, the reception interval measurement unit 204, the second threshold value determination unit 205, the frame display time determination unit 206, and the frame display unit 207 are processed according to program control, for example. Realized by a CPU that executes

  In addition, each unit in the playback terminal 200 of the present embodiment may be realized by a hardware circuit. As an example, a receiving unit 201, a decoding unit 202, a video frame buffering unit 203, a reception interval measuring unit 204, a second threshold value determining unit 205, a frame display time determining unit 206, and a frame display unit 207 are respectively provided with LSI (Large Realized by Scale Integration). Further, they may be realized by one LSI.

  Next, the outline of the present invention will be described. FIG. 6 is a block diagram showing an outline of a video reproduction apparatus according to the present invention. The video playback device 20 according to the present invention includes a measurement unit 21 (for example, a reception interval measurement unit 204) that measures a reception interval that is an interval of time at which a video frame is received, and a time at which an unreproduced video is captured. A determination unit 22 (for example, a second threshold value determination unit 205) that determines a lower limit value of the imaging time to be reproduced at the same time as the imaging time (for example, the reproduction buffer time) based on the measured reception interval;

  With such a configuration, the video playback device can extend the range of the playback buffer time that is played back at the same speed while preventing depletion of unplayed video.

  The video playback device 20 also includes an adjustment unit (for example, a frame display) that adjusts the playback time of the unplayed video based on the upper limit value and the lower limit value of the shooting time during which the unplayed video is played back at the same time as the shooting time. A time determination unit 206) may be provided.

  With such a configuration, the video playback device can adjust the playback time of an unplayed video according to the fluctuation of the bandwidth of the communication network.

  In addition, the video playback device 20 may include a playback unit (for example, a frame display unit 207) that plays back an unplayed video with the playback time adjusted by the adjustment unit.

  With such a configuration, the video reproduction apparatus can reproduce an unreproduced video with a reproduction time adjusted according to a change in the bandwidth of the communication network.

  Further, the determination unit 22 may determine the lower limit value by an exponential smoothing moving average method.

  With such a configuration, the video playback apparatus can dynamically control the lower limit value according to fluctuations in arrival of video frames.

  The determination unit 22 may determine the lower limit value by an exponential smoothing moving average method using a first smoothing coefficient (for example, coef_1) and a second smoothing coefficient (for example, coef_2).

  With such a configuration, the video reproduction apparatus can control the lower limit value according to the relationship between the previously controlled lower limit value and arrival delay.

  In addition, the video playback device 20 includes a storage unit (for example, a video frame buffering unit 203) that stores received video frames, and unplayed video is composed of video frames stored in the storage unit. May be.

  With such a configuration, the video playback apparatus can adjust the playback speed of the buffered video frames according to the current state of the communication network.

10 Video playback system
20 Video playback device
21 Measuring unit
22 Decision part
100 distribution terminals
101 Encoder
102 Transmitter
103 Throughput measurement unit
104 Parameter determination part
200 playback terminals
201 Receiver
202 Decryption unit
203 Video frame buffering section
204 Reception interval measurement unit
205 Second threshold value determination unit
206 Frame display time determination section
207 Frame display
300 communication network

Claims (10)

A measurement unit that measures a reception interval, which is an interval of time when a video frame is received
And a determination unit that determines a lower limit value of the shooting time that is played back at the same time as a shooting time that is a time when an unplayed video is shot, based on the measured reception interval. apparatus. The video playback apparatus according to claim 1, further comprising an adjusting unit that adjusts a playback time of the unplayed video based on an upper limit value and a lower limit value of the shooting time in which the unplayed video is played back at the same time as the shooting time. The video playback apparatus according to claim 2, further comprising: a playback unit that plays back an unplayed video with a playback time adjusted by the adjustment unit. The video reproduction device according to any one of claims 1 to 3, wherein the determination unit determines the lower limit value by an exponential smoothing moving average method. The video reproduction device according to claim 4, wherein the determination unit determines the lower limit value by an exponential smoothing moving average method in which the first smoothing coefficient and the second smoothing coefficient are used. A storage unit for storing received video frames;
The video reproduction device according to any one of claims 1 to 5, wherein an unreproduced video is composed of video frames stored in the storage unit. Measure the reception interval, which is the time interval when the video frame was received,
A video playback method, wherein a lower limit value of the shooting time that is played back at the same time as a shooting time that is a time when an unplayed video is shot is determined based on the measured reception interval. The video playback method according to claim 7, wherein the playback time of the unplayed video is adjusted based on an upper limit value and a lower limit value of the shooting time in which the unplayed video is played back at the same time as the shooting time. On the computer,
The measurement processing for measuring the reception interval, which is the time interval when the video frame was received, and the lower limit of the shooting time that was played back at the same time as the shooting time when the unplayed video was shot were measured A video playback program for executing a determination process that is determined based on the reception interval. On the computer,
The video playback program according to claim 9, wherein an adjustment process for adjusting a playback time of an unplayed video is executed based on an upper limit value and a lower limit value of the shooting time during which an unplayed video is played back at the same time as the shooting time.