JP5973616B1 - Receiving terminal and video acquisition method thereof - Google Patents

Abstract

A receiving terminal capable of improving video quality for viewing in a situation where a communication environment changes, and a video acquisition method therefor are provided. A receiving terminal is a terminal that supports adaptive streaming, and holds an acquisition control unit that changes a bit rate of a segment to be acquired according to a communication environment, and a segment acquired by the acquisition control unit. And a content playback unit 14 that plays back the segment held in the buffer holding unit 13, and the acquisition control unit 11 acquires the first bit rate (for example, low bit rate) segment. If there is a grace period before the segment is reproduced, the segment is reacquired at a second bit rate (for example, a high bit rate) higher than the first bit rate, and the replacement is performed. [Selection] Figure 1

Description

  The present invention relates to a receiving terminal and a video acquisition method thereof, and more particularly, to a high-quality video acquisition method in adaptive streaming.

  Conventionally, a method of changing a file to be acquired according to throughput (hereinafter referred to as adaptive streaming) is known (see Patent Documents 1 to 6). When the server prepares a plurality of bit rate files for one content, the bit rate of the content to be acquired can be selected according to the throughput as long as the receiving terminal supports adaptive streaming. That is, a high bit rate is selected when the throughput is high, and a low bit rate is selected when the throughput is low.

  Content playback can be performed without interruption by dynamically changing the bit rate of the segment to be acquired according to the communication environment. In an unstable environment where the communication environment fluctuates, the content is reproduced with the high bit rate segment and the low bit rate segment intermingled.

JP 2013-135471 A JP2013-089977A JP 2012-235463 A JP-T-2015-501568 Special table 2015-501090 gazette Special table 2013-534684 gazette

  When the communication environment is temporarily deteriorated in a state where the communication environment is unstable, an attempt is made to acquire a segment having a size that can be acquired in the communication state. Therefore, when the throughput decreases, the segment acquired at that time has a low bit rate (low quality). Even if the communication environment is improved after the acquisition of such a low quality segment and before the reproduction, there is a problem that the segment once obtained is reproduced with the low quality.

  An object of the present invention is to provide a receiving terminal capable of improving video quality for viewing in a situation where a communication environment changes and a video acquisition method thereof in view of the above-described conventional technology.

In order to achieve the above object, the invention according to the first aspect is a receiving terminal that supports adaptive streaming, wherein an acquisition control unit that changes a bit rate of a segment to be acquired according to a communication environment, and the acquisition control unit And a playback unit that plays back the segment held in the buffer holding unit, and the acquisition control unit acquires the first bit rate segment, If there is a grace period before playback, the segment is reacquired at a second bit rate that is higher than the first bit rate, replaced, and the acquisition control unit further determines the segment playback time. t, s is the time taken to download the next segment, u is the time taken to download the replacement segment, When the rearrangement process time is p, when a u <t-s-p is summarized in that to determine that a grace until reproduction of the segment.

In order to achieve the above object, the invention according to the second aspect is a video acquisition method in which a receiving terminal compatible with adaptive streaming acquires video, and acquires a first bit rate segment according to a communication environment. A buffer holding step for holding the segment acquired in the acquisition step, and a second bit having a bit rate higher than the first bit rate if there is a grace period before the segment acquired in the acquisition step is reproduced. and re-acquisition of a segment at a rate, comprising: a re-acquisition step of performing the replacement, and a reproduction step of reproducing the segments re-captured by the capture segments or the re-acquisition step by the obtaining step, further, the re In the acquisition step, the segment playback time is t, and the next segment is downloaded. S the time, the time required to download the segments replacing u, the replacement processing time when the p, summarized in that to determine that when a u <t-s-p, there is a grace before reproduction of said segments And

  ADVANTAGE OF THE INVENTION According to this invention, it is possible to provide the receiving terminal which can improve the video quality to view in the condition where a communication environment changes, and its video acquisition method.

It is a functional block diagram of the receiving terminal in the embodiment of the present invention. It is a flowchart which shows operation | movement of the receiving terminal in embodiment of this invention. It is a flowchart which shows operation | movement of the receiving terminal in embodiment of this invention. It is a figure which shows the file reproduction | regeneration and acquisition flow image when not acquiring again. It is a figure which shows a reacquisition flow image. It is a figure for demonstrating the bit-rate calculation system of the next segment at the time of buffer reduction. It is a figure which shows the adaptive streaming in embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following embodiments exemplify a receiving terminal for embodying the technical idea of the present invention, and the configuration of the apparatus and the configuration of data are not limited to the following embodiments. Absent.

(Receiving terminal)
FIG. 1 is a functional block diagram of a receiving terminal 10 according to the embodiment of the present invention.

  The content server 20 is a device that distributes video content and the like through the Internet. In the following description, video content may be simply referred to as “video” or “content”.

  The receiving terminal 10 is a terminal such as an STB or a mobile phone that supports adaptive streaming. Once a low bit rate segment has been acquired, but there is a delay before playback of that segment and there is sufficient throughput, the low bit rate segment is reacquired at a high bit rate and replaced. . Since the segment to be replaced is reacquired, the number of buffers to be held can be changed by setting. When the content server 20 prepares only one bit rate file for one content, the prepared bit rate content is acquired regardless of the throughput.

  Functionally, the receiving terminal 10 includes an acquisition control unit 11, a reacquisition control unit 12, a buffer holding unit 13, and a content reproduction unit 14. Hereinafter, each functional unit will be described.

  The acquisition control unit 11 is a functional unit that controls content acquisition. Specifically, when an acquisition instruction is received from the buffer holding unit 13, an acquisition request is issued to the content server 20, and when the content is acquired from the content server 20, the acquired content is transferred to the buffer holding unit 13. Further, when a reacquisition instruction is received from the reacquisition control unit 12, a reacquisition request is issued to the content server 20, and when content is reacquired from the content server 20, a content replacement instruction is issued to the buffer holding unit 13.

  The reacquisition control unit 12 is a functional unit that controls reacquisition of content. Specifically, after a low bit rate segment is acquired, if there is a grace period before the segment is played back, the acquisition control unit 11 is instructed to reacquire the segment at a high bit rate.

  The buffer holding unit 13 is a functional unit that holds a buffer (unreproduced content) in a memory. Specifically, the content acquired or reacquired by the acquisition control unit 11 is held. The number of buffers held by the buffer holding unit 13 can be changed by setting.

  The content reproduction unit 14 is a functional unit that reproduces content. Specifically, the content held in the buffer holding unit 13 is reproduced and output to an external device such as a display (not shown).

(Operation)
FIG. 2 is a flowchart showing the operation of the receiving terminal 10 in the embodiment of the present invention.

  FIG. 2A shows a video playback flow of adaptive streaming. First, the reproduction buffer is accumulated and reproduction is started (S1 → S2). Further, when the acquisition of the next segment is started and the acquisition of the next segment is completed, the buffer free space is confirmed (S3 → S4 → S5). At this time, if there is no buffer free space, the process waits until the buffer decreases (S5 → S6). If the buffer decreases, the next segment acquisition bit rate is calculated (S8), and the process returns to step S3. On the other hand, if there is free buffer capacity and the file is not finished, the next segment acquisition bit rate is calculated (S5 → S7 → S8), and the process returns to step S3.

  In the present embodiment, characteristic processing (S11 to S15) is executed while waiting for buffer reduction (S6). That is, as shown in FIG. 2B, it is determined whether there is a segment to be replaced in the buffer (S11). If there is a segment to be replaced in step S11, a bit rate at which the segment can be acquired is calculated, and reacquisition is started (S12 → S13). If acquisition is completed before the buffer is decreased, the process returns to step S11. If acquisition is not completed before the buffer is decreased, the next segment acquisition bit rate is calculated (S8), and the process returns to step S3. On the other hand, if there is no segment to be replaced in step S11, the process waits until the buffer decreases (S15). If the buffer decreases, the next segment acquisition bit rate is calculated (S8), and the process returns to step S3.

  FIG. 3 is a flowchart showing the operation of the receiving terminal 10 in the embodiment of the present invention. Hereinafter, the adaptive streaming reacquisition determination operation will be described in detail with reference to FIG.

  The preconditions are as follows. That is, the time required for DL (download) of one segment is basically within t seconds, the maximum buffer length is T seconds, the maximum number of retained segments is n, the playback time per segment is t seconds, and the content type (rank) is H, M1,..., Mx, L, the time required for segment replacement is p seconds, and the time required for DL of the latest segment is s seconds.

  First, it is determined whether there is room to re-DL the low bit rate segment (S21). Specifically, it is determined whether the total buffer is equal to or greater than the maximum buffer length T.

  Next, when the setting for performing reacquisition is on, it is determined whether there is a segment other than the high bit rate in the buffer (S22 → S23). At this time, if there is a segment other than the high bit rate, it is determined whether or not the high bit rate segment can be reacquired until n + 1 segment is acquired (S24).

  That is, if the time taken for DL of the latest segment is s, there is a remaining time of ts. The time u required to acquire a high segment to be replaced with the throughput calculated from the DL of the latest segment is calculated. Therefore, it is determined that reacquisition is possible when u <t−s−p, and it is determined that reacquisition is not possible when u> t−s−p (S24).

  If it is determined in step S24 that reacquisition is possible, a high bit rate segment is reacquired (S25), and it is determined whether reacquisition is completed (S26). The determination in step S26 is performed when the acquisition time for the n + 1 segment is reached. If it is determined that the reacquisition is not complete, the data being reacquired is discarded (S27). If it is determined that the reacquisition is complete, the process returns to step S21.

  If it is determined in step S24 that reacquisition is impossible, it is determined whether the segment to be reproduced next is a segment other than the high bit rate (S28). If it is determined that the segment is not a segment other than the high bit rate, the buffer change (T ± t Second) is determined (S29).

  If the buffer change is not permitted in step S29, it is determined whether reacquisition of segments other than the high bit rate is permitted (S31). Specifically, when DL is performed at the medium bit rates M1 to Mx and replacement is performed with segments other than the high bit rate, the determination in step S31 is YES. On the other hand, when DL is performed at the medium bit rates M1 to Mx and no replacement is performed with segments other than the high bit rate, the determination in step S31 is NO.

  If it is allowed in step S29 that the buffer is reduced, it is determined whether a high bit rate segment can be reacquired by xt-s-p (S30). x is an allowable upper limit number of buffer changes. If it is determined that reacquisition is possible, the process proceeds to step S25. If it is determined that reacquisition is impossible, the process proceeds to step S31.

  If it is determined in step S31 that reacquisition of segments other than the high bit rate is permitted, it is determined whether the reacquisable bit rate is higher than the acquired bit rate (S32). At this time, if it is determined that the bit rate is higher than the acquired bit rate, the segment is acquired again at a bit rate other than the high bit rate (S33), and the process proceeds to step S26.

  If NO is determined in steps S22, S23, S31, and S32, a new segment is acquired as soon as the segment being played is finished (S34), and the segment acquired at n + 1 is a low bit rate or other than a low bit rate. (S35). In the determination of step S35, it is desirable to consider the improvement limit.

  If the segment acquired at n + 1 has a low bit rate, a segment with a low bit rate is acquired. On the other hand, if the segment acquired at n + 1 is other than the low bit rate, it is determined whether the acquisition of segments other than the low bit rate has been completed (S36). The determination in step S36 is made when the n + 2 segment acquisition time is reached.

  If it is determined in step S36 that the acquisition has not been completed, it is determined whether or not to continue the acquisition (S37). Specifically, when the remaining data size of the high bit rate> the data size of the low bit rate, it is determined that the acquisition is not continued, and the acquisition is continued when the remaining data size of the high bit rate <the data size of the low bit rate. Then, it is determined (S37). If it is determined in step S37 that the acquisition is not continued, the data being acquired is discarded and a low bit rate segment is acquired (S38 → S39). On the other hand, if it is determined in step S36 that the acquisition has been completed, the same processing is performed for the next segment Seg_n + 1 (S40 → S21 →...).

(File playback and acquisition flow image when not acquiring again)
FIG. 4 is a diagram showing a file playback and acquisition flow image when re-acquisition is not performed. The downward arrows in the figure indicate the passage of time. A right-pointing arrow indicates that the segment is being played back or being acquired, or is waiting. In the figure, “Seg” means the segment, and the number after “Seg” means the segment playback order.

  The receiving terminal 10 acquires the video from the content server 20, performs playback sequentially while having a buffer, and acquires the next buffer when playing back one segment. Specifically, the reproduction is started with the reproduction initial buffer (five in the example) accumulated. Since the segment Seg_1 is reproduced, the unreproduced buffer is decreased, and acquisition of the next segment Seg_6 is started.

  Note that when the acquisition of the segment Seg_6 is completed, the playback of the segment Seg_1 may not be finished. In this case, the next segment Seg_7 is not acquired until the segment Seg_2 is played back, and the standby state is set.

(Re-acquisition flow image)
FIG. 5 is a diagram showing a re-acquisition flow image. Of the initial playback buffers (five in the example), the third segment has a low bit rate and the remaining segments have a high bit rate as viewed from the first segment being played. Here, the low bit rate segment is represented by [L], and the high bit rate segment is represented by [H].

  First, it is assumed that the next segment [H] is acquired during playback of the first segment, and the time required for the acquisition is s. At this time, it is determined whether the acquired segment [L] can be reacquired by [H] by ts-p. If it is determined that re-acquisition is possible, the re-acquisition of [H] is completed with u, and the replacement with [H] re-acquired from [L] once acquired with p is completed.

(The bit rate calculation method for the next segment when the buffer is reduced)
FIG. 6 is a diagram for explaining a bit rate calculation method for the next segment when the buffer is decreased. FIG. 6A shows a situation in which H quality cannot be acquired in time. It is assumed that the H quality data size is xMbyte and the L quality data size is yMbyte. FIG. 6B is a table showing the lower limit of throughput. The quality rank, the throughput lower limit at the time of acquisition (normal time), and the throughput lower limit at the time of buffer decrease are associated and managed. The table shown in FIG. 6B is set in units of files and is variable.

  At present, the bit rate of the next segment is determined based on the throughput that the H quality can be acquired in t seconds. However, in the following cases (1) and (2), since the time taken to acquire the next segment is tv seconds, the H quality cannot be acquired within the time. In addition, a similar event occurs in acquisition of the next segment, and a situation in which cancellation is likely to occur occurs.

(1) Acquired in H quality, acquired in H quality over v seconds after t seconds (2) Acquired in H quality, discarded in H quality after t seconds, and acquired in L quality over v seconds

  Therefore, in the present embodiment, the following solution is adopted. In other words, if the segment acquisition is not completed within t seconds (when the buffer is reduced), compare the bit rate of the next segment to be acquired with the "throughput at the time of the most recent segment acquisition" and the "lower throughput limit at the time of buffer decrease" And decide. The throughput at the time of the latest segment acquisition is as follows.

In the case of (1): x / (t + v) (Mbps)
In the case of (2): (x ′ + y) / (t + v) (Mbps)

(Concrete example)
FIG. 7 is a diagram showing adaptive streaming in the embodiment of the present invention. The relationship between the local buffer on the receiving terminal 10 and the segment file on the content server 20 is shown. In the figure, t is the segment playback time (video time), s is the time required for the DL of the next segment, u is the time required for the DL of the replacement segment, and p is the replacement processing time (low bit rate → high Bit rate processing), q is the time from the start to the completion of replacement. p may be a fixed value that is determined, and is unnecessary in the first place if it is at a μs level such as 10 ms.

  As shown in FIG. 7, the DL of the segment Seg_n + 1 is started when the reproduction of the segment Seg_1 is started. The segment Seg_1 is discarded from the buffer after t seconds (after the end of reproduction), and the DL of the segment Seg_n + 2 is started together with the start of the reproduction of the segment Seg_2.

  When the replacement from the low bit rate to the high bit rate is attempted for the segment Seg_3 at t> s, if the time q required for the replacement becomes larger than ts, the buffer becomes shorter than the maximum buffer length T. .

  Based on the above conditions, a case where there is no buffer change, a case where there is a buffer change (small), and a case where there is a buffer change (many) will be described. The case of buffer change (small) means that the number of buffers held by the buffer holding unit 13 is allowed to decrease. The case of buffer change (many) means that the number of buffers held by the buffer holding unit 13 is allowed to increase. The allowable upper limit number x of buffer changes is “2”.

(A) When there is no buffer change When reacquisition at a bit rate other than a high bit rate is allowed, a reacquisable bit rate is calculated, and DL is performed at the reacquisable bit rate. On the other hand, if reacquisition at a bit rate other than the high bit rate is not permitted, the next segment is DL without waiting for re-acquisition and waiting for ts seconds.

(B) In case of buffer change (small) DL if high bit rate can be acquired at 2t-s-p, and re-acquisable bit if high bit rate cannot be acquired at 2t-s-p The rate is calculated and DL is performed at the reacquisable bit rate. In this case, u is given a margin by not acquiring the next segment until the two-segment reproduction is completed.

(C) When there is a buffer change (many) After DL segment Seg_n + 1, immediately DL the next segment Seg_n + 2 (the time taken for this is s ′), and 2t−s−s ′> q If there is, a high bit rate reacquisition determination is performed in the next loop. In this case, q is given a margin by acquiring segment n + 2 before starting the next segment reproduction.

  As described above, the receiving terminal 10 according to the embodiment of the present invention is a terminal that supports adaptive streaming, and the acquisition control unit 11 that changes the bit rate of the segment to be acquired according to the communication environment, and the acquisition control unit 11, and a content playback unit 14 that plays back the segment held in the buffer holding unit 13. The acquisition control unit 11 has a first bit rate (for example, a low bit rate). After acquiring a (rate) segment, if there is a grace period before playback of the segment, re-acquire the segment at a second bit rate (for example, a high bit rate) higher than the first bit rate and replace it. To implement. As a result, the acquired low bit rate segment can be replaced with the high bit rate segment, so that it is possible to improve video quality for viewing in a situation where the communication environment changes. Also, even if a low bit rate segment is acquired, if there is no margin in the communication band, buffer accumulation is given priority without performing re-acquisition, so that continuous video reproduction can be given the highest priority. In particular, since the mobile phone is in an unstable radio wave state, the embodiment of the present invention is very effective when applied to the mobile phone.

  Further, the acquisition control unit 11 sets u <ts−s−, where t is the playback time of the segment, s is the time required to download the next segment, u is the time required to download the replacement segment, and p is the replacement processing time. When p, it may be determined that there is a grace period before the segment is played back. This makes it possible to accurately determine whether or not there is a grace period before the segment is played back.

  Further, the number of buffers held by the buffer holding unit 13 may be changed depending on the setting. This makes it possible to perform reacquisition while increasing or decreasing the number of buffers or dynamically changing the number of buffers.

  Further, when it is allowed to reduce the number of buffers held by the buffer holding unit 13, assuming that the allowable upper limit number of buffer changes is x, if a high bit rate segment can be acquired with xt−s−p, If a bit rate segment is downloaded and if a high bit rate cannot be obtained by xt-sp, the reacquisable bit rate may be calculated and the reacquisable bit rate segment may be downloaded. Thereby, for example, it is possible to give u a margin by not acquiring the next segment until the two-segment reproduction is completed.

  Also, when it is allowed to increase the number of buffers held by the buffer holding unit 13, after the segment Seg_n + 1 is downloaded, the next segment Seg_n + 2 is immediately downloaded, the allowable upper limit number of buffer changes is x, and the segment Seg_n + 2 If xt−s−s′−... −s′x> q, assuming that the time taken for downloading is s ′ and the time taken for the segment Seg_n + x is s′x, a high bit rate is reacquired in the next loop. A determination may be made. Thereby, for example, by acquiring segment n + 2 before the start of the next segment reproduction, it is possible to give q a margin.

  The video acquisition method according to the embodiment of the present invention is a video acquisition method in which the receiving terminal 10 compatible with adaptive streaming acquires video, and has a first bit rate (for example, a low bit rate) according to the communication environment. If there is a grace period between the acquisition step of acquiring the segment, the buffer holding step of holding the segment acquired in the acquisition step, and the playback of the segment acquired in the acquisition step, the bit rate higher than the first bit rate Re-acquisition step of performing re-acquisition of a segment at a second bit rate (for example, high bit rate) and performing replacement, and playback step of reproducing the segment acquired in the acquisition step or the segment re-acquired in the re-acquisition step With. As a result, the acquired low bit rate segment can be replaced with the high bit rate segment, so that it is possible to improve video quality for viewing in a situation where the communication environment changes.

  Note that the present invention can be realized not only as the receiving terminal 10 and its video acquisition method, but also as a video acquisition program for causing a computer to execute each step included in this video acquisition method. It goes without saying that such a program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as the Internet.

DESCRIPTION OF SYMBOLS 10 ... Receiving terminal 20 ... Content server 11 ... Acquisition control part 12 ... Re-acquisition control part 13 ... Buffer holding part 14 ... Content reproduction part

Claims (2)

A receiving terminal that supports adaptive streaming,
An acquisition control unit that changes the bit rate of the segment to be acquired according to the communication environment;
A buffer holding unit for holding a segment acquired by the acquisition control unit;
A playback unit for playing back the segment held in the buffer holding unit,
The acquisition control unit re-acquires a segment at a second bit rate that is higher than the first bit rate when there is a grace period before the segment is played after acquiring the segment at the first bit rate. , Replace ,
Further, the acquisition control unit assumes that the playback time of the segment is t, the time required to download the next segment is s, the time required to download the replacement segment is u, and the replacement processing time is p, u <ts−s−. When it is p, it is determined that there is a grace period before the segment is reproduced . A video acquisition method in which a receiving terminal compatible with adaptive streaming acquires video,
An acquisition step of acquiring a segment of the first bit rate according to a communication environment;
A buffer holding step for holding the segment acquired in the acquiring step;
A reacquisition step of performing re-acquisition by performing re-acquisition of a segment at a second bit rate higher than the first bit rate and performing replacement when there is a grace period before playback of the segment acquired in the acquisition step;
Playing back the segment acquired in the acquisition step or the segment re-acquired in the re-acquisition step, and
Further, in the re-acquisition step, if the segment playback time is t, the time to download the next segment is s, the time to download the replacement segment is u, and the replacement processing time is p, u <ts−s−. If p, determine that there is a grace period before the segment is played
A video acquisition method characterized by the above.