JP6397341B2 - Reception device, buffer management method, and program - Google Patents

Description

  The present invention relates to a receiving apparatus that receives content composed of a plurality of segments for each segment and performs streaming reproduction, a buffer management method for the receiving apparatus, and a program for causing the receiving apparatus to function.

  In the streaming video distribution on the Internet nowadays, a transition from a streaming distribution method using a dedicated server and a dedicated protocol to a streaming distribution method using an HTTP protocol by a general-purpose Web server is progressing. It has become mainstream in distribution. As such a streaming distribution method (adaptive streaming) based on the HTTP protocol, an original technology by an IT vendor is prevalent, and MPEG-DASH (ISO / IEC 2309-1) which is an international standard intended to unify these streaming methods. ) Was formulated.

  The basic concept is the same for both technologies. The web server divides a stream in which video content is encoded with one or more qualities (screen size and bit rate) into files of several seconds to tens of seconds each. Manifest files that describe things (segments) and attributes and URLs of the moving image contents are prepared. The receiving device receives the segments one after another by selecting the timely quality in consideration of the screen size of the receiving device and the state of the network bandwidth of the transmission path from the manifest file, and joins and plays back the video content. (For example, refer nonpatent literature 1).

  FIG. 6 shows an example of adaptive streaming. In FIG. 6, a video is encoded with a plurality of qualities, and segments divided into units of several seconds are generated for each quality. In this example, the encoder splits the video into low quality segments 1-4, medium quality segments 1'-4 ', and high quality segments 1 "-4". In the manifest file, the URL of the segment and attribute information such as the bit rate and image size for each quality are registered. The segment and manifest file are supplied to the web server.

  The receiving device receives the manifest file from the Web server, grasps the bit rate and segment configuration from the manifest file (step A), and determines the quality in units of segments based on the screen size of the receiving device and the status of the network receiving bandwidth. Is received by the HTTP protocol (step B). Subsequently, the received segments are connected in order according to the manifest file and reproduced (step C). Thus, by repeating the processes of steps B and C, a series of contents can be received and played back.

  In the receiving apparatus, a viewing environment compatible with multi-devices (PCs, television receivers, tablet devices, etc.) can be constructed by using an HTML5 browser that is widely used in many devices.

  However, since a video element (video tag) that is a function for playing back a moving image in an HTML5 browser can only refer to a media file as a source, in order to realize such adaptive streaming viewing, It was necessary to install a browser plug-in separately. For this reason, it was difficult to build a viewing environment that can be used by all browsers. Under such circumstances, an international non-profit organization that promotes standardization of technologies used on the Web as a function for realizing adaptive streaming viewing using video elements (video tags) of HTML5 browser and Javascript (registered trademark). In W3C, standardization of MSE (Media Source Extensions) is progressing (see Non-Patent Document 2, for example).

  By using MSE, video elements can refer to a buffer as a source. Since the segment received by Javascript can be inserted into the buffer, various adaptive streaming viewing can be realized by the video element of HTML5 and Javascript.

  Here, an MSE buffer model will be described with reference to FIG. FIG. 7 is a diagram illustrating an example of an MSE buffer model. The buffer (SourceBuffer) manages segments in association with information on the playback start time (playback start position) and playback duration (scale). In FIG. 7, the hatched blocks indicate segments, the playback start time is 0, the playback duration is 5 seconds, the playback start time is 5 and the playback duration is 5 seconds, the playback start time is 15 and playback continues. A 5 second segment is inserted in the buffer. At this time, the buffering range (buffered attribute) of the buffer is 0 to 10 seconds and 15 to 20 seconds.

  In addition, as shown in FIG. 8, it is possible to insert another segment whose playback time overlaps the range where the segment has already been inserted and overwrite it. FIG. 8 shows an example in which a segment having a playback start time of 12 and a playback duration of 10 seconds is inserted as a new segment in the example shown in FIG. As a result, the range of 12 to 22 seconds is replaced with a new segment, and the buffering range is also updated to 0 to 10 seconds and 12 to 22 seconds.

  In general, when viewing streaming, the reception bandwidth always fluctuates depending on the network congestion, so data that is several seconds to several tens of seconds ahead of the current playback time is received and buffered as far as the buffer capacity allows. By inserting it, the viewing stability can be improved.

  In viewing the content, the content may be viewed from the beginning to the end, but seek playback may be performed in which playback is performed by designating an arbitrary position of the content during viewing. However, when seeking playback, if unreplayed segments remain close to the maximum buffer capacity, even if a segment at the specified position (time) is received, it cannot be inserted into the buffer and playback is performed. Could not start.

  FIG. 9 shows an example in which playback cannot be started during a seek operation. In FIG. 9, it is assumed that the buffer has a capacity for inserting three segments. The current playback time is 15, and a segment of 15 to 30 seconds is inserted in the buffer. Assume that seeking to the position at time 5 is performed in this state. At this time, the segment of 15 seconds to 30 seconds is left in the buffer in an unreproduced state. Even if the segment at time 5 is acquired in this state, since three segments remain in the buffer, they cannot be inserted into the buffer. Therefore, it is in a situation where seek regeneration cannot be started.

  In MSE, after the version shown in Non-Patent Document 3, an API called remove method is added, and it is possible to delete a segment of a buffer by specifying an arbitrary range.

  Further, in Patent Document 1, when data is stored in a buffer, if the buffer capacity is full, old data is preferentially erased from reproduced data, or reproduced data remains in the buffer. In the case where there is no data, a technique is disclosed in which new data is preferentially erased from data remaining in the buffer before any position is designated.

JP 2013-16920 A

“Technical Overview and Standardization / Related Technology Trends of Next Generation Video Distribution Technology“ MPEG-DASH ”, Journal of the Institute of Image Information and Television Engineers, Vol.67, No.2, 2013, p.109-115 W3C, “Media Source Extensions W3C Candidate Recommendation 17 July 2014”, [Search January 6, 2015], Internet URL: http://www.w3.org/TR/media-source/ W3C, “Media Source Extensions W3C Editor's Draft 25 February 2013”, [Search January 6, 2015], Internet <URL: https://dvcs.w3.org/hg/html-media/raw-file/668a1c82fb88/ media-source / media-source.html>

  However, there are already a large number of receiving apparatuses in which a version prior to the version shown in Non-Patent Document 3 is installed, and these receiving apparatuses cannot delete the segment of the buffer using the remove method.

  Further, according to the technique disclosed in Patent Document 1, it is possible to perform apparently seamless switching during the waiting time that has occurred when the user designates an arbitrary position in streaming reproduction and performs reproduction. When MSE is used, the method for erasing data from the buffer is implementation-dependent, and this technique cannot always be applied.

  An object of the present invention made in view of such circumstances is to provide a receiving apparatus, a buffer management method, and a program capable of stably performing a seek operation in streaming reproduction.

  In order to solve the above-described problem, a receiving apparatus according to the present invention is a receiving apparatus that receives content composed of a plurality of segments for each segment and performs streaming reproduction, and includes information on a reproduction start time and a reproduction duration time. Corresponding buffer for storing segments, and new playback at the playback time specified by the seek playback command in the buffered segment range indicating the range in which the segment is inserted in the buffer when the seek playback command is received A buffer control unit that acquires information on a redundant segment range that does not include time, and a dummy data that is a segment that includes the redundant segment range and that is smaller in size than the data stored in the redundant segment range. Minimal that produces a minimal segment that is an encoded segment A segment generation unit and a playback unit that reads and decodes a segment from the buffer, and the buffer control unit secures a free space in the buffer by overwriting the redundant segment range with the minimal segment. It is characterized by.

  Further, in the receiving device, the minimum segment generation unit encodes a black image to generate the minimum segment.

  Furthermore, in the receiving apparatus, the minimum segment generation unit generates the minimum segment by connecting template segments stored in advance.

  In order to solve the above-described problem, a buffer management method according to the present invention is a buffer management method in a receiving apparatus that performs streaming playback by receiving content composed of a plurality of segments for each segment, and includes a playback start time And storing the segment in the buffer in association with the information of the playback duration, and when receiving the seek playback command, in the buffered segment range indicating the range in which the segment is inserted in the buffer, by the seek playback command A step of obtaining information of a redundant segment range that is a range not including a new playback time that is a specified playback time, and a segment of a range including the redundant segment range, the data being stored in the redundant segment range Are also encoded with small dummy data. Generating a minimal segment that is a segment, overwriting the redundant segment range with the minimal segment, ensuring free space in the buffer, and reading and decoding the segment from the buffer. It is characterized by that.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the receiving device.

  According to the present invention, when performing streaming reproduction, a stable seek operation can be performed without a function of designating and deleting an arbitrary segment from the buffer.

It is a figure which shows the structural example of the receiver which concerns on one Embodiment of this invention. It is a figure which shows an example of MPD (Media Presentation Description). It is a figure which shows an example of the template segment in the receiver which concerns on one Embodiment of this invention. 4 is a flowchart illustrating a buffer management method of a receiving apparatus according to an embodiment of the present invention. It is a figure which shows the example which overwrite-inserts the minimum segment in a buffer in the receiver which concerns on one Embodiment of this invention. It is a figure which shows an example of the conventional adaptive streaming. It is a figure which shows an example of the buffer model of the conventional MSE. It is a figure which shows the example which overwrite-inserts a segment in a buffer in the buffer model of the conventional MSE. It is a figure which shows the example which cannot start reproduction | regeneration at the time of a seek operation | movement in the buffer model of the conventional MSE.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment described below, as an example, the manifest file is in the MPEG-DASH (ISO / IEC 239-1) MPD (Media Presentation Description) format, and the segment is in the ISOBMFF (ISO / IEC 14496-12) format.

  FIG. 1 is a diagram illustrating a configuration example of a receiving apparatus according to an embodiment of the present invention. The receiving device 1 and the distribution server 2 are connected via the Internet. The receiving device 1 receives content composed of a plurality of segments from the distribution server 2 for each segment, and performs streaming playback.

  The distribution server 2 is a server that distributes the manifest file or segment of the URL specified by the receiving device 1, and can be a general Web server, for example.

  In the example of FIG. 1, the reception device 1 includes a communication I / F 10, a manifest reception unit 11, a segment request unit 12, a buffer 13, a buffer control unit 14, a minimal segment generation unit 15, and a reproduction unit 16. A playback control unit 17, a playback control user I / F 18, and a display unit 19.

  The manifest receiving unit 11 receives a manifest file (MPD) of a desired content from the distribution server 2 via the communication I / F 10, and a segment URL list in which a reproduction time and a segment URL are associated with each other based on the manifest file. Is output to the segment request unit 12.

  FIG. 2 shows an example of MPD. MPD has a hierarchical structure in the order of Period, AdaptationSet, and Representation. Period indicates a section obtained by dividing the program into one or more in the time direction. In the example of FIG. 2, the start time is 0 and the time is 3 minutes. Period is composed of one or a plurality (one in the example of FIG. 2) AdaptationSet. AdaptationSet shows information on media components (video, audio, text, etc.), and multiple parameters (bit rate, screen size, number of audio channels, etc.) with the same media (one in the example of Fig. 2) Components can be included. Information of each media component included in the AdaptationSet is indicated by Representation. Representation is composed of one or a plurality of segments.

The URL of a segment at a certain playback time can be obtained from MPD information. The MPD shown in FIG. 6 will be described as an example. From the start value on the second line, the period start time is zero. From the timescale value in the fourth row, the time scale of the AdaptationSet (the number of scales per second) is 90000. From the duration value in the sixth row, the duration of the segment is 450,000. From the value of startNumber on the sixth line, the segment start number is 1. From the above, the segment number Number for a certain reproduction time t (second) can be calculated by the following equation (1). The numbers below the decimal are rounded down.
Number = (t-start) x timescale / duration + startNumber (1)

  Next, the segment URL template “content_v_ $ Number $ .mp4” is acquired from the media value on the sixth line, and the segment URL is acquired by replacing $ Number $ with the previously calculated Number value. be able to.

  Returning to FIG. 1 again, the segment request unit 12 inputs the reproduction time information of the segment to be acquired from the buffer control unit 14, extracts the URL of the segment corresponding to the input reproduction time information from the segment URL list, The distribution server 2 is requested to send a segment request including the extracted segment URL, and the segment corresponding to the request is received.

  The buffer 13 receives a segment from the buffer control unit 14, stores the segment in association with the playback start time and playback duration of the segment, and buffers the playback time and playback duration information of all currently stored segments. Output to the control unit 14. Further, the playback time segments requested from the playback unit 16 are sequentially output to the playback unit 16.

  The reproduction control user I / F 18 is a function for controlling reproduction by a user operation. A playback start / stop button may be prepared, the start time and end time of the content may be displayed, and the current playback position may be graphically displayed based on the playback time information input from the playback control unit 17. For example, at the bottom of the playback screen, the content start time is displayed as a horizontally long bar with the left end and end time at the right end, and a knob-shaped operation part is placed so as to overlap with it. By moving from the left end to the right end, the user can grasp at a glance which part of the entire reproduction time is being reproduced.

  Further, an arbitrary reproduction position may be designated by moving this knob by a user operation such as a mouse. By a user operation, a playback start command, a stop command, a seek playback command, and new playback time information are output to the playback control unit 17.

  The playback control unit 17 outputs the current playback state (playing or stopped) and playback time information input from the playback unit 16 to the playback control user I / F 18. When a playback start command and a stop command are input from the playback control user I / F 18, the playback unit 16 is instructed to start and stop playback. When a seek playback (playback position change) command is input from the playback control user I / F 18, new playback time information that is information indicating the playback time after the change is output to the buffer control unit 14, and the buffer control unit 14 When a notification indicating that the buffer specified value has been exceeded is input from, new playback time information is output to the playback unit 16 and playback start is instructed.

  When the playback unit 16 receives the playback start command and the new playback time information from the playback control unit 17, the playback unit 16 sequentially reads out the segment of the playback start time indicated by the new playback time information from the buffer 13, and reads the read segment into the segment media. Are decoded and output to the display unit 19, and reproduction time information that is information of a reproduction time updated along with reproduction is output to the reproduction control unit 17. When a stop command is input from the playback control unit 17, playback is stopped.

  The display unit 19 displays the decoded media data input from the playback unit 16 according to the format of the media.

  When the new playback time information is input from the playback control unit 17, the buffer control unit 14 is a redundant segment that does not include the new playback time in the buffered segment range indicating the range in which the segment is inserted in the buffer 13. Get segment range information. The redundant segment range information is an overwrite start time indicating the start time of the redundant segment range, and an overwrite duration time indicating the duration of the redundant segment range.

  The minimum segment generation unit 15 generates a minimum segment that is a segment in a range including the redundant segment range, and is a segment in which dummy data having a size smaller than the data stored in the redundant segment range is encoded, and the buffer control unit 14 Output to. That is, when the overwrite start time information and the overwrite duration time information are input from the buffer control unit 14, a minimum segment that starts from the input overwrite start time information (or slightly before) and is longer than the overwrite duration information is generated. . The dummy data is not particularly limited as long as the data is small in size, but can be a black image, for example.

  The minimum segment generation unit 15 may generate a minimum segment by connecting template segments stored in advance. Here, the template segment is obtained by encoding dummy media data having a small data size and segmenting it in advance. The template segment may be stored in the receiving device 1 or may be acquired from the distribution server 2 at the start of content reproduction, for example. The minimal segment is generated by replacing the template segment parameters with the overwrite start time information and the overwrite duration information.

  Here, an example of the template segment will be described with reference to FIG. FIG. 3 shows an example when the template segment is in ISOBMFF (ISO / IEC 14496-12) format. Note that a plurality of Movie Fragments can be included in one segment, but in this example, only one is included.

  Some parameters related to the present invention will be described. The timescale included in sidx represents the time scale. earliest_presentation_time indicates the reproduction time of the first moving image data of the first Movie Fragment of this segment. subsegment_duration indicates the playback duration for each Movie Fragment. Also, baseMediaDecodeTime included in moof / traf / tfdt in the Movie Fragment indicates the decoding time of the Movie Fragment. For example, when timescale = 90000, the segment of playback start time 10 and playback duration 5 seconds is earliest_presentation_time = baseMediaDecodeTime = timescale × 10 = 900,000. Also, subsegment_duration = timescale × 5 = 450,000.

  That is, earliest_presentation_time may be corrected to 1350000 in order to generate a minimal segment with an overwrite start time of 15 and an overwrite duration of 5 seconds. If the playback duration is to be corrected, the correction range extends to the information in mdat, and the processing load increases. Therefore, by connecting a plurality of template segments until the reproduction duration exceeds the overwrite duration, a desired minimum segment can be generated without correcting the reproduction duration. For example, template segments having playback durations of 1 second, 2 seconds, and 5 seconds, respectively, may be prepared and appropriately combined according to the length of the overwrite duration time.

(Buffer management method)
Next, a buffer management method during the seek operation of the receiving apparatus 1 according to the present invention will be described.

  FIG. 4 is a flowchart illustrating a buffer management method according to an embodiment of the present invention. This flowchart starts from a state in which a manifest file has already been acquired and a segment URL list has been created. In addition, in anticipation that the reception band constantly fluctuates according to the network congestion situation, playback starts after a segment with a buffer specified value (for example, 5 seconds) or more set in advance starting from the playback start time is inserted into the buffer 13. Shall begin.

  In addition, the buffer 13 has a range from the start time to the end time of the content, and stores the segment in association with the information of the playback start time and the playback continuation time. A range in which the content segment is inserted in the buffer 13 is referred to as a buffered segment range.

  First, when a seek operation is executed from the playback control user I / F 18 and a seek playback command is received (step S001), the playback control unit 17 extracts new playback time information from the seek playback command and outputs it to the buffer control unit 14. (Step S002).

  Next, the buffer control unit 14 confirms whether or not there is a segment in the buffer 13 in a range that is equal to or greater than a preset buffer value from the new playback time (step S003). If step S003 is YES, the data stored in the buffer 13 can be reproduced as it is, and the reproduction control unit 17 starts reproduction (step S013). On the other hand, when step S003 is NO, since it is necessary to acquire data, it is confirmed whether or not there is free space for one segment in the buffer 13 (step S004).

  If there is no free space for one segment in the buffer 13 (step S004-NO), it is confirmed whether or not there is a redundant segment range that is a buffered segment range that does not include the new playback time in the buffer 13 ( Step S005). When the redundant segment range exists in the buffer 13 (step S005-YES), the redundant segment range is extracted, the minimum value of the redundant segment range is set as the overwrite start time, and the length of the redundant segment range is set as the overwrite duration time. The minimum segment generation unit 15 generates a minimum segment based on the input overwrite start time and overwrite overwrite time, and outputs the minimum segment to the buffer control unit 14 (step S006). The buffer control unit 14 inserts the input minimum segment into the buffer 13 to overwrite the redundant segment range (step S007), and returns to step S004.

  FIG. 5 is a diagram illustrating an example in which a minimal segment is overwritten and inserted into the buffer 13. As shown in FIG. 5, for example, when a segment for 15 seconds from start time 15 to start time 30 is stored in the buffer 13 and the new reproduction time is 5, the segment stored in the buffer 13 is wasted ( It can be considered as a redundant segment. Therefore, in this case, the start time 15 to 30 is set as the redundant segment range, and the generated minimal segment is overwritten on the redundant segment range. In this figure, in order to show that the data capacity of the minimal segment is small, the area of the minimal segment is displayed in a pseudo-small manner. In this way, a free space can be secured by overwriting a minimal segment with a small amount of data.

  Next, returning to step S005, if there is no redundant segment range in the buffer 13 (step S005-NO), the process waits until the regenerated segment is erased from the buffer 13 (step S008), and returns to step S004.

  Returning to step S004, if there is free space for one segment in the buffer 13 (step S004-YES), the segment request unit 12 performs the following operation. If there is no segment including the new playback time in the buffer 13, it is confirmed whether or not the segment URL including the new playback time is present in the segment URL list (step S009). If there is a segment including the new playback time in the buffer 13, the segment following the segment including the maximum value of the buffered segment range including the new playback time (that is, following the buffered segment including the new playback time). It is confirmed whether or not the (segment) URL exists in the segment URL list (step S009). When the URL of the segment exists in the segment URL list (step S009-YES), the URL of the segment is extracted from the segment URL list. The segment request unit 12 transmits a segment request including the extracted URL of the segment to the distribution server 2, receives the segment corresponding to the request, and outputs it to the buffer control unit 14 (step S010). The buffer control unit 14 inserts the received segment into the buffer 13 (step S011).

  If step S009 is NO, nothing is done and the process proceeds to step S012. In step S012, the playback control unit 17 confirms whether playback is currently being performed. If playback has not started (step S012-NO), the process returns to step S003. On the other hand, if playback has started (step S012-YES), it is confirmed whether the end of the buffer 13 has been reached or whether the stop button has been pressed in the playback control user I / F 18, and none of them is satisfied. (Step S014-NO), it returns to step S004. If either one is satisfied (step S014-YES), the reproduction is stopped and the process is terminated.

  As described above, even when the free space necessary for inserting the segment in the buffer 13 is insufficient, the redundant segment range which is a buffered segment range not including the new reproduction time is overwritten with the minimal segment, Free space can be secured quickly, and stable playback can be achieved by shortening the time until the start of playback after a seek operation.

  As an example, a segment with a playback duration of 5 seconds at 8 Mbps is 5 MB, whereas a 5 second minimum segment that encodes a black image is about 7 KB, so the redundant segment range is overwritten with the minimum segment. Thus, it is possible to secure a free space near 5 MB (4.993 MB).

  As described above, the receiving device 1 can secure the free space of the buffer 13 by overwriting the redundant segment range with the minimal segment when receiving the seek reproduction command. Thus, the receiving apparatus 1 can stably perform a seek operation without the function of designating and erasing an arbitrary segment from the buffer 13.

  Note that a computer can be suitably used to cause the above-described function to function as the receiving device 1, and such a computer stores a program describing processing contents for realizing each function of the receiving device 1 in a storage unit of the computer. It can be realized by reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks and steps described in the embodiments can be combined into one, or one constituent block and steps can be divided.

1 Receiving Device 2 Distribution Server 10 Communication I / F
DESCRIPTION OF SYMBOLS 11 Manifest reception part 12 Segment request part 13 Buffer 14 Buffer control part 15 Minimal segment production | generation part 16 Playback part 17 Playback control part 18 Playback control user I / F
19 Display

Claims (5)

A receiving device that receives content composed of a plurality of segments and performs streaming playback for each segment,
A buffer for storing a segment in association with information of a playback start time and a playback duration;
Redundant segment range that does not include the new playback time that is the playback time specified by the seek playback command in the buffered segment range that indicates the range in which the segment is inserted in the buffer when the seek playback command is received A buffer control unit for acquiring the information of
A minimum segment generation unit that generates a minimum segment that is a segment of a range including the redundant segment range and that is a segment obtained by encoding dummy data that is smaller in size than data stored in the redundant segment range;
A playback unit that reads and decodes the segment from the buffer,
The receiving apparatus according to claim 1, wherein the buffer control unit secures a free capacity of the buffer by overwriting the redundant segment range with the minimal segment.   The receiving apparatus according to claim 1, wherein the minimum segment generation unit generates the minimum segment by encoding a black image.   The receiving apparatus according to claim 1, wherein the minimal segment generation unit generates the minimal segment by connecting template segments stored in advance. A buffer management method in a receiving apparatus that receives content composed of a plurality of segments and performs streaming reproduction for each segment,
Storing the segment in a buffer in association with the information of the playback start time and playback duration;
Redundant segment range that does not include the new playback time that is the playback time specified by the seek playback command in the buffered segment range that indicates the range in which the segment is inserted in the buffer when the seek playback command is received Obtaining the information of
Generating a minimal segment that is a segment of a range that includes the redundant segment range and that is encoded with dummy data that is smaller in size than the data stored in the redundant segment range;
Securing free space in the buffer by overwriting the redundant segment range with the minimal segment;
Reading and decoding segments from the buffer;
A buffer management method comprising:   The program for functioning a computer as a receiver as described in any one of Claim 1 to 3.

Images