JP6581884B2 - Reception device, buffer management method, and program - Google Patents

Description

  The present invention relates to a receiving apparatus that receives content composed of a plurality of segments for each segment and performs streaming reproduction, a buffer management method for the receiving apparatus, and a program for causing the receiving apparatus to function.

  In the streaming video distribution on the Internet nowadays, a transition from a streaming distribution method using a dedicated server and a dedicated protocol to a streaming distribution method using an HTTP protocol by a general-purpose Web server is progressing. It has become mainstream in distribution. As such a streaming distribution method (adaptive streaming) based on the HTTP protocol, an original technology by an IT vendor is prevalent, and MPEG-DASH (ISO / IEC 2309-1) which is an international standard intended to unify these streaming methods. ) Was formulated.

  The basic concept is the same for both technologies. The web server divides a stream in which video content is encoded with one or more qualities (screen size and bit rate) into files of several seconds to tens of seconds each. Manifest files that describe things (segments) and attributes and URLs of the moving image contents are prepared. The receiving device receives the segments one after another by selecting the timely quality from the manifest file in consideration of the screen size of the receiving device and the state of the network bandwidth of the transmission path, etc., and joins and plays back one segment of video content (For example, refer nonpatent literature 1).

  FIG. 5 shows an example of adaptive streaming. In FIG. 5, the video is encoded with a plurality of qualities, and segments divided into units of several seconds are generated for each quality. In this example, the encoder splits the video into low quality segments 1-4, medium quality segments 1'-4 ', and high quality segments 1 "-4". In the manifest file, the URL of the segment and attribute information such as the bit rate and image size for each quality are registered. The segment and manifest file are supplied to the web server.

  The receiving device receives the manifest file from the Web server, grasps the bit rate and segment configuration from the manifest file (step A), and determines the quality in units of segments based on the screen size of the receiving device and the status of the network receiving bandwidth. Is received by the HTTP protocol (step B). Subsequently, the received segments are connected in order according to the manifest file and reproduced (step C). Thus, by repeating the processes of steps B and C, a series of contents can be received and played back.

  In the receiving apparatus, a viewing environment compatible with multi-devices (PCs, television receivers, tablet devices, etc.) can be constructed by using an HTML5 browser that is widely used in many devices.

  However, since a video element (video tag) that is a function for playing back a moving image in an HTML5 browser can only refer to a media file as a source, in order to realize such adaptive streaming viewing, It was necessary to install a browser plug-in separately. For this reason, it was difficult to build a viewing environment that can be used by all browsers. Under such circumstances, an international non-profit organization that promotes standardization of technologies used on the Web as a function for realizing adaptive streaming viewing using video elements (video tags) of HTML5 browser and Javascript (registered trademark). In W3C, standardization of MSE (Media Source Extensions) is progressing (see Non-Patent Document 2, for example).

  By using MSE, video elements can refer to a buffer as a source. Since the segment received by Javascript can be inserted into the buffer, various adaptive streaming viewing can be realized by the video element of HTML5 and Javascript.

  Here, the buffer model of MSE will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of an MSE buffer model. The buffer (SourceBuffer) manages segments in association with the segment start time and playback duration. In FIG. 6, hatched blocks indicate segments, the segment start time, which is the playback start time for each segment, is 0 seconds, the playback duration is 5 seconds, the segment start time is 5 seconds, and the playback duration is 5 seconds. Three segments having a second and a segment start time of 15 seconds and a playback duration of 5 seconds are inserted in the buffer. At this time, the buffering range (buffered attribute) of the buffer is 0 to 10 seconds and 15 to 20 seconds.

  In addition, as shown in FIG. 7, it is possible to insert another segment whose playback time overlaps the range where the segment has already been inserted and overwrite it. FIG. 7 shows an example in which a segment having a segment start time of 12 seconds and a playback duration of 10 seconds is inserted as a new segment in the example shown in FIG. As a result, the range of 12 to 22 seconds is replaced with a new segment, and the buffering range is also updated to 0 to 10 seconds and 12 to 22 seconds.

  As a method of providing a streaming service for moving images, the switching of content linked in time series, such as inserting video clips such as CMs in the middle of one program, or providing programs one after another according to the organization like broadcasting, etc. However, a service form for viewing is conceivable. In MPEG-DASH, a manifest file description method that enables such a service is defined.

  FIG. 8 shows a conceptual diagram of a manifest file in the MPEG-DASH MPD (Media Presentation Description) format. The MPD format has a hierarchical structure in the order of Period, AdaptationSet, and Representation. Period indicates a section (one content) obtained by dividing the manifest file in the time direction. In the example of FIG. 8, each content duration is 300 seconds, 30 seconds, and 300 seconds of Period. It shows that. Period is composed of one or a plurality of AdaptationSets.

  AdaptationSet indicates information on media components (video, audio, text, etc.), and can include a plurality of components with different parameters (bit rate, screen size, audio channel number, etc.) on the same media. Information of each media component included in the AdaptationSet is indicated by Representation. Representation is composed of one or a plurality of segments.

  By using such a manifest file, it becomes possible to sequentially receive segments (Initialization Segment, Media Segment in FIG. 8) that form content connected in time series in accordance with the playback time.

  Regarding content switching in streaming playback, an information processing apparatus and an information processing method are known that can switch content to be played back when the playback content is good when the buffer amount is insufficient ( For example, see Patent Document 1).

JP 2011-77597 A

“Technical Overview and Standardization / Related Technology Trends of Next Generation Video Distribution Technology“ MPEG-DASH ”, Journal of the Institute of Image Information and Television Engineers, Vol.67, No.2, 2013, p.109-115 W3C, “Media Source Extensions W3C Candidate Recommendation 17 July 2014”, [searched November 6, 2015], Internet <URL: http://www.w3.org/TR/media-source/>

  However, when trying to play back the content linked in time series as described above, if the content is generated by a different encoder, the time length of all segments constituting the content is slightly shorter than the duration value of Period. May be. Then, a time gap is generated in the buffer at the time of content switching (change of Period).

  FIG. 9 shows an example in which a time gap is generated in the buffer. In FIG. 9, the period before switching is Period # 1, the period after switching is Period # 2, the duration parameter of Period # 1 is 300 seconds, and the total time length of all the segments of Period # 1 is 299.9 seconds. An example is shown. In this case, a gap of 0.1 second is generated in the buffer at the connection portion of Period # 1 and Period # 2. In such a case, there was a risk that playback would stop at the transition of Period.

  Also, Patent Document 1 describes switching content to be played back at a time when the playback content is good when the buffer amount is insufficient with regard to switching content in streaming playback. It did not mention the details of the details of the buffer, and it was not always a technique that could be used to solve the problem.

  An object of the present invention made in view of such circumstances is a receiving apparatus, a buffer management method, and a buffer management method capable of smoothly switching contents when streaming a program composed of contents linked in time series. To provide a program.

  In order to solve the above problems, a receiving apparatus according to the present invention is a receiving apparatus that receives a program composed of a plurality of segments connected in time series and performs streaming playback for each segment. A segment acquisition unit for acquiring a segment, a buffer for storing the segment in association with a segment start time and a playback continuation time, and a time scale for all the segments constituting the content stored in the buffer. A filler segment generation unit that generates a filler segment that is equal to or greater than the difference between the content scale and the time scale of all the segments when the content length obtained from the meta information indicating the scale is shorter than the content scale, and the filler segment Insert into the buffer following the last segment of the content A buffer control unit that, characterized in that it comprises a.

  Furthermore, in the receiving apparatus according to the present invention, the filler segment generation unit, when the media format of the segment inserted in the buffer is video, sets the filler segment as a segment encoding a white image or a black image. It is characterized by.

  Furthermore, the receiving device according to the present invention is characterized in that, when the media format of the segment inserted in the buffer is voice, the filler segment generation unit sets the filler segment as a segment encoding silence. To do.

  Furthermore, the receiving apparatus according to the present invention is characterized in that the filler segment generation unit generates the filler segment by connecting template segments prepared in advance.

In order to solve the above-described problem, the buffer management method according to the present invention is a receiving apparatus that receives a program composed of a plurality of segments connected in time series for each segment and performs streaming playback. A buffer management method according to claim 1, wherein a step of acquiring a segment, a step of storing the segment in a buffer in association with a segment start time and a playback duration, and a step of storing all segments constituting the content stored in the buffer When the time scale is shorter than the content scale which is meta information indicating the content scale, a step of generating a filler segment whose playback duration is equal to or greater than the difference between the content scale and the time scale of all the segments; and the filler A segment before the last segment of the content Characterized in that it comprises the steps of inserting into the buffer, the.

  In order to solve the above problems, a program according to the present invention causes a computer to function as the receiving device.

  According to the present invention, in a receiving apparatus that performs streaming playback of a program composed of contents linked in time series using a buffer that manages segments in association with information on segment start time and playback duration, Even when the time scales of all the segments constituting the content are shorter than the content scales of the set individual contents, the content can be switched smoothly, thereby enabling stable viewing.

It is a block diagram which shows the structural example of the receiver which concerns on one Embodiment of this invention. It is a figure which shows an example of the manifest file which the receiver which concerns on one Embodiment of this invention acquires. It is a figure which shows an example of the filler segment which the receiver which concerns on one Embodiment of this invention produces | generates. 4 is a flowchart illustrating a buffer management method of a receiving apparatus according to an embodiment of the present invention. It is a figure which shows an example of the conventional adaptive streaming. It is a figure which shows an example of the buffer model of the conventional MSE. It is a figure which shows the example which overwrite-inserts a segment in a buffer in the buffer model of the conventional MSE. It is a figure which shows the concept of the conventional manifest file. It is a figure explaining the conventional subject.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment described below, as an example, the manifest file is in the MPEG-DASH (ISO / IEC 239-1) MPD (Media Presentation Description) format, and the segment is in the ISOBMFF (ISO / IEC 14496-12) format. Further, the time corresponding to the head of the segment among the reproduction times is referred to as “segment start time”.

  FIG. 2 shows an example (extract) of a manifest file. In the example of FIG. 2, two periods with a content duration of 3 minutes are connected, and each Period has one AdaptationSet, and each AdaptationSet has one Representation. ing.

The URL of the segment at a certain playback time can be obtained from information in the manifest file. A description will be given based on the example of FIG. From the start value on the second line, the period start time is zero. From the timescale value in the fourth row, the adaptationSet timescale (number of scales per second) is 90000. From the duration value in the sixth row, the duration of the segment is 450,000. From the value of startNumber on the sixth line, the segment start number is 1. From the above, the segment number for a certain reproduction time t (second) can be calculated by the following equation (1). The number below the decimal is rounded down.
Number = (t-start) x timescale / duration + startNumber (1)

  Next, the segment URL template “content_v_ $ Number $ .mp4” is acquired from the media value on the sixth line, and the segment URL is acquired by replacing $ Number $ with the previously calculated Number value. be able to.

  Although the above example has been described based on the first period of FIG. 2, the second period can be calculated in the same manner. In the present embodiment, information reproduced by the entire MPD file is referred to as “program”, and information reproduced by each period is referred to as “content”. That is, the program is configured by linking contents in time series.

(Receiver configuration)
Next, a configuration example of a receiving apparatus according to an embodiment of the present invention will be described with reference to FIG.

  The receiving device 1 is connected to the distribution server 2 via the Internet, receives content composed of a plurality of segments from the distribution server 2 for each segment, and performs streaming reproduction.

  The distribution server 2 is a server that distributes the manifest file or segment of the URL specified by the receiving device 1, and can be a general Web server, for example.

  1 includes a communication I / F 10, a manifest acquisition unit 11, a segment acquisition unit 12, a filler segment generation unit 13, a buffer 14, a buffer control unit 15, a reproduction control unit 16, A playback control user I / F 17, a playback unit 18, and a display unit 19 are provided.

  The reproduction control user I / F 17 outputs a reproduction command and a stop command to the reproduction control unit 16 by a user operation. The reproduction command includes reproduction start time information indicating the reproduction start time.

  The playback control user I / F 17 prepares a playback start / stop button, and based on the start and end times of content and the current playback time information input from the playback control unit 16, the current playback time (playback position). ) May be displayed graphically. For example, at the bottom of the playback screen, the content start time is displayed as a horizontally long bar with the left end and end time at the right end, and a knob-shaped operation part is placed so as to overlap with it. By moving from the left end to the right end, the user can grasp at a glance which part of the entire reproduction time is being reproduced. Further, an arbitrary reproduction time may be designated by moving this knob by a user operation such as a mouse.

  When receiving a stop command from the playback control user I / F 17, the playback control unit 16 instructs the playback unit 18 to stop.

  In addition, when a playback command is input from the playback control user I / F 17, the playback control unit 16 extracts a playback start time from the playback command and outputs playback start time information to the buffer control unit 15. Then, when the notification indicating that the buffer specified value (for example, 5 seconds) has been exceeded is input from the buffer control unit 15, the playback control unit 16 outputs the playback start time information to the playback unit 18 and instructs the playback start. .

  Further, the playback control unit 16 outputs the current playback state (playing and stopped) and the current playback time information input from the playback unit 18 to the playback control user I / F 17.

  The manifest acquisition unit 11 acquires a manifest file (MPD file) of a desired program from the distribution server 2 via the communication I / F 10 when an application is started or a program is selected. Then, a segment URL list in which the segment start time and the URL of the segment are associated with each other is generated based on the manifest file, and is output to the segment acquisition unit 12 and the buffer control unit 15.

  The buffer control unit 15 determines the segment start time of the acquisition target segment based on the reproduction start time information input from the reproduction control unit 16 and the buffering range (buffered segment range) information input from the buffer 14. . Then, the URL of the segment associated with the segment start time is extracted from the segment URL list input from the manifest acquisition unit 11 and output to the segment acquisition unit 12. Further, when a segment is input from the segment acquisition unit 12, the buffer control unit 15 inserts the segment into the buffer 14.

  The segment acquisition unit 12 acquires a corresponding segment based on the URL of the acquisition target segment input from the buffer control unit 15 and outputs the acquired segment to the buffer control unit 15. Specifically, a segment request (request) including the segment URL is generated and transmitted to the distribution server 2 via the communication I / F 10, and the distribution server 2 responds as a response to the request via the communication I / F 10. Get the segment to perform. The segment acquisition unit 12 may input the segment start time from the buffer control unit 15, and the segment acquisition unit 12 may extract the URL of the segment associated with the segment start time from the segment URL list.

  The buffer 14 receives a segment from the buffer control unit 15 and stores the segment in association with the segment start time and the reproduction duration time.

  Further, the buffer 14 outputs the buffering range information to the buffer control unit 15. Here, the buffering range information is information indicating the range in which the content segment is inserted in the buffer 14, and is, for example, information on the segment start time and playback duration of all currently stored segments. .

  Further, the buffer 14 sequentially outputs the segments requested from the reproduction unit 18 to the reproduction unit 18.

  The playback unit 18 stops playback when a stop command is input from the playback control unit 16.

  Further, when the playback unit 18 inputs a playback command including playback start time information from the playback control unit 16, the playback unit 18 sequentially inputs segments from the buffer 14 based on the playback start time, and the input segments are converted into the segment media format ( Video, audio, text, etc.) and output to the display unit 19. The playback unit 18 outputs current playback time information, which is playback time information updated with playback, to the playback control unit 16.

  The display unit 19 displays the decoded media data input from the playback unit 18 according to the format of the media.

  Below, the production | generation of a filler segment is demonstrated.

  When the segment inserted into the buffer 14 is the last segment of the currently playing period (Period), the buffer control unit 15 sets the time length of all segments (maximum value of the buffering range) and the end of the period. Compare time. Here, the end time of the period is a time obtained by adding the content scale to the start time (start) of the period. The content scale is obtained from meta information (duration) indicating the scale of the content. If the period start time is 0, the period end time and the content scale are the same.

  When the maximum value of the buffering range constituting the content stored in the buffer 14 is smaller than the end time of the period, that is, when the time scale of all the segments is shorter than the content scale, the buffer control unit 15 And the difference between the period end time and the maximum value of the buffering range is determined as a gap time, and filler information indicating the insertion start time and the gap time is output to the filler segment generation unit 13. In addition, when a filler segment is input from the filler segment generation unit 13, the buffer control unit 15 inserts the filler segment into the buffer 14 following the last segment of the content.

  When the filler information is input from the buffer control unit 15, the filler segment generation unit 13 generates a filler segment that starts from the insertion start time and has a reproduction duration of the gap time or more, and outputs the filler segment to the buffer control unit 15. Here, the filler segment is, for example, data obtained by encoding a white image or a black image when the media format of the segment is video. When the media format of the segment is audio, silence is encoded data. If the playback duration of the filler segment is greater than or equal to the gap time, it is possible to eliminate the gap at the transition of the period by overwriting the segment of the next period on the filler segment and prevent playback from stopping at the transition of the period. it can.

  In addition, the filler segment generation unit 13 can segment media data having a small data size in advance and prepare it as a template, and use the data obtained by replacing the template parameters based on the filler information. The template of the filler segment can be stored in the receiving device 1, or may be acquired from the distribution server 2, for example, at the start of content reproduction.

  An example of the filler segment will be described with reference to FIG. FIG. 3 shows an example in which the filler segment is in ISOBMFF (ISO / IEC 14496-12) format. Note that a plurality of Movie Fragments can be included in one segment, but in this example, only one is included.

Some parameters related to the present invention will be described. The timescale included in sidx represents the time scale. earliest_presentation_time indicates the playback time of the first media data of the first Movie Fragment of this segment. subsegment_duration indicates the playback duration for each Movie Fragment. Also, baseMediaDecodeTime included in moof / traf / tfdt in the Movie Fragment indicates the playback duration of the Movie Fragment. For example, when timescale = 90000, the segment with the playback start time t and playback duration d seconds is
earliest_presentation_time = 90000 × t
subsegment_duration = baseMediaDecodeTime = 90000 × d
It becomes.

  That is, in order to generate a filler segment with an insertion start time of 15 seconds and a playback duration of 0.5 seconds, earliest_presentation_time may be corrected to 1350000 and baseMediaDecodeTime to 45000. If the playback duration is to be corrected, the correction range extends to the information in mdat, and the processing load increases. Therefore, by preparing a template segment in advance and connecting a plurality of template segments until the reproduction duration exceeds the gap time, a desired filler segment can be generated without correcting the reproduction duration.

(Buffer management method at the start of playback)
Next, a buffer management method at the start of reproduction of the receiving apparatus 1 will be described with reference to the flowchart shown in FIG.

  The flowchart shown in FIG. 4 starts from a state in which a manifest file has already been acquired and a segment URL list has been created. In addition, the receiving apparatus 1 inserts a segment having a buffer specified value (for example, 5 seconds) or more set in advance into the buffer 14 with the reproduction start time as a starting point in anticipation that the reception band constantly fluctuates according to the network congestion situation. It is assumed that the playback starts after being done.

  First, when a playback start operation is executed in the playback control user I / F 17 (step S001), the playback control unit 16 sets the playback start time (0 second) at the beginning of the program when starting a new playback. When reproduction is resumed after an operation such as pause, the reproduction start time at the time when the pause operation is performed is extracted as reproduction start time information and output to the buffer control unit 15 (step S002).

  Next, the buffer control unit 15 checks whether or not there is a segment in the buffer 14 in a range that is equal to or greater than a preset buffer specified value from the reproduction start time information (step S003). If the segment does not exist (step S003-NO), the process proceeds to step S004. If the segment exists (step S003-YES), reproduction starts (step S012), and then the process proceeds to step S004.

  In step S004, the buffer control unit 15 confirms whether or not the URL of the segment following the last segment in the buffering range including the reproduction time exists in the segment URL list. If the URL of the segment exists (YES in step S004), the buffer control unit 15 extracts the URL of the segment from the segment URL list and outputs it to the segment acquisition unit 12 (step S005). Subsequently, the segment acquisition unit 12 transmits a segment request including the extracted segment URL to the distribution server 2, receives the segment corresponding to the request, and outputs the segment to the buffer control unit 15 (step S005). The buffer control unit 15 inserts the segment received in step S005 into the buffer 14 (step S006).

  Next, the buffer control unit 15 confirms whether or not the segment inserted into the buffer 14 is the last segment of the currently playing period (Period) (step S007). If it is the last segment of the period (step S007-YES), the maximum value of the buffering range is compared with the end time of the period (step S008).

  When the maximum value of the buffering range is smaller than the end time of the period (step S008-YES), the filler segment generation unit 13 sets the maximum value of the buffering range to the insertion start time, the period end time, and the maximum of the buffering range. A filler segment is generated with the difference from the value equal to or greater than the playback duration as a playback duration, and is output to the buffer control unit 15. The buffer control unit 15 inserts the filler segment into the buffer 14 (step S009).

  If NO in steps S004, S007, and S008, it is confirmed whether or not playback is currently being performed (step S010). If it is not being reproduced (step S010-NO), the process returns to step S003. If playback is in progress (YES in step S010), it is confirmed whether the end of the buffer 14 has been reached or whether the stop button has been pressed in the playback control user I / F 17 (step S011). If any condition is satisfied (step S011-YES), the process is terminated. If step S010 is NO, the process returns to step S003. If step S011 is NO, the process returns to step S004.

  As described above, when the time scale of all the segments constituting the period is shorter than the content scale of the period being played back, a filler segment having a duration longer than the difference between the content scale of the period and the time scale of all segments is selected. By generating and inserting the last segment of the period (the last segment of the content) into the buffer 14, the gap of the buffer 14 between the periods (contents) can be filled, so that the switching of the periods (contents) is smooth Thus, stable viewing is possible.

  Note that a computer can be suitably used to cause the above-described function to function as the receiving device 1, and such a computer stores a program describing processing contents for realizing each function of the receiving device 1 in a storage unit of the computer. It can be realized by reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks or processing steps described in the embodiments can be combined into one, or one can be divided into a plurality.

1 Receiving Device 2 Distribution Server 10 Communication I / F
DESCRIPTION OF SYMBOLS 11 Manifest acquisition part 12 Segment acquisition part 13 Filler segment production | generation part 14 Buffer 15 Buffer control part 16 Playback control part 17 Playback control user I / F
18 Playback section 19 Display section

Claims (6)

A receiving apparatus that receives a program configured by linking time-series content composed of a plurality of segments and performs streaming playback for each segment,
A segment acquisition unit for acquiring segments;
A buffer for storing the segment in association with a segment start time and a playback duration;
When the time scale of all the segments constituting the content stored in the buffer is shorter than the content scale obtained from the meta information indicating the content scale, the playback duration is the time scale of the content scale and all the segments. A filler segment generator that generates a filler segment greater than or equal to
A buffer controller that inserts the filler segment into the buffer following the last segment of the content;
A receiving apparatus comprising:   2. The filler segment generation unit according to claim 1, wherein when the media format of the segment inserted into the buffer is video, the filler segment is a segment obtained by encoding a white image or a black image. Receiver.   2. The receiving apparatus according to claim 1, wherein, when the media format of the segment inserted into the buffer is voice, the filler segment generation unit sets the filler segment as a segment in which silence is encoded. .   The receiving device according to any one of claims 1 to 3, wherein the filler segment generation unit generates the filler segment by connecting template segments prepared in advance. A buffer management method in a receiving device that receives a program configured by linking time-series content composed of a plurality of segments and performs streaming playback,
Obtaining a segment;
Storing the segment in a buffer in association with the segment start time and playback duration;
When the time scale of all segments constituting the content stored in the buffer is shorter than the content scale that is meta information indicating the scale of the content, the playback duration time of the content scale and the time scale of all the segments is set. Generating filler segments greater than or equal to the difference;
Inserting the filler segment into the buffer following the last segment of the content;
A buffer management method comprising:   The program for functioning a computer as a receiver as described in any one of Claim 1 to 4.

Images