JP6609468B2 - Receiving device, reproduction time control method, and program - Google Patents

Description

  The present invention relates to a receiving device, a playback time control method, and a program that receive content composed of a plurality of media components for each segment.

  In the streaming video distribution on the Internet nowadays, a transition from a streaming distribution method using a dedicated server and a dedicated protocol to a streaming distribution method using an HTTP protocol by a general-purpose Web server is progressing. It has become mainstream in distribution. As such a streaming distribution method (adaptive streaming) based on the HTTP protocol, an original technology by an IT vendor is widespread, and MPEG-DASH (ISO / IEC 2309-1) which is an international standard intended to unify these streaming methods. ) Was formulated.

  The basic concept is the same for both technologies. The web server divides a stream in which video content is encoded with one or more qualities (screen size and bit rate) into files of several seconds to tens of seconds each. Manifest files that describe things (segments) and attributes and URLs of the moving image contents are prepared. The receiving device receives the segments one after another by selecting the timely quality from the manifest file in consideration of the screen size of the receiving device and the state of the network bandwidth of the transmission path, etc., and joins and plays back one segment of video content (For example, refer nonpatent literature 1).

  FIG. 5 shows an example of adaptive streaming. In FIG. 5, the video is encoded with a plurality of qualities, and segments divided into units of several seconds are generated for each quality. In this example, the encoder splits the video into low quality segments 1-4, medium quality segments 1'-4 ', and high quality segments 1 "-4". In the manifest file, the URL of the segment and attribute information such as the bit rate and image size for each quality are registered. The segment and manifest file are supplied to the web server.

  The receiving device receives the manifest file from the Web server, grasps the bit rate and segment configuration from the manifest file (step A), and determines the quality in units of segments based on the screen size of the receiving device and the status of the network receiving bandwidth. Is received by the HTTP protocol (step B). Subsequently, the received segments are connected in order according to the manifest file and reproduced (step C). Thus, by repeating the processes of steps B and C, a series of contents can be received and played back.

  In the receiving apparatus, a viewing environment compatible with multi-devices (PCs, television receivers, tablet devices, etc.) can be constructed by using an HTML5 browser that is widely used in many devices.

  However, since a video element (video tag) that is a function for playing back a moving image in an HTML5 browser can only refer to a media file as a source, in order to realize such adaptive streaming viewing, It was necessary to install a browser plug-in separately. For this reason, it was difficult to build a viewing environment that can be used by all browsers. Under such circumstances, an international non-profit organization that promotes standardization of technologies used on the Web as a function for realizing adaptive streaming viewing using video elements (video tags) of HTML5 browser and Javascript (registered trademark). In W3C, standardization of MSE (Media Source Extensions) is progressing (see Non-Patent Document 2, for example).

  By using MSE, video elements can refer to a buffer as a source. Since the segment received by Javascript can be inserted into the buffer, various adaptive streaming viewing can be realized by the video element of HTML5 and Javascript.

  Here, the buffer model of MSE will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of an MSE buffer model. The buffer (SourceBuffer) manages segments in association with the segment start time and playback duration. In FIG. 6, hatched blocks indicate segments, the segment start time, which is the playback start time for each segment, is 0 seconds, the playback duration is 5 seconds, the segment start time is 5 seconds, and the playback duration is 5 seconds. Three segments having a second and a segment start time of 15 seconds and a playback duration of 5 seconds are inserted in the buffer. At this time, the buffering range (buffered attribute) of the buffer is 0 to 10 seconds and 15 to 20 seconds.

  In addition, as shown in FIG. 7, it is possible to insert another segment whose playback time overlaps the range where the segment has already been inserted and overwrite it. FIG. 7 shows an example in which a segment having a segment start time of 12 seconds and a playback duration of 10 seconds is inserted as a new segment in the example shown in FIG. As a result, the range of 12 to 22 seconds is replaced with a new segment, and the buffering range is also updated to 0 to 10 seconds and 12 to 22 seconds.

  As a method of providing a video streaming service, a service format (multi-angle) that allows users to select desired video from multiple media components and view it while switching, such as enabling viewing of video at a different angle in the same program in addition to the main video. Service). In MPEG-DASH, a manifest file description method that enables such a service is defined.

  FIG. 8 shows a conceptual diagram of a manifest file in the MPEG-DASH MPD (Media Presentation Description) format. The MPD format has a hierarchical structure in the order of Period, AdaptationSet, and Representation. Period indicates a section obtained by dividing the manifest file in the time direction. In the example of FIG. 8, each content has a duration of 300 seconds. Period is composed of one or more AdaptationSets.

  AdaptationSet indicates information of media components (video, audio, text, etc.), and can include a plurality of components expressed in the same media format (videos of different angles, audio of different languages, etc.). In the example of FIG. 8, there are two videos (mimeType = video / mp4) and one audio (mimeType = audio / mp4). Information of each media component included in AdaptationSet is represented by Representation. Representation consists of one or more segments.

  By making such a manifest file, it becomes possible to sequentially receive the segments (Initialization Segment, Media Segment in FIG. 8) forming the media component (AdaptationSet) selected in the Period, and select the media component. By correcting, it is possible to switch to reception of a segment forming another media component and display another video.

  A video switching method for streaming video data is known as a technique for seamlessly switching the representation of multimedia content between the representations while streaming using a network streaming protocol (see, for example, Patent Document 1). ).

JP 2014-209751 A

“Technical Overview and Standardization / Related Technology Trends of Next Generation Video Distribution Technology“ MPEG-DASH ”, Journal of the Institute of Image Information and Television Engineers, Vol.67, No.2, 2013, p.109-115 W3C, “Media Source Extensions W3C Candidate Recommendation 17 July 2014”, [searched November 6, 2015], Internet <URL: http://www.w3.org/TR/media-source/>

  However, in general, when viewing streaming, the reception band always changes depending on the network congestion, so data from several seconds to several tens of seconds ahead of the current playback time is received and inserted into the buffer. In general, the viewing stability is improved. For this reason, when switching the media components sharing the reproduction time in this way, it may take time until the switching, or segments that have already been acquired may be wasted.

  FIG. 9 shows an example of switching media components. In FIG. 9, the media component before switching (AdaptationSe) is AdaptationSet # 1, and the media component after switching is AdaptationSet # 2. Also, the hatched block indicates the segment, and the segment playback duration is 5 seconds. Switching of media components is assumed to be triggered by a switching instruction. FIG. 9A shows a playback state immediately before a media component switching instruction, and shows a state in which a 10-second segment is acquired and inserted into the buffer prior to the current playback time.

  FIG. 9B shows an example in which a segment of AdaptationSet # 2 is added after the last segment inserted into the buffer after switching of media components. In this case, since the media component is switched about 10 seconds after the switching instruction, the response until the switching is slow and the convenience of the user may be impaired.

  FIG. 9C shows an example in which the segment of AdaptationSet # 2 is overwritten after the segment including the current playback time after switching the media component. In this case, although the media component is switched immediately after the switching instruction, the previously acquired segment of AdaptationSet # 1 is overwritten with the newly acquired segment of AdaptationSet # 2, so the already acquired segment is wasted .

  The invention described in Patent Document 1 does not touch on the detailed situation of the buffer as described above, and is not necessarily a technique that can be used to solve the problem.

  An object of the present invention made in view of such circumstances is that a content composed of a main video, a video at a different angle (multi-angle) associated with the main video, and a video at a different playback speed (fast forward / rewind). It is to reduce the waste of buffering when switching media components.

In order to solve the above-described problem, a receiving device according to the present invention is a receiving device that receives content composed of a plurality of media components for each segment, and includes a manifest acquisition unit that acquires a manifest file, and the manifest file. If there are multiple media components that are analyzed and share the same media format and playback time, change the playback start time of each media component so that the playback times of each media component do not overlap . a manifest conversion unit for determining a reproduction offset time indicating the difference, the buffer control unit that determines the URL and segment start time of the segment to be acquired based on the playback start time of the acquisition target on the basis of URL of the segment segment Take Comprising a segment acquisition section that, the, the buffer control unit, the segment acquired by the segment acquisition unit, and stored in the buffer in association with the segment start time, when the switching operation of the media components is performed is the current playback time, based on the new playback start time obtained by adding the reproduction offset time indicating the difference between the media components before and after the switching, characterized that you determine the URL and the segment start time of the segment of the acquisition target .

  Further, the reproduction offset time is longer than the time scale of the content so that the reproduction times of the respective media components do not overlap.

In order to solve the above problem, a playback time control method according to the present invention is a playback time control method in a receiving device that receives content composed of a plurality of media components for each segment, and acquires a manifest file. When the manifest acquisition step and the manifest file are analyzed and there are multiple media components sharing the same media format and playback time, the playback start time of the media component is changed so that the playback times of the media components do not overlap. a manifest conversion step of determining a reproduction offset time indicating the difference between the reproduction start time of each media component, determining a URL and segment start time of the segment of the acquisition target on the basis of the playback start time, the segments It is seen containing a segment acquisition step of acquiring the acquisition target segment based on the URL, and the buffer control step, when the switching operation of the media components is executed, the current playback time, the media components before and after the switching The URL of the segment to be acquired and the segment start time are determined based on a new playback start time obtained by adding a playback offset time indicating a difference .

  In order to solve the above problems, a program according to the present invention causes a computer to function as the receiving device.

  According to the present invention, when a media component is switched, a buffer area different from the media component before switching can be used, and buffering waste can be reduced. In particular, when a media component switching operation is performed at a short interval, the chance of re-acquiring a previously acquired segment is reduced, so that the network usage efficiency is improved and stable viewing is possible.

It is a block diagram which shows the structural example of the content delivery system which concerns on one Embodiment of this invention. It is a figure which shows an example of the manifest file which the receiver which concerns on one Embodiment of this invention acquires. An example of the state of the buffer at the time of switching of a media component in the receiving apparatus according to an embodiment of the present invention is shown. It is a flowchart which shows the reproduction time control method of the receiver which concerns on one Embodiment of this invention. It is a figure which shows an example of the conventional adaptive streaming. It is a figure which shows an example of the buffer model of the conventional MSE. It is a figure which shows the example which overwrite-inserts a segment in a buffer in the buffer model of the conventional MSE. It is a figure which shows the concept of the conventional manifest file. It is a figure explaining the conventional subject.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment described below, as an example, the manifest file is in the MPEG-DASH (ISO / IEC 2309-1) MPD (Media Presentation Description) format.

  A receiving apparatus according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram illustrating a configuration example of a receiving apparatus according to an embodiment of the present invention.

  The receiving device 1 is connected to the distribution server 2 via the Internet, receives content composed of a plurality of media components from the distribution server 2 for each segment, and performs streaming reproduction.

  The distribution server 2 is a server that distributes the manifest file or segment of the URL specified by the receiving device 1, and can be a general Web server, for example.

  The receiving apparatus 1 illustrated in FIG. 1 includes a communication I / F 10, a manifest acquisition unit 11, a manifest conversion unit 12, a segment acquisition unit 13, a buffer 14, a buffer control unit 15, a reproduction control unit 16, and a reproduction. A control user I / F unit 17, a playback unit 18, and a display unit 19 are provided.

  The reproduction control user I / F unit 17 outputs a reproduction command and a stop command to the reproduction control unit 16 by a user operation. The reproduction command includes reproduction start time information indicating the reproduction start time.

  The playback control user I / F unit 17 prepares a playback start / stop button and a media component switching button, and based on the content start time, end time, and current playback time information input from the playback control unit 16. In addition, the reproduction time (reproduction position) for current display may be graphically displayed. For example, at the bottom of the playback screen, the content start time is displayed as a horizontally long bar with the left end and end time at the right end, and a knob-shaped operation part is placed so as to overlap with it. By moving from the left end to the right end, it is possible to grasp at a glance which part of the entire reproduction time is being reproduced.

  The manifest acquisition unit 11 acquires a manifest file (MPD file) of a desired program from the distribution server 2 via the communication I / F 10 when the application is started or a program is selected, and outputs the manifest file to the manifest conversion unit 12.

  The manifest conversion unit 12 analyzes the manifest file input from the manifest acquisition unit 11, and when there are a plurality of media components that share the same media format and playback time, the playback time (time axis) of each media component overlaps. The playback start time is converted so as not to occur, and a segment URL list in which the segment start time corresponding to the start time of the segment and the URL of the segment are associated with each other is generated and output to the buffer control unit 15. In addition, the manifest conversion unit 12 generates a playback offset time list that is a list in which a media identifier that is an identifier of each media component and a playback offset time that indicates a difference in playback start time of each media component are associated with each other. Output to the control unit 16. The playback offset time is preferably set to be equal to or longer than the content time scale so that the playback times of the media components do not overlap.

  FIG. 2 shows an example (extract) of a manifest file. FIG. 2A shows a manifest file before conversion. In this example, there is one content (Period) with a content start time (start) of 0 seconds and a content length (duration) of 3 minutes, and the media has the same media format (mimeType = "video / mp4") There are two media components (AdaptationSet) (id = "main": main video, id = "sub": different angle video). Furthermore, there is one Representation for each AdaptationSet, and in this example, two AdaptationSets share the playback time.

  The manifest conversion unit 12 performs conversion as an example of a converted manifest file shown in FIG. 2B in order to prevent the reproduction times of the AdaptationSet from overlapping. In the converted manifest file, lines 109 and 110 are added to the manifest file before conversion. By arranging two AdaptationSets in a Period with different playback times, overlapping of buffer areas can be avoided.

Next, generation of a segment URL list will be described based on the converted manifest file in FIG. From the start value on the 102nd line, the period start time is 0 seconds. From the timescale value on the 104th line, the time scale of the AdaptationSet (the number of scales per second) is 90000. From the duration value in the 106th line, the duration of the segment is 450,000. The start number of the segment is 1 from the value of startNumber on the 106th line. From the above, the segment number for a certain reproduction time t (second) can be calculated by the following equation. The number below the decimal is rounded down.
Number = (t−start) × timescale / duration + startNumber (1)

  Next, the segment URL template “content_v1_ $ Number $ .mp4” is obtained from the media value on line 106 in FIG. 2, and $ Number $ is replaced with the previously calculated Number value to obtain the URL of the segment. Can be obtained. Although the above example has been described based on the first period of FIG. 2, the second period can be calculated in the same manner.

  When a playback command is input from the playback control user I / F 17, the playback control unit 16 extracts the playback start time from the playback command and outputs playback start time information indicating the playback start time to the buffer control unit 15. Then, when the notification indicating that the buffer specified value (for example, 5 seconds) has been exceeded is input from the buffer control unit 15, the reproduction control unit 16 outputs the reproduction start time information to the reproduction unit 18 and instructs the reproduction start. .

  When the reproduction control unit 16 inputs a stop command from the reproduction control user I / F 17, the reproduction control unit 16 instructs the reproduction unit 18 to stop.

  Further, the playback control unit 16 outputs the current playback state (playing and stopped) and the current playback time information input from the playback unit 18 to the playback control user I / F 17.

  The buffer control unit 15 determines the segment start time of the acquisition target segment based on the reproduction start time information input from the reproduction control unit 16 and the buffering range (buffered segment range) information input from the buffer 14. . Then, the URL of the segment associated with the segment start time is extracted (determined) from the segment URL list input from the manifest conversion unit 12 and output to the segment acquisition unit 13. Further, when a segment is input from the segment acquisition unit 13, the buffer control unit 15 inserts the segment into the buffer 14.

  The segment acquisition unit 13 acquires a corresponding segment based on the URL of the acquisition target segment input from the buffer control unit 15 and outputs the acquired segment to the buffer control unit 15. Specifically, a segment request (request) including the segment URL is generated and transmitted to the distribution server 2 via the communication I / F 10, and the distribution server 2 responds as a response to the request via the communication I / F 10. Get the segment to perform.

  The buffer 14 has a range from the start time to the end time of all the periods described in the manifest file, and stores the segment input from the buffer control unit 15 in association with the segment start time and the playback duration.

  Further, the buffer 14 outputs the buffering range information to the buffer control unit 15. Here, the buffering range information is information indicating the range in which the content segment is inserted in the buffer 14, and is, for example, information on the segment start time and playback duration of all currently stored segments. .

  Further, the buffer 14 sequentially outputs the segments requested from the reproduction unit 18 to the reproduction unit 18.

  When the playback unit 18 inputs a playback command including playback start time information from the playback control unit 16, the playback unit 18 sequentially inputs segments from the buffer 14 based on the playback start time, and the input segments are converted into the segment media format (video, audio, And the like and output to the display unit 19. The playback unit 18 outputs current playback time information, which is playback time information updated with playback, to the playback control unit 16.

  In addition, when the playback unit 18 inputs a stop command from the playback control unit 16, the playback unit 18 stops playback.

  The display unit 19 displays the decoded media data input from the playback unit 18 according to the format of the media.

  Hereinafter, an operation when a media component switching operation is executed will be described.

  When the playback control unit 16 inputs a media component switching command from the playback control user I / F unit 17, the playback offset time associated with the media identifier after switching is selected from the playback offset time list input from the manifest conversion unit 12. get. Further, the playback control unit 16 acquires the current playback time from the current playback time information input from the playback unit 18. Then, by adding the playback offset time to the current playback time, a new playback start time that is the playback time after switching the media component is calculated. Then, new playback start time information indicating the new playback start time is output to the buffer control unit 15, and when a notification indicating that the buffer specified value has been exceeded is input from the buffer control unit 15, the new playback start time information is input to the playback unit 18. Is output to instruct to change the playback time.

  In order to perform such processing, the new playback start time that is the playback time on the actual buffer 14 is different from the current playback time that is the apparent playback time displayed on the playback control user I / F unit 17. At the time of component switching, the current playback time is the time obtained by subtracting the playback offset time from the new playback start time.

  The buffer control unit 15 determines the segment start time of the acquisition target segment based on the new playback start time information input from the playback control unit 16 and the buffering range information input from the buffer 14. Then, the URL of the segment associated with the segment start time is extracted from the segment URL list input from the manifest acquisition unit 11 and output to the segment acquisition unit 13.

  When the playback unit 18 inputs new playback start time information from the playback control unit 16, the playback unit 18 sequentially inputs segments from the buffer 14 based on the new playback start time information, decodes the input segments in accordance with the segment media format, and displays them. To the unit 19.

  FIG. 3 shows an example of the buffer status at the time of switching the media component. In this example, the manifest acquisition unit 11 acquires a manifest file describing information about one content (Period) composed of two media components (AdaptationSet), and the manifest conversion unit 12 converts the manifest file into each of the manifest files. It is assumed that it has been converted into a manifest file describing information related to two contents (Period) composed of one media component (AdaptationSet). Also, the playback start time (start) of each media component is 300 seconds, the content scale (duration) is 3 minutes, and the playback offset time is the content scale.

  FIG. 3 (a) shows a playback state immediately before a media component switching instruction. One segment of a media component is inserted from 300 seconds, and a segment for 10 seconds is acquired prior to the current playback time. It shows the state of being inserted into the buffer. FIG. 3B shows a state in which after switching the media component, the segment of the other media component is inserted from 495 seconds so that the buffer areas of the media components do not overlap. As described above, in the present invention, when the media component is switched, a buffer area different from the buffer area used by the media component before the switching is used.

(Playback time control method)
Next, a playback time control method during a media component switching operation will be described using the flowchart shown in FIG.

  In the flowchart shown in FIG. 4, it is assumed that the manifest file has already been acquired and the segment URL list after the manifest conversion process has been created. In addition, in anticipation that the reception band constantly fluctuates depending on the network congestion situation, switching is performed after a segment having a buffer specified value (for example, 5 seconds) or more set in advance is inserted into the buffer 14 starting from the reproduction start time. It is assumed that later media component playback starts. The display playback time range is a range from the start time to the end time of the Period described in the manifest file received from the distribution server 2, which is the original time scale of the content (before conversion of the manifest file).

  First, when a media component switching operation is performed in the playback control user I / F unit 17 (step S001), the playback control unit 16 reads the media after switching from the playback offset time list input from the manifest conversion unit 12. An identifier is extracted (step S002). Then, the playback control unit 16 calculates the new playback start time by adding the playback offset time associated with the media identifier after switching to the playback time for display (step S003), and outputs it to the buffer control unit 15.

  Next, the buffer control unit 15 confirms whether or not there is a segment in the buffer 14 that is equal to or greater than a predetermined buffer specified value from the new playback start time (step S004). If YES in step S004, the playback time is changed to a new playback start time (step S009). Then, playback of the media component after switching is started (step S010), and the process ends.

  On the other hand, in the case of NO in step S004, the buffer control unit 15 obtains the URL of the segment following the last segment in the buffering range including the new reproduction start time from the segment URL list input from the manifest conversion unit 12. Extract (step S005) and output to the segment acquisition unit 13. Then, the segment acquisition unit 13 transmits a request including the extracted URL of the segment to the distribution server 2, receives the segment corresponding to the request (step S006), and inserts the received segment into the buffer 14 (step S007). ).

  Next, the playback control unit 16 confirms whether the display playback time has reached the end time of the content or whether the user I / F has pressed the stop button (step S008). If step S008 is NO, the process returns to step S004. If step S008 is YES, the process ends.

  As described above, the receiving apparatus 1 according to the present invention is configured to display content including a main video, a video at a different angle (multi-angle) associated with the main video, and a video at a different playback speed (fast forward / rewind). When performing streaming playback, the playback offset time of each media component is set so that the buffer areas of each media component do not overlap. Therefore, when the media component is switched, it is possible to use a buffer area different from the media component before the switching, and buffering waste can be reduced. In particular, when media components are switched at short intervals, the chance of reacquisition of previously acquired segments is reduced, so that the network usage efficiency is improved and stable viewing is possible.

  Note that a computer can be suitably used to cause the above-described function to function as the receiving device 1, and such a computer stores a program describing processing contents for realizing each function of the receiving device 1 in a storage unit of the computer. It can be realized by reading and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks or processing steps described in the embodiments can be combined into one, or one can be divided into a plurality.

1 Receiving Device 2 Distribution Server 10 Communication I / F
DESCRIPTION OF SYMBOLS 11 Manifest acquisition part 12 Manifest conversion part 13 Segment acquisition part 14 Buffer 15 Buffer control part 16 Playback control part 17 Playback control user I / F
18 Playback section 19 Display section

Claims (4)

A receiving device that receives content composed of a plurality of media components for each segment,
A manifest acquisition unit for acquiring a manifest file;
When there are multiple media components that analyze the manifest file and share the same media format and playback time, the playback start time of the media component is changed so that the playback times of the media components do not overlap , A manifest conversion unit that determines a playback offset time indicating a difference in playback start time;
A buffer controller for determining a URL and segment start time of the segment of the acquisition target on the basis of the playback start time,
A segment acquisition unit that acquires the segment to be acquired based on the URL of the segment ,
The buffer control unit stores the segment acquired by the segment acquisition unit in a buffer in association with the segment start time,
When a media component switching operation is executed, based on a new playback start time obtained by adding a playback offset time indicating a difference between media components before and after switching to the current playback time, the URL of the segment to be acquired and the segment receiving apparatus characterized that you determine the start time.   The receiving apparatus according to claim 1, wherein the playback offset time is equal to or longer than a time scale of content so that playback times of the respective media components do not overlap. A playback time control method in a receiving device for receiving content composed of a plurality of media components for each segment,
A manifest acquisition step for acquiring a manifest file;
When there are multiple media components that analyze the manifest file and share the same media format and playback time, the playback start time of the media component is changed so that the playback times of the media components do not overlap , A manifest conversion step for determining a playback offset time indicating a difference in playback start time;
A buffer control step of determining the URL and segment start time of the segment of the acquisition target on the basis of the playback start time,
A segment acquisition step of acquiring the segment to be acquired based on the URL of the segment;
Said segments obtained by said segment acquisition step, seen including a storage step of storing in a buffer, the in association with the segment start time,
The buffer control step, when a media component switching operation is performed, is based on a new playback start time obtained by adding a playback offset time indicating a difference between media components before and after switching to the current playback time. A playback time control method, wherein a URL of a segment and the segment start time are determined . A program for causing a computer to function as a receiving device according to claim 1 or 2.

Images