JP6535273B2 - Receiving device, segment acquisition method, and program - Google Patents

Description

  The present invention relates to a receiving apparatus that receives live streaming, a segment acquisition method, and a program.

  In the streaming video delivery on the Internet in recent years, the transition from a streaming delivery method using a dedicated server and a dedicated protocol to a streaming delivery method using an HTTP protocol by a general-purpose Web server is progressing, and for many devices It is mainstream in delivery. As streaming delivery methods (adaptive streaming) using such HTTP protocol, proprietary techniques by IT vendors are widespread, and MPEG-DASH (ISO / IEC 23009-1), which is an international standard intended to unify these streaming methods. ) Was formulated.

  The basic concept of each technology is the same, and the Web server is divided into a few seconds to several tens of seconds each of a stream encoded with moving image content at one or more qualities (screen size and bit rate). Prepare a manifest file in which objects (segments) and attributes and URLs of those moving image contents are described. The receiving device selects the timely quality from the manifest file in consideration of the screen size of the receiving device and the network bandwidth of the transmission path, etc., receives the segments one after another, joins them into one moving image content, and reproduces them. (See, for example, Non-Patent Document 1).

  An example of adaptive streaming is shown in FIG. In FIG. 5, the video is encoded with multiple qualities, and segments divided into units of several seconds are generated for each quality. In this example, the encoder divides the video into low quality segments 1-4, medium quality segments 1'-4 ', and high quality segments 1' '-4' '. In the manifest file, attribute information such as the URL of the segment and the bit rate and image size for each quality are registered. Segment and manifest files are provided to the web server.

  The receiving device receives the manifest file from the Web server, grasps the bit rate and the segment configuration from the manifest file (Step A), and quality on a segment basis based on the screen size of the receiving device and the status of the network reception bandwidth. Are selected and received by the HTTP protocol (step B). Subsequently, the received segments are connected in order according to the manifest file and reproduced (step C). By repeating the processes of steps B and C in this way, it is possible to receive a series of contents and continue reproduction.

  In the receiving apparatus, by using the HTML 5 browser, which is widely used in many devices, it is possible to construct a viewing environment compatible with multi-devices (PC, television receiver, tablet device, etc.).

  However, since video elements (video tags), which are a function to play moving pictures in HTML5 browser, can only refer to media files as sources, to realize such adaptive streaming viewing, however, It was necessary to install a browser plug-in separately. Therefore, it was difficult to construct a viewing environment that can be used commonly by any browser. Under such circumstances, an international non-profit organization promoting standardization of technologies used on the Web as a function for realizing adaptive streaming viewing by HTML5 browser video elements (video tags) and Javascript (registered trademark) The standardization of MSE (Media Source Extensions) is progressing in W3C, which is (see, for example, Non-Patent Document 2).

  By using the MSE, the video element can reference the buffer as a source. Since it becomes possible to insert segments received by Javascript into the buffer, various adaptive streaming viewing can be realized by HTML5 video elements and Javascript.

  Here, the buffer model of the MSE will be described with reference to FIG. FIG. 6 is a diagram showing an example of the MSE buffer model. A buffer (SourceBuffer) manages segments in association with information on segment start time and playback duration time. In FIG. 6, shaded blocks indicate segments, and the segment start time, which is the segment unit playback start time, is 0 seconds, the playback duration is 5 seconds, the segment start time is 5 seconds, and the playback duration is 5 Three segments of second and segment start time of 15 seconds and reproduction duration of 5 seconds are inserted in the buffer. At this time, the buffering range (buffered attribute) of the buffer is 0 to 10 seconds and 15 to 20 seconds.

  Generally, when watching streaming, the reception bandwidth always fluctuates according to the congestion status of the network, so as long as the buffer capacity allows, data from several seconds to tens of seconds ahead of the current playback time is received and stored in the buffer. By inserting it, the viewing stability can be improved.

  In the case of content viewing, in addition to a viewing method of viewing through the content from the beginning to the end, convenience can be improved by enabling a seek operation to specify and play an arbitrary position of the content during viewing.

  In addition, a video display device, a buffer management method, and a video display system are known that enable a seemingly seamless switching during the waiting time that occurs when the user specifies an arbitrary position and performs playback in streaming playback. (See, for example, Patent Document 1).

JP, 2013-16920, A

“Technical overview and standardization and related technology trends of next-generation video distribution technology“ MPEG-DASH ”, Journal of the Institute of Image Information and Television Engineers, Vol. 67, No. 2, 2013, p. 109-115 W3C, "Media Source Extensions W3C Candidate Recommendation 17 July 2014", [Search on November 6, 2015], Internet <URL: http://www.w3.org/TR/media-source/>

  In the case of streaming viewing using MSE, the seek operation can be performed by setting the playback time after seek in the currentTime property of the video element and acquiring a segment including the set playback time. An example of the state of the buffer of a general receiving apparatus at this time is shown in FIG.

  FIG. 7 shows an example in which the reproduction continuation time of the video segment is 10 seconds, the reproduction continuation time of the audio segment is 8 seconds, and the new reproduction start time after the seek is 18 seconds. The shaded blocks indicate segments. In this case, the video acquires segment a (the segment start time is 10 seconds) including the playback time 18 seconds, and the audio acquires the segment b (the segment start time is 16 seconds) including the playback time 18 seconds Can be started.

  However, depending on the receiving apparatus, although the new reproduction start time after the seek is as the set value, there is one that starts reproduction from the segment start time which is the head of the reproduction video segment. In order to decode the encoded video signal, it is necessary to decode from an I frame (intra frame), and the segment head is an I frame. In such a receiving apparatus, reproduction is attempted from the position of 10 seconds which is the head of the video segment a in FIG. 7, but there is no segment at the position of 10 seconds in the audio buffer. There is a problem that it is necessary to separately acquire the segment of and it is not possible to immediately start the reproduction.

  Further, according to Patent Document 1, when data is stored in the buffer, when the buffer capacity is full, old data is preferentially erased from the reproduced data, or the reproduced data remains in the buffer. If not, it has been described that the new data is preferentially erased from the data remaining in the buffer before arbitrary position designation, but it is not a technique that can solve the above-mentioned problem.

  An object of the present invention made in view of such circumstances is to provide a receiving apparatus, a segment acquiring method, and a program capable of stably performing a seek operation in streaming reproduction.

  In order to solve the above problems, a receiving apparatus according to the present invention is a receiving apparatus in which video and audio are separated into separate streams, and each of which receives content composed of a plurality of segments for each segment, the manifest A manifest acquisition unit that generates a segment URL list in which a segment start time corresponding to a segment start time and a segment URL are associated based on a file, a reproduction control unit that acquires a reproduction start time, and the reproduction time A buffer control unit that determines the segment start time of the segment to be acquired and extracts the URL of the segment associated with the segment start time from the segment URL list; and the acquisition target of the segment based on the URL of the segment A segment acquisition unit for acquiring segments, and When the playback operation is executed, the reproduction control unit acquires new reproduction start time information indicating a new reproduction start time, and the buffer control unit acquires the video segment including the new reproduction start time from the segment URL list. A URL and a segment start time of the video segment are extracted, and a URL of an audio segment including the segment start time is extracted from the segment URL list.

  Furthermore, in order to solve the above problems, the segment acquisition method according to the present invention is a segment acquisition method in which video and audio are separate streams, and each content is obtained from a plurality of segments. A manifest acquisition step of generating a segment URL list in which the segment start time corresponding to the start time of the segment is associated with the URL of the segment based on the manifest file, and a reproduction control step of acquiring the reproduction start time A buffer control step of determining a segment start time of a segment to be acquired based on the reproduction time, and extracting a URL of a segment associated with the segment start time from the segment URL list; and a URL of the segment Segment to be acquired A segment acquiring step of acquiring a new reproduction start time information indicating a new reproduction start time when the seek operation is executed; and the buffer control step includes the segment URL A video segment URL including the new reproduction start time and a segment start time of the video segment are extracted from a list, and an audio segment URL including the segment start time is extracted from the segment URL list. .

  Further, in order to solve the above-mentioned problems, a program according to the present invention is characterized in that a computer is caused to function as the above-mentioned receiving device.

  According to the present invention, an audio segment including the segment start time of the video segment including the new reproduction start time can be quickly even in the reception starting from the segment start time of the video segment including the new reproduction start time after seek. It can be acquired. Therefore, the time until the reproduction start after the seek operation can be shortened and the reproduction can be performed stably.

It is a figure which shows the structural example of the receiver which concerns on one Embodiment of this invention. It is a figure which shows an example (extract) of a manifest file. It is a figure which shows an example of the condition of the buffer at the time of the seek operation of the receiver concerning one embodiment of the present invention. It is a flowchart which shows the segment acquisition method at the time of seek operation of the receiver concerning one Embodiment of this invention. It is a figure which shows an example of the conventional adaptive streaming. It is a figure which shows an example of the buffer model of the conventional MSE. In the conventional receiving device, it is a figure showing an example of the situation of the buffer at the time of seek operation.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In the following description, of the reproduction times, the time to start reproduction is referred to as “reproduction start time”, and the time to start reproduction on a segment basis is referred to as “segment start time”.

  First, the manifest file prepared in the distribution server will be described. In this embodiment, as an example, the manifest file is in the format of MPD (Media Presentation Description) of MPEG-DASH (ISO / IEC 23009-1), and the segment is in the format of ISOBMFF (ISO / IEC 14496-12).

  FIG. 2 shows an example (excerpt) of the manifest file. The MPD format has a hierarchical structure in the order of Period, AdaptationSet, and Representation.

  Period indicates a section in which a program is divided in a time direction (one or more). Period consists of one or more AdaptationSets.

  AdaptationSet indicates information of media components (video, audio, text, etc.). Period shown in FIG. 2 includes AdaptationSet (3rd to 8th lines) indicating video information and AdaptationSet (9th to 12th lines) indicating audio information. In addition, AdaptationSet can include multiple components with different parameters (bit rate, screen size, number of audio channels, etc.) in the same medium. In the example of FIG. 2, the number of video and audio media components is one each.

  Representation indicates information of each media component included in AdaptationSet, and is composed of one or more segments.

The URL of the segment at a certain playback time can be obtained from MPD information. It demonstrates based on the example of FIG. From the value of start on the second line, the start time of Period is 0, and from the value of duration, the content scale is 3 minutes. From the value of timescale on the fourth line, the time scale of AdaptationSet (the number of graduations per second) is 90000. From the value of duration on line 6, the duration of the segment is 900000. From the value of startNumber on the sixth line, the segment start number is 1. From the above, the segment number for a certain reproduction time t (seconds) can be calculated by the following equation (1). In addition, we cut off less than a few.
Number = (t-start) x timescale / duration + startNumber (1)

  Next, the segment URL template "video_ $ Number $ .mp4" is acquired from the value of media on the sixth line, and the segment URL is acquired by replacing $ Number $ with the previously calculated Number value. be able to. Although the above described the video (the third to eighth lines), it is possible to obtain the same for the audio (the ninth to twelfth lines).

(Configuration of receiver)
Next, an exemplary configuration of a receiving apparatus according to an embodiment of the present invention will be described with reference to FIG.

  The receiving device 1 is connected to the distribution server 2 via the Internet, receives content from the distribution server 2 for each segment, and performs streaming reproduction. This content is a separate stream of video and audio, each consisting of multiple segments. That is, the content is composed of a plurality of video segments and a plurality of audio segments.

  The distribution server 2 is a server that distributes the manifest file and the segment of the URL designated by the receiving device 1, and can be, for example, a general web server.

  The receiver 1 shown in FIG. 1 includes a communication I / F 10, a manifest acquisition unit 11, a segment acquisition unit 12, a buffer 13, a buffer control unit 14, a reproduction control unit 15, and a reproduction control user I / F 16. , Playback unit 17 and display unit 18.

  The reproduction control user I / F 16 outputs a reproduction instruction, a stop instruction, and a seek instruction (reproduction time change instruction) to the reproduction control unit 15 by user operation. The reproduction instruction includes reproduction start time information indicating a reproduction start time, and the seek instruction includes new reproduction start time information indicating a new (after change) reproduction start time.

  The playback control user I / F 16 prepares the playback start / stop button, and based on the content start / stop time and the current playback time information input from the playback control unit 15, the current playback time (playback position ) May be displayed graphically. For example, a horizontally long bar is displayed on the lower part of the playback screen with the start time of the content at the left end and the end time at the right end, and the knob-like operation part is arranged to overlap it. By moving from the left end to the right end, the user can grasp at a glance which part of the entire reproduction time is currently being reproduced. In addition, by moving this knob by user operation with a mouse or the like, an arbitrary reproduction time may be designated.

  The manifest acquisition unit 11 acquires a manifest file (MPD) of desired content from the distribution server 2 via the communication I / F 10 at the time of application activation or content selection, and based on the manifest file, segment start time and segment A segment URL list associated with the URL is generated and output to the segment acquisition unit 12 and the buffer control unit 14.

  When the reproduction control unit 15 receives a stop command from the reproduction control user I / F 16, the reproduction control unit 15 instructs the reproduction unit 17 to stop.

  Also, when the reproduction control unit 15 receives a reproduction instruction from the reproduction control user I / F 16, the reproduction control unit 15 extracts reproduction start time information from the reproduction instruction and outputs the reproduction start time information to the buffer control unit 14. Then, when the reproduction control unit 15 receives a notification indicating that the buffer regulation value (for example, 5 seconds) is exceeded from the buffer control unit 14, the reproduction control unit 15 outputs reproduction start time information to the reproduction unit 17 and instructs reproduction start. .

  Further, the reproduction control unit 15 outputs the current reproduction state (during reproduction and stop) and the current reproduction time information input from the reproduction unit 17 to the reproduction control user I / F 16.

  The buffer control unit 14 acquires segments (video segment and audio segment) to be acquired based on the reproduction start time information input from the reproduction control unit 15 and the buffering range (buffered segment range) information input from the buffer 13. Determine the segment start time of Then, the URL of the segment associated with the segment start time is extracted from the segment URL list, and is output to the segment acquisition unit 12.

  The segment acquiring unit 12 inputs the URL of the segment to be acquired from the buffer control unit 14. Then, based on the URL of the segment, the corresponding segment is acquired and output to the buffer control unit 14. Specifically, a segment request (request) including the URL of the segment is generated and transmitted to the distribution server 2 via the communication I / F 10, and the distribution server 2 responds as a response to the request via the communication I / F 10 Get the segment to The segment acquisition unit 12 may input the segment start time from the buffer control unit 14 and extract the URL of the segment associated with the segment start time from the segment URL list.

  The buffer 13 receives a segment from the buffer control unit 14, associates the segment start time and the playback duration time of the segment, and stores the video segment and the audio segment separately.

  Also, the buffer 13 outputs the buffering range information to the buffer control unit 14. Here, the buffering range information is information indicating a range in which the segment of the content is inserted in the buffer 13, and is, for example, information of segment start time and reproduction duration time of all currently stored segments. .

  Also, the buffer 13 sequentially outputs the segments requested from the reproduction unit 17 to the reproduction unit 17.

  When the reproduction unit 17 receives a stop command from the reproduction control unit 15, the reproduction unit 17 stops reproduction.

  When the reproduction unit 17 receives a reproduction instruction from the reproduction control unit 15, the reproduction unit 17 sequentially inputs segments from the buffer 13 based on the reproduction start time information, and the input segment is in the format of the segment media (video, audio, text, etc. According to the above, the signal is decoded and output to the display unit 18. The reproduction unit 17 outputs, to the reproduction control unit 15, current reproduction time information, which is information on the reproduction time updated with the reproduction.

  The display unit 18 displays the decoded media data input from the playback unit 17 in accordance with the format of the media.

  The following describes the case where a seek operation is performed.

  When the seek control unit 15 receives a seek command from the play control user I / F 16, the play control unit 15 extracts new play start time information from the seek command and outputs the information to the buffer control unit 14. Then, when the reproduction control unit 15 receives a notification indicating that the buffer specified value (for example, 5 seconds) is exceeded from the buffer control unit 14, the reproduction control unit 15 outputs new reproduction start time information to the reproduction unit 17 and instructs reproduction start. Do.

  When the reproduction unit 17 receives new reproduction start time information from the reproduction control unit 15, the reproduction unit 17 sequentially inputs segments from the buffer 13 based on the new reproduction start time information, and the input segment has the format of the segment media (video, Decoding is performed according to voice, text, etc., and output to the display unit 18.

  When the new reproduction start time information is input from the reproduction control unit 15, the buffer control unit 14 receives the URL of the video segment including the new reproduction start time from the segment URL list and the segment start time corresponding to the start time of the video segment. Extract Then, the URL of the audio segment including the segment start time of the video segment is extracted from the segment URL list, and is output to the segment acquisition unit 12.

  The segment acquiring unit 12 inputs the URL of the segment to be acquired (video segment and audio segment) from the buffer control unit 14. Then, based on the URL of the segment, the corresponding segment is acquired and output to the buffer control unit 14.

  An example of the state of the buffer at the time of seek operation is shown in FIG. Similar to FIG. 7, the video segment playback duration time is 10 seconds, the audio segment playback duration time is 8 seconds, and the new playback start time after seek is 18 seconds. The video acquires a segment a (the segment start time is 10 seconds) including the new reproduction start time 18 seconds. On the other hand, the audio is not the new reproduction start time, and a segment c (segment start time is 8 seconds) including the segment start time 10 seconds of the video segment is acquired. Therefore, the video and audio can be immediately started from 10 seconds of the segment start time of the video segment.

(Segment acquisition method during seek operation)
Next, a segment acquisition method at the time of seek operation of the receiving device 1 will be described with reference to the flowchart shown in FIG.

  In the flowchart illustrated in FIG. 4, it is assumed that the manifest file has already been acquired, and the segment URL list also starts from the created state. Also, in anticipation of the reception band constantly fluctuating according to the congestion status of the network, the reception device 1 inserts into the buffer 13 a segment equal to or larger than a buffer specified value (for example, 5 seconds) preset from the reproduction start time. It is assumed that playback is started after it is done.

  First, when a seek operation is performed by the user in the reproduction control user I / F 16 (step S001), the reproduction control unit 15 extracts new reproduction start time information from the seek command and outputs it to the buffer control unit 14 (step S002). ).

  Next, the buffer control unit 14 acquires a new reproduction start time from the new reproduction start time information. Then, the buffer control unit 14 extracts the URL of the video segment including the new reproduction start time and the segment start time corresponding to the start time of the video segment from the segment URL list (step S003), and the extracted video segment The URL of the voice segment including the segment start time is extracted from the segment URL list (step S 004).

  Next, the segment acquisition unit 12 generates a segment request including the URL of the video segment extracted in step S 003 and the URL of the audio segment extracted in step S 004, and transmits the segment request to the distribution server 2. A segment corresponding to the request is received from the buffer control unit 14 and output to the buffer control unit 14 (step S 005). Then, the buffer control unit 14 inserts the video segment and the audio segment received in step S005 into the buffer 13 (step S006).

  Next, the buffer control unit 14 confirms whether or not there is a segment equal to or larger than the buffer specified value from the new reproduction start time in each of the video and audio buffers 13 (step S 007). If it exists, reproduction is started by the reproduction unit 17 (step S010), and the process is ended.

  On the other hand, if step S 007 is NO, the buffer control unit 14 extracts the URL of the segment following the segment including the maximum value of the buffering range from the segment URL list (step S 008). Then, the segment acquisition unit 12 transmits a segment request including the URL of the extracted segment to the distribution server 2, receives the segment corresponding to the request (step S008), and inserts the received segment into the buffer 13. (Step S009).

  As described above, according to the receiving device 1 of the present invention, even if the specification starts playback starting from the segment start time of the video segment including the new playback start time after the seek, the video segment including the new playback start time It is possible to quickly obtain an audio segment including the segment start time. Therefore, the time until the start of reproduction after the seek operation can be shortened and the reproduction can be performed stably.

  Note that a computer can be suitably used to function as the above-described receiving device 1, and such a computer stores a program in which the processing content for realizing each function of the receiving device 1 is described in the storage unit of the computer. This can be realized by reading and executing this program by the CPU of the computer. This program can be recorded on a computer readable recording medium.

  Although the embodiments described above have been described as representative examples, it will be obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes are possible without departing from the scope of the claims. For example, with respect to the configuration blocks or processing steps described in the embodiments, it is possible to combine a plurality into one or divide one into a plurality.

1 Receiver 2 Distribution Server 10 Communication I / F
11 Manifest Acquisition Unit 12 Segment Acquisition Unit 13 Buffer 14 Buffer Control Unit 15 Reproduction Control Unit 16 Reproduction Control User I / F
17 playback unit 18 display unit

Claims (3)

A receiver that receives video and audio in separate streams and receives content composed of a plurality of segments for each segment.
A manifest acquisition unit that generates a segment URL list in which the segment start time corresponding to the start time of the segment and the URL of the segment are associated based on the manifest file;
A playback control unit that obtains a playback start time;
A buffer control unit which determines a segment start time of a segment to be acquired based on the reproduction start time, and extracts a URL of a segment associated with the segment start time from the segment URL list;
A segment acquisition unit for acquiring the segment to be acquired based on the URL of the segment;
When a seek operation is performed, the reproduction control unit acquires new reproduction start time information indicating a new reproduction start time, and the buffer control unit acquires a video segment including the new reproduction start time from the segment URL list. A receiving apparatus, which extracts a URL and a segment start time of the video segment, and extracts a URL of an audio segment including the segment start time from the segment URL list. A segment acquisition method in which video and audio are in separate streams, and each content is composed of a plurality of segments and acquired for each segment.
A manifest acquisition step of generating a segment URL list in which the segment start time corresponding to the start time of the segment and the URL of the segment are associated based on the manifest file;
A reproduction control step of acquiring a reproduction start time;
A buffer control step of determining a segment start time of a segment to be acquired based on the reproduction start time, and extracting a URL of a segment associated with the segment start time from the segment URL list;
Obtaining a segment to be obtained based on the URL of the segment;
When a seek operation is performed, the reproduction control step acquires new reproduction start time information indicating a new reproduction start time, and the buffer control step generates a video segment including the new reproduction start time from the segment URL list. A segment acquisition method characterized by extracting a URL and a segment start time of the video segment and extracting a URL of an audio segment including the segment start time from the segment URL list.   A program for causing a computer to function as the receiving device according to claim 1.

Images