JP5552650B2 - Electronic device and reproduction control method - Google Patents

Description

  Embodiments described herein relate generally to an electronic device capable of reproducing content data and a reproduction control method.

  In recent years, various electronic devices such as personal computers, PDAs, and smartphones have been developed. Many electronic devices of this type have a streaming reproduction function for reproducing multimedia content data while receiving the multimedia content data via the Internet.

  In streaming reproduction of multimedia content data, a part of the multimedia content data received from the content server is stored in a stream buffer in the electronic device.

Japanese Patent Laid-Open No. 10-224749

  The player software executed on the electronic device displays a user interface called a seek bar (or progressive bar). The user can change the playback position in the multimedia content data by operating the seek bar.

  When a data position that is not stored in the buffer is designated as the playback position, the player software starts obtaining the corresponding data portion from the content server, and stores the data portion in the buffer (seek). This buffering takes a certain time. Accordingly, a certain waiting time is required until reproduction from the designated data position is started due to a time lag from when a new data portion is requested until the data portion is stored in the buffer.

  In general, in multimedia content data in a file format for streaming, a video data portion and an audio data portion whose reproduction times are close to each other are continuously arranged.

  However, there is multimedia content data in a file format that is not intended for streaming. In such multimedia content data, there is a possibility that a video data portion and an audio data portion that are close to each other and to be played back are discontinuously arranged. In such seeks for multimedia content data, there is a possibility that the response of seek will be reduced by the process for receiving discrete data portions from the content server, that is, there is a time lag for buffering these data portions. It can grow. Therefore, even if the reproduction of the multimedia content data from the designated data position is once resumed, the reproduction may be immediately stopped again. This is because there is a possibility of waiting time for buffering another data portion to be reproduced following the data portion at the designated data position.

  An object of the present invention is to provide an electronic device and a content playback control method capable of smoothly playing back multimedia content data.

According to the embodiment, the electronic device reproduces the multimedia content data in which the video data and the audio data are multiplexed while receiving it from the server. The electronic device includes a buffer, a reproduction unit, and a control unit. The buffer stores the received multimedia content data. The reproduction means reproduces a video data portion and an audio data portion in the multimedia content data stored in the buffer. The control means causes the reproduction means to stop the reproduction when the reproduction position is changed to a first data position in the multimedia content data that is not yet stored in the buffer, and the multimedia content A first buffering process for sequentially receiving each data portion following the first data position in the data from the server and storing it in the buffer is started. The control means causes the reproduction means to restart the reproduction from the first data position after the first buffering process is started, and after the reproduction is resumed, the control means not stored in the buffer. When a video data part in media content data is requested by the playback means, the first buffering process is stopped, and each data part subsequent to the requested video data part is received from the server, and Start a new buffering process to store in the buffer. The control means continues the execution of the first buffering process when an audio data part in the multimedia content data not stored in the buffer is requested by the reproduction means after the reproduction is resumed. In addition, the reproduction means skips the reproduction of the requested audio data part.

FIG. 2 is a perspective view illustrating an appearance of the electronic apparatus according to the embodiment. 4 is an exemplary view showing an example of a content playback screen displayed by player software executed by the electronic apparatus of the embodiment. FIG. 2 is an exemplary block diagram showing the system configuration of the electronic apparatus of the embodiment. FIG. 2 is an exemplary block diagram showing the configuration of player software executed by the electronic apparatus of the embodiment. FIG. FIG. 5 is a diagram for explaining a buffer management operation executed by the player software of FIG. 4. The figure for demonstrating the seek control operation performed by the player software of FIG. The figure which shows the example of arrangement | positioning of the audio data part and video data part in the multimedia content data for streaming. The figure which shows the example of arrangement | positioning of the audio data part and video data part in the multimedia content data which are not for streaming. The figure which shows the other example of arrangement | positioning of the audio data part and video data part in the multimedia content data which are not for streaming. The figure for demonstrating the phenomenon in which the response of a seek is reduced. The figure for demonstrating the example of buffer release control operation | movement. The figure for demonstrating the request | requirement with respect to the data which existed in the released buffer area which may generate | occur | produce when using the buffer release control operation | movement of FIG. FIG. 5 is a diagram for explaining a buffer release control operation executed by the player software of FIG. 4. The figure for demonstrating the reproduction | regeneration stop operation | movement performed when the amount of buffered data decreases. The figure for demonstrating a mode that reproduction | regeneration is stopped suddenly. The figure for demonstrating the reproduction | regeneration stop operation | movement performed by the player software of FIG. 5 is a flowchart showing a procedure of a reproduction control operation executed by the player software of FIG. 6 is a flowchart for explaining a procedure of a seek control operation executed by the player software of FIG. 6 is a flowchart for explaining a procedure of a buffer release operation executed by the player software of FIG. 6 is a flowchart for explaining a procedure executed by the player software of FIG. 4 when the amount of buffered data is reduced.

Hereinafter, embodiments will be described with reference to the drawings.
FIG. 1 is a perspective view illustrating an external appearance of an electronic apparatus according to an embodiment. This electronic device can be realized as, for example, a slate personal computer (PC), a laptop PC, a smartphone, or a PDA. In the following, it is assumed that the electronic device is realized as a slate type personal computer 10. As shown in FIG. 1, the slate type personal computer 10 includes a computer main body 11 and a touch screen display 17.

  The computer main body 11 has a thin box-shaped housing. The touch screen display 17 incorporates a liquid crystal display (LCD) and a touch panel. The touch panel is provided so as to cover the screen of the LCD. The touch screen display 17 is attached to be superposed on the upper surface of the computer main body 11.

  The computer 10 has a content reproduction function for reproducing multimedia content data by streaming. This content reproduction function is executed by player software installed in the computer 10. The content playback operation by the player software may be executed using a multimedia framework provided by the operating system. The computer 10 has a built-in communication device that executes communication with a wired or wireless network, and communication between the computer 10 and the Internet 20 is executed via this communication device.

  Multimedia content data is a data stream in which video data and audio data are multiplexed. The video data may be compression encoded, and the audio data may be compression encoded.

  The player software receives multimedia content data from the content server 21 via a network such as the Internet 20. The player software reproduces the multimedia content data while receiving the multimedia content data. In the computer 10, the received data portion in the multimedia content data is stored in a buffer. The buffer is a part of the memory in the computer 10. After a certain amount of data is stored in the buffer, the player software starts playing the multimedia content data.

  In the reproduction processing of multimedia content data, each of the video data portion and the audio data portion stored in the buffer is reproduced. During the multimedia content data playback process, the player software may perform a prefetch process. The prefetching process is a buffering process in which each data part after the current playback position is sequentially received from the content server 21 and stored in the buffer.

  FIG. 2 is an example of a content playback screen displayed on the display 17 by the player software. The content playback screen displays a video display area 31 and a seek bar 33. The video display area 31 is a display area for displaying video data images (moving images) in the multimedia content data. The seek bar 33 is a graphical user interface that allows the user to control the playback position of the multimedia content data.

  The left end and the right end of the seek bar 33 correspond to the start position and the end position of the multimedia content data to be reproduced, respectively. During the playback period of the multimedia content data, the playback position of the multimedia content data is automatically updated as time passes. As a value indicating the data position in the multimedia content data such as the playback position, an offset value starting from the head position of the multimedia content data may be used. The offset value corresponding to a certain data position indicates the data size from the top position of the multimedia content data to the data position.

  The seek bar 33 displays a slider 35. A slider 35 indicates the current playback position in the multimedia content data. That is, the slider 35 indicates the progress of the reproduction of the multimedia content data. The user can change the playback position in the multimedia content data by moving the slider 35 using the pointing device.

  The seek bar 33 further displays the reproduced section 37 and the seek-enabled section 39 on the seek bar 33. The reproduced section 37 indicates a data section that has been reproduced and the data portion remains in the buffer in the multimedia content data sequence. The seekable section 39 indicates a data section in the multimedia content data sequence that has not been played back and whose data portion is already stored in the buffer. The above-described prefetching process (buffering process) may be performed so that the seekable section 39 is as long as possible for a certain time.

  When the slider 35 is moved to an arbitrary position in the seekable section 39, the reproduction position is immediately changed to the position in the seekable section 39 indicated by the slider 35. When the slider 35 is moved to an area outside the seekable section 39, that is, when the playback position is changed to the first data position in the multimedia content data that is not yet stored in the buffer, the player software Start the buffering process. In this buffering process, the player software sequentially receives from the content server each data portion subsequent to the first data position (seek start position) in the multimedia content data and stores it in the buffer. The reproduction of the multimedia content data is stopped until a predetermined amount of data is stored in the buffer by the buffering process. After a predetermined amount of data is stored in the buffer by the buffering process, the reproduction of the multimedia content data is resumed from the first data position.

FIG. 3 is a diagram showing a system configuration of the computer 10.
As shown in FIG. 3, the computer 10 includes a CPU 101, a north bridge 102, a main memory 103, a south bridge 104, a graphics controller 105, a sound controller 106, a BIOS-ROM 107, a LAN controller 108, and a solid state drive (SSD) 109. Wireless LAN controller 112, embedded controller (EC) 113, EEPROM 114, HDMI control circuit 3 and the like.

  The CPU 101 is a processor that controls the operation of each unit in the computer 10. The CPU 101 executes an operating system (OS) 201 and various application programs loaded from the SSD 109 to the main memory 103. The application program includes a player application program 202. The player application program 202 is the above-described player software, and is executed on the operating system (OS) 201.

  The CPU 101 also executes the BIOS stored in the BIOS-ROM 107. The BIOS is a program for hardware control.

  The north bridge 102 is a bridge device that connects the local bus of the CPU 101 and the south bridge 104. The north bridge 102 also includes a memory controller that controls access to the main memory 103. The north bridge 102 also has a function of executing communication with the graphics controller 105 via a PCI EXPRESS standard serial bus or the like.

  The graphics controller 105 is a display controller that controls the LCD 17 </ b> A used as a display monitor of the computer 10. A display signal generated by the graphics controller 105 is sent to the LCD 17A. The LCD 17A displays an image based on the display signal. A touch panel 17B is disposed on the LCD 17A. The touch panel 17B is a pointing device for inputting on the screen of the LCD 17A. The user can operate a graphical user interface (GUI) displayed on the screen of the LCD 17A using the touch panel 17B. For example, the user can instruct execution of a function corresponding to the button by touching the button displayed on the screen.

  The HDMI terminal 2 is an external display connection terminal. The HDMI terminal 2 can send an uncompressed digital video signal and a digital audio signal to the external display device 1 with a single cable. The HDMI control circuit 3 is an interface for sending a digital video signal to the external display device 1 called an HDMI monitor via the HDMI terminal 2. That is, the computer 10 can be connected to the external display device 1 via the HDMI terminal 2 or the like.

  The south bridge 104 controls each device on a peripheral component interconnect (PCI) bus and each device on a low pin count (LPC) bus. Further, the south bridge 104 includes an ATA controller for controlling the SSD 109.

  The south bridge 104 has a built-in USB controller for controlling various USB devices. Further, the south bridge 104 has a function of executing communication with the sound controller 106. The sound controller 106 is a sound source device and outputs audio data to be reproduced to the speakers 18A and 18B. The LAN controller 108 is a wired communication device that executes, for example, IEEE 802.3 standard wired communication. The wireless LAN controller 112 is a wireless communication device that executes wireless communication of, for example, the IEEE 802.11 standard.

  The EC 113 is a one-chip microcomputer including an embedded controller for power management. The EC 113 has a function of turning on / off the computer 10 in accordance with the operation of the power button by the user.

Next, the functional configuration of the player application program 202 will be described with reference to FIG.
The player application program 202 includes a data source 301, a demultiplexer 302, a video decoder 303, an audio decoder 304, a renderer 305, and a playback control unit 306. The data source 301 sequentially executes processing for receiving multimedia content data from the content server 21 under the control of the playback control unit 306, and stores each received data portion in the buffer 401 (buffering). In the process of receiving multimedia content data from the content server 21, for example, HTTP streaming may be used. In this case, the data source 301 requests a data part from the content server 21 and receives the requested data part from the content server 21.

  The demultiplexer 302, the video decoder 303, the audio decoder 304, and the renderer 305 function as a playback unit that plays back the video data unit and the audio data unit included in the data unit stored in the buffer 401, respectively.

  The demultiplexer 302 receives data from the buffer 401 in the data source 301 and separates the data into a video data portion and an audio data portion. The video decoder 303 decodes the video data part and sends the decoded video data part to the renderer 305. The audio decoder 304 decodes the audio data part and sends the decoded audio data part to the renderer 305. The renderer 305 displays the decoded moving image of the video data portion on the video display area 31. Furthermore, the renderer 305 outputs sound corresponding to the decoded audio data part via a speaker or the like.

  The playback control unit 306 controls the playback operation of content data by the player application program 202. More specifically, the playback control unit 306 controls the data source 301, the demultiplexer 302, the video decoder 303, the audio decoder 304, and the renderer 305, respectively.

  The multimedia content data buffering process starts with an offset value included in the access request sent from the demultiplexer 302 to the data source 301 as a starting point. Here, the offset value indicates the data size (offset) from the start position of the multimedia content data to the reproduction target position of the multimedia content data. Execution of reproduction is waited until a certain amount of data is stored in the buffer 401. When a certain amount of data is stored in the buffer 401, the reproduction of the multimedia content data is started. During reproduction, the demultiplexer 302 issues access requests sequentially. If the data part of the offset value included in the access request is stored in the buffer 401, the data part is immediately returned from the data source 301 to the demultiplexer 302.

  The offset value specified by the access request from the demultiplexer 302 is recorded in the data source 301 as the final access offset. This final access offset is used for buffer amount management processing by the buffer amount management unit 402.

  In this embodiment, instead of using two stream buffers corresponding to video data and audio data, video data and audio data are stored in the same buffer 401. The configuration in which the same buffer 401 is shared for video data storage and audio data storage makes it possible to reduce the required memory amount and the required data transfer rate.

  In the present embodiment, the data source 301 further includes a media identifier setting unit 403. The media identifier setting unit 403 determines the video data final access offset (video) depending on whether the access request from the demultiplexer 302 is a request for video data (video access request) or a request for audio (audio access request). (Final access offset) and audio data final access offset (audio final access offset) are individually managed.

  Next, an example of the video final access offset and the audio final access offset will be described with reference to FIG. The video final access offset is an offset value specified by the last video access request. The audio final access offset is an offset value specified by the last audio access request.

  Here, it is assumed that multimedia content data in which a video data portion and an audio data portion whose reproduction times are close to each other is arranged relatively apart from each other is reproduced using streaming. Such multimedia content data includes content data in a format such as ASF (Advanced Systems Format). The content data file corresponding to the ASF includes a header portion including information such as a reproduction time length of the content, a data portion, and index information. The index information indicates an offset value of each video data part existing in the data part. The demultiplexer 302 receives the index information from the reproduction control unit 306, and calculates the offset values of the video data unit and the audio data unit based on the index information.

  In FIG. 5, a data portion (also referred to as a data section) from an offset value of 1000 bytes to an offset value of 2000 bytes is buffered in the multimedia content data sequence. The data section from the maximum value of the video final access offset and the audio final access offset to the end of the data in the buffer 401 is displayed as a seekable section 39 on the above-described seek bar 33, that is, as a buffered data section. The For example, if the video final access offset indicates 1500 bytes and the audio final access offset indicates 1300 bytes, the data from 1500 bytes to the data end (2000 bytes) in the buffer 401 is displayed as the seekable section 39. That is, in this embodiment, the seek-enabled section 39 is displayed on the seek bar 33 based on the difference between the maximum value of the video final access offset and the audio final access offset and the offset value of the data end in the buffer 401. Is done.

  When the user moves the slider 35 to a data position later than the seek-enabled section 39, that is, when the playback position is changed to a position outside the buffered period, the playback control unit 306 stops playback. At the same time, the buffer 401 is released, and buffering is performed again using the data source 301 (seeking outside the buffered period). More specifically, the playback control unit 306 starts a buffering process in which each data portion following the changed playback position is sequentially received from the content server 21 and stored in the buffer 401. When a certain amount of data is stored in the buffer 401, the playback control unit 306 restarts playback. The demultiplexer 302 issues an access request including an offset value (for example, a data position later in time than the 2000th byte) indicating the reproduction position after the change. In response to the reception of this access request, the data source 301 reads the data part requested for access from the buffer 401 and sends it to the demultiplexer 302.

  Next, the seek control operation of the present embodiment will be described with reference to FIG.

  When seeking outside the buffered period, the demultiplexer 302 may send a video access request and an audio access request to the data source 301 in the following order.

(1) Request for video 1 (2) Request for audio 2 (3) Request for video 3 (4) Request for audio 4 Video 1, audio 2, video 3 and audio 4 are data parts to be reproduced in this order. . In multimedia content data not intended for streaming, the offset values of video 1, audio 2, video 3 and audio 4 are not arranged in the order of video 1, audio 2, video 3 and audio 4, as shown in FIG. For example, audio 2, video 1, audio 4, and video 3 may be arranged side by side. In this case, as described above, the seek response may be significantly reduced. This will be specifically described below.

  When seeking outside the buffered period, the buffer 201 is first released. Then, a buffering process for sequentially receiving each data portion following the offset value corresponding to video 1 and storing it in the buffer 401 is started. When a certain amount of data is stored in the buffer 401, the reproduction is resumed. Multiplexer 302 sends a request for video 1 to data source 301. Since video 1 is stored in buffer 401I, video 1 is read from buffer 401, sent to demultiplexer 302, and video 1 is played back. After this, the demultiplexer 302 sends a request for audio 2 to the data source 301. At this time, since the audio 2 is not stored in the buffer 401, the reproduction of the multimedia content data is stopped again. The playback control unit 306 starts buffering of the data portion that follows the offset value corresponding to the audio 2 this time. When a certain amount of data is stored in the buffer 401, the reproduction is resumed. Audio 2 is sent to demultiplexer 302, and audio 2 is played back. After this, the demultiplexer 302 sends a request for video 3 to the data source 301. At this point, video 3 may not be stored in buffer 401. Therefore, the reproduction of the multimedia content data is stopped again.

  In the present embodiment, the following seek control operation is executed in order to prevent the occurrence of a situation where the reproduction operation is frequently stopped due to such a decrease in seek response.

  When the playback position is changed to the first data position in the multimedia content data that has not yet been stored in the buffer 401, the playback control unit 306 temporarily stops playback and releases the buffer 401. Then, the playback control unit 306 uses the data source 301 to start the first buffering process. In the first buffering process, a process of sequentially receiving each data portion following the first data position in the multimedia content data from the content server 21 and storing it in the buffer 401 is executed. After a certain amount of data is stored in the buffer 401 by the first buffering process, the playback control unit 306 causes the playback unit to resume playback of the multimedia content data from the first data position. After the reproduction is resumed, that is, after a certain amount of data is stored in the buffer 401 by the first buffering process, the audio data part in the multimedia content data that is not stored in the buffer 401 is May be required. In this case, usually, since the starting point of the buffering process is changed, the first buffering process is stopped, and instead, the requested audio data part is received from the content server 21 and stored in the buffer 401. The second buffering process may be started.

  In the present embodiment, the playback control unit 306 skips execution of the second buffering process for receiving the requested audio data unit from the content server 21 and storing it in the buffer 401, and performs the above-described first processing. While continuing the buffering process, the playback unit causes the playback of the requested audio data part to be skipped. That is, an access request for audio data not stored in the buffer 401 is ignored.

  Assume that the playback position is changed to an offset value corresponding to video 1 located outside the buffered section. In the first buffering process, each data portion subsequent to the offset value corresponding to video 1, that is, video 1, audio 4, video 3... Is sequentially buffered. That is, the playback control unit 306 receives the video 1, audio 4, video 3... From the content server 21 in this order by sequentially transmitting a data acquisition request to the content server 21 while increasing the offset value. When a certain amount of data, for example, a data portion corresponding to video 1 and a part of the data portion corresponding to audio 4 are stored in buffer 401 in the first buffering process, the reproduction of multimedia content data is resumed. Video 1 is first played. Thereafter, the demultiplexer 302 requests the audio 2 data. Since this request is an access request for audio data not stored in the buffer 401, the playback control unit 306 ignores this access or request and receives the audio 2 from the content server 21 and stores it in the buffer 401. Skip the ring process. Further, the playback control unit 306 causes the playback unit to skip playback of the audio 2. By skipping the buffering process for the audio 2, the process of sequentially buffering each data portion subsequent to the video 1 requested first is continuously executed.

  When a request for audio 4 occurs, audio 4 may already be buffered. In this case, the reproduction of the audio 4 is not skipped, and the audio 4 is reproduced as usual.

  Therefore, in the present embodiment, when the playback position is changed to the first data position in the multimedia content data that is not yet stored in the buffer 401 during the playback of the multimedia content data not intended for streaming, During a certain period of time, only video playback is performed and audio is not played back. After a certain period of time, both video and audio are played. Thus, by omitting buffering and playback for some audio data parts, it is possible to smoothly play multimedia content data in which the video data part and the audio data part whose playback times are close to each other are discontinuous. it can.

  Next, an arrangement example of the audio data portion and the video data portion in the multimedia content data will be described with reference to FIGS.

  FIG. 7 shows multimedia content data having a format for streaming. Each of the audio data parts A1, A2, A3,... Is a data part corresponding to a reproduction time of 1 second, for example. Similarly, each of the video data portions V1, V2, V3,... Is a data portion corresponding to a reproduction time of 1 second, for example. The audio data portion and the video data portion having the same subscript number are data to be reproduced at the same time or close to each other. As shown in FIG. 7, in multimedia content data having a format for streaming, a video data portion and an audio data portion having a reproduction time close to each other are continuous.

  FIG. 8 shows multimedia content data not intended for streaming. FIG. 9 shows another multimedia content data not intended for streaming.

  In this way, in multimedia content data not intended for streaming, the video data portion and the audio data portion that are close to each other in reproduction time are discontinuous.

  When seeking the multimedia content data of FIG. 8 or FIG. 9, the offset values specified by successive access requests do not increase monotonously, but the offset value to be accessed is increased or decreased as shown in FIG. There are things to do. This causes a decrease in seek response, as described above.

  Assume that the playback position is changed to a position corresponding to the video data V4 in a state where the data section after the audio data portion A1 of the multimedia content data in FIG. 9 is not buffered. The reproduction of the multimedia content data is temporarily stopped. The buffer 201 is released, and processing for sequentially buffering the data portions (V4, V5, V6, A4, A5, A6, V7, V8,...) Following the video data V4 is started. For example, when the video data V4 and V5 are buffered, the reproduction of the multimedia content data is resumed from the video data V4. Thereafter, a request for the audio data A4 is generated. The audio data A4 has not been buffered yet. Therefore, the reproduction is stopped again. Moreover, since the offset position of the audio data A4 is located behind the offset value indicating the end of the data in the current buffer 401 (for example, the offset value corresponding to the video data V5), the buffer 201 is released again. May be. Then, the starting point of buffering is changed, thereby starting the process of sequentially buffering the data parts (A4, A5, A6, V7, V8,...) Following the audio data A4.

  In the present embodiment, the following seek operation is performed on the multimedia content data of FIG.

  Assume that the playback position is changed to a data position corresponding to the video data V4 in a state where the data section after the audio data portion A1 is not buffered. The reproduction of the multimedia content data is temporarily stopped. The buffer 201 is released, and processing for sequentially buffering the data portions (V4, V5, V6, A4, A5, A6, V7, V8,...) Following the video data V4 is started. For example, when the video data V4 and V5 are buffered, the reproduction of the multimedia content data is resumed from the video data V4.

  Thereafter, an access request for the audio data A4 is generated. The audio data A4 has not been buffered yet. Therefore, buffering and reproduction of the audio data A4 are skipped. Therefore, since the starting point of the buffering process is not changed, the buffering process for each remaining unbuffered data part is continued. After the reproduction of the audio data A4 is skipped, an access request for the video data V5 is generated. Since the video data V5 is already buffered, the video data V5 is immediately sent to the playback unit. Thereafter, an access request for the audio data A5 is generated. If the audio data A5 is already buffered, the audio data A5 is also immediately sent to the playback unit.

  Next, a buffer release control operation for releasing the buffer 401 will be described with reference to FIGS. The release of the buffer is a process of setting a storage area in the buffer 401 in which data is stored to a free state in which no data is stored.

  FIG. 11 shows the buffer release control operation when the video final access offset and the audio final access offset are not distinguished. When the amount of data from the last access offset value to the offset value indicating the end of the data in the buffer 401 is smaller than the first threshold, in other words, the final access offset value and the offset value indicating the end of the data in the buffer 401 Is less than the first threshold, the data range from the beginning of the data in the buffer 401 to the last access offset value is invalidated to increase the size of the free storage area in the buffer 401, The storage area in the buffer 401 corresponding to the data range is released. For example, it is assumed that a data portion of multimedia content data having an offset value of 1000 bytes to an offset value of 2000 bytes is stored in the buffer 401. Assume that the first threshold is 600 bytes. When the final access offset value reaches 1500 bytes, the difference between the final access offset value 1500 bytes and the offset value 2000 bytes at the end of the data in the buffer 401 becomes less than the first threshold value. In this case, the data range from the first 1000 bytes of the data part to the final access offset 1500 bytes is invalidated.

  However, if such a buffer release control operation is applied to multimedia content data that is not intended for streaming, the data portion at the offset position that is likely to be requested next is buffered. The data part may be invalidated. FIG. 12 shows this state. In FIG. 12, it is assumed that the difference between the final access offset value corresponding to the (N−1) th access request and the offset value indicating the end of the data in the buffer 401 is less than the first threshold value. . When the (N-1) th access request is generated, the data range from the beginning of the data in the buffer 401 to the (N-1) th final access offset value is invalidated, thereby releasing the storage area corresponding to the data range. Is done. Next, an Nth access request is generated. The Nth access request may include an offset value that requests a data portion that was already in the released storage area.

  In the present embodiment, in order to prevent the occurrence of a situation where the data portion at the offset position that is highly likely to be requested next is buffered, the data portion is invalidated. A buffer release control operation as shown in FIG. 13 is executed.

  That is, in the buffer release control operation of this embodiment, the minimum value of the video final access offset and the audio final access offset is used as a reference value for the buffer release control operation.

  When the difference between the minimum value of the video final access offset and the audio final access offset and the offset value indicating the end of the data in the buffer 401 is less than the first threshold, the playback control unit 306 determines that the buffer 401 In order to increase the size of the free storage area, the data range from the beginning of the data in the buffer 401 to the above-mentioned minimum value is invalidated, and the storage area in the buffer 401 corresponding to the data range is released.

  For example, it is assumed that a data portion of multimedia content data having an offset value of 1000 bytes to an offset value of 2000 bytes is stored in the buffer 401. Assume that the first threshold is 800 bytes. The video final access offset value is 1500 bytes, and the audio final access offset value is 1250 bytes. In this case, the minimum value of the video final access offset and the audio final access offset is 1250 bytes, which is the audio final access offset value. Then, the data range from the first 1000 bytes of the data part to the audio final access offset 1250 bytes is invalidated, and the storage area in the buffer 401 corresponding to the data range is released. What remains in the buffer 401 is data from an offset value of 1250 bytes to an offset value of 2000 bytes.

  Here, the case where the audio final access offset value is smaller than the video final access offset value is illustrated here, but when the audio final access offset value is larger than the video final access offset value, the video final access offset value is Used as the minimum of the video final access offset and the audio final access offset.

  Next, with reference to FIG. 14 to FIG. 16, the reproduction stop operation executed when the buffered data amount is reduced will be described.

  FIG. 14 shows the control of the playback stop operation when the video final access offset and the audio final access offset are not distinguished. When the amount of data from the last access offset value to the offset value indicating the end of the data in the buffer 401 is smaller than the second threshold value, in other words, the final access offset value and the offset value indicating the end of the data in the buffer 401 Is less than the second threshold value, the reproduction of the multimedia content data is stopped until a certain amount of data is accumulated in the buffer 401.

  For example, it is assumed that a data portion of multimedia content data having an offset value of 1000 bytes to an offset value of 2000 bytes is stored in the buffer 401. The second threshold is assumed to be 150 bytes. When the final access offset value becomes 1900 bytes, for example, the difference between the final access offset value 1900 bytes and the offset value 2000 bytes at the end of the data in the buffer 401 becomes less than the second threshold value. In this case, the reproduction of the multimedia content data is stopped until a certain amount of data is accumulated in the buffer 401.

  However, if such playback stop control is applied to multimedia content data that is not intended for streaming, the playback operation stops suddenly even though there is a sufficient seekable section 39 on the seek bar 401. There is a possibility of being. FIG. 15 shows this state. In FIG. 15, the seek bar 33 at the time of the (N-1) th access is shown. At this time, a fixed amount of data located behind the last access offset (offset specified by the (N-1) th access request) is stored in the buffer 401 as the seekable section 39. However, in multimedia content data not intended for streaming, the audio data portion and the video data portion that are temporally close to each other are discontinuous, and therefore the offset value specified by the Nth access request is the (N−1) th. It may specify a position that is relatively far behind in time than the offset specified in the access request. In this case, a phenomenon occurs in which the reproduction operation is suddenly stopped even though there is a sufficient seekable section 39 on the seek bar 401.

  In the present embodiment, as shown in FIG. 16, the seek bar 33 is displayed on the basis of the difference between the maximum value of the video final access offset and the audio final access offset and the offset value indicating the end of the data in the buffer 401. The buffered data section (seekable section 39) is displayed, and playback stop control is executed based on the maximum value of the video final access offset and the audio final access offset.

  That is, when the difference between the maximum value of the video final access offset and the audio final access offset and the offset value indicating the end of the data in the buffer 401 is less than the second threshold, the playback control unit 306 The reproduction of the multimedia content data is stopped until a certain amount of data is accumulated in 401.

  In FIG. 16, it is assumed that the audio final access offset value is larger than the video final access offset value. In this case, the buffered data section (seekable section 39) is displayed on the seek bar 33 based on the difference between the offset value indicating the end of the data in the buffer 401. The display of the seek bar 33 in FIG. 16 corresponds to the (N-1) th access time.

  By such reproduction stop control, it is possible to prevent the phenomenon that the reproduction operation is suddenly stopped even though there is a sufficient seekable section 39 on the seek bar 401.

  Next, with reference to the flowchart of FIG. 17, the procedure of the reproduction control process executed by the player application program 202 will be described.

  In the player application program 202, based on the index information of the multimedia content data to be played, the playback control unit 306 sequentially receives each data portion in the multimedia content data from the content server 21 and stores it in the buffer 401. Start the buffering process. When a certain amount of data is stored in the buffer 401, a reproduction request is issued from the reproduction control unit 306 to the demultiplexer 302 (step S11), thereby starting reproduction processing for reproducing multimedia content data by streaming. (Step S12). Note that first, a playback request may be issued from the playback control unit 306 to the demultiplexer 302, and then the buffering process may be started.

  In the reproduction process, the demultiplexer 302 requests data (video or audio) to be reproduced next from the data source 301 based on the index information of the multimedia content data (step S13). In the data source 301, information indicating the requested media type (video or audio) is set in the media identifier setting unit 403 (step S14). Further, in the data source 301, the video final access offset or the audio final access offset is updated according to the media type requested to be accessed (step S15). Then, under the control of the playback control unit 306, a buffer management operation using these video final access offset and audio final access offset is executed (steps S16 and S17). In step S16, the above-described buffer release control operation (buffer release process) is executed. In step S17, the above-described reproduction stop operation (rebuffering process) for storing a certain amount of data in the buffer 401 by temporarily stopping reproduction is executed. Details of the procedures of the buffer release control operation and the reproduction stop operation will be described later with reference to FIGS.

  When data (video or audio) to be reproduced next is sent from the data source 301 to the demultiplexer 302 (step S18), the data is reproduced by being decoded by the video decoder 303 or the audio decoder 304 (step S19). ).

  Until the reproduction of the multimedia content data is completed, the reproduction process from step S12 to step S19 is repeatedly executed. For example, when the playback progresses and the playback position reaches the end of the multimedia content data, the playback process is completed (step S21).

  Next, the seek control operation executed by the player application program 202 will be described with reference to the flowchart of FIG.

  When the playback position is changed to a data position (first data position) in the multimedia content data that has not yet been stored in the buffer 401 by the movement operation of the slider 35 by the user, the following seek control operation is executed. Is done.

  That is, the playback control unit 306 causes the demultiplexer 302 to stop the playback operation and requests the demultiplexer 302 to change the playback position to the first data position (step S31). Then, seek processing is started under the control of the reproduction control unit 306 (step S32). In the seek process, the demultiplexer 302 requests data (video or audio) to be reproduced next from the data source 301 based on the index information of the multimedia content data (step S33). Immediately after the start of the seek process, in step S33, the demultiplexer 302 calculates an offset value of the video data portion corresponding to the first data position based on the index information of the multimedia content data, and accesses including the offset value Send the request to the data source 301. In the data source 301, information indicating the requested media type (video or audio) is set in the media identifier setting unit 403 (step S34). Further, in the data source 301, the video final access offset or the audio final access offset is updated according to the media type requested to be accessed (step S35). Then, under the control of the reproduction control unit 306, the following buffering control operation is executed.

  When the access requested data is not stored in the buffer 401 (NO in step S36), the playback control unit 306 determines whether the access requested media type is video or audio (step S41). . If the access-requested media type is video (YES in step S41), the playback control unit 306 releases the entire storage area of the buffer 401, and each of the data units following the offset value specified by the access request. Are sequentially received from the content server 21 and stored in the buffer 401 (step S42). In the first buffering process, the playback control unit 306 repeatedly executes a process of acquiring each data part subsequent to the offset value specified by the access request from the content server 21.

  When a certain amount of data is buffered by the first buffering process, the playback control unit 306 restarts the playback operation. When data (video or audio) to be reproduced next is sent from the data source 301 to the demultiplexer 302 (step S37), the data is reproduced by being decoded by the video decoder 303 or the audio decoder 304 (step S38). ). Until the reproduction of the multimedia content data is completed, the processing from steps S32 to S38 is repeatedly executed. When the playback progresses and the playback position reaches the end of the multimedia content data, the playback process (seek) is completed (step S40).

  If the access requested data is not stored in the buffer 401 (NO in step S36), and the media type of the requested data is audio data (NO in step S41), the playback control unit 306 Then, the buffering and reproduction of the requested data (audio data) is skipped (step S43), and the process proceeds to step S33. That is, when an access request is made for audio data that is not stored in the buffer 401, the playback control unit 306 obtains audio data requested by the access request from the content server 21, and demultiplexes the audio data. Each of the processes to be reproduced by 302 is skipped. Therefore, during the seek period, even if audio data that has not yet been buffered is requested, the seek start point is not changed in order to obtain the audio data. Thus, the first buffering process is continuously executed.

  If an unbuffered video data part is requested during the seek process, the buffering process in step S42 is executed. In this case, a process of sequentially receiving the requested video data part and the data part subsequent to the requested video data part and storing them in the buffer 401 is executed.

  The first buffering process is started when the playback position is changed to the first data position. After a certain amount of data is buffered, the seek process from step S32 is started. Also good.

  Next, the procedure of the buffer release control operation (buffer release process) executed in step S16 of FIG. 17 will be described with reference to the flowchart of FIG. As shown in FIG. 19, in the buffer release process, the playback control unit 306 uses the buffer amount management unit 402 to determine whether the following conditions are satisfied.

Threshold> BLOffset − MIN (VFOffset, AFOffset)
Here, BLOffset indicates the offset value at the end of the data stored in the buffer 401. VFOffset indicates the video final access offset value. AFOffset indicates the audio last access offset value. MIN (VFOffset, AFOffset) indicates a function for selecting a minimum value from two values of VFOffset and AFOffset.

  When the condition of threshold> BLOffset−MIN (VFOffset, AFOffset) is satisfied (YES in step S51), the playback control unit 306 invalidates the data from the beginning of the data in the buffer 401 to MIN (VFOffset, AFOffset). The storage area in the buffer 401 in which the data is stored is released (step S52).

  Next, the procedure of the reproduction stop operation (rebuffering process) executed in step S17 of FIG. 17 will be described with reference to the flowchart of FIG. As illustrated in FIG. 20, in the rebuffering process, the playback control unit 306 uses the buffer amount management unit 402 to determine whether the following conditions are satisfied.

Threshold> BLOffset−MAX (VFOffset, AFOffset)
Here, BLOffset indicates the offset value at the end of the data stored in the buffer 401. VFOffset indicates the video final access offset value. AFOffset indicates the audio last access offset value. MAX (VFOffset, AFOffset) indicates a function for selecting the maximum value from two values of VFOffset and AFOffset.

  When the condition of threshold> BLOffset−MAX (VFOffset, AFOffset) is satisfied (YES in step S61), the reproduction control unit 306 stops reproduction until a certain amount of data is stored in the buffer 401 (step S62).

  As described above, according to the present embodiment, when the reproduction position is changed to the first data position in the multimedia content data that is not yet stored in the buffer 401, the first data position follows. A first buffering process for sequentially receiving each data part from the content server 21 and storing it in the buffer 401 is started. When an audio data part not stored in the buffer 401 is requested after a predetermined amount of data is stored in the buffer 401, the requested audio data part is received by the content server 21 and stored in the buffer 401. The execution of the second buffering process for storing is skipped, the first buffering process is continued, and the reproduction of the requested audio data part by the reproducing part is skipped. Therefore, if the playback position is changed to the first data position in the multimedia content data that has not been stored in the buffer 401 during the playback of the multimedia content data not intended for streaming, Only plays the video, not the audio. After a certain period of time, both video and audio are played. By omitting buffering and playback for some audio data parts in this way, even if the multimedia content data to be played is content data in which the video data portion and the audio data portion that are close to each other in playback time are discontinuous. Even so, the content data can be reproduced smoothly.

  Note that the playback control processing procedure of this embodiment can all be executed by software. For this reason, it is possible to easily realize the same effect as that of the present embodiment only by installing and executing this program on a normal computer through a computer-readable storage medium storing the program for executing the reproduction control processing procedure. Can do.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 10 ... Computer, 21 ... Content server, 33 ... Seek bar, 202 ... Player application program, 301 ... Data source, 302 ... Demultiplexer, 303 ... Video decoder, 304 ... Audio decoder, 305 ... Renderer, 306 ... Playback control part, 401 ... buffer, 402 ... buffer amount management unit.

Claims (6)

An electronic device that reproduces multimedia content data in which video data and audio data are multiplexed while receiving from a server,
A buffer for storing the received multimedia content data;
Playback means for playing back a video data portion and an audio data portion in the multimedia content data stored in the buffer;
When the playback position is changed to the first data position in the multimedia content data that is not yet stored in the buffer, the playback means stops the playback, and the playback unit stops the playback. Control means for starting a first buffering process for sequentially receiving each data part following one data position from the server and storing it in the buffer;
The control means includes
Causing the reproduction means to resume the reproduction from the first data position after the first buffering process is started;
When a video data portion in the multimedia content data that is not stored in the buffer is requested by the playback means after the playback is resumed, the first buffering process is stopped and the requested video is stopped. A new buffering process for receiving each data part following the data part from the server and storing it in the buffer is started,
After the reproduction is resumed, if an audio data part in the multimedia content data not stored in the buffer is requested by the reproduction means, the execution of the first buffering process is continued and the reproduction means An electronic device that causes the requested audio data portion to be skipped.   When the audio means in the multimedia content data that is not stored in the buffer is requested by the reproduction means, the control means assigns each data part subsequent to the requested audio data part to the server. The electronic device according to claim 1, wherein the execution of the first buffering process is continued without executing a new buffering process for receiving the data from and storing in the buffer.   The control means is further configured to determine an offset value in the multimedia content data respectively designated by a video access request and an audio access request received from the playback means during playback of the multimedia content data as a video final access offset. And an audio final access offset, and a difference between a minimum value of the video final access offset and the audio final access offset and an offset value indicating an end of data in the buffer is less than a first threshold value. The electronic device according to claim 1, wherein when it is smaller, the storage area in the buffer corresponding to the data range from the beginning of the data in the buffer to the minimum value is released.   The control means is further configured to determine an offset value in the multimedia content data respectively designated by a video access request and an audio access request received from the playback means during playback of the multimedia content data as a video final access offset. And a buffer on the seek bar based on a difference between a maximum value of the video final access offset and the audio final access offset and an offset value indicating an end of data in the buffer. If the difference between the maximum value and the offset value indicating the end of the data in the buffer is smaller than a second threshold value when the ringed data section is displayed, the reproduction means reproduces the multimedia content data. Stop The electronic apparatus according to claim 1, characterized in that is configured to. A reproduction control method for reproducing multimedia content data in which video data and audio data are multiplexed while receiving from a server,
Storing the received multimedia content data in a buffer;
Playing the video data portion and the audio data portion in the multimedia content data stored in the buffer;
If the playback position is changed to a first data position in the multimedia content data that is not yet stored in the buffer, the playback is stopped and the first data position in the multimedia content data Starting a first buffering process for sequentially receiving each data part from the server and storing it in the buffer;
Resuming the playback from the first data position after the first buffering process is started;
When a video data portion in the multimedia content data that is not stored in the buffer is requested after the reproduction is resumed, the first buffering process is stopped and the requested video data portion is followed. Starting a new buffering process for receiving each data part from the server and storing it in the buffer;
When the reproduction of the audio data part in the multimedia content data that is not stored in the buffer is requested after the reproduction is resumed, the first buffering process is continued and the requested audio data part Playback control method that skips playback. A program for causing a computer to reproduce multimedia content data in which video data and audio data are multiplexed while receiving from a server,
Storing the received multimedia content data in a buffer;
Replaying a video data portion and an audio data portion in the multimedia content data stored in the buffer;
If the playback position is changed to a first data position in the multimedia content data that is not yet stored in the buffer, the playback is stopped and the first data position in the multimedia content data Starting a first buffering process for sequentially receiving each data part from the server and storing it in the buffer;
Resuming the playback from the first data position after the first buffering process is started;
When a video data portion in the multimedia content data that is not stored in the buffer is requested after the reproduction is resumed, the first buffering process is stopped and the requested video data portion is followed. A procedure for starting a new buffering process for receiving each data part from the server and storing it in the buffer;
When the reproduction of the audio data part in the multimedia content data that is not stored in the buffer is requested after the reproduction is resumed, the first buffering process is continued and the requested audio data part A program for causing the computer to execute a procedure for skipping the reproduction of.

Images