JP5557939B2 - Portable terminal, information reproduction system, and information reproduction method - Google Patents

Description

  The present invention relates to a portable terminal, an information reproduction system, and an information reproduction method.

  2. Description of the Related Art Conventionally, there are portable terminals that can view AV content including a plurality of media elements such as video, audio, and text data. Such contents such as digital broadcasting and video streaming have a huge amount of data, so that the mobile terminal performs processing by a high-speed wireless communication device, a high-performance output device, and an arithmetic device. However, since there are many media elements included in the content, it is necessary to adjust the display area and size of each element in a single mobile terminal, and the content cannot be reproduced sufficiently. On the other hand, Patent Literature 1 and Patent Literature 2 disclose configurations for reproducing content using a plurality of portable terminals.

JP 2000-122957 A JP 2003-5947 A

  However, in the configurations described in Patent Literature 1 and Patent Literature 2, when media elements included in content such as streaming data are divided and played back, the difference in processing capability between the playback devices and the portable terminal and each playback device Due to the data transfer delay, the playback timings of a plurality of playback devices may shift.

  Accordingly, an object of the present invention is to provide a portable terminal, an information reproducing system, and an information reproducing method capable of reproducing contents such as streaming data in synchronization with other reproducing apparatuses in order to solve such problems. To do.

In order to solve the above problems, an information reproduction system of the present invention is an information reproduction system in which a first portable terminal and a second portable terminal reproduce contents in synchronization. The first mobile terminal reproduces the first data acquisition means for acquiring the content and the plurality of media elements included in the content acquired by the first data acquisition means by the first mobile terminal and the second mobile terminal. possible whether in response to a first media element to be reproduced in the first mobile terminal, the reproduction pattern determining means for determining a second media element to be reproduced to the second mobile terminal, the first mobile terminal is a second mobile from the transmission of the data to the terminal, the first mobile terminal has a constant measuring means measures the data transfer time is the time it takes to receive a response from the second mobile terminal, is determined by the reproduction pattern determining unit 1 Media elements and a media element extracting means for the second media element is extracted from the content, the second mobile terminal a second media element extracted by the media element extracting means Sends a second media element of the second reproduction start timing determined as the time after the time obtained by adding the data transfer time measured by the measuring means at the time of sending to the mobile terminal notifies the second mobile terminal And a first synchronized playback control means for playing back the first media element at the playback start timing . The second portable terminal has second data acquisition means for acquiring the second media element transmitted by the first synchronous reproduction control means, and second data acquisition means at the reproduction start timing notified by the first synchronous reproduction control means. And second synchronized playback control means for playing back the second media element acquired by

The information reproduction method of the present invention is an information reproduction method in an information reproduction system in which a first portable terminal and a second portable terminal reproduce contents in synchronization. The information reproduction method includes: a first data acquisition step for acquiring content in the first mobile terminal; and a plurality of media elements included in the content acquired in the first data acquisition step. depending on whether the second mobile terminal can reproduce the first media element to be reproduced in the first mobile terminal, the reproduction pattern determining step of determining a second media element to be reproduced to the second mobile terminal, the reproduction pattern A media element extraction step of extracting from the content the first media element and the second media element determined in the determination step; and after the first mobile terminal transmits data to the second mobile terminal, the first mobile terminal from the mobile terminal and measuring step to measure a data transfer time is the time until receiving the response, the media element extraction scan The second media element extracted in-up and transmits to the second mobile terminal, the second media element of the second measured data transfer time to time after the time obtained by adding the measuring step in time to be transmitted to the mobile terminal A first synchronized playback step of notifying the second mobile terminal of the playback start timing determined as and playing back the first media element at the playback start timing . Further, in the second portable terminal , the information reproduction method includes a second data acquisition step for acquiring the second media element transmitted in the first synchronous reproduction step, and a reproduction start timing notified in the first synchronous reproduction step. It comprises a second synchronization step of reproducing the second media element acquired in the second data acquisition step.

The portable terminal of the present invention is a portable terminal that reproduces content in synchronization with another portable terminal . According to whether the portable terminal and other portable terminals can reproduce data for each of a plurality of media elements included in the data acquired by the data acquiring unit and the data acquired by the data acquiring unit. a first media element to be reproduced in the portable terminal, and the reproduction pattern determining means for determining a second media element to be reproduced to another mobile terminal, the first media element and the second media element that is determined by the reproduction pattern determining means a media element extracting means for extracting from the content, the mobile terminal from the transmission of the data to another portable terminal, measuring a data transfer time is a time until the mobile terminal receives the response from other mobile terminals constant measuring means for, sends the second media element extracted by the media element extraction unit to other mobile terminal, the second The playback start timing determined Deer elements as the time after the time obtained by adding the data transfer time measured by the measuring means at the time of transmitting to other mobile terminals, and notify the other mobile terminal, the reproduction start timing Synchronized playback control means for playing back the first media element .

  According to this configuration, among the plurality of media elements included in the content, the first media element to be played back on the first mobile terminal and the second media element to be played back on the second mobile terminal are determined. One content can be divided for each media element and reproduced on the first portable terminal and the second portable terminal. Then, at the reproduction start timing determined based on the data transfer time between the first portable terminal and the second portable terminal, the first media element and the second media element are reproduced on the first portable terminal and the second portable terminal, respectively. Thus, reproduction can be started simultaneously in consideration of a delay time caused by data transfer from the first portable terminal to the second portable terminal. For this reason, it becomes possible to reproduce | regenerate the one content divided | segmented for every media element synchronously with the 1st portable terminal and the 2nd portable terminal.

Furthermore , among the plurality of media elements included in the content, a media element that can be processed by the first mobile terminal can be a first media element, and a media element that can be processed by the second mobile terminal can be a second media element. . For this reason, it is possible to reliably reproduce the media element in the first portable terminal and the second portable terminal.

The second synchronized playback control means notifies the playback end time of the playback interval in the second mobile terminal to the first mobile terminal for each playback interval that divides the playback time of the content, and the first synchronized playback control means The second mobile terminal is instructed to specify the playback rate so that the difference between the playback end time of the playback interval in the second mobile terminal notified by the synchronous playback control means and the playback end time of the playback interval in the first mobile terminal becomes small. The second synchronized playback control means may play back the second media element at the playback rate instructed by the first synchronized playback control means.

  Processing executed in the first mobile terminal and the second mobile terminal changes according to time. For this reason, based on the difference between the time actually required by the first mobile terminal to play the first media element in the playback interval and the time actually required by the second mobile terminal to play the second media element in the playback interval. Thus, by determining the playback rate of the second mobile terminal, the difference between the playback end time of the playback interval of the first mobile terminal and the playback end time of the playback interval of the second mobile terminal can be suppressed. For this reason, even if the playback time of the playback interval changes for each of the first mobile terminal and the second mobile terminal due to changes in the processing load of the first mobile terminal and the second mobile terminal, by changing the playback rate , Synchronization loss can be reduced. As a result, it becomes possible for the first mobile terminal and the second mobile terminal to play content in synchronization.

The first synchronized playback control means may transmit the second media element to the second portable terminal at a transmission rate determined based on the data transfer time so that the data to be reproduced in the second portable terminal is not insufficient. Good. According to this configuration, it is possible to suppress a shortage of data to be reproduced in the second mobile terminal by determining the transmission rate for transmitting the second media element to the second mobile terminal in consideration of the data transfer time. . For this reason, in the second portable terminal, it is possible to suppress a delay in reproduction, and the first portable terminal and the second portable terminal can reproduce the content in synchronization.

The measuring unit measures the data transfer time for each playback interval that divides the playback time of the content, and the first synchronous playback control unit determines whether the data to be played back on the second portable terminal is based on the measured data transfer time. You may adjust a transmission rate so that it may not run short . The data transfer time between the first portable terminal and the second portable terminal may change depending on the state of the communication path such as traffic between the first portable terminal and the second portable terminal. According to this configuration, by adjusting the transmission rate based on the data transfer time measured at each reproduction interval, it is possible to obtain a transmission rate according to the change in the data transfer time. For this reason, in the 2nd portable terminal, the delay of reproduction | regeneration can further be suppressed.

The second portable terminal is configured to detect whether or not the first portable terminal is in a disconnected state in which content cannot be acquired, and when the disconnected state of the first portable terminal is detected by the disconnection detecting unit. You may further provide the continuous reproduction | regeneration control means which controls so that a content may be reproduced | regenerated continuously in a portable terminal. In this case, the first synchronized playback control means transmits the address information of the content to the second mobile terminal, and the second data acquisition means receives the first mobile terminal when the disconnection detecting means detects the disconnection state of the first mobile terminal. Based on the address information transmitted by the one synchronous playback control means and the time stamp of the second media element that is not played back on the second mobile terminal among the second media elements acquired by the second data acquisition means , of the content, it acquires the unreproduced portion not reproduced in the first mobile terminal and the second mobile terminal, continuous reproduction control means, out of the unplayed portion obtained by the second data acquisition means, the second mobile terminal May play media elements that can be played . According to this configuration, even if the first mobile terminal cannot acquire content, the second mobile terminal can acquire an unreproduced portion of the content. For this reason, it becomes possible to continuously reproduce the content on the second portable terminal. That is, it is possible to continue playback dynamically according to changes in the playback device and network environment.

The first mobile terminal may further include a first buffer that stores the first media element extracted by the media element extraction unit, and the second mobile terminal receives the second media element acquired by the second data acquisition unit. When the disconnection state of the first mobile terminal is detected by the disconnection detection means, the disconnection detection means for detecting whether or not the first mobile terminal is in a disconnection state where content cannot be acquired, You may further provide the continuous reproduction | regeneration control means which controls so that a content may be reproduced | regenerated continuously in a 2nd portable terminal. The second data acquisition means is not played back on the first mobile terminal among the first media elements stored in the first buffer when the disconnection detecting means detects the disconnection state of the first mobile terminal. The first media element is acquired from the first mobile terminal, and the continuous playback control means is configured to select a second media element out of the first media element acquired by the second data acquisition means and the second media element stored in the second buffer . A part of the content is reconstructed using the second media element that is not played back on the mobile terminal, and the media element that can be played back by the second mobile terminal is played out of the part of the reconfigured content. May be. According to this configuration, even if the first portable terminal is disconnected, the transmitted and unplayed content can be reconfigured in the second portable terminal. For this reason, it is possible to continuously reproduce the transmitted content on the second portable terminal. That is, it is possible to continue playback dynamically according to changes in the playback device and network environment.

  The reproduction start timing may be an absolute time. The data transfer time between the first portable terminal and the second portable terminal may change depending on the data amount of the communication path between the first portable terminal and the second portable terminal. Even in such a case, by setting the reproduction start timing to an absolute time, the first portable terminal and the second portable terminal can respectively change the first media element and the second media element without being affected by the change in the data transfer time. Can be played back synchronously.

  The reproduction start timing may be a relative time. In this case, even when there is a difference between the absolute time held by the first mobile terminal and the absolute time held by the second mobile terminal, the first mobile terminal and the second mobile terminal synchronize content. It can be played back.

  The content may be streaming data. In this case, streaming data divided for each media element can be synchronously reproduced by a plurality of devices.

  According to the present invention, a plurality of devices can reproduce contents such as streaming data in synchronization.

1 is a diagram schematically showing a functional configuration of an information reproduction system according to a first embodiment of the present invention. It is a figure which shows the hardware constitutions of the 1st portable terminal of FIG. It is a figure which shows an example of a streaming data management table. It is a figure which shows an example of a reproduction | regeneration management table. It is a figure which shows the reproduction | regeneration state of the streaming data in the information reproduction | regeneration system of FIG. It is a sequence diagram which shows the connection establishment process of the information reproduction system of FIG. It is a sequence diagram which shows an example of the synchronous reproduction | regeneration control process of the information reproduction | regeneration system of FIG. It is a sequence diagram which shows an example of the continuous reproduction | regeneration control process of the information reproduction | regeneration system of FIG. It is a sequence diagram which shows an example of the continuous reproduction | regeneration control process of the information reproduction | regeneration system of FIG. It is a figure which shows an example of the transition of the screen of the 1st portable terminal and 2nd terminal device of FIG. 6 and FIG. It is a figure which shows an example of the transition of the screen of the 1st portable terminal of FIG. 6 and FIG. 7, and a 2nd portable terminal. It is a figure which shows an example of the transition of the screen of the 1st portable terminal of FIG. 8, and a 2nd portable terminal. It is a figure which shows an example of the transition of the screen of the 1st portable terminal of FIG. 9, and a 2nd portable terminal. It is a figure which shows roughly the function structure of the information reproduction system which concerns on 2nd Embodiment. It is a figure which shows an example of the screen of the 1st portable terminal of FIG. It is a sequence diagram which shows an example of the synchronous reproduction | regeneration control process of the information reproduction | regeneration system of FIG.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numerals, and redundant description will be omitted.

(First embodiment)
FIG. 1 is a diagram schematically illustrating a functional configuration of the information reproduction system according to the first embodiment. As shown in FIG. 1, the information reproduction system 10 includes a first mobile terminal 1 (mobile terminal), a second mobile terminal 2 (other mobile terminals), and a streaming server 3. The first portable terminal 1 and the second portable terminal 2 are systems for reproducing information such as streaming data in a multi-device manner. Multi-device playback is a playback method in which a plurality of media elements constituting content such as one streaming data is distributed to a plurality of terminals, and the media elements distributed in each terminal are synchronously played back.

  The first mobile terminal 1 is a device that performs multi-device playback together with the second mobile terminal 2, and is a main device that leads multi-device playback processing. The first mobile terminal 1 is, for example, a device such as a tablet terminal, a PDA (Personal Digital Assistants), a notebook PC, a smartphone, and a mobile phone. The first mobile terminal 1 is configured to be communicable with the second mobile terminal 2 via the local network LN, and is connected to the network NW. The first portable terminal 1 communicates with the second portable terminal 2 by short-range wireless communication such as a wireless local area network (LAN) or Bluetooth, or a dedicated cable, for example. The first mobile terminal 1 communicates with the network NW by a wireless WAN (Wide Area Network) such as 3G (3rd Generation) or LTE (Long Term Evolution).

  FIG. 2 is a diagram illustrating a hardware configuration of the first mobile terminal 1. As shown in FIG. 2, the first portable terminal 1 physically includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102 as a main storage device, a ROM (Read Only Memory) 103, a hard disk. An auxiliary storage device 104 such as a network card, a communication module 105 which is a data transmission / reception device such as a network card, an input device 106 which is an input device such as a touch panel, a mouse and a keyboard, an output device 107 which is an output device such as a liquid crystal display and a speaker, time It is comprised by hardware, such as the timer 108 which measures this. Each function of the first terminal device 1 to be described later allows the input device 106, the output device 107, the timer 108, and the like to be controlled under the control of the CPU 101 by loading predetermined computer software on hardware such as the CPU 101 and the RAM 102. This is realized by operating and reading and writing data in the RAM 102 and the auxiliary storage device 104.

  The second mobile terminal 2 is a device that performs multi-device playback together with the first mobile terminal 1, and is a sub-device that performs multi-device playback processing based on an instruction from the first mobile terminal 1. This 2nd portable terminal 2 is apparatuses, such as a smart phone, a mobile phone, a tablet terminal, PDA, and a notebook PC, for example. The second mobile terminal 2 is configured to be communicable with the first mobile terminal 1 via the local network LN, and is connected to the network NW. The second portable terminal 2 communicates with the first portable terminal 1 by, for example, short-range wireless communication such as wireless LAN or Bluetooth, or a dedicated cable. The second mobile terminal 2 communicates with the network NW by a wireless WAN such as 3G or LTE.

  The second mobile terminal 2 has a hardware configuration similar to that of the first mobile terminal 1 and includes hardware such as a CPU, RAM, ROM, communication module, auxiliary storage device, input device, output device, and timer. . Each function of the second portable terminal 2 to be described later operates an input device, an output device, a timer, and the like under the control of the CPU by reading predetermined computer software on hardware such as a CPU and a RAM. This is realized by reading and writing data in the RAM and the auxiliary storage device.

  The streaming server 3 is a device that distributes streaming data to the first mobile terminal 1 or the second mobile terminal 2. The streaming server 3 is connected to the network NW. The streaming server 3 includes an information processing device such as a server device, and stores a CPU, a RAM, a ROM, a communication module for communicating with the first portable terminal 1 and the second portable terminal 2 on the network NW, and information. It has hardware such as storage devices. By operating these components, each function of the streaming server 3 described later is exhibited.

  The network NW may be configured by either wired or wireless, for example, a network such as a wireless WAN, a wired WAN, a mobile communication network, and the Internet. The first portable terminal 1, the second portable terminal 2, and the streaming server 3 are connected to each other via a network NW.

  Next, the functional configuration of the information reproduction system 10 will be described. As shown in FIG. 1, the streaming server 3 functionally includes a communication unit 31, a streaming data storage unit 32, a streaming data management unit 33, and a reproduction management unit 34.

  The communication control unit 31 functions as a communication control unit that performs data transmission / reception with the network NW and session management. When the communication control unit 31 receives a connection request from the first portable terminal 1 or the second portable terminal 2, the communication control unit 31 transmits a connection response indicating whether or not the connection is possible, and communicates with the first portable terminal 1 or the second portable terminal 2. Establish a communication connection. Further, when the communication connection is disconnected during data transmission, the communication control unit 31 stores the transmission data in the buffer and puts the data transmission into the suspended state.

  The streaming data storage unit 32 functions as a streaming data storage unit that stores a plurality of streaming data reproduced in the first mobile terminal 1 and the second mobile terminal 2. The streaming data stored in the streaming data storage unit 32 is managed by the streaming data management unit 33.

  The streaming data management unit 33 functions as a streaming data management unit that manages meta information of streaming data stored in the streaming data storage unit 32. When receiving the streaming data distribution request from the first portable terminal 1 or the second portable terminal 2, the streaming data management unit 33 distributes the streaming data to the first portable terminal 1 or the second portable terminal 2 that has transmitted the streaming data distribution request. A streaming data delivery response including meta information of streaming data corresponding to the request is transmitted via the communication control unit 31.

  The streaming data management unit 33 manages the meta information of the streaming data stored in the streaming data storage unit 32 using the streaming data management table. In addition, the streaming data management unit 33 refers to the streaming data management table in the negotiation with the first mobile terminal 1 and the second mobile terminal 2.

  FIG. 3 is a diagram illustrating an example of the streaming data management table. As shown in FIG. 3, the streaming data management table includes a streaming data ID, a file name, URL (Uniform Resource Locator) information, media configuration information, codec information, data rate information, DRM (Digital Rights). Management) information is stored in association with each other. The streaming data ID is identification information for identifying the streaming data stored in the streaming data storage unit 32. The file name is information indicating the file name of the streaming data indicated by the streaming data ID. The URL information is address information indicating the location on the network NW of the streaming data indicated by the streaming data ID.

  The media configuration information is information indicating media elements that constitute the streaming data indicated by the streaming data ID. This media element includes, for example, video data (video), audio data (voice), text data, and the like. In this example, video data is represented as “v”, audio data is represented as “a”, and text data is represented as “t”. The codec information is information indicating the codec of each media element constituting the streaming data indicated by the streaming data ID. This codec information includes information indicating the codec of video data (H.264, WMV, etc.), information indicating the codec of audio data (AAC, AAC +, AMR, etc.), and the like.

  The data rate information is information indicating a data rate that the streaming server 3 can provide for each media element constituting the streaming data indicated by the streaming data ID. This data rate information includes information indicating the data rate of video data, information indicating the data rate of audio data, and the like. The DRM information is information for managing the copyright of the streaming data indicated by the streaming data ID. The DRM information includes, for example, information indicating the reproduction time limit of streaming data, multi-device information indicating whether the streaming data can be reproduced by multi-device, and the like.

  In this streaming data management table, for example, streaming data with a streaming data ID “1” has a file name “aaa.flv”, a URL “aaa / index.html / aaa.flv”, and video data and audio. It is shown that it is composed of data and text data. The video data codec of the streaming data is “H.264”, the audio data codec is “AAC +”, and the text data codec is “text”. Further, it is shown that the data rate of the video data is “756 kbps / 384 kbps” and the data rate of the audio data is “128 kbps / 64 kbps”. In addition, the reproduction time limit of the streaming data is December 31, 2012, which indicates that multi-device reproduction is possible.

  For example, when receiving the notification of the playback rate from the first mobile terminal 1 or the second mobile terminal 2, the streaming data management unit 33 acquires a data rate that can be provided by the streaming server 3 from the streaming data management table. Then, the streaming data management unit 33 determines a data transmission rate from the data rates that can be provided by the streaming server 3 based on the notified reproduction rate. Then, the streaming data management unit 33 transmits a data transmission rate response specifying the data transmission rate and the transmission data size to the first portable terminal 1 or the second portable terminal 2.

  In addition, when receiving the streaming data transmission request from the first portable terminal 1, the streaming data management unit 33 acquires the streaming data corresponding to the streaming data transmission request from the streaming data storage unit 32. Then, the streaming data management unit 33 divides the streaming data into divided data having a predetermined data size in the order of reproduction. Then, the streaming data management unit 33 transmits the divided data to the first portable terminal 1 via the communication control unit 31 in the order of reproduction. Note that the first divided data may be included in the streaming data delivery response.

  In addition, when the streaming data management unit 33 receives a streaming data transmission request in which the continuous playback start time is specified from the second portable terminal 2 that is a sub device, the streaming data management unit 33 sets the divided data after the continuous playback start time in the playback order. 2 Transmit to the mobile terminal 2 via the communication control unit 31. Also, when the streaming data management unit 33 receives a streaming data transmission request for which the continuous playback start time is not specified from the second portable terminal 2, the divided data after the time indicated by the transmitted time stamp received from the playback management unit 34 Are transmitted to the second portable terminal 2 via the communication control unit 31 in the order of reproduction.

  The reproduction management unit 34 functions as a reproduction management unit that dynamically manages a reproduction state in a specific client including the first portable terminal 1 and the second portable terminal 2. The playback management unit 34 manages the playback state of the streaming data using the playback management table.

  FIG. 4 is a diagram illustrating an example of the reproduction management table. As shown in FIG. 4, the playback management table associates the streaming data ID, the file name, the playback device ID, the transmitted data amount information, the transmitted time stamp, and the data transmission rate information. I manage. The streaming data ID is identification information for identifying the streaming data being reproduced. The file name is information indicating the file name of the streaming data indicated by the streaming data ID.

  The playback device ID is identification information for identifying the device to which the streaming data indicated by the streaming data ID is supplied. That is, the playback device ID is identification information for identifying the device that is playing the streaming data indicated by the streaming data ID. The playback device ID includes a main device ID and a sub device ID. The main device ID is identification information for identifying the main device performing multi-device playback. The sub device ID is identification information for identifying a sub device that performs multi-device playback. When reproduction is performed by a single device, identification information for identifying the device is registered in the main device ID, and nothing is registered in the sub device ID. As identification information for identifying a device, for example, device subscriber identification information such as IMSI (International Mobile Subscriber Identity) in the UMTS (Universal Mobile Telecommunications System) system is used.

  The transmitted data amount information is information indicating the amount of data transmitted to the playback device among the streaming data indicated by the streaming data ID. The transmitted time stamp is information indicating the amount of transmitted data as an elapsed time from the playback start time of streaming data. The playback start time is expressed as hh: mm: ss in 00:00:00, and the elapsed time is It is the counted information. The data transmission rate information is information indicating the data transmission rate from the streaming server 3 of the streaming data indicated by the streaming data ID. The reproduction management unit 34 dynamically manages the transmitted data amount information, the transmitted time stamp, and the data transmission rate information according to the transmission state of the streaming data.

  In this playback management table, for example, the streaming data with the streaming data ID “1” has the file name “aaa.flv” and the main device ID “35 *******”. It is shown that multi-device playback is performed by the device and the device whose sub-device ID is “35 *******”. Further, it is shown that the data amount of the streaming data transmitted is 30000 kbytes, the transmitted time stamp is 27 minutes 25 seconds, and the data transmission rate is 2048 kbps.

  In addition, when receiving the streaming data transmission request for which the continuous reproduction start time is not specified from the second portable terminal 2 that is the sub device, the reproduction management unit 34 transmits the streaming data transmission time corresponding to the streaming data transmission request. The stamp is transmitted to the streaming data management unit 33.

  Functionally, the first portable terminal 1 includes an input unit 11, an output unit 12, a terminal information management unit 13, a communication control unit 14, a buffer control unit 15, a measurement unit 16 (measurement means), A reproduction control unit 17.

  The input unit 11 functions as an input unit that receives a user operation via the input device 106. The input unit 11 transmits operation information regarding the received operation to the reproduction control unit 17. The output unit 12 functions as an output unit that plays back the media element extracted by the playback control unit 17 using the output device 107.

  The terminal information management unit 13 functions as a terminal information management unit that manages the terminal capability (static information) and processing load (dynamic information) of the first mobile terminal 1. The terminal information management unit 13 includes, for example, codec information such as video codec and audio codec supported by the first mobile terminal 1, reproducible rate information such as video reproducible rate and audio reproducible rate, and screen resolution of the output device 107. And the communication protocol information such as the WAN protocol supported by the first portable terminal 1 are managed as the first capability information. In addition, the terminal information management unit 13 manages, for example, load information indicating processing load, memory consumption information indicating memory consumption, battery remaining information indicating remaining battery power, and the like as the first state information. .

  The communication control unit 14 functions as a communication control unit that performs data transmission / reception, session management, and communication bearer control. The communication control unit 14 includes an external NW communication unit 141 and a local NW communication unit 142. The external NW communication unit 141 performs data transmission / reception and session management with the network NW. For example, the external NW communication unit 141 transmits a connection request to the streaming server 3 on the network NW, receives a connection response from the streaming server 3, and establishes a communication connection with the streaming server 3.

  The local NW communication unit 142 performs data transmission / reception and session management with the local network LN. For example, the local NW communication unit 142 transmits a connection request to the second portable terminal 2 on the local network LN when the streaming data can be reproduced by multiple devices. Then, the local NW communication unit 142 receives a connection response from the second mobile terminal 2 and establishes a communication connection with the second mobile terminal 2.

  In addition, the local NW communication unit 142 performs operation information indicating an operation of turning off the power of the first mobile terminal 1 from the input unit 11 and an operation of disconnecting the communication connection between the first mobile terminal 1 and the streaming server 3. When the operation information shown is received, a disconnection request is transmitted to the second portable terminal 2. Further, the local NW communication unit 142 transmits a disconnection request to the second portable terminal 2 when detecting that the communication connection with the streaming server 3 has been disconnected.

  The buffer control unit 15 controls the buffer size of the transmission / reception buffer used for transmission / reception of data with the network NW and the amount of data in the buffer. The buffer control unit 15 controls the buffer size of the transmission / reception buffer used for transmission / reception of data with the local network LN and the amount of data in the buffer. Further, the buffer control unit 15 stores the first media element and the second media element received from the media element extraction unit 173 in the buffer based on a predetermined amount of buffering data. When the buffer control unit 15 receives the first media element transfer request from the second mobile terminal 2, the buffer control unit 15 sends the first media element stored in the buffer to the second mobile terminal 2 via the local NW communication unit 142. Send.

  The measurement unit 16 functions as a measurement unit that measures the data transfer time between the first mobile terminal 1 and the second mobile terminal 2. For example, the measurement unit 16 measures RTT (round trip time), which is the time from when data is transmitted to the second portable terminal 2 until the response is received from the second portable terminal 2. The measurement unit 16 measures the RTT at each reproduction interval (hereinafter simply referred to as “interval”) obtained by dividing the reproduction time of the streaming data by a predetermined time interval. The measurement unit 16 transmits RTT information indicating RTT to the reproduction control unit 17 every time measurement is performed.

  The reproduction control unit 17 functions as a reproduction control unit that controls reproduction of streaming data in the first portable terminal 1. The reproduction control unit 17 includes a data acquisition management unit 171 (first data acquisition unit), a reproduction pattern determination unit 172 (reproduction pattern determination unit), a media element extraction unit 173 (media element extraction unit), and a synchronous reproduction control unit. 174 (first synchronous reproduction control means).

  The data acquisition management unit 171 functions as a first data acquisition unit that acquires streaming data. For example, the data acquisition management unit 171 receives operation information indicating an operation for acquiring streaming data from the input unit 11. Then, the data acquisition management unit 171 transmits a streaming data distribution request to the streaming server 3 via the external NW communication unit 141. Then, the data acquisition management unit 171 receives a streaming data delivery response including the streaming data meta information corresponding to the streaming data delivery request from the streaming server 3 via the external NW communication unit 141.

  Further, the data acquisition management unit 171 transmits a streaming data transmission request to the streaming server 3 via the external NW communication unit 141. Then, the data acquisition management unit 171 receives the streaming data divided data corresponding to the streaming data transmission request from the streaming server 3 via the external NW communication unit 141. The data acquisition management unit 171 transmits the received divided data to the buffer control unit 15 and stores it in the buffer.

  Based on the processing capabilities of the first mobile terminal 1 and the second mobile terminal 2, the playback pattern determination unit 172 is a first mobile terminal among a plurality of media elements included in the streaming data acquired by the data acquisition management unit 171. 1 functions as a reproduction pattern determination means for determining the first media element to be reproduced in 1 and the second media element to be reproduced by the second portable terminal 2. Specifically, the reproduction pattern determination unit 172 sends a capability notification request to the second mobile phone via the local NW communication unit 142, for example, in order to acquire second capability information indicating the processing capability of the second mobile terminal 2. Transmit to terminal 2. Then, the reproduction pattern determination unit 172 receives the second capability information from the second portable terminal 2 via the local NW communication unit 142. Further, the reproduction pattern determination unit 172 acquires the first capability information from the terminal information management unit 13.

  Further, the playback pattern determination unit 172 determines the first media element and the second media element based on the meta information, the first capability information, and the second capability information of the streaming data. For example, when the multi-device information indicates that multi-device playback is not possible, the playback pattern determination unit 172 sets all the media elements constituting the streaming data as the first media element. When the multi-device information indicates that multi-device playback is possible, the playback pattern determination unit 172 determines whether or not the first mobile terminal 1 and the second mobile terminal 2 correspond to the codec of the media element constituting the streaming data. . When the second mobile terminal 2 supports the codec of the media element, the media element is set as the second media element. When the second mobile terminal 2 does not support the media element, the media element is set as the first media element.

  Further, when the first mobile terminal 1 and the second mobile terminal 2 correspond to the codec of the media element constituting the streaming data, the playback pattern determination unit 172 causes the number of media elements or the processing load to be equalized. , It may be included in either the first media element or the second media element. In addition, the reproduction pattern determination unit 172 displays a plurality of candidates for the first media element and the second media element on the output device 107 by the output unit 12, and the first media is selected by the user selecting one of the candidates. The element and the second media element may be determined.

  The media element extraction unit 173 functions as a media element extraction unit that extracts the first media element and the second media element determined by the reproduction pattern determination unit 172 from the streaming data. The media element extraction unit 173 extracts the first media element and the second media element from the divided data in the buffer. Then, the media element extraction unit 173 transmits the extracted first media element and second media element to the buffer control unit 15 and stores them in the buffer.

  The synchronized playback control unit 174 transmits the second media element extracted by the media element extraction unit 173 to the second portable terminal 2, and the playback start time determined based on the data transfer time measured by the measurement unit 16 To the second portable terminal 2, and functions as a first synchronized playback control means for playing back the first media element using the output device 107 at the playback start time. More specifically, the synchronized playback control unit 174 determines the playback start time based on the RTT measured by the measuring unit 16. That is, data equivalent to the RTT from when the first portable terminal 1 transmits a reproduction start request to the second portable terminal 2 until the reproduction start response from the second portable terminal 2 reaches the first portable terminal 1. A transfer delay occurs. For this reason, the synchronized playback control unit 174 determines a playback start time after the time obtained by adding a data transfer delay from the current time.

  Also, the synchronized playback control unit 174 calculates the playback rate of the first mobile terminal 1 and the playback rate of the second mobile terminal 2 based on the first capability information and the second capability information. That is, the synchronized playback control unit 174 calculates a playback rate according to the processing capability of each terminal so that the streaming data can be played back synchronously between the first mobile terminal 1 and the second mobile terminal 2.

  In addition, the synchronized playback control unit 174 adjusts the data transmission rate to the second mobile terminal 2 and the amount of buffering data to be transmitted to the second mobile terminal 2 in advance based on the RTT measured by the measurement unit 16. . That is, the synchronized playback control unit 174 determines the data transmission rate, the transmission data amount, and the buffering data amount to the second mobile terminal 2 so that the data to be played back in the second mobile terminal 2 is not insufficient due to the data transfer delay To do.

  In addition, the synchronized playback control unit 174 transmits a playback start request specifying the playback start time, playback rate, data transmission rate, and transmission data amount to the second portable terminal 2 via the local NW communication unit 142. Further, the synchronized playback control unit 174 receives a playback start response from the second portable terminal 2 via the local NW communication unit 142. Then, when the received playback start response indicates that playback is possible, the synchronized playback control unit 174 causes the buffer control unit 15 to transmit the first media element stored in the buffer to the output unit 12, and sends a playback start request. Is reproduced by the output device 107 at the reproduction start time designated in FIG.

  When the received playback start response indicates that playback is possible, the synchronized playback control unit 174 uses the determined data transmission rate and transmission data amount to store the second media element stored in the buffer in the second mobile terminal. 2 to send. In addition, the synchronized playback control unit 174 transmits the second media elements for the determined buffering data amount to the second portable terminal 2 via the local NW communication unit 142 until the playback start request is transmitted. At this time, the synchronized playback control unit 174 adds the URL information of the streaming data and transmits the second media element to the second portable terminal 2. In addition, the synchronized playback control unit 174 transmits the second media element to the second mobile terminal 2 based on the determined data transmission rate and transmission data amount after starting the synchronized playback.

  Although the description is omitted in the above description, since the second media element extracted by the media element extraction unit 173 is stored in the buffer, the synchronized playback control unit 174 performs the second media element stored in the buffer. The element is acquired by the buffer control unit 15, and the second media element is transmitted to the second portable terminal 2 via the local NW communication unit 142.

  In addition, the synchronized playback control unit 174 calculates an interval playback end time, which is a time when playback of the interval ends, every time playback of each interval of streaming data ends in the first mobile terminal 1. In addition, the synchronized playback control unit 174 indicates an interval playback end time that is a time when playback of each interval of streaming data ends in the second mobile terminal 2 from the second mobile terminal 2 via the local NW communication unit 142. Receive information. Then, the synchronized playback control unit 174 compares the nth (n is an integer greater than or equal to 1) interval playback end time of the first mobile terminal 1 with the nth interval playback end time of the second mobile terminal 2, and calculates the difference. calculate. Then, the synchronized playback control unit 174 calculates the playback rate at the (n + 2) th interval of the second mobile terminal 2 so that the calculated difference is reduced in order to eliminate the synchronization shift.

  For example, when the n-th interval playback end time of the first mobile terminal 1 is later than the n-th interval playback end time of the second mobile terminal 2, the playback rate of the first mobile terminal 1 is decreased. In addition, when the n-th interval playback end time of the first mobile terminal 1 is earlier than the n-th interval playback end time of the second mobile terminal 2, the playback rate of the second mobile terminal 2 is decreased. Then, the synchronized playback control unit 174 instructs the playback rate at the (n + 2) th interval to the second portable terminal 2.

  Further, the synchronized playback control unit 174 may adjust the data transmission rate and the transmission data amount to the second portable terminal 2 based on the RTT measured by the measurement unit 16 during the streaming data playback.

  Functionally, the second portable terminal 2 includes an input unit 21, an output unit 22, a terminal information management unit 23, a communication control unit 24, a buffer control unit 25, a reproduction control unit 27, and a disconnection detection unit. 28.

  The input unit 21 functions as an input unit that receives a user operation via the input device. The input unit 21 transmits operation information regarding the received operation to the reproduction control unit 27. The output unit 22 functions as output means for playing back the second media element or streaming data by the output device 107.

  The terminal information management unit 23 functions as a terminal information management unit that manages the terminal capability (static information) and processing load (dynamic information) of the second portable terminal 2. The terminal information management unit 23 includes, for example, codec information such as video codec and audio codec supported by the second mobile terminal 2, reproducible rate information such as video reproducible rate and audio reproducible rate, and screen resolution of the output device. The resolution information shown and communication protocol information such as the WAN protocol supported by the second portable terminal 2 are managed as the second capability information.

  In addition, the terminal information management unit 23 manages, for example, load information indicating processing load, memory consumption information indicating memory consumption, battery remaining information indicating battery remaining amount, and the like as second state information. . When receiving the capability notification request from the first mobile terminal 1 via the local NW communication unit 242, the terminal information management unit 23 receives capability information including the terminal capability of the second mobile terminal 2 via the local NW communication unit 242. 1 Transmit to the mobile terminal 1.

  The communication control unit 24 functions as a communication control unit that performs data transmission / reception, session management, and communication bearer control. The communication control unit 24 includes an external NW communication unit 241 and a local NW communication unit 242. The external NW communication unit 241 performs data transmission / reception and session management with the network NW. For example, the external NW communication unit 241 transmits a connection request to the streaming server 3 on the network NW, receives a connection response from the streaming server 3, and establishes a communication connection with the streaming server 3. The local NW communication unit 242 performs data transmission / reception and session management with the local network LN. When the local NW communication unit 242 receives a connection request from the first mobile terminal 1, the local NW communication unit 242 transmits a connection response indicating whether or not connection is possible.

  The buffer control unit 25 controls the buffer size of the transmission / reception buffer used for transmission / reception of data with the network NW and the amount of data in the buffer. The buffer control unit 25 controls the buffer size of the transmission / reception buffer used for data transmission / reception with the first portable terminal 1 via the local network LN and the amount of data in the buffer. Further, the buffer control unit 25 stores the second media element and streaming data acquired by the data acquisition management unit 271 in the buffer. Further, the buffer control unit 25 receives the instruction to discard the buffered data from the continuous reproduction control unit 275 and discards the data in the buffer.

  The reproduction control unit 27 functions as a reproduction control unit that controls reproduction of streaming data in the second portable terminal 2. The reproduction control unit 27 includes a data acquisition management unit 271 (second data acquisition unit), a synchronous reproduction control unit 274 (second synchronous reproduction control unit), and a continuous reproduction control unit 275 (continuous reproduction control unit). ing.

  The data acquisition management unit 271 functions as a second data acquisition unit that acquires streaming data to be played back on the second portable terminal 2. The data acquisition management unit 271 receives URL information of the second media element and streaming data from the first mobile terminal 1 via the local NW communication unit 242. Then, the data acquisition management unit 271 transmits the received second media element to the buffer control unit 25 and stores it in the buffer.

  In addition, the data acquisition management unit 271 performs the unreproduced portion of the streaming data based on the URL information received from the first mobile terminal 1 when the disconnection detection unit 28 described below detects the disconnection state of the first mobile terminal 1. To get. Specifically, when the disconnection detection unit 28 determines that the communication between the first mobile terminal 1 and the streaming server 3 is disconnected, the data acquisition management unit 271 is based on the URL information received from the first mobile terminal 1. Then, the streaming data distribution request is transmitted to the streaming server 3 via the external NW communication unit 241. Then, the data acquisition management unit 271 receives a streaming data delivery response including the streaming data meta information corresponding to the streaming data delivery request from the streaming server 3 via the external NW communication unit 241.

  Further, the data acquisition management unit 271 notifies the streaming server 3 of the playback rate determined by the continuous playback control unit 275 described later. Then, the data acquisition management unit 271 receives from the streaming server 3 a data transmission rate response in which the data transmission rate and the transmission data size are specified. Then, the data acquisition management unit 271 transmits a streaming data transmission request to the streaming server 3 via the external NW communication unit 141. Then, the data acquisition management unit 271 receives the streaming data divided data corresponding to the streaming data transmission request from the streaming server 3 via the external NW communication unit 241. Then, the data acquisition management unit 271 transmits the received divided data to the buffer control unit 25 and stores it in the buffer.

  Note that the data acquisition management unit 271 may transmit the streaming data transmission request to the streaming server 3 by specifying the continuous reproduction start time. In this case, the data acquisition management unit 271 uses the time indicated by the time stamp notified from the continuous playback control unit 275 as the continuous playback start time, and transmits the streaming data transmission request by specifying the continuous playback start time.

  In addition, when the disconnection detection unit 28 detects the disconnection state of the first mobile terminal 1, the data acquisition management unit 271 transmits the first media element that is not reproduced on the first mobile terminal 1 from the first mobile terminal 1. You may get it. In this case, when the data acquisition management unit 271 determines that the communication between the first portable terminal 1 and the streaming server 3 has been disconnected by the disconnection detection unit 28, the data acquisition management unit 271 acquires the buffered first media element. A first media element transfer request is transmitted to the first portable terminal 1. When the data acquisition management unit 271 receives the buffered first media element from the first portable terminal 1, the data acquisition management unit 271 transmits the received first media element to the buffer control unit 25 and stores it in the buffer.

  The synchronized playback control unit 274 functions as a second synchronized playback control unit that plays back the second media element received by the data acquisition management unit 271 based on the playback start time notified by the synchronized playback control unit 174. When the synchronized playback control unit 274 receives a playback start request specifying the playback start time, playback rate, data transmission rate, and transmission data amount from the synchronized playback control unit 174 via the local NW communication unit 242, the synchronized playback control unit 274 starts playback. A reproduction start response including availability information is transmitted.

  Further, the synchronized playback control unit 274 causes the buffer control unit 25 to transmit the second media element stored in the buffer to the output unit 22. Then, the synchronized playback control unit 274 causes the output device to play the second media element at the playback start time specified in the playback start request. At this time, the synchronized playback control unit 274 plays back the second media element at the playback rate specified in the playback start request.

  In addition, the synchronized playback control unit 274 calculates an interval playback end time, which is a time when playback of the interval ends, every time playback of each interval of the streaming data ends in the second portable terminal 2. Then, the synchronized playback control unit 274 transmits information indicating the interval playback end time to the first mobile terminal 1.

  The continuous playback control unit 275 controls to continuously play streaming data on the second mobile terminal 2 when a disconnection state of the first mobile terminal 1 is detected by a disconnection detection unit 28 described later. Functions as a means. The continuous reproduction control unit 275 reproduces the unreproduced portion of the streaming data acquired by the data acquisition management unit 271. Specifically, the continuous playback control unit 275 acquires an unplayed portion of the streaming data acquired by the data acquisition management unit 271 from the buffer, and causes the output unit 22 to play back.

  When the data acquisition management unit 271 acquires from the first mobile terminal 1 the first media element that has not been played back on the first mobile terminal 1, the continuous playback control unit 275 extracts the first media element from the buffer. get. Then, the continuous playback control unit 275 combines the first media element and the buffered second media element that has not been played back on the second mobile terminal 2, and reconstructs the divided data of the streaming data.

  Further, the continuous playback control unit 275 acquires the second capability information from the terminal information management unit 23, and determines the playback rate of the streaming data to be continuously played based on the second capability information. Here, the streaming data to be continuously reproduced includes the divided data reconstructed by the continuous reproduction control unit 275 and the streaming data acquired from the streaming server 3 by the data acquisition management unit 271.

  Further, the continuous playback control unit 275 determines whether the streaming data to be continuously played back includes media elements exceeding the processing capability of the second portable terminal 2 based on the second capability information. If the continuous playback control unit 275 determines that the streaming data to be continuously played back includes a media element exceeding the processing capability of the second portable terminal 2, the media element is discarded. For example, when the audio data of the codec that is not supported by the second portable terminal 2 is included in the streaming data to be continuously reproduced, the continuous reproduction control unit 275 discards the audio data.

  The continuous reproduction control unit 275 causes the output unit 22 to reproduce the reconstructed divided data. At this time, the continuous reproduction control unit 275 notifies the data acquisition management unit 271 of the time stamp of the unacquired divided data. The continuous reproduction control unit 275 causes the output unit 22 to reproduce the divided data received from the streaming server 3 after reproducing the reconstructed divided data.

  Further, when the continuous playback control unit 275 cannot receive the buffered first media element from the first mobile terminal 1, the continuous playback control unit 275 determines that the first mobile terminal 1 is in an unusable state, and stores the buffer in the buffer control unit 25. Instruct to discard the second (unplayed) second media element. At this time, the continuous reproduction control unit 275 may notify the data acquisition management unit 271 of the time stamp of the unreproduced divided data.

  The disconnection detection unit 28 functions as a disconnection detection unit that detects whether or not the first mobile terminal 1 is in a disconnection state in which streaming data cannot be acquired from the streaming server 3. For example, the disconnection detection unit 28 determines that the communication between the first mobile terminal 1 and the streaming server 3 has been disconnected by receiving a disconnection request from the first mobile terminal 1. Further, the disconnection detection unit 28 detects that the data transfer rate of the data received from the first mobile terminal 1 is reduced, thereby disconnecting the communication between the first mobile terminal 1 and the streaming server 3. It is determined that

  Next, the reproduction state of streaming data in the information reproduction system 10 having the above-described configuration will be described. FIG. 5A is a diagram illustrating a playback state in which the first mobile terminal 1 and the second mobile terminal 2 are performing synchronous playback. FIG. 5B is a diagram illustrating a state in which the first mobile terminal 1 is disconnected and the second mobile terminal 2 is connected. It is a figure which shows the reproduction | regeneration state which is reproducing | regenerating independently. As shown in FIG. 5A, in a state where the first portable terminal 1 and the second portable terminal 2 are synchronously playing streaming data (first state ST1), the streaming data is distributed from the streaming server 3. The first portable terminal 1 is receiving streaming data.

  The first mobile terminal 1 establishes a connection with the local network LN with the second mobile terminal 2. Then, the first mobile terminal 1 separates the received streaming data into a plurality of media elements, and extracts a first media element to be played back on the first mobile terminal 1 and a second media element to be played back on the second mobile terminal 2 To do. Then, the first mobile terminal 1 stores the first media element in the buffer and transfers the second media element to the second mobile terminal 2. On the other hand, the second portable terminal 2 stores the received second media element in the buffer. Thereafter, the first mobile terminal 1 performs synchronous playback control and playback rate adjustment with the second mobile terminal 2, and performs synchronous playback together with the second mobile terminal 2.

  On the other hand, as shown in FIG. 5B, the communication between the first portable terminal 1 and the streaming server 3 is disconnected due to user selection, disconnection of the wireless WAN, or the like (second state ST2). First, the second mobile terminal 2 detects that the transfer rate of data transmitted from the first mobile terminal 1 is continuously decreasing, or receives a disconnection request from the first mobile terminal 1. Then, playback of the second media element is stopped. Then, the second mobile terminal 2 transmits a first media element transfer request to the first mobile terminal 1.

  Thereafter, when the second mobile terminal 2 receives the first media element from the first mobile terminal 1, the second mobile terminal 2 combines the second media element buffered in the second mobile terminal 2 and regenerates the divided data of the streaming data. Configure. Then, the second mobile terminal 2 adjusts the playback rate according to the processing capability of the second mobile terminal 2, and starts playback of the reconstructed divided data. Next, the second mobile terminal 2 accesses the streaming server 3 based on the URL information received from the first mobile terminal 1 and requests the streaming server 3 for data that has not been played back and has not been received from the first mobile terminal 1. To do. Further, when the second mobile terminal 2 cannot receive the first media element from the first mobile terminal 1, the second mobile terminal 2 requests the streaming server 3 for data after the stop of playback.

  Then, the second mobile terminal 2 negotiates with the streaming server 3. At this time, the data transmission rate, the buffered data size, and the reproduction rate are determined in consideration of the processing capability of the second portable terminal 2 and the amount of buffered data. Then, when communication with the streaming server 3 is established, the second mobile terminal 2 starts playback from the playback stop position.

  Next, an information reproduction method in the information reproduction system 10 will be described with reference to FIGS. FIG. 6 is a sequence diagram showing a connection establishment (negotiation) process of the information reproduction system 10. This process is started when the first portable terminal 1 receives operation information indicating an operation for acquiring streaming data from the input unit 11.

  The communication between the first portable terminal 1 and the second portable terminal 2 is performed via the local NW communication unit 142 and the local NW communication unit 242, and the communication between the first portable terminal 1 and the streaming server 3 is external. The communication between the second mobile terminal 2 and the streaming server 3 is performed via the external NW communication unit 241 and the communication control unit 31, and is performed via the NW communication unit 141 and the communication control unit 31. In the description of 9, this is omitted.

  As shown in FIG. 6, first, the data acquisition management unit 171 of the first portable terminal 1 transmits a streaming data distribution request to the streaming server 3 (step S01). When the streaming data management unit 33 of the streaming server 3 receives the streaming data distribution request from the first mobile terminal 1, the streaming data distribution response including the meta information of the streaming data corresponding to the streaming data distribution request is sent to the first mobile terminal. 1 (step S02).

  Next, when receiving a streaming data delivery response from the streaming server 3, the data acquisition management unit 171 transmits meta information to the reproduction pattern determination unit 172. Then, the playback pattern determination unit 172 determines whether or not the multi-device information included in the meta information indicates that multi-device playback is possible (step S03).

  When the multi-device information indicates that multi-device playback is possible, the local NW communication unit 142 transmits a connection request to the second portable terminal 2 on the local network LN (step S04). When the local NW communication unit 242 of the second portable terminal 2 receives the connection request from the first portable terminal 1, it transmits a connection response indicating whether or not connection is possible. Then, the local NW communication unit 142 receives a connection response indicating that connection is possible from the second portable terminal 2, and establishes a communication connection with the second portable terminal 2 (step S05).

  When the communication connection between the first mobile terminal 1 and the second mobile terminal 2 is established, the reproduction pattern determination unit 172 transmits a capability notification request to the second mobile terminal 2 (step S06). When receiving the capability notification request from the first portable terminal 1, the terminal information management unit 23 of the second portable terminal 2 transmits the second capability information to the first portable terminal 1 (step S07). Then, the reproduction pattern determination unit 172 receives the second capability information from the second portable terminal 2. Further, the reproduction pattern determination unit 172 acquires the first capability information from the terminal information management unit 13.

  Next, the playback pattern determination unit 172 determines the first media element and the second media element based on the meta information of the streaming data, the first capability information, and the second capability information (step S08, playback pattern determination step). ). For example, the playback pattern determination unit 172 determines whether the first mobile terminal 1 and the second mobile terminal 2 are compatible with the codec of the media element constituting the streaming data, for example. To determine which media elements can be played. Then, the playback pattern determination unit 172 sets a media element that can be played back on the first mobile terminal 1 as a first media element and a media element that can be played back on the second mobile terminal 2 as a second media element. In addition, when there are a plurality of combinations of the first media element and the second media element, the reproduction pattern determination unit 172 displays the combination candidates of the first media element and the second media element on the output device 107 by the output unit 12. To do. Then, the playback pattern determination unit 172 determines the first media element and the second media element by selecting one of the candidates by the user.

  On the other hand, the data acquisition management unit 171 transmits a streaming data transmission request to the streaming server 3 (step S09). When receiving the streaming data transmission request from the first mobile terminal 1, the streaming data management unit 33 acquires the streaming data corresponding to the streaming data transmission request from the streaming data storage unit 32. Then, the streaming data management unit 33 divides the streaming data into divided data having a predetermined data size in the order of reproduction.

  Subsequently, the streaming data management unit 33 transmits the divided data to the first portable terminal 1 in the order of reproduction (step S10). Then, the data acquisition management unit 171 receives the divided data of the streaming data corresponding to the streaming data transmission request from the streaming server 3. The data acquisition management unit 171 transmits the received divided data to the buffer control unit 15 and stores it in the buffer (first data acquisition step). Note that steps S09 and S10 may be performed in parallel with steps S03 to S08.

  Next, the media element extraction unit 173 extracts the first media element and the second media element determined by the reproduction pattern determination unit 172 from the divided data in the buffer (step S12, media element extraction step). Then, the media element extraction unit 173 transmits the extracted first media element and second media element to the buffer control unit 15 and stores them in the buffer.

  Thereafter, the synchronized playback control unit 174 transmits the second media element in the buffer to the second portable terminal 2 (step S13). Further, the synchronous playback control unit 174 transmits the URL information (content URL) of the streaming data together with the second media element (step S14). The data acquisition management unit 271 of the second mobile terminal 2 receives the second media element and the URL information from the first mobile terminal 1, transmits the received second media element to the buffer control unit 25, and stores it in the buffer. (Step S15).

  FIG. 7 is a sequence diagram showing a synchronized playback control process in the first state ST1 of the information playback system 10. This process follows the process of FIG.

  As shown in FIG. 7, the streaming data management unit 33 transmits the divided data to the first portable terminal 1 in the order of reproduction, and the data acquisition management unit 171 receives the divided data from the streaming server 3 (Step S21, the second). 1 data acquisition step). This step S21 is the same process as step S10 in FIG. 6, and is repeatedly performed at regular time intervals until the transmission of the streaming data is completed. The subsequent processing is performed in parallel with this step S21.

  The measurement unit 16 of the first mobile terminal 1 measures the RTT for the second mobile terminal 2 (step S22, measurement step). The synchronized playback control unit 174 determines the data transmission rate and the buffering data amount based on the RTT measured in step S22 so that the data to be played back in the second portable terminal 2 is not short (step S23). In addition, the synchronized playback control unit 174 determines a playback start time after a time obtained by adding RTT to the current time. The synchronous playback control unit 174 also plays back the playback rate of the first mobile terminal 1 and the playback rate of the second mobile terminal 2 based on the first capability information and the second capability information so that the streaming data can be played back synchronously. And calculate.

  Then, the synchronized playback control unit 174 transmits the URL information of the streaming data together with the second media element, and the data acquisition management unit 271 transmits the second media element received from the first portable terminal 1 to the buffer control unit 25. Then, it is stored in the buffer (step S24, second data acquisition step). This step S24 is the same processing as step S13 to step S15 of FIG. Step S24 is repeatedly performed according to the data transmission rate determined in step S23 or the like.

  Next, the synchronized playback control unit 174 transmits a playback start request specifying the playback start time, playback rate, data transmission rate, and transmission data amount to the second mobile terminal 2 (step S25). Then, when receiving the reproduction start request from the synchronous reproduction control unit 174, the synchronous reproduction control unit 274 transmits a reproduction start response including information on whether or not reproduction can be started (step S26).

  When the synchronized playback control unit 174 receives a playback start response indicating that playback can be started from the second mobile terminal 2, the synchronized playback control unit 174 causes the buffer control unit 15 to transmit the first media element stored in the buffer to the output unit 12. Then, the output device 107 is caused to reproduce at the reproduction start time designated in the reproduction start request (first synchronous reproduction step). In addition, the synchronous playback control unit 274 causes the buffer control unit 25 to transmit the second media element stored in the buffer to the output unit 22, and plays it on the output device at the playback start time specified in the playback start request (step S27, second synchronous reproduction step).

  Thereafter, the synchronized playback control unit 174 causes the output device 107 to play back the first media element stored in the buffer according to the playback rate of the first mobile terminal 1. In addition, the synchronized playback control unit 274 causes the output device to play back the second media element stored in the buffer according to the playback rate of the second mobile terminal 2. Subsequent processing is performed during this synchronous reproduction, and is performed in units of intervals.

  The measuring unit 16 measures RTT in the first interval (step S28). Thereafter, when the first mobile terminal 1 finishes reproducing the first interval, the synchronized reproduction control unit 174 calculates a first interval reproduction end time, which is a time when the reproduction of the first interval ends (step S29).

  On the other hand, the synchronized playback control unit 274 calculates the first interval playback end time when the playback of the first interval is completed in the second portable terminal 2 (step S30). Then, the synchronized playback control unit 274 transmits information indicating the first interval playback end time to the first mobile terminal 1, and the synchronized playback control unit 174 transmits information indicating the first interval playback end time to the second mobile terminal 2. (Step S31). Moreover, the measurement part 16 measures RTT in a 2nd interval (step S32). Then, the synchronous reproduction control unit 174 adjusts the data transmission rate and the transmission data size based on the RTT measured in step S32 so that the data to be reproduced in the second portable terminal 2 is not short (step S33). .

  Next, the synchronized playback control unit 174 compares the first interval playback end time of the first mobile terminal 1 with the first interval playback end time of the second mobile terminal 2, and calculates the difference. Then, the synchronous playback control unit 174 calculates the playback rate at the third interval of the second mobile terminal 2 so that the calculated difference is small in order to eliminate the synchronization shift (step S34). Then, the synchronized playback control unit 174 instructs the playback rate at the third interval to the second portable terminal 2 (step S35).

  Thereafter, when the reproduction of the second interval is completed in the first mobile terminal 1, the synchronous reproduction control unit 174 calculates a second interval reproduction end time that is a time when the reproduction of the second interval is completed (step S36). On the other hand, the synchronized playback control unit 274 calculates the second interval playback end time when the second mobile terminal 2 finishes playing the second interval (step S37).

  Thereafter, the same processing as Steps S31 to S37 in the second interval is repeated in the third interval (Steps S38 to S44). In subsequent intervals, the same processing as in steps S31 to S37 is repeated, and the synchronized reproduction is continued.

  FIG. 8 is a sequence diagram illustrating an example of the continuous reproduction control process in the second state ST2 of the information reproduction system 10. As shown in FIG. 8, this process is started when the first mobile terminal 1 is disconnected from the communication connection with the streaming server 3 due to reasons unintended by the user, such as disconnection of radio (step S51). . When the communication connection with the first portable terminal 1 is disconnected during data transmission to the first portable terminal 1, the communication control unit 31 of the streaming server 3 stores the transmission data in the buffer and suspends the data transmission. (Step S52). Then, the reproduction management unit 34 stores the transmitted time stamp in the reproduction management table.

  When the local NW communication unit 142 of the first mobile terminal 1 detects that the communication connection with the streaming server 3 is disconnected, the local NW communication unit 142 transmits a disconnection request to the second mobile terminal 2 (step S53). And the cutting | disconnection detection part 28 of the 2nd portable terminal 2 determines with the communication with the 1st portable terminal 1 and the streaming server 3 having been cut | disconnected by having received the cutting | disconnection request | requirement from the 1st portable terminal 1 (step S54). . In step S <b> 53, the first mobile terminal 1 may not transmit a disconnection request to the second mobile terminal 2. In this case, the disconnection detection unit 28 detects that the data transfer rate of the data received from the first mobile terminal 1 has decreased, thereby disconnecting the communication between the first mobile terminal 1 and the streaming server 3. It is determined that it has been performed (step S54).

  If it is determined in step S54 that the communication between the first mobile terminal 1 and the streaming server 3 has been disconnected, the data acquisition management unit 271 acquires the buffered first media element in the first mobile terminal 1. The first media element transfer request is transmitted to the first portable terminal 1 (step S55). When the buffer control unit 15 receives the first media element transfer request from the second mobile terminal 2, the buffer control unit 15 transmits the first media element stored in the buffer to the second mobile terminal 2 (step S56). Then, when the transfer of the first media element stored in the buffer is completed, the local NW communication unit 142 disconnects the communication connection (connection) with the second mobile terminal 2.

  When the data acquisition management unit 271 receives the buffered first media element from the first portable terminal 1, the data acquisition management unit 271 transmits the first media element to the buffer control unit 25 and stores it in the buffer. Thereafter, the continuous playback control unit 275 combines the buffered second media element in the second portable terminal 2 and the received first media element to reconstruct the divided data of the streaming data (step S57). Subsequently, the continuous playback control unit 275 acquires the second capability information from the terminal information management unit 23, and determines the playback rate of the reconstructed divided data based on the second capability information.

  In addition, the continuous playback control unit 275 determines whether or not a media element exceeding the processing capability of the second mobile terminal 2 is included in the reconstructed divided data based on the second capability information. Then, when it is determined that the reconstructed divided data includes a media element that exceeds the processing capability of the second portable terminal 2, the continuous playback control unit 275 discards the media element (step S58). . Then, the continuous reproduction control unit 275 causes the output unit 22 to reproduce the reconstructed divided data (step S59). At this time, the continuous reproduction control unit 275 may notify the data acquisition management unit 271 of the time stamp of the unacquired divided data.

  Next, the data acquisition management unit 271 transmits a streaming data distribution request to the streaming server 3 based on the URL information received from the first portable terminal 1 (step S60). When the streaming data management unit 33 of the streaming server 3 receives the streaming data distribution request from the second mobile terminal 2, the streaming data distribution response including the streaming data meta information corresponding to the streaming data distribution request is sent to the second mobile terminal 2. (Step S61).

  Subsequently, when receiving the streaming data delivery response from the streaming server 3, the data acquisition management unit 271 notifies the streaming server 3 of the playback rate determined by the continuous playback control unit 275 (step S62). When receiving the notification of the playback rate from the second portable terminal 2, the streaming data management unit 33 acquires a data rate that can be provided by the streaming server 3 from the streaming data management table. Then, the streaming data management unit 33 determines the data transmission rate and the transmission data size based on the reproduction rate notified in step S62. Thereafter, the streaming data management unit 33 transmits a data transmission rate response designating the data transmission rate and the transmission data size to the second portable terminal 2 (step S63).

  Next, when receiving the data transmission rate response from the streaming server 3, the data acquisition management unit 271 transmits a streaming data transmission request to the streaming server 3 (step S64). When the streaming data management unit 33 receives the streaming data transmission request from the second portable terminal 2, the streaming data management unit 33 acquires the transmitted time stamp from the reproduction management unit 34. Then, the streaming data management unit 33 transmits the divided data after the time indicated by the transmitted time stamp to the second portable terminal 2 in the order of reproduction (step S65).

  Next, the data acquisition management unit 271 receives the divided data of the streaming data corresponding to the streaming data transmission request from the streaming server 3. Subsequently, the data acquisition management unit 271 transmits the received divided data to the buffer control unit 25 and stores it in the buffer. Then, after the reproduction of the reconstructed divided data is finished, the continuous reproduction control unit 275 causes the output unit 22 to reproduce the divided data received from the streaming server 3 according to the reproduction rate notified in step S62.

  In step S63, the streaming data management unit 33 may measure the RTT for the second portable terminal 2 and determine the data transmission rate and the transmission data size in consideration of the RTT. In addition, the second mobile terminal 2 may determine the buffering data size in consideration of RTT.

  FIG. 9 is a sequence diagram illustrating another example of the continuous reproduction control process in the second state ST2 of the information reproduction system 10. As shown in FIG. 9, this process is started when the user intentionally performs an operation of turning off the power of the first portable terminal 1 (step S71). When the communication connection with the first portable terminal 1 is disconnected during data transmission to the first portable terminal 1 (step S72), the communication control unit 31 of the streaming server 3 stores the transmission data in the buffer and stores the data. Transmission is suspended (step S73). Then, the reproduction management unit 34 stores the transmitted time stamp in the reproduction management table.

  Moreover, the local NW communication part 142 of the 1st portable terminal 1 will transmit a cutting | disconnection request | requirement to the 2nd portable terminal 2, if operation which turns off the power supply of the 1st portable terminal 1 is detected (step S74). And the cutting | disconnection detection part 28 of the 2nd portable terminal 2 determines with the communication with the 1st portable terminal 1 and the streaming server 3 having been cut | disconnected by having received the cutting | disconnection request | requirement from the 1st portable terminal 1 (step S75). . If it is determined in step S75 that the communication between the first mobile terminal 1 and the streaming server 3 has been disconnected, the data acquisition management unit 271 acquires the buffered first media element in the first mobile terminal 1. The first media element transfer request is transmitted to the first portable terminal 1 (step S76).

  When the user intentionally performs an operation to turn off the power of the first mobile terminal 1, the power of the first mobile terminal 1 is turned off. The element cannot be transmitted to the second mobile terminal 2. For this reason, since the data acquisition management unit 271 cannot receive the buffered first media element from the first portable terminal 1, the continuous reproduction control unit 275 determines that the first portable terminal 1 is in an unusable state. (Step S77).

  Then, the continuous playback control unit 275 instructs the buffer control unit 25 to discard the buffered (unplayed) second media element. When receiving an instruction to discard the buffered data from the continuous reproduction control unit 275, the buffer control unit 25 discards the second media element in the buffer (step S78). At this time, the continuous reproduction control unit 275 notifies the data acquisition management unit 271 of the time stamp of the unreproduced divided data.

  Next, the data acquisition management unit 271 transmits a streaming data distribution request to the streaming server 3 based on the URL information received from the first portable terminal 1 (step S79). When the streaming data management unit 33 of the streaming server 3 receives the streaming data distribution request from the second mobile terminal 2, the streaming data distribution response including the streaming data meta information corresponding to the streaming data distribution request is sent to the second mobile terminal 2. (Step S80).

  When the data acquisition management unit 271 receives the streaming data delivery response from the streaming server 3, the continuous playback control unit 275 compares the data rate information with the second capability information, and determines a lower data rate as the playback rate. . Next, the data acquisition management unit 271 notifies the streaming server 3 of the playback rate determined by the continuous playback control unit 275 (step S81).

  When receiving the notification of the playback rate from the second portable terminal 2, the streaming data management unit 33 acquires a data rate that can be provided by the streaming server 3 from the streaming data management table. Then, the streaming data management unit 33 determines the data transmission rate and the transmission data size based on the reproduction rate notified in step S62. Thereafter, the streaming data management unit 33 transmits a data transmission rate response specifying the data transmission rate and the transmission data size to the second portable terminal 2 (step S82).

  Then, when receiving the data transmission rate response from the streaming server 3, the data acquisition management unit 271 transmits a streaming data transmission request to the streaming server 3 (step S83). At this time, in the streaming data transmission request, the data acquisition management unit 271 specifies the time indicated by the time stamp notified in step S78 as the continuous reproduction start time. When receiving the streaming data transmission request from the second portable terminal 2, the streaming data management unit 33 acquires the continuous reproduction start time specified in the streaming data transmission request. Then, the streaming data management unit 33 transmits the divided data after the continuous reproduction start time to the second portable terminal 2 in the reproduction order (Step S84).

  Then, the data acquisition management unit 271 receives the divided data of the streaming data corresponding to the streaming data transmission request from the streaming server 3. Subsequently, the data acquisition management unit 271 transmits the received divided data to the buffer control unit 25 and stores it in the buffer. Then, the continuous playback control unit 275 stores the necessary data amount (buffering data size) in the buffer, and then outputs the divided data stored in the buffer according to the playback rate notified in step S81. To play.

  In step S82, the streaming data management unit 33 may measure the RTT for the second portable terminal 2 and determine the data transmission rate and the transmission data size in consideration of the RTT. In addition, the second mobile terminal 2 may determine the buffering data size in consideration of RTT.

  Next, transition of the screens of the first mobile terminal 1 and the second mobile terminal 2 will be described with reference to FIGS. 10 and 11 are diagrams illustrating examples of screen transitions of the first mobile terminal and the second mobile terminal during multi-device playback.

  As shown in FIG. 10, while the first mobile terminal 1 is negotiating with the streaming server 3 (corresponding to steps S01 to S02 in FIG. 6), the first mobile terminal 1 is “buffering”. The screen A01 on which the message is displayed is output. At this time, the second portable terminal 2 outputs a screen B01 on which nothing is displayed (step S101). Subsequently, when the streaming data is capable of multi-device playback, the first mobile terminal 1 outputs a screen A02 on which a message “Do you use multi-device playback? [Y / N]” is displayed. At this time, the second portable terminal 2 outputs a screen B02 on which nothing is displayed (step S102).

  When the user selects “Y” on the screen A02 (corresponding to step S03 in FIG. 6), the first mobile terminal 1 displays a screen A03 on which a message such as “Please connect the device. [OK]” is displayed. Is output. At this time, the second portable terminal 2 outputs a screen B03 on which nothing is displayed (step S103). Then, the user establishes a connection between the first portable terminal 1 and the second portable terminal 2 using Bluetooth or the like, and selects “OK” on the screen A03 (corresponding to step S04 to step S05 in FIG. 6).

  Subsequently, after acquiring the second capability information (corresponding to step S06 to step S07 in FIG. 6), the first portable terminal 1 outputs a screen A04 on which reproduction pattern candidates are displayed. At this time, the second portable terminal 2 outputs a screen B04 on which a message such as “Waiting ... (select playback pattern on the first portable terminal 1)” is displayed (step S104).

  Subsequently, as shown in FIG. 11, when candidate 1 is selected from screen A04 in step S104 (corresponding to step S08 in FIG. 6), first mobile terminal 1 “plays video. [OK] ] "Is displayed on the screen A05. On the other hand, the second portable terminal 2 outputs a screen B05 on which a message such as “Play voice and text. [OK]” is displayed (step S105).

  When the user selects “OK” on the screen A05 and the screen B05, the first media element and the second media element are extracted from the divided data of the streaming data, and the first media element is stored in the buffer of the first mobile terminal 1. Is stored, and the second media element is stored in the buffer of the second portable terminal 2. Further, the data transmission rate and the like are determined (corresponding to steps S12 to S15 in FIG. 6 and steps S21 to S24 in FIG. 7). During this time, the first mobile terminal 1 and the second mobile terminal 2 output screens A06 and B06 on which a message “Preparing for playback” is displayed (step S106).

  Next, when preparation for playback is completed, the first mobile terminal 1 and the second mobile terminal 2 output screens A07 and B07 on which a message such as “Start playback [OK]” is displayed (step S107). ). Then, when the user selects “OK” on the screen A07 and the screen B07, video data is reproduced on the first portable terminal 1, and audio data and text data are reproduced on the second portable terminal 2 (step S25 in FIG. 7). To step S27). At this time, the first mobile terminal 1 outputs the video data playback screen A08, and the second mobile terminal 2 outputs the audio data and text data playback screen B08 (step S108).

  FIG. 12 is a diagram illustrating an example of screen transitions of the first mobile terminal and the second mobile terminal when the first mobile terminal 1 is disconnected. As shown in FIG. 12, when the communication connection between the first mobile terminal 1 and the streaming server 3 is disconnected (corresponding to steps S51 to S54 in FIG. 8), the first mobile terminal 1 “disconnects from the server”. The screen A11 on which a message such as “Continue playback on the second mobile terminal 2” is displayed is output. On the other hand, the second portable terminal 2 outputs a screen B11 on which a message such as “Preparing for continuous reproduction” is displayed (step S111).

  During this step S111, the video data buffered in the first mobile terminal 1 is transmitted to the second mobile terminal 2, the divided data of the streaming data is reconfigured, and preparation for continuous playback is performed (FIG. 8). Equivalent to step S55 to step S58). When preparation for continuous playback is completed and the amount of data necessary for playback is stored in the buffer, streaming data is continuously played back on the second portable terminal 2 (corresponding to steps S59 to S65 in FIG. 8). . At this time, the first mobile terminal 1 outputs a screen A12 on which nothing is displayed, and the second mobile terminal 2 outputs a playback screen B12 of video data, audio data, and text data (step S112).

  FIG. 13 is a diagram illustrating an example of screen transitions of the first mobile terminal and the second mobile terminal when the power of the first mobile terminal 1 is turned off. As illustrated in FIG. 13, when the power of the first mobile terminal 1 is turned off by the user (corresponding to steps S <b> 71 to S <b> 75 in FIG. 9), the power of the first mobile terminal 1 is in the off state. A screen A21 showing is output. On the other hand, the second portable terminal 2 outputs a screen B21 on which a message such as “Preparing for continuous reproduction” is displayed (step S121).

  During this step S121, the second portable terminal 2 accesses the streaming server 3 and acquires unreproduced streaming data (corresponding to steps S76 to S84 in FIG. 9). When the amount of data necessary for reproduction is stored in the buffer, the streaming data is continuously reproduced in the second portable terminal 2 (corresponding to step S84 in FIG. 9). At this time, the first portable terminal 1 outputs a screen A22 indicating that the power is off, and the second portable terminal 2 outputs a reproduction screen B22 of video data, audio data, and text data (step S122). ).

  Next, the effect of the information reproduction system 10 will be described. The information reproduction system 10 is an information reproduction system in which the first portable terminal 1 and the second portable terminal 2 reproduce streaming data in synchronization. The first mobile terminal 1 is included in the data acquisition management unit 171 that acquires streaming data, and the streaming data acquired by the data acquisition management unit 171 based on the processing capabilities of the first mobile terminal 1 and the second mobile terminal 2. A playback pattern determining unit 172 that determines a first media element to be played back on the first mobile terminal 1 and a second media element to be played back on the second mobile terminal 2, and the first mobile terminal 1. The measurement unit 16 that measures the data transfer time between the mobile terminal 2 and the second mobile terminal 2, and the media element extraction unit 173 that extracts the first media element and the second media element determined by the playback pattern determination unit 172 from the streaming data. And the second media element extracted by the media element extraction unit 173 is transmitted to the second portable terminal 2. A synchronous playback control unit 174 that notifies the second mobile terminal 2 of the playback start time determined based on the data transfer time measured by the measurement unit 16 and plays back the first media element at the playback start time; Prepare. The second portable terminal 2 includes a data acquisition management unit 271 that acquires the second media element transmitted by the synchronized playback control unit 174, and a data acquisition management unit 271 that is at the playback start time notified by the synchronized playback control unit 174. And a synchronized playback control unit 274 that plays back the acquired second media element.

  Thereby, according to the processing capability of the 1st portable terminal 1 and the 2nd portable terminal 2, among the several media elements contained in streaming data, the 1st media element reproduced in the 1st portable terminal 1 and the 2nd portable terminal By determining the second media element to be played back on the terminal 2, one streaming data can be divided for each media element and played back on the first mobile terminal 1 and the second mobile terminal 2. Then, at the reproduction start time determined based on the data transfer time between the first portable terminal 1 and the second portable terminal 2, the first media element and the second media element respectively at the first portable terminal 1 and the second portable terminal 2. Is played, the first mobile terminal 1 and the second mobile terminal 2 start playback simultaneously in consideration of the delay time caused by the data transfer from the first mobile terminal 1 to the second mobile terminal 2. Can do. For this reason, it becomes possible to reproduce | regenerate the one streaming data divided | segmented for every media element in the 1st portable terminal 1 and the 2nd portable terminal 2 synchronizing.

  The synchronized playback control unit 274 notifies the first mobile terminal 1 of the playback end time of the playback interval in the second mobile terminal 2 for each playback interval that divides the playback time of the streaming data. In addition, the synchronized playback control unit 174 is based on the difference between the playback end time of the playback interval in the second mobile terminal 2 notified by the synchronized playback control unit 274 and the playback end time of the playback interval in the first mobile terminal 1. The playback rate is instructed to the second portable terminal 2. Then, the synchronized playback control unit 274 plays the second media element at the playback rate instructed by the synchronized playback control unit 174.

  The processing executed in the first mobile terminal 1 and the second mobile terminal 2 changes according to time. Therefore, the difference between the time actually required by the first mobile terminal 1 to play the first media element in the playback interval and the time actually required by the second mobile terminal 2 to play the second media element in the playback interval. By determining the playback rate of the second mobile terminal 2 based on the above, the difference between the playback end time of the playback interval of the first mobile terminal 1 and the playback end time of the playback interval of the second mobile terminal 2 can be suppressed. it can. Therefore, even if the playback time of the playback interval changes for each of the first mobile terminal 1 and the second mobile terminal 2 due to changes in the processing load of the first mobile terminal 1 and the second mobile terminal 2, the playback rate is changed. By changing it, synchronization loss can be reduced. As a result, the first mobile terminal 1 and the second mobile terminal 2 can play streaming data in synchronization.

  The synchronized playback control unit 174 may transmit the second media element to the second mobile terminal 2 at a transmission rate determined based on the data transfer time. According to this configuration, by determining the transmission rate for transmitting the second media element to the second portable terminal 2 in consideration of the data transfer time, the data to be reproduced in the second portable terminal 2 is insufficient. Can be suppressed. For this reason, in the 2nd portable terminal 2, the delay of reproduction | regeneration can be suppressed and the 1st portable terminal 1 and the 2nd portable terminal 2 can reproduce | regenerate streaming data synchronizing.

  The measurement unit 16 may measure the data transfer time for each playback interval that divides the playback time of the streaming data, and the synchronous playback control unit 174 may adjust the transmission rate based on the measured data transfer time. The data transfer time between the first portable terminal 1 and the second portable terminal 2 may change depending on the state of the communication path such as traffic between the first portable terminal 1 and the second portable terminal 2. According to this configuration, by adjusting the transmission rate based on the data transfer time measured at each reproduction interval, it is possible to obtain a transmission rate according to the change in the data transfer time. For this reason, in the 2nd portable terminal 2, the delay of reproduction | regeneration can further be suppressed.

  In the second mobile terminal 2, the disconnection detection unit 28 that detects whether or not the first mobile terminal 1 is in a disconnection state in which streaming data cannot be acquired, and the disconnection detection unit 28 detects the disconnection state of the first mobile terminal 1 In this case, the second portable terminal 2 may further include a continuous reproduction control unit 275 that controls the streaming data to be reproduced continuously. In this case, the synchronized playback control unit 174 transmits the URL information of the streaming data to the second portable terminal 2, and the data acquisition management unit 271 detects the disconnection state of the first portable terminal 1 by the disconnection detection unit 28. In addition, based on the URL information transmitted by the synchronous playback control unit 174, the unplayed portion of the streaming data is acquired, and the continuous playback control unit 275 determines the unplayed portion of the streaming data acquired by the data acquisition management unit 271. You may replay it. According to this configuration, even if the first mobile terminal 1 cannot acquire streaming data, the second mobile terminal 2 can acquire an unreproduced portion of the streaming data. For this reason, the streaming data can be continuously reproduced on the second portable terminal 2. In other words, it is possible to continue playback dynamically according to a change in playback device or a change in network environment depending on the viewing style or the like.

  The data acquisition management unit 271 acquires, from the first mobile terminal 1, the first media element that is not played back on the first mobile terminal 1 when the disconnection detection unit 28 detects the disconnection state of the first mobile terminal 1. The continuous playback control unit 275 may reconstruct the streaming data using the first media element acquired by the data acquisition management unit 271 and the second media element that is not played back on the second mobile terminal 2. Good. According to this configuration, even if the first mobile terminal 1 is disconnected, the streaming data that has been transmitted and has not been played back is transmitted to the second mobile terminal 2 without re-acquiring the streaming data from the streaming server 3. Can be reconfigured. For this reason, in the second portable terminal 2, it is possible to continuously play the transmitted streaming data. In other words, it is possible to continue playback dynamically according to a change in playback device or a change in network environment depending on the viewing style or the like.

  As described above, in the information reproduction system 10, the streaming data is divided among the devices in units of media elements constituting the streaming data according to the attributes of the mobile devices such as smartphones and tablet terminals and the viewing style of the user, and the streaming data is divided into a plurality of devices Play in sync with. For this reason, the attractiveness of the content can be fully utilized, and the battery consumption and the memory consumption can be suppressed. Further, even if the user turns off one device for some reason or disconnection of unintentional wireless communication, the other device can continue to view streaming data.

(Second Embodiment)
FIG. 14 is a diagram schematically illustrating a functional configuration of an information reproduction system according to the second embodiment. As shown in FIG. 14, the information reproduction system 10A further includes a device 4 (reproduction device), includes a first portable terminal 1A (portable terminal) instead of the first portable terminal 1, and a streaming server. 3 is different from the information reproduction system 10 in that it is not provided. Moreover, in the information reproduction system 10, the 2nd portable terminal 2 hold | maintains absolute time (it has a time measuring means), However, in the information reproduction system 10A, the apparatus 4 does not have a time measuring means. Is different.

  The information reproduction system 10 </ b> A is a system that performs multi-device reproduction of content stored in the first mobile terminal 1 </ b> A using the second mobile terminal 2 (another mobile terminal, playback device) and a plurality of playback devices among the devices 4. is there. The device 4 is one or more devices connected to the local network LN and capable of reproducing any media element. The device 4 includes, for example, a personal computer 5, a speaker 6, a television 7, and the like. The device 4 is not limited to these, and may be any device that can be connected to the local network LN and can reproduce any media element.

  1st portable terminal 1A is replaced with the terminal information management part 13, the measurement part 16, and the reproduction | regeneration control part 17, the point provided with the terminal information management part 13A, the measurement part 16A (measurement means), and the reproduction | regeneration control part 17A, and content It is different from the first portable terminal 1 in that it further includes a storage unit 32A, a content management unit 33A, and a reproduction management unit 34A.

  The content storage unit 32 </ b> A functions as a content storage unit that stores a plurality of contents to be played back on the second mobile terminal 2 and the device 4. The first mobile terminal 1A stores content acquired from an external content server, content recorded by the first mobile terminal 1A, a slide show created by the first mobile terminal 1A, and the like in the content storage unit 32A. The content stored in the content storage unit 32A is managed by the content management unit 33A.

  The content management unit 33A functions as a content management unit that manages meta information of content stored in the content storage unit 32A. The content management unit 33A manages the meta information of the content stored in the content storage unit 32A using a content management table similar to the streaming data management table of FIG.

  The reproduction management unit 34A functions as a reproduction management unit that dynamically manages the reproduction state in the reproduction apparatus including the second portable terminal 2 and the device 4. The reproduction management unit 34A manages the reproduction state of content using a reproduction management table similar to the reproduction management table of FIG.

  The terminal information management unit 13A functions as a terminal information management unit that manages the terminal capabilities (static information) and processing loads (dynamic information) of the playback devices of the second portable terminal 2 and the device 4 respectively. The terminal information management unit 13A, for example, codec information such as video codec and audio codec supported by the playback device, playback rate information such as video playback rate and audio playback rate, and resolution indicating the screen resolution of the output device 107 Information is managed as reproduction ability information. In addition, the terminal information management unit 13A manages, for example, load information indicating processing load, memory consumption information indicating memory consumption, battery remaining information indicating remaining battery power, and the like as reproduction state information.

  The measurement unit 16A functions as a measurement unit that measures the data transfer time between the first portable terminal 1A and each playback device. For example, the measurement unit 16A may measure RTT, which is the time from when data is transmitted to the playback device to when the response is received from the playback device, and half of the time may be used as the data transfer time. Then, the measurement unit 16A transmits transfer time information indicating the data transfer time to the reproduction control unit 17A. The measurement unit 16A may measure the data transfer time at a predetermined time interval.

  When the playback device does not have a function of transmitting a reception response to the first mobile terminal 1A, the measurement unit 16A measures the data transfer time by another method. For example, when the playback device is the speaker 6, the measurement unit 16 </ b> A outputs audio data having a predetermined sound pattern, and measures the time until the sound pattern of the output audio data is output from the speaker 6. Specifically, the measurement unit 16A outputs audio data to the speaker 6 and receives sound output from the speaker 6 using a microphone (not shown). Then, the measurement unit 16A determines whether or not the sound pattern of the output audio data is output from the speaker 6 by comparing the received sound pattern with the sound pattern of the output audio data. When the voice patterns match, the measurement unit 16A calculates the time from when the audio data is output until the voice pattern is received, and the time is used as the data transfer time.

  When the playback device is the personal computer 5 or the television 7, the measuring unit 16 </ b> A outputs video data of a predetermined image, and the time until the output video data image is displayed on the personal computer 5 or the television 7. Measure. More specifically, the measurement unit 16A outputs video data to the personal computer 5 or the television 7 and takes an image displayed on the personal computer 5 or the television 7 using a camera (not shown). Then, the measurement unit 16A performs image processing on the captured image and compares it with the output video data to determine whether the image of the output video data is displayed on the personal computer 5 or the television 7. When it is determined that the image of the output video data is displayed, the measurement unit 16A calculates the time from when the video data is output until the image is displayed, and the time is used as the data transfer time. When the playback device is the personal computer 5 or the television 7, the data transfer time may be measured by an audio pattern as with the speaker 6.

  The playback control unit 17A functions as playback control means for controlling multi-device playback of content in the playback device. The reproduction control unit 17A includes a data acquisition management unit 171A (first data acquisition unit), a reproduction pattern determination unit 172A (reproduction pattern determination unit), a media element extraction unit 173A (media element extraction unit), and a synchronous reproduction control unit. 174A (first synchronous reproduction control means).

  The data acquisition management unit 171A functions as a first data acquisition unit that acquires content. For example, the data acquisition management unit 171 receives operation information indicating an operation for reproducing content from the input unit 11. Then, the data acquisition management unit 171 acquires the meta information of the content from the content management unit 33A. Further, the data acquisition management unit 171A acquires content from the content storage unit 32A, transmits the content to the buffer control unit 15, and stores it in the buffer.

  The playback pattern determination unit 172A functions as a playback pattern determination unit that determines which playback device plays back each of the plurality of media elements included in the content acquired by the data acquisition management unit 171A. Specifically, the playback pattern determination unit 172A determines a playback device for each media element included in the content based on the meta information of the content.

  For example, when the multi-device information indicates that multi-device playback is possible, the playback pattern determination unit 172A causes the output device 107 to display a screen for allowing the user to select whether or not to perform multi-device playback. When the multi-device playback of the content is selected by the user, the playback pattern determination unit 172A determines the media for each media element constituting the content based on the playback capability information managed by the terminal information management unit 13A. The playback device corresponding to the element codec is extracted. Then, the playback pattern determination unit 172A causes the output device 107 to display a list of the extracted playback devices so that the user can select a playback device that plays back each media element.

  FIG. 15 is a diagram illustrating an example of a screen of the first mobile terminal 1A. As shown in FIG. 15, the reproduction pattern determination unit 172A, for example, allows the user to select a reproduction device that reproduces the sound of the media element, in the second mobile terminal 2, the personal computer 5, the speaker 6, and the television 7. The list is displayed on the output device 107.

  When receiving operation information indicating an operation for selecting one of the playback devices from the input unit 11 for each media element constituting the content, the playback pattern determination unit 172A determines the playback device that plays back the media element. . Also, the playback pattern determination unit 172A is a playback device corresponding to the codec of the media element so that the number of media elements or the processing load is equalized based on the terminal information managed by the terminal information management unit 13A. Among them, a playback device for playing back the media element may be determined.

  On the other hand, when the multi-device information indicates that multi-device playback is possible, when the multi-device playback of the content is not selected by the user, and when the multi-device information indicates that multi-device playback is not possible, the playback pattern determination unit 172A extracts a playback device capable of playing back all media elements constituting the content based on the terminal information managed by the terminal information management unit 13A. Then, the reproduction pattern determination unit 172A causes the output device 107 to display a list of the extracted reproduction devices so that the user can select a reproduction device for reproducing the content. In addition, the reproduction pattern determination unit 172A may select a predetermined reproduction device.

  The media element extraction unit 173A functions as a media element extraction unit that extracts a media element from content for each playback device determined by the playback pattern determination unit 172A. The media element extraction unit 173A extracts media elements for each playback device from the data in the buffer. Then, the media element extraction unit 173A transmits the extracted media element to the buffer control unit 15 and stores it in the buffer.

  The synchronized playback control unit 174A plays back each of the plurality of media elements extracted by the media element extraction unit 173A on the device determined by the playback pattern determination unit 172A based on the data transfer time measured by the measurement unit 16A. It functions as a first synchronized playback control means. For example, the synchronized playback control unit 174A transmits the media element extracted by the media element extraction unit 173A to the playback device based on the data transfer time measured by the measurement unit 16A. More specifically, the synchronized playback control unit 174A determines the playback start timing based on the data transfer time measured by the measuring unit 16A. That is, there is a data transfer delay equivalent to the data transfer time from when the first mobile terminal 1A transmits a media element to each playback device until the playback device receives the media element. For this reason, the synchronized playback control unit 174A determines the timing for transmitting the media element for each playback device in accordance with the data transfer time of the playback device so that the timing at which each playback device receives the media element is matched.

  Further, the synchronized playback control unit 174A adjusts the data transmission rate to each playback device based on the data transfer time measured by the measurement unit 16A. That is, the synchronized playback control unit 174A determines the data transmission rate and transmission data amount to each playback device so that the data to be played back in each playback device is not short due to the data transfer delay, and determines the determined data transmission rate and transmission. The media element stored in the buffer is transmitted to each playback device in the amount of data.

  Although not described in the above description, since the media element extracted by the media element extraction unit 173A is stored in the buffer, the synchronized playback control unit 174A performs buffer control on the media element stored in the buffer. The media element is transmitted to the second mobile terminal 2 and the device 4 via the local NW communication unit 142.

  In addition, the synchronized playback control unit 174A adjusts the data transmission rate and the transmission data amount to the second mobile terminal 2 and the device 4 based on the data transfer time measured by the measurement unit 16A during the playback of the content. Also good.

  FIG. 16 is a sequence diagram showing an example of the synchronized playback control process of the information playback system 10A. This process is started when the first mobile terminal 1A receives operation information indicating an operation for content reproduction from the input unit 11.

  As shown in FIG. 16, first, the data acquisition management unit 171A of the first mobile terminal 1A acquires the meta information of the content to be reproduced from the content management unit 33A, and transmits it to the reproduction pattern determination unit 172A. Further, the data acquisition management unit 171A acquires content from the content storage unit 32A, transmits the content to the buffer control unit 15, and stores it in the buffer.

  Next, the reproduction pattern determination unit 172A determines whether or not the multi-device information included in the meta information indicates that multi-device reproduction is possible. When the multi-device information indicates that multi-device playback is possible, the playback pattern determination unit 172A causes the output device 107 to display a screen for allowing the user to select whether or not to perform multi-device playback.

  When the operation information indicating the operation for multi-device playback is received from the input unit 11 (step S201), the playback pattern determination unit 172A determines the codec of the media element constituting the content based on the meta information of the content. Determine. Then, based on the playback capability information managed by the terminal information management unit 13A, the playback pattern determination unit 172A extracts, for each media element constituting the content, a playback device corresponding to the codec of the media element. . Subsequently, the reproduction pattern determination unit 172A causes the output device 107 to display a list of the extracted reproduction devices so that the user can select a reproduction device that reproduces each media element.

  When receiving operation information indicating an operation for selecting one of the playback devices from the input unit 11 for each media element constituting the content, the playback pattern determination unit 172A determines the playback device that plays back the media element. (Step S202). Here, a case will be described in which the personal computer 5 is determined as a playback device that plays back video data and text data, and the speaker 6 is determined as a playback device that plays back audio data.

  Next, the local NW communication unit 142 transmits a connection request to each playback device determined by the playback pattern determination unit 172A (step S203). When each playback device receives a connection request from the first portable terminal 1A, each playback device transmits a connection response indicating whether or not connection is possible. Then, the local NW communication unit 142 receives a connection response indicating that connection is possible from the playback device, and establishes a communication connection with the playback device (step S204).

  When the communication connection between the first portable terminal 1A and each playback device is established, the measurement unit 16A measures the data transfer time for each playback device (step S205). Further, the media element extraction unit 173A extracts each media element from the content stored in the buffer (step S206). Then, the media element extraction unit 173A transmits the extracted media element to the buffer control unit 15 and stores it in the buffer.

  In addition, the synchronized playback control unit 174A determines the data transmission rate and the amount of transmission data based on the data transfer time measured in step S205 so that the data to be played back in the playback device is not short (step S207). Then, the synchronized playback control unit 174A determines the playback start timing based on the data transfer time measured by the measuring unit 16A. That is, the synchronized playback control unit 174A determines the timing for transmitting the media element for each playback device in accordance with the data transfer time of the playback device so that the timing of receiving the media element by each playback device is matched.

  Then, the synchronized playback control unit 174A transmits the media elements stored in the buffer to each playback device at the transmission timing determined for each playback device at the data transmission rate and transmission data amount determined for each playback device ( Step S208). Thereafter, step S208 is repeatedly performed according to the data transmission rate determined in step S207 or the like. When receiving the media element, the playback device sequentially plays back the received media element (step S209). At this time, the synchronized playback control unit 174A transmits the media element so that each playback device receives the media element almost simultaneously, so that synchronized playback is realized.

  Note that the measurement unit 16A may measure the data transfer time at a predetermined time interval. In this case, the synchronous reproduction control unit 174A adjusts the data transfer rate and the transmission data amount according to the measured data transfer time. May be.

  Next, operational effects of the information reproduction system 10A will be described. The information reproduction system 10A is an information reproduction system in which a plurality of reproduction apparatuses reproduce contents in synchronization. The first mobile terminal 1A reproduces the data acquisition management unit 171A for acquiring the content and each of the plurality of media elements included in the content acquired by the data acquisition management unit 171A on any of the plurality of playback devices. A reproduction pattern determination unit 172A for determining the data transfer time, a measurement unit 16A for measuring data transfer times between the first portable terminal 1A and the plurality of reproduction devices, and a media element extraction unit 173A for extracting a plurality of media elements from the content. And synchronous reproduction control for reproducing each of the plurality of media elements extracted by the media element extraction unit 173A at a timing based on the data transfer time measured by the measurement unit 16A by the apparatus determined by the reproduction pattern determination unit 172A 174A. The playback device plays back the media element transmitted by the synchronized playback control unit 174A.

  Thus, by determining which of a plurality of playback devices plays back each of a plurality of media elements included in the content, one content is divided into media elements and played back on the plurality of devices. can do. Then, based on the data transfer time between the first mobile terminal 1A and the plurality of playback devices, each media element is played back in the determined device, whereby the data from the first mobile terminal 1A to each playback device. Considering the delay time caused by the transfer, it is possible to reduce the deviation of the reproduction start timing in each device. For this reason, it becomes possible to reproduce | regenerate the one content divided | segmented for every media element synchronously with several apparatuses.

  Thus, in the information reproduction system 10A, the first portable terminal 1A is used like a remote controller, and data acquired or recorded by the first portable terminal 1A is synchronously reproduced by a plurality of reproduction apparatuses. According to this information reproduction system 10A, the load of reproduction processing can be distributed to a plurality of reproduction apparatuses. Furthermore, in the information reproduction system 10A, for example, a reproduction apparatus can be selected according to the user's preference such as improvement of sound quality by stereo reproduction, so that user satisfaction can be improved.

  The terminal device, the information display system, and the information display method according to the present invention are not limited to those described in the above embodiment. For example, in the first embodiment, in the negotiation between the first portable terminal 1 and the streaming server 3, playback from the first portable terminal 1 to the streaming server 3 is performed as in the negotiation between the second portable terminal 2 and the streaming server 3. It is good also as a structure which notifies a rate and transmits the response of a data transmission rate from the streaming server 3 to the 1st portable terminal 1. FIG.

  In the first embodiment, the streaming data playback interval is a time interval that can be sampled. However, by setting the playback interval to a shorter time interval, the accuracy of the synchronized playback control can be improved.

  Moreover, in the said 1st Embodiment, although the 1st portable terminal 1 and the 2nd portable terminal 2 presuppose that absolute time is hold | maintained, absolute time may be inaccurate. In this case, synchronous playback control may be performed using relative time. For example, the first portable terminal 1 transmits a reproduction start request including information that reproduction is started after a set time [msec] to the second portable terminal 2. Then, the first mobile terminal 1 corrects the set time by RTT and adjusts the reproduction start timing. Specifically, the reproduction is started after (set time + RTT / 2) from the time when the reproduction start request is transmitted to the second portable terminal 2.

  In the first embodiment, the playback pattern determination unit 172 determines the memory consumption of the first mobile terminal 1 and the second mobile terminal 2 when there are a plurality of combinations of combinations of the first media element and the second media element. The first media element and the second media element may be determined according to the memory consumption amount. For example, when the memory consumption of the second portable terminal 2 is larger than the memory consumption of the first portable terminal 1, a media element with a small processing load may be used as the second media element. Further, when the memory consumption of the second portable terminal 2 is smaller than the memory consumption of the first portable terminal 1, more media elements may be used as the second media element.

  In the first embodiment, the playback pattern determination unit 172 determines the remaining battery level of the first mobile terminal 1 and the second mobile terminal 2 when there are a plurality of combinations of combinations of the first media element and the second media element. The first media element and the second media element may be determined according to the remaining battery level. For example, when the remaining battery level of the second mobile terminal 2 is greater than the remaining battery level of the first mobile terminal 1, a media element with a large processing load may be used as the second media element. Further, when the remaining battery level of the second mobile terminal 2 is less than the remaining battery level of the first mobile terminal 1, a media element with a small processing load may be used as the second media element.

  Furthermore, the playback pattern determination unit 172 may determine the first media element and the second media element by combining one or more pieces of information included in the capability information.

  In addition, the playback pattern determination unit 172 displays all media elements constituting the streaming data on the output device 107 by the output unit 12, and each media element is set to either the first media element or the second media element by the user. By doing so, the first media element and the second media element may be determined.

  In the first embodiment, the synchronized playback control unit 174 transmits the URL information of the streaming data together with the divided data of the streaming data. However, the present invention is not limited to this, and the URL information of the streaming data is separated from the divided data. You may transmit to the 2nd portable terminal 2. Further, when the power is turned off, the synchronized playback control unit 174 may transmit the URL information of the streaming data to the second mobile terminal 2. That is, the synchronized playback control unit 174 may transmit the URL information of the streaming data to the second mobile terminal 2 before the communication between the first mobile terminal 1 and the second mobile terminal 2 is disconnected.

  In the first embodiment, multi-device playback of streaming data has been described. However, data to be subjected to multi-device playback is not limited to streaming data, and may be data including a plurality of media elements. For example, moving image data captured by the first mobile terminal 1, content acquired in advance by the first mobile terminal 1, a slide show with audio and text, and the like may be targeted for multi-device playback in the information playback system 10.

  In the first embodiment, the plurality of media elements constituting the streaming data received by the first mobile terminal 1 are distributed to the first mobile terminal 1 and one second mobile terminal 2 so that each terminal Although the description has been made centering on the configuration in which the media elements distributed in the above are synchronously reproduced, the present invention is not limited to this. For example, a plurality of media elements constituting content such as streaming data received by the first mobile terminal 1 are distributed to the first mobile terminal 1 and two or more playback devices, and the media elements distributed in each terminal It is good also as a structure which reproduces | regenerates synchronously. In this case, the first mobile terminal 1 may cause two or more playback devices to perform multi-device playback without performing playback of media elements. Further, the playback device is not limited to the second mobile terminal 2, and may be a device that can play back any media element. Examples of the playback device include a personal computer, a speaker, and a television.

  In the second embodiment, the information playback system 10A may include the streaming server 3 when performing multi-device playback of streaming data. In this case, the first mobile terminal 1 </ b> A may perform processing with the streaming server 3 in the same manner as the first mobile terminal 1.

  In the second embodiment, the playback pattern determination unit 172A determines whether each of the plurality of media elements included in the content acquired by the data acquisition management unit 171A is the first mobile terminal 1A or one or more playback devices. It may be determined whether or not to play on the device. In other words, the first mobile terminal 1A may play back any one of the media elements, and the playback device is a medium other than the media element played back by the first mobile terminal 1A among the plurality of media elements constituting the content. You may play the element.

  DESCRIPTION OF SYMBOLS 1,1A ... 1st portable terminal (mobile terminal), 2 ... 2nd portable terminal (other portable terminals, reproducing | regenerating apparatus), 4 ... Apparatus (reproducing apparatus) 10, 10A ... Information reproducing | regenerating system, 16, 16A ... Measurement Unit (measuring unit), 28 ... cutting detection unit (cutting detection unit), 171, 171A ... data acquisition management unit (first data acquisition unit), 172, 172A ... reproduction pattern determination unit (reproduction pattern determination unit), 173 173A: Media element extraction unit (media element extraction unit), 174, 174A: Synchronous reproduction control unit (first synchronous reproduction control unit), 271 ... Data acquisition management unit (second data acquisition unit), 274 ... Synchronous reproduction control unit (Second synchronous reproduction control means), 275... Continuous reproduction control section (continuous reproduction control means).

Claims (16)

An information reproduction system in which a first portable terminal and a second portable terminal reproduce contents in synchronization,
The first portable terminal is
First data acquisition means for acquiring the content;
Depending on whether or not the first mobile terminal and the second mobile terminal can play back each of a plurality of media elements included in the content acquired by the first data acquisition means , the first mobile terminal A playback pattern determining means for determining a first media element to be played back and a second media element to be played back by the second mobile terminal ;
Wherein the first mobile terminal transmits data to the second mobile terminal, the first mobile terminal measuring means to measure the data transfer time is the time it takes to receive a response from the second mobile terminal When,
Media element extraction means for extracting the first media element and the second media element determined by the reproduction pattern determination means from the content;
Transmits the second media element extracted by the media element extraction unit to the second mobile terminal, the data measured by said measuring means at the time of transmitting the second media element to said second mobile terminal A first synchronized playback control means for notifying the second mobile terminal of the playback start timing determined as a time after the transfer time is added, and playing back the first media element at the playback start timing ;
With
The second portable terminal is
Second data acquisition means for acquiring the second media element transmitted by the first synchronous playback control means;
Second synchronized playback control means for playing back the second media element acquired by the second data acquiring means at the playback start timing notified by the first synchronized playback control means;
Information reproduction system comprising: a. The second synchronized playback control means notifies the playback end time of the playback interval in the second mobile terminal to the first mobile terminal for each playback interval that divides the playback time of the content,
The first synchronized playback control means includes a playback end time of the playback interval in the second mobile terminal notified by the second synchronized playback control means and a playback end time of the playback interval in the first mobile terminal. Instructing the playback rate to the second mobile terminal so that the difference is small ,
The information playback system according to claim 1 , wherein the second synchronized playback control unit plays the second media element at the playback rate instructed by the first synchronized playback control unit. The first synchronized playback control means sends the second media element to the second portable terminal at a transmission rate determined based on the data transfer time so as not to run out of data to be played back in the second portable terminal. The information reproducing system according to claim 1 , wherein the information reproducing system transmits the information to the information reproducing system. The measuring means measures the data transfer time for each playback interval that divides the playback time of the content,
Said first synchronized playback control means, based on said measured data transfer time, the second as the data to be reproduced do not run out of the portable terminal, according to claim 3, characterized in that adjusting the transmission rate Information reproduction system described in 1. The second portable terminal is
Disconnection detecting means for detecting whether or not the first portable terminal is in a disconnection state where the content cannot be acquired;
Continuous playback control means for controlling to continuously play back the content on the second mobile terminal when the disconnection state of the first mobile terminal is detected by the disconnection detection means;
Further comprising
The first synchronized playback control means transmits address information of the content to the second portable terminal,
The second data acquisition unit includes the address information transmitted by the first synchronous reproduction control unit and the second data acquisition unit when a disconnection state of the first portable terminal is detected by the disconnection detection unit. Among the contents, based on the time stamp of the second media element that is not played back on the second mobile terminal among the second media elements acquired by the first mobile terminal and the second mobile terminal obtaining missed reproducing portion not reproduced in the terminal,
The continuous regeneration control means, out of the acquired unplayed portion by the second data acquisition means, claims 1 to 4, wherein the second mobile terminal, wherein the playing media elements playable The information reproduction system according to any one of the above. The first portable terminal is
A first buffer for storing the first media element extracted by the media element extraction means;
The second portable terminal is
A second buffer for storing the second media element acquired by the second data acquisition means;
Disconnection detecting means for detecting whether or not the first portable terminal is in a disconnection state where the content cannot be acquired;
Continuous playback control means for controlling to continuously play back the content on the second mobile terminal when the disconnection state of the first mobile terminal is detected by the disconnection detection means;
Further comprising
The second data acquisition unit includes the first media element stored in the first buffer when the disconnection detection unit detects a disconnection state of the first mobile terminal. Obtaining the first media element that has not been played from the first mobile terminal;
The continuous reproduction control means is not reproduced on the second portable terminal among the first media element acquired by the second data acquisition means and the second media element already stored in the second buffer. and the second medium element, to reconfigure a portion of the content by using, among some of the contents reconstituted, the second mobile terminal, wherein the playing media elements playable information reproduction system according to any one of claims 1 to 4. The playback start timing, the information reproducing system according to any one of claims 1 to 6, characterized in that the absolute time. The playback start timing, the information reproducing system according to any one of claims 1 to 6, characterized in that the relative time. The content information reproducing system according to any one of claims 1 to 8, characterized in that the streaming data. A mobile device that plays content in sync with other mobile devices ,
Data acquisition means for acquiring the content;
First media to be played on the mobile terminal according to whether or not the mobile terminal and the other mobile terminal can play back each of a plurality of media elements included in the content acquired by the data acquisition means Playback pattern determining means for determining an element and a second media element to be played back by the other mobile terminal ;
Media element extraction means for extracting the first media element and the second media element determined by the reproduction pattern determination means from the content;
Wherein the mobile terminal transmits data to the other mobile terminals, and a constant measuring means measures the data transfer time is a time until the mobile terminal receives the response from the other mobile terminal,
Transmits the second media element extracted by the media element extraction unit to the other mobile terminals, the data measured by said measuring means at the time of transmitting the second media element to the other mobile terminals Synchronous playback control means for notifying the other mobile terminal of the playback start timing determined as the time after the time when the transfer time is added, and playing back the first media element at the playback start timing ;
A portable terminal comprising: The synchronized playback control means, for each reproduction interval obtained by dividing the playback time of the content, the reproduction end time of the playback interval in the another mobile terminal, the difference between the reproduction end time of the playback interval in the portable terminal The mobile terminal according to claim 10 , wherein a playback rate is instructed to the other mobile terminal so as to be reduced . The measuring means measures the data transfer time for each playback interval that divides the playback time of the content,
The synchronized playback control means moves the second media element to the other portable device at a transmission rate adjusted so as not to run out of data to be reproduced in the other portable terminal based on the measured data transfer time. The mobile terminal according to claim 10 or 11 , wherein the mobile terminal is transmitted to a terminal. An information playback method in an information playback system in which a first mobile terminal and a second mobile terminal play back content synchronously,
In the first mobile terminal,
A first data acquisition step of acquiring the content;
Depending on whether or not the first mobile terminal and the second mobile terminal can reproduce each of the plurality of media elements included in the content acquired in the first data acquisition step, the first mobile terminal A playback pattern determining step for determining a first media element to be played back in step 2 and a second media element to be played back on the second mobile terminal ;
A media element extraction step of extracting the first media element and the second media element determined in the reproduction pattern determination step from the content;
Measurement step the first mobile terminal from the transmission of data to the second mobile terminal, the first mobile terminal to measure the data transfer time is the time it takes to receive a response from the second mobile terminal When,
Transmits the second media element extracted in said media element extraction step in the second mobile terminal, the data of the second media element has been measured in the measuring step in time to be transmitted to the second mobile terminal A first synchronized playback step of notifying the second mobile terminal of the playback start timing determined as a time after the time when the transfer time is added, and playing back the first media element at the playback start timing ;
With
In the second mobile terminal ,
A second data acquisition step of acquiring the second media element transmitted in the first synchronous playback step;
A second synchronized playback step of playing back the second media element acquired in the second data acquisition step at the playback start timing notified in the first synchronized playback step ;
An information reproducing method comprising:   An information playback system in which a first mobile terminal and one or more playback devices play back content synchronously,
  The first portable terminal is
  First data acquisition means for acquiring the content;
  For each of a plurality of media elements included in the content acquired by the first data acquisition means, the media element is set according to whether or not the first mobile terminal and the one or more playback devices can play back. Playback pattern determination means for determining which of the first portable terminal and the one or more playback devices to play back;
  Measuring means for measuring, for each of the one or more playback devices, a data transfer time, which is a time from when the first portable terminal transmits data to the playback device until the playback device receives the data; ,
  Media element extraction means for extracting the plurality of media elements from the content;
  First synchronized playback control means for transmitting each of the plurality of media elements extracted by the media element extracting means to the device determined by the playback pattern determining means;
With
  The first synchronized playback control means is configured to adjust each of the plurality of media elements according to the data transfer time measured by the measuring means so as to match the timing received by the apparatus determined by the playback pattern determining means. Is determined for each device,
  The apparatus determined by the playback pattern determining means, when receiving the media elements transmitted by the first synchronized playback control means, sequentially plays the received media elements.   A mobile terminal that plays content in synchronization with one or more playback devices,
  Data acquisition means for acquiring the content;
  For each of a plurality of media elements included in the content acquired by the data acquisition means, depending on whether or not the mobile terminal and the one or more playback devices can play back the media elements, Playback pattern determination means for determining which of the one or more playback devices to play back;
  A measuring means for measuring a data transfer time, which is a time from when the portable terminal transmits data to the playback device to when the playback device receives the data, for each of the one or more playback devices;
  Media element extraction means for extracting the plurality of media elements from the content;
  Synchronous reproduction control means for transmitting each of the plurality of media elements extracted by the media element extraction means to an apparatus determined by the reproduction pattern determination means;
With
  The synchronized playback control means transmits each of the plurality of media elements according to the data transfer time measured by the measuring means so as to match the timing received by the device determined by the playback pattern determining means. A portable terminal that determines a timing for each device.   An information reproduction method in an information reproduction system in which a first portable terminal and one or more reproduction devices reproduce contents in synchronization,
  In the first mobile terminal,
  A first data acquisition step of acquiring the content;
  For each of a plurality of media elements included in the content acquired in the first data acquisition step, depending on whether or not the first mobile terminal and the one or more playback devices can play back the media elements, A playback pattern determining step for determining which of the first portable terminal and the one or more playback devices to play back;
  A measurement step of measuring, for each of the one or more playback devices, a data transfer time, which is a time from when the first portable terminal transmits data to the playback device until the playback device receives the data; ,
  A media element extraction step of extracting the plurality of media elements from the content;
  A first synchronized playback step of transmitting each of the plurality of media elements extracted in the media element extraction step to the device determined in the playback pattern determination step;
With
  In the first synchronized playback step, each of the plurality of media elements is set according to the data transfer time measured in the measurement step so as to match the timing received by the device determined in the playback pattern determination step. Determine the transmission timing for each device,
  The apparatus determined in the reproduction pattern determination step, when receiving the media element transmitted in the first synchronous reproduction step, sequentially reproduces the received media element.