KR20160086185A - Delivery of synchronised soundtracks for electronic media content - Google Patents

Abstract

The present invention relates to a method and a system for streaming a soundtrack from a server to a remote user device for an electronic media content subscriber. The soundtrack is defined by a plurality of audio areas. Each of the audio areas is defined by an audio track for a playback in the audio area, a starting position in electronic media content, corresponding to a position at which an audio area playback will be started, and a stopping position in the electronic media content, corresponding to a position at which the audio area playback will be stopped. The soundtrack streaming is based on control data, generated by the remote user device. The purpose of the present invention is to provide an improved system for transmitting a synchronized soundtrack for electronic media content.

Description

[0001] DELIVERY OF SYNCHRONIZED SOUNDTRACKS FOR ELECTRONIC MEDIA CONTENT FOR ELECTRONIC MEDIA CONTENT [0002]

The present invention relates to a system and method for delivery of synchronized soundtracks for electronic media content, such as, but not limited to, text only or multimedia content including text provided in combination with images, video or animation . The playback or transmission of such a sound track is synchronized with the reading speed of the user's personal.

To improve the reading experience, various systems have been proposed in which an audio soundtrack is created for the content of the text, such as an e-book, and played back to the user while the user is reading the text. Some of these systems aim to synchronize the playback of the soundtrack with the user's reading speed.

One such system is described in co-pending International PCT Patent Application Publication No. WO2013 / 015694, the contents of which are incorporated herein by reference in their entirety. In such a system, a sound track for electronic text, such as an e-book, is generated that includes different types of audio tracks that are configured to be played simultaneously and / or sequentially while the user is reading the electronic text. The reproduction of the sound track is synchronized with the reading speed of the user. The soundtrack includes a combination of different audio types including ambient noise, sound effects, and background music that are configured to be played at a particular point or at a particular point in the electronic text in a synchronized manner when the user is reading a particular portion of the electronic text can do. The sound track in WO2013 / 015694 includes a number of audio regions designated to be reproduced during a specified text portion of the electronic text. Each audio area includes audio data defining an audio track to be played back, a start position in the electronic text where the audio area starts playing back, and a stop position in the electronic text in which the audio area is to stop playing. In one described embodiment, the electronic text is indexed based on the number of words and the start and stop positions of the audio region are associated with start and stop words.

WO2013 / 015694 describes a variety of user playback systems for displaying electronic text and controlling playback of associated sound tracks. Such user playback systems may be implemented in a variety of user hardware devices including portable consumer electronic devices such as smart phones, tablets and dedicated ebook devices, or personal computers on desktops or laptops. The sound track data file containing all the audio tracks can be played back after the whole has been downloaded in advance and stored locally on the user device. In some embodiments, the soundtrack and audio track file are combined with an electronic text file as a single multimedia data file for download and playback. The user playback system processes the locally stored downloaded file and adjusts the playback of audio according to a read position counter that estimates the user's read position in the electronic text. This user playback system configuration uses the heavy throughput and memory resources of the hardware device to deliver synchronized soundtracks. In particular, soundtrack data files and audio track files can be very large and can take up a significant amount of local storage on a hardware device, which limits the user's ability to store a library of soundtracks for a user's ebook. Downloading of this large file to a device, especially a mobile device, can also be costly if it is downloaded via a user's mobile or other subscribed data connection. The user playback system also uses the processing resources to determine the current reading position and then to search and adjust the playback of the audio track file in a synchronized manner.

In this specification, which refers to patent specifications, other external documents, or other sources of information, its purpose is generally to provide a context for describing features of the invention. Unless specifically stated otherwise, references to such external documents should not be construed as an admission that such documents, or sources of such information, are conventional or, to some extent, form part of the general knowledge known in the art .

It is an object of the present invention to provide an improved system for the delivery of synchronized soundtracks for electronic media content or to provide at least a useful choice to the general public.

In a first aspect, the present invention relates generally to an audio player, comprising an audio track for playback in an audio region, a start position within the electronic media content corresponding to where the playback of the audio region is to begin, and a corresponding electronic media content A stream of compiled sound tracks defined by a plurality of audio regions, each defined by a stop position in the stream, to a reader remote user device of the electronic media content,

Receiving control data from the remote user device over the network, the control data indicating the read speed of the user; And

Streaming the compiled soundtrack to the user device over the network based on the received control data.

In a first aspect, the compiled soundtrack is a precompiled soundtrack and is selected from a range of precompiled soundtracks that are precompiled at different read rates. Preferably, the method further comprises selecting a precompiled sound track for streaming having an associated read speed that most closely matches control data indicative of the user ' s read speed.

In a second aspect, the compiled sound track is a real time compiled sound track and the method further comprises compiling a sound track in real time based on control data indicative of the user ' s reading speed. The sound track is progressively compiled and the method preferably includes the step of streaming the compiled portion of the sound track while the remainder is being compiled.

The method further comprising receiving from the remote user device control data indicative of the user's desired start reading position and transmitting the compiled audio data at a data location within the compiled sound track corresponding to the user's desired & Preferably, the method further comprises starting to stream the sound track to the user device. For example, the data location may be at the beginning of the compiled sound track or at an offset position within the compiled sound track.

The method comprising: receiving control data from the remote user device indicative of a change in a user's read position to a new read location in the electronic media content; and determining a stream of the compiled sound track based on the received control data And a step of correcting the error. The step of modifying the streaming of the compiled sound track further comprises adjusting the streaming of the compiled sound track to a new data location in the compiled sound track corresponding to the new reading location .

The method includes receiving from a remote user device control data indicative of a change in a user ' s read speed to a new user read speed, and modifying streaming of the compiled sound track based on the received control data .

Wherein the step of modifying the streaming of the compiled soundtrack based on the new user readout speed in the first form, wherein the compiled soundtrack is a precompiled soundtrack, Selecting a new precompiled sound track having a read rate and starting streaming of the new precompiled sound track. Preferably, the method further comprises initiating a transitioning protocol for transitioning the stream from the currently precompiled soundtrack to the selected precompiled soundtrack. In one example, the transition protocol comprises receiving from a remote user device control data representative of a current playback location of a user in a streamed sound track, and receiving the newly selected precompile at a data location corresponding to the current playback location of the user And starting a stream of sounded tracks. In another example, the transition protocol comprises the steps of analyzing the remaining unreamed portions of the current precompiled soundtrack, identifying a transition position based on the transition criteria, and comparing the data corresponding to the identified transition position And starting streaming of the newly selected precompiled sound track at a location.

Wherein the step of modifying the streaming of the compiled soundtrack based on the new user readout position in the second form, wherein the compiled soundtrack is a real time compiled soundtrack, Starting compiling, and starting streaming of the newly compiled soundtrack. The method preferably further comprises executing or initiating a transition protocol for transitioning streaming from the current real-time compiled sound track to the new real-time compiled sound track. In one example, the transition protocol comprises receiving from the remote user device control data indicating the current playback position of the user in the streamed sound track, and receiving the new real time playback position at a data location corresponding to the current playback location of the user And starting streaming of the compiled soundtrack. In another example, the transition protocol includes analyzing the remaining unreamed portions of the current real-time compiled soundtrack, identifying a transition position based on the transition criteria, and comparing the data corresponding to the identified transition location And starting streaming of the new real-time compiled sound track at a location.

In some embodiments, the method includes streaming the compiled soundtrack as a single digital audio file. In another embodiment, the method comprises streaming the compiled sound track as a plurality of digital audio files for simultaneous playback at the remote user location. For example, the audio region of the sound track may be categorized into different types, such as music, mood or effects, and the compiled sound track may be provided as a separate audio file, each containing an audio region associated with one type have.

In a second aspect, the present invention is generally directed to a method of reproducing audio content in an audio domain, the audio track for playback in an audio domain, the starting position in the electronic media content corresponding to where the playback of the audio domain begins, A sound track server for streaming a compiled sound track defined by a plurality of audio regions, each defined by a stop position within the content, to a reader remote user device of the electronic media content,

Receiving from the remote user device via the network control data indicative of the read speed of the user;

And a sound track server for streaming a compiled sound track configured to stream the compiled sound track to the user device over the network based on the received control data.

The sound track server may further include one or more of the features mentioned in the first aspect of the present invention.

In a third aspect, the present invention is generally directed to a method of reproducing audio content, comprising the steps of: providing an audio track for playback in an audio region, a start position in the electronic media content corresponding to where the playback of the audio region is to begin, CLAIMS What is claimed is: 1. A method of reproducing a compiled sound track defined by a plurality of audio regions, each defined by a stop position within a content, on a reader user device of the electronic media content,

Transmitting control data indicating the read speed of the user to a remote user server via a network;

Receiving an input stream of a compiled soundtrack having an associated read rate corresponding to the transmitted control data from the remote server over the network; And

Processing the streamed compiled sound track and starting playback at the user device of the streamed compiled sound track.

The method comprising: generating control data representing the user's desired start reading position to start the stream of the received compiled sound track at a data location in the compiled sound track corresponding to the user's desired start reading position To the remote server.

Preferably, the method further comprises temporarily storing the input stream of the compiled sound track in a memory of the user device to be ready for playback. The method comprising: receiving a user input representing a new read position with the electronic media content; determining whether the new read position corresponds to a play position in a temporarily stored portion of the compiled sound track; and Transitioning the playback to a new playback position in the temporarily saved compiled soundtrack if the temporarily saved compiled soundtrack is in the buffer, or transitioning the playback to a new playback position if the temporarily saved compiled soundtrack is outside the buffer And transmitting control data indicating the new read position to the remote server to start an updated stream of the compiled soundtrack. The method may further comprise implementing a transitioning protocol to transition the playback to a new playback location or an updated input stream with the buffer.

The method comprising the steps of receiving a user input indicative of a change in read speed or detecting a change in read speed, and controlling the readout speed to start the updated stream of the compiled sound track at a new read speed And transmitting the data to the remote server. The method may further comprise implementing a transitioning protocol to transition the playback to the updated input stream.

In a fourth aspect, the present invention provides, in general, an audio track for playback in an audio domain, a start position in the electronic media content corresponding to where the playback of the audio domain begins, and a corresponding electronic media A user apparatus for reproducing a compiled sound track defined by a plurality of audio regions, each defined by a stop position in content, for a reader of electronic media content,

A user interface configured to receive user input to control the user device;

An audio output system operable to generate an audio output for the user to hear; And

The processor comprising:

Transferring control data indicating the read speed of the user to a remote server via a network;

Receiving an input stream of a compiled sound track having an associated read rate corresponding to the transmitted control data from the remote server over a network;

And a user device configured to process the streamed compiled sound track to begin playing the sound track through the audio output system in response to user input.

In one form, the user device further comprises a display configured to play back the electronic media content such that both the audible sound track and the visible electronic media content are presented to the user in the same user device.

In another aspect, a user device is configured to provide an audible soundtrack to a user and is in data communication with a second separate user device that includes a display and is configured to provide visual electronic media content to a user. For example, data communication between a pair of user equipments can be done directly or indirectly through radio frequency (RF), near field radio (NFC), Bluetooth, Wi-Fi or other such wireless communication media or protocols, A wired communication medium may be provided when a pair of user devices are connected via a cable.

The user device may further include one or more of the features mentioned for the third aspect of the present invention.

In a fifth aspect, the present invention relates generally to a method and apparatus for reproducing audio in an audio domain, the audio track for playback in an audio domain, the starting position in the electronic media content corresponding to where the playback of the audio domain begins, A system for streaming a compiled sound track defined by a plurality of audio regions, each defined by a stop position in content, to a reader remote user device of an electronic media content, the system comprising a user device and a remote server , The user device

A user interface configured to receive user input to control the user device;

An audio output system operable to generate an audio output for the user to hear;

A data communication module configured to communicate with a remote server via a network; And

The processor comprising:

Transferring control data indicating the read speed of the user to a remote server via a network;

Receiving an input stream of a compiled sound track having an associated read rate corresponding to the transmitted control data from the remote server over a network;

And to process the streamed compiled soundtrack to start playing the soundtrack through the audio output system in response to user input,

The remote server

From the user device, control data indicating the read speed of the user;

And a system configured to stream the compiled sound track to the user device based on the received control data.

In a first embodiment, the user device further comprises a display configured to display electronic media content. In such an embodiment, the user device is capable of searching electronic media content (such as an e-book server, store or library) or searching electronic media content from a data file stored locally or in a database that can be retrieved, streamed, And a second application configured to receive and process streamed compiled sound tracks from a remote server and to control playback of the compiled sound tracks via the audio output system of the user device As shown in FIG. For example, the first application may be an Internet browser viewing a web page, electronic book terminal software for viewing an electronic book, or any other text, document or electronic media content viewer. For example, the second application may be an on-demand soundtrack player or an internet browser accessing a soundtrack player website managed by a soundtrack server.

In a first aspect, these first and second applications may operate independently without exchanging data. In a second form, the first application may include a title or detail identifying an item of the electronic media content that is reading, and / or the current reading position of the user, the reading speed, data representing page turning, scrolling information, To the second application or to make the data useful for retrieval by such second application. The data communication link between the first and second applications may be via an application programming interface (API) or the like.

In this embodiment, a first application running on a user device is configured to display electronic media content in response to user interaction, and a second application is configured to display a compiled sound track for electronic media content And controls playback. For example,

Load electronic media content for display via a first application;

Transmit control data requesting a streamed compiled sound track corresponding to electronic media content to a remote server via a second application;

And to receive and play the streamed compiled sound track received from the remote server via the second application.

In a second embodiment, the user device further comprises a display configured to display electronic media content. In this embodiment, the user device is configured to execute a single multimedia application configured to control display of electronic media content via user interaction and to control playback of the streamed compiled sound track.

In a first form, the multimedia application may be an Internet browser configured to receive and process electronic media content received from a remote server in the form of a website server and a streamed compiled sound track. In one form, the Internet browser may receive electronic media content and streamed compiled soundtracks as a single combined data stream, or alternatively, electronic media content and streamed compiled soundtracks in a different form may be in separate data streams Lt; / RTI > In one form, the electronic media content may be downloaded entirely for display, or alternatively downloaded as chunks, displayed, or streamed. Such soundtrack streaming and / or playback functionality may be performed by the Internet browser itself or by a plug-in application program to a browser, or by an application embedded in a web page.

In a second aspect, the multimedia application may be an application-specific application configured to search for and display electronic media content and to control the playback of a streamed compiled sound track from a multimedia content server. In one form, the multimedia content server may provide electronic media content in the form of an electronic book, and the streamed compiled sound track corresponds to a sound track for the electronic book. In a first form, the electronic book data may be transmitted in a separate data stream (and the audio stream may be transmitted in a separate data stream) regardless of whether it is downloaded in advance or chunked or streamed. In another form, the e-book data and audio stream may be transmitted in a combined multimedia data stream.

In a third embodiment, a user device is configured to provide an audible sound track to a user and is in data communication with a second separate user device that includes a display and is configured to present the user with visual electronic media content. For example, data communication between a pair of user equipments can be done directly or indirectly through radio frequency (RF), near field radio (NFC), Bluetooth, Wi-Fi or other such wireless communication media or protocols, A wired communication medium may be provided when a pair of user devices are connected via a cable.

In such an embodiment, the second separate user device (text display device) may be configured to display, for example, the user's current reading location, reading speed, data indicative of page turnover, scrolling information, Data indicative of user read information including data indicative of title or details identifying an item of electronic media content to a first user device (audio reproduction device), or to data useful for retrieval by such first user device .

Such a system may have one or more of the characteristics mentioned for the first to fourth aspects of the present invention.

In a sixth aspect, the present invention provides, in general, a method of reproducing audio content in an audio domain, the audio track for playback in an audio domain, a start position in an electronic media content corresponding to where the playback of the audio domain begins, CLAIMS What is claimed is: 1. A method for streaming an uncompiled sound track defined by a plurality of audio regions, each defined by a stop position in content, to a reader remote user device of an electronic media content,

Transmitting a sound track data file including data defining an audio region of the sound track to the remote user device over a network;

Receiving control data from the remote user device over the network requesting one or more audio track files corresponding to an audio region of the sound track; And

And transmitting one or more audio track files to the remote user device over the network in response to the received control data.

In one aspect, the method includes receiving random or periodic control data requesting a separate audio track file, and, upon request, transmitting the requested audio track file, one by one, to a remote user device.

In another aspect, the method includes receiving control data having a single request for all audio track files associated with the sound track, transmitting all audio track files to the remote user device, either singly, batchwise, or otherwise . The method includes transmitting audio track files in a playback order in their soundtrack.

In another aspect, the method includes receiving random or periodic control data including a request for placement of an audio track file, and, upon request, transmitting the arrangement of the requested audio track file to the remote user device .

In a seventh aspect, the present invention relates generally to a system and method for controlling playback of an audio file, the audio track comprising an audio track for playback in an audio domain, a start position in the electronic media content corresponding to where the playback of the audio domain begins, A sound track server for streaming an uncompiled sound track defined by a plurality of audio regions, each defined by a stop position in content, to a reader remote user device of an electronic media content,

Transferring a sound track data file containing data defining an audio region of the sound track to a remote user device over a network;

Receiving control data from the remote user device over a network requesting one or more audio track files corresponding to an audio region of the sound track;

And a sound track server configured to transmit one or more audio track files over the network to the remote user device in response to the received control data.

The sound track server may further include one or more of the features mentioned with respect to the sixth aspect of the present invention.

In the eighth aspect, the audio track for reproduction in the audio area, the start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and the stop position in the electronic media content corresponding to the stop of reproduction of the audio area CLAIMS What is claimed is: 1. A method for reproducing an uncompiled sound track defined by a plurality of audio regions,

Transmitting control data requesting a sound track for reproduction to a remote server via a network;

Receiving a sound track data file defining the audio region of the sound track from the remote server in response to the transmitted control data;

Processing the sound track data file to find at least a next audio region to be played back based on a user read position variable indicating the user's read position in the electronic media content;

Transmitting periodic or arbitrary control data requesting an audio track file corresponding to at least the next audio region to be reproduced to one or more audio track file sources;

Receiving a requested audio track file from the one or more audio track file sources and storing the requested audio track file in a memory; And

And adjusting the reproduction of the stored audio track file as defined by the associated audio region in the sound track data file, based on the user read position variable.

In an embodiment, the sound track data file includes source data representing one or more audio track file sources requesting each audio track file. In some forms, the sound track data file may define a single audio track file source requesting all audio track files. In another form, the sound track data file may define two or more different audio track file sources that request an audio track file.

In a first embodiment, the audio track file source is a remote server, and the method comprises requesting all audio track files from the remote server.

In a second embodiment, the audio track file source comprises the remote server and one or more other audio track file sources, the method further comprising the step of, based on the source data in the soundtrack data file, And requesting the audio track file from a track file source. For example, some audio track files may be obtained from the remote server, and other audio track files may be obtained from one or more other audio track file sources.

In a third embodiment, the one or more audio track file sources are separate from the remote server, and the method comprises requesting all audio track files from one or more separate audio track file sources. For example, all audio track files are obtained outside the remote server.

In one aspect, the method includes transmitting control data requesting individual audio track files one by one to the one or more audio track file sources.

In another aspect, the method includes transmitting control data to the one or more audio track file sources, the control data including a single request for all audio track files associated with the audio track of the soundtrack.

In another aspect, the method includes transmitting control data requesting placement of an audio track file to the one or more audio track file sources.

In another form, the method comprises the steps of: maintaining a sliding download window with respect to the user read position variable (corresponding to a play position in the soundtrack); and if all audio track files associated with the audio region in the sliding window And sending control data to the one or more audio track file sources requesting all of the audio track files associated with the audio region in the sliding download window to be received and stored based on the variable and ready for future playback. The size of the sliding download window may be configurable.

In one aspect, the method further comprises deleting the audio track file immediately or after a configurable predetermined delay from memory after the corresponding audio region of the audio track file has completed playing.

In one aspect, the method includes analyzing a sound track data file for an audio track file that is repeated in more than one audio region of the sound track, analyzing the audio track file as a repeated audio track upon receiving and storing the audio track file . The method further comprises retaining each of the repeated audio tracks in a memory until the last audio region associated with the audio track completes reproduction.

In a ninth aspect, the present invention relates generally to a system and method for audio playback, comprising an audio track for playback in an audio region, a start position in an electronic media content corresponding to where the playback of the audio region is to begin, CLAIMS What is claimed is: 1. A user device for reproducing an uncompiled sound track defined by a plurality of audio regions, each defined by a stop position in content, for a reader of electronic media content,

A user interface configured to receive user input to control the user device;

An audio output system operable to generate an audio output for the user to hear; And

The processor comprising:

Transmitting control data requesting a sound track for playback to a remote server via a network;

Receiving a sound track data file defining the audio region of the sound track from the remote server in response to the transmitted control data;

Processing the sound track data file to find out at least a next audio region to be played back based on a user read position variable indicating the read position of the user in the electronic media content;

Transmitting periodic or arbitrary control data requesting an audio track file corresponding to at least the next audio region to be reproduced to one or more audio track file sources;

Receiving a requested audio track file from the one or more audio track file sources and storing the requested audio track file in a memory of the user device;

And an uncompacted sound track playback user device configured to adjust the playback of the stored audio track file as defined by the associated audio region in the sound track data file based on the user read position variable.

In one form, the user device further comprises a display configured to play back the electronic media content such that both the audible sound track and the visible electronic media content are presented to the user in the same user device.

In another aspect, a user device is configured to provide an audible soundtrack to a user and is in data communication with a second separate user device that includes a display and is configured to provide visual electronic media content to a user. For example, data communication between a pair of user equipments can be done directly or indirectly through radio frequency (RF), near field radio (NFC), Bluetooth, Wi-Fi or other such wireless communication media or protocols, A wired communication medium may be provided when a pair of user devices are connected via a cable.

The user device may further include one or more of the features mentioned for the eighth aspect of the present invention.

In a tenth aspect, the present invention relates generally to a system and method for reproducing audio data in an audio domain, the audio track for playback in an audio domain, the start position in the electronic media content corresponding to where the playback of the audio domain begins, CLAIMS What is claimed is: 1. A system for streaming uncompiled sound tracks defined by a plurality of audio regions, each defined by a stop position in content, to a reader remote user device of an electronic media content, the system comprising a user device and a remote server Wherein the user device comprises a user device and a remote server,

A user interface configured to receive user input to control the user device;

An audio output system operable to generate an audio output for the user to hear;

A data communication module configured to communicate with a remote server via a network; And

The processor comprising:

Transmitting control data requesting a sound track for playback to a remote server via a network;

Receiving, in response to the transmitted control data, a sound track data file defining the audio region of the sound track from the remote server;

Processing the sound track data file to find out at least a next audio region to be played back based on a user read position variable indicating the read position of the user in the electronic media content;

Transmitting periodic or arbitrary control data requesting an audio track file corresponding to at least the next audio region to be reproduced to the remote server;

Receiving the requested audio track file from the remote server and storing it in the memory of the user device;

And to adjust the reproduction of the stored audio track file as defined by the associated audio region in the sound track data file based on the user read position variable,

The remote server comprising:

Transmit a sound track data file to the user device over the network in response to a control signal from the user device;

Receiving from the user device via the network control data requesting one or more audio track files corresponding to the audio track of the sound track;

And a system configured to transmit the one or more audio track files via the network to the user device in response to the received control data.

In a first embodiment, the user device further comprises a display configured to display electronic media content. In such an embodiment, the user device is capable of searching electronic media content (such as an e-book server, store or library) or searching electronic media content from a data file stored locally or in a database that can be retrieved, streamed, And a second application configured to receive and process streamed compiled sound tracks from a remote server and to control playback of the compiled sound tracks via the audio output system of the user device As shown in FIG. For example, the first application may be an Internet browser viewing a web page, electronic book terminal software for viewing an electronic book, or any other text, document or electronic media content viewer. For example, the second application may be an on-demand soundtrack player or an internet browser accessing a soundtrack player website managed by a soundtrack server.

In a first aspect, these first and second applications may operate independently without exchanging data. In a second form, the first application may include a title or detail identifying an item of the electronic media content that is reading, and / or the current reading position of the user, the reading speed, data representing page turning, scrolling information, To the second application or to make the data useful for retrieval by such second application. The data communication link between the first and second applications may be via an application programming interface (API) or the like.

In this embodiment, a first application running on a user device is configured to display electronic media content in response to user interaction, and a second application is configured to display a compiled sound track for electronic media content And controls playback. For example,

Load electronic media content for display via a first application;

Transmit control data requesting a streamed compiled sound track corresponding to electronic media content to a remote server via a second application;

And to receive and play the streamed compiled sound track received from the remote server via the second application.

In a second embodiment, the user device further comprises a display configured to display electronic media content. In this embodiment, the user device is configured to execute a single multimedia application configured to control display of electronic media content via user interaction and to control playback of the streamed compiled sound track.

In a first form, the multimedia application may be an Internet browser configured to receive and process electronic media content received from a remote server in the form of a website server and a streamed compiled sound track. In one form, the Internet browser may receive electronic media content and streamed compiled soundtracks as a single combined data stream, or alternatively, electronic media content and streamed compiled soundtracks in a different form may be in separate data streams Lt; / RTI > In one form, the electronic media content may be downloaded entirely for display, or alternatively downloaded as chunks, displayed, or streamed. Such soundtrack streaming and / or playback functionality may be performed by the Internet browser itself or by a plug-in application program to a browser, or by an application embedded in a web page.

In a second aspect, the multimedia application may be an application-specific application configured to search for and display electronic media content and to control the playback of a streamed compiled sound track from a multimedia content server. In one form, the multimedia content server may provide electronic media content in the form of an electronic book, and the streamed compiled sound track corresponds to a sound track for the electronic book. In a first form, the electronic book data may be transmitted in a separate data stream (and the audio stream may be transmitted in a separate data stream) regardless of whether it is downloaded in advance or chunked or streamed. In another form, the e-book data and audio stream may be transmitted in a combined multimedia data stream.

In a third embodiment, a user device is configured to provide an audible sound track to a user and is in data communication with a second separate user device that includes a display and is configured to present the user with visual electronic media content. For example, data communication between a pair of user equipments can be done directly or indirectly through radio frequency (RF), near field radio (NFC), Bluetooth, Wi-Fi or other such wireless communication media or protocols, A wired communication medium may be provided when a pair of user devices are connected via a cable.

In such an embodiment, the second separate user device (text display device) may be configured to display, for example, the user's current reading location, reading speed, data indicative of page turnover, scrolling information, Data indicative of user read information including data indicative of title or details identifying an item of electronic media content to a first user device (audio reproduction device), or to data useful for retrieval by such first user device .

Such a system may have one or more of the characteristics mentioned for the first to fourth aspects of the present invention.

In another aspect, the invention is generally embodied in a computer-readable medium having computer-readable instructions stored thereon that, when executed on a processing device, causes the processing device to perform one or more of the methods defined in the above aspects of the invention .

Justice

As used herein and in the claims, the term "sound track ", unless stated otherwise, is a sound track to be played back while reading electronic media content, comprising a plurality of audio regions configured to be played at predetermined read positions or regions within the electronic media content Means a sound track of the type described in WO2013 / 015694 in which the playback timing of the sound track is synchronized with the user based on the user's estimated read position and / or read speed.

The phrases " streaming ", "stream ", or" streamed ", as used in this specification and claims, unless otherwise stated, are intended to refer to a sound track having the form of a compiled soundtrack ready for reproduction or an uncompounded soundtrack Regardless of whether the streaming is controlled by a server, a user device, or a combination thereof, the user can enable the user to start playing the soundtrack without downloading the entire soundtrack, and continuously streaming the soundtrack &Quot; is intended to mean streaming a sound track from a remote server to a user device in any form, including streaming data, and arranging the sound track, and transmitting it to a chunk, portion, or section.

The phrase "compiled soundtrack" used in the present specification and claims includes sound tracks compiled in the form of digital audio files ready for playback unless otherwise specified, duration of the soundtrack to suit readers having different read rates The compiled sound track is intended to be compiled into any digital audio format for playback by a digital audio player or audio player software, And / or compressed, and includes real-time compiled audio files or precompiled audio files.

As used herein and in the claims, the phrase "uncompiled sound track" refers to an audio track file associated with each audio region and a sound track data file defining playback data for multiple audio regions of the sound track Wherein the uncompiled sound track processes the soundtrack data file and determines a user's read position in the electronic media content based on data, a counter, or a variable, For playback by a sound track player which adjusts the synchronized reproduction of the audio track file of the audio area by readout of the audio track file.

The phrase "user device" as used in this specification and claims is intended to encompass all types of electronic devices, including but not limited to smart phones, mobile phones, tablets, e-book terminals or electronic book devices, laptops and notebooks, gaming machines or consoles, smart televisions, Is intended to mean any form of programmable electronic hardware device, platform, or system, including, but not limited to, a portable or non-portable consumer electronics device,

The term "network" as used herein and in the claims is intended to encompass all types of networks, including, but not limited to, the Internet, an intranet or extranet, a local area network (LAN), and / Any network or data communication channel or network or device that can transmit and / or receive data to another device, regardless of the remote server or other user device, including but not limited to a wireless local area network (WLAN) Channel < / RTI >

The term "server" as used in this specification and claims, unless otherwise stated, performs a dedicated server or server function, regardless of the website server, dedicated soundtrack server, electronic media content server, or any other form of server Is intended to mean any form of electronic server that is capable of transmitting or streaming sound tracks to, or transmitting or streaming to, a user device over a network, regardless of platform, such as one or more clients or May be configured to assist or manage the user device, and may include any form of data communication between the user device and the server, whether wired, wireless, or a combination thereof.

The phrase "electronic media content" as used in this specification and claims is intended to mean any electronic media content, including electronic text, either alone or in combination with fixed or dynamic images or graphics, The electronic media content can be any format or file type that can be made in an electronic display and includes, but is not limited to, electronic books, web pages, digital video, or slide show presentations.

As used in the specification and claims, the term " comprising " means "consisting at least in part ". As interpreted in the present specification and claims, including the term " comprising ", there may also be features other than those listed in the opening paragraph. Relevant terms such as "include" should be interpreted in the same manner.

As used herein, the terms "and / or" means "and" or ", or both.

As used herein, "(s)" following a noun means plural and / or singular forms of a noun.

The present invention includes configurations in which only the following examples are given.

In the following description, specific details are given for a complete understanding of the embodiments. However, those skilled in the art will appreciate that the embodiments may be practiced without these specific details. For example, software modules, functions, circuits, etc., may be shown in block diagram form for clarity of illustration. In other instances, well-known modules, structures, and techniques may not be shown in detail for purposes of understanding the present embodiments.

It should also be noted that the present embodiment can be described as a process shown as a flowchart, a structural diagram, or a block diagram. Although the flowchart can describe the operation as a continuous process, many operations can be performed in parallel or concurrently. Also, the order of such operations can be rearranged. The process is terminated when its operation is completed. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, and the like in a computer program. When the process corresponds to a function, its termination corresponds to the return of the function to the calling function or main function.

Preferred embodiments of the present invention will be described by way of example only with reference to the following drawings.
1 is a schematic data plot of a portion of text and the associated audio region of a sound track configured for playback during reading of the electronic text.
Figure 2 is a schematic data plot similar to Figure 1, except that the sound track of the present invention configured for electronic media content including text and images is shown.
3A is a schematic diagram of a sound track delivery system according to a first embodiment of the present invention in which a separate application of the user device controls the display of electronic media content and the playback of a sound track based on an audio stream from a remote server.
3B is a block diagram of a sound according to a second embodiment of the present invention, employing a single multimedia application to control the display of electronic media content and playback of a sound track based on the electronic media content and audio data stream from the remote multimedia content server. Fig.
4 is a block diagram of an embodiment of the present invention in which a user device controls the display of electronic media content based on an audio data stream from a remote server and another user device employs a pair of user devices Fig. 7 is a schematic diagram of a sound track delivery system according to a third embodiment of the invention.
5a is a block diagram of a first embodiment of the present invention for transferring a precompiled sound track audio file selected from a library of precompiled sound tracks in a range of different read rates from a sound track server to a remote user device via an audio stream. Fig. 2 is a schematic view of a sound track delivery system according to the invention;
FIG. 5B is a flow chart showing resynchronization to a new read position in the system of FIG. 5A.
FIG. 5C is a flow chart showing a transition protocol for a change in read speed in the system of FIG. 5A. FIG.
6 is a schematic diagram of a soundtrack delivery system according to a second aspect of the present invention for delivering a real time compiled soundtrack audio file from a soundtrack server to a remote device via an audio stream;
7 is a schematic diagram of a soundtrack delivery system according to a third aspect of the present invention for delivering uncompiled soundtracks from a soundtrack server to a remote user device via an audio stream;

1. Synchronized soundtrack overview

The present invention relates to the transfer of a sound track of the type described in WO2013 / 015694 mentioned above. These soundtracks are created to enhance the user's reading experience for items of electronic text.

These soundtracks can be played in a variety of formats such as aeh, djvu, epub, fb2, html, xhtml, azw, lit, prc, mobi, exe, pdb, txt, pdb, pdf, ps, rgo, pdb, tr2, tr3, wol, rtf, doc, docx, asc, lit, wri, odt, text, pwi, rtx, sdw, strings, utf8, utf6, wp *, wpw, wri, wtx, asc, csv, lwp, dtp, indd, pmd, ppp, pub, fm Including but not limited to those associated with file extensions of any type of markup language type including but not limited to SGML, HTML, XHTML, HTML5, XML, and LaTex, or It can be applied to electronic text as a file type. Such electronic text may have the form of any electronic media content, either static or dynamic (e.g., video or other animation), including text only or text combined with an image or graphics. For example, the electronic media content may have the form of an electronic book, a web page, or any other electronic media content format.

Soundtracks are customized to electronic media content and customized to the content of electronic text to enhance the reading experience, especially for readers of electronic text. These soundtracks may include different types of audio tracks and may be configured to be played simultaneously, subsequently, or both such as while the user is reading the electronic text. The reproduction of such a sound track is synchronized with the user's reading speed (e.g., words per minute or scrolling speed or other measurements). In one example, the electronic text may be a novel or non-fiction book, and the sound track may be configured to play at a particular portion of the electronic text or at a particular point in a synchronized manner when the user is reading a specific portion below the electronic text Which may include combinations of different audio types, including ambient noise, sound effects, and background music.

The soundtrack includes a number of audio regions designed to reproduce during a specified text portion of the electronic text. Each audio region includes audio data defining audio tracks to be played back and audio characteristics defining the desired characteristics of playback. This audio data specifies the start text position in the electronic text where the audio area will start playing and the stop text position in which the audio area will stop playing. In one embodiment, the electronic text is assigned a numeric text index based on the number of electronic text or words or characters, and the start and stop text positions are defined by respective corresponding start and stop index values. In one example, the indexing of the text is based on the number of words in the electronic text such that each word has a unique index value that represents its position in the electronic text. The index value for each word may correspond to the number of words in the electronic text or the word count. In this example, the audio region is played back between the start and stop index values indicating the selected start word and stop word in the electronic text for that audio region.

When the user reads the electronic text, a read position counter or identifier corresponding to the text position in the electronic text that the user is currently presumably reading is maintained. This read position counter increases or varies at a rate based at least in part on a user read speed variable indicative of the user ' s read speed. The reproduction of the audio area of the sound track can be synchronized with the reading speed of the user by this reading position counter so that the reproduction of the audio area is triggered when the reading position counter matches the starting index value of the starting word for the audio area, Similarly, when the read position counter matches the stop index value of the stop word in the audio area, it is stopped.

In alternative embodiments, other read location referencing schemes may be employed in contrast to word or character based indexing. In other embodiments, the start and stop positions of each audio region may be defined by any other document or text location referencing scheme or system. One such example is the ePub canonical fragment identifier (CFI). In this example, the start and stop positions in the audio region will be defined by the CFI parameter or range, and the read position variable is set such that the readout position variable < RTI ID = 0.0 > Or in an increasing CFI format.

The sound track includes a plurality of audio regions and the reproduction of two or more regions may be partially or entirely overlapped so that two or more audio regions can be reproduced simultaneously if desired. The sound track is provided with multiple channels for playback and the overlapping audio regions are designated as different channels in a set of available channels to allow for simultaneous playback. This provides different sound tracks such as sound effects, ambient noise, or background music simultaneously playing back and mixing together to provide a full soundtrack that enhances the user's reading experience with the electronic text. Usually, the audio data for each audio region includes audio types such as "music "," atmosphere ", or "effect ". The selected audio type is intended to define the nature of the audio domain. For example, if the audio region is for background music, then the selected audio type would be "music ". If the audio region is a sound effect, the "effect" audio type is selected. If the audio region is intended to be ambient sound or noise, the "Atmosphere" audio type is selected.

For example, Figure 1 shows a portion of an electronic media content that includes only electronic text. 1 also overrides the text with a schematic representation of the number of audio regions 10 that are configured for playback during reading of the electronic text in a schematic representation and the audio region 10 is a bar extending below the portion of the associated text, Respectively. The start of each bar or underline of the audio area indicates the start position at which the audio track of the audio area starts to play, and the end 10 indicates the position in the electronic text in which the audio track of the audio area stops playback. Each audio region may extend over a single word or group of words, lines, multiple lines, or even a page. As shown, the audio regions can be superimposed and can be played simultaneously.

In Fig. 2, the electronic media content may include multimedia content, including text and images, statically or dynamically. Figure 2 shows an electronic text including a static image 20. In such electronic media content, the sound track audio region may have start and stop positions corresponding to text or images. For example, the audio region may be configured to start at a specific word and then terminate when the user is expected to see a particular image embedded in the text. Alternatively, the audio region may be configured to start playback when the user is expected to see a particular embedded image. For image-based multimedia content, such as cartoons, the read location identifier may identify an image that the user is expected to see and the read rate may be, for example, in the form of a scroll or scan rate for an image, such as an image per second or per minute Lt; / RTI >

2. Streaming the sound track from the remote server to the user device

survey

The present invention relates primarily to the streaming of synchronized sound tracks from remote to user devices over a network. These soundtracks can be streamed into two alternate forms: compiled soundtracks or uncompiled soundtracks. In one embodiment, a single user device is provided for displaying visible electronic media content and for receiving a streamed sound track for audible reproduction. In another embodiment, a pair of separate user devices for data communication are provided. One of these user devices provides visible electronic media content and the other user device receives a streamed sound track for audible playback. The data communication between these devices allows the sound track reproduction to be synchronized with the display of the electronic media content.

In the first form, when streaming as a compiled soundtrack, the server compiles a sound track prepared for playback by any suitable standard or custom digital audio player software, regardless of the plug-in or function of the standalone application or other application Streamed digital audio files. A single digital audio file of these soundtracks can be streamed or alternatively can be streamed to a particular type of soundtrack, such as a 'music' or 'effect' or 'mood', or some other breakdown, Two or more audio files (including audio regions) may be streamed together to be ready for playback together to reproduce the entire soundtrack. Alternatively, the user may deactivate one or more audio files from playback to effectively reduce one or more types of audio regions from playback. The audio file of the sound track delivered by the server is compiled at a configured read rate that can be selected by the user to match the user ' s individual read rate so that the playback timing of the audio region corresponds to their predicted read position in the electronic media content. .

In a second form, when streaming as an uncompiled soundtrack, the server streams the soundtrack data file and the audio track file relating to the audio region of the soundtrack to the user device. Such a user device comprises a customized soundtrack playback module or function configured to process the soundtrack data file and to adjust the synchronized playback of the audio track file of the audio area with the user's estimated or predicted readout position of the electronic media content . Such a soundtrack playback module or function may be a stand-alone application or integrated with other applications such as a plug-in. The sound track playback module maintains a user read position variable indicating the user's expected read position of the electronic media content and triggers the playback of the audio region defined in the sound track data file based on this user read position variable and is described in WO2013 / 015694 Lt; RTI ID = 0.0 > soundtrack < / RTI > playback module. By way of example, if the audio region of the soundtrack is defined by the start and stop positions associated with the word index or count as described above, then the user readout position variable may be a counter corresponding to the estimated word of the electronic text that the user is reading . This counter may then be increased in a rate based at least in part on a configurable read speed variable indicative of the user ' s read speed. These counters can also be used for any page reading, such as page turning, scrolling, user clicking on the mouse cursor for the current word the user is reading, or tapping (for touch screen display) May be updated at random or periodically to the actual reading position of the user by various user interactions or user inputs including location update inputs, or by other means for updating user read position variables.

In the case of the streaming of both sides, for a compiled and uncompiled sound track, the sound track may be transmitted at a data transmission rate that is the rate of data reproduction at the user device, or more preferably, May be streamed at a higher transmission rate, taking into account data transmission delays or interruptions caused by device performance, and the like. When streaming at a data transmission rate higher than the data reproduction rate, a buffer for temporarily storing input data prepared for future reproduction is provided to the user device. The size of these buffers can be fixed or dynamically changed during streaming. This buffer size may be configurable and controlled by the user device or server. If desired, the buffer size may be set to zero to effectively limit the data transmission rate to the data reproduction rate. Optionally, the system may be configured to change the bit rate or quality of the sound track delivered to the user device based on the bandwidth of the data channel between the server and the user device.

Such soundtracks may be combined with or otherwise streamed to the corresponding electronic media content, and the presentation and playback of the electronic media content and soundtrack may be implemented as separate or combined application programs in embodiments employing a single user device , Or in a separate application program running on a separate device in an embodiment employing a pair of user devices in data communication. In some forms, a user may have electronic media content already stored on the user device, and may only request a corresponding sound track from the server for playback on that user device or on a separate user device while reading the electronic media content. With reference to Figures 3a, 3b and 4, some examples of these different system configurations will be described.

Example 1 - A separate application on your device for text and soundtracks

3A illustrates a first example 30 of a system configuration in which a user device 32 executes a separate application for controlling the display of electronic media content and the playback of a synchronized sound track received from the sound track server 34 Respectively. In the system configuration 30, the user device 32 is connected to an electronic media content server (such as an e-book server, store or library) on the display 38 of the user device or to a database that is retrieved, streamed, (Electronic media content or text application) responsive to the retrieval, processing and display of electronic media content from a locally stored data file. This text application 36 may have any form of electronic media content viewer or editor. In one example, such a text application may be electronic book terminal software for displaying electronic media content in the form of an electronic book file. Alternatively, in another example, the text application may be an Internet browser viewing electronic media content in the form of a web page.

The second application (soundtrack application) is operable to receive, process and play back the audio data stream representing the synchronized soundtrack for the electronic media content received from the remote soundtrack server (34) over the network (42). Such a soundtrack application may interact with the soundtrack server 34 via control data 42 and may receive one or more audio streams 44 from the soundtrack server 34 and process the input audio data, Or an application program that is capable of playing a sound track through the audio output system 46 of the computer 32. It will be appreciated that the audio output 46 may be any type of audio transducer, including an integrated or external speaker, headphone, earphone, and the like.

In a first form, the text application 36 and the soundtrack application 40 can operate independently without exchanging data. In a second form, the applications 36, 40 may transmit or create useful data to each other via a data communication channel or link 41, such as an API. For example, the text application 36 may identify an item of electronic media content that is being read, for example, the user's current reading location, reading speed, data indicative of page turnover, scrolling information, To the soundtrack application 40, or to data useful for retrieval by such a soundtrack application 40. In some embodiments,

In one example, the soundtrack application 40 may be a custom soundtrack player application configured for the purpose of accessing a synchronized soundtrack for associated electronic media content from a remote soundtrack server 34. [ Alternatively, the soundtrack application may be an add-on or a plug-in for another main user software application. In one such example, the soundtrack application may be integrated with an Internet browser, and such a soundtrack application may be a plug-in application to an Internet browser. In another example, the soundtrack application may be a soundtrack library managed by the soundtrack server 34 or an internet browser accessible to a soundtrack player web page.

The exemplary operation of the first example system configuration 30 will now be described. First, the user loads the electronic media content to be displayed through the text application 36. The user then initiates their soundtrack application 40 and selects a synchronized soundtrack corresponding to the electronic media content from the soundtrack server 34 based on the loaded electronic media content or the application automatically selects . For example, a user may operate the soundtrack application 40 to access a soundtrack library 35 of soundtrack servers displaying useful soundtracks for different electronic media content. These sound track libraries 35 represent all the sound tracks stored in the sound track database 37 that are integrated with or accessible by the sound track server. For example, if the user is displaying an e-book in the text application 36, the sound track server 40 can be inspected via the sound track application 40 for one or more associated sound drags configured or tailored for the particular e-book have. The user interacts with the sound track server via control data 42 sent by their sound track application 40 in response to user input / interaction. Once the soundtrack has been selected, the server 34 may stream the soundtrack to the user device 32 via one or more audio data streams 44 for playback by the soundtrack application 40.

Stream compiled soundtracks

If the server is configured to stream the compiled soundtrack, a user read speed setting is configured for the user and this can be stored in the user read information module 31. [ The sound track application 40 may be implemented in various ways, including, for example, selecting a user numeric read speed, selecting a read speed from a list or scale, and performing a read speed check provided by the sound track server via the sound track application 40. [ And may be sent via control data 42 to the server or the user's read rate may be extracted from historical data stored in the soundtrack application 40 or the soundtrack server 34, have. Once the user read speed setting is configured, the server is configured to stream the compiled sound track corresponding to the user read speed setting. As will be described in more detail later, the compiled soundtrack may be progressively compiled before being streamed or compiled in real time at a particular read rate, or alternatively the library of soundtracks that are precompiled may be in a range of different read rates (Either automatically by the server or manually by the user) to be streamed to the user device, which may be available at the server at the server and matches the user's read speed setting most closely. The soundtrack application 40 may store or buffer the input audio data stream in memory and the user may begin playing the audio file ready to play the soundtrack through the soundtrack application as it begins to read, May be automatically initiated by the soundtrack application 40 based on data from the text application 40 representing the user's interaction with the user.

Uncompiled soundtrack

If the server is configured to stream an unkill filed soundtrack, the soundtrack data file and the audio track file defining the audio region may be provided to the server 34 (e. G., In response to control data 42 from the soundtrack application 40 of the user device) To the user device 32 via one or more audio data streams 44. [ The soundtrack application 40 stores or buffers the input audio data stream in memory. As will be discussed further below, the soundtrack data file may be transmitted in its entirety or transmitted in a predefined amount or portion. Likewise, the audio track files may be sent one by one in the sequence of playback or in a group corresponding to a predefined amount or portion of the soundtrack. The user configures the user's read speed setting (manually or automatically detected) to the soundtrack application 40 of the user device using one of the techniques described above. The user can start playing the sound track as it begins to read, or it can happen automatically as described above, which causes the soundtrack application to display the user's estimated or predicted readout position in the electronic media content, To process the soundtrack data file based on the user read position variable representing the audio track file and to trigger an adjusted reproduction of the audio track file corresponding to the audio region.

Example 2 - A single application on your device for text and soundtracks

3B shows a user device 32 executing a single application (a multimedia application) to control the display of electronic media content and playback of a synchronized sound track, while the remainder is another system configuration 60 (similar to the first system configuration 30) In the second example. Like numbers refer to like elements and parts in the drawings. In this example, a single multimedia application 62 is configured to run on the user device 32 and is capable of displaying electronic media content and playing a soundtrack, in a first system configuration 30, ). ≪ / RTI >

In such a system configuration 60, the multimedia application 62 may be configured to receive an audio stream 44 representing a sound track synchronized to the electronic media content seen at the user device in a manner similar to that described for the first system configuration 30. [ And may be configured to interact with the multimedia content server 64 by control data 42.

The multimedia application 60 may obtain electronic media content from an electronic media content server (such as an e-book server, store or library) or from a data file stored locally or in a database that is retrieved, streamed or accessible from online or the Internet. In one example, the multimedia data application may completely download, progressively download, or stream the electronic media content from the multimedia content server 64 via the electronic media content data channel 66. In this example, the multimedia content server 64 may include an electronic media content module 68 configured to control delivery of electronic media content to the user device via the network 43.

In one example, the multimedia application 62 is a custom software application program configured to interact with a multimedia content server, including multimedia content that includes electronic media content and associated synchronized sound tracks to be displayed and played, respectively, Lt; RTI ID = 0.0 > a < / RTI > custom software application program. For example, the multimedia content server may provide a library 7 of e-books and corresponding synchronized sound tracks tailored to such e-books. Such electronic book data may be transferred to the electronic media content data stream 66 and the sound track may be streamed to the user device with the audio data stream 44. [ Alternatively, the electronic media content and the audio data stream do not have to be separate and can be combined into a single multimedia data stream.

In another example, the multimedia application 62 may take the form of an Internet browser. In this example, the multimedia content server may be a web site server, and the electronic media content may be a web page that is downloaded to the Internet browser via the electronic media content data stream 66. At the same time, the Internet browser may have a sound track player function as a plug-in application, or alternatively the sound track player function may be an application embedded in web page content configured to control the audio stream stream 44 for sound track playback.

The soundtrack application function of the multimedia application 62 operates similarly to the first system configuration 30 as described above in terms of streaming a compiled or uncompiled soundtrack. The multimedia application 62 has a unique knowledge of the user's interaction with the electronic media content so as to facilitate coordination of synchronized playback of compiled or uncompiled sound tracks as will be described in more detail later Based on the user interaction / input (e.g., page turning, scrolling, any read location update input, etc.), the updated user read position may be determined periodically or at random.

Example 3 - A pair of user devices

Figure 4 shows a third example of another system configuration 80 provided with a pair of devices 82,84. The first user device 84 (audio playback device) has an audio output 85 and is configured to provide an audible sound track to the user and the second user device 82 (text display device) ) And is configured to provide visible electronic media content to the user. This third system configuration 80 is similar to the first system configuration 30 of Figure 3a except that the text application 36 and the soundtrack application 40 are separated to operate on separate user devices. Like numbers refer to like elements and parts in the drawings and will not be repeated.

For example, the data communication link 86 between a pair of user devices 82, 84 may be a radio frequency (RF), short range wireless communication (NFC), Bluetooth, infrared, WiFi, RTI ID = 0.0 > and / or < / RTI > The data connections between the user devices 82 and 84 function for similar purposes to the data communication channels 41 described in the system configuration 30 of FIG. In particular, the text display device 82 may be used to identify items of the electronic media content that are reading, for example, the user's current reading location, reading speed, data indicative of page turnover, scrolling information, Data indicative of user read information including data indicative of title or detail may be made available or transmitted for retrieval by audio playback device 84 over data link 86. [

Electronic media content for display on the text display device 82 may be stored in a database that is locally stored or accessible in such a device or such data 87 may be stored in an electronic media Downloaded from the content server 88, streamed, or retrieved. The electronic media content server 88 may be any type of electronic book server, store or library, a web site server, or any other electronic media server that is described for the system configuration 60 of FIG. 3B, May have any other form of electronic media content server, such as having the same electronic media content module 68 and electronic media content library 70. [

Text display device 82 is capable of executing an electronic book terminal device, tablet or smart phone, text application 36 executing an electronic book terminal application or other text application, Such as a desktop computer or any other electronic device capable of displaying electronic media content. Likewise, the audio playback device 84 may be a tablet, a smartphone, a computer, a dedicated audio playback device, or any other electronic device capable of executing a soundtrack playback application 40 to generate audible playback of the audio output 85. [ As shown in FIG. A user can operate any pair of such devices through another device 84 that are in data communication to view electronic media content in one device 82 while listening to a synchronized sound track.

Configure the server for electronic media content

As described above in the various system configurations (30, 60, 80), the electronic media content data may, in some embodiments, be streamed from the same server as the soundtrack, or alternatively from a separate server, Downloaded, or retrieved. In particular, in some forms, a multimedia server is provided that provides both soundtracks and electronic media content, and in another form a soundtrack server is provided and an electronic media content server is provided. In the case of a separate server, each server may be owned and operated by the same entity or by a separate entity.

Various embodiments or implementations of a soundtrack delivery system will now be described with reference to Figures 5A-7. In particular, an embodiment for streaming a compiled sound track will be described with reference to Figures 5A-6, and an embodiment for streaming an uncompiled sound track will be described with reference to Fig. This embodiment can be applied to one of the system configurations of Figs. 3A to 4. It is to be appreciated that although a single user equipment system configuration such as that of FIG. 3A and / or FIG. 3B is described, it may be adapted or modified to operate with a pair of user equipment as described in the system configuration of FIG.

3. Stream precompiled soundtracks

survey

In FIG. 5A, a first form of sound track delivery system 100 is configured to stream a precompiled sound track from a sound track server 104 to a user device 102. These soundtrack servers have an integrated or accessible soundtrack database 106 that includes a range of pre-compiled soundtrack audio files that are each compiled for different user read rates. Such a soundtrack server may automatically select a precompiled soundtrack audio file that most closely matches the user ' s read speed, or may allow the user to manually select and copy such files to the user device 102 via the network 108 .

hardware

User device 102 may be any programmable electronic device. The user device may include a display or display screen 113 such as a user interface 112 such as a processor 110, a memory 111, a keyboard, a mouse, a touch display screen, a control button, A communication module 114 for connecting to the sound track server 104 via the network 108 and an audio output section 115 capable of delivering sound to the user through an integrated or external speaker, . Such communication modules may communicate with a sound track server over a network by any wired or wireless communication medium, or a combination of these. For example, the communication module for a smartphone may be a 3G or 4G cellular data connection, or alternatively a WiFi connection to a broadband router. For a personal computer, the communication medium may be a wired or wireless connection to a local area network (LAN).

System behavior

In operation, the user device may be operable to access a soundtrack from the soundtrack server 104 via a soundtrack application or a multimedia application running on the user device as described with respect to Figures 3A-4. As previously described, the electronic media content may be streamed from a sound track server, a separate third-party server, or alternatively may be locally loaded by a user device in an alternative configuration. For your understanding, I'll focus on streaming soundtracks.

Select a soundtrack

The user can manually select a sound track for the electronic media content from the sound track library of the sound track server displaying the useful sound track data file 116 at the server. Alternatively, the soundtrack data file for the electronic media content may be automatically selected by the soundtrack server if it is providing electronic media content, or if downloaded or stored media content from a third party server is known. The user device interacts with the soundtrack server via the control data stream 118.

Then, a user read speed setting can be configured for the user. The user read speed setting may be transmitted from the user device to the sound track server via the control data stream 118. [ Alternatively, such a sound track server may store the user read speed for the user based on historical user read speed information from previous interactions.

In one form, the sound track server may include a sound track selector module 120 configured to maintain a user read speed variable indicative of a user ' s read speed. These soundtrack selector modules are configured to examine the soundtrack database for precompiled soundtrack audio files that most closely match the user read speed variable. In particular, the soundtrack database 106 includes a library of pre-compiled soundtracks at different user read rates for each soundtrack data file 116 of the soundtrack library. For example, a soundtrack database may include precompiled soundtrack audio files at a reading speed of 200 words per minute (wpm), 250 wpm, 300 wpm, 350 wpm, 400 wpm, and so on. Each of these precompiled soundtrack audio files will have a different duration corresponding to the compiled user settling rate. Faster read speeds have shorter duration audio files than slower read speeds. Each of the precompiled soundtrack audio files 122 has a read rate identifier that indicates the compiled or generated read rate. Such a read speed identifier may be in a sound track filename, or may be stored in a data or folder structure that identifies the speed of the pre-compiled checklist, embedded in the associated header file or information of the audio file, or encoded. The soundtrack selector 120 is configured to determine which precompiled soundtrack audio file 122 has a read speed identifier or has been precompiled at a read speed that most closely matches the user read speed variable. These selected sound tracks are then queued to be streamed to the user device via the audio streaming module 124 of the sound track server.

In an alternative form, the user can simply look at the precompiled soundtrack audio file in the database and select a soundtrack with a read rate identifier that he wishes to receive without constructing a user read speed variable at the soundtrack server.

Streaming soundtracks

Once the precompiled soundtrack audio file 122 is selected, the audio streaming module 124 sends a stream of the precompiled soundtrack audio file to the user device < RTI ID = 0.0 > 0.0 > 102 < / RTI > The audio stream 126 may be started at the beginning of the precompiled audio track or at the offset position of the precompiled soundtrack audio file corresponding to the user's desired starting reading position in the electronic media content. In one form, the user device 102 is configured to send control data 118 indicative of a user-requested starting reading position in the electronic media content such that the input audio stream begins at a location within the pre-compiled soundtrack audio file .

At the user device, the input audio stream 126 is temporarily received and stored in a buffer in memory prepared for playback. This buffer size may be configured by the sound track server either by the user device or alternatively, for example, via header information provided in the audio stream. Such a buffer size may be configured or varied to accommodate, for example, the processing and data connection rates of the user equipment. When the buffer is filled or the total capacity is reached, the control data 118 is transferred from the user device to the soundtrack server to stop data transmission by the audio stream until the capacity is resumed by the user device when it becomes available to the buffer .

When the user starts reading electronic media content, playback can begin with the audio output through a direct control of the soundtrack application or the soundtrack player of the multimedia application. For example, soundtrack playback may be triggered through user interaction with a soundtrack or multimedia application, such as by generating a read start signal by operation of a soundtrack or a play button on a graphical user interface (GUI) . Similarly, the user can stop sound track playback through a pause button or the like of the GUI. Alternatively, the sound track playback start and stop can be directly and automatically controlled through the user's interaction with electronic media content capable of issuing a signal user read start or stop signal. In such an embodiment, the sound track player may be a hidden background process that can not be directly controlled by the user.

In one form, the buffer size or range is configurable in relation to the current playback position in the audio stream. It may be large or small according to the user's needs within the capacity of the user equipment, and each is interchanged. If the buffer is small or absent (in terms of mobile or broadband data allocation subscriptions), data usage may be less, but due to network latency and / or latency, and the ability to quickly resynchronize to a new read location, The possibility that the track is blocked or not consecutive may be higher. Larger buffers may be more likely to consume more data, although the likelihood of continuous soundtrack playback is higher and more quickly resynchronized to a new read location. These buffers extend forwardly of the current playback position and optionally extend beyond the current playback position so that the portion of the audio stream to be played back is retained in the memory for a period of time. This buffer is essentially a sliding buffer for a reference point determined by the current playback position of the soundtrack in the audio stream.

Resynchronizing the soundtrack to a new reading location

In Figures 5A and 5B, the user may be enabled to cause the user device to generate data representing the new user read location in the electronic media content that the user is reading, or it may be automatically detected by the user device based on user interaction . For example, a user may indicate a new read location based on user interaction / input such as page turning, scrolling, any read location update input, and so on. For example, a user may jump or skip to a new page or portion of electronic media content and begin reading from a new read location.

Upon detecting the new read position 130, the sound track application or multimedia application running on the user device is configured to determine if the new user read position is within the range of the buffered portion of the streamed sound track in memory (131 ).

If the new read position is within buffer range 132, a pre-configured or configurable playback position transition is initiated (133). A variety of transition options are available. In one example, a feed 134 may be implemented to cause the soundtrack playback to suddenly skip or jump to a new playback position and continue playback from the new position. In another example, a stop transition 135 is implemented to stop the current playback and receive a signal that the sound track is to be played at the new location, but wait for a user start (activation of the play button etc.) before restarting the play at the new location. In another example, a fading transition 136 may be implemented, such that the current sound track reproduction position may gradually fade so that the new sound track reproduction position may gradually be sharpened, such as a crossfade, or alternatively the current sound track may be faded Only new soundtracks can be sharpened.

For example, if the new read location is outside of the buffered audio stream 137, such as when a user skips many pages or chapters of an e-book, then the soundtrack application or multimedia application running on the user device may be a new user It may be configured to send control data 118 to a soundtrack server that represents the read location 138, which may optionally be stored in the read location module 109 of the server. In response, the audio streaming module resets the audio stream by stopping the current data transfer 139 and restarting the data transfer of the precompiled sound track at the new playback position corresponding to the new current reading position. Upon receipt of the new audio stream, the user device is configured to implement the stream transition protocol 140. A variety of transition options are available. In one example, a feed 141 may be implemented in which the sound track reproduction immediately jumps to the new audio stream upon arrival at the user device. In another example, a stop transition 142 may be performed in which playback of the current stream is stopped, the buffer is cleared, and a new audio stream is temporarily stored in the user device waiting for playback start by the user. In another example, a fading transition 143 may be implemented in which the current audio stream gradually fades while the new shade track progressively sharpens, such as a crossfade, or only when the current audio stream is completely blurred, The two streams are matched by bit / tempo / content type so that they can be sharp or the crossfade to the new stream into the old stream is very smooth.

Read Speed Change - Transition Protocol

The read speed of the user may be increased or decreased during reading of the electronic media content. The soundtrack application or multimedia application of the user device may be manually updated by the user either by selecting a new read speed from the range or by inputting a new numeric read speed or through another user input indicating a change in read speed . Alternatively, the multimedia application controls the display of the electronic media content and the sound track playback automatically detects a significant change in the user read speed.

5A and 5C, when a change in read speed is detected 150, control data 118 indicating a new read speed is transmitted from the user device to the sound track server. The soundtrack server is configured to determine whether the currently selected precompiled soundtrack audio file should transition to another precompiled soundtrack audio file that more closely matches the new user read speed. If a new precompiled soundtrack audio file is selected for streaming (151), the transition protocol 152 is executed by the soundtrack server for playback transitions to the new precompiled soundtrack audio file. Various examples of possible transition protocols for switching the synchronized soundtrack between the current user read rate and the new user read rate will be described in more detail. It will be appreciated that other transition protocols may also be implemented.

Feeding

In one example, the transition protocol may be feed as described further with reference to step 152A. In this protocol, the current reading position of the user of the electronic media content is determined in step 154. This may be determined, for example, via control data 118 sent from the user device to the soundtrack server. The soundtrack server then initiates a new audio stream 156 corresponding to the new precompiled soundtrack audio file selected for the updated read position. The playback position of the new audio stream of the new precompiled sound track audio file corresponds to the current reading position of the user as determined at step 154. [ The new audio stream initiated at step 156 has a parallel relationship with the current audio stream for the old read rate. At the user device, upon receipt of a new incoming audio stream, the user device clears the buffer at step 158 and suddenly switches to the new audio stream and begins playback. Finally, the old audio stream is stopped at 160 by the server.

Fade transition

In another example, the soundtrack server may implement a fade transition protocol as described with reference to step 152B. In the fade transition protocol, steps 154, 156, and 160 are the same as those performed for feed 152A. The main difference is step 162 in which the user equipment implements a fade transition to a new input audio stream. For example, upon receipt of a new input audio stream, the user device is configured to make the old audio stream progressively blur while at the same time sharpening the new audio stream. Alternatively, the old audio stream may become completely blurred before the new audio stream becomes clear.

Timely Transition Protocol

In another example, the soundtrack server may implement a timely transition protocol as described with reference to step 152C. In such a timely transition protocol, the soundtrack server is configured to transmit the remaining unstreamed portion of the pre-compiled soundtrack audio file of the current stream to the appropriate transition location (i.e., the portion of the audio file that has not yet been transmitted to the user device from the soundtrack server) And the like). In one example, the soundtrack server may perform signal processing analysis of the unstreamed portion of the audio file to determine the lull state or gap or silence time. Alternatively, the server may analyze the corresponding soundtrack data file containing the timing data for the audio region of the precompiled audio file, and may analyze or calculate the next appropriate transition position based on the predetermined transition parameters or criteria . For example, the transition criterion may be such that only the appropriate transition position has no audio region corresponding to the effect and / or the atmosphere, or alternatively a sound track in which there is no audio region playing in both the matching bit, tempo, It can be dictated that it is a location. Various transition criteria may be configured according to the nature of the soundtrack, which includes no transition or default transition in the event that no criterion can be easily or quickly satisfied in a particular time frame. The transition analysis is performed at step 164. Once the transition location is then determined, the soundtrack server is configured (166) to switch the audio stream data transfer to a new precompiled audio file instead of the current audio file when the audio data associated with the transition location is reached (166). At the user device, very smooth transitions from old audio files at older reading speeds to new audio files at updated reading speeds can be achieved with audio technologies such as fade in, fade out, volume and equalizer matching, and other audio processing techniques ≪ / RTI >

Streaming live compiled soundtracks

survey

In FIG. 6, a second form of the sound track delivery system 200 is configured to stream real-time compiled sound tracks from the sound track server 204 to the user device 102. In such a system configuration 200, the soundtrack server may include a library of soundtrack data files 116 and an integrated or accessible soundtrack database (not shown) containing audio track files associated with the audio regions defined for both of these soundtrack data files 206). In this system configuration 200, a sound track data file for the selected sound track is processed in the sound track server 204 and the compiled sound track audio file is progressively compiled to be streamed to the user on demand or in real time There is provided a real-time sound track compiler 220 configured to generate a real-time sound track. These sound tracks are compiled according to a configurable user read speed setting corresponding to the user ' s read speed.

As will be described, the operation of the second type sound track delivery system 200 is substantially similar to that of the first type sound track delivery system 100 and the same reference numerals denote the same components and functions. The main difference to the second type system 200 is that, as opposed to a sound track audio file in which the entirety is precompiled, as in the first type system 100, the compiled sound track is the selected sound track data file, Is compiled or generated in real time for the configured user read speed from the audio track file associated with the defined audio area in the data file. In addition, the system operations associated with the transition protocol for selecting a sound track, audio streaming, resynchronizing to a new read position, and changing the read rate are substantially similar, as will be further discussed.

Real-time compilation and soundtrack streaming

Once the soundtrack is selected, the user device sends control data 218 to the soundtrack server 204 indicating the user's reading speed and the desired start reading position in the electronic media content. Alternatively, the server 204 may retrieve the user ' s read rate setting from the historically stored data. Then, the real time sound track compiler 220 starts compiling or producing the audio file representing the sound track starting at the desired read position and starting with the audio region playback timing corresponding to the user's configured read speed setting. The sound track server is configured to gradually start streaming of such audio files as they are being compiled into the audio-visual device via the audio streaming module 224. In particular, the compiled portion of the soundtrack can be streamed at the same time while the sound compiler is compiling the remainder of these audio files in real time with custom user-read speed settings.

In one embodiment, the soundtrack compiler may produce a single audio file transmitted in a single audio stream 326 to the user device 102. In an alternative embodiment, such a soundtrack compiler may produce two or more separate audio files representing such a soundtrack. For example, such a soundtrack compiler may progressively produce a music audio file containing all music audio regions, a sound effect audio file containing all sound effect audio regions, and an ambient audio file containing all ambient audio regions . All of these audio files will have the same duration and can be played simultaneously on the user device to reproduce all the sound tracks or the selected audio files will be played simultaneously so that the sound of the other area is reduced and only the specific type of audio area You can create a soundtrack.

The degree to which the sound track compiler 320 compiles or produces the sound track in advance beyond that sent to the user device may be configured or changed based on the variable compilation setting. For example, for faster readers, the soundtrack compiler can compile more audio files in advance for the stream compared to slower readers. Alternatively, or in addition, the control data 218 from the user device may control the amount of the pre-compiled sound track of the current portion being transmitted to the user device. For example, the control data 218 from the user device indicating that the user's buffer is full causes the soundtrack compiler to wait until the additional additional data in the audio file You can stop compiling the part.

Sound track playback and transition

Upon receipt of an input audio stream of a real-time compiled audio file, the user device may buffer the audio data in memory and may start and stop playback as previously described with reference to the first type system 100.

For the transition to the new read position, the same sequence of operations as described with reference to Figure 5B is applied to the first form system 100, with substantially few adjustments. In particular, if the new read location is outside the buffer range, the server may immediately reset the audio stream to the new playback location if it is in the compiled but not yet streamed portion of the audio file, or alternatively, Is located beyond the compilation, the soundtrack compiler is configured to restart the compilation of the soundtrack at the new location and then reset the audio stream to the new compiled portion of the soundtrack's audio file . Stream transition protocol 140 may then be implemented as described with reference to FIG. 5B.

In the change of the reading speed, the same sequence of operations as described for Fig. 5C are applied to the first type system 100 substantially except for a small number of adjustments. When a new read rate is detected 150 the sound track server does not select 151 a new precompiled soundtrack to be streamed as in the first type system 100, The soundtrack server is configured to start the soundtrack compiler 220 to begin compiling a new audio file of configured soundtracks at the new read rate. Then, a transition protocol 152 for a new audio file of the sound track at this new read location can be implemented as described with reference to FIG. 5C.

5. Streaming uncompiled soundtracks

survey

In FIG. 7, a third form of the sound track delivery system 300 is configured to stream uncompiled sound tracks from the sound track server 304 to the user device 102. In this system configuration 300, the soundtrack server includes a library of soundtrack data files and an integrated or accessible soundtrack database 306 containing audio track files associated with the audio regions defined for both of these soundtrack data files. . In operation, the soundtrack server is configured to stream a soundtrack data file and an audio track file over one or more audio data streams to a user device 102 (the same member numbers denote the same components and functions previously described) have. The sound track application or multimedia application of such a user device may process the sound track data file and, based on the user read position variable indicating the current reading position of the user in the electronic media content, A sound track player 330 is provided that is configured to control the player.

Streaming soundtracks

The user may select a sound track to stream or these sound tracks may be displayed on the user device in a configuration in the form of an electronic media server in which the sound track server is coupled with, for example, a synchronized sound track to provide electronic media content. Can be automatically selected by the sound track server. Interaction with this sound track server may occur via control data 318. [

Once a soundtrack for streaming is selected, the entire soundtrack data file is first transferred to the user device as an audio data stream 326. Then, the sound track data file 332 is processed by the sound track player. In particular, the sound track player includes an audio track file requestor module 331 that controls the gradual download of audio track files associated with a sound track from a sound track server. This audio track file requestor module 331 sends control data 318 to the audio track file dispatcher module 320 of the server requesting the requested audio track file. In response to this request, the dispatcher module 320 retrieves the requested audio track file from the soundtrack database and then sends the file to the user device as an audio data stream. The sound track player stores these requested audio track files in memory to be ready for playback.

The sound track player includes a read position module 333 that maintains a user read position variable (such as a counter as previously described) that represents the user's predicted or estimated read position in the electronic media content. The user read position variable may be increased or varied at a rate based on the configured user read speed setting and may be updated randomly or periodically based on user interaction / input (e.g., page turning, scrolling, . This read location module is also configured to receive a control input signal indicating a user read start or a read stop, thereby controlling the read position variable (e.g., to start or stop the counter).

In a first form, this requestor module 331 erases the progressive search of the downloaded soundtrack data file and the audio track file based on the user readout positional variable. In one example, such a requestor module processes the sound track data file to find each next audio region to be played back based on the user read position variable, and sends the audio data stream to the user device, Lt; RTI ID = 0.0 > 320 < / RTI > This may be done one by one for each audio region or alternatively by placement or chunks of audio track files that may be requested in advance for the next batch or chunked audio region to be played back based on the user read position variable. In another example, such a requestor module may process a soundtrack data file and maintain a sliding download window associated with a user readout position variable. In particular, the sliding download window may be an extended configurable size from the user read location variable. This requestor module 331 may be configured to ensure that it has requested all audio track files corresponding to any audio area starting in the sliding download window. The larger these windows, the more audio track files are downloaded prior to their playback and this reduces the likelihood of blocking soundtrack playback due to network delays.

In a second form, the requestor module 331 can be configured to download all the audio track files associated with the soundtrack from the server, one by one in the order of reproduction of the associated audio region.

In a third form, the sound track may be divided into a plurality of predetermined sections or parts, and such information may be embedded in the sound track data file. For example, in a sound track for an e-book, the sound track may be divided into corresponding sections of each elbow of the e-book. In this form, the requestor module 331 is configured to download the entire batch of audio files for each section prior to playing this section. Optionally, such a requestor module 331 may be configured to download a batch of audio files for one or more subsequent sections in advance during simultaneous playback of the first downloaded section of the audio file.

Play soundtrack

The sound track player 330 is configured to process the sound track data file and to signal to reproduce an audio track file associated with the defined audio region of the sound track based on the user read position variable maintained by the read position module 333. [ And a reproduction module 334. In particular, such a playback module starts playback of an audio track file associated with the audio area when the read position variable matches the prescribed start position for that audio area in the manner previously described, Stop playback of the audio track file when the read position variable matches the stop position. As described in WO2013 / 015694, two or more audio track files may be played simultaneously when there are two or more overlapping audio regions, and each audio region may be reproduced at a read speed such as volume and panning envelope control and other audio effects and characteristics And may have other associated characteristics for regeneration that may or may not be dependent.

Purging and retention policies

In such an embodiment, the requestor module 331 may delete such audio track files stored in memory after the audio track file is jammed or after a configurable predetermined time period, if the user again flips to play a portion of the soundtrack . In addition, the requestor module 331 is configured to process the sound track data file to find any audio region that defines the same audio track file. This repeated audio track file is displayed to be stored in the memory of the user device until the last audio area is reproduced.

Sourcing audio track files from different sources

In this example, the requestor module 331 is configured to request all audio track files from the soundtrack server 304. In an alternative embodiment, the sound track data file from the sound track server 304 may be configured to define one or more different sources or servers for at least some or all of the audio track files. In this embodiment, the requestor module 331 is coupled to any other one or more server or file source or file services, whether remote from a user device or a music library or a database stored locally on the user device, To obtain all audio track files from one or more servers or file sources or file services that may be downloaded, streamed, or otherwise obtained from, or otherwise separated from, the sound track server 304 providing the sound track data files . In one example, the user device can obtain a large audio track file for the audio region associated with the effect and mood from the soundtrack server, in accordance with the soundtrack data file, and stored in a different music server, An audio track file for the music audio area can be obtained from the database. In another example, a user device may be capable of playing audio tracks of a sound track from one or more sources separated from a database accessible or locally stored by a library or user device, such as a sound track server 304, such as a file server, You can get all the track files. The alternative source to the sound track server 304 may be operated by the user or by a different entity to the operator of the sound track server 304. [

6. General

Embodiments of the present invention may be implemented by hardware, software, firmware, middleware, microcode, or any combination thereof. When implemented in software, firmware, middleware or microcode, program code or code segments that perform the necessary tasks may be stored in a machine-readable medium, such as a storage medium, or in another storage. A process can perform the tasks that it needs. A code segment may represent a procedure, function, subprogram, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or hardware circuit by transmitting and / or receiving information, data, arguments, parameters, or memory content. Information, arguments, parameters, data, etc., may be passed or transmitted through any suitable means including memory sharing, memory passing, token passing, network transmission,

In the above, the storage medium may be any type of storage medium, including read only memory (ROM), random access memory (RAM), magnetic disk storage medium, optical storage medium, flash memory device and / or other machine readable medium for storing information Lt; RTI ID = 0.0 > and / or < / RTI > The terms "machine-readable medium" and "computer-readable medium" include, but are not limited to, portable or fixed storage devices, optical storage devices, and / or various other media capable of storing, It is not.

Various logical blocks, modules, circuits, elements and / or components described in connection with the examples disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC) Programmable gate array (FPGA) or other programmable logic component, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general purpose processor is a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, circuit, and / or state machine. A processor may also be implemented as a combination of computing components, such as, for example, a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such combination of configurations.

The methods or algorithms described in connection with the examples disclosed herein may be implemented in the form of processing units, programming instructions or other instructions, either directly in hardware, in a software module executable by a processor, or in a combination of the two, Or may be distributed to multiple devices. A software module may reside in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. The storage medium may be coupled to the processor such that the processor can read information and write information to the storage medium. Alternatively, the storage medium may be integrated into the processor.

One or more of the components and functions shown in the figures may be rearranged and / or combined into a single component without departing from the present invention, or may be implemented with multiple components. Additional elements or components may be added without departing from the invention. Also, the features described herein may be implemented in software, hardware, or a combination thereof.

In various aspects, the invention may take the form of a computer-implemented process, a machine (such as an electronic device, or a general purpose computer or other device that provides a platform upon which a computer program may be run), a process executed by such machine, Can be implemented. Such articles of manufacture may include computer program products or digital information products, including computer program instructions or computer-readable data on which computer-readable storage media are stored, and processes and machines for creating and using such articles of manufacture.

The foregoing description of the invention includes preferred forms of the invention. Modifications may be possible without departing from the scope of the invention as defined in the appended claims.

Claims (41)

Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, A method of streaming a compiled sound track defined by an audio region of an electronic media content to a reader remote user device of an electronic media content,
Receiving control data from the remote user device over the network, the control data indicating the read speed of the user; And
Streaming the compiled sound track to the user device over the network based on the received control data. The method of claim 1, wherein the compiled soundtrack is a precompiled soundtrack and is selected from a range of precompiled soundtracks that are precompiled at different read rates . 3. The method of claim 2, further comprising selecting a precompiled soundtrack for streaming having an associated readout rate that most closely matches control data representing the user ' s read rate, A method of streaming a track. 2. The method of claim 1 wherein the compiled sound track is a real time compiled sound track and the method further comprises compiling a sound track in real time based on control data representing the read speed of the user How to stream compiled soundtracks to. 5. The method of claim 4, wherein the sound track is incrementally compiled and the method comprises streaming the compiled portion of the sound track while the remainder is being compiled. Streaming method. 6. The method according to any one of claims 1 to 5, further comprising: receiving from the remote user device control data indicative of the user's desired start reading position; Further comprising: starting streaming the compiled sound track to the user device at a data location within the sound track. ≪ Desc / Clms Page number 21 > 7. The method of claim 6, wherein the data location is at the beginning of the compiled sound track or at an offset location within the compiled sound track. 6. The method according to any one of claims 1 to 5, further comprising the steps of: receiving control data from the remote user device indicating a change in a user's read position to a new read location in the electronic media content; ≪ / RTI > further comprising modifying the stream of the compiled soundtrack based on the stream of the compiled soundtrack. 9. The method of claim 8, wherein modifying the streaming of the compiled soundtrack comprises: adjusting streaming of the compiled soundtrack to a new data location in the compiled soundtrack corresponding to the new readout location The method comprising the steps of: 6. The method according to any one of claims 1 to 5, further comprising the steps of: receiving from the remote user device control data indicative of a change in a user's read speed to a new user read speed; Further comprising modifying the streaming of the compiled soundtrack. ≪ Desc / Clms Page number 22 > 11. The method of claim 10, wherein the compiled soundtrack is a precompiled soundtrack and modifying the streaming of the compiled soundtrack based on the new user readout speed is closer to the new user readout speed Selecting a new precompiled soundtrack having a matching reading rate and starting streaming the new precompiled soundtrack. ≪ Desc / Clms Page number 19 > 12. The method of claim 11, further comprising initiating a transition protocol for transitioning the stream from the currently precompiled sound track to the newly selected precompiled sound track . 13. The method of claim 12, wherein the transition protocol comprises the steps of: receiving control data from the remote user device indicating a current playing position of a user in a streamed sound track; And starting streaming of the compiled soundtrack. ≪ Desc / Clms Page number 20 > 13. The method of claim 12, wherein the transition protocol comprises the steps of: analyzing the remaining unstreamed portions of the precompiled sound track and identifying a transition position based on the transition criteria; And starting streaming of the newly selected precompiled sound track at a location of the newly selected precompiled sound track. 11. The method of claim 10 wherein the compiled sound track is a real time compiled sound track and modifying the streaming of the compiled sound track based on the new user read position comprises: Starting the compilation of the track and starting streaming of the newly compiled sound track. ≪ Desc / Clms Page number 12 > 16. The method of claim 15, further comprising initiating a transition protocol for transitioning streaming from the current real-time compiled sound track to the new real-time compiled sound track . 17. The method of claim 16, wherein the transition protocol comprises: receiving control data from the remote user device indicating a current playing position of the user in the streamed sound track; And starting streaming of the real-time compiled soundtrack. 17. The method of claim 16, wherein the transition protocol comprises: analyzing the remaining unreamed portions of the current real-time compiled sound track and identifying a transition position based on the transition criteria, And starting streaming of the new real-time compiled soundtrack at a location of the compiled soundtrack. 6. A method according to any one of claims 1 to 5, comprising streaming the compiled sound track as a single digital audio file. 6. A method according to any one of claims 1 to 5, comprising streaming the compiled sound track as a plurality of digital audio files for simultaneous playback at the remote user location How to stream soundtracks. 21. The method of claim 20 wherein the audio tracks of the sound track are categorized into different types and the compiled sound tracks are provided as separate audio files each comprising an audio region associated with one type How to stream soundtracks. Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, A sound track server for streaming a compiled sound track defined by the audio region of the electronic media content to a reader remote user device of the electronic media content,
Receiving from the remote user device via the network control data indicative of the read speed of the user;
And to stream the compiled sound track to the user device over the network based on the received control data. Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, A method of reproducing a compiled sound track defined by an audio region of an electronic media content on a reader-user device of an electronic media content,
Transmitting control data indicating the read speed of the user to a remote user server via a network;
Receiving an input stream of a compiled soundtrack having an associated read rate corresponding to the transmitted control data from the remote server over the network; And
Processing the streamed compiled sound track and starting playback of the streamed compiled sound track at the user device. Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, And reproducing the uncompiled sound track defined by the audio region of the electronic media contents on the reader user device of the electronic media contents,
Transmitting control data requesting a sound track for reproduction to a remote server via a network;
Receiving a sound track data file defining the audio region of the sound track from the remote server in response to the transmitted control data;
Processing the sound track data file to determine at least a next audio region to be played back based on a user read position variable indicating the read position of the user in the electronic media content;
Transmitting periodic or arbitrary control data requesting an audio track file corresponding to at least the next audio region to be reproduced to one or more audio track file sources;
Receiving a requested audio track file from the one or more audio track file sources and storing the requested audio track file in a memory; And
Adjusting the reproduction of the stored audio track file as defined by the associated audio region in the sound track data file based on the user read position variable. 25. The method of claim 24, wherein the sound track data file includes source data representing one or more audio track file sources requesting each audio track file. 26. The method of claim 25, wherein the sound track data file defines a single audio track file source requesting all audio track files. 26. The method of claim 25, wherein the sound track data file defines two or more different audio track file sources requesting an audio track file. 27. The method of claim 26, wherein the audio track file source is a remote server and the method comprises requesting all audio track files from the remote server. 28. The method of claim 27, wherein the audio track file source comprises the remote server and one or more other audio track file sources, the method further comprising: And requesting the audio track file from a track file source. 30. The method of claim 29, wherein some audio track files are obtained from the remote server and the other audio track files are obtained from one or more other audio track file sources. 27. The method of claim 26, wherein the one or more audio track file sources are separate from the remote server, the method further comprising: receiving all audio tracks from the separate one or more audio track file sources ≪ / RTI > wherein the requesting step comprises requesting the file to play the uncompiled sound track. 29. The method of claim 27, wherein the one or more audio track file sources are separate from the remote server, the method further comprising: receiving all audio tracks from the separate one or more audio track file sources ≪ / RTI > wherein the requesting step comprises requesting the file to play the uncompiled sound track. 33. A method as claimed in any one of claims 24 to 32, comprising transmitting control data requesting individual audio track files one by one to the one or more audio track file sources Way. 33. A method according to any one of claims 24 to 32, comprising transmitting control data to the one or more audio track file sources, the control data including a single request for all audio track files associated with the audio track of the soundtrack Lt; RTI ID = 0.0 > soundtrack. ≪ / RTI > 33. A method according to any one of claims 24 to 32, comprising transmitting control data requesting placement of an audio track file to the one or more audio track file sources Way. 33. A method according to any one of claims 24 to 32, further comprising the steps of: maintaining a sliding download window with respect to the user read position variable; and maintaining all audio track files associated with the audio region in the sliding window Transmitting to the one or more audio track file sources control data requesting all of the audio track files associated with the audio region in the sliding download window to be received and stored for future playback. ≪ RTI ID = 0.0 & How to play uncompiled soundtracks. 37. The method of claim 36, wherein the size of the sliding download window is configurable. 33. The method of any one of claims 24 to 32, further comprising deleting the audio track file immediately after the memory track has finished playing, or after a configurable predetermined delay from the memory Lt; RTI ID = 0.0 > soundtrack. ≪ / RTI > 33. The method of any one of claims 24 to 32, further comprising: analyzing a sound track data file for an audio track file that is repeated in more than one audio region of the sound track; Tagging the audio track as an audio track, and retaining each of the repeated audio tracks in memory until the last audio region associated with the audio track completes reproduction. ≪ RTI ID = 0.0 > Track playback method. Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, A user device for reproducing an uncompiled sound track defined by an audio region of an electronic media content for a reader of electronic media content,
A user interface configured to receive user input to control the user device;
An audio output system operable to generate an audio output for the user to hear; And
The processor comprising:
Transmitting control data requesting a sound track for playback to a remote server via a network;
Receiving a sound track data file defining the audio region of the sound track from the remote server in response to the transmitted control data;
Processing the sound track data file to find out at least a next audio region to be played back based on a user read position variable indicating the read position of the user in the electronic media content;
Transmitting periodic or arbitrary control data requesting an audio track file corresponding to at least the next audio region to be reproduced to one or more audio track file sources;
Receiving a requested audio track file from the one or more audio track file sources and storing the requested audio track file in a memory of the user device;
And adjust the reproduction of the stored audio track file as defined by the associated audio region in the sound track data file based on the user read position variable. Each of which is defined by an audio track for reproduction in the audio area, a start position in the electronic media content corresponding to where the reproduction of the audio area is to be started, and a stop position in the corresponding electronic media content, CLAIMS What is claimed is: 1. A method for streaming an uncompiled sound track defined by an audio region of an electronic media content to a reader remote user device of an electronic media content,
Transmitting a sound track data file including data defining an audio region of the sound track to the remote user device over a network;
Receiving control data from the remote user device over the network requesting one or more audio track files corresponding to an audio region of the sound track; And
And transmitting one or more audio track files to the remote user device over the network based on the received control data.

Images