WO2015120143A1 - User generated audio elements which provides for interaction with radio programming systems - Google Patents

Abstract

A soundtracking system comprised of hardware and software, which permits a user to up-load an audio selection recorded using an application on a user input device. The selection and its associated information is processed by the soundtracking system to form an audio form where the processing can include any number of actions that help in the indexing, storage, retrieval, curating, manipulation, cataloging, distribution, re-distribution, publication, and play of the audio selection, as well as a set of assembled audio selections. In addition, the soundtracking system of the present invention can be used to collect, catalog, store, sort, group and index a set of sounds. Thus, a posting user has the ability to provide a "scrapbook" of sounds, such as a set of sounds from the user's life experience that are meaningful to them or a permissioned user has the ability to make a playlist or sound track available for publication or play.

Description

USER GENERATED AUDIO ELEMENTS WHICH PROVIDES FOR

INTERACTION WITH RADIO PROGRAMMING SYSTEMS

FIELD OF THE INVENTION

[0001] The present invention provides a social media service involving a soundtracking or sound scrapbooking system (i.e. a system which pastes together in an organized fashion a series of user provided sound elements, with the ability to modify the elements, and for storage, broadcast or re-play for a specific purpose and at a later time). This system has hardware and software for receiving sound recordings or audio elements generated by users who record voice or other audio on their mobile device or computer. The soundtracking system permits processing of one or more audioforms that are created by the system from the sound recording and interpretational objects that are determined from the audio elements and associated information. The processing includes functions such as indexing, sorting and storage functions. Then the audio form or forms can be transferred, for example, through a web services platform from the posting user's device to a cloud-based platform for consumption and sharing by other users. The audio may also be transferred to a web- based password -protected component of the system for curation by a user who has permission from the system to access the audio. Soundtracking or sound scrapbooking describes the process by which the posting user submits their personal audio or the system receives the audio and the system processes the submissions for indexing, storage and publication or consumption such as by social media, a website, or broadcast platforms and the inclusion of the ability to sort and group sound submissions according to a specific end-use.

[0002] The present invention further involves a hardware and software audio scrap booking or sound tracking system for processing user generated sound in which the system receives sound bites as audio elements from a user via an input device that accesses the internet or other wire driven communication means such as a computer or land or cell telephone. In this instance, sound tracking refers to "recording" in both the sense of capturing sound for replay, as well as the grouping, of sound or sounds, and librarying or cataloging the sound or group of sounds for one or more particular purpose including for example, promotion, publication or distribution, archiving, matchmaking, and music-making. The processing done by the present invention includes receiving the sound as an audio element along with associated information, transcoding the audio element to the proper audio format, transcribing any speech or verbal content to determine additional associated information conveyed in the audio element, processing the associated information along with any additional associated objects (such as internally generated associated objects including for example formatting or archiving information) to form a set of interpretational objects, and associating the interpretational objects with the audio element to form a audio form. The audio form is further subjected to additional processing; including indexing, digesting, filtering, censoring, cataloging, storing, curating, combination, distribution, redistribution or and publishing or play, which processing can be automatically generated by the audio soundtracking or scrapbooking system in accordance with pre-defined rules or which can be individually tailored to the needs or desires of a permissioned user (generally a person like a disc jockey or other human entity, such as a business).

BACKGROUND OF THE INVENTION

[0003] Broadcast radio has long recognized the value of adding user generated content to its broadcast format, such as in the form of pre-recorded and live audience participation and listener call-in. Adding this content to broadcast adds excitement and an air of spontaneity. It can be used as well for promotional purposes, in which case a broadcaster might aggregate a series of recordings or call to give depth to the listening experience. Likewise, in certain reporting situations, especially including sports broadcast, the use of user-generated content shows audience support and buy-in. The advent of social media allows textual and video content to be shared immediately; however, it is relatively novel to think of the provision of sound tracking, cataloging and distribution for user-generated sound content, in particular as sound bites. There are various applications for which this system can be used depending on the ways in which the generated audio forms are sorted, grouped and distributed. SUMMARY OF THE INVENTION

[0004] The present invention provides a soundtracking system, which permits an end-user (i.e. in this case, a sound generating or posting user) to up-load an audio selection recorded on a phone, smart-pad or computer using an application. The selection is processed by the soundtracking system to allow further processing of the audio selection, and its associated information where the processing can include any number of actions that help in the indexing, storage, retrieval, curating, manipulation, cataloging, distribution, re-distribution, publication, and play of the audio selection, as well as a set of assembled audio selections.

[0005] In addition, the soundtracking system of the present invention can be used to collect, catalog, store, sort, group and index a set of sounds. Thus, a posting user has the ability to provide a "scrapbook" of sounds, such as a set of sounds from the user's life experience that are meaningful to them. The invention would allow a parent to record, categorize and store sound recordings that benchmark their children's lives, or for any one to capture ambient sounds and use them to capture and re-create the audio sense of a life experience, such as the sound of a favorite beachside retreat, a late summer evening, or a busy city street.

[0006] The present invention has further application depending on the system by which the sounds are organized and distributed for personal interaction, such as employee search or match making purposes. In this embodiment of the invention, the soundtracking system processes the audio forms according to a set of rules so as to sort and group the audio forms for a particular distribution and consumption. For example, the system may use a defined set of interpretational objects associated with an audio form to sort and group a subset of audio forms, and to determine the distribution of the selected audio forms to selected audience members (who might be posting users, permissioned users and audience members for whom a set of audio forms is assembled into a playlist and subsequently played).

[0007] Additionally, the system can gather "soft", or passive information or data, including, for example, information about where a posting user posts from, the time of posts, and how often the posting user posts. The present invention also includes a user interface in which a curator, such as a DJ, can review audio submissions and associated information, i.e. one or more of the audio forms. The system may also provide logs relating posting user interaction and related data from social media sources including for example, Facebook, Twitter, InstaGram, Linkedln, 4-Square, and Pinterest to name a few.

[0008] The soundtracking system of the present invention can provide events during which posting users are alerted and invited to up-load audio content. For example, members of the posting user group can be prompted to participate in a posting event by alerts, delivered by SMS, IM, or e-mail or by social media including for example but not limited to FaceBook® and Twitter® or by mobile device "apps" or applications directed to programming content or other application based directives. The audience can use various devices to input their audio selection (assuming that the device can capture and transmit the selective using a phone or internet connection), such as a computer, work station, PDA, cellular device, cable box, mobile pad, game station, or vehicle infotainment system. In a further embodiment of the invention, members of the audience can upload audio files, which may then be moderated or screened by the soundtracking system and/or staff which use it. Then, those audio forms may be made available for broadcast to a general broadcast audience, such as a broadcast radio audience, or to a audience group, including the posting user, or an authorized representative or a otherwise permissioned user.

[0009] The present invention also provides a software system in which an end- user's participation can automatically (i.e., without the need for a person to participate) be provided for access, publication or distribution.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is schematic diagram illustrating the present invention;

[0011] Figure 2 is a representation of the screen which is used to prompt the posting user to participate by announcing a soundtracking event;

[0012] Figure 3 is a representation of a web-based control screen; and

[0013] Figure 4 is diagram of the software link of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] Figure 1 shows a schematic view of the audience interaction system in accordance with the present invention. The system shown 10 generally represents a hardware and software system which can enable user captured contents to be captured and stored for further manipulation by the user or another entity. Storage happens on a server, which can be a dedicated server, and which can be a series of distributed servers or can be cloud based storage. The hardware and software of the present invention allows manipulation including transcription, categorization, indexing, grouping, organization, listing, selection and storage. The system further allows the resultant play list to be distributed, such as by social media or to a user or set of selected users such as a friend group selected through social media, including for example Facebook, Twitter, Linkedln, Pinterest, Instagram or others.

[0015] An additional aspect of the present invention is that it allows audience members to up-load selections as an MP3, AC3, Wav, or OGG format file. The selection goes to the software link for further manipulation.

[0016] Figure 2 is a representation of the screen which is used to prompt the posting user to participate by announcing a soundtracking event.

[0017] Figure 3 illustrates the Web-Based Control Screen that helps to initiate the generation of user based audio content.

[0018] Figure 4 shows a schematic view of the software system in accordance with the present invention in the context of the interaction between the external data source and specifically, the posting user interface, and the soundtracking or sound scrapbooking system. While the invention is described with respect to a separate and external sound tracking system (i.e. which is implemented and managed apart from the broadcast station), it should be understood that the data source can be internal to the broadcast station and operate as part of the automation system and where the link of the present invention is used as a mechanism to import files external to the initial playlist, file system, and/or automation system.

[0019] The software system, shown generally at 10, further interacts such as by way of the internet 1 12, with an external data source, which is specifically a posting user system shown generally at 1 14. [0020] The software linking system 1 10 of the present invention includes major components, each responsible for a separate process, that can be combined together to create the functionality required to enable the broadcast station automation system to play a selection chosen through a voting session or an open selection event.

[0021] The first component 200 is an input device which may include consumer electronics such as a computer table or smart phone including hardware that allows audio recording.

[0022] As a second component 300, the linking system receives content via a web- based API. The information is formatted, for example in XML, and transferred via a TCP/IP connection as a stream of bytes. The format for XML coming from the user input device is a generally known to one skilled in the art. The software linking system re-formats the received audio content system via HTTP POST or SMS via a web- based API. The data is sent along with authorization credentials (username and password) for security. Some of the HTTP POST parameters used by this API are: The system is not limited to these HTTP Post parameters and may use other parameters to exchange information between the automation system, website, audience user interface, and web based system.

[0023] The third component 400 comprises the software link which connects the user interface with a manipulation system.

[0024] The fourth component 500 is the hardware and software system by which a authorized user can manipulate the audio received content and enable the creation of a playlist comprising a linked series of audio selections.

[0025] The posting user may record an 1 1.11 second audio element through any capable device with a web browser, native app (Android, iPhone, iPad), or call-in phone recording system. The audio element is transferred from the device to the soundtracking system server via HTTP post or MMS (multimedia text message). The posting user has an option to create a personal login on the system, which the system will associate with audio elements and other objects submitted by the posting user. The soundtracking system automatically transcodes the audio element into mp3, m4a, and wav file formats on the server. The soundtracking system automatically time stamps audio elements as they are received. The soundtracking system automatically transcribes the audio element into text and associates the audio element with the text transcription as an "interpretational object." The soundtracking system automatically creates an "interpretational object" with sentiment analysis of each audio element. The soundtracking system associates each audio element with a series of objects, known as "associated objects" including but not limited to a user action, the method by which the audio element was submitted to the soundtracking system, posting user interaction with social media, songs, song ratings, photos, the location of the posting or permissioned user, and other posting user qualifications for example according to information supplied on a user form, and additional interpretational objects (i.e. based transcription and sentiment analysis) and video or images. This creates an audio form from the audio element and associated objects which can be further processed by the soundtracking system for the purpose of sorting, indexing, grouping, distributing, or publication. Audio elements or audio forms may be modified, i.e. manipulated or "curated" by the posting user or another permissioned user on the soundtracking system, causing the audio to loop, overlay on other audio, or concatenate multiple audio elements into one file or stream. This curating user may add sound effects to audio elements or alter the audio with a number of effects including changing the pitch, phase, tempo, frequency response/equalization.

[0026] The posting or permissioned user may download and share an audio form or the audio element of an audio form or altered audio element along with other associated objects via Twitter®, Facebook®, Instagram®, Sound Cloud®, and other social networks. This user may cause the soundtracking system to create a playlist of audio elements to be played in a particular order. This user may create a playlist of audio based on relevant associated or interpretational objects or similarities in audio transcription or social graph (connection to other users). The soundtracking system may create a live stream playing back a playlist of audio related by a common object. The user may cause the system to suppress or censor certain audio elements from playing. The system may deliver an audio element to the user simultaneous with associated objects, such as photos, videos, ratings for the audio element and events.

[0027] For the purpose of this description, an audio form is the total experience of an audio element, its associated objects which also comprises images, videos, user ratings and interpretation objects such as keywords, meta data, transcription, sentiment analysis, images, user data, location and pre-selected definitional schema, including personal data supplied by a user, like skills and qualifications or preferences, depending on the end use criteria. A posting user may opt into receiving alerts with respect to an audioform's distribution or when the audio form is changed by the system via social media including Facebook and Twitter, e-mail message, text message, and multimedia text message. These alerts can be attached to specific users, events, or other objects in the system. Alerts are delivered to users when, for example, a new audio message is uploaded attached to the object or user for which the original user wished to be alerted, or when their personal audio is published on e.g. a broadcast medium such as radio, i.e. posting user submits audio and 100 people post likes, posting user gets alert that 100 people like the audio. The system may deliver alerts to users related to the level of distribution or consumption of certain audio elements. The user may search for audio based on what the person said or based on other users mentioned, as identified in the transcription or interpretation objects for each audio element. The user may cause the system to submit audio elements to a broadcast radio station for on-air broadcast. A permissioned user may download a zip/batch of audioforms based on associated objects or audio content. Any user may customize the dashboard that displays access to audio elements. Audio elements and their associated objects may be created for private or direct one-to-one distribution or to a limited group of privileged users such as friends. The user may create keywords, ratings, and other meta data that is stored in discrete interpretation objects and associated with one or several audio elements.

[0028] In one application, the present invention is used for replay on a radio station. In this case, the system API is accessible through the following URL scheme (items in <> are placeholders, items in [] are optional): http://api drhub.com/?kev=<station kev>&method=<method name>f&<param1 >=<val 1 >[&<param2>=<val2>...]]. The station_key: A unique key assigned to each station, usually the call letters of the station. Method_name: The API method being called. Parameters vary by the method called. It returns a JSON object. When an error occurs, there will be an 'error' key in the object, usually an object itself with a 'message' key detailing the error. The invention in particular expects the following: http://apUdrhub.com/?key=<station key>&metbod^~save open mic[&service=<emailjs ms|twitter>&contact=<email addresslsms numberjtwitter user>&name=<full name>l The contact details (which allow the system to alert the user that their audio selection is about to be on the air, or otherwise) can be omitted if the user chooses not to submit them. It also expects an audio file POSTed under the key "audio" with an extension of either m4a or mp3 as appropriate. Upon successful submission, the API will return an object with the 'success' key set to 1 . The file will be saved to our servers and accessible through the software link or "Control Room", where you can make notes about it, play it back, download it (as MP3 or M4A), and message the user (if they submitted contact details).

[0029] The following description represents user stories, which catalogs the experience of an end-user in interacting with the system of the present invention.

[0030] USER STORIES:

1. When the app first opens, it is already listening (similar to Shazam or Sound Hound), so that the user can dictate commands.

I see a menu (bottom of screen?) with "commands" (record, play, search, -and "follow" if logged in?, login).

Any user can listen to audio, but only users who are logged in can post audio.

2. Menu Command: Record

If not logged in... Go to Sign In Screen (5)...

If logged in...

2a. I hear a tone that indicates that the recording has started.

I can record audio for up to 1 1.1 1 seconds.

On the screen, I see equalizer that moves with the audio and a countdown on time.

I see a pause button and a stop button.

Once the audio is stopped, I can preview the audio.

2b. At the end of 1 1.11 seconds, I hear a tone.

Once audio is recorded, I see a submit button... with options to add keywords, mood, photo, tag friends.

Server automatically gets location and transcription.

Once I hit the submit button, the audio is posted to the feed, and I'm sent back to the newsfeed with me/my friends with my post highlighted at the top of the feed.

I see a (?) with possible commands. 3. Menu Command: Play

If not logged in ...

See 3B ... Start playing a playlist of trending audio (based on parameters TBD)

Message displayed somewhere on screen: "Login to post audio, customize your feed, follow friends"

Every 5-6 audio elements, Call to action "stop creeping, start contributing"

(need several Morgan Freeman prompts)

I am logged in ...

See 3B ... playlist based on your friends

3B: What's on the screen during each audio element:

Showcase Showdown Wheel...

List of elements - now playing element in center of screen?

Play button is on left of screen

indicates your position in the playlist of audio elements

transparent progress bar shows pos. of audio (within 11 seconds)

User can scroll up / down to move up and down in playlist.

Inside the audio element window ... mood, keywords, earmark (our version of star/favorite), image (may show up only when playing), rate up / rate down, share on

Twitter, echo (our version of retweet), chain (our version of reply) ~ see #8, loop,

If I click on "echo" ~ the audio is added in the background to my feed.

If the user doesn't rate any audio after 10 elements play, remind the user to rate audio.

4. Menu Command: Search

Search box for keyword (server will also use transcription)

Filter by mood, location

Or see only audio I've starred (earmarked)

(all audio driven, in addition to text / buttons for mood/location)

5. Sign-In Screen

Login ( go to 1 )

Create an Account

5a. Create An Account

Stepl

- e-mail

- Password

Step2

- Choose a username

- Record your username (optional)

- Record your 11 second bio (optional)

- Location (optional)

- Profile Picture (optional)

- Link to Twitter ... auto-post all of your audio toTwitter.

Once you've finished Step2, go to (6) Follow People.

6. Follow

Option 1 - Search for user

- Access Contacts on phone

Option 2 - Invite friends

- Share on FB / Twitter

See who I'm following

See who is following me.

8. Chain (reply)

I see the audio element at the top of the screen (where I clicked chain) I see elements of (2a) and (2b)

9. Settings (go back to 5a-Step2)

- Record your username (optional)

- Record your 11 second bio (optional)

- Location (optional)

- Profile Picture (optional) - Link to Follow Screen

[0031] While in accordance with the patent statutes the best mode and preferred embodiment have been set forth, the scope of the invention is not limited thereto, but rather by the scope of the attached claims.

Claims

WHAT IS CLAIMED IS

1 . A method of creating a soundtrack comprising the steps of

receiving audio content generated by a first and associated information; using a soundtracking system having hardware and software to define an audio element from the audio content;

using the soundtracking system to define a set of one or more

interpretational objects comprising information relating to the user or the audio element from the associated information;

using the soundtracking system to manipulate the audio element by one or more of transcoding into a audio wave format, transcribing verbal content in the audio element, analyzing the audio element for verbal or emotional content, characterizing the audio element, organizing the audio element, indexing the audio element, and storing the audio element;

using the soundtracking system to associate the interpretational object with the audio element to create an audioform;

using the soundtracking system to allow a permissioned user access to and to curate the audio form;

using the soundtracking system to allow the permissioned user to publish the audio form; and

alerting the first user details of the publication of the audioform.

2. A method of creating a soundtrack as set forth in claim 1 wherein the audio content is received by HTTP or SMS.

3. A method of creating a sound track as set forth in claim 1 wherein the step of publishing comprises playback of the audioform on broadcast radio.

4. A method of creating a sound track as set forth in claim 1 wherein the step of publishing comprises playback of the audioform on a private sound system.

5. A method of creating a sound track as set forth in claim 1 comprising the further step of defining a set of interpretational objects such that an audioform can be analyzed for the purpose of matchmaking.

6. A method of creating a sound track as set forth in claim 1 comprising the further step of defining a set of interpretational objects such that an audioform can be used for the purpose of application to a job or program.

7. A method of creating a sound track as set forth in claim 1 comprising the further step of defining a set of interpretational objects such that an audioform can be used for the purpose of promotion.

8. A method of creating a sound track as set forth in claim 1 comprising the further step of defining a set of interpretational objects such that an audioform can be used for the purpose of advertising.

9. A method of creating a sound track as set forth in claim 1 comprising the further step of defining a set of interpretational objects such that an audioform can be used for the purpose of promoting sports.

10. A method of matching based on user generated audio elements comprising the steps of

receiving audio elements and associated information from a posting user using a computer and software to transcode the audio elements into a file in one of the following formats, WAV, MP3 MP4; and

digesting the associated information to form a set of interpretational objects;

collecting the audio elements and interpretational objects to create an audio form; gathering a plurality of audio forms and archiving them on a server and using hardware and software to interpret the associated information to index the audio forms for storage and retrieval;

allowing access to the plurality of audio forms to a matching system so as to select a subset of the plurality of audio forms according to a pre-selected criteria;

broadcasting the audio elements of the subset of audio forms to a selected end user.

1 1 .A method of matching as set forth in claim 10 wherein a user selects one or more of the pre-selected criteria.

12. A method of matching as set forth in claim 1 1 wherein the pre-selected criteria includes educational criteria.

13. A method of matching as set forth in claim 1 1 wherein the pre-selected criteria includes musical preference.

14. A method of archiving user generated sound elements comprising the steps of receiving audio elements and associated information from a posting user posting over the internet with a device and using a computer and software to transcode the audio elements into a file in one of the following formats, WAV, MP3 MP4 on a remote server; and

digesting the associated information to form a set of interpretational objects;

collecting the audio elements and interpretational objects to create an audio form;

gathering a plurality of audio forms and archiving them on the remote server and using hardware and software located on that server or on the user's device to interpret the associated information to index the audio forms for storage and retrieval; broadcasting the audio elements of the audio forms at the instigation of the user in a selected organizational sequence.

15. A method as set forth in claim 14 wherein the selected organizational sequence is chronological according to the time that the audio element was generated.

16. A method as set forth in claim 14 wherein the selected organizational

sequence is depending on a tag that the user associates with the audio element.