WO2013191755A1 - Procédé obtenant l'identification d'une tonalité de rappel et d'autres informations de paires d'appels sur des combinés d'appel entrants et sortants - Google Patents

Procédé obtenant l'identification d'une tonalité de rappel et d'autres informations de paires d'appels sur des combinés d'appel entrants et sortants Download PDF

Info

Publication number
WO2013191755A1
WO2013191755A1 PCT/US2013/030564 US2013030564W WO2013191755A1 WO 2013191755 A1 WO2013191755 A1 WO 2013191755A1 US 2013030564 W US2013030564 W US 2013030564W WO 2013191755 A1 WO2013191755 A1 WO 2013191755A1
Authority
WO
WIPO (PCT)
Prior art keywords
phone
rbt
heard
call
metadata
Prior art date
Application number
PCT/US2013/030564
Other languages
English (en)
Inventor
Martin GINDI
Original Assignee
Tribeca Mobile Innovations Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tribeca Mobile Innovations Inc. filed Critical Tribeca Mobile Innovations Inc.
Priority to US14/564,889 priority Critical patent/US20150304491A1/en
Publication of WO2013191755A1 publication Critical patent/WO2013191755A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/22—Arrangements for supervision, monitoring or testing
    • H04M3/2227—Quality of service monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/42017—Customized ring-back tones

Definitions

  • RBTs ringback tones
  • caller the calling party's
  • the audio clip may be a default tone or the callee may select it.
  • An audio clip that the callee intends to use as an RBT is saved into a callee's profile.
  • the callee may have many audio clips in his profile and the callee may designate that certain RBTs be used for particular callers.
  • Figures 1 a, 1 b and 1 c Architecture arrangement of physical components of the present system, representing alternative arrangements. This also shows the location of certain databases.
  • Figures 2(1 ), 2(2), 2a and 10 Schema and sample data as part of illustrative examples for tables used in the system.
  • Figures 3, 3a and 3b Process flow for three main components of the system including; starting events (figure 3), the 'what I heard' service (figure 3a), and the display and return of what I heard visual Metadata. (Figure 3b).
  • Figure 4 Schematic drawings of various visual displays supported by the system.
  • Figure 5 Schematic representation of the audio match service.
  • Figure 6 Schema for data returned and sometimes stored by various APIs in the system. Also includes a list of example rules used for processing certain return values.
  • Figures 7(1 ) and 7(2) Time flow diagram for two alternate methods for obtaining a phone's phone number if the OS restricts access.
  • Figures 9(1 ), 9(2), 9a(1 ), 9a(2) and 12 Represent embodiments of various display methods for 'what I heard' and 'what I played'.
  • Figure 13 is a high-level block diagram showing an example architecture of a mobile device
  • Figure 14 is a front view of a mobile device suitable for processing. DETAILED DESCRIPTION
  • Phone A calls Phone B.
  • Phone B had, previous to the call provisioned his or her line on their carrier that provides service for phone B to play a 'Ring Back Tones'. Ring-back tones are some audio recording played in the earpiece of phone A until phone B 'picks up' the phone.
  • extra Metadata related to phone A may have been set up on phone line B.
  • the current disclosure provides methods for Phone A to view a human readable message, immediately after the call, or within a number of seconds, regarding the RBT just heard or other relevant Metadata. We will call this display and the software apparatus that performs this 'what I heard' (even in the case where no RBT is in fact played and only relevant Metadata is returned). Similarly, Phone B will display 'what I played' information on Phone B's handset.
  • Figure 3 shows a flowchart of the 'what I heard' or 'what I played' method. The detailed description begins with Figure 3 to give an overview of the functional aspects of the disclosure.
  • the method begins at step 201 A where phone A calls phone B.
  • the system records the audio on an outgoing call just in case we may need it for later audio processing.
  • the method determines if phone B has connected the call. This step can occur if the user of phone B has answered the call or if phone B's voicemail activates. Upon the connection of the call, the system stops recording and saves the audio clip at step 201 D.
  • Steps 202A-G represent a number of events that can trigger the system to determine what RBT was played or which call related Metadata to display. It is possible for the system to receive either one or more of these events in order to begin a 'what I heard' event. For example, multiple trigger events could occur if a call has completed and the user of phone A later opens the 'what I heard' mobile app.
  • Event 202A is an indication from the phone's operating system that the call has completed.
  • Event 202B is an external notification. For example, an outside source such as from server 30 or 40, or from a carrier's network 10 or 20 have the ability to provide this notification.
  • Event 202C occurs when a user manually initiates the 'what I heard' mobile app.
  • Event 202D occurs when a user has indicated in the app's settings that the app periodically polls the system.
  • Event 202E occurs when an off-line process runs that goes thorough lists of phone A-phone B call pairs.
  • Event 202F occurs when a RBT determination is delivered within the low level protocol exchange during call setup.
  • Step 202G occurs when the user's phone initiates its own third party application.
  • Figure 3a shows a flow chart of a 'what I heard' method.
  • the method utilizes the various servers and databases shown in Figures 1 a, and 1 b to determine the RBT or call related Metadata.
  • the various components of Figures 1 a, 1 b and 1 c are described in detail herein or are known to those of ordinary skill in the relevant art.
  • Step 203 determines if the event includes any data from the 'what I heard' mobile app. If the system identifies RBT data or other Metadata, then the method proceeds to 219 to collect the remaining Metadata.
  • step 203A the system determines the carrier associated with Phone B.
  • Step 203A first consults table 180 to determine if the phone number is known locally. If it is a carrier is determined directly. If not, step 203A contacts the phone number to carrier service 60 to determine the carrier for phone B. After determining the carrier for phone B, the method proceeds to step 205.
  • 'carrier' here is simply a key used in accessing table 100 and may or may not be an actual carrier.
  • Steps 205-214 and 219 constitute a submethod of determining a RBT. The method uses RBT authorities to determine RBTs or other Metadata.
  • RBT authorities are either audio-match or record-based.
  • the audio-match RBT authority determines the RBT through analyzing the audio of the RBT.
  • the record-based RBT authority determines the RBT through analyzing the metadata of the RBT. These steps can be repeated if one RBT authority cannot determine what was heard or other Metadata. If the current RBT authority cannot determine the RBT or other Metadata, the system moves on to another RBT authority.
  • Step 204 is an alternate entry to the 'what I heard' submethod. It provides server 40 or a third party app or server 95 a way of accessing the submethod as an API to determine what was played.
  • Step 205 uses the carrier to determine an RBT authority. The system has access to an ordered list of RBT authorities. If this is the first pass through the 'what I heard' submethod, the system starts at the first RBT authority on the ordered list for this carrier. It should be noted that an RBT authority may in fact be related to an RBT or it may simply provide other Metadata related to this call pair. Subsequent iterations of the 'what I heard' submethod uses the next RBT authority available on the ordered list.
  • step 206 the system determines if there are any RBT authorities available on the ordered list. If there are no RBT authorities available, the method proceeds to step 207. At step 207, the system returns a message that it could not determine the RBT.
  • Step 208 is a decision block that determines whether the RBT authority is an audio- match or record-based authority. If the RBT authority is an audio match authority, processing continues to step 213. Otherwise, for a non-Audio match authority, the process continues to step 209
  • the record-based RBT authority is contacted with the appropriate information to determine the RBT or to retrieve a record that may be decoded to produce a RBT determination.
  • This RBT Authority may be a remote Server 40 or may be a software apparatus housed in Server 30.
  • Step 210 is a decision block where three possible paths ensue. If the RBT or other Metadata is directly returned, the system exits the subroutine. If the RBT authority returned a Raw RBT record, the system decodes this record and sees if it applies to the current phone A and proceeds to step 21 1 . If the authority determined that the RBT record is not supported by this authority, processing continues to step 205 to try another authority.
  • the system applies stored rules to the record to determine which RBT phone A heard. For example, if phone A is on a list of per-phone-number custom RBTs, the RBT listed on the list associated with the phone number is returned. It may also store the RBT in table 120.
  • the method proceeds to decision block 212 which determines whether an RBT or other Metadata was determined. If an RBT or other Metadata was determined, the process exits the subroutine. If an RBT was not determined in step 21 1 , at the subroutine returns to step 205. In some embodiments, the system may fail to determine an RBT in step 21 1 if the stored rules were not sufficient to determine the RBT.
  • the system fails if the RBT requires processing by another RBT authority. For example, a jukebox of RBTs requires further processing by an audio-match RBT authority.
  • the method upon exiting the subroutine, stores the determined RBT in table 120 with its associated phone number. If decision block 208 had determined that an audio authority is needed, processing continues to step 213.
  • the system uses a RBT audio match service to match the clip recorded in step 201 B against a list of pre-analyzed or on-the-fly-analyzed audio clips.
  • decision block 214 determines whether the RBT audio match service found a match for the RBT. If the audio match service was successful, the process proceeds to step 219. If there was no match, the method returns to step 205 where another RBT authority is used.
  • the system Upon exiting the subroutine with an identified RBT, the system proceeds to decision block 215 which determines if the user has the 'what I heard' mobile app. If the system determines that the user does not have the app, the method proceeds to step 218 where the system will send a 'what I heard' or 'what I played' message by some alternate means to the owner of phone A and/or Phone B. If that phone is capable of running the app, the alternate message contains instructions on how to download the app or access the identified RBT information on the Web, social network, or other electronic means.
  • Step 216 determines which display mechanism will be used to display the results. It is also possible that multiple methods may be chosen (e.g., display on a widget but also a pop-up).
  • the system displays the results through a software widget at a spot on the phone's home screen in real-time. As new RBTs or other Metadata, are heard and/or played, this area updates with the lastest results.
  • the widget displays one or more previously identified RBTs.
  • the widget is included in the screen space of another app (an 'in-app' widget').
  • the system activates a pop-up RBT app in real-time on the phone's main screen. The RBT is displayed in app.
  • the app displays one or more previously identified RBTs or other Metadata. It may physically display in full screen or it may be a partial screen.
  • a small portion of the phone display is used to display the identified RBT or other Metadata.
  • the notification bar commonly displays small text and graphic messages alerting the user of an incoming message or reminder.
  • the user is allowed to activate the RBT app through the notification bar.
  • drag-down notification is a variation on 217C.
  • the phone has the ability go to a notification area that would show the identified RBT or other Metadata and allow an app to be run.
  • 217E Normal phone app, the identified RBT or other Metadata is shown within the app.
  • the app display may take up the entirety of the phone display or it may take up a smaller portion or the phone display.
  • the app displays one or more previously identified RBTs or other Metadata. This normal display may also be entered from 217A, B, C, D and F
  • numeric badge the phone device supports a small icon, usually with a number in it, to indicate to the phone's owner that there is some event that occurred. Usually this icon is not shown at all if there are no events.
  • this badge is displayed on the first identified RBT and then incremented on additional identified RBTs or other Metadata.
  • the phone's owner runs the app (217E) the identified RBTs or other Metadata since the last run of the app are shown.
  • Phone Call App 217G is the phone's own phone call app. This Phone Call App may also be a third party app that acts as a third party consumer (95) of the 'what I heard' service.
  • Broadcast and store actions 217J perform 3 separate broadcast operations: 220A broadcasts a RBT or other Metadata determination using a phone's inter-process broadcast mechanism, sending it to any application that is listening. This step also saves this information in a common data store. (Exact features are dependent on a phone's abilities.).
  • 202B stores RBT or other Metadata associated with outgoing calls as part of the person's contact information (likewise it stores RBT or other Metadata information for incoming calls).
  • 202C stores RBT Metadata along with recent phone calls in the phone's call log.
  • Descriptions of 217A to 217J include the embodiment of the processes in the system into specific software apparatus that may be executed on various physical handsets.
  • 217G is a special case of display for an identified RBT or other Metadata. It returns the calculated RBT that was requested in 204 by server 40 or third party server 95.
  • 'What I heard/played' generally refers to the determination of 'what was played or heard' on a phone A or Phone B.
  • 'What I heard' Service generally refers to the service, defined herein for determining 'what I heard' or 'what I played' and for the associated non-RBT and enhanced RBT determinations.
  • RBT Ring Back Tone
  • Phone Number Used generically to refer to a string of digits that is used to access a Phone A or Phone B.
  • Jukebox a collection of RBTs played in a sequence or randomly on phone A's when they call phone B's.
  • Raw record vs. 'what I heard' determination.
  • a raw record is a phone owner's actual RBT administration record that contains rules for playing an RBT on this line. The 'what I heard' service then applies rules to it to determine 'what I heard'.
  • a 'what I heard' determination by contrast is a record retrieved or sent to a service that contains some or part of the actual RBT played.
  • the raw record method in some embodiments may be used to determine non-RBT related Metadata.
  • Audio Features Frequencies, pitch and amplitude values extracted from audio clips. Many features are extracted from a single audio clip.
  • Audio Matching The process of taking Audio Features extracted from a recently recorded audio clip and comparing them to a list of some baseline features from many Audio clips. The process makes a 'best match' of many features over the baseline set.
  • Phone A The Mobile Originating (MO) or 'outgoing' phone. Also referred to as the Caller phone
  • Metadata Information associated with a 'what I heard' determination, including its title, image, code values, corporate information, customer service agent name, name of a separate app that needs to be run etc.
  • This Metadata may be a set of defined data values such as those just mentioned or may be a block of display code such as HTML5 code that is to be displayed.
  • Server used herein to represent a physical processor and its associated components.
  • Service a function, often tied to a particular server. Usually have inputs and outputs in the form of an API or other data.
  • Enhanced Ringback Tone Additional or replacement determination for 'what I heard' or 'what I played' such as information needed to retrieve a coupon mentioned in an RBT advertisement
  • Non-RBT Arbitrary determination and display related to a phone call where no RBT is played. Instead other relevant information about a call pair is determined and displayed.
  • RBT Authority A server, or possibly a local server, that can provide raw records or 'what I heard' determinations given a Phone B and/or a phone A.
  • RBT Authority is associated with a carrier or locally on the phone or server 30, however, in the case of a non-RBT authority it is associated with local or remote server that can provide arbitrary call pair information. In this case it is referred to as a non-RBT information authority.
  • Non-RBT Information Authority A non-RBT authority differs from an RBT Authority in the type of metadata returned on a call pair query.
  • Carrier A service that connects two handsets referred to by the global dial plan.
  • the carrier itself generally plays the Ringback tone, (or they subcontract this service to a third party who provides it through the carrier's network.
  • carrier is simply used as a term to index into an RBT Authority and may or may not be an actual carrier.
  • PBX Private Branch Exchange. Used by entities, small or large corporations or other organizations to direct phone calls to a general, global dial plan number to an employee or other person in the organizations. PBXs also may provide DTMF or voice directed announcements and call routing. The PBX is generally connected to a carrier and may be looked at as a further switch after the carrier has routed the call.
  • Figures 1 a, 1 b and 1 c illustrates the hardware and database tables involved in the system as well as communications paths between these elements.
  • Phone A places a phone call to Phone B.
  • phone B The outgoing (MO) calling party (phone A) and incoming (MT) called party (phone B) throughout.
  • Phone A may either be a Mobile handset (1 ) or a landline (2).
  • Phone A is serviced by carrier network A (10).
  • Phone B may be a mobile handset (3) or a landline (4) on the same carrier (10) or may be on a mobile handset (5) or landline (6) on another carrier network 20.
  • phone A or Phone B are mobile phones they may or may not support the following hardware and software features: 1 ) call started event, 2) call established event 3) Call completed event, 4) ability to respond to external (the handset) notifications 5) ability for an application to run periodically in the background, 6) ability to display information passively on the home screen, 7) ability for an application to 'pop-up' (to become visible and accept input) on the home screen from some event, 8) possess a notification bar that shows various events, 9) has the ability to expand on events in the notification bar, 10) has the ability for the user to run applications from a menu of application, 1 1 ) has a badge that can show a back-ground updateable number next to or over the entry in the menu of applications.
  • the phone 12 has the ability to record the incoming audio stream between the time an outgoing call is started (dialed) up to the point that the call connects (picked up by phone B) 13) has a call log table 1 A, 3A or 5A. 14) The phone may broadcast data to another app. 15) the phone may store data to be picked up by other apps. 16) The phone has an address book (contact list).
  • Phone A or Phone B may house the 'what I heard' software apparatus (app) if there are sufficient hardware and software features to run the app and business rules dictate that the software apparatus be available on those devices. In another embodiment no software apparatus runs on the phone at all.
  • the 'what I heard' software apparatus is housed on a particular phone. It may also house a version of server 50, the audio match service.
  • a third party app, 95 may run on a phone A or Phone B. This app may exist on its own and read broadcast data from the 'what I heard' service on the phone or it may incorporate a binary version of the 'what I heard' service within itself.
  • Phones A and B contact a carrier network 10 or 20 to place and/or receive phone calls.
  • Phone A and B are connected to the internet, 70. Through the internet it may contact the 'what I heard' server 30, any number of RBT Authority servers 40, the Audio processing server 50, phone number to carrier service 80, 'Has App' service 90, text message delivery service 97 and web server 98.
  • Notification service 60 may send asynchronous notifications to Phone A and Phone B. In turn, these notifications may originate from carrier networks 10 or 20, 'what I heard' server 30 or an RBT Authority server 40. Text messages may be received via text message service 97 which in turn originate from server, the 'has app' service 90 (described in inventor's concurrently filed application, entitled, "METHOD FOR EXCHANGING FILES OR MESSAGES WHEN ONLY ONE SIDE OF A CALL HAS AN APPLICABLE SOFTWARE APPLICATION,” inventor: Martin Gindi, attorney docket no. 82227.8003. US01 , filed , which is incorporated herein by reference in its entirety), or server 30.
  • the on-phone version of the 'what I heard' service may provide 'what I heard' broadcast and data to third party apps 95 on the phone.
  • Carrier Network 10 and 20 are Carrier Network 10 and 20
  • Carrier 10, and 20 are the phone switches, cell towers etc. that setup the real time voice connection between two phones and also play the ring-back tone. While server 40, the server that houses a user's RBT profile, may be 'owned' by a carrier for our purposes we consider it a separate entity from 10 and 20. For our purposes, Carrier network 10 originates a phone call from a Phone A. That call can go to a phone B on the same network or be routed to phone B on carrier network 20
  • an RBT is played, it is played by the incoming side (phone B side) of carrier network 10 or by the incoming (phone B) side of carrier network 20.
  • Carrier Networks 10 and 20 may have Call Log tables 10A and 20A that may be, in some variations, accessed from phone A or B and Server 30 through an API. These tables contain lists of calls, incoming, outgoing and missed as recorded by the Network.
  • Carrier networks 10 or 20 my contact the notification service 60 to have it send notifications to phone A or phone B.
  • carrier network 10 or 20 may send notifications to server 30 which then forwards the notification to a notification service 60.
  • the Carrier network could send notifications directly to phone A or phone B
  • Carrier networks 10 or 20 may contact server 30. It also may have an active connection to a server 40 that is part of the carrier's corporate network.
  • Carrier network 10 or 20 may contact text message service 97 to send text messages to phone A or phone B.
  • the internet 70 connects the phones, servers and services that comprise the system. IP or DNS names are used to establish connection sessions between entities. Once connected, illustratively, various protocols such as HTML get/put and XML are used to exchange data. Note that while servers 30, 40, and 50 show direct connections, these connections may or may not in fact route through the internet.
  • the 'what I heard' server 30 houses the 'what I heard' API service.
  • Server 30 also houses the 'track info' API. It may house a local RBT Authority 40 as well as housing the audio match service 50. It also houses the offline processing application started in step 202E.
  • the various APIs provided by server 30 are enumerated below
  • Server 30 contacts Notification service 80 that in-turn forwards these notifications to Phones A and B. In one variation these notifications may go directly to phones A and B. Server 30 uses phone number to carrier service 80 and the Has App service 90. It uses the Text Message delivery service 97 to send text messages to phones A and B. It contacts RBT Authority 40 for various services provided by the RBT authority and contacts the Audio Match server 50 as needed.
  • the services housed on server 30 are provided to phones A and B, RBT Authority servers 40, Carrier Networks 10 and third party apps and servers, 95
  • RBT Authority 40 are servers that provide RBT information through APIs. In one variation, these authorities provide information based on phone numbers and GPS unrelated to RBTs.
  • the actual APIs provided by these servers vary according to each carrier and are discussed herein as illustrative examples. Examples include such things as a raw RBT record related to a phone B, a full 'what I heard' determination given phone A and B, track-info information, track clips and carrier customer log-ins.
  • the system is designed to interface to a broad range of the specific APIs provided by an RBT Authority, however, a list of broad requirements for RBT Authorities connected to the system are enumerated below.
  • RBT authorities to Carrier Networks 10 and 20, the system treats them as logically distinct. They may communicate at will.
  • An RBT authority 40 houses the API services it provides. It may also physically house all 'what I heard' services provided by server 30 or 50.
  • the 'what I heard' service 30 it may contact all the entities described with server 30 above. Additionally it may contact the notification service 80 on its own volition to send notifications to phones A and Phone B.
  • the API services provided by an RBT Authority are accessed by phones A and B and Server 30. In one embodiment they may also be accessed from the third party App or Service 95.
  • the Notification Service 60 is a service provided by third parties such as phone manufactures or carriers that forward notifications from third party servers (such as server 3) to send asynchronous notifications to apps residing on Phones A or B. Often, the apps on phones A and B register that they allow notifications to be sent to it. Notifications may be implemented between the notification service 60 and phones A or B in many different ways, illustratively as polled TCP/IP sockets or as text messages. The system however is designed to operate with all services
  • certain phone models allow the server to send notifications to phones A or B directly without an intervening notification service.
  • notifications may be sent directly between phones.
  • the notification server 60 houses the notification service.
  • the notification service 60 sends notifications to phone A and Phone B
  • the API to send notifications is contacted by servers 30 and 40 and Phone A and Phone B as well as Carrier networks 10 and 20.
  • the API to register notifications is sent from phone A and Phone B. It may also be sent from servers 30 and 40 as well as Carrier networks 10 and 20
  • the phone number to carrier service is provided by third parties to all applications and servers to determine the carrier associated with a particular phone number.
  • This same service may also include device information such as the make/model of the phone as well as things such as screen size and other hardware capability. These services are updated frequently to keep up with the fact that a user may move his or her phone number to other carriers.
  • the system is designed to operate with all services.
  • Phone Number to Carrier server 80 houses the Phone number to carrier service.
  • Service 80 does not contact any other entity in the system.
  • the system uses a 'has app service' 90 to determine if one or both of phone A or phone B has the appropriate 'what I heard' software apparatus installed, and if not, if the phone is capable of running this app.
  • the 'has app' service also delivers device information to server 30, that is information specific to the hardware/software on phone A or Phone B allowing it to make more informed in decisions about delivering information to the phones. As an example, it may decide that a device supports 'notification bar only' and does not support 'widgets', in which case the 'what I heard service' would send a notification to the phone.
  • the 'has app' server 95 houses the 'has app' service.
  • the 'has app' server contacts the text message service 97 and notification service 80 to send alternate deliveries to phones A and Phone B. It also refers in some of these notifications to web server 98. Additionally, internally it contacts the phone number to carrier service 80 and may pass some of this information to any server that uses its services (to save money by not calling the phone number to carrier service twice for the same phone number)
  • a Third Party server or app on a phone may use the 'what I heard' services and APIs of an RBT Authority. They are shown in the system to illustrate that they may access the services.
  • These third party apps or services may be housed in separate servers or they may reside on phone A or phone B. When they reside on a phone they may use the data and broadcast methods of the 'what I heard' service or they may incorporate binary versions of the 'what I heard' service within themselves.
  • the 'what I heard' service When the 'what I heard' service is incorporated within the third party app, it may access all the services and servers that the on phone 'what I heard' service may access. It may register for broadcast information from 'what I heard' services broadcasting on this phone.
  • the 'what I heard' service when incorporated into the third party app, it may receive notifications from the notification service 80.
  • the text message delivery service 97 delivers text messages to phones A and B on behalf of a server.
  • the text message delivery server 97 houses the 'has app' service.
  • Server 97 directly sends text messages to phones A and B.
  • Server 97 is used by the 'has app' service 90 to send text messages to phone A and B.
  • the 'what I heard' services sends text messages via server 97 to phones A and B
  • Web server 98 is referred to by text messages sent by the 'has app' service and by phone number variation schemes described below for phones that do not have access to their own phone numbers.
  • the web server in the exception case, contacts server 30. [00126] Phones A and B contact this server to see web pages. Audio Match Server 50
  • the Audio Match Server provides an audio match service between clips recorded on phone A or phone B during ring-back time against various sets of referenced audio clips stored in table 51 . It provides an API to do so as described below.
  • the Audio match service may be housed on server 50 or it may be housed on phone A or B as well as on server 30. It may also be housed on server 40 when server 40 hosts the 'what I heard' service.
  • the audio match server may contact a server to get referenced audio clips. These clips may be housed on server 30 or 40.
  • the audio match service may be accessed from phones A or B, server 30 or server 40 when it houses service 30.
  • the Private Branch Exchange, PBX 98 is a private switch that routes to 'extensions', generally handsets 981 within a company or other organization. These handsets are alternate 'phone B's in our system.
  • the carrier network 10 or 20 connects phone A to a PBX by use of a global phone number.
  • Most PBXs today use a DTMF (phone pad) or voice recognition system 982 to further route the call from phone A to an employee or other member of an organization possessing a handset 981 .
  • the person on phone A may only interact with 982.
  • 982 any be an automatic account system that verbally tells you your bank balance. In this case, 982 would be considered a Phone B.
  • a ringback tone is played exclusively during the ringback period by carrier 10 or 20. Once connected to a PBX, the ringback period is over and any audio heard on handset 10 comes from the PBX and is not a ringback tone for our purposes.
  • the person at handset 981 may also act as a phone A, calling out to out to a phone B at the other end of a carrier network
  • a 'direct dial' number is shown as the incoming phone number on phone B or some general PBX number may be shown.
  • the PBX may provide services whereby it calls out automatically and plays automated audio, this acting as a phone A. (i.e. 982 would act as a phone A)
  • the PBX keeps information about calls in table 983. This information includes routing information, e.g. the handset 981 that phone A or the call out to phone be was connected to.
  • the PBX provides a front end server that allows access to call information that may or may not be physically part of the PBX.
  • Server 40 in this case takes as input phone A's phone number, and possibly other identifying information such as a timestamp and provides information stored in 983 about the call.
  • the PBX may contact the notification service 60 to have it send notifications to phone A or phone B.
  • the PBX may send notifications to server 30 which then forwards the notification to a notification service 60.
  • the PBX could send notifications directly to phone A or phone B. These may be done through internet 70
  • the PBX may contact server 30. This may be through internet 70
  • the PBX may contact text message service 97 to send text messages to phone A or phone B. This may be through internet 70
  • Figure 2 shows the columns of database tables as they exist on server 30. As shown in Figures 1 a, 1 b and 1 c though, there are equivalent tables on server 40. The system is not concerned with the structure of these tables on 40 and is shown for illustrative purposes only; server 30 indicates retrieves data from these tables via API to 40.
  • Figure 6 show illustrative API returns from raw data and 'what I heard' calls from server 30 to server 40. These show a typical return value. These illustrative return values also describe the abstract data stored in certain columns of table on server 30. Indeed the actual return values from API calls are sometimes stored in these tables.
  • RBT Authority info (100) (physical table 31 and optionally, 1 B, 3B and 5B).
  • This table allows server 30 or phones 1 , 3 or 5 to contact various RBT authorities. When used for non-RBT determinations this table is referred to as an Information authority'.
  • RBT and information authorities differ only in nomenclature and not their methods. These authorities may be located on phones 1 , 3, 5, server 30 or server 40.
  • the table is accessed by carrier name as returned from the phone number to carrier service 80 or may be accessed without carrier name.
  • Server 30 or phone 3 or 5 then obtains records with increasing ordinals, one at a time, until one of the authorities returns a 'what I heard/played' success.
  • Column 101 record ID, is a simple sequential number referring to this row in the table.
  • Column 102, carrier, the carrier for this record and if carrier is searched may match the carrier being requested. Multiple carriers may be listed in this column as belonging to this RBT Authority.
  • Column 103 is the RBT authority to be used in this record.
  • the server contains an address if the server is remote: (e.g., AT&T RBT Server 1 ), or it may indicate an Audio match service to contact, or it may indicate 'local-data' in order to access the local user RBT Store table 33, local-audio' to use a local audio match service, local-enough' indicating that there is enough track data in a 'what I heard' determination or 'local-code' indicating that a software apparatus local to server 30 is to be used as the RBT (or information) authority.
  • Column 104 is the ordinal for this RBT authority for this carrier (i.e. they repeat for each carrier or aggregator); the service will try Authority #1 first and try it, then #2 and so on.
  • Column 104 may also contain special codes for special actions including: 'have enough' and 'track-info- ⁇ for use when the table resides in 1 B, 3B or 5B.
  • Column 106 is a pointer to a rule in table 130. These are the rules to apply to records retrieved from an RBT Authority.
  • Figure 2 shows table 100 filled with a number of illustrative examples.
  • Record 1 is for a corporate RBT check. As an illustrative example this type of RBT a large group of phone numbers belongs to a group that has particular rules for those phones.
  • column 103 indicates that this check is available on RBT Authority 'AT&T Server-1 '
  • column 105 indicates that an API request should indicate 'Corporate_RBT1 ' so that the server know to return that record.
  • Column 106 points to rule record 1001 in 130
  • rule 1001 indicates that a 'what I heard' is returned, that the track Metadata be retrieved via API and that the record be stored in 120 for cache purposes
  • Record 2 is the next AT&T RBT authority to be tried should AT&T ordinal 1 fail (i.e. the number is not in the corporate list or the time of day does not require the ID to be played).
  • Record 2 allows for access to phone B's raw RBT record in order to calculate an RBT heard/played.
  • Column 103 indicates server 'AT&T server 2' should be used.
  • Column 105 indicates that the API should request 'Raw RBT profile';
  • column 106 indicates rule 1002 in table 130 should be used.
  • This rule has a set of rules that define how to process the retrieved raw RBT record. This rule would return the RBT heard/played, however, this rule also states that if a jukebox is determined, then go on to ordinal 3.
  • Record 3 is performed if a jukebox was returned in ordinal 2 to cause an audio match on the small set of possibilities in the jukebox. If a match occurs it is that track, if not, we return the jukebox anyway.
  • Column 103 indicates to use 'audio-1 ' match server that contains the clip signatures for AT&T.
  • Column 105 has server 30 asking a server 50 to process a jukebox, (as say opposed to a match over the whole catalog).
  • Column 106 points to rule 1003 in table 130. Rule 1003 states 1 ) if there is an audio match return it, if not, 2) return the original jukebox as a match.
  • Record 4 shows a carrier aggregator that is an RBT authority that supports multiple carriers; this aggregator returns raw RBT records. There is no secondary RBT authority.
  • Column 102 indicates that three carriers match this RBT Authority, 'Orange', 'O2' and 'vodaphone'.
  • Column 103 indicates if any of these are matched go to RBT Authority 'Europe aggregator-1 '.
  • 105 asks for a raw profile and 106 uses rule 1004. That rule is the same as rule 1002 in record 2 above, except that it doesn't allow for Jukebox processing by an audio service. It also gets Metadata from table 1 10
  • Records 5, 6, 7 and 8 are a set of records for a carrier 'Verizon' that is processed locally on server 30 with access to table 120 (and on the audio server) rather than going to a remote RBT authority.
  • Record 5 is a 'what I heard' cache check, it uses both phone A and phone B to access table 120 to find a recent determination of 'what I heard'.
  • Column 103 is set to 'local-data
  • column 105 is set to 'What_l_Heard_AB' which can be used as the search type key for column 124 in the User RBT Record table (120)
  • column 106 would use rule 1005, that rule, unlike rule 1001 , indicates the track Metadata is stored with the record that will be retrieved from 120. Also that it should not store the data in 120 (which would result in the retrieved record being stored again.)
  • Record 6 is equivalent to record 1 , except that it a) indicates to search local table 120 for type 'VZW Corp.' (column 103 is set to 'local-data', column 105 is set to 'VZW Corp.'). and 2) it uses a rule 1005, the same rule as was described with record 5 to get the 'what I heard' directly and not store it.
  • Record 7 is equivalent to record 2, except it indicates in column 103 to search for search type 'RBT_PROFILE_REQ3' in table 120.
  • Rule 1006 differs from rule 1002 only in that it does not store the retrieved record again in 120.
  • Record 9 demonstrates a raw corporate record that has a time-of-day rule. It is an alternate ordinal 1 for AT&T.
  • a raw corporate record CORP_RAW_RBT is made.
  • Rule 1007 indicates to use the only the TOD rule in the retrieved raw record. It then stores this raw record as A/B for cache purposes.
  • Record 10 demonstrates an "enhanced RBT housed at an external carrier.
  • the T-Mobile server 'Ad-Server-1 ' is accessed after a call.
  • the user would have heard an RBT at the beginning of the call, say about Pepsi.
  • the current GPS location of the phone, along with Phone A, Phone B and direction are sent with request 'Send_GPS' in column 105.
  • the carrier employs a third party RBT advertisement content and that server 40 contacts (or is) a server provided by that third party to process advertisement requests. That relationship is out of the scope of this disclosure.
  • the server returns info about an advertisement that is based on the GPS value and Phone A and B values. Rule 1005 is used since the format of the return resembles a 'what I heard'.
  • Record 1 1 shows querying a Server 40 associated with a PBX.
  • phone A would have been talking to a customer service rep through Time Warner Cable's (TWC) PBX.
  • TWC Time Warner Cable's
  • the 'what I heard' query here would return, as an illustrative example, the company name, the name and photo of the customer service rep and a way of calling this customer service rep directly.
  • This record 1 1 in table 100 contains information needed to contact the appropriate server, in this case server TWC-4.
  • the record also indicates that a 'what I heard' determination is expected.
  • Record 1 1 further refers to rule 1012 (column 106), referring to table 130 which then refers to rule '13' in rule list 560. This rule says to expect a direct 'what I heard' determination with 'who you were talking to' Metadata.
  • Record 12 shows a local software apparatus delivering non-RBT metadata related to a particular phone number.
  • This illustrative example shows calling an American Idol voting line that registers a vote for a particular contestant. After the call, the 'what I heard' software apparatus will thank you for voting for that individual and allow you among other things to download the song the contestant just sung.
  • the second record in Table 180 contains the illustrative record for any phone A calling this phone B, in this case it is voting for a particular 'American Idol' contestant using one of two 1 -800 (866) numbers assigned to vote for that particular contestant.
  • Table 181 contains this list of phone numbers used to vote for this contestant.
  • Column 182 has a key that will be used to access table 100, record 12(in figurel O).
  • This record 12 in table 100 indicates that the local-to-server-30 software apparatus be contacted (via the special value 'Local-code' in column 103) and that it need only pass 'phone b' (request column 105).
  • the record also indicates that a 'what I heard' determination is expected.
  • Record 12 further refers to rule 1013(column 106), referring to table 130 which then refers to rule '14' in rule list 560. This rule says to expect a direct 'what I heard' determination with a generic set of data to display, in this case it has two images, two sets of text and two action URLs connected to two buttons.
  • Record 101 instructs the phone to determine 'what I heard' by calling the 'what I heard' service on server 30.
  • Column 103 indicates the server 30 to contact, TMI-1 '.
  • Ordinal (104) is 1 (note, in our illustrative example record 102 is an alternate ordinal 1 , either record 101 or 102 would be on a phone).
  • Column 105 indicates that the phone will use server 30's 'what I heard' service that takes Phone A, Phone B and direction as input, and returns 'what I heard/played' to the phone.
  • Column 106 indicates that rule 1005 would be used to interpret the results (a 'what I heard' is expected with track info and it is not to be stored)
  • Record 102 is an alternate record to 101 , it indicates that a server 40 be contacted directly to get the raw record for Phone B (from both a phone A and a Phone B).
  • Column 103 indicates that 'AT&T server 2' is contacted. The ordinal is 1 .
  • Column 105 indicates that the phone will request a raw record from AT&T server 2.
  • Column 106 indicates that rule 1006 would be used to interpret the results (Raw record processing with a failover to audio processing)
  • Record 103 provides this audio processing local to the phone using on- the-fly signature processing of the returned tracks in the jukebox in record 102.
  • Column 103 indicates that 'local-audio' be used. The ordinal is 2, following record
  • Record 104 is a special record relevant to the phone only it is an RBT authority specifically searched for by the software apparatus on the phone to process events 202B that contain all the Metadata information needed in the event.
  • 103 indicates 'local-enough'.
  • the ordinal (104) is 'enough', the special search value.
  • Column 105 indicates that 'enough' be performed.
  • Column 106 indicates that rule 1005 would be used to interpret the results (a 'what I heard' is expected with track info and it is not to be stored).
  • Record 105 is a special record relevant to the phone only it is an RBT authority specifically searched for by the software apparatus on the phone to process events 202B that contain no Metadata for the 'what I heard' included in the event that only had a trackID and that server 30 needs to be contacted to return the Metadata associated with the trackID.
  • Column 103 indicates that the server TMI-1 needs to be contacted.
  • the ordinal (104) is 'track-info'.
  • Column 105 indicates that the 'track-info' API be used when contacting server TMI-1 .
  • Column 106 indicates that rule 1010 would be used for this 'what I heard' (use API for track info, no store).
  • RBT Metadata (1 10) (physical table 32). This table stores data describing a particular RBT of varying types. Examples include Music RBTs, Advertisement RBTs and jukeboxes, arbitrary collection of various types of RBTs that may be played in some (usually random) order.
  • Column 1 1 1 is a simple sequential number referring to this row in the table.
  • Column 1 12 contains the type of Metadata contained in the row. Examples include 'music track' and 'info'. These are schemas that describe the method of decoding data in column 1 14. Examples of these schemas are 530, 540 and 550.
  • Column 1 14 is the Metadata itself.
  • This table is populated from large data transfers from carriers and RBT audio clip providers. It may also be populated by caching the API return values from various calls to server 40. It may also be populated manually.
  • Figure 2 shows table 1 10 filled with a number of illustrative examples.
  • Record 1 of table shows music track data. It would be accessed in this case when an audio match returned a music track
  • Column 1 12 indicates 'track data' as the type.
  • Column 1 13 has a unique id, in this case '39', and column 1 14 has the actual data in the form of 530. It contains the track title (532) artist (533), album (534), album art (535) and the audio review clip (536)
  • Record 2 has the info needed for an advertising clip.
  • Column 1 12 indicated 'info with URL'
  • 123 has a unique ID, in this case '42'
  • column 1 14 has the actual data in the form of 550, in this case 553, the 'info' is a URL 'www.pepsi.com'
  • User RBT Store 120 (Physical table 33 and 3D and 5D). This table stores information related to a user's RBT. It may be raw or 'what I heard'. The table allows for multiple entries for the same user, allowing the ordinal system of table 100 to be used to access various possible RBTs in sequence. As an example an ordinal of 1 in table 100 could specify an advisement RBT and ordinal 2 in table 100 a music track with raw data, if there is then no ordinal 1 in this table, but there is an ordinal 2, then a music track has been played previously and not an Advertisement.
  • Column 121 is a simple sequential number referring to this row in the table.
  • Column 122 contains the phone number or other unique identifier (such as email address or login name or other number unique to the phone) for Phone B. This column many also contain multiple phone numbers to form a group.
  • Column 123 optionally contains a phone number A that is used only when this table contains specific results, such as 'what I heard' determinations between two phones.
  • Column 124 is a search type identifier for this type of user profile record, Often it is the same as column 105 request type.
  • Column 125 is the type of user data contained in the row. Examples include 'Raw' and 'what I heard'. These are schemas that describe the way of decoding data in column 126. Examples of these schemas are 500 and 520.
  • Column 126 is the User record itself.
  • This table is populated from large data transfers from carriers and RBT audio clip providers. It may also be populated by caching the API return values from various calls to server 40.
  • table 120 exists on phone' B in physical tables 3D and 5D. This is used to store the first raw RBT record retrieved for use in a scheme, described below, that uses this record for all 'What I played' determinations on phone B. Only one record is stored in this table on the phone.
  • Figure 2 shows table 120 filled with a number of illustrative examples.
  • the search type (124) is set to the value in 105 for local RBT authority record 5.
  • Data type 125 'what I heard' indicates schema 125 is used to interpret the 'love shack' what I heard record stored in 126.
  • Record 2 shows a corporate RBT 'group', here there are many phone numbers in the group, any match to a number is relevant to this group.
  • Column 122 has 666-1212 and 777-1212.
  • Phone A column 123 is blank.
  • Column 124 is 'corporate RBT2'.
  • Data (126) is in 'what I heard schema' (125) and indicates that 'Corp. Msg' is played.
  • Record 3 shows a record where a Raw RBT record is stored, a rule from 130 will be applied to it to calculate what a phone A would have heard.
  • Data (126) is in in 'what I heard schema' (125). This data record has all the song(s) and RBT configuration associated with line 888-1212
  • Rule table 130 (Physical table 34). This table contains the rules for interpreting API results returned from 40 and data in table 120. The rules are from a simple set of rules available in the system. These rules are dependent on actual RBT authorities and are thus illustrative in the current disclosure. Some of the rules are arranged in sequence and are meant to be executed in sequential order to calculate or extract an RBT from a retrieved record (marked as 'ID' in illustrative table 560). Other rules are meant to be executed after an RBT is calculated or extract. These are marked 'result' in illustrative table 560)
  • Table 560 contains illustrative rules.
  • Column 131 is a rule ID.
  • Column 132 contains rules for this row.
  • Figure 2 shows table 130 filled with a number of illustrative examples that refer to the illustrative rules in 560.
  • Rule IDs 1003 and 1005 are variations on the 'what I heard' rules.
  • 1003 uses rule 7 instead of 8 which indicate that the track info is to be retrieved from local Metadata table 1 10.
  • 1005 uses rule 6 indicating that the data is stored within the 'what I heard'.
  • Rules 1004 and 1006 are variations on the raw rules. 1004 does not allow for jukebox failover and has no storage rule. 1006 allows for the jukebox failover but it also has no storage rule.
  • Rule 1007 is used to process a corporate RBT record that returns just a raw time of day rule. It then stores the record in the cache.
  • Rule 1009 is a special rule for audio processing of a jukebox, here, the audio match determined the RBT within a returned jukebox.
  • Rule 1009 indicates that it should get the Metadata from that previously returned record.
  • Rule 1010 is used to indicate that a simple API access is needed for a 'what I heard'.
  • Rule 1012 says to expect a direct 'what I heard' determination with 'who you were talking to' metadata (rather than RBT related metadata).
  • Rule 1013 says to expect a direct 'what I heard' determination with a generic set of data to display, in this two images, two sets of text and two action URLs connected to two buttons.
  • Audio clip match table 140 (Physical table 51 ). This table stores audio signature clips for audio match server 50.
  • the system supports two distinct versions of this table, 1 ) a database of pre-extracted features, usually done off line and 2) a list of signatures extracted from reference clips just-in-time (on the fly) right before an attempt is made to match the just recorded clip. These two differ in physical location and storage of the table, but are logically similar.
  • Table 140 contains this set of extracted features.
  • Column 141 has a unique record identifier.
  • Column 142 and 143 are used to select sub-sets from the main list.
  • Column 142 is 'search type' it can be used to collect extracted features. A match would occur only against those with this search type.
  • Column 143 'search unique ID' is a unique identifier for this record. It could for instance contain the trackID that is referenced in a jukebox (545).
  • the 'extracted features' from the baseline analysis for a track are stored in column 144. Alternately this column is a URL to a reference clip.
  • Column 145 is the type of user data contained in the row and is used along with column 146 (ID) as search keys into the Metadata table 1 10.
  • Figure 2 shows table 140 filled with a number of illustrative examples.
  • Pending Verification table 150 (physical table 35). This table supports two facilities related to phone devices that do not allow the 'what I heard' software apparatus to access its own phone number. The first facility allows global phone numbers to be verified, the second facility allows for unique ID to global phone number mapping
  • Column 151 has the global phone number. It is stored in both uses of the table.
  • Column 152 has 'state', for the verification function the state may be 'pending verification' or 'verified'. For the unique number to global phone number facility the values are 'pending pick-up' and 'picked up'.
  • Column 153 has the unique number stored only in the unique number to global phone number facility
  • Figure 2 shows table 150 filled with illustrative examples
  • Phone number translate table 160 (physical table 36). This table supports two facilities related to translating global phone numbers and string and binary values. The first facility translates global phone numbers to string and binary values. The second facility translates string and binary values to global phone numbers.
  • Column 161 has the global phone number
  • column 162 has the associated string or binary value
  • column 163 has the type of value associated with column 162. These are dependent on the specific RBT authorities or users of 'what I heard' systems. [00208] This table may be populated by large transfers from an RBT Authority, or populated by an algorithmic means.
  • Figure 2 shows table 160 filled with an illustrative example.
  • Column 171 contains a phone number
  • Column 172 the text string that will be displayed in the address book and contains the 'what I heard' or 'what I played' indication.
  • Column 173 is the call direction. It is either 'outgoing' or 'incoming/missed'
  • Figure 2 shows table 172 filled with an illustrative example.
  • This table provides a mapping between specific phone numbers; groups or ranges or phone numbers and a RBT/lnformation Authority table 40 entry that usually, but not always, access a non-RBT authority.
  • This table is used instead of a carrier lookup (it is accessed first and if there is an entry here, no carrier is looked up, as such it may be considered a local carrier lookup table).
  • this table may in-fact point to an RBT authority that is carrier. In that case this table acts as a carrier-lookup facility.
  • Column 181 contains an abstract field that may be a single phone number or represent a group of arbitrary numbers, a range of arbitrary numbers or any combination thereof.
  • Column 182 is the RBT authority name, which is used as a key to match column 102 or authority table 100.
  • FIG. 1 shows table 180 filled with a illustrative examples.
  • the record for TWC (Time Warner Cable) shows a combination of single numbers and ranges of numbers that all access the same server 40, and thus the same after call information and experience.
  • the information authority value TWC accesses column 102 in table 100 retrieving record 1 1 in that table for the TWC.
  • the record for American Idol has two numbers associated with the American Idol line. A match on this retrieves record 12 in table 100 which then indicates that local-code' (local to server 30) will be called, passed phone B to get a 'what I heard' determination for calling this phone B.
  • Events 202A through 202E as well as a call from an external entity (204) trigger 'what I heard/played' processing.
  • Step 202A is an automatic event sent by a phone's operating system indicating that a call has completed. It is sent if the phone supports the generation of this type of event and business rules allow for this type of event service.
  • This completion event is for when: 1 ) an outgoing call, originated on Phone A, is completed by either party ending the call (or by the outgoing party aborting the call before the incoming has answered), 2) an incoming call on Phone B is completed.
  • a call is only classified as an incoming call if Phone B answers the call. 3) if Phone B does not answer the call then it is a 'missed call', This event is triggered as a call completion even after phone B fails to pick up the call or phone A aborts the call before phone B picks up.
  • a software apparatus on phone B configures the phone operating system to send these 3 events to it on call completions.
  • This software apparatus receives these events when the event occurs.
  • Step 202A records which of these three events occur.
  • the event itself may include the 'other sides' phone number or the software apparatus may issue further instructions to retrieve this information from the phone OS.
  • the software apparatus also collects its own phone number. The results of this step thus includes 1 ) the phone number of the current phone, 2) the phone number of the remote phone and 3) the 'direction' of this call (incoming, outgoing or missed).
  • a variation on this event has a single 'call completion' event.
  • the software apparatus further determines if the event was 'incoming, 'outgoing' or 'missed' by querying the phone operating system.
  • step 202A In another embodiment of step 202A some or all of the remote number or direction may not be available in this case, after the incoming, outgoing or missed event is generated, the software apparatus searches the phone's call log, 1 A, 3A or 5A for the last call received or the last call(s) not processed by the software apparatus to obtain a list of one or more calls to be processed further. It is scanned if the phone supports it and business rules allow for this type of scan. For each of these calls the system retrieves the 'other sides' phone number along with call direction.
  • the system may have a separate event each for incoming, outgoing or missed, in which case only the call log for that type of call is searched, or it may just get a general 'call completed' event in which case the incoming outgoing and missed logs are all searched. In all these cases the phone number of the current phone is retrieved from the phone operating system or it may be in the call log.
  • the software apparatus may store the information from these events for a run of the app 202C or a periodic poll (202D) to further process the event, or it may hand it off directly to the 'what I heard' processing section.
  • a periodic poll 202D
  • the call log, or latest calls may be retrieved via API from carrier networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should this call log information be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • Step 202B is an event from some entity external to this software apparatus.
  • these entities include: server 30, server 40, network 10 or 20 or servers 40, networks 10 and 12 via server 30.
  • some third party entity 60 forwards the event to the handset or the event generating entity may send it directly to the handset.
  • this event may occur in another software apparatus on this same phone, or on some other phone (and be forwarded directly or through third party entity 60). External notifications are received if the phone supports it and business rules allow for this type of event service.
  • a special case of notification is when the phone OS does not send external notifications right away to the software apparatus. Instead, if queues them until the app is later run (via a 202C). Once running the OS sends these events to the app. Processing then proceeds on each of these queued notifications within the running version of the app.
  • These notification events may have the following payloads or combinations: 1 ) no payload at all (i.e. just the event), 2) the 'what I heard/played' track number (or the complete track), 3) the 'other side' phone number and the direction of the call, 4) a reference number that can be used to pick up further information on a server for this call. 5) A reference to a previously stored 'what I heard' result on this phone. Additionally the external notification may contain a number of these payloads. (I.e. the record of a number of phone calls are sent at the same time)
  • the phone's operating system forwards this event to the software apparatus:
  • the raw event, with its possible data may 1 ) simply be stored at this point and further processing continue when the app is run (202C) or 2) a poll later sees these store values (202D).
  • notification increments the badge value 217F and data may be stored in raw form now or processing allowed to continue through 'what I heard' to step 217F now.
  • the software apparatus scan call logs 1 A, 3A or 5A for calls since the last time the apparatus was run, returning one or more calls to be processed along with 'call direction' a retrieved from these files.
  • the call log, or latest calls may be retrieved via API from networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists.
  • step 203 If 'what I heard' or 'what I played' is included in the event 'what I heard/played' processing is not required (shown in step 203), directionality may be indicated by the message saying 'heard' or 'played' or it may be an explicit part of the value included in the event. Additionally the 'other sides' phone number may or may not be included.
  • a reference to a previously stored 'what I heard' results may be returned, if for example some other entity had stored these results and is sending a notification for this software apparatus to display it. In this case processing would continue to 215 via 203.
  • Step 202B thus passes one of the following on to the 'what I heard/played' section: 1 ) the phone's phone number, the other side's phone number and call direction 2) a reference number to be used later to retrieve call/what I heard information or 3) 'what I heard/played' itself along with the other sides' phone number and call direction (if the latter two are provided). It may pass a single call or multiple calls.
  • step 202C the software apparatus is invoked manually to display 'what I heard'. It may be invoked from the list of applications on the phone, from the notification bar in 217C, the drag down in 217E or noticing a badge in 217F. It may also be run from the widget, 217A or pop-up, 217B. Running of the application is done if the phone supports running applications manually and business rules allow for this type of event service. [00241] When the application is run, it processes various lists of calls and events. It may process one or more of these lists:
  • the scan of call logs occur if the phone supports it and business rules allow for this type of scanning.
  • the call log, or latest calls may be retrieved via API from carrier networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should they be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • the various stored raw outputs from set 202A and 202B are processed, these include 1 )other phone, this phone and direction triplets, 2) 'what I heard' results from notifications,, reference numbers to servers and references to already stored 'what I heard' results. Each of these would in turn be passed to step 203 for further processing.
  • a periodic event such as a system timer causes processing of 'what I heard/played' to start. Polling is done if the phone supports it and business rules allow for this type of service.
  • the periodic event processes various lists of calls and events. It may process one or more of these lists:
  • the call log lists in 1 A, 3A and 5A for calls since the last invocation of the app. retrieving Other phone numbers' and call direction along with the phone number of this phone and passes the information for each of the calls found to step 203 in turn.
  • the scan of call logs occur if the phone supports it and business rules allow for this type of scanning
  • the call log, or latest calls may be retrieved via API from carrier networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should they be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • the various stored raw outputs from set 202A and 202B are processed, these include 1 )other phone, this phone and direction triplets, 2) 'what I heard' results from notifications,, reference numbers to servers and references to already stored 'what I heard' results. Each of these would in turn be passed to step 203 for further processing.
  • step 202E lists of Phone A and Phone B are processed on servers 30 or 40 with the intention of sending notification or a 'has app' alternate messages to Phone A and/or Phone B. These lists optionally directly include a third field, 'what I heard/played' in this call to allows for faster processing by removing the need to do a 'what I heard' determination.
  • Step 202F call progress protocol
  • CDMA Code Division Multiple Access
  • GSM Global System for Mobile communications
  • CND Call Number Delivery protocol
  • the data transmitted during call establishment should have some element of 'what I heard', ideally at least the title of the RBT that is playing. Additionally, processing and communications during this phase of call establishment is limited, so full processing of a 'what I heard' is not practical.
  • 202F proceeds to step 221 immediately to store this information in a known, accessible location and processes it no further. So on receipt of this information, 202F stores the information in a known accessible location to be picked up. It may also send a broadcast style inter-process event to listening app to pick up this information (or include it in the broadcasts).
  • Step 202F is embodied within the low level call establishment software apparatus within a phone.
  • step 202F is incorporated into a third party's application in this case it saves the information or broadcasts it as above, but in addition it may simply pass the information to another section of the third party app.
  • While the system does not generally cover how a carrier's network generates this information and sends it in the low level protocol, it does cover the case where the network, having both Phone A and Phone B's phone numbers calls the 'what I heard' API service on server 30 to determine 'what I heard'.
  • 202F may also occur more readily during a VOIP type call since the protocol and app that places/receives VOIP calls are more readily changeable. Indeed, full 'what I heard' service may proceed during call setup.
  • Step 202g is a run of a third call app.
  • This type of app on the phone places a phone call to a remote handset or receives an incoming call from a remote handset and stays active for the duration of the call.
  • every phone has at least one of these applications embodied by the phone's normal call placement app. (the app that allows you to dial or receive incoming calls). It is possible however for other apps to provide this functionality, whether for a normal voice call or for a VOIP application.
  • a 202G starts this type of app. it is used exclusively in conjunction with display method 217I, display within this type of app.
  • steps 203 through 214 and step 219 determine 'what I heard/played', determine that you did not hear an RBT or that the determination is inconclusive.
  • step 203 causes the 'what I heard' processing to be skipped, jumping to step 215.
  • step 203A determines a carrier that is used as a simple key into the RBT authority table 100, column 102.
  • step 203a tries to see if the phone number is locally known in table 180. To do this it makes an abstract match of this phone number to the phone number, phone number list or phone number range in column 181 . If there is a match, column 182 is used as the carrier key and processing continues at step 205. If there is no match in table 180, step 203a contacts the phone number to carrier service 80.
  • This service takes as input a phone number and returns the carrier that is currently servicing this number. In addition, this service returns 'device information' such as the phone model, operating system on the phone, screen size and so on. Some of this later information is used later in the display phase.
  • Step 205 starts the iterative processes of finding an RBT or information authority for the 'carrier' code returned in step 203A.
  • the process starts by searching for an RBT authority whose 'carrier code' column 102 contains the 'carrier code ' retrieved in step 203A and whose ordinal is 1 .
  • the information in this record, all the records that follow it and phone A and Phone B are combined as described below to calculate an RBT using this authority. If steps 212, 203, 214 or 210 return a 'what I heard', then the iterative loop is exited. If this iteration does not determine 'what I heard', or in one variation, the determination was for a Jukebox that needs further processing', the loop is repeated again at 205 with the same carrier column 102 match and the next ordinal in numeric sequence.
  • Each new RBT Authority tried returns the following 1 ) from column 103 whether to a) contact an audio match service (50), b) an external RBT authority (40), c) to process the results locally on any of 30, 1 , 3 or 5 or d) that there is no available or remaining authority. 2) An API request rule from column 105 to pass to the appropriate authority to request processing. 3) The rule in column 106 to be used in processing the result from the RBT Authority.
  • step 205 returns that there are no RBT authorities that match this carrier and the current ordinal
  • step 206 forwards processing to step 207 to return that there are no more authorities.
  • This determination means any of the following 1 ) there were authorities for this carrier, however none of them had, in a past iteration of the 'what I heard/played' loop returned an RBT that was played or heard for this phone number. In this case 'no more authorities' means that there is in fact no RBT that was played. 2) That there were no RBT authorities as all, in which case it is possible that an RBT was played on this line, however, since this system does not have access to an authority for this carrier, we could not determine if an RBT had been played. 3-b-ii. Obtaining 'what I heard (or played)'
  • Step 208 directs processing to either an RBT record based approach (209) (method 1 below) or to an audio matching approach (213) (method 2 below)
  • Step 209 starts the RBT record based approach.
  • Phone number B is combined with the request string/rule returned from step 205 and stored in column 105 of table 100.
  • phone A is also combined with the string/rule.
  • Step 209 may be performed on either the 'what I heard' server 30 as the RBT authority, on phone A or B, or through a remote RBT authority 40.
  • column 105 indicates a remote authority along with what is generally a string to assemble an API request to server 40. Phone B and sometimes Phone A are inserted into the request string and sent to server 40.
  • Server 40 may return one of three things, 1 ) Direct determination of 'what I heard/played', 2) a raw RBT profile record for phone B or 3) that the RBT authority was capable of determining with a certainty which RBT was heard/played but that in fact no RBT was played. As illustrative example of the later, the RBT authority is an RBT advertising authority and given both Phone A and Phone B that it did not play one of its RBTs this value would be returned)
  • Step 209 determines what return value type is expected as a return from the remote RBT Authority by consulting with rule column 106.
  • rule column 106 has 3 rule types. 1 ) return value 'raw' is expected in the illustrative format 500 2) return value 'what I heard expected' in illustrative return value 520 or 3) either raw or what I heard is returned, and step 209 should look at the format of the return value to see if 500 or 520 is returned.
  • Step 210 determines the flow of the process based on these three possible return values.
  • 209 extracts 'what I heard/played' from the returned record.
  • Illustrative example 520 shows a return of track, jukebox, info, an error indication or an empty RBT. If the record indicates that a track is returned, it would be in the format 530. If the record indicates that a jukebox is returned, it is in the format of 540, which would then contain a list of constituent tracks in form 530. If an informational type RBT is returned it is in the format 550. In another variation, only partial Metadata needed for the display section is included in the 'what I heard'. In either case, processing continues to step 219 where the remaining Metadata is collected.
  • a local authority is indicated by column 103 set to any of: local-data, local-audio or local-enough.
  • column 105 has a search type key used to access table 120.
  • Table 120 contains the same values in Colum 126 that can be returned from the API call, and thus a local RBT record authority differs little from the remote RBT record authority above. These differences include:
  • Access to this same data is achieved by searching for the appropriate data in table 120 rather than an API call.
  • phone B is used to search column 122
  • phone B is used to search column 123
  • the retrieved authority column 105 is used to search column 124 to obtain a record from table 120.
  • Phone B is used only when column 105's type indicates that it should be used.
  • Retrieved column 126 is then used in the same way as the remote RBT Authority access above.
  • the 'what-l-Heard' API required of a server 40 may also be implemented as a software apparatus on Server 30. I.e. the services hosted by a remote server 40 may instead be accessed by calling a software apparatus on server 30. It implements the 'APIs expected of server 40' and described above, taking phone numbers as input and returning 'what I heard' or 'raw' determinations.
  • this local service 40 is accomplished by the addition of a defined value 'Local-code' as a permissible value in table 100, column 102 (RBT- Authority).
  • the 'request' column, 105 may contain information on directing the system to a particular local software apparatus or it may contain values that are passed to some common local software apparatus.
  • Step 209 would decode 'local-code' and direct flow to this local software apparatus.
  • this feature is by no means exclusive to 'non-RBTs' (one could easily implement a full local RBT based 'what I heard' system as a local software apparatus), this feature, used in conjunction with the local phone number to carrier table (180) will enable display, using all the display methods in the system of a set of metadata based off of many phone A's calling into a single, known, phone B. (see our illustrative example below).
  • the record may be a raw record, in which case processing continues to step 21 1 , otherwise it is a direct 'what I heard' message and processing continues in step 219.
  • the process uses the record returned along with the rule pointed to by column 106 for this RBT authority to calculate the information needed.
  • the rule itself is stored in rule table 130.
  • the rule table 130 is an abstract table that contains a string of rules on interpreting a retrieved record. Rules are obtained in negotiations with RBT Authorities and the system cannot predict rules or the format of retrieved records that may be revealed in the future.
  • figure 560 shows some possible rules. The discussion on using these rules show how these illustrative rules may be applied to illustrative retrieved records in 500, 520, 530, 540 and 550. Sections below will illustrate the use of these rules. Each of these sections also define the data that would be extracted from the retrieved records and then passed to the display section starting in 215
  • the retrieved rule may also indicate that the retrieved record should be stored in table 120 as a cached value to be used to retrieve the record locally at another time. There are two methods of storing this record:
  • a raw RBT profile record contains RBT settings for a phone B. These settings are combined with a set of rules for interpreting the settings and Phone A to determine which if any RBT was played for Phone A. step 21 1 processes these.
  • a music RBT system plays a music track to various callers into phone B. These tracks don't have to strictly be music, but are in-fact some audio track.
  • the illustrative raw music profile and rule would have a retrieved RBT record in the form of 500 and would use a rule from table 130, record 1002.
  • a carrier supports four features for a music RBT: 1 ) a per-MDN feature where the owner of phone B can enter a number of Phone A's and assign a music track to each, say 'Let it Be' when 555-1212 calls and 'Love Shack' when 444-1212 calls. 2) a time of day feature where the owner of phone B can set say 'Sergeant Pepper' on Mon-Fri 9AM-5PM and 'Hey Jude' on Saturday morning 10AM-1 1 AM.
  • a line default which is a track to play if neither time-of day nor 'per-MDN' is set and 4) a carrier wide music track such as some classical music track to be played in none of 1 , 2 or 3 are set. All of these settings are stored in an RBT profile for phone B
  • 500 shows how the schema for how this data can be sent from carrier 40 or stored in table 120.
  • 501 is the section that describes the carrier wide default.
  • 502 contains a track in the form of 530, a jukebox in the form of 540 or an info RBT record in the form of 550. It has a carrier default RBT (i.e. empty not allowed).
  • Section 503 contains a line default 504 that is also a track, jukebox or info RBT, but in this case can also be empty.
  • Selection 505 contains a list of per-MDN RBTs. each item in the list is in the form 506 which contains the phone number(507) for the phone A MDN that may call phone B and (509) the track, jukebox or RBT.
  • Section 509 has the time of day rules. There can be a set of rules here as described above.
  • a rule item is shown in 510 which consist of two things, a time of day rule (51 1 ) and a track/jukebox/info item to be played if the rule is satisfied.
  • the rule 51 1 is in the form of days-of-the-week and time span (i.e. M-F 9AM-5PM).
  • Track (530), Jukebox (540) and info (550) are described in the Retrieving Metadata section below.
  • step 21 1 involves retrieving partial data during raw processing; in this case 21 1 would retrieve the track info from table 1 10 or via API from a remote RBT authority using a track-identification key stored in the raw record for each possible track.
  • Parameters to access the additional Metadata are encoded in the access values in columns 103 and 105.
  • Rules consist of two sections, an 'if-then-else' section and a On result' rule.
  • Illustrative rule 1002 is used to interpret our illustrative record 500. The rules in 1002 are applied in order. If one succeeds then processing continues to the result section.
  • Rule 1 (the first rule in 1002) (rule 1 is shown in 560-1 ) If time of day set(51 1 ) in raw profile 500 and the current time fits in this range, then use the track (or jukebox or info track) indicated in the raw record(512). If not 2) Rule 2 (560-2) check if phone A is in list of caller IDs list (505/507), if it is use track, juke or info 508. If not, 3) Rule 3 (560-3) if the user has a 'default RBT set (503) use track, juke or info 504. Else 4) rule 4 (560-4) says use the default AT&T ring-back in 502 (which is never empty).
  • Rule 5 states; however, if this all returns a jukebox, return the jukebox, but act as if there were in fact no match. (This will force the next RBT Authority).
  • Rule 6 (560-10) states that on a successful RBT determination, store the retrieved record (cache it) in table 120.
  • the system may directly return a 'what I heard/played' from various sources: from an event in step 203, from a local or remote RBT authority in 209 or from an audio match in 214.
  • 219 is executed to extract 'what I heard' from the message and to obtain the information needed for the display section.
  • a music 'what I heard' includes the track info or points to the track info that is stored at a remote RBT authority or in a local store such as in table 1 10.
  • the retrieved record from this RBT Authority would be in the schema of 520.
  • the value in 521 would be a track (a track object 530), a trackID value or a juke (jukebox object 540).
  • Track 530 and jukebox 540 are described in Metadata below. Some or all of this data may be present.
  • Step 219 takes this record and is then processed using the illustrative rules 1001 , 1003 or 1005 from table 130 to pass values to the display section.
  • Rule 1001 step 8 indicates that the retrieved record would be in what I heard schema 500 and that it should expect and extract a trackID from the record that is then used to retrieve Metadata information from a remote RBT authority via an API.
  • Rule 1001 indicates to apply rule 9 once the data is collected and store the results in table 120 using both phone A and Phone B.
  • Rule 1005 would then be used to retrieve this cached value from that table. (The item stored now has all Metadata so 1005 indicates it should expect all data)
  • Rule 1003 is an illustrative example of an audio match on a jukebox.
  • the audio match section would return a 'what I heard' RBT.
  • Rule 7 indicates that the returned data has a trackID and that the Metadata for it be retrieved from the local 1 10 Metadata table, rule 1001 then indicates to apply rule 9 once the data is collected and store the results in table 120 using both phone A and Phone B.
  • Rule 1 1 is special for an audio RBT jukebox match (and is the complimentary rule to a rule 5 that invoked audio match from rule 1002). This rule indicates what to do on success or failure of the audio match.
  • On success rule 7 is applied to the success value (track ID then local Metadata), however, on failure, we return the jukebox that was passed in to the audio section to indicate 'one of these RBTs were played, we don't know which'
  • Rule 1005 indicates that the retrieved RBT record would be in the schema 520, however it should expect that the track info in the format 530 is there or that it contains a jukebox in the format of 540.
  • This rule is the complement of rule 1001 ; as a result there is no 'store in table 120, as this would result in the record being stored twice in 120.
  • a 'what I heard' record for an advertisement that is played during RBT time would contain a title of the advertisement, an image of the advertisement designed to fit into the display (akin to 'album art') and a URL to an HTML5 snippet that is displayed when an 'interact' button is pressed.
  • This is an example of an 'enhanced-RBT' determination where the data returned in the determination and resulting display don't limit themselves to what was heard or played but rather displays metadata possibly tangentially related to the RBT, in this case a follow-on advertisement to the advertisement RBT.
  • this type of RBT may take a GPS value passed from the phone to determine a particular display to be shown based on geographic information.
  • An advertisement 'what I heard' is similar to a music 'what I heard' except that the retrieved record 520 would contain an info item in the form 550 or a info ID that then points to local or remote Metadata in the form of 550.
  • the info data 1 field, 553 would contain the URL for the advertising image and info data 2 field (554) contains the URL for the interact HTML5.
  • An advertisement would return 1 ) an advertising title, 2) an advertising image and 3) a URL to an HTML5 'interact' code snippet.
  • Type' advertisement is also passed to the display section.
  • RBT is determined for 'T-Mobile' (column 102 set to T-mobile' in this example)
  • Rule 1005 is used to parse the return.
  • the return is contained in a 'what I heard' determination with all data included.
  • the returned data will include the name of the company, a URL to an image and a pointer to an HTML5 snippet that is displayed when an 'interact' button is pressed.
  • Ad-server-1 would return an advertisement relative to phone A, phone B, direction and GPS in a 520 data format.
  • the retrieved record 520 would contain an info item in the form 550 or an info ID that then points to local or remote Metadata in the form of 550.
  • Info title 552 would contain the name of the company being advertised,
  • info data 1 field, 553 would contain the URL for the advertising image and info data 2 field (554) contains the URL for the interact HTML5
  • the carrier in this case T-Mobile, may contract the processing of RBT- advertisements to a third party, and this server may be administered by that service. In one variation, this redirection to the third party server is performed by server 40 and is out of the scope of this disclosure.
  • phone numbers within the carrier that have RBT-advertisement installed are download into table 180, the non-carrier table to direct any determinations for one of those phone numbers directly to the third party advertisement server.
  • Figure 2010 shows an example embodiment of this illustrative example.
  • a 'what I heard' can be a corporate RBT. This RBT would be played on lines of phones owned by the Acme Company. If a phone a calls into a phone B from say 9-5 on Monday through Friday, an RBT would be an audio track that might say ' thank you for calling one of our employees. After the call you will be asked to rate this employee'. After the call you will see an image of the employee. The 'interact' button would point to an HTML5 code snippet that would allow you to rate his or her performance.
  • rule 1001 is similar to that of music or advertisement; however Phone B would be provided to the API so that the RBT Authority can customize the returned data in the schema of 550 with title, image and URL for employee who owned phone B
  • the system may be configured to return 'what I heard' when no RBT was in-fact heard. This is because the 'what I heard' record determination model merely attempts to 'mirror' the RBT played by network 10 or 20 by doing a shadow calculation of the RBT that could be played. This record based calculation can return any record related to this Phone A, phone B and call direction triplet. As an illustrative example, an advertisement for the company you just called could 'pop-up' when the call complete however you did not hear an RBT. The RBT Authority would be set up to use phone-A, Phone-B and call direction to return a relevant 'what I heard' return value that is a targeted advertisement for those 3 values.
  • This no-RBT case would use the event (202A, B, C and D) and display procedures in the system along with a 'what I heard' determination that is not related to an RBT played 217A through 217J.
  • a no-RBT determination is in-place-of and not 'in-addition-to' an actual RBT.
  • non-RBT determinations are provided by non-carrier entities and thus phone numbers for these non-RBT determinations are stored in the non-carrier table 180 resulting in processing of the determination at a non-Carrier RBT authority.
  • non-RBT determinations are done at a carrier in which case the decision to return an RBT, enhanced-RBT or non-RBT determination is out of the scope of this disclosure.
  • Step 203A searches table 180 for a match on the phone number in column 181 . If this had been a call 'to' TWC, the 'what I heard' software apparatus on phone A would use Phone B's number as a key. Had TWC called you, the software apparatus on phone B would use phone A, TWC's number that would appear as the 'incoming call from' number to as a key.
  • the non-carrier value is then used as a key to table 100, column 102 to retrieve the RBT Authority record 1 1 .
  • TWC server TWC-4 would query their PBX using the passed phone A or Phone B to find the agent info, including the agent's name, photo and return contact information. The internal to TWC-4 query to the TWC PBX that determines this information is out of the scope of this disclosure.
  • TWC-4 returns this determination.
  • the retrieved record 520 would contain an info item in the form of 550.
  • Info title 553 would contain the words Time Warner Cable and the contact name, info data 1 (553) would a URL to the agent's image, and info data 2 (554), then call back information
  • Figure 2008 shows an example embodiment of this illustrative example.
  • Non-RBT determination involves the use of table 180 and a local-code RBT authority record to display a particular experience based on any phone A calling a particular phone B.
  • a person on phone A calls a voting line for, as an example American Idol to vote for Jennifer Hudson.
  • the call completes information on American Idol and on Jennifer Hudson is displayed and the user has the option of downloading the song that Jennifer Hudson just sang.
  • This no-RBT case would use the event (202A, B, C and D) and display procedures in the system along with a 'what I heard' determination that is not related to an RBT played 217A through 217J.
  • a no-RBT determination is in-place-of and not 'in-addition-to' an actual RBT.
  • Step 203A searches table 180 for a match on the phone number in column 181 . This matches the second record in table 180, returning 'carrier' 'American Idol' from column 182
  • This local software apparatus returns this determination.
  • the retrieved record 520 would contain an info item in the form of 550.
  • Info title 553 would contain the words Time Warner Cable and the contact name, info data 1 (553) would a URL to the agent's image, and info data 2 (554), then call back information
  • Item 2009 shows an embodiment of this illustrative example.
  • the system supports audio matching of a clip recorded on phone A that was recorded from the time the call started to the time the call was 'picked up' against a collection of clips.
  • the audio matching service is treated as an RBT Authority. It may be used as the primary authority for a carrier or it may be used as a back-up authority should another RBT return no match or 'ambiguous' match. This later case of ambiguous match is illustratively a jukebox, but if could be other small sets of possible audio files.
  • the system supports two types of generation of the collection of clips for which to match against: 1 ) offline or 2) 'on-the-fly'.
  • offline feature extraction of the reference clips occurs at some time before the match is attempted.
  • the system takes a list of reference clips and extracts their features in real time, just before it attempts to match against the just-recorded clip.
  • Audio matching depends on its setup as an authority and whether phone A supports the required events to start and stop a recording and that supports the actual recording of the incoming audio stream from the time the user starts the call to the time the call is 'picked up'. Steps 201 A, 201 B and 201 C are only performed if all this is true.
  • step 201 A the app registers to receive a 'call placed' event. On receiving this event it start recording the incoming audio stream in step 201 B. The app also registered for receiving the 'call connect' event. On receiving this event in step 201 C the app stops the recording in 201 D that was started in step 201 B. The app then stores this recorded clip for possible use. It is possible, should there be no audio RBT authority, that this clip is not used in the 'what I heard process'. In all cases it is discarded after 'what I heard' processing occurs.
  • Processing is directed to step 213 when 208 sees an RBT Authority record with the RBT Authority, column 103 set to local-audio' to use local audio processing, or column 103 indicates a server 50 to contact and the request in column 105 indicates that audio processing is performed. Both remote and local proceed as follows:
  • Step 213 is entered with 1 ) a recorded clip 2) the carrier for phone B and 3) the jukebox calculated from the previous RBT authority.
  • RBT authority column 105 the request column, with an indication on how to process this list.
  • values in this column encode the following: a) collection size to use: values include 'carrier', which indicates that the match should occur against clips marked with 'carriers' to indicate a match of just carrier or 'jukebox' to indicate that the list should match against carrier and the records in 140 whose 'search unique ID' are equal to the trackID in the passed RBT list. And b) a flag indicating if 'offline' or 'on-the fly' is to be used.
  • step 213 extracts features from the reference clips before matching. If at the same time, 'carrier' is indicated, records from 140 are selected where column 142 is the passed carrier and the value in signatures is a URL to a reference clip instead of the signature itself. This becomes the list of on- the-fly clips. If at the same time 'jukebox' is indicted, the passed jukebox list has potential tracks in it. Each of these tracks contains 536, the URL of the track itself. Step 213 then takes the list of on-the-fly clips, and uses the audio matching service to produce a list of potential signatures
  • Step 213 now prepares the recorded clip for match.
  • Many carriers add a 'please enjoy the music while your party is reached' audio clip to the voice stream.
  • a recorded clip during a phone's ringing period would include this (901 ).
  • This clip is also administrable by the user and may or may not be played, for each particular carrier we make a copy of a truncated piece of the recorded clip containing only the number of seconds it takes to play the carrier phrase and remove anything after that (906) we then match 906 to a pre-analyzed clip of the phrase in 905 If there is a match we produce a clip without the phrase 903.
  • the 'please enjoy' phrase may have a so called 'audio water mark'.
  • Metadata in the system is any of the information associated with a 'what I heard/played' determination. In one embodiment it also contains the actual RBT played in the form of a playable audio track. Metadata is passed to the display section. It is a different set of data for different types of 'what I Heard' determinations such as music, ad or corporate above. However it is treated generically, except for a few exceptions noted throughout until it is actually displayed.
  • Metadata can be 1 ) part of a 'what I heard' determination. This includes notification events in 202B, in 'what I heard' determinations or raw record determinations returned from server 40 or contained in records stored in 120. (these records in 120 may in turn have come from 202B or server 40, or through other processes not described herein, 2) be explicitly requested via an API request to server 40 by server 30 passing in an ID and getting Metadata on the return 3) be contained in table 1 10 and searched for by a data type column 1 12 and ID column 1 13, 4) be returned by server 30 in an API request for Metadata from a phone A or B (which in turn may call RBT Authority 40)
  • Metadata is an abstract collection of bytes that is then extracted from its source by using a schema to parse the data. This collection of bytes may be returned in API calls or it may be stored as a collection of bytes in 120. After a parse it then stored in a form, such as an array, structure or object that can be used by the display section. We will simply refer to the fields in these items and not discuss its specific implementation. A 'type' is also returned to the display section, illustrative examples include 'music track', 'advertisement', or 'corporate RBT'.
  • 530 is an illustrative example of a music track
  • fields include Track ID(531 ) a number that uniquely identifies the track in an external catalog to be used by function (described in inventor's concurrently filed application, entitled, "METHOD FOR SENDING AND RECEIVING METADATA CONTAINING REFERENCES TO DIGITAL MEDIA OR CONSUMER GOODS BETWEEN CALLING PARTIES," inventor: Martin Gindi, attorney docket no. 82227.8002.
  • Track Name (532) a text string with the name of the identified track, Track artist(533), a text string with the name of the artist of the identified track, Track album(534), a text string with the name of the album, Track Album art or URL(535) the actual image in-lined with the data or a URL pointing to an image associated with the album associated with this track.
  • Track Clip or URL (536) the actual audio clip or a URL to the track that was just identified.
  • 550 is an illustrative example of a more generic form of the Metadata that can be displayed on a 'what I heard/played'. It is used for such things as the corporate RBT described above or the advertising RBT, also described above', fields include info ID (551 ) a number that uniquely identifies the info to some in an external entity to be used by function, Info Title (552) a text string describing what was heard, generic data in 553 and 554 (there could be more of these fields as needed by the type of thing heard.)
  • info ID a number that uniquely identifies the info to some in an external entity to be used by function
  • Info Title 552
  • 553 a text string describing what was heard
  • generic data in 553 and 554 (there could be more of these fields as needed by the type of thing heard.)
  • 553 is a URL to an image for this ad
  • 554 is a URL to an HTML5 snippet.
  • 540 is an illustrative example of a jukebox.
  • a jukebox may be any collection of individual tracks one of which may have been heard. This collection illustratively consists of tracks in the form of 530 or 550.
  • fields include Jukebox ID (541 ) a number that uniquely identifies the jukebox to some in an external entity to be used by function, Jukebox Title (542) a text string describing this jukebox, 543 is a list of individual items in this jukebox, 544 is an individual item in the list and 545 is the data in the form of 530 or 550. There may be many 544's and 545's in the list. The title and ID are passed along with an array or some other type of list of the values in 554 and 545 to the display section once they are parsed.
  • Metadata is collected as needed in step 21 1 , when the raw record doesn't contain enough information, and step 219 when a 'what I heard' determination doesn't contain enough information.
  • the special 'local-enough' value is used to possibly further process full or partial 'what I Heard' indications returned during the 'event' phase (all the steps 202).
  • the request and rule in the RBT authority indicate what to do with these indications.
  • the metadata, or the rules associated with a particular metadata return may include reference to an action that can be performed as a result of this 'what I heard' determination.
  • RBT determinations may include actions such as 'rate', purchase full track, recommend a new one and so on.
  • a determination for American Idol voting line may include the action to get the track just sung by the person you voted for or have a link to americanidol.com
  • the return of track-info, 530 schema used in conjunction with rule 6 in table 560 that 'rate, recommend, buy full track' are displayed.
  • various rules such as rule 14 in table , associated with an info scheme 550 return includes two buttons that have html actions and words to put on the button.
  • html5 return values described throughout this disclosure may contain these actions buttons and code for the actions themselves.
  • the current disclosure is responsible for the display of the action associated with the 'what I heard' determination and to store relevant information that may have come with the button when the button is pressed.
  • the actual actions such as rating, recommend, get the American idol song are out of the scope of this disclosure.
  • Execution of 'what I heard', step 203 through 214 and 219 are designed to be highly distributed. As illustrative examples of combinations that can occur; all these steps may occur on server 30, all these steps may occur on a phone A or phone b, some of the steps on phone A/B and some on server 30, some on phone A/B with API requests of 80 and 40, or server 30 incorporated in server 30, with server 40 providing services to phones A/B.
  • Server 30 can act as a 'what I heard' service to third party entities 95. Audio processing could occur on server 50 or on phone A/B. server 50 may reside on server 30.
  • Servers 30, 40 and 50 include or are expected to provide API services to the other entities in order to effect this distribution.
  • server 30 provides APIs that may be accessed by Phone A or B, server 40, third parties 95 or network 10 or 20. These APIs include 1 ) what-l-Heard/played and 2) get meta-data.
  • the 'what I heard/played' takes as input Phone A, Phone B and optionally the recorded clip from steps 201 A to 201 D. It returns 'what I heard/played', an error or partial success for a jukebox, indicating that the calling entity could do audio processing if it so desired. Additionally this API may optionally take 'carrier' (for when it is called from server 40 or 95). Server 30 provides this API service by processing all the steps 203 through 214 and 219 using the appropriate defined RBT authorities for phone B, contacting server 40, 50 and 80 as needed.
  • 'Get Metadata' provides Metadata as defined above and used in steps 21 1 and 219 when those steps are executed on phone A or Phone B.
  • phone A/B would call this service whenever it only has the title or ID of a 'what I heard' to retrieve the remaining Metadata needed and the Metadata does not reside on phone A or B.
  • 'get Metadata' takes as input any or all of the following; a 'date type' (e.g. track or info,), an ID, and/or partially retrieved information such as 'title' or 'advertising company'. It returns Metadata associated with the request.
  • Server 30 searches table 1 10 using data type and ID and sometimes accessing the data itself to search for matches on title. It may also search table 120 for matches on the data. Additionally server 30 may pass the 'get Metadata' call on to server 40.
  • 'get Metadata' may be called from server 40, 50 or third party 95.
  • the 'what I heard' API to server 30 supports phone numbers passed in 'global dial plan' format or a string or binary value. See 'phone numbers' section below for details.
  • the following are implemented as APIs on server 30 if business rules require them. These are all related to various global phone number verification schemes. 1 )'verify request' takes as input a global phone number. It stores the global phone number and sends a verification text message to that phone number via text service 97. 2) 'Verify check' is called at a later date to return the verification status of a passed global phone number. It returns the status, 'pending' or 'verified' to the user. 3) 'Store unique/phone pair'. Stores the pair of unique number and global phone number. 4) 'Get phone via unique' takes a unique number and returns the stored global phone number.
  • a GPS value usually containing the location of the current handset may be included in the API call to server 30. This value may be used in rules and request values for a local RBT authority or for passing on this value to a server 403. b.ii.5.f. APIs on-the-phone
  • the 'what I heard' service that is resident on the phone may be accessed by third party apps on the phone (on phone versions of 95) in a number of ways.
  • Binary 'what I heard' service here, steps 204, 203A, 205 through 214, 219 and the return 217G are compiled into a binary deliverable for each type of phone OS that is supported by the service.
  • the third party app would then include this binary deliverable in their distribution.
  • the third party will be provided with a specification on the use of this API, values to pass and parameters to expect on return.
  • the API accepts Phone A and Phone B as input and optionally a recorded clip of the RBT heard. It returns 'what I heard' display Metadata, or an error.
  • a third party app may pick up 'what I heard' display Metadata from the output of 220B, the call log, and 220C, the address book.
  • the setup and use of these data stores is dependent on the phone. Additionally, depending on phone capabilities, 220A or 220B may not be implemented.
  • server 40 provides all the APIs described in 30 above.
  • Server 40 RBT authorities may not accept global dial plan phone numbers, instead requiring a string or binary value.
  • Server 30 supports a one-to-one translation scheme to translate these before sending. (See notes on phone numbers below)
  • server 50 When steps 213 and 214 are executed on server 50, server 50 provides a single 'Audio-ID' API to server 40 or to phones A and B and in other variations to server 40 and third party 95.
  • the 'Audio ID' API takes as input all or some of; 1 ) the clip to be ID'd, 2) the jukebox being searched, 3) carrier, 4) collection size (carrier or Juke), 5) offline or on-the-fly indication, 6) please calculate Metadata.
  • Input 1 thorough 5 relate directly to values discussed above in the audio ID section, step 6 optionally forces step 219 to be performed.
  • server 50 contacts server 30 for Metadata before returning results.
  • the 'Audio ID API' returns 1 ) data type and ID of ID'd track or 2) the Metadata for the track or 3) no id made. 3.b.ii.5.d. APIs expected of networks 10 and 20
  • network 10 or network 20 may provide lists of call logs that access tables 10A or 20A, the list of calls, incoming, outgoing and missed as recorded by the network.
  • the API illustratively it takes as input a phone number, and returns recent phone calls. It may implement this as one call for each type of call action, i.e. incoming to this phone number, outgoing from this phone number or missed calls to this phone number, or it may return all types at once. It may allow for 'all calls' to be returned or just calls since the last time the app was called.
  • server 40 As part of the processing distribution in the system, certain APIs may be needed from server 40 to server 30 or phone A/B. These server 40's are operated by differing entities worldwide. The APIs provided, if any, by these server 40s and thus the details of these APIs can only be described in illustrative fashion. As described above, server 30 as well as phone A/B may provide 'what I heard' service without calling server 40 at all.
  • Raw record this API takes as input a phone B and returns a raw RBT record that is processed in step 21 1 .
  • This returned record may contain Metadata for the various RBTs or jukeboxes within the raw record or it may provide ID codes that need to be resolved either on server 30 or phone A/B or by an additional API call to server 40.
  • What_i_heard this takes as input phone A and Phone B and directly returns 'what I heard'. This returned record may contain Metadata for this 'what I heard' or it may provide ID codes that need to be resolved either on server 30 or phone A/B or by an additional API call to server 40.
  • Get_Meta_Data API service provided by server 40 similar to that described above with server 30.
  • Server 30 may pass the request to its API on to server 40.
  • All of these API returns from server 40 may be cached in table 120 or 1 10 on server 30 and in some variations on phones A and B.
  • a server 40 API may also accept a GPS value for the server 40 to use in determining 'what I heard' or as a parameter in retrieving a raw record.
  • Service 60 provides the ability to send asynchronous notifications to phone A and B.
  • the API provides all or some of the following: 1 ) the phone a way of registering and sending its own phone number along with the registration, 2) callback or polled API from a server to retrieve the registration event with the phone number 3) an API that the server may call to send a notification.
  • the specific format of these values and the exact interaction with this API is defined by the notification service provider. The system will adapt accordingly to use these values.
  • Service 80 provides phone number to carrier information to server 30 or to phone A or phone B.
  • the API takes as input a phone B phone number and returns a carrier name suitable for matching column 102 in the RBT Authority table.
  • the specific format of these values and the exact interaction with this API is defined by the phone number to carrier information service provider. The system will adapt accordingly to use these values.
  • Service 90 provides 'has app' service. This API and service is described in detail below in 'When Other Side does not have app'
  • Service 97 provides text message delivery service for server 30 directly or through the 'has app' service 90.
  • the API takes a destination phone number and text for a text message.
  • the specific format of these values and the exact interaction with this API is defined by the text message delivery service provider. The system will adapt accordingly to use these values.
  • RBT Authority there may be a RBT Authority on phone A or B and another on server 30.
  • An illustrative example of this would have the phone's RBT Authority table indicating that it should call the RBT authority API on server 30 with a 'What I heard' request. Then the RBT Authority table on server 30 would indicate that it would first call server 40 with a raw request, server 30 would do this, calculating the actual RBT played from the record it gets from 40, returning a 'what I heard' to phone A or B
  • Steps 202A, B, C and D start on the phone. Depending on a phone's capability all or some of these may be implemented. Each of these has certain data available to them and each would then require different things of the 'what I heard' system,
  • Step 202A call complete events have phone A and phone B and thus need to call a full 'what I heard' service.
  • Step 203 is executed on the phone and always directs flow downward though 203A (203A may or may not be implemented on the phone, so it proceeds to 205.)
  • An Audio server 50 may be implemented on the phone.
  • a table 140 may be installed on the phone or it may implement an 'on-the-fly' only version of the service.
  • the user may switch display methods manually, as an illustrative example, the widget may display a small part of the last 'what I heard' and some action of the user may cause the 'normal phone app' display to show more of it.
  • the display section has the title and image associated with the overall jukebox. It also has the track information (title, artist, album, and image) of each constituent track, as a general mater it can display this information 3 ways; 1 ) display the Jukebox title and image, 2) display the list of titles, artists, albums and album images, with words to the effect of 'you heard one of these', or 3) a 'button' element on the screen, usually in conjunction with #1 to further display the constituent tracks.
  • track information title, artist, album, and image
  • Info ID(551 ) along with the 'carrier' and 'RBT Authority' may uniquely identify this Ad or Corporate RBT. Additionally, the pointer URL stored in 553 or 554 may encode enough information that the originator can use to uniquely identify this track.
  • the image or the image URL is in the Metadata. If the image is includes it would be in line binary data in the Metadata. If it is a URL, the phone retrieves the image by accessing the URL included and using the results as an image. For a music track the image or URL is stored in 535, for a Jukebox it is the jukebox image in 546, for an ad or corporate RBT it is stored in 553 or 554.
  • the 'what I heard' app allows for various events to trigger 'what I heard' processing on the phone. These events 202A through 202D cause the display action 217A through 217F. Each of the sections below on each of these display actions will describe how the event and subsequent 'what I heard' processing would occur and how its completion would cause the 'what I heard' visual display to be updated with the correct information.
  • the system displays the 'what I heard' visual display in a passive screen location within a container on the phone's home screen. Alternately, this widget may also be included the screen space of another app (an 'in-app' widget'). Visually, the 'what I head' visual display in the widget is updated within a few seconds of call termination with the 'what I heard' visual display of the last call.
  • a home screen widget contains parameters that define the size the widget relative to the full screen size. It generally takes a fraction of the overall screen size. Some phones allow the user to place the widget wherever he or she desires at different spot on the screen. Others are positioned absolutely. Some systems allow them to be manually installed and deleted, others do not. In- app widgets are widget containers that are not displayed on the home screen, but rather in the container of another, third party app. this app controls location of the in- app widget within the app.
  • the operating system that the widget resides allows an event to be caught via a callback or event catcher or a poll to occur in order to update the 'what I heard' visual display, additionally, to allow interaction after 'what I heard' is displayed the operating system allows certain areas within the widget area assigned to the 'what I heard' app to be 'touched' for selection purposes or to allow some other manual means of selecting controls.
  • the call completion event registers for call completion events as described above. This call completion event may execute within the execution space of the widget or it may be a separate executable.
  • the call completion event 202A starts a thread (or it causes the main thread to execute) 'what I heard' processing. Once completed this asynchronous processing updates the area designated for the 'what I heard' visual display.
  • the widget service on the phone allows for URLs to be processed at some scheduled time. On receiving a 202A event, processing continue until the middle of step 209. In that step the URL to contact the remote RBT authority is calculated and passed to the widget synchronously processing handler for execution sometime in the future. The widget mechanism sends the URL and then notifies the app to continue in the middle of step 209 which processes the resulting 'what I heard' or raw response. The widget would then update the area designated for the 'what I heard' visual display.
  • Event 202B an external notification. As noted, these events have, some, all or no data included with them. In the case of 'no data', processing of step 202B described how the call log is scanned for new calls. This would result in 'some' of the Metadata being retrieved and processing proceeds as described below. This would be done within the widget's executable or separate executable as described in 202A. A 202B event may also have multiple phone calls included. Each is processed in turn. Alternately the call log API on networks 10 or 20 is called to scan for the same purpose.
  • Event 202C running an app, does not directly start processing in a widget and is not considered here.
  • Event 202D a periodic poll
  • the widget picks up 'what I heard' data from some external event or notification stored for its retrieval or it scans the call log for recent call.
  • the periodic poll is started within the widget execution space, when the poll is triggered, the widget process this the same way as 202B; if there was all the Metadata waiting for the widget when the poll occurs it is directly displayed. If there is partial data, it continues as per a 202A event.
  • Event 202F call progress protocol is not directly relevant to this display method (i.e. it does not trigger this display method directly), rather it stores 'what I heard' data in a known location that me be picked up by this display method.
  • Event 202G run of a call app, is not relevant here as it is only relevant to display method 2171
  • the call log, or latest calls may be retrieved via API from networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should they be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • This widget software apparatus stays active once installed on the phone by the user. It processes events 202A, 202B or 202D in a continuous loop using the execution mechanisms described herein. Depending on business rules, the widget may continue to display previous 'what I heard' determinations on the same widget area.
  • the widget allows transition to the normal phone app display mechanism 217E by doing a 'run app' 202C and may be triggered by an 'interact' button or some other button.
  • the widget process deposits the 'what I heard' Metadata result in a location that may be picked up on the execution of 202C.
  • it may send an interprocess event for any other process that may be listening indicating that this Metadata is available.
  • the Metadata itself may be part of those inter-process messages. On certain phones, this inter-process message may also start the 'what I heard' app. this is treated as a 'run app' (202C)
  • Figures 2001 and 2005 show illustrative embodiments of the widget display method:
  • What I heard widget (2001 ): Illustrative embodiment of the widget display method, 217A showing a real time display of 'what I heard'. This shows 'what I heard' Metadata including Album art, track title, artist name and album. This example also shows an interact button related to interacting with the current 'what I heard'. Also shows a display of past 'what I heard' and 'what I played' results.
  • What I Played Widget (2005) Illustrative embodiment of the widget display method, 217A showing a real time display of 'what I played'. This shows 'what I played' Metadata including Album art, track title, artist name and album. This example also shows an interact button related to interacting with the current 'what I played' .Also shows a display of past 'what I heard' and 'what I played' results.
  • the app pop up, one of the events 202A, 202 B or 202D causes an app to become visible on the main display of the phone. This app then shows 'what I heard' or played.
  • the pop-up may take up all of the display normally reserved for regular applications or it can pop-up over a small area of the display over the image that had been visible prior to the pop-up.
  • Figure 320 illustrates the pop-up.
  • figure 320 it is shown as sliding in from the bottom. This is one illustrative example. Illustratively it may also appear without a transition, slide in from the left or right, spin or fade in, or parts of it can slide in from various directions or it can rotate in. The specific transition is a business decision and it based on the phone's capability.
  • 217B is often available in conjunction with other display methods, (i.e., both a widget and a pop-up). Depending on business rules, 217B may be activated or deactivated by the user or continually activated without the ability to turn it off.
  • the app configures itself to allow the OS to activate it on receipt of a certain event as described below.
  • the call completion event registers for call completion events as described above.
  • This call completion event may be the event that activates the app or the call completion event may activate a separate executable.
  • the app spawns a separate thread to perform the 'what I heard' service. While this thread runs a visual 'processing' visual is shown (conventionally these are rotating arrows, but it may be many things). Alternately, 'what I heard' may run in the main thread. On completion of 'what I heard' the processing visual display is removed and the pop up would then update the area designated for the 'what I heard' visual display within the pop-up window.
  • the call completion event is sent to a separate executable
  • that separate executable may, depending on business rules and the phone's capability, after storing the phone numbers and call direction, immediately send the app activation event. Processing continues the same as the previous paragraph. Alternately, complete 'what I heard' processing may occur in the separate executable. If that is the case, once the separate executable completes and has collected all the Metadata for the 'what I heard', the Metadata is stored by this separate executable at a location that the pop-up may pick up, then the separate executable sends the activation event to the pop-up. The pop up would then update the area designated for the 'what I heard' visual display within the pop-up window.
  • Event 202B an external notification: As noted, these events have, some, all or no data included with them. In the case of 'no data', processing of step 202B described how the call log is scanned for new calls. This would result in 'some' of the Metadata being retrieved and processing proceeds as described below. This would be done within the pop-up's executable space or in the separate executable as described in 202A. A 202B may have multiple phone calls returned, in which case, each are processed in turn. Alternately the call log API on network 10 or 20 is called to scan for the same purpose
  • Event 202C running an app, does not directly start processing in a popup and is not considered here.
  • Event 202D a periodic poll, is processed in a separate executable to start.
  • the separate executable registers for a call-back at some interval.
  • This separate executable picks up 'what I heard' data from some external event or notification stored for its retrieval or it scans the call log for recent calls.
  • this separate executable may completely process 'what I heard', or it can pass information to the pop-up to continue execution. In either case it sends the activation event to the pop-up.
  • Event 202F call progress protocol is not directly relevant to this display method (i.e. it does not trigger this display method directly), rather it stores 'what I heard' data in a known location that me be picked up by this display method.
  • Event 202G run of a call app, is not relevant here as it is only relevant to display method 2171
  • the call log, or latest calls may be retrieved via API from networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should they be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • This pop-up software apparatus is meant to display the results of the current 'what I heard', and optionally some previous 'what I heard determinations' it is either dismissed by the user, or is caused to be dismissed depending on business rules by other events in the system such as a screen timeout or preemption by another app.
  • a version of this preemption occurs when a phone call is placed or received resulting in the dismissal of the current version of the pop-up. After the call is competed, another 202A, 202B or 202D would trigger a new pop-up.
  • the pop-up allows transition to the normal phone app display mechanism 217E by doing a 'run app' 202C and may be triggered by an 'interact' button or some other button.
  • the pop-up process deposits the 'what I heard' Metadata result in a location that may be picked up on the execution of 202C.
  • it may send an inter-process event for any other process that may be listening indicating that this Metadata is available.
  • the Metadata itself may be part of that inter-process message. On certain phones, this inter-process message may also start the 'what I heard' app. this is treated as a 'run app' (202C)
  • Figures 2000 and 2004 show illustrative embodiments of the pop-up display method:
  • What I heard pop-up (2000) Illustrative embodiment of the pop-up display method, 217B showing a real time display of 'what I heard'. This shows 'what I heard' Metadata including Album art, track title, artist name and album. This example also shows an interact button related to interacting with the current 'what I heard'. Also shows a display of past 'what I heard' and 'what I played' results.
  • What I Played pop-up (2004) Illustrative embodiment of the pop-up display method, 217B showing a real time display of 'what I played'. This shows 'what I played' Metadata including Album art, track title, artist name and album. . This examples also shows an interact button related to interacting with the current 'what I played'. Also shows a display of past 'what I heard' and 'what I played' results.
  • Displays 217C, Notification bar, 217D, Drag down notification and 217F and Numeric badges represent ways of displaying out-of-band messages to a phone user.
  • the implementation of these three types of displays in the what-l-heard app is similar and differs only in the final display step.
  • Figure 310 shows a 217C navigation bar and a pop-up alternate.
  • the notification bar may be used on phones that support a small amount of screen space, sometimes, the top of the phone to display small text and graphic messages. Alternately, some phones produce a small temporary pop-up that fades shortly after appearing. These two methods are equivalent.
  • the 'what I heard/played' message is displayed here.
  • this bar or popup is not actionable, that is, the larger app to see more of the information cannot be launched from the navigation bar, instead, they require the user to go to the drag- down display (217D). Should a phone however support direct app launch from this area the procedure described here supports it.
  • Figure 330 shows a 217D drag down notification.
  • this type of display 'what I heard' is seen when the phone owner takes an action to view a list of notifications on his or her phone. Often this is performed by a 'drag down' finger action, but it can in fact be other actions, illustratively including a button press, or navigation to a location and pressing select.
  • the 'what I heard' notification displayed here allows the larger application to be run to see more information or to interact with the RBT.
  • Figure 340 shows a 217F badge.
  • this display mechanism no direct information about 'what I heard' is displayed to the user. Instead, a small circle or other type of display is shown next to the icon on the phone that launches the 'what I heard' app. The icon's presence and count indicate that there have been 'heard RBTs' since last the app was run'. The phone's user knows this convention and launches the app to find what RBTs have arrived.
  • this section does not cover systems that display text, or graphics as notifications within the phone OS framework itself, later requiring the user to take an action to start the app... I.e. a system where notifications are displayed on the phone without being sent directly to the app if the app is not running.
  • the notification display of the OS displays the message without involving the 'what I heard' software apparatus.
  • the follow-on manual start from these types of external display notification is treated as a 'run app' 202C.
  • This section also does not cover OS badge systems that increase a badge value without calling the app to increase the badge.
  • 202A, 202B and 202D are embodied in these display types as a background process.
  • the OS is configured to execute this background process on receipt of any of these events.
  • the software apparatus is run manually or automatically at startup and it puts itself in the background.
  • the background process registers that it is waiting for 202A or 202B events, or it sets a timer to trigger a 202D at specified intervals.
  • the call completion event registers for call completion events as described above.
  • the background process retrieves phone A, Phone B and direction and starts execution of 'what I heard' in the background. Upon completion it displays the 'what I heard' visual display as described below.
  • Event 202B an external notification. As noted, these events have, some, all or no data included with them. In the case of 'no data', processing of step 202B described how the call log is scanned for new calls. This would result in 'some' of the Metadata being retrieved and processing proceeds as described below. This would be done within the background process. A 202B event may contain many phone calls, each are processed in turn. Alternately the call log API on networks 10 or 20 is scanned for the purpose.
  • Event 202C running an app, does not directly start processing in a widget and is not considered here.
  • Event 202D a periodic poll
  • the background process picks up 'what I heard' data from some external event or notification stored for its retrieval or it scans the call log for recent call.
  • the periodic poll is started within the background process' execution space, when the poll is triggered, the background process processes this the same way as 202B; if there was all the Metadata waiting for the background process when the poll occurs it displays the 'what I heard' visual display as described below. If there is partial data, the background process starts execution of 'what I heard'. Upon completion it displays the 'what I heard' visual display as described below.
  • Event 202F call progress protocol is not directly relevant to this display method (i.e. it does not trigger this display method directly), rather it stores 'what I heard' data in a known location that me be picked up by this display method.
  • Event 202G run of a call app, is not relevant here as it is only relevant to display method 2171
  • the call log, or latest calls may be retrieved via API from networks 10 or 20. These logs at a minimum have phone A, phone B and call direction and may contain more 'what I heard' data. If business rules allow it, and network 10 or 20 support this, then an API call is made to network 10 or 20 to retrieve call lists. Should they be retrieved, processing continues as if a local call log had been scanned as described in the previous paragraph.
  • a text string that is a proper subset of the illustrative display string is passed to the OS API that displays the information in the notification bar or as a 'pop-up' notification window.
  • the OS API that displays the information in the notification bar or as a 'pop-up' notification window.
  • a small icon may also be displayed.
  • this area or pop-up is selectable so the user may see more information.
  • a 217D display is also needed to allow the user to see more information.
  • the call to the OS that produced 217C also stores the same message in a 217D type display.
  • a drag-down notification a text string that is a proper subset of the illustrative display string is passed to the OS API that displays this information in the drag-down list. Selection of this 'what I heard' item in the drag down list allows the user to see more information.
  • the OS feature that displays badges is called. Depending on the phone, this feature takes either an absolute value or an increment command. In some variations the called activates the badge, in others; a non-zero value for the badge count automatically displays the badge.
  • the Metadata information needed for a badge is minimal. Depending on the phone and business rules, it need only be the fact that a call has occurred. Upon receipt of this information, if needed the 'what I heard' software apparatus activates the badge. If an incremental OS call is supported it is called, if a numeric value is needed the background process keeps a count, adds one to it, and passes it to the OS feature. Reading of the any of the 'what I heard' Metadata requires the app to be started with as 202C.
  • This background process software apparatus stays active once started; it processes events 202A, 202B or 202D in a continuous loop using the execution mechanisms described herein. If the process is stopped, then the system is incapable of displaying 217C, 217D or 217E displays
  • a 217E display starts with a 'run app' (202C), however, once 217E is active on the phone, 202A, 202B and 202D events may be sent to it and will update the 'what I heard' display accordingly.
  • 202C is started manually by the user from: 1 ) the phone's app selector menu, 2) the phone's notification bar, notification pop up (217C) or drag down notification (217D) 3) by requesting more information from the widget (217A) or popup (217B). 4) Startup by another app on the phone, possibly with startup parameters containing Phone A, Phone B and direction. It may alternately pass a contact record from the phone's address book.
  • the app then responds to 202A call completion or 202B external notifications. It may implement its own 202D periodic poll to check the call log, access the call log API on network 10 or 20 or to check for newly deposited Metadata. Additionally a manual 'refresh' button may check the call log or check for newly deposited Metadata.
  • the normal phone app display (217E) is executed as a foreground executable by the operating system.
  • 'what I heard' retrieval may proceed in a new thread.
  • event 202A, 202B and 202D may be executed in background threads.
  • All the initial activations of the 217E process may display a number of 'what I heard' items in a loop. There may be a number of deposited 'what I heard' Metadata items, a number of queued OS system notification and/or a number of new calls in the call log and/or calls from accessing the call log API on network 10 or 20. Each one would be processed as follows.
  • the process looks at an agreed upon storage location, an in-memory location or the contents of an app startup message. For queued system messages, the process indicates to the OS that it is processing stored OS notification events of a specific type. The OS then sends these stored OS events to the process. For a call log the process scans the incoming, outgoing and missed called lists for calls it has not processed. The process keeps a bit on each call pair to indicate it had previously processed this call pair.
  • the user may remain in the app and 'what I heard' displays will be automatically (or manually) updated by the following: 1 ) a call completion event 202A, 2) an external notification sent directly to this process (202B), 3) a periodic poll to pick up new call log information(202D for call log), 4) a periodic poll to pick up Metadata deposited by other processes (202D for Metadata scan), 5) manual 'refresh' button to perform both types of poll at this time 6) an inter-process event with Metadata or an indication that 'what I heard' Metadata may be picked up. 7) A periodic poll to pick up new call log information through the call log API to networks 10 or 20.
  • the call completion event registers for call completion events as described above.
  • the 'what I heard' app arranges for a call back or for event processing.
  • a thread is started to do 'what I heard' processing, or 'what I heard' processing is done in-line.
  • 'what I heard's visual displays are shown within the area allocated for them in the 'normal phone app' location [00519] 2)
  • the 'what I heard' app registers to receive the external notification events.
  • a 202B event may contain many phone calls, each are processed in turn.
  • call logs 1 A, 3A or 5A are scanned for new calls since the last check of the call-log.
  • a thread is started to do 'what I heard' processing using the retrieved Phone A, Phone B and direction, or this 'what I heard' processing is done in-line.
  • 'what I heard's visual displays are shown within the area allocated for them in the 'normal phone app' location.
  • Metadata On a periodic poll for deposited Metadata the depository is scanned. This Metadata may have come from 217A, 217B, 217C, 217D or 217F. For each deposited Metadata found, each may contain all the Metadata needed for the 'what I heard' display or will need to have additional 'what I heard' processing. If there is enough data as determined by the business rules each of these 'what I heard' visual displays are shown within the area allocated for them in the 'normal phone app' location.
  • the 'what I heard' app registers to receive an inter-process communication this IPC may originate from any of the background event processing in separate processes that are part of 217A, 217B, 217C, 217D and 217F.
  • This IPC may have full or partial Metadata contained within the IPC mechanism or it may signal that the 'what I heard' app should scan the 'what I heard' depository for what I heard Meta. Data in both the IPC itself and picked up Metadata may have enough 'what I heard' information or not. Processing on this data continues as per the 'polls' above.
  • Event 202F call progress protocol is not directly relevant to this display method (i.e. it does not trigger this display method directly), rather it stores 'what I heard' data in a known location that me be picked up by this display method.
  • Event 202G run of a call app, is not relevant here as it is only relevant to display method 2171
  • Figures 2002 and 2006 show the run-app display method with full graphics.
  • Figures 2003 and 2007 show the run-app display method with just text.
  • What I heard Graphic App (2002): Illustrative embodiment of the running app display method, 217E showing a real time graphical display of 'what I heard'. This shows 'what I heard' Metadata including Album art, track title, artist name and album.
  • What I Played Graphic App (2006) Illustrative embodiment of the running app display method, 217E showing a real time graphical display of 'what I played'. This shows 'what I played' Metadata including Album art, track title, artist name and album.
  • What I Played pop-up (2004) Illustrative embodiment of the pop-up display method, 217B showing a real time display of 'what I played'. This shows 'what I played' Metadata including Album art, track title, artist name and album. . This examples also shows an interact button related to interacting with the current 'what I played'. Also shows a display of past 'what I heard' and 'what I played' results.
  • What I Played Text app (2007) Illustrative embodiment of the running app display method, 217E showing a real time a 'just text' display of 'what I played'. This shows 'what I played' Metadata including track title, artist name and album.
  • step 202G the run phone call app event. Please see the description of 202G for starting this type of app.
  • Event 202F the call progress protocol event, or by the third party app acting like a third party app 95 by calling the 'what I heard' API.
  • An event 202F during call establishment would have deposited the collected 'what I heard' display Metadata at a known and accessible location.
  • the call app retrieves this information from that location.
  • the app may have registered for the broadcast event that event 202F may broadcast.
  • the obtained Metadata may have some or all display Metadata depending on what was transmitted during call establishment.
  • a call to 'what I heard' during call establishment may take two forms: a call to on-the-phone 'what I heard' binary API, or use the 'what I heard' API on server 30. in both cases, the third party app sends phone A and phone B values and is returned full 'what I heard' display Metadata.
  • Step 217J distributes the results of 'what I heard' determinations to other apps on this phone. They share this step 217J because they interface to the flow of the 'what I heard' software apparatus in the same way.
  • Step 217J may be implemented on the phone as a further step of the 'what I heard' determination embodied in the various display methods 217A, 217B, 217C, 217D, 217E and 217F.
  • Each of these descriptions explained various embodiments of the 'what I heard' determination.
  • 217J and its sub steps 220A, 220B and 220C may also be implemented by themselves without 217A, 217B, 217C, 217D, 217E, and 217F on phones that allow background processing and for which business rules allow it.
  • a background process implements the various background processes described in those display methods does not implement the final display step.
  • the background call processing described in the app-pop-up for call completion events would run, however the final 'pop-up' would not occur, instead 217J and one of its sub-steps would run. 3.c.vii.1 . broadcast and general data store
  • Step 220A uses a phone's data distribution model to distribute 'what I heard' display Metadata to other apps on the phone that may be interested in 'what I heard' information such as those that described in 2171, the call app.
  • These data distribution models allow one app on a phone to distribute data to another app on that same phone.
  • Inter-process broadcast type events that include the data:
  • various receiving apps register for a particular broadcast event.
  • the sending app registers that it is the source of these broadcast events.
  • the sending app places the data in a broadcast call.
  • the phone OS then arranges for all registered receivers to receive the broadcast event and to collect the distributed data.
  • As embodied in the 'what I heard' software apparatus on a phone it registers as the sending app and when 'what I heard' display Metadata is available it places it in a broadcast call.
  • Passive data stores that are polled In this type of distribution model the sending app places the data in a pre-arranged location. Sometimes the phone OS provides this common store in a structured manner, (e.g. as relational database or a data dictionary); others simply provide a storage location for which all applications have permissions to access. After the sending app places the data here, the receiver periodically checks for data stored here and accesses it if it sees it. As embodied in the 'what I heard' software apparatus on a phone, when 'what I heard' display Metadata is available it deposits that data in this store.
  • a structured manner e.g. as relational database or a data dictionary
  • Passive data stores that have 'data changed' events distributed This type of distribution is a combination of numbers 1 , 2 and 3 above.
  • the sending app places the data in the same passive data store as 3, then uses the broadcast or specific process directed events in 1 or 2, however it does not include the data in these events.
  • the sending app As embodied in the 'what I heard' software apparatus on a phone, it firsts deposits the data as in 3, and then sends events as described in 1 and 2
  • Private App A calls App B, passing it data:
  • the sending app starts (executes) the receiving app, passing it the data in the startup call.
  • 'what I heard' display Metadata places the data in the execution parameters and executes the pre-arranged app. It may also do this function when a 'what I heard' software apparatus user selects a particular action and reasonably expects to be directed to a different app.
  • the 'what I heard' software apparatus might display 'run this app' button on an advertising RBT. On pressing it this advertising app is run and the 'what I heard' determination is passed to it.
  • Step 220B Address book. This display method adds 'what I heard' and 'what I played' to the phone's address book seen by users and which may be accessed by other apps.
  • the address book displays the name, device type (mobile, home etc.) picture and other contact information associated with a phone number. Additionally, a 'call to action' is sometimes available for various registered phone call and other contact related additions to the call log.
  • Step 202C populates the results of 'what I heard' and 'what I played' into the one contact entry associated with the contact.
  • the contact entry for 'George' on a particular phone would read "George played 'Let It Be' for you (this is the 'what I heard') and 'you played 'Love Shack' for him when he calls' (this is the 'what I played').
  • the 'what I heard' service populates the contact list in a random access, random time manner. That is, as a user calls and is called from a contact the data is stored in the list (or updated as needed). This results in the information being spotty in the address book. If available it would also add a 'call to action' to start up the 'what I heard' software apparatus.
  • the phone OS provides a facility for third party apps to add this information to the address book and 2) 'what I heard' functionality is integrated by the phone owner when the phone is manufactured into the address book.
  • Phone provides facility.
  • the phone OS provides a way for a new contact information provider to register with the contact system.
  • the 'what I heard' software apparatus does this, also indicating that it should be called via a 202C with the contact record when the OS provided call to action for this contact provider is selected by the user.
  • the phone OS expects the provider of contact information to provide a call-back to return the information that it will display on a contact in the address book.
  • the 'what I heard' software apparatus stores 'what I heard' or 'what I played' determinations in table 170.
  • An insert inserts these three things, and update uses 171 and 173 as keys, replacing 172 if needed.
  • the 'what I heard' software apparatus provides a call back to provide the information to be displayed with this contact from the 'what I heard' system. This callback extracts the phone number for this contact and uses it to search column 171 . If one record is returned (an 'outgoing' or 'incoming/missed'), just that column 172 is returned as the information to display. If two records are returned, the two strings 172 are concatenated to produce a sentence about the incoming and outgoing determination.
  • [00569] 2 Integrated with the phone's manufacturer. The specifics of integrating with a particular phone manufacture cannot be determined, however broadly the integration points and methods are similar to the phone OS provided facility; 1 ) A means for the address book to call the 'what I heard' software apparatus on a call to action with a contact record, or with just phone B. 2) a means to store determinations into the address book using the phone number of the other side as a key or by a callback.
  • Step 220C Call log.
  • This display method adds 'what I heard' and 'what I played' to the phone's global call log seen by users and which may be accessed by other apps.
  • a call log is the list of incoming, outgoing and missed calls on a phone that usually show the phone number, name of the contact, picture, date and call direction. Furthermore, a 'call to action' is sometimes available for various registered phone call and contact related additions to the call log.
  • Step 220C adds 'what I heard' to this display on outgoing calls and 'what I played' on incoming and missed calls. If available it also adds a 'call to action' to start up the 'what I heard' software apparatus.
  • a call log display generally has two components, the call information itself and a reference to the contact from the phone's address book associated with this call.
  • the contact information is provided in 'address book' 220B facility described above.
  • the call log update may occur in two ways:
  • the 'has app' service, 90 is integrated into the 'what I heard' service to a) send 'what I heard/played' indications to phones that do not have the 'what I heard' software apparatus and b) to encourage owners of phones that are capable of downloading the software apparatus to do so. In both cases it does so by sending text messages or postings to third party social network sites.
  • the 'has app' service is optional in the flow of 'what I heard' and its use depends on business rules.
  • the 'has app' service 90 is treated as a black-box for our purposes, however the inputs, configurations and outputs needed for the service to perform the functions needed are described below.
  • the 'has app' service 215 executes in three variations. In one variation, 215 is integrated into the API 'what I heard' service running on server 30. In another variation it executes within the 'what I heard' process on phone 1 , 3, or 5. In a third variation it is used as part of the 'offline processing service', 202E on server 30. In all cases the 'has app' service 90 is called with certain inputs, configurations and outputs.
  • service 90 When service 90 is called from server 30, it executes in two modes: 1 ) 'service mode' when the 'what I heard' API provided by 30 is called as a service to other servers, 40 or 95 or from the offline processing service, 2) 'phone mode' when the 'what I heard' API provided by server 30 is called from phone 1 , 3 or 5. Both these modes will be described together below, with differences in the modes defined.
  • the inputs, configurations, and outputs provided the 'Has App' service 90 when called from 1 , 3 or 5 is the same as when called in 'phone mode' from server 30. These will be described only in the context of 'phone mode' from server 30.
  • the API call for both modes sent to server 30 has the following additional parameters over the normal 'what I heard' service is provided when business rules call for the 'has app' service. 1 ) The 'request has app service' bit set to true. Let us call this parameter 4-1 . 2) The human readable name for the owner of this phone. Let us call it 4-2. This name was obtained either from a known location on the phone or the 'what I heard' app asks for it at the time of first installation of the app. If it is empty, the phone number of the other side.
  • server 30 passes parameters 4-2, 4P-1 , 4P-2 and 4P-3 along with these additional parameters to service 90: 1 ) the other side's phone number as was provided in the 'what I heard' API. Let us call this 4P-2) Text strings for various alternate delivery methods for which the 'what I heard service may choose'.
  • Each of these text strings contain: words that, according to business rules contain 'what I heard' Metadata and this phone's number or name (4-2) and a URL pointer, illustrative variations on these alternates and the type of alternate 'what I heard message' that can be sent include; a) other side's device is capable of downloading a version of the 'what I heard app: a text message with a URL to an on-phone app catalog and a pointer to the app itself so the user can download the app. Call this 4T-1 . b) Other sides may receive text messages but can't have the app: a text message with URL pointing to web server 96. The URL encodes the title, artist, album of the track and the remote's phone number or 4-2. Call this 4T-2. c) Various strings, each appropriate to a social networking provider containing the similar URL to web server 96 or to pages within a social network. Call this 4T-3
  • the 'has app' service 90 will be configured with the following cascading logic: 1 ) if other side has app, do nothing and return 'has app'. 2) If the other side is capable of running the app and their carrier rules allow for text messages, send a text message 4T-1 . 3) Otherwise, if the remote can receive a text message and their carrier rules allow for text messages, send text message 4T- 2. Otherwise 4) if social networking credentials were supplied in 4P-1 send to the user on that social network site as supplied in 4P-2. These messages are sent in step 218.
  • Has app service 90 will return the following to step 215: 1 ) Remote has app already. Let us call this 4R-1 , 2) the other side does not have app but the service 90 has in some way sent a 'what I heard' message. Let us call this 4R-2, 3) the other side does not have the app and enclosed is a message that should manually be sent from the phone as just described. Let us call this 4R-3. 4) There is no way of getting a message to other side. Let us call this 4R-4. If 'has app' 90 has been called directly from a phone 1 , 3 or 5, return these values to the phone.
  • 'has app' is used in the following situations. 1 ) During 'offline processing 202E for both sides of the recorded call pair. 2) During 'what I heard/played' processing that originates from phones 1 , 3 or 5 for the 'other side' of call. (Phone A if called from phone B or phone B if called from phone A). 3) API call from 40 or 95 for both sides of the call pair
  • the 'offline list processing' started in step 202E and run on server 30, processes lists of phone call pairs between phones A and Phones B. It sends 'what I heard' messages to Phones A, and 'what I played' messages to phones B. it sends external notifications via 217H to phones that 'has app' and uses the 'has app' service to send alternate messages via 218.
  • a list of calls that have completed is provided to server 30.
  • Process 202E iterates through this list. Additionally the process selects first phone A, then phone B and performs its function on both in turn.
  • 202E performs one 'what I heard' indication for the pair, as the 'what I played' as seen on phone B is the same as 'what I heard' on phone A. To do this it does a 'what I heard' determination (outgoing call) with Phone A and Phone B as values.
  • the 'what I heard' service entered in 204 and proceeding with steps 203A, 205 through 214 and 219 proceed as described above, collecting 'what I heard' display visual Metadata for the pair if an RBT (or non-RBT type display) is indicated. If nothing is returned from the determination in 207, processing moves on to the next pair. This processing continues until there are no more pairs.
  • the list may already include some or all the 'what I heard' display Metadata of a predetermined 'what I heard' determination for the call pair. Should this be partial Metadata, the process enters at 203, sending control immediately to 219 to get more information.
  • 4P-1 and 4P-2 are null.
  • Force manual 4P- 3 is always false, that is it is an automatic send from server.
  • 4T-1 the text message when the other side 'can have' the app but doesn't is set as per above (without a name 4-2, instead the other side's phone number is inserted there).
  • 4T- 2 the text message when the other side 'can't have the app' is set per above (also with 4-2 being the other sides phone number).
  • 4T-3 social networking messages are set to null.
  • the 'has app' service is configured as above, the 'has app' service will send out a text message to the other side if it does not have the app and the carrier allows these messages and the phone allows text messages as described above. These messages are sent out via 218.
  • an external notification is sent from server 30 through notification service 60 to the phone being processed.
  • This notification contains as much Metadata retrieved from the 'what I heard' service as the notification can hold. It also contains human readable messages if the device type requires that, (if all the Metadata cannot fit, a track-ID) is provided so that the what I heard processing on the phone can pick up the Metadata via an API call to 30
  • 'Phone Number' has been used throughout to mean either a number within the global dialing plan or a private string or binary digits used for addressing.
  • this number is characterized as a string of digits and certain special characters such as '+', ' * ' or '#' that allow a phone user to place a phone call to another land line or mobile phone throughout the world.
  • a private string or binary digits it can be, illustratively, any random collection of characters such as an email address or string of hex digits.
  • This number may also be a special coded address special to a notification system 60
  • Phones that run the 'what I heard' software apparatus may not have access to the phone's own global phone number. If that is the case then the 'what I heard' software apparatus might have access to a unique private number that the phone OS assigns to this device, such as a serial number or other uniquely defined number or if it does not have any unique OS provided number, a unique number may be generated by the 'what I heard' software apparatus.
  • a unique private number that the phone OS assigns to this device, such as a serial number or other uniquely defined number or if it does not have any unique OS provided number, a unique number may be generated by the 'what I heard' software apparatus.
  • Many carriers' server 40 use global dialing plan numbers in the API's, however some may use private strings or binary digits and thus may require these on RBT Authority calls.
  • the 'what I heard' system operates in two distinct modes. In one mode, all calls are within the same network 10 i.e. from phone 1 or 2 to phone 3 or 4. In this scenario the 'phone number' for Phone A and Phone B throughout this document may be either a global dialing plan number or a private string or binary value.
  • the phone does not allow the 'what I head' software apparatus to obtain its own global phone number the software apparatus obtains it.
  • the following illustrative examples provide a means of obtaining this number, but there may be other methods.
  • server 30 implements a verification API that takes this number to be verified and stores it in 612 in the 'pending verification table' 150 with column 151 set to the proposed global phone number and column 152 set to 'pending'.
  • Server 30 then sends a text message to the global phone number in 613 to a text message service 97 in column 604.
  • the text message service forwards this message back to the phone in 614. Should the correct global phone number have been entered, the phone that sent the verification request in 61 1 will get this text message.
  • the user responds to this text in 614A and sends the text message to server 30.
  • Server 30 had arranged to have this text message sent to itself. Server 30, gets this 615 in column 603 and uses the 'from' global phone number in the text message to access table 151 .
  • the 'what I heard' software apparatus issues another API call to server 30 in 617 asking if verification has occurred, passing it the still unverified global phone number.
  • Server 30 gets this 617 in column 603.
  • server 30 implements this API by using the passed global phone number to access table 150 using column 151 as key, returning column 152 in 618. If 'verified' is returned, a 'verified' global phone number is saved on the phone. In this case whenever this phone's own number is needed this value is used. If 618 had not returned 'verified', every time 'this phone' number is needed, another attempt is made to call 617. In each return of not 'verified', the 'what I heard' service cannot be called with the unverified number and no 'what I heard' is shown.
  • the OS runs the app in 671 , starting the app in column 653.
  • the app puts in a unique ID into this URL before sending it, storing this unique ID within the app for later in 672.
  • the web page shown illustratively says ' thank you for downing the app', press OK to continue'.
  • the web server then sends a URL to the web server 98 in column 655 via 674. This URL has the unique ID and the cookie that stored the global phone number.
  • the web server then forwards unique ID and global phone number to server 30 in column 657 via 675.
  • the app calls an API implemented on server 30 via 678. This API sends the unique ID stored in 672.
  • the API returns the global phone number.
  • Column 657 receives the request; searches table 150 for the passed unique ID in column 153, fetching the global phone number from 151 . It sets the state, column 152 to 'picked up'.
  • the response is sent to the app in 679 (to column 657).
  • the app stores this global phone number in 680. The app will now use this number as its global phone number and use this value whenever 'this phone' global phone number is needed.
  • Carrier log in The user's global phone number is generally stored at the carrier. This can be obtained by calling a login API on server 40. The user is asked for userlD and password for their carrier account in the 'what I heard' software apparatus. These values are sent via API to server 40. Server 40 responds with a user record that contains a global phone number. The app will only allow log in to the carrier for which the app is written. (I.e. the AT&T version of the app will allow only log in to AT&T and not Verizon). The returned global phone number is stored. The app will now use this number as its global phone number and use this value whenever 'this phone' global phone number is needed.
  • the carrier may have a 'single-log-in' facility on the phone whereby a log-in to other carrier apps on the phone (or on the phone's browser) will allow the 'what I heard' software apparatus to access the customer record containing the 'global dialing plan phone number' without asking the 'what I heard' software apparatus' user to log in again.
  • server 30 normalizes passed strings or binary values to global dial plan numbers.
  • the 'what I heard' API into server 30 supports this by allowing, depending on business needs a 'type' value to be associated with both phone A and phone B.
  • a special value for 'type' is 'global dial plan phone number' this indicates to the system that the value is a global phone number.
  • Other values passed are arbitrary and matches the arbitrary values inserted into table 160.
  • step 203A checks both phone A and Phone B in turn to see if the type value is 'global dial plan phone number' . If it is, the carrier is looked up as normal with no further action. If it is other than 'global dial plan phone number', 'type' and the passed string/binary value are used to search table 160, columns 163 and 162 respectively. The returned 'global dial plan phone number' is now used as the value for phone A and phone B are now used throughout the remaining 'what I heard' steps (unless it is later translated to a string/binary value below).
  • This feature may be implemented on both server 30 and phones 1 , 3 and 5.
  • C Mechanism to translate global phone numbers to private strings or binary values on calls to RBT authorities
  • Server 40 RBT authorities may not accept global dial plan phone numbers instead requiring a string or binary value.
  • Server 30 supports a one-to-one translation scheme to translate these before sending.
  • the request column 105 of the RBT Authority table encodes a request that requires a translation. It also includes the 'type' to translate to. As an illustrative example it can be: 'translate_to_skype_email'. Before sending the request to the referenced RBT Authority it uses the 'global dial plan phone number' that was received (or translated to) in the 'what I heard' API and this type to search table 160 columns 161 and 163 respectively. The retrieved string/binary value from column 162 is passed to the RBT Authority.
  • This feature may be implemented on both server 30 and phones 1 , 3 and 5.
  • Third party call apps are applications on mobile phones that provide audio or video calling services but are not integrated into the phone's normal phone services. That is, there is no call completion events 202A provided on call progress and there is no centralized, known call log that can be scanned on a poll for 202D or for running app 202C to scan. Illustrative examples of this include VOIP apps that perform full call functions. Similarly certain phone OSs do not provide call completion or call log events or may not allow the 'what I heard' software apparatus to obtain this information.
  • the values are in 'global dialing plan' format, but they may be string/binary and type. 3) store these same values in a known location and send an inter-process message as described in the 'what I heard' service description above, 4) store these same values in a known location and have a poll 202D or running app 202C pick it up.
  • these methods could provide more information over phone A and phone B as described in the 'what I heard' API (such as track title etc.).
  • Voice Over IP services allow applications on mobile phones users to place audio calls and in some cases video calls over the IP networks.
  • the normal (default) phone services on the phone support the normal voice services in an integrated manner while the VOIP services are provided as separate apps.
  • the normal voice service is integrated into the phone by having easy access (usually one button) into the phone service while VOIP calling is generally a separate app run a separate way.
  • normal phone services may provide call completion events 202A, log calls into the phone-wide call log and address books.
  • the normal phone functions utilize the global dialing plan phone numbers while VOIP calls may use their own addressing method such as user names or email addresses in addition to or in place of the global dial plan numbers'.
  • VOIP networks here, a networks is a collection of a network and handsets and PCs that both support the same VOIP service
  • VOIP service may be self-contained networks or interconnected with traditional phone networks.
  • both endpoints to a call are the VOIP service, as such; phone A and Phone B's VOIP specific strings or binary addresses may consistently be used throughout without translation.
  • the VOIP service When VOIP networks are interconnected to traditional phone networks the VOIP service generally allows the 'global dialing plan phone number be use used throughout.
  • the VOIP service assigns a 'global dialing plan phone number' to the VOIP app.
  • a user on VOIP calls a phone on a traditional phone network, it uses that handset's 'global dialing plan phone number'. That incoming handset sometimes sees the assigned VOIP number as the incoming call and other times it sees a generic global dial plan phone number representing the VOIP network.
  • a handset on a traditional phone network uses the assigned-by-VOIP 'global dialing plan phone number' to call the VOIP user. This results in the VOIP app running on the called handset.
  • either the VOIP app provides both numbers in 'global dial plan' phone format in calls to 'what I heard' or there exists a translation in table 160 between its string/binary form and the 'global data plan phone number' form.
  • Notification services 60 forward notifications from some server to the phone. These notification servers use types of addresses. 'Global dialing plan phone number' and 'Unique ID'.
  • the phone, notification server and server sending the notification all agree on a unique ID.
  • the server sending the notification then uses the unique ID to send a notification to the app.
  • the unique ID version generally requires a registration to be sent from the handset to the notification service saying that it is interested in receiving notifications. This registration either assigns the unique ID from the notification service or is given the unique ID from the handset. To complete the registration process this unique ID is then sent from the handset (or forward from the notification server) to the server that is sending the notification.
  • the registration is sent to a server 30, it uses table 160 to store these values, putting the unique ID (and any credential blobs) in column 162 and the 'global dialing plan phone number' in column 161 . It sets type appropriate for the value, e.g. 'Apple Notification Service'. Later, when it wishes to send a notification on its own volition it searches the table using the 'global dialing plan notification' phone number to match column 161 and 'type' (column 163). Column 162 is then returned as the unique ID to use for the notification.
  • the 'what I heard' software apparatus is responsible for sending all the needed notification registrations for all expected notifications from any server that may send a notification. This is generally done at first app startup.
  • the raw RBT record for a phone B is unlikely to change on either 30 or 40 during the run of the software apparatus.
  • having the raw RBT record on the phone allows an RBT determination to occur local to the phone on subsequent incoming phone calls from any phone A without a retrieval to server 30 or 40.
  • an RBT Authority record on phone B that retrieves a raw record is modified to point to a rule that includes a 'store w/b' rule (similar to example RBT authority record 2, but on the phone).
  • This record would be Ordinal 2.
  • ordinal 1 below that tries to retrieve from the cache would fail as the record is not in the database).
  • An ordinal 1 record would, similar to RBT authority record 7 would also be on the phone that would, as the first rule, try to retrieve from the local table 120.
  • This cache may be kept either for the amount of time that the 'what I heard' software apparatus is resident in memory, or another method, such as time residing in table 120 may be used to purge this message. Additionally, other sections of the 'what I heard' software apparatus, will allow administration of the raw RBT record. This manipulation may also serve as a signal to invalidate the cache.
  • This illustrative example shows: 1 ) a notification from 40 with phone B in it is sent to a phone A via a notification service 60 2) a background notification processes this on a phone. 3) the phone contacts the 'what I heard' RBT API on server 30, 4) server 30 first sees if a corporate RBT is active on phone B by getting a raw corporate RBT Time-of-Day record from server 40, not finding that the current time is within the range it then 5) calls server 30 to get a raw RBT profile for the Phone B. This record has all needed Metadata. 6) Server B analyzes the raw record and assembles a 'what I heard' return with all needed Metadata. 7) Server 30 stores it in a cache for a later retrieval.
  • Time flow chart 1000 shows the flow of this illustrative example.
  • This example uses RBT Authority record 101 on the phone to indicate that a 'what I heard/played' is called on server TMI-I and that a 'what I heard/played' determination is returned with full Metadata.
  • Record 9 (which is an alternate ordinal 1 to carrier AT&T) on the server indicates that a raw time of day record is returned from the first server at the carrier.
  • Record 2 on the server indicates that a raw user record is returned form server 2. The contents of these records were described above with table 100
  • Column 1001 starts the process in 1020A.
  • Server 40 after call completion decides to send a notification to phone A.
  • phone A As an illustrative example here it includes just the phone number of phone B. It sends the notification to the notification service 60 (1020)
  • Step 1004 the background process receives 1022 in step 202B.
  • Phone side 'what I heard' processing is entered at 203. Since there is no 'what I heard' determination in the notification, only the phone number and call direction, processing at 203 passes to 'what I heard' processing in 205. It looks through the phone's RBT authority table in step 205 and finds record 101 indicating that it should contact server TMI-1 in step 209 and ask for 'what I heard' processing. It passes its phone number as phone A and the number received in the notification as phone B. It assembles this request in 1023 and sends it to server 30 in 1024.
  • Column 1006 receives this 'what I heard' request on server 30 in step 204. It first contacts the phone number to carrier service 80 in step 203A, sending request 1025 to it. Column 1007 receives this and sends the response back to 1006 in 1026. The determination in this case is that it is for AT&T.
  • Column 1008 which is AT&T server-1 (a server 40 type server), receives this request and using phone B that was passed and returns the raw corporate RBT record associated with B. it returns it through 1028.
  • AT&T server-1 a server 40 type server
  • 1028 is received in column 1006 on server 30 and processing continues through 210 to step 21 1 to do raw calculation.
  • This RBT authority (record 9), uses rule 1007.
  • Rule 1007 states that a TOD check only occur (and that the record is stored also.
  • the returned record from AT&T server-1 indicates that an RBT is played, as an illustrative example, from 9AM to 5PM Monday through Friday, it happens to be 6PM on Monday so, applying rule 1007 there is no RBT to be played and step 212 determines that 'no RBT' was played and processing loops back to 205 to search for RBT authority for AT&T of ordinal 2.
  • Step 209 contacts this server and sends 1029.
  • Column 1009 receives this 1029 on AT&T Server 2 (a server 40 type server).
  • AT&T server 2 searches for the raw record associated with phone B and returns this as the response in 1030.
  • Step 21 1 Column 1006, server 30 receives this response in step 210 which, seeing it raw proceeds to step 21 1 to do raw calculation.
  • This RBT authority (record 2), uses rule 1002.
  • Rule 1002 states that Time of Day, MDN, user default and carrier default are checked in that order. (And that the record be stored also)
  • the returned record from AT&T server-2 indicates that user had no TOD set and no MDNs set, but it did have a line default of say 'Let It Be' with the full Metadata). This results in 'let it be' being played for everyone, so 'Let It Be' is the determination.
  • Step 21 1 also stores this record returned from AT&T server-2 in table 120 (shown as 1031 ) with both phone A and Phone B stored. Server 30 proceeds through to 217G that sends the return value to phone A in 1032.
  • server 30 has 'what I heard' cached
  • Time flow chart 1 100 shows the flow of this illustrative example.
  • This example uses RBT Authority record 101 on the phone to indicate that a 'what I heard/played' is called on server TMI-1 and that a 'what I heard/played' determination is returned with full Metadata.
  • Record 5 on the server indicates that a cached 'what I heard' may be found in table 120. The contents of these records were described above with table 100
  • Column 1 101 starts the process in the phone's OS.
  • the RBT app had previously registered that it should be sent 'call completion events'.
  • call completion starts the process sending the event shown as 1 1 1 1 .
  • This is an outgoing call so it sends the 'outgoing call complete' event.
  • Column 1 102 the background process of the RBT app receives this call completion event and processes it starting at step 202A.
  • the first thing it does is to collect this phone's phone number at Phone A, and the phone number of the call that was just completed as phone B.
  • the business rule for this version of the RBT app allows for the 'what I heard' access to occur while the app is up, so the background process of the RBT app issues a broadcast event that causes the app to be visible.
  • Column 1 103 receives 1 1 14, the API request on server 30, starting step 204. It first contacts the phone number to carrier service 80 in step 203A, sending request 1 1 15 to it with 'phone B'. Column 1 104 receives this and sends the response back to 1 103 in 1 1 16. The determination in this case is that it is for Verizon.
  • the record indicates that a locally stored 'what I heard' should be searched in table 120 using both phone A and Phone B. We will assume that some recent request had stored the record matching Phone A and Phone B and that the search returns a record. This is shown in 1 1 17.
  • Step 210 determines that 'what I heard was included in the record so it jumps to 217G to return it.
  • Server 30 returns this record that has all the needed Metadata sending it in 1 1 18.
  • the RBT app on the phone receives 'what I heard' determination in column 1 102. It first stops and removes the rotating arrow in 1 1 19. Continuing 'what I heard' processing on the phone sees a 'what I heard' in 210 and proceeds to 217B to display the 'what I heard' Metadata results in the pop-up. This is shown in 1020.
  • This illustrative example shows: 1 ) the network equipment for the carrier of Phone A sending a notification to phone A with the track name and track ID (partial Metadata) via a notification service. 2) phone A only supports a notification center and does not send the notification to the RBT app if the app is not running, instead, putting it in the notification center 3) The user sees the notification and runs the app from the notification center which passes the notification data to the app. 4) the app, seeing it needs more Metadata, calls server 30 to get it. 5) Server 30 doesn't have the data so it calls server 40. 6) The data is returned through server 30 to the app that then updates the display in the running app with this data.
  • Time flow chart 1200 shows the flow of this illustrative example.
  • Network 10 sees the call completion in 1201 A and decides to send a notification to phone A.
  • Network 10 includes a human readable message that as an illustrative example says 'your friend just played 'Let It Be' for you.' it also contains the track-ID associated with 'let it be' so the phone can use it as a reference to retrieve all the needed Metadata. It sends the notification to the notification service 60 (1210)
  • Column 1203 receives the notification 121 1 on phone A's OS.
  • the 'what I heard' software apparatus previous to this time had registered with the OS that it is interested in receiving this type of notification.
  • This type of phone though does not forward notifications directly to apps (and thus start them automatically), instead, it displays the text 'your friend just played 'Let It Be' for you' in a special area on the phone designated for notifications (1212).
  • the phone associates this notification with the registered phone app so that when the user selects it, it starts the app, passing it the notification at startup.
  • the user selects this notification from the phone's notification area in 1213.
  • 1213 causes the app to start at 1214.
  • the phone OS starts the app at 202C.
  • the app runs in column 1204. It starts at 202C seeing the waiting actual notification (that was set up by the OS) and retrieves it.
  • Seeing a 'what I heard' determination with a track ID step 203 passes control to 219. 219 see that only the trackID was provided and searches the phone's RBT authority for the special record Track_info'. this retrieves record 105 from the RBT authority table 100 which indicates that server TMI-1 be accessed to get the track-info passing the 'track info' request from column 105. This request is sent via 1215.
  • Column 1205 receives 1215 on server 30, which is the special API request to retrieve just the track info. Server 30 knows that this information is not in its table 1 10 so it passes the request via 1216 [00659]
  • Column 1206 receives this 1216 track-info request on server 40. Server 40 retrieves this information and returns in via 1217
  • This illustrative example shows: 1 ) running the app directly 2) scanning the call log for both incoming and outgoing calls 3) the first call, an incoming one where this phone acts as a phone B, sends a 'what I played' RBT API on server 30 passing the incoming call as phone A and its own phone number as phone B, 4) Server 30 calls the server 40 servicing the carrier associated with phone B 5) server 40 returns a raw record for phone B to server 30. 6) Server 30 figures what would be played for phone A. 7) server 30 returns 'what I played' to the phone, 8) the phone displays this 'what I played'.
  • the second call is processed, an outgoing call where the phone acts as a phone A, sends a 'what I heard' request with the outgoing number as phone B and its own number as phone A.
  • Server 30 calls the server 40 servicing the carrier associated with this new phone B 1 1 ) server 40 returns a raw record for phone B to server 30. 12) server 30 figures what would be heard on phone A. 13) server 30 returns 'what I heard' to the phone, 14) the phone displays this second 'what I heard'.
  • Time flow chart 1300 shows the flow of this illustrative example.
  • This example uses RBT Authority record 101 on the phone to indicate that a 'what I heard/played' is called on server TMI-1 and that a 'what I heard/played' determination is returned with full Metadata.
  • Record 2 on the server modified for our purposes to be ordinal 1 , indicates a raw record is to be expected from AT&T, and record 4 on the server, indicating that it is a carrier aggregator that supports carriers Orange, 02 and Videophone and that it too returns a raw RBT profile.
  • the user decides to start the app from the phone's app menu in 1310, column 1301 .
  • the user decides to start the app.
  • the app is started in 202C.
  • 202C in this case scans the incoming and outgoing call logs for calls it has not processed since the last time it scanned the log. It will notice one incoming and one outgoing call in our illustrative example.
  • 131 1 it processes the first call, an incoming one, so the number in the call log is a phone A, and the phone that the app is on is a phone B for 'what I heard' purposes. It starts thread A as column 1302, passing it these phone numbers via 1312.
  • Thread A column 1302 starts 'what I played' processing. It passes 203 and 203A going to 205 where it looks through the phone's RBT authority table in step 205 and finds record 101 indicating that it should contact server TMI-1 in step 209 and ask for 'what I heard' processing. It passes phone A and phone B numbers. It assembles this request and sends it to server 30 in 1313. It should be noted that this Thread A now suspends waiting for the response and the phone actually proceeds to 1321 to start a separate thread for the second phone number while thread A waits. (1321 is described below)
  • Column 1304 receives this 'what I heard' request 1313 on server 30 in step 204. It first contacts the phone number to carrier service 80 in step 203A, sending request 1314 to it. Column 1305 receives this and sends the response back to 1304 in 1315. The determination in this case is that it is for AT&T.
  • the record indicates that AT&T server-2 be contacted asking for RBT_PROFILE_REQ1 asking for a raw user record for phone B. This request is sent to that server in 1316 at step 209
  • Step 210 server 30 receives this response in step 210 which, seeing it raw proceeds to step 21 1 to do raw calculation.
  • This RBT authority (record 2), uses rule 1002.
  • Rule 1002 states that Time of Day, MDN, user default and carrier default are checked in that order. (And that the record be stored also)
  • the returned record from AT&T server-2 indicates that user had no TOD set and no MDNs set, but it did have a line default of say 'Let It Be' with the full Metadata). This results in 'Let It Be' being played for everyone, so 'Let It Be' is the determination.
  • Server 30 proceeds through to 217G that sends the return value to thread A on the phone in 1318.
  • Thread A receives this 1318 response.
  • the phone's RBT authority record 101 was used to send this and its rule 1005 indicates that it is a 'what I heard' response with all the data needed.
  • Thread A posts all the Metadata in 1319 so the running app displays it in 1320. Thread A now dies.
  • the second phone number the outgoing call example is actually processed while thread A is waiting for the response. These occur in 1321 and the thread is started in 1322.
  • the phone's number is a phone A, and the one from the call log, phone B. From here, items 1323 through 1328 proceed in the same manner as 1313 through 1320, except that the carrier determination is '02', and thus RBT Authority record 4 is used.
  • Thread B on the phone receives the 'what I heard' response in 1328.
  • the phone's RBT authority record 101 was used to send this and its rule 1005 indicates that it is a 'what I heard' response with all the data needed. Thread B then posts all the Metadata in 1329 so that the main app displays it in 1330. Thread B now dies.
  • This illustrative example shows: 1 ) in a background process the recording of the audio clip between call initiated and call connected events on phone A takes place. 2) on call completion Phone A sends a 'what I heard' request with phone B and its own number as phone A. 3) server 30 has the raw record cached for phone B. 4) it calculates 'what I heard' from the raw record and concludes that a jukebox was played. 5) Server 30 returns 'what I heard' with full Metadata for each track in the jukebox. This includes a URL to the actual clip for each item in the jukebox. 6) The phone' background process sees that a jukebox is returned and starts audio match processing.
  • Audio processing uses the URL for each actual clip and collects all these possible clips.
  • the phone produces audio match signatures for each of the possible clips.
  • the phone makes an audio signature for the clip recorded at the start.
  • An audio match is performed matching the recorded clip against the list of possible clips. 1 1
  • On a match the Metadata from the matched record is assembled into a 'notification bar' event. 12) The full set of Metadata is stored. 13) When the user selects the event from the notification bar the app is run. 13) The app sees the stored Metadata and displays it.
  • Time flow chart 1400 shows the flow of this illustrative example.
  • This example uses RBT Authority record 201 on the phone to indicate that the 'what I heard/played' API is called on server TMI-1 and that a 'what I heard/played' determination is returned with full Metadata.
  • This record further states, through rule 1008 that should an RBT be returned in this 'what I heard' it will try to fail-over to an audio match.
  • This record 201 is an alternate 'ordinal 1 ' on the phone.
  • Record 103 on the phone is used to effect audio processing as ordinal 2.
  • Record 4 on the server indicates that it is a carrier aggregator that supports carriers Orange, 02 and Videophone and that it too returns a raw RBT profile. The contents of these records were described above with table 100
  • Column 1401 starts this process in 141 OA.
  • the 'what I heard' app's background process previously registered with the phone OS to get all the phone progress events including 'call initiated', 'call connected' and 'call ended'.
  • the phone OS on phone A sends the 'call initiated' event in 1410.
  • the background process of phone A receives this event in 1402, starting step 201 A that starts recording the 'heard' audio stream in 141 1 . This records the RBT audio heard.
  • the phone OS sends a 'call connected' event via 1412 to 1402.
  • the background process picks up the event and stops the recording of the audio stream in 1412B, resulting in only the pre-connected audio being recorded.
  • the background process in column 1402 receives 1413 in step 202A.
  • 202A collects the current phone's number as Phone A, and the number that was called as Phone B.
  • 'what I heard processing is entered at 203. Since there is no 'what I heard' determination in the notification, only the phone numbers, processing at 203 passes to 'what I heard' processing in 205. It looks through the phone's RBT authority table in step 205 and finds record 201 indicating that it should contact server TMI-1 in step 209 and ask for 'what I heard' processing. It passes Phone A and Phone B. it assembles this request and sends it to server 30 in 1414.
  • Column 1404 receives this 'what I heard' request on server 30 in step 204. It first contacts the phone number to carrier service 80 in step 203A, sending request 1415 to it. Column 1405 receives this and sends the response back to 1404 in 1416. The determination in this case is that it is for '02'
  • the record indicates that the aggregator service 'Europe Aggregator-1 ' RBT_Profile_REQ3.
  • Phone B is sent in the request. This request is sent to that server in 1417 at step 209
  • 1417A is received in column 1404 on server 30 and processing continues through 210 to step 21 1 to do raw calculation.
  • This RBT authority uses rule 1004.
  • Rule 1004 states that a raw RBT is expected, however it does not failover (here on the server) if a Jukebox is encountered.
  • Rule 1002 states that Time of Day, MDN, user default and carrier default are checked in that order.
  • the returned record from Europe Aggregator-1 indicates that an MDN based called ID is used when phone A calls in. (i.e. phone A passed in the API is the same as phone number in the MDN in the MDN based RBT section.).
  • the RBT in this MDN entry is a jukebox.
  • This same return from server 40 also has the track-info Metadata for all constituent tracks.
  • a 'what I heard' is returned indicting that a jukebox was played, and here are all the possible tracks. (It cannot determine which one, which is what the audio processing will do). This 'what I heard' is returned in 1418
  • step 213 This directs flow to step 213 in 1419.
  • step 213 extracts the URLs pointing to audio clips. In turn it sends a 1420 request to server 40's track repository, column 1407 (which may be separate from the 'RBT record' processor 1406 on server 40). Column 1407 returns the track in 1421 . Each track in the jukebox is done this way repeating 1420 and 1421 until all track audio has been retrieved.
  • this Metadata is used to assemble a short message, illustratively 'Your friend just played 'Let It Be'. This is sent to the OS API to display at the top notification bar in the system. The full set of Metadata is also sent for the system to later send to the app when it is run. This short message appears at the top of the phone in 1425A.
  • the user selects this notification from the phone's notification area in 1426. This causes the app to start.
  • the phone OS starts the app at 202C in 1427.
  • the app runs in column 1403. It starts at 202C seeing the waiting actual notification, it gets it in 1428 (that was stored from 1425 in the OS). Since all the Metadata was stored and retrieved in the notification call, the 'what I heard' visual Meta is displayed on the phone using step 217E.
  • This offline processing illustrative example shows the processing for two call pairs.
  • Phone A has the app
  • phone B doesn't have the app, but it can get the app.
  • the phone that has the app has a widget that will display the results.
  • Phone A can't get the app but allows text messages and phone B has no path to it by an alternate means. This results in two 'what I heard' determinations and four separate passes through 'has app'.
  • This illustrative example shows: 1 ) a process starts on server 30 to process calls; 2) it encounters call pair record number 1 . 2) Phone A and phone B in this record are extracted and passed to 'what I heard' processing. 3) On determining that something was heard, between these two (i.e. phone B played for phone A) (via a cached value) a 'has app' is done on Phone A 4) has app returns that this Phone A 'has the app'; the process then sends a notification via the notification service to phone A of 'what I heard'. All the Metadata is sent in the notification. 5) Phone A receives this in a background process. 6) It then sends all the Metadata that was in the notification to the widget which then displays it.
  • a text message is sent to this phone A with a URL to a page that lets the user interact more with this heard RBT.
  • the user follows the link to the web page. 16) The process then sends phone B of the second record to 'what I played. 17) on determining that something was played, a 'has app' is done 18) has app returns that this phone is unreachable by any means (perhaps it is a landline) and discards the result of the 'what I heard'.
  • Time flow chart 1500 shows the flow of this illustrative example.
  • this example uses RBT Authority record 104 on the phone to indicate that a 'what I heard/played' that is received later in a notification has all the required Metadata.
  • This is a special record with the ordinal column set to 'enough' meaning that there is enough data.
  • Record 5 on the server indicates that a 'what I heard' is in the cache in table 120.
  • Column 1501 starts this offline process on server 30 it executes the process 202E.
  • 202E runs through the supplied list.
  • a search of 120 with phone A and phone B shows a cached 'what I heard' with all Metadata in 1521 .
  • step 215. This sends a 'has app' lookup request for phone A to the 'has app' server 90 via 1522 to column 1502.
  • the 'has app service returns 'has app' in 1523 to column 1501 .
  • 1501 receives this 'has app' result. This causes processing on server 30 to continue to step 217H which then sends a notification 1524 with all the Metadata to the phone A via notification server 60 in 1503. Column 1503 gets the 1524 and forwards it to phone A column 1504 via 1525.
  • Phone A receives the notification 1525 in column 1504.
  • the background process on phone A receives this notification.
  • the background process then passes this complete 'what I heard' Metadata display information to the widget half of the 'what I heard' app. (this is not an actual send but a pass within the app of the data) the widget displays this Metadata via 217A in 1526 and the user sees 'what I heard' from this call sent from the 'offline process'
  • the offline process then moves to processing 'has app' for phone B of this first call record in 1527.
  • the 'what I heard' determination that was collected during processing of phone A is the same 'what I played' for phone B so the stored determination for this pair may be used for both 'has app' determinations.
  • the call 1528 to the 'has app' service in column 1502 determines that phone B does not have the app, but is on a phone type that may have the app.
  • the 'has app' service then sends, illustratively, a text message that says 'you recently heard 'Let It Be' from your friend, click here to download the app to learn more.' (This text message was supplied by 30 to then 'has app' service and passed with 1528).
  • the text message is sent to phone B via 1529.
  • Phone B in column 1505 receives this text message from 1529. At some later time the user clicks on the link in the message to get the app in 1529A. This starts the download store in 1529BThis directs the user to the on phone app store in column 1506. The user may download the app in 1529C.
  • the offline process then moves to processing the next phone A and Phone B pair in 1531 .
  • phone A and phone B are used with a call direction 'outgoing', asking 'what I heard on phone A when I called phone B'.
  • the same 'what I heard' processing as 1520 occurs here in 1531 retrieving a cached 'what I heard' using phone A and phone B from 120 and using it for 'what I played' determination with all the Metadata.
  • the call 1532 to the has app service for phone A in column 1502 determines that phone A does not have the app and that phone 'cant' have the app, but the phone is capable of text messages.
  • the 'has app' service then sends, illustratively, a text message that says 'you recently heard 'let it be' from your friend, click here to learn more.' (This text message was supplied by 30 to the' has app' service and passed with 1532). The text message is sent to phone B via 1533.
  • Phone A in column 1507 receives this text message from 1533. At some later time in 1533A the user clicks on the link in the message to go to the web page referred to in the text message via 1534. This directs the user to the web site in column 1508. The user sees this web page and may interact with it in 1535.
  • the call 1538 to the has app service in column 1502 determines that phone B does not have the app and that phone 'cant' have the app, and the phone is NOT capable of text messages.
  • the 'has app' service returns 'can't reach phone with any alternate message' in 1539. This results in 1540 the 'what I played' determination being dropped on the floor.
  • This illustrative example shows an example of the phone going directly to RBT authorities, one for each of two carriers directly, bypassing server 30. it shows: 1 ) outgoing phone call 1 occurring, 2) the background processes contacting the 'phone to carrier' service, 3) direct call for a raw record to a server 40 of this carrier, 4) after raw processing, the display of this 'what I heard' in the widget. 5) A second call coming in, 6) steps repeated on a second server for a different carrier.
  • Time flow chart 1600 shows the flow of this illustrative example.
  • This example uses RBT Authority record 108 on the phone to indicate that a raw record is requested from AT&T-server 2 and to use rule 1006 to process it.
  • Record 109 on the phone indicates the same thing, except that server Verizon-1 is contacted. The contents of these records were described above with table 100
  • Column 1602 the background process on phone A receives 1610 in step 202A.
  • This version of the software apparatus is configured to contact the phone number to carrier service and to search the RBT authority with the returned carrier name. It contacts the phone number to carrier service 80 in step 203A; sending request 161 1 to it for phone B.
  • Column 1603 receives this and sends the response back to 1602 in 1612. The determination in this case is that it is for AT&T.
  • the record indicates that AT&T server-2 be contacted asking for RBT_PROFILE_REQ1 .
  • Phone B is sent in the request. This request is sent to that server in 1613 at step 209
  • Column 1605 which is AT&T server-2 (a server 40 type server), receives this request and using phone B that was passed returns the raw RBT record associated with B. it returns it through 1614 [00713] 1614 is received in column 1602, the background process on phone A and processing continues through 210 to step 21 1 to do raw calculation.
  • This RBT authority (record 108) uses rule 1006. Rule 1006 states that Time of Day, MDN, user default and carrier default are checked in that order. The returned record from AT&T server-2 indicates that user had no TOD set and no MDNs set, but it did have a line default of say 'Let It Be' with the full Metadata.
  • the background process then passes the what I heard Metadata display information to the widget half of the 'what I heard' app in 1615 (this is not an actual send but a pass within the app of the data)
  • Network calls server 30 What I heard' and provides it in the low level protocol to the phone.
  • the phone's call app displays then stores this in the call log and address book
  • This illustrative example shows the network providing 'what I heard' at a low level to the phone and the phone's call app playing it. It also shows how the network can call server 30 to determine 'what I heard'. It shows: 1 ) an outgoing call started, 2) the network contacting server 30 to determine what I heard 3) a 'ringing' notification from the network where the network adds in this 'what I heard'. 3) The phone's app that handles phone calls updates the display with 'what you are listening to now'. 4) On call termination the phone's pop up that has 'call time 10:00' is updated to include 'what I heard'. 5) The 'what I heard" determination is stored in the phone's call log and address book 6) the 'what I heard' widget is updated with this outgoing call.
  • the phone OS in column 1702 gets 171 1 and sends a low level phone protocol call connection request 1712 to the network 10 to start the call connection process.
  • the carrier network 10 receives 1712 in column 1703 and determines what is being played (or about to be played) by the incoming side.
  • the network Had the incoming handset, phone B been on network 10, the network has access to the RBT profile of B and could, if it so desired, determine 'what I heard' directly.
  • the phone being on network 20, network 10 cannot readily access this record, instead it opts to call a 'what I heard' service on server 30. (Note, network 10 could also make this access to server 30 when phone B is on its network if it so desires).
  • Carrier Network 10 then forwards a 'what I heard' API request to server 30 in 1713.
  • 1713 is received on server 30 in column 1704. This illustrative example does not show the interaction and setup required on server 30 to determine 'what I heard' for clarity. It returns a 'what I heard' with full Metadata in 1714.
  • step 202F is incorporated into the phone OS by the phone's manufacturer.
  • Step 202F returns this 'what I heard' to the phone OS and the phone OS uses whatever mechanism it normally uses to convey a 'call proceeding message', appending 'what I heard' to the phone's app via a 1716.
  • 202F stores phone A, phone B, direction and this 'what I heard' information for other apps to get. It may also broadcast this information on phones with broadcast event mechanisms. (The phone's App could also have listened for a broadcast event or it could have passed the location that 202F stores the 'what I heard')
  • 1716 is received by the phone's call app in column 1701 .
  • the required information is extracted from the returned 'what I heard' display Metadata (only a portion is sent by the network) and displayed illustratively in 1717 within the phone's call app as 'What you're hearing now' (e.g. you are hearing 'Let It Be' while you wait for 'George' to answer).
  • the phone OS in column 1702 sends a 'call completed' event in 1721 to apps that had registered to receive this broadcast event.
  • step 202A is received by the 'what I heard' background process in the 'what I heard' software apparatus by step 202A. (Shown in column 1707). For clarity the steps that the background process takes to determine 'what I heard' are not shown, instead they are represented by 1722, which takes phone A, phone B and direction from the step 202A and returns full 'what I heard' display Metadata. (Alternately, had 202F included complete 'what I heard' info and stored it, 1722 should retrieve this.)
  • step 220C invokes step 220C to send, via 1723, the 'what I heard' determination to the phone's call log.
  • the call log receives this in column 1705.
  • step 220B invokes step 220B to send, via 1724, the 'what I heard' determination to the phone's Address book (also called the contact list).
  • the address book receives this in column 1706.
  • the background process in 1707 invokes the display method in step 217A to display, via 1725, the 'what I heard' determination in the 'what I heard' software apparatus' widget.
  • the widget displays this in 1726 (steps not shown for clarity.)
  • the user now, through the widget may interact with this 'what I heard' display.
  • Figure 13 is a high-level block diagram showing an example architecture of a mobile device.
  • the mobile device includes processor(s) 1302 and a memory 1304 coupled to an interconnect 1306.
  • the interconnect 1306 shown in Figure 13 is an abstraction that represents any one or more separate physical buses, point to point connections, or both, connected by appropriate bridges, adapters, or controllers.
  • the processor(s) 1302 may include central processing units (CPUs) of the mobile device and, thus, control the overall operation of the mobile device by executing software or firmware.
  • CPUs central processing units
  • the processor(s) 1302 may be, or may include, one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs), or the like, or a combination of such devices.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • PLDs programmable logic devices
  • the memory 1304 represents any form of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, or the like, or a combination of such devices.
  • the software or firmware executed by the processor(s) may be stored in a storage area 1310 and/or in memory 1304, and typically include an operating system 1308 as well as one or more applications 1318.
  • Data 1320 utilized by the software or operating system is also stored in the storage area or memory.
  • the storage area 1310 may be a flash memory, hard drive, or other mass-storage device.
  • the mobile device includes an input device 1312, which enables a user to control the device.
  • the input device 1312 may include a keyboard, trackpad, touch-sensitive screen, or other standard electronic input device.
  • the mobile device also includes a display device 1314 suitable for displaying a user interface, such as the display.
  • a wireless communications module 1316 provides the mobile device with the ability to communicate with remote devices over a network using a short range or long range wireless protocol.
  • FIG 14 is a front view of a mobile device 1400 suitable for processing.
  • the mobile device 1400 may include a housing 1401 , a plurality of push buttons 1402, a directional keypad 1404 (e.g., a five-way key), a microphone 1405, a speaker 1406, and a display 1410 carried by the housing 1401 .
  • the mobile device 1400 may also include other microphones, transceivers, photo sensors, and/or other computing components generally found in PDA phones, cellular phones, smartphones, portable media players, portable gaming devices, portable email devices (e.g., Blackberrys), or other mobile communication devices.
  • the display 1410 includes a liquid-crystal display (LCD), an electronic ink display, and/or other suitable types of display configured to present a user interface.
  • the mobile device 1400 may also include a touch sensing component 1409 configured to receive input from a user.
  • the touch sensing component 1409 may include a resistive, capacitive, infrared, surface acoustic wave (SAW), and/or another type of touch screen.
  • the touch sensing component 1409 may be integrated with the display 1410 or may be independent from the display 1410.
  • the touch sensing component 1409 and the display 1410 have generally similar sized access areas. In other embodiments, the touch sensing component 1409 and the display 1410 may have different sized access areas.
  • the touch sensing component 1409 may have an access area that extends beyond a boundary of the display 1410.
  • the mobile device 1400 also includes a 12-key numerical keypad 1412 capable of receiving text or numerical input from a user.
  • the mobile device 1400 may include a full QWERTY keyboard for receiving user input.
  • the mobile device 1400 may also provide a software keyboard or keypad on the display 1410 to enable a user to provide text or numerical input through the touch-sensing component 1409.
  • Non-RBT determination containing the record of the agent within a corporation that was just called
  • This illustrative example shows a phone A which has the software apparatus calling an organization's PBX, being connected to an agent through that PBX and, after the call completes, seeing the agent's phone name, photo and direct call back information. It shows 1 ) the call being placed 2) the owner of phone A navigating through the PBX to an agent. 3) The 'what I heard' software apparatus being started by a call completion event 202A. 4) Use of the local number to carrier table to direct flow based on this particular phone number, 5) the 'what I heard software' apparatus' querying a remote server 40 with phone A's phone number to obtain the needed metadata about the call that was just completed as a non-RBT determination. 5) Displaying that information in the pop-up style 217B
  • the first record in Table 180 contains the illustrative record for this organization; in this case it is Time Warner Cable (TWC).
  • Column 181 contains a list of phone numbers that may be used to contact TWC and that support the 'what I heard' service 40 that will be accessed by the 'what I heard' software apparatus.
  • Column 182 has a key that will be used to access table 100, record 1 1 (in figurel O).
  • This record 1 1 in table 100 contains information needed to contact the appropriate server, in this case server TWC-4.
  • the record also indicates that a 'what I heard' determination is expected.
  • Record 1 1 further refers to rule 1012 (column 106), referring to table 130 which then refers to rule '13' in rule list 560. This rule says to expect a direct 'what I heard' determination with 'who you were talking to' Metadata.
  • the time flow chart 1800 shows the flow of this illustrative example.
  • Column 1801 is the phone A's Operating system. In our case it is responsible for placing phone calls, conducting the call, and then starting the 'what I heard' software apparatus when the call completes.
  • Column 1802 is the 'what I heard' software apparatus on phone A.
  • 1803 is phone A's carrier
  • 1804 is TWC's carrier
  • 1805 is TWC's PBX
  • 1806 is the desk phone of the TWC agent that is connected to through the PBX and also represents the agent him or herself.
  • 1807 is the 'what I heard' server 30.
  • 1808 is TWC's call completion query server that takes a phone # and timestamp and returns the agent, agent photo and call back information. This server acts as a server 40 type in our system.
  • the illustrative example starts at step 1810 by the user phoning TWC.
  • the phone OS in column 1801 contacts their own carrier, carrier A in column 1803 through a message 181 1 .
  • Carrier A contacts TWC carrier B in column 1804 through a message in 1812.
  • Carrier B connects to TWC PBX in column 1805 through step 1813.
  • Steps 181 1 though 1813 serve only to connect phone A to the TWC PBX.
  • TWC's PBX starts its automated call direction service in 1814 to deliver audio prompts to the user in 1815.
  • the user presses DTMF keys or speaks (1816) to this call director to direct his or her call too an agent in column 1806.
  • Step 1816A is internal to the PBX and has the effect of connecting the caller on phone A to an agent.
  • the agent is shown in step 1817.
  • the call completes in step 1819, column 1801 (either side hangs up).
  • the phone's OS in column 1801 sends a call completion event in 1820, starting the software apparatus in column 1802.
  • the software apparatus contacts 'what I heard' server 30 in column 1807 with 1821 .
  • Table 180 is consulted first to see if 'phone B' is in this table. We find TWC's number there, and then using the retrieved column 182, access table 100 to determine that TWC-4 server should be contacted.
  • Server 30 contacts the TWC this call completion server, acting as a server 40 in column 1808, through 1822.
  • the TWC call completion server contacts the PBX in column 1805 through the query in 1823 and it responds back in 1824.
  • Server 40 in column 1808 responds back to server 30 with a 'what I heard' determination (in its own style and in this case a 'non-RBT' determination) with the agent's name, a URL to his or her phone and a dial string for directly dialing this agent back with an 1825.
  • Server 30 in column 1807 responds back to the 'what I heard' software apparatus on phone A, column 1802 through the 'what I heard' determination in 1826 with metadata in the form needed for rule 13 in table 560.
  • the 'what I heard' software apparatus in column 1802 receives this indication and displays the agent's name, photo and direct contact number in a pop-up style display in step 1827.
  • Item 2008 shows this determination as seen through an embodiment of the current disclosure.
  • This illustrative example shows a phone A which has the software apparatus calling a particular phone B number and receiving information related to that number. It shows 1 ) the 'what I heard' software apparatus being started by a call completion event 202A. 2) Use of the local number to carrier table to direct flow based on this particular phone B number, 3) the 'what I heard software' apparatus' calling a local-to-server-30 software apparatus with phone B's phone number to obtain the needed metadata about the call that was just completed as a non-RBT determination. 4) Displaying that information in the pop-up style 217B.
  • the second record in Table 180 contains the illustrative record for any phone A calling this phone B, in this case it is voting for a particular 'American Idol' contestant using one of two 1 -800 (866) numbers assigned to vote for that particular contestant.
  • Table 181 contains this list of phone numbers used to vote for this contestant.
  • Column 182 has a key that will be used to access table 100, record 12 (in figurel O).
  • This record 12 in table 100 indicates that the local-to-server-30 software apparatus be contacted (via the special value 'Local-code' in column 103) and that it need only pass 'phone b' (request column 105).
  • the record also indicates that a 'what I heard' determination is expected.
  • Record 12 further refers to rule 1013 (column 106), referring to table 130 which then refers to rule '14' in rule list 560.
  • This rule says to expect a direct 'what I heard' determination with a generic set of data to display, in this case it has two images, two sets of text and two action URLs connected to two buttons.
  • the time flow chart 1900 show the flow of this illustrative example.
  • Column 1901 is the phone A's Operating system. In our case it is responsible for starting the 'what I heard' software apparatus when the call completes.
  • Column 1902 is the 'what I heard' software apparatus on phone A.
  • 1903 is the 'what I heard' server 30.
  • 1904 is the local number to carrier table (180); we show it here because it replaces the call to server 80 seen in other illustrative examples.
  • 1905 is the local-to-server-30 'what I heard' software apparatus.
  • the illustrative example starts at step 1910 with the call completing in column 1901 (either side hangs up).
  • the phone's OS in column 1901 sends a call completion event in 191 1 , starting the software apparatus in column 1902.
  • the software apparatus contacts 'what I heard' server 30 in column 1903 with 1912.
  • Table 180 is consulted first to see if 'phone B' is in this table (1913).
  • the retrieved record from table 100 indicates 'local-code', directing the system to use the local-to-server-30 'what I heard' software apparatus (1914).
  • Image 2009 shows this determination as seen through an embodiment of the current disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Quality & Reliability (AREA)
  • Information Transfer Between Computers (AREA)
  • Telephonic Communication Services (AREA)

Abstract

L'invention concerne un procédé et un système pour identifier divers détails d'une tonalité de rappel (« RBT ») ou d'autres métadonnées liées à des paires d'appels après la réalisation d'un appel (« le système »). Le système comprend une interface d'utilisateur adéquate pour l'utilisation dans divers dispositifs mobiles. L'interface d'utilisateur permet à un utilisateur d'identifier de multiples caractéristiques d'une RBT telles que le nom de la chanson, le nom de l'artiste et l'image de la couverture d'album. De plus, les métadonnées améliorant la RBT, telles un coupon lié à une publicité de la RBT peuvent être affichées. Les métadonnées affichées peuvent être des informations liées à des paires d'appels non liées à une RBT telles que le nom d'un représentant du service client avec lequel vous avez parlé récemment. L'interface d'utilisateur produit également des actions appropriées à réaliser après l'appel telles que l'achat de la chanson complète liée à la RBT, l'évaluation de la RBT, l'utilisation du coupon ou l'évaluation du représentant du service client.
PCT/US2013/030564 2012-06-19 2013-03-12 Procédé obtenant l'identification d'une tonalité de rappel et d'autres informations de paires d'appels sur des combinés d'appel entrants et sortants WO2013191755A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/564,889 US20150304491A1 (en) 2012-06-19 2013-03-12 Method providing a graphical user interface readout of the identification of a ringback tone on the incoming and outgoing call handsets

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US201261663418P 2012-06-22 2012-06-22
US201261663394P 2012-06-22 2012-06-22
US201261663479P 2012-06-22 2012-06-22
US61/663,479 2012-06-22
US61/663,394 2012-06-22
US61/663,418 2012-06-22

Publications (1)

Publication Number Publication Date
WO2013191755A1 true WO2013191755A1 (fr) 2013-12-27

Family

ID=49769180

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2013/030564 WO2013191755A1 (fr) 2012-06-19 2013-03-12 Procédé obtenant l'identification d'une tonalité de rappel et d'autres informations de paires d'appels sur des combinés d'appel entrants et sortants
PCT/US2013/031798 WO2013191770A1 (fr) 2012-06-22 2013-03-14 Envoi et réception de métadonnées contenant des références à des supports numériques ou des biens de consommation entre abonnés appelants

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2013/031798 WO2013191770A1 (fr) 2012-06-22 2013-03-14 Envoi et réception de métadonnées contenant des références à des supports numériques ou des biens de consommation entre abonnés appelants

Country Status (1)

Country Link
WO (2) WO2013191755A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10127604B2 (en) * 2014-08-27 2018-11-13 Verizon Patent And Licensing Inc. Identification and caller options relating to custom ringback audio

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105634981B (zh) * 2014-10-30 2019-08-16 阿里巴巴集团控股有限公司 内容缓存和传输方法及其系统

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060023862A1 (en) * 2004-07-27 2006-02-02 Geoff Sutcliffe Methods, systems, devices, and products for providing ring backs
US20060182247A1 (en) * 2005-01-28 2006-08-17 Batni Ramachendra P Change to playback characteristic of ringback tone
WO2007005660A2 (fr) * 2005-07-05 2007-01-11 Microsoft Corporation Annonce d'informations de presence au cours d'un retour d'appel telephonique
US20070121821A1 (en) * 2005-11-23 2007-05-31 Chengchun Su System and Method for Displaying Ring Back Tone and Caller Display in Picture/Video Format
US20110225061A1 (en) * 2010-03-09 2011-09-15 Keith Chad C Method For Automating Onboarding Of User Generated Ringback Tones To Sales Distribution Channel

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060240851A1 (en) * 2003-03-21 2006-10-26 Vocel, Inc. Interactive messaging system
US7917148B2 (en) * 2005-09-23 2011-03-29 Outland Research, Llc Social musical media rating system and method for localized establishments
KR100783328B1 (ko) * 2006-02-28 2007-12-10 이재영 휴대단말기의 사진 공유와 코멘트 추가방법 및 그 장치
KR20070113738A (ko) * 2006-05-26 2007-11-29 엠브릿지 주식회사 국제 문자메시지 서비스 방법
US8577715B2 (en) * 2009-02-27 2013-11-05 Blackberry Limited Pushed ringtones based on device-side content

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060023862A1 (en) * 2004-07-27 2006-02-02 Geoff Sutcliffe Methods, systems, devices, and products for providing ring backs
US20060182247A1 (en) * 2005-01-28 2006-08-17 Batni Ramachendra P Change to playback characteristic of ringback tone
WO2007005660A2 (fr) * 2005-07-05 2007-01-11 Microsoft Corporation Annonce d'informations de presence au cours d'un retour d'appel telephonique
US20070121821A1 (en) * 2005-11-23 2007-05-31 Chengchun Su System and Method for Displaying Ring Back Tone and Caller Display in Picture/Video Format
US20110225061A1 (en) * 2010-03-09 2011-09-15 Keith Chad C Method For Automating Onboarding Of User Generated Ringback Tones To Sales Distribution Channel

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10127604B2 (en) * 2014-08-27 2018-11-13 Verizon Patent And Licensing Inc. Identification and caller options relating to custom ringback audio

Also Published As

Publication number Publication date
WO2013191770A1 (fr) 2013-12-27

Similar Documents

Publication Publication Date Title
US8175651B2 (en) Devices and methods for automating interactive voice response system interaction
US8537980B2 (en) Conversation support
KR100806409B1 (ko) 주문형 호출경보시스템 및 방법
US8526577B2 (en) System and method to access content from a speech-enabled automated system
US8572269B2 (en) CSIP proxy for translating SIP to multiple peer-to-peer through network resources
US20080187112A1 (en) Method and system for delivering podcasts to communication devices
US7953211B2 (en) Automated ringback update system
US20070243887A1 (en) Platform for telephone-optimized data and voice services
CN101918940A (zh) 用于移动设备的智能网页供应系统和方法
KR20130118346A (ko) 데이터 채널 증강된 자동 교환 음성 응답 시스템을 위한 방법 및 장치
CN1581900A (zh) 动态图片主叫用户识别
US20120057688A1 (en) Method and apparatus for using a search engine advantageously within a contact center system
JP2009516425A (ja) リングバック・トーンの選択を援助するためのリングバック・トーン好み情報
US20100235443A1 (en) Method and apparatus of providing a locket service for content sharing
US20170034353A1 (en) System and method for dynamic call diversion
US20150304491A1 (en) Method providing a graphical user interface readout of the identification of a ringback tone on the incoming and outgoing call handsets
US20090181614A1 (en) Method and system for providing playback of digital audio content available through a computer network
WO2012034318A1 (fr) Procédé et appareil de mise en œuvre d'une réponse vocale interactive à partir d'un réseau de la prochaine génération
US6493434B1 (en) Update of web audio messages via audio user interface
US20140128039A1 (en) System and Method for Storing and Managing Voicemails
WO2013191755A1 (fr) Procédé obtenant l'identification d'une tonalité de rappel et d'autres informations de paires d'appels sur des combinés d'appel entrants et sortants
US9860329B2 (en) Determining customized audio services
US20080014911A1 (en) Group sharing of media content
KR20040071744A (ko) 무선 장치에서 인터넷 컨텐츠를 얻기 위한 방법 및 장치
US9344563B2 (en) System for and method of recycling numbers seamlessly

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13807393

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 14564889

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 26/05/2015)

122 Ep: pct application non-entry in european phase

Ref document number: 13807393

Country of ref document: EP

Kind code of ref document: A1