CN105453026A - Auto-activating smart responses based on activities from remote devices - Google Patents

Abstract

An electronic device with one or more processors and memory includes a procedure for using a digital assistant to automatically respond to incoming communications. In some embodiments, the device obtains a speech input from a user, and, in response to obtaining the speech input, the device determines whether the speech input includes instructions for performing a specified action in response to receipt of a subsequent incoming communication from one or more specified senders. After storing the instructions, the device obtains an incoming communication from a respective sender. The device determines whether the respective sender is one of the one or more specified senders, and, upon determining that the respective sender is one of the one or more specified senders, the device performs the specified action in accordance with the instructions and thereafter automatically deleting the instructions.

Description

Based on the movable automatic activation intelligent response from remote equipment

the cross reference of related application

This application claims U.S.Provisional Serial the 61/862nd that submit on August 6th, 2013, that be entitled as " AUTO-ACTIVATINGSMARTRESPONSESBASEDONACTIVITIESFROMREMOTE DEVICES ", the right of priority of No. 927, its full content is incorporated herein by reference for all objects.

Technical field

The disclosed embodiments relate generally to digital assistant, and relate more particularly to carry out from dynamic response importing communication into.

Background technology

Just as mankind's personal assistant, digital assistant can perform asked task and provide asked suggestion, information or service.Ability for the digital assistant meeting the request of user depends on that digital assistant is to request or the correct understanding of instruction.Progress in natural language processing has made user natural language and digital assistant can be used to carry out alternately with form that is spoken or text.This digital assistant can the input of interpreting user to infer the intention of user, inferred intention is converted to the task and parameter that can take action, executable operations or configuration service to execute the task, and produce the intelligible output of user.Ideally, the output produced by digital assistant should meet the intention of the user expressed by natural language interaction between user and digital assistant.

Ability for generation of the digital assistant of the response of the satisfaction of asking for user depend on to digital assistant can natural language processing, knowledge base, artificial intelligence.In addition, although a lot of Third party system of current existence and service, there is not digital assistant and import communication in response to obtaining from one or more appointment sender and automatically perform the effective means of required movement.

Summary of the invention

Embodiment disclosed herein provides for operand word assistant to import communication in response to receiving from one or more appointment sender and automatically perform the method for required movement, equipment, system and non-transitory computer readable storage medium.

Some embodiments provide and make digital assistants to importing the method communicating and carry out from dynamic response into.Perform the method at one or more electronic equipment place, each electronic equipment comprises one or more processor and storer.First, phonetic entry is obtained from user.In response to acquisition phonetic entry, determine whether phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement.After this, instruction is stored.After storage instruction, obtain from corresponding sender and import communication into.Determine whether corresponding sender is one or more appointment senders specified in sender.When determining that corresponding sender is an appointment sender in one or more appointment sender, perform required movement according to this instruction.After this, (such as, when not having human intervention) deletes this instruction automatically.

In certain embodiments, determine whether phonetic entry comprises instruction and comprise: determine whether phonetic entry comprises for one or more appointment is follow-up imports communication type into and perform the instruction of required movement in response to receiving from one or more appointment sender.Determine corresponding sender be in one or more appointment sender one specify sender and import into communication be one or more appointment follow-up import in communication type one specify follow-up import communication type into time, perform required movement according to this instruction.After this, (such as, when not having human intervention) deletes this instruction automatically.

In certain embodiments, the follow-up communication type of importing into of the follow-up corresponding appointment of importing in communication type of one or more appointment is in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, importing communication into is call, and wherein required movement sends speech message.

In certain embodiments, speech message is the record of voice to user at least partially.

In certain embodiments, speech message is digitized version or the text-to-speech version of input text at least partially.

In certain embodiments, the method comprises and performs natural language processing to convert phonetic entry to text to phonetic entry, and wherein stores instruction and comprise instruction is stored as text.

In certain embodiments, perform required movement to comprise: according to this instruction, write the response to corresponding sender; And this response is communicated to sender.

In certain embodiments, this communication is in the following: call, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, this communication uses and imports the identical means of communication that communicates into.

In certain embodiments, this is write and comprises according to this instruction and write the response to corresponding sender based on contextual information at least in part.

In certain embodiments, contextual information comprises in the following: the positional information be associated with user, pattern information or behavioural information.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the contacts list be associated with the user of equipment, judge that whether corresponding sender is that in one or more appointment sender one specifies senders.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the caller ID be associated with corresponding sender, judge that whether corresponding sender is that in one or more appointment sender one specifies senders.

In certain embodiments, the method comprises the address that mark is associated with the sender of communication, and determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the address be associated to the corresponding sender of communication, judge that whether corresponding sender is that in one or more appointment sender one specifies senders.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the content importing communication into received from corresponding sender, judge that whether this sender is that in one or more appointment sender one specifies senders.

In certain embodiments, follow-up communication type of importing into is specified to be in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, importing communication into is in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, action is selected from the group be made up of the following: send speech message to sender; SMS message is sent to sender; MMS or photo is sent to sender; Email is sent to sender; Event invitation is sent to sender; One or more calendar is shared with sender; One or more custom attributes is shared with sender.

In certain embodiments, when further comment or the instruction not from the user of equipment, execution and deletion are performed automatically.

In certain embodiments, perform required movement to comprise: perform the first sub-action; And perform the second sub-action different from the first sub-action.

In certain embodiments, this instruction specifies expiration period further; And the method comprises: determining that corresponding sender is not that in one or more appointment sender one specifies sender and expiration period past tense, automatically delete this instruction.

In certain embodiments, corresponding sender is the first corresponding sender; And the method comprises: determining that the first corresponding sender is not that in one or more appointment sender one specifies sender and expiration period not past tense, abandon deleting this instruction; Obtain from the different from the first corresponding sender second corresponding sender and import communication into; Determine whether the second corresponding sender is one or more appointment senders specified in sender; And determining that the second corresponding sender is that in one or more appointment sender one specifies sender and expiration period not past tense, performing required movement according to this instruction, and after this automatically deleting this instruction.

According to some embodiments, a kind of non-transitory computer readable storage medium with instruction stored therein, this instruction, when being performed by the electronic equipment with one or more processor, makes equipment perform the operation of any one method in method described above.According to some embodiments, electronic equipment comprises one or more processor, storer and one or more program; One or more program is stored in memory and is configured to be performed by one or more processor, and one or more program comprises the operation for performing any one method in method described above.According to some embodiments, graphical user interface on a kind of electronic equipment, this electronic equipment has display, storer and the one or more processors of one or more programs for storing in execute store, and graphical user interface comprises the user interface according to any one method in method described above.According to some embodiments, a kind of electronic equipment, comprises the device of the operation for performing any one method in method described above.According to some embodiments, a kind of signal conditioning package, the electronic equipment for the device of the operation comprised for performing any one method in method described above uses.

According to some embodiments, electronic equipment comprises sound reception unit, and this sound reception unit is configured to obtain phonetic entry from user.Electronic equipment also comprises the processing unit being coupled to sound reception unit.In response to acquisition phonetic entry, this processing unit is configured to determine whether phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement and the instruction from first user.After this, processing unit is configured to store instruction.After storage instruction, processing unit is configured to obtain from corresponding sender import communication into.After this, processing unit is configured to determine whether corresponding sender is one or more appointment senders specified in sender.When determining that corresponding sender is an appointment sender in one or more appointment sender, processing unit is configured to perform required movement according to this instruction.After this, processing unit is configured to automatically delete this instruction.

The one or more embodiments of the detail of the theme described in this instructions have been set forth by reference to the accompanying drawings with following description.Other features of theme, aspect and advantage will become apparent from specification, drawings and the claims.

Accompanying drawing explanation

Fig. 1 is the block diagram that the environment operated wherein according to the digital assistants of some embodiments is shown.

Fig. 2 is the block diagram of the digital assistants client illustrated according to some embodiments.

Fig. 3 A is the block diagram that digital assistant according to some embodiments or its server section are shown.

Fig. 3 B is the block diagram of the function illustrated according to the digital assistants shown in Fig. 3 A of some embodiments.

Fig. 3 C is the diagram of the part according to the body shown in Fig. 3 B of some embodiments.

Fig. 4 be according to some embodiments for importing communication in response to receiving from one or more appointment sender and automatically perform the diagram of a part for the body of required movement.

Fig. 5 illustrates the process flow diagram of the example process for operand word assistant according to some embodiments.

Fig. 6 illustrates the process flow diagram of the example process for operand word assistant according to some embodiments.

Fig. 7 is the functional block diagram of the electronic equipment according to some embodiments.

Similar reference number runs through accompanying drawing and refers to corresponding parts.

Embodiment

Fig. 1 is the block diagram of the operating environment 100 according to the digital assistants of some embodiments.Term " digital assistants ", " virtual assistant ", " intelligent automation assistant " or " automatic digital assistant " refer to that decipher natural language that is oral and/or textual form inputs to infer user view, and based on any information handling system that the user view inferred performs an action.Such as, in order to carry out the user view inferred, it is one or more that system can perform in following content: carry out identification mission stream by design in order to the step and parameter realizing the user view inferred; Specific requirement from the user view inferred is input in task flow; By streams of executing the task such as calling program, method, service, API; And the output response of the sense of hearing (such as, voice) generated user and/or visual form.

Specifically, digital assistants (DA) can accept at least in part for natural language instructions, request, state, tell about and/or inquire form user request.Usually, user asks or seeking digital assistants makes informedness answer, or seeking digital assistants executes the task.Gratifying response for user's request is to provide asked informedness and answers, performs the task or the combination of both of asking.Such as, user can propose such as to digital assistants " I now where? " and so on problem.Based on the current location of user, digital assistants may answer that " you are at Central Park." user also can ask to execute the task, such as, " me please be reminded calling this afternoon 4 mother." responsively, digital assistants can confirm request, and then in the electronic calendar of user, create suitably prompting.During performing the task of asking, digital assistants carries out with user sometimes alternately in section for a long time in the continuous dialogue relating to repeatedly message exchange.Exist and carry out alternately with solicited message or the many additive methods performing various task with digital assistants.Respond except providing speech and take except action by programming, digital assistants also provides the response of other videos or audio form, such as, as text, alarm, music, video, animation etc.

The example of digital assistants on January 10th, 2011 submit to, the U.S. patent Nos patent application serial numbers the 12/987th of the applicant that is entitled as " IntelligentAutomatedAssistant ", described by having in No. 982, its full content is incorporated to herein by reference.

As shown in fig. 1, in certain embodiments, digital assistants is implemented according to client-server model.Digital assistants is included in client-side aspects 102a, 102b (being hereafter called " DA-client 102 ") of the upper execution of subscriber equipment 104a, 104b, and the server portion 106 (being hereafter called " DA-server 106 ") performed on server system 108.DA-client 102 is communicated with DA-server 106 by one or more network 110.DA-client 102 provides client-side function, such as user oriented input and output process, and communicates with DA-server 106.The DA-client 102 that DA-server 106 is any amount provides server side function, and the DA-client 102 of described any amount is positioned on corresponding subscriber equipment 104 separately.

In certain embodiments, DA-server 106 comprises the I/O interface 112 of curstomer-oriented end, one or more processing module 114, data and model 116, and to the I/O interface 118 of external service.The I/O interface of curstomer-oriented end promotes the input and output process of the curstomer-oriented end of digital assistants server 106.One or more processing module 114 utilizes data and model 116 to determine the intention of user based on natural language input, and carries out tasks carrying based on the user view inferred.In certain embodiments, DA-server 106 communicates to finish the work or Information Monitoring with external service 120 by one or more network 110.I/O interface 118 to external service promotes that this type of communicates.

The example of subscriber equipment 104 to include but not limited in the service of handheld computer, personal digital assistant (PDA), panel computer, laptop computer, desk-top computer, cell phone, smart phone, enhancement type general use grouping wireless electricity (EGPRS) mobile phone, media player, navigator, game console, televisor, telepilot or these data processing equipments two or more combination arbitrarily or other data processing equipments.The more details of associated subscriber equipment 104 provide with reference to the illustrative user device 104 shown in Fig. 2.

Subscriber equipment 104 communicates with other equipment 130 via one or more communication network 110.Other equipment 130 comprise the computing equipment be associated with the user of the user being different from subscriber equipment 104, such as, and mobile phone, computing machine or there is any other electronic equipment of one or more processor, storer and communication interface.The example of one or more communication network 110 comprises LAN (Local Area Network) (LAN) and wide area network (WAN), such as the Internet.One or more communication network 110 uses any known procotol to implement alternatively, comprise various wired or wireless agreement, such as such as Ethernet, USB (universal serial bus) (USB), live wire (FIREWIRE), global system for mobile communications (GSM), enhanced data gsm environment (EDGE), CDMA (CDMA), time division multiple access (TDMA) (TDMA), bluetooth, Wi-Fi, Internet telephony agreement (VoIP), Wi-MAX or any other suitable communication protocol.

Server system 108 is implemented on the one or more free-standing data processing equipment or distributed network of computing machine.In certain embodiments, server system 108 also adopts the various virtual unit of third party's service provider (such as, third party cloud service provider) and/or service to provide bottom computational resource and/or the infrastructure resources of server system 108.

Although the digital assistants shown in Fig. 1 comprises client-side aspects (such as, DA-client 102) and server portion is (such as, DA-server 106) both, but in certain embodiments, the function of digital assistants is implemented as installation stand alone type application on a user device.In addition, the function between the client part of digital assistants and server section divides alterable in various embodiments.Such as, in certain embodiments, the every other function of digital assistants for only providing user oriented input and output processing capacity, and is delegated to the thin-client of back-end server by DA-client 102.

Fig. 2 is the block diagram of the subscriber equipment 104 according to some embodiments.Subscriber equipment 104 comprises memory interface 202, peripheral interface 206 and one or more processor 204.Various parts in subscriber equipment 104 are by one or more communication bus or signal wire coupling.Subscriber equipment 104 comprises various sensor, subsystem and is coupled to the peripherals of peripheral interface 206.The various functions of sensor, subsystem and peripherals Information Monitoring and/or promotion subscriber equipment 104.

Such as, motion sensor 210, optical sensor 212 and proximity transducer 214 are coupled to peripheral interface 206 to promote orientation, illumination and close to sensing function.Other sensors 216 one or more, such as positioning system (such as, gps receiver), temperature sensor, biometric sensors, gyroscope, compass, accelerometer etc., be also connected to peripheral interface 206, to promote correlation function.

In certain embodiments, camera sub-system 220 and optical sensor 222, for promoting camera function, are such as taken pictures and record video clips.Facilitate communication function by one or more wired and/or radio communication subsystem 224, described wired and/or radio communication subsystem can comprise various communication port, radio frequency receiver and transmitter and/or optics (as infrared) receiver and transmitter.Audio subsystem 226 is coupled to loudspeaker 228 and microphone 230, to promote the function of support voice, such as, and speech recognition, speech reproduction, digital recording and telephony feature.

In certain embodiments, I/O subsystem 240 is also coupled to peripheral interface 206.I/O subsystem 240 comprises touch screen controller 242 and/or other input control devices 244 one or more.Touch screen controller 242 is coupled to touch-screen 246.Touch-screen 246 and touch screen controller 242 such as can use in multiple touch sensitivity technology any one detect and contact and move or their interruption, described multiple touch sensitivity technology such as capacitance technology, resistive technologies, infrared technique, surface acoustic wave technique, proximity sensor arrays etc.Other input control devices 244 one or more can be coupled to other input/control devicess 248, such as one or more button, rocker switch, thumb roller, infrared port, USB port and/or pointing device (such as stylus).

In certain embodiments, memory interface 202 is coupled to storer 250.Storer 250 can comprise high-speed random access memory and/or nonvolatile memory, such as one or more disk storage device, one or more optical storage apparatus, and/or flash memories (such as NAND or NOR).

In certain embodiments, storer 250 stores operating system 252, communication module 254, graphical user interface module 256, sensor processing module 258, phone module 260 and application 262.Operating system 252 comprises for the treatment of basic system services and the instruction for performing hardware dependent tasks.Communication module 254 promotes the communication with one or more optional equipment, one or more computing machine and/or one or more server.Graphical user interface module 256 promotes graphical user interface process.Sensor processing module 258 promotes sensor relevant treatment and function.Phone module 260 promotes the process that phone is relevant and function.The various functions that application module 262 promotes user to apply, such as electronic information transmission, web page browsing, media processing, navigation, imaging and/or other processes and function.

As described in this description, storer 250 also stores client-side digital assistant instruction (such as, in digital assistants client modules 264) and various user data 266 is (such as, specific to lexical data, the preference data of user, and/or the electronic address list, do list, shopping list etc. of other data such as user) to provide the client-side function of digital assistants.

In various embodiments, digital assistants client modules 264 can by the various user interfaces of subscriber equipment 104 (such as, I/O subsystem 244) accept Speech input (such as, phonetic entry), Text Input, touch input and/or gesture input.Digital assistants client modules 264 can also provide the output of audio frequency (such as, voice output), video and/or tactile form.Such as, output can be provided as voice, sound, alarm, text message, menu, figure, video, animation, vibration and/or its combination above.During operation, digital assistants client modules 264 uses communication subsystem 224 and digital assistants server communication.

In certain embodiments, digital assistants client modules 264 utilize various sensor, subsystem and peripherals from the surrounding environment of subscriber equipment 104 gather additional information with set up with user, active user alternately and/or active user input the situation be associated.In certain embodiments, digital assistants client modules 264 contextual information or its subset are inputted with user together be provided to digital assistants server to help to infer the intention of user.In certain embodiments, digital assistants also uses contextual information to determine and how to prepare export and be sent to user.

In certain embodiments, the contextual information with user's input comprises sensor information, the image of such as illumination, neighbourhood noise, environment temperature, surrounding environment or video etc.In certain embodiments, contextual information also comprises the physical state of subscriber equipment 104, such as apparatus orientation, device location, device temperature, power level, speed, acceleration, motor pattern, cellular signal strength etc.In certain embodiments, also the information (such as operational process, installation procedure, past and current network activity, background service, error log, resource use etc.) relevant to the application state of subscriber equipment 104 is provided to digital assistants server as inputting with user the contextual information be associated.

In certain embodiments, digital assistants client modules 264 optionally provides in response to the request from digital assistants server the information (such as, user data 266) be stored on subscriber equipment 104.In certain embodiments, digital assistants client modules 264 also extracts when digital assistants server 106 is asked from the additional input of user via natural language dialogue or other user interfaces.Described additional input is sent to digital assistants server 106 by digital assistants client modules 264, carries out intent inference and/or meet the user view expressed in user's request to help digital assistants server 106.

In various embodiments, storer 250 comprises extra-instruction or less instruction.In addition, the various functions performing subscriber equipment 104 can be included in one or more signal transacting and/or special IC in hardware and/or firmware.

Fig. 3 A is the block diagram of the exemplary digital assistance system 300 according to some embodiments.In certain embodiments, digital assistant 300 is implemented in freestanding computer system.In certain embodiments, digital assistant 300 distributes across multiple computing machine.In certain embodiments, some in the module of digital assistants and function are divided into server section and client part, wherein client part is positioned at subscriber equipment (such as, subscriber equipment 104) go up and pass through one or more network and server section (such as, server system 108) communication, such as one or more network 110.In certain embodiments, digital assistant 300 is the embodiment of the server system 108 (and/or digital assistants server 106) shown in Fig. 1.It should be pointed out that digital assistant 300 is only an example of digital assistant, and this digital assistant 300 can have than more or less parts, two or more parts capable of being combined being shown, maybe can having the difference configuration of parts or layout.Various parts shown in Fig. 3 A at hardware, the software instruction for being performed by one or more processor, firmware (comprising one or more signal processing integrated circuit and/or special IC), or can realize in their combination.

Digital assistant 300 comprises storer 302, I/O (I/O) interface 306, network communication interface 308, and one or more processor 304.These parts are communicated with one another by one or more communication bus or signal wire 310.

In certain embodiments, storer 302 comprises non-transitory computer readable storage medium, such as high-speed random access memory and/or non-volatile memory medium (such as, one or more disk storage device, flash memory device or other non-volatile solid state memory equipment).

In certain embodiments, the input-output apparatus 316 of digital assistant 300 such as display, keyboard, touch-screen and microphone are coupled to Subscriber Interface Module SIM 322 by I/O interface 306.I/O interface 306, is combined with Subscriber Interface Module SIM 322, receives user's input (such as, phonetic entry, input through keyboard, touch input etc.) and correspondingly processes these inputs.In certain embodiments, digital assistant 300 comprises relative to any one (such as when digital assistants is implemented on free-standing subscriber equipment) in the parts described by the subscriber equipment 104 in Fig. 2 and I/O interface and communication interface.In certain embodiments, digital assistant 300 represents the server section that digital assistants is specifically implemented, and by being positioned at client-side aspects on subscriber equipment (subscriber equipment 104 such as, shown in Fig. 2) and user carries out alternately.

In certain embodiments, network communication interface 308 comprises wireless transmission and receiving circuit device 314 and/or one or more wired connection port 312.One or more wired connection port receives via one or more wireline interface such as Ethernet, USB (universal serial bus) (USB), live wire etc. and sends signal of communication.Wireless circuit device 314 receives RF signal and/or optical signalling from communication network and other communication facilitiess and RF signal and/or optical signalling is sent to communication network and other communication facilitiess.Radio communication can use any one in multiple communication standard, agreement and technology, such as GSM, EDGE, CDMA, TDMA, bluetooth, Wi-Fi, VoIP, Wi-MAX or any other suitable communication protocol.Network communication interface 308 makes to communicate via network (such as the Internet, Intranet and/or wireless network be cellular phone network, WLAN (wireless local area network) (LAN) and/or Metropolitan Area Network (MAN) (MAN) such as) between digital assistant 300, server system 108 and other equipment 130.

In certain embodiments, non-transitory computer readable storage medium storage program, module, the instruction and data structure of storer 302 or storer 302, comprise whole or its subset in following content: operating system 318, communication module 320, Subscriber Interface Module SIM 322, one or more application 324 and digital assistant module 326.One or more processor 304 performs these programs, module and instruction, and reads data from data structure or data are write data structure.

Operating system 318 (such as, Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS or embedded OS such as VxWorks) comprise for control and management General System task (such as, memory management, memory device control, electrical management etc.) various software part and/or driver, and promote various hardware, communication between firmware and software part.

Communication module 320 promotes to be communicated by network communication interface 308 between digital assistant 300, server system 108 and other equipment 130 (computing equipment such as, be associated with the user of the user being different from subscriber equipment 104).Such as, communication module 320 communicates with the communication interface 254 of the equipment 104 shown in Fig. 2 alternatively.Communication module 320 also comprises various parts, and these various parts are for the treatment of the data received by wireless circuit device 314 and/or wired connection port 312.

Subscriber Interface Module SIM 322 receives order from user's (such as, from keyboard, touch-screen, sensing equipment, controller and/or microphone) and/or input via I/O interface 306, and generating user interface object over the display.Subscriber Interface Module SIM 322 also prepares to export (such as, voice, sound, animation, text, icon, vibration, tactile feedback, illumination etc.) and it is sent to user via I/O interface 306 (such as, by display, voice-grade channel, loudspeaker and Trackpad etc.).

One or more application 324 comprises the program and/or module that are configured to be performed by described one or more processor 304.Such as, if digital assistant is implemented on free-standing subscriber equipment, then one or more application 324 comprises user's application, such as game, calendar application, navigation application or mail applications alternatively.If digital assistant 300 is implemented on server farm, then one or more application 324 comprises resource management applications, diagnostic application or dispatch application alternatively.

Storer 302 also stores digital assistant module 326 (or server section of digital assistants).In certain embodiments, digital assistant module 326 comprises following submodule or its subset or superset: I/O processing module 328, voice are larded speech with literary allusions this (STT) processing module 330, natural language processing module 332, dialogue stream processing module 340, task flow processing module 342, service processing module 346 and respond module 350 automatically.Each in these modules all has the access rights to one or more or its subset in the data of following digital assistants 326 and model or superset: body 334, glossarial index 336, user data 338, task flow model 344, service model 348 and user journal 360.

In certain embodiments, be used in processing module, data and model that digital assistant module 326 implements, digital assistant 300 performs at least some in following content: the intention identifying the user expressed the natural language input received from user; Draw energetically and obtain the information (such as, by eliminating the ambiguity of words, title, purpose etc.) needed for the intention for fully inferring user; Determine the task flow of the intention realizing inferring; And stream of executing the task is to realize the intention inferred.In certain embodiments, when for various reasons, when not having or can not provide satisfied response to user, digital assistants also takes suitable action.

In certain embodiments, as shown in Figure 3 B, I/O processing module 328 by the I/O equipment 316 in Fig. 3 A and user interactions or by the network communication interface 308 in Fig. 3 A and subscriber equipment (such as, subscriber equipment 104 in Fig. 1) alternately to obtain user's input (such as, phonetic entry) and the response (such as, as voice output) inputted user is provided.I/O processing module 328 optionally obtains soon together or after the user input is received and inputs with the user from subscriber equipment the contextual information be associated in company with receiving user's input.Contextual information comprises data, vocabulary specific to user, and/or inputs relevant preference to user.In certain embodiments, contextual information is also included in the software and hardware state of equipment when user asks that receives (such as, the subscriber equipment 104 in Fig. 1), and/or the information relevant to the surrounding environment of the user when receiving user and asking.In certain embodiments, I/O processing module 328 also sends to user and asks relevant follow-up problem with user, and receives answer from user.When user asks to be received by I/O processing module 328 and user's request comprises phonetic entry, phonetic entry is forwarded to STT processing module 330 to carry out speech text conversion by I/O processing module 328.

STT processing module 330 receives phonetic entry (user spoken utterances such as, caught in voice record) by I/O processing module 328.In certain embodiments, STT processing module 330 uses various sound model and language model phonetic entry to be identified as the sequence of phoneme, and it is identified as the sequence with the words of one or more written or symbol the most at last.Voice turn text processing module 330 can use any suitable speech recognition technology, sound model and language model such as hidden Markov model, implemented based on the speech recognition of dynamic time warping (DTW) and other statistics and/or analytical technology.In certain embodiments, voice present treatment of larding speech with literary allusions can perform by third party's service execution or on the equipment of user at least in part.To lard speech with literary allusions the result of present treatment once STT processing module 330 obtains voice, the sequence of such as words or symbol, result is just sent to natural language processing module 332 to carry out intent inference by STT processing module 330.

Dependent voice lard speech with literary allusions present treatment more details on September 20th, 2011 submit to, the U.S. patent Nos patent application serial numbers the 13/236th that is entitled as " ConsolidatingSpeechRecognitionResults ", described by having in No. 942, its whole disclosure is incorporated to herein by reference.

The natural language processing module 332 (being sometimes also called herein " natural language processor ") of digital assistants obtains the sequence (" or symbol sebolic addressing ") of words or the symbol generated by STT processing module 330, and attempts this symbol sebolic addressing to be associated with one or more " can perform intention " identified by digital assistants.Expression can be performed by digital assistants and have the task of the task flow be associated implemented in task flow model 344 " can to perform intention ".The task flow that is associated is the action a series of by programming and step that digital assistants is taked to execute the task.The limit of power of digital assistants depends on to be implemented and the value volume and range of product of the task flow stored in task flow model 344, or in other words, depends on the value volume and range of product of " can perform intention " that digital assistants identifies.But the validity of digital assistants also depends on that digital assistants is from the ability to infer correct " one or more perform intention " the user of natural language expressing request.

In certain embodiments, except the symbol sebolic addressing obtained from STT processing module 330, natural language processor 332 also receives the contextual information of asking with user to be associated, such as, from I/O processing module 328.Natural language processor 332 optionally uses contextual information come clear and definite, supplementary and/or be defined in the information comprised from the symbol sebolic addressing of STT processing module 330 reception further.Contextual information comprises such as user preference, the hardware of subscriber equipment and/or application state, before user's request, period or the sensor information of collecting soon afterwards, previously mutual (such as, talking with) between digital assistants and user etc.

In certain embodiments, natural language processing is based on body 334.Body 334 for comprising the hierarchy of many nodes, each node represents " can perform intention " or to one or more relevant " attribute " in " can intention be performed " or other " attribute ".As mentioned above, task that " can perform intention " representative digit assistant can perform, that is, this task is for " executable " or can be carried out." attribute " representative is intended to or parameter that the sub-aspect of another attribute is associated with can perform.The connection that can perform in body 334 between intention node and attribute node defines the parameter represented by attribute node and how to be subordinated to by performing the task of being intended to node on behalf.

In certain embodiments, body 334 by can perform intention node and attribute node form.In body 334, each intention node that performs is connected directly to or is connected to one or more attribute node by one or more middle attribute node.Similarly, each attribute node is connected directly to or is connected to by one or more middle attribute node and one or morely performs intention node.Such as, as shown in Figure 3 C, body 334 comprises " dining room reservation " node (that is, can perform intention node) alternatively.Attribute node " dining room ", " date/time " (for reservation) and " colleague's number " are all connected directly to perform and are intended to node (that is, " dining room reservation " node).In addition, attribute node " style of cooking ", " price range ", " telephone number " and " position " are the child nodes in attribute node " dining room ", and are all connected to " dining room reservation " node by middle attribute node " dining room ".And for example, as shown in Figure 3 C, body 334 also can comprise " setting is reminded " node (that is, another can perform intention node).Attribute node " date/time " (reminding for setting) and " theme " (for prompting) are all connected to " setting is reminded " node.Due to attribute " date/time " to carry out the task of dining room reservation and set both tasks of reminding relevant, therefore attribute node " date/time " is connected to " dining room reservation " node in body 334 and " setting prompting " node both.

The attribute node that intention node connects together with it can be performed, be sometimes described in " territory ".In this discussion, each territory is associated with corresponding performed intention, and relates to and perform with specific the group node (and the relation between them) being intended to be associated.Such as, the body 334 shown in Fig. 3 C is included in the example of the dining room subscribing domain 362 in body 334 and reminds the example in territory 364.Dining room subscribing domain comprises can perform intention node " dining room reservation ", attribute node " dining room ", " date/time " and " colleague's number " and sub-attribute node " style of cooking ", " price range ", " telephone number " and " position ".Remind territory 364 to comprise can to perform intention node and " prompting is set " and attribute node " theme " and " date/time ".In certain embodiments, body 334 is made up of multiple territory.One or more attribute node is shared with other territories one or more alternatively in each territory.Such as, except dining room subscribing domain 362 and prompting territory 364, " date/time " attribute node is associated with many not same areas (such as, routing territory, travel reservations territory, film ticket territory etc.) alternatively.

Although Fig. 3 C illustrates two example domain in body 334, other territories (maybe can perform intention) comprise such as " telephone calling ", " search direction ", " arrangement meeting ", " transmission message " and " answer of problem is provided ", etc." transmission message " territory and " transmission message " can perform and be intended to node and be associated, and comprise attribute node such as " one or more recipient ", " type of message " and " message text " alternatively.Attribute node " recipient " is such as defined by sub-attribute node such as " recipient's name " and " message addresses " alternatively further.

In certain embodiments, body 334 comprises digital assistants and can to understand and to its all territories of working (and thus can perform intention).In certain embodiments, body 334 alternatively such as by adding or removing whole territory or node, or is revised by the relation between the node in amendment body 334.

In certain embodiments, alternatively by multiple relevant perform the node cluster that is intended to be associated " the super territory " in body 334 under.Such as, " travelling " super territory comprises with the relevant attribute node of travelling alternatively and can perform and be intended to trooping of node.The performed intention node relevant with travelling comprises " plane ticket booking ", " hotel reservation ", " automobile leasing ", " route planning ", " searching point of interest " alternatively, etc.Performed intention node under same super territory (such as, " travelling " super territory) has multiple shared attribute node sometimes.Such as, one or more in attribute node " reference position ", " destination ", " sailing date/time ", " date of arrival/time " and " colleague's number " is sometimes shared for the performed intention node of " plane ticket booking ", " hotel reservation ", " automobile leasing ", " route planning ", " searching point of interest ".

In certain embodiments, each node in body 334 is intended to one group of relevant words and/or phrase is associated with following maybe can be performed by the attribute of node on behalf.The words of the respective sets be associated with each node and/or phrase are so-called be associated with node " vocabulary ".The words of the respective sets be associated with each node and/or term storage maybe can be able to performed in the glossarial index 336 being intended to be associated with the attribute representated by node.Such as, return Fig. 3 B, the vocabulary be associated with the node of " dining room " attribute comprises words such as " cuisines ", " drinks ", " style of cooking ", " hunger ", " eating ", " Pizza ", snack food, " meals " etc. alternatively.And for example, the vocabulary be associated with the node of the performed intention of " telephone calling " comprise alternatively words and phrase such as " calling ", " making a phone call ", " dialing ", " with ... take on the telephone ", " calling out this number ", " phoning " etc.Glossarial index 336 optionally comprises words and the phrase of different language.

Natural language processor 332 from STT processing module 330 receiving symbol sequence (such as, text string), and determines which node the word in symbol sebolic addressing involves.In certain embodiments, if the words in discovery symbol sebolic addressing or phrase are associated with the one or more nodes in body 334 (via glossarial index 336), then described words or phrase are by " triggering " or " activation " those nodes.Based on the quantity and/or the relative importance that activate node, selection one of can perform in intention can be performed the task of being intended to as user view, digital assistants be performed by natural language processor 332.In certain embodiments, the territory with " triggering " node is at most selected.In certain embodiments, the territory of most high confidence level (such as, based on the relative importance of its each trigger node) is selected to have.In certain embodiments, based on territory is selected in the quantity of trigger node and the combination of importance.In certain embodiments, in the process selecting node, also consider additive factor, whether correctly the previous decipher of such as digital assistants from the similar request of user.

In certain embodiments, digital assistants also by the name storage of special entity in glossarial index 336, make when a title in these titles being detected in user's request, natural language processor 332 can identify the particular example that this title relates to attribute or sub-attribute in body.In certain embodiments, the title of special entity is the title of enterprise, dining room, people, film etc.In certain embodiments, digital assistants is searched for and is identified special entity title from other data sources, the address list of other data sources described such as user, movie database, musician's database and/or dining room database.In certain embodiments, when the words that natural language processor 332 identifies in symbol sebolic addressing is title (title in such as user communication record) of special entity, be to select in the body that user asks can perform in the process of intention, for additional importance given in this words.

Such as, when identifying words " Mr. Santo " and find that in glossarial index 336 surname " Santo " is the contact person's a period of time in the address book of user in user's request, then user's request probably corresponds to " transmission message " or " telephone calling " territory.And for example, when finding word " ABC coffee-house " and find that in glossarial index 336 term " ABC coffee-house " is the title of particular restaurant in city, user place in user's request, then user's request may correspond to " dining room reservation " or " discovery dining room " territory.

User data 338 comprises the information specific to user, such as specific to the address book of the default language of the vocabulary of user, user preference, station address, home location, user and second language, user, and other short-terms of every user or long-term information.In certain embodiments, natural language processor 332 uses the information specific to user to carry out in replenish user input the information that comprises to limit user view further.Such as, " my friends is invited to participate in my birthday party " for user's request, natural language processor 332 can calling party data 338 to determine that " friend " is that who and " birthday party " will in when and where holding, and not need user to provide this type of information clearly in its request.

Based on symbol string search body other details on Dec 22nd, 2008 submit to, the U.S. patent Nos patent application serial numbers the 12/341st that is entitled as " MethodandApparatusforSearchingUsingAnActiveOntology ", described by having in No. 743, its whole disclosure is incorporated to herein by reference.

In certain embodiments, once natural language processor 332 identifies based on user's request can perform intention (or territory), natural language processor 332 just generating structured is inquired about to represent identified performed intention.In certain embodiments, structuralized query comprises the parameter for one or more nodes that can perform in the territory of intention, and at least some parameter in described parameter is filled with the customizing messages and requirement of specifying in user's request.Such as, user " helps me to subscribe the seat of 7 in evening in sushi shop to DA statement." in this case, natural language processor 332 can input based on user and can perform intention and be correctly identified as " dining room reservation ".According to body, the structuralized query in " dining room reservation " territory comprises parameter such as { style of cooking }, { time }, { date }, { colleague's number } etc. alternatively.In certain embodiments, based on the information be included in user spoken utterances, natural language processor 332 is for dining room subscribing domain generating portion structuralized query, and wherein partial structured inquiry comprises parameter { style of cooking=sushi class } and { time=evening 7 point }.But in this example, user spoken utterances comprises the information being not enough to the structuralized query be associated with territory.Therefore, based on current available information, in structuralized query, do not specify other call parameters such as { colleague's number } and { date }.In certain embodiments, natural language processor 332 carrys out some parameters of interstitital textureization inquiry by the contextual information received.Such as, in certain embodiments, if request " neighbouring " sushi shop, natural language processor 332 be used for from the gps coordinate of subscriber equipment 104 come interstitital textureization inquiry in { position } parameter.

In certain embodiments, structuralized query (comprising any completed parameter) is sent to task flow processing module 342 (being sometimes also called " task flow processor ") herein by natural language processor 332.Task flow processing module 342 is configured to receive the structuralized query from natural language processor 332, completes structuralized query (if desired), and execution " completing " user finally asks required action.In certain embodiments, complete the necessary various process of these tasks to provide in task flow model 344.In certain embodiments, task flow model 344 comprises the process for obtaining the additional information from user, and for performing and the task flow being intended to the action be associated can being performed.

As mentioned above, in order to complete structuralized query, task flow processor 342 needs to initiate, with the additional dialogue of user, to obtain additional information, and/or to understand fully the language that may have ambiguity sometimes.When this type of is necessary alternately, task flow processor 342 calls conversation process processing module 340 (being sometimes also called " dialog processor ") herein to carry out the dialogue with user.In certain embodiments, how dialog processor 340 is determined (and/or when) to user's query additional information, and receive and process user response.By I/O processing module 328 problem be supplied to user and receive answer from user.In certain embodiments, dialog processor 340 presents dialogue via audio frequency and/or video frequency output to user and exports, and receives the input from user responded via oral or physics (such as, clicking).Continue example above, when task flow processor 342 calls dialogue stream processor 334 to determine " colleague's number " and " date " information for the structuralized query be associated with territory " dining room is subscribed ", dialog processor 340 Generating Problems such as " total how many people's dining? " " specifically when having dinner? " to be sent to user.Once receive the answer from user, the available missing information interstitital textureization inquiry of dialog processor 340, or information is passed to task flow processor 342 to complete missing information according to structuralized query.

Once task flow processor 342 completes structuralized query for performing intention, task flow processor 342 just continues perform and can perform the final task being intended to be associated.Therefore, task flow processor 342 to be executed the task step in flow model and instruction according to the special parameter comprised in structuralized query.Such as, comprise alternatively for contacting dining room and in fact asking at special time for the step of the reservation of specific colleague's number and instruction for the task flow model that can perform intention " dining room reservation ".Such as, by using structuralized query such as: { dining room is subscribed, dining room=ABC coffee-house, date=2012/3/12, time=afternoon 7 point, colleague number=5 people }, task flow processor 342 performs following steps alternatively: (1) signs in the server of ABC coffee-house or dining room reservation system such as (2) inputting date, time and the number information that goes together in the form on website, (3) submit form to, and (4) make calendar for this reservation in user's calendar.

In certain embodiments, task flow processor 342 is answered the assist task of asking in lower completing user input or the informedness of the asking in providing user to input of service processing module 346 (being sometimes also called herein " service processor ").Such as, service processor 346 can represent task flow processor 342 telephone calling, arranges calendar, invocation map search, other users application that invoke user equipment is installed or apply alternately with described other, and call third party's service (such as, dining room subscribe portal website, social network sites, banking portal site etc.) or with third party's service interaction.In certain embodiments, by the respective service model in service model 348 specify every serve needed for agreement and application programming interface (API).Service processor 346 for the suitable service model of service access, and generates the request for this service according to the agreement of service model needed for this service and API.

Such as, if online booking service has been enabled in dining room, then dining room can submit service model to, and this service model specifies the call parameter carrying out subscribing and the API value of call parameter being sent to online booking service.When being asked by task flow processor 342, service processor 346 can use the web address be stored in service model to set up the network of serving with online booking to be connected, and the call parameter (such as, time, date, colleague's number) subscribed is sent to online booking interface with the form of the API served according to online booking.

In certain embodiments, natural language processor 332, dialog processor 340 and task flow processor 342 jointly and use repeatedly to infer and limit the intention of user, acquisition information further to define and refine user view and finally generate response (such as, export user to, or finish the work) to meet the intention of user.

In certain embodiments, meet after user asks required all tasks in executed, digital assistants 326 formulates confirmation response, and by I/O processing module 328, this response is sent to user.If user asks seek information to be answered, then confirmation response presents asked information to user.In certain embodiments, digital assistants also asks the response satisfaction whether user's indicating user produces digital assistants 326.

Natural language processor 332 receives the symbol sebolic addressing corresponded to from the phonetic entry of subscriber equipment 104 (Fig. 1) from STT processing module 330.In certain embodiments, natural language processor 332 infers that phonetic entry comprises for importing to from one or more the follow-up of appointment sender the instruction communicating and carry out from dynamic response into.As following about Fig. 4 discuss in detail, when instruction at least comprise (1) one or more appointment sender and (2) required movement time, the task flow that instruction triggers is associated with the automatic response field 400 of body 334.In some other embodiments, natural language processor 332 infers that the corresponding territory different from automatic response field 400 is triggered, and digital assistants 326 proceeds the task flow model be associated with corresponding territory (such as, dining room discussed above subscribing domain).

In certain embodiments, digital assistants 326 comprises automatic respond module 350.In certain embodiments, automatic respond module 350 is parts of task flow processor 342.In some other embodiments, automatic respond module 350 is separated with task flow processor 342.Automatic respond module 350 is configured to perform step asks with the user of the automatic response field 400 of satisfied triggering.In other words, automatic respond module 350 is configured to perform the task flow relevant with automatic response field 400.

After natural language processor 332 determines that automatic response field 400 is triggered, automatic respond module 350 follow-up imports communication in response to specifying sender to receive from one or more appointment sender, instruction is obtained from natural language processor 332, and perform required movement, or required movement is performed.In certain embodiments, automatic respond module 350 comprises designated module 352, determination module 354, writes module 356 and execution module 358.

Instruction module 352 is configured in storaged voice input included for importing the instruction communicating and respond into.The instruction stored at least comprises required movement and one or more appointment sender.In certain embodiments, the instruction stored comprises one or more designated communication type, alternatively for the expiration period of instruction and/or the condition precedent (conditionprecedent) of execution instruction.In certain embodiments, instruction module 352 comprises the storer for storing the instruction received from user.In some other embodiments, in the storer that instruction module 352 is separated in and instruction module 352, store instruction.In certain embodiments, when (namely, instruction) when being included in phonetic entry, instruction module 352 is also configured to storage for the expiration period (or duration) (such as, in following two days or by the end of some this afternoon 5) of instruction and the condition precedent performing required movement.Instruction module 352 is also configured at execution required movement or expiration period past tense, automatic delete instruction.In certain embodiments, when there is no human intervention or instruction, at execution or expiration period past tense, immediately delete instruction.

In certain embodiments, subscriber equipment 104 to receive from other equipment other equipment 130 be associated with the user of the user being different from subscriber equipment 104 via one or more communication network 110 and imports communication into.In certain embodiments, determination module 354 is configured to judge that whether follow-up sender (sender of such as, call, SMS or Email) of importing communication into is that in one or more appointment senders indicated in instruction one specifies senders.In one embodiment, determination module 354, at least in part based on the address book of user, judges whether the follow-up sender of importing communication into is one or more appointment senders specified in sender.Such as, the telephone number in the address corresponding to the telephone number with the user corresponding to designated user that import communication into compares by determination module 354.In another embodiment, determination module 354, at least in part based on the caller-identification information be associated with the follow-up sender of importing communication into, judges whether the follow-up sender of importing communication into is one or more appointment senders specified in sender.In another embodiment, determination module 354 identifies the e-mail address (or other unique identifiers) that communicate with follow-up importing into (such as voip call or the communication based on other the Internets) is associated, and at least in part based on identified e-mail address, judge whether the follow-up sender of importing communication into is one or more appointment senders specified in sender.

In a further embodiment, determination module 354 based on to the follow-up aftertreatment of importing communication into, judges whether the follow-up sender of importing communication into is one or more appointment senders specified in sender at least in part.Such as, determination module 354 is based on name, telephone number, e-mail address or stay other identification informations in voice mail or speech message (such as, " hello; I is JohnWhite; please call back to me "), determine or infer the follow-up identity importing the sender of communication into.In another example, determination module 354 determines the follow-up identity importing the sender of communication into based on the name in the signature boxes of Email.In another example, determination module 354 is based on name, telephone number, e-mail address or stay other identification informations in SMS (such as, " hello; I is JackWhite; write in reply the performance address of tonight to me "), determine the follow-up identity importing the sender of communication into.

In certain embodiments, determination module 354 is also configured to determine whether follow-up communication of importing into is the communication type of being specified by instruction.Such as, user is to DA statement " if mother transmits, then write in reply say that I on the way home to her ".In this example, only when receiving text message or SMS from mother, just perform required movement (such as, transmitting to mother).In another example, user to DA statement " if my wife calls me, allow she know I in gymnasium and will will return home very much ".In this example, only when receiving call from the wife of user, just perform required movement, such as, user just will will return home in gymnasium to allow the wife of user know very much.

In certain embodiments, when instruction instruction expiration period, determination module 354 is further configured to determine whether to receive in expiration period and follow-up imports communication into.In certain embodiments, when instruction instruction is for performing the condition precedent of required movement, determination module 354 be further configured to determine to continue after the receipt import communication into before or after the receipt continuous import communication into while whether meet condition precedent.

Write module 356 to be configured to write the follow-up response of importing communication into based on instruction included in phonetic entry.In certain embodiments, writing module 356 when writing response, utilizing contextual information, the position be such as associated with user, the pattern in past or behavior or calendar.Such as, suppose that the user of subscriber equipment 104 requires that DA " if my wife contacts me, transmits and says ' I on the way home ' to her ".In this example, (such as, based on the program of in program described above) determine from the wife of user receive follow-up import communication into after, digital assistants, at least in part based on the instruction be associated with user and positional information, writes the text message of the wife to user.Such as, state that " dear, hello by writing the text message that module 356 writes.I according to the current magnitude of traffic flow, should can get home on the way home for about 15 minutes ".In this example, 15 minute period was estimated based on the position of the current location of user, the family of user and/or current traffic flow condition by digital assistants.

In certain embodiments, execution module 358 is configured to, after determination module 354 determines that the follow-up sender of importing communication into is a sender in one or more appointment sender, perform the action of being specified by instruction.In some other embodiments, execution module 358 is configured to, after determination module 354 determines that follow-up importing into communicates as a designated communication type in one or more designated communication type, the action of being specified by instruction is performed.Such as, required movement sends verbal messages to sender, SMS or text message is sent to sender, MMS is sent to sender, transfer immediate news to sender, Email is sent to sender, some other forms of electronics or non electronic communications are sent to sender, event or calendar invitation is sent to sender, transmission or sharing media content are (such as, audio file, video segment, photo, URL etc.), one or more calendar or appointment entries is shared with sender, or share other custom attributes one or more (such as with sender, information defined by the user or the sundry item of media content).In certain embodiments, execution module is configured to use and carries out default response with follow-up importing into the identical means of communication that communicates to follow-up communication of importing into.

The more details of Values assistant are found in U.S. patent Nos application the 12/987th that on January 18th, 2010 submits to, that be entitled as " IntelligentAutomatedAssistant ", No. 982, on June 3rd, 2011 submit to, the U.S. patent Nos application the 61/493rd that is entitled as " GeneratingandProcessingDataItemsThatRepresentTaskstoPerf orm ", in No. 201, its whole disclosure is incorporated to herein by reference.

Fig. 4 illustrates the automatic response field 400 (herein sometimes also referred to as " from dynamic response " movable intention) of the body 334 be associated with movable intention 402, and movable intention 402 follow-up imported communication in response to specifying from one or more appointment sender sender to receive into and automatically performed required movement.

Sender's attribute node 404 is specified to indicate the follow-up sender of importing communication into.In certain embodiments, sender's node 404 is specified to comprise the sub-attribute node of mark appointment sender, the name in the telephone directory of such as user, telephone number, e-mail address etc.Required movement node 406 indicate from specify sender receive follow-up import communication into time the action that will perform.In certain embodiments, required movement node 406 comprises the details 414 (or information) that mark will be included in dynamic response and the type of action 416 that will be performed.

In certain embodiments, details child node 414 comprises the spoken words from the text of phonetic entry or record, the digitized version from the Text Input of user or text-to-speech version and/or the current location for being included in the subscriber equipment 104 (Fig. 1) in dynamic response.In certain embodiments, if the request of user comprises details or the instruction of the deficiency for writing automatic response message, then automatically respond module 350 (Fig. 3 A) is configured such that dialog process module 340 (Fig. 3 A) initiates the dialogue with user, to draw additional details for writing complete automatic response message or instruction.In some other embodiments, if the request of user comprises details or the instruction of the deficiency for writing automatic response message, then respond module 350 (Fig. 3 A) is configured such that writing module 356 (Fig. 3 A) completes any unessential details do not comprised in instruction, to write automatic response message automatically.Such as, writing module 356 (Fig. 3 A) utilizes the use of the contextual information be associated with user (such as, the telephone directory of user, positional information, pattern or behavioural information, user journal or calendar) to write complete automatic response message.

In certain embodiments, type of action child node 416 specifies the type of the action that will be performed, such as, to/with appointment sender send/share voice mail or speech message, SMS or text message, MMS, Email, event invitation, media content, one or more calendar or one or more custom attributes.In certain embodiments, if the request statement DA of user will with the means of specifying sender's communication information still not have designated communication, then use the means of communication the most often using or give tacit consent to for response message.

Such as, the user of subscriber equipment 104 is to DA statement " if my father and mother attempt to contact me, then transmit and tell them address in my new apartment, and allow them specify me to call back them ".In this case, because phonetic entry comprises one or more appointment sender (such as, anyone in the father and mother of user) and the required movement that will be performed is (such as, send text message), so movable intention is designated " from dynamic response " movable intention 402 by natural language processor 332 (Fig. 3 D).According to body 334, the structuralized query for automatic response field 400 comprises such as { specifies sender 404}, { details 414} and the { parameter of type of action 416}.Such as, use structuralized query, such as { from dynamic response, specify sender=father and mother, the address in the apartment of details=user and " will call back ", type of action=SMS or text message }, task flow processor 342 (Fig. 3 A) or its parts perform following steps: instruction included in the input of (1) storaged voice; (2) receive and follow-up import communication (such as, text message or call) into; (3) determine that the follow-up sender of importing communication into specifies sender (the arbitrary individuals in the father and mother of such as, user); (4) according to the details provided in instruction (or information) write response text message (such as, " and my address is No. 123, Jia Nifuniya state Sen Niweier city Ai Erwen forest track, 94086.I will call back to you at one's leisure at me "); (5) text message is made to be sent to a people in the father and mother of user or two people; And (6) are making after text message is sent out, automatic delete instruction.

Expiration period node 408 is specified and must be received the follow-up time period of importing communication into from one or more appointment sender.Such as, the user of subscriber equipment 104 to DA statement " if Gary attempted to contact me in following 30 minutes, by I today can future time neutral send e-mails to him for meet ".In this case, because phonetic entry comprises one or more appointment sender (such as, Gary) and the required movement that will be performed (such as, send the face time of suggestion), so movable intention is designated " from dynamic response " by natural language processor 332 (Fig. 3 A) action can be intended to 402.According to body 334, the structuralized query for automatic response field 400 comprises parameter, such as { specifies sender 404}, { details 414}, { type of action 416} and { expiration period 408}.Such as, use structuralized query, such as { from dynamic response, specify sender=Gary, details=next available face time, type of action=Email, expiration period=30 minute }, task flow processor 342 (Fig. 3 A) or its parts perform following steps: instruction included in the input of (1) storaged voice; (2) receive and follow-up import communication (such as, text message, call or Email) into; (3) determine whether the follow-up sender of importing communication into is specify sender (such as, Gary); (4) determine follow-up to import communication whether received in expiration period (such as, in 30 minutes that receive instruction) into; (5) according to the respond with e-mails of always writing day of the details provided in instruction (or information) and user, (such as, " hello for Gary.My schedule indicates my next visible face time to be some this afternoon 1 (America eastern time) "); (6) Email is made to be sent to Gary; And (7) are making after Email is sent out, automatic delete instruction.

The condition that condition node 410 must meet before specifying in and performing required movement.Such as, the user of subscriber equipment 104 " if Suzie attempts to contact me, when only HALF MOON BAY is more than 75 ° when she contacts me, sends the direction from her family address to HALF MOON BAY to her " to DA statement.In this case, because phonetic entry comprises one or more appointment sender (such as, Suzie) and the required movement that will be performed (such as, sending direction), so movable intention is designated " from dynamic response " movable intention 402 by natural language processor 332 (Fig. 3 A).According to body 334, the structuralized query for automatic response field 400 comprises parameter, such as { specifies sender 404}, { required movement 406}, { details 414} and { condition precedent 410}.Such as, use structuralized query, such as { from dynamic response, specify sender=Suzie, required movement=sending direction, details=from Suzie family address to the direction of HALF MOON BAY, the temperature of the HALF MOON BAY of condition precedent=when Suzie contacts me is more than 75 ° }, task flow processor 342 (Fig. 3 A) or its parts perform following steps: instruction included in the input of (1) storaged voice; (2) receive and follow-up import communication (such as, text message, call or Email) into; (3) determine whether the follow-up sender of importing communication into is specify sender (such as, Suzie); (4) determine to continue upon receipt import communication into time whether meet condition precedent (such as, the temperature of HALF MOON BAY is more than 75 °); (5) direction from Suzie family address to HALF MOON BAY is made to be sent to Suzie (that such as, use acquiescence or the most frequently used means of communication); And (6) are making after direction is sent out, automatic delete instruction.

Specify import into communication type node 412 indicate follow-up import communication into must be received to perform the means of communication of required movement by it.In certain embodiments, the child node imported communication type node 412 into and comprise identification communication means is specified, such as SMS, text message, call, Email, VoIP, video call etc.Such as, the user of subscriber equipment 104 is to DA statement " if Varun transmits to me, then I will reach the standard grade over the course of 15 mins to allow him know ".In this case, because phonetic entry comprises one or more appointment sender (such as, Varun) and the required movement that will be performed (such as, send the message having information), so movable intention is designated " from dynamic response " by natural language processor 332 (Fig. 3 A) action can be intended to 402.According to body 334, the structuralized query for automatic response field 400 comprises parameter, such as { specifies sender 404}, { required movement 406}, { details 414}, { communication type 412} is imported in appointment into.Such as, use structuralized query, such as { from dynamic response, specify sender=Varun, required movement=transmission has the message of information, details=user will reach the standard grade over the course of 15 mins, specify and import communication type=SMS or text message into }, task flow processor 342 (Fig. 3 A) or its parts perform following steps: instruction included in the input of (1) storaged voice; (2) receive and follow-up import communication (such as, text message, call or Email) into; (3) determine whether the follow-up sender of importing communication into is specify sender (such as, Varun); (4) determine whether follow-up communication of importing into is specify to import communication type (such as, SMS or text message) into; (5) response message (such as, " hello for Varun, and I will reach the standard grade over the course of 15 mins ") is write according to the details provided in instruction (or information); (6) message of information is made to be sent to David (that such as, use acquiescence or the most frequently used means of communication); And (7) are making after the message of information is sent out, automatic delete instruction.

Fig. 5 illustrates the process flow diagram of the method 500 of the operand word assistant according to some embodiments.In certain embodiments, electronic equipment (such as, corresponding DA-client 102 or DA-server 106 in Fig. 1) place's manner of execution 500 of one or more processor and storer is being comprised.In certain embodiments, illustrated in Fig. 3 A-3B digital assistants 326 place manner of execution 500.In certain embodiments, by store in storer (such as, non-transitory computer readable storage medium), the instruction set management method 500 that performed by the one or more processors of electronic equipment.

Electronic equipment (such as, subscriber equipment 104) receives phonetic entry (such as, Fig. 3 A, receives phonetic entry at I/O processing module 328 place) from the user of subscriber equipment 104.Such as, user is to digital assistants (DA) statement " if Christina transmits, then I is driving and am inciting somebody to action very near that to allow her know ".On the contrary, I/O processing module 328 (Fig. 3 A) sends phonetic entry to STT processing module 330 (Fig. 3 A).After processed voice input, STT processing module 330 (Fig. 3 A) sends the symbol sebolic addressing representing phonetic entry to natural language processor 332 (Fig. 3 A).

Natural language processor 332 receives (502) symbol sebolic addressing from STT processing module 330, and attempts symbol sebolic addressing to be associated with the one or more movable intention by digital assistants identification.

Natural language processor 332 determines whether (504) phonetic entry comprises for importing to from one or more the follow-up of appointment sender the instruction communicating and carry out from dynamic response into.When instruction at least comprises (1) one or more appointment sender (such as, Christina) and (2) required movement (such as, have the response of information), the task flow that instruction triggers is associated with the automatic response field 400 of body 334.

Determining that phonetic entry does not comprise for when importing instruction that communication carries out from dynamic response into from one or more appointment sender follow-up ("No" branch), digital assistants performs (506) and performs other processes one or more be associated with the task flow being different from the task flow be associated with automatic response field 400 (such as, above discussed dining room subscribing domain).

Determining that phonetic entry comprises for when importing instruction that communication carries out from dynamic response into from one or more appointment sender follow-up ("Yes" branch), whether instruction that digital assistants is determined (508) comprises the expiration period for instruction.

When determining the expiration period that instruction comprises for instruction ("Yes" branch), digital assistants stores (510) expiration period.Such as, during the expiration period for instruction to be stored in instruction module 352 in included storer or to be stored in storer that and instruction module 352 is separated by instruction module 352 (Fig. 3 A).

After storing expiration period ("Yes" branch) or when determining the expiration period that instruction does not comprise for instruction ("No" branch), whether instruction that digital assistants is determined (512) comprises the condition precedent for performing required movement.

When determining that instruction comprises condition precedent ("Yes" branch), digital assistants stores (514) condition precedent.Such as, during condition precedent to be stored in instruction module 352 in included storer or to be stored in storer that and instruction module 352 is separated by instruction module 352 (Fig. 3 A).

After storage condition precedent ("Yes" branch) or when determining that instruction does not comprise condition precedent ("No" branch), instruction included in the phonetic entry that digital assistants is determined (516) receives whether complete (that is, instruction comprises enough details for performing required movement or information).

When determining that instruction is imperfect ("No" branch), automatic respond module 350 makes (518) dialog process module 340 (Fig. 3 A) initiate with the dialogue of user to collect complete details or instruction, to perform required movement.Such as, user requires that DA " if Tony makes a phone call, sends out mail and tells him Movie show times ".In this example, user fails to indicate concrete movie name and the date or time scope for show time.In this case, automatic respond module 350 makes dialog process module 340 initiate the dialogue with user, to determine that user wants to send to movie name and the date of the show time of Tony.

When determining that instruction is complete ("Yes" branch), digital assistants stores the instruction that (520) at least comprise one or more appointment sender and required movement.Such as, during one or more appointment sender and required movement to be stored in instruction module 352 in included storer or to be stored in storer that and instruction module 352 is separated by instruction module 352 (Fig. 3 A).

Fig. 6 illustrates the process flow diagram of the method 600 of the operand word assistant according to some embodiments.In certain embodiments, electronic equipment (such as, corresponding DA-client 102 or DA-server 106 in Fig. 1) place's manner of execution 600 of one or more processor and storer is being comprised.In certain embodiments, illustrated in Fig. 3 A-3B digital assistants 326 place manner of execution 600.In certain embodiments, by store in storer (such as, non-transitory computer readable storage medium), the instruction set management method 600 that performed by the one or more processors of electronic equipment.

Storing, being used for communicates to follow-up the importing into from one or more appointment sender carries out after the instruction of dynamic response, and electronic equipment (such as, Fig. 1, subscriber equipment 104) receives (602) from corresponding sender and imports communication into.Such as, importing communication into is in the following one: Email, call, video call, voip call, SMS or text message, MMS, speech message etc.

Whether digital assistants is determined (604) triggers for importing to from one or more the follow-up of appointment sender the instruction communicating and carry out from dynamic response into.Automatic respond module 350 (Fig. 3 A) or its parts (such as, Fig. 3 A, determination module 354) determine whether corresponding sender is one or more senders specified in senders included in instruction.As discussed above, exist for determining that whether the follow-up corresponding sender of importing communication into is multiple programs that in one or more appointment sender one specifies sender.In certain embodiments, automatic respond module 350 (Fig. 3 A) or its parts (such as, Fig. 3 A, determination module 354) also determine whether import communication into is the communication type specified by instruction.

Determining to trigger for when importing instruction that communication carries out from dynamic response into from one or more appointment sender follow-up, method 600 then "Yes" be branched off into step 612 (if or instruction comprise expiration period and/or condition precedent, then to the step of in step 606 or 608 or in certain embodiments, to step 610).Such as, corresponding sender is that in one or more appointment sender specifies sender, and in some cases, imports the designated communication type communicating and utilize in one or more designated communication type into.Determining not trigger for when importing instruction that communication carries out from dynamic response into from one or more appointment sender follow-up, method 600 is branched off into step 602 along "No" and imports communication into receive another and repeat deterministic process.

In certain embodiments, when instruction comprises expiration period, digital assistants or its parts (such as, Fig. 3 A, determination module 354) determine (606) whether expiration period is pass by.Determining expiration period past tense before communication is imported in reception into, method 600 is branched off into step 614 along "Yes", and respond module 350 (Fig. 3 A) or its parts (such as, Fig. 3 A, instruction module 352) delete instruction automatically automatically.Determining expiration period not past tense before communication is imported in reception into, method 600 is branched off into step 612 (or in certain embodiments, step 608 or 610) along "No".

In certain embodiments, when instruction comprises the condition precedent for performing required movement, digital assistants or its parts (such as, Fig. 3 A, determination module 354) determine whether (608) meet condition precedent.Determine receive import communication into before or receive import communication into while also do not meet condition precedent time, method 600 is branched off into step 602 along "No", with receive another import into communication and repeat deterministic process.Determining receive import communication into before or receive import communication into while met condition precedent time, method 600 is branched off into step 612 (or in certain embodiments, step 610) along "Yes".

In certain embodiments, digital assistants or its parts (such as, Fig. 3 A, writes module 356) write (610) customized responses to corresponding sender.According to details included in instruction or information, and the contextual information be associated based on the user of the subscriber equipment 104 with Fig. 1 alternatively (such as, with user-dependent position, pattern or behavioural information, calendar or schedule), write customized responses.Such as, user requires digital assistants " if Swapnil attempts to contact me, I in session to allow him know, and be proposed in the time durations meeting that this afternoon, I had time ".In this example, (such as, address book based on being associated with user) determine from Swapnil receive follow-up import communication into after, digital assistants is write to the message of Swapnil, set forth " I just in session; we meet and client thing are discussed at 4 in afternoon ", and send to Swapnil and invite for the calendar of 4 this afternoon.In this example, DA at least in part based on the same day user calendar or schedule (such as, contextual information) write to the message of Swapnil.

Determining to trigger to when importing instruction that communication carries out from dynamic response into from one or more appointment sender follow-up, digital assistants performs (612) required movement.Such as, automatic respond module 350 (Fig. 3 A) or its parts (such as, Fig. 3 A, execution module 358) send (such as to corresponding user via a kind of means of communication in multiple means of communication, in step 610, by writing that module 356 writes in Fig. 3 A) customized responses.In certain embodiments, execution module uses and imports identical means of communication that communication uses into send customized responses.Such as, required movement comprises to specifying sender to send in the following: Email, call, video call, voip call, SMS or text message, MMS, speech message etc.

In certain embodiments, execution module is configured to, when further comment or the instruction not from user, perform required movement.In certain embodiments, perform required movement comprise the sub-action of execution first and perform the second sub-action.Such as, user requires that DA " if my wife makes a phone call, tells that she meets with me in Cyrano dining room tonight, and after transmitting 30 minutes, also sends the calendar of 7 invitation in afternoon being entitled as ' night of appointment ' to her ".In this example, receive in response to the wife from user and import call into, execution module 358 (Fig. 3 A) is configured to perform the first sub-action included in instruction (such as, wife to user sends text message, statement " tonight meets with me in Cyrano dining room ").And, further in this example, execution module 358 (Fig. 3 A) to be also configured to after sending text message to the wife of user 30 minutes, perform the second sub-action (wife such as, to user sends the calendar of 7 invitation in afternoon that night being entitled as ' night of appointment ') included in instruction.

In one example, automatic respond module 350 (Fig. 3 A) is by specifying sender to send speech message in response to sender's import call into from specifying, and this speech message comprises the record of the voice of the user from the phonetic entry comprising instruction or receiving the phonetic entry of recording before instruction.In another example, automatic respond module 350 (Fig. 3 A) by the speech message of specifying sender to send digitized version as the Text Input from user or text-to-speech version, in response to importing communication call into from appointment sender.

After execution required movement, instruction that electronic equipment is deleted automatically (614).Such as, automatic respond module 350 (Fig. 3 A) or its parts (such as, Fig. 3 A, instruction module 352) instruction stored in included in instruction module 352 (Fig. 3 A) or that and instruction module 352 (Fig. 3 A) is separated storer is deleted.In certain embodiments, instruction module 352 (Fig. 3 A), except one or more appointment sender included in delete instruction and instruction action, also deletes expiration period and/or condition precedent automatically.

The particular order that should be appreciated that the operation in Fig. 5-6 is only for exemplary and be described, and the order described by not being intended to indicate is the unique order that these operations can perform.Those skilled in the art will recognize that the various modes of rearrangement operation described herein.

According to some embodiments, Fig. 7 shows the functional block diagram of the electronic equipment 700 of the principle configuration of the embodiment according to various description.The functional block diagram of equipment is realized by the combination of hardware, software or hardware and software, to perform the principle of the embodiment of various description alternatively.It will be appreciated by those skilled in the art that the functional block described in Fig. 7 is combined alternatively or is separated into sub-frame to realize the principle of the embodiment of various description.Therefore, the description of this paper is supported any possible merging, the separation of functional block described herein alternatively or further defines.

As shown in Figure 7, electronic equipment 700 comprises sound reception unit 702, and it is configured to obtain phonetic entry from user.In certain embodiments, electronic equipment 700 comprises loudspeaker unit 704 alternatively, and it is configured to generate sound, and touch screen display unit 706, and it is configured to display information and receives one or more touch input.Electronic equipment 700 also comprises processing unit 708, and it is coupled to sound reception unit 702 (and being coupled to loudspeaker unit 704 and touch screen display unit 706 alternatively).In certain embodiments, processing unit 708 comprises instruction-determining unit 710, storage unit 712, communication acquisition unit 714, sender's judging unit 716, performance element 718, delete cells 720, communication type determining unit 722, converting unit 724 and writes unit 726.

Processing unit 708 is configured to: in response to acquisition phonetic entry, (such as, utilizing instruction-determining unit 710) determines whether phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement; And (such as, utilizing storage unit 712) stores instruction.Processing unit 708 is also configured to: after storage instruction, and (such as, utilizing communication to obtain unit 714) obtains from corresponding sender and import communication into; And (such as, utilizing transmission judging unit 716) determines whether corresponding sender is one or more appointment senders specified in sender.Processing unit 708 is further configured to when determining that corresponding sender is a sender in one or more appointment sender, according to instruction (such as, utilize performance element 718) perform required movement, and after this (utilize delete cells 720) automatic delete instruction.

In certain embodiments, determine whether phonetic entry comprises instruction and comprise: (utilizing instruction-determining unit 710) determines whether phonetic entry comprises for one or more appointment is follow-up imports communication type into and perform the instruction of required movement in response to receiving from one or more appointment sender; Whether processing unit 708 (utilizing communication type determining unit 722) to determine to import into communication be that follow-up of importing in communication type of one or more appointment specifies and follow-uply imports communication type into if being configured to; And perform required movement to comprise: determine corresponding sender be in one or more appointment sender one specify sender and import into communication be one or more appointment follow-up import in communication type one specify follow-up import communication type into time, come (such as according to this instruction, utilize performance element 718) perform required movement, and after this (such as, utilize delete cells 720) and automatically delete this instruction.

In certain embodiments, the follow-up communication type of importing into of the follow-up corresponding appointment of importing in communication type of one or more appointment is in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, importing communication into is call, and required movement sends speech message.

In certain embodiments, speech message is the record of voice to user at least partially.

In certain embodiments, speech message is digitized version or the text-to-speech version of input text at least partially.

In certain embodiments, processing unit 708 is configured to (such as, utilizing converting unit 724) performs natural language processing to convert phonetic entry to text to phonetic entry, and wherein stores instruction and comprise instruction is stored as text.

In certain embodiments, perform required movement and comprise: according to this instruction, (such as, unit 726 is write in utilization) writes the response to corresponding sender; And this response communicates to sender by (such as, utilizing performance element 718).

In certain embodiments, this communication is in the following: call, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, this communication uses and imports the identical means of communication that communicates into.

In certain embodiments, this is write and comprises: according to this instruction and at least in part based on contextual information, and (such as, unit 726 is write in utilization) writes the response to corresponding sender.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the contacts list be associated with the user of equipment, (such as, utilizing sender's judging unit 716) judges whether corresponding sender is one or more appointment senders specified in sender.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the caller ID be associated with corresponding sender, (such as, utilizing sender's judging unit 716) judges whether corresponding sender is one or more appointment senders specified in sender.

In certain embodiments, processing unit 708 is configured to (such as, utilize sender's judging unit 716) identify the address be associated with the sender of communication, and determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the address be associated to the corresponding sender of communication, (such as, utilizing sender's judging unit 716) judges whether corresponding sender is one or more appointment senders specified in sender.

In certain embodiments, determine that whether corresponding sender is that in one or more appointment sender one specifies sender to comprise: based on the content importing communication into received from corresponding sender, (such as, utilizing sender's judging unit 716) judges whether corresponding sender is one or more appointment senders specified in sender.

In certain embodiments, importing communication into is in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

In certain embodiments, action is selected from the group be made up of the following: send speech message to sender; SMS message is sent to sender; MMS or photo is sent to sender; Email is sent to sender; Event invitation is sent to sender; One or more calendar is shared with sender; One or more custom attributes is shared with sender.

In certain embodiments, when further comment or the instruction not from the user of electronic equipment 700, (such as, utilizing performance element 718) execution and (such as, utilizing delete cells 720) deletion are performed automatically.

In certain embodiments, perform required movement to comprise: (such as, utilizing performance element 718) performs the first sub-action; And (such as, utilizing performance element 718) performs the second sub-action different from the first sub-action.

In certain embodiments, this instruction specifies expiration period further; And processing unit 708 is further configured to: determining that corresponding sender is not that in one or more appointment sender one specifies sender and expiration period past tense, (such as, utilizing delete cells 720) deletes this instruction automatically.

In certain embodiments, corresponding sender is the first corresponding sender; And processing unit 708 is configured to: determining that the first corresponding sender is not that in one or more appointment sender one specifies sender and expiration period not past tense, abandon deleting this instruction; (such as, utilize communication obtain unit 714) obtains from the different from the first corresponding sender second corresponding sender and imports communication into; (such as, utilizing sender's judging unit 716) determines whether the second corresponding sender is one or more appointment senders specified in sender; And determining that the second corresponding sender is that in one or more appointment sender one specifies sender and expiration period not past tense, according to this instruction (such as, utilize performance element 718) perform required movement, and after this automatically delete this instruction.

In order to the object explained, describe aforementioned explanation with reference to specific implementation mode.But above illustrative discussion is not intended to detailed or disclosed implementation is limited in disclosed precise forms.Based on above instruction, many amendments and modification are possible.Select and describe implementation to think ratio juris and practical application disclosed in explaining best, thus making others skilled in the art can utilize idea of the present disclosure best, and be applicable to the various amendments of the specific use expected.

Although term " first ", " second " etc. can be used to describe various element herein, but these elements should not limited by these elements.These terms are only for being distinguished from each other between element.Such as, first phonetic entry can be called as the second phonetic entry, and similarly, second phonetic entry can be called as the first phonetic entry, and the implication of description can not be changed, as long as " the first phonetic entry " occur unanimously named and " the second phonetic entry " occur and unanimously named.First phonetic entry and the second phonetic entry are all phonetic entries, but they are not identical phonetic entries.

The object of term as used herein just in order to describe specific implementation mode, and and not intended to be as limitations on claims.As in the description of implementation and claims use, " one ", " one " and " being somebody's turn to do " of singulative is intended to also contain plural form, unless context clearly has contrary instruction.It will also be understood that, term "and/or" used herein refers to and contains any and all possible combination of the one or more projects in the project listed explicitly.It will also be understood that, the feature that term " comprises " and/or " comprising " specifies existence to state when using in this manual, integer, step, operation, element and/or parts, but do not get rid of existence or add one or more further feature, integer, step, operation, element, parts and/or its grouping.

As used herein, based on context, term " if " can be interpreted as being meant to stated condition precedent for genuine " when ... time " or " ... time " or " in response to determining " or " according to determining " or " in response to detecting ".Similarly, based on context, phrase " if determining [condition precedent stated for true] " or " if [condition precedent stated is for true] " or " when [condition precedent stated is for true] " can be interpreted as being meant to the condition precedent that condition precedent that " determining " or " determining one " state states for true time or " in response to determining " or " according to determining " or " detecting " or " in response to detecting " is true.

Claims (53)

1. make digital assistants to importing the method communicating and carry out from dynamic response into, described method comprises:

There are one or more electronic equipment places of one or more processor and storer separately:

Phonetic entry is obtained from user;

In response to the described phonetic entry of acquisition, determine whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement;

Store described instruction;

After the described instruction of storage, obtain from corresponding sender and import communication into;

Determine whether corresponding sender is described one or more appointment sender specified in sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, performing described required movement according to described instruction, and after this automatically deleting described instruction.

2. method according to claim 1, wherein determines whether described phonetic entry comprises instruction and comprise: determine whether described phonetic entry comprises for one or more appointment is follow-up imports communication type into and perform the instruction of required movement in response to receiving from one or more appointment sender;

Described method comprises: whether import communication described in determining into is that follow-up of importing in communication type of described one or more appointment specifies and follow-uply imports communication type into; And

Wherein perform described required movement to comprise: determine corresponding sender be in described one or more appointment sender one specify sender and described in import into communication be described one or more appointment follow-up import in communication type one specify follow-up import communication type into time, perform described required movement according to described instruction, and after this automatically delete described instruction.

3. method according to claim 2, the follow-up corresponding appointment of importing in communication type of wherein said one or more appointment is follow-up, and to import communication type into be in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

4. the method according to any one of claim 1-3, wherein said to import communication into be call, and wherein said required movement sends speech message.

5. method according to claim 4, wherein said speech message be the record of voice to described user at least partially.

6. the method according to any one of claim 4-5, wherein said speech message be digitized version or the text-to-speech version of input text at least partially.

7. the method according to any one of claim 1-6, comprises and performs natural language processing to convert described phonetic entry to text to described phonetic entry, and wherein stores described instruction and comprise described instruction is stored as text.

8. the method according to any one of claim 1-3 and 7, wherein performs described required movement and comprises:

According to described instruction, write the response to corresponding sender; And

Described response is communicated to described sender.

9. method according to claim 8, wherein said communication is one in the following: call, voice mail, video call, SMS, MMS, VOIP calling or Email.

10. the method according to Claim 8 according to any one of-9, wherein said communication uses imports with described the identical means of communication that communicates into.

11. methods according to Claim 8 according to any one of-10, wherein said writing comprises according to described instruction and writes the response to corresponding sender based on contextual information at least in part.

12. methods according to claim 11, wherein said contextual information comprises one in the following: the positional information be associated with described user, pattern information or behavioural information.

13. methods according to any one of claim 1-12, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the list of the contact person be associated with the described user of described equipment, judge that whether corresponding sender is that in described one or more appointment sender one specifies sender.

14. methods according to any one of claim 1-12, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the caller ID be associated with corresponding sender, judge that whether corresponding sender is that in described one or more appointment sender one specifies sender.

15. methods according to any one of claim 1-12, comprise the address identifying and be associated with the described sender of described communication; And

Wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the described address be associated with the corresponding sender of described communication, judge that whether described sender is that in described one or more appointment sender one specifies sender.

16. methods according to any one of claim 1-12, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the content importing communication described in receiving from corresponding sender into, judge that whether described sender is that in described one or more appointment sender one specifies sender.

17. methods according to any one of claim 1-3 and 7-16, wherein said to import communication into be in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

18. methods according to any one of claim 1-3 and 7-16, wherein said action is selected from the group be made up of the following:

Speech message is sent to described sender;

SMS message is sent to described sender;

MMS or photo is sent to described sender;

Email is sent to described sender;

Event invitation is sent to described sender;

One or more calendar is shared with described sender; And

One or more custom attributes is shared with described sender.

19. methods according to any one of claim 1-18, wherein when further comment or the instruction not from the described user of described equipment, described execution and described deletion are performed automatically.

20. methods according to any one of claim 1-19, wherein perform described required movement and comprise:

Perform the first sub-action; And

Perform the second sub-action different from described first sub-action.

21. methods according to any one of claim 1-20, wherein said instruction specifies expiration period further; And

Described method comprises: determining that corresponding sender is not that in described one or more appointment sender one specifies sender and described expiration period past tense, automatically delete described instruction.

22. methods according to claim 21, wherein said corresponding sender is the first corresponding sender; And described method comprises:

Determining that the described first corresponding sender is not that in described one or more appointment sender one specifies sender and described expiration period not past tense, abandon deleting described instruction;

Obtain from the different from described first corresponding sender second corresponding sender and import communication into;

Determine whether the described second corresponding sender is described one or more appointment sender specified in sender; And

Determining that the described second corresponding sender is that in described one or more appointment sender one specifies sender and described expiration period not past tense, performing described required movement according to described instruction, and after this automatically deleting described instruction.

The non-transitory computer readable storage medium of 23. 1 kinds of one or more programs of storage, described one or more program comprises the instruction for operand word assistant, described instruction is when being performed by the electronic equipment with one or more processor, make described equipment executable operations, described operation comprises:

Phonetic entry is obtained from user;

In response to the described phonetic entry of acquisition, determine whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement;

After the described instruction of storage, obtain from corresponding sender and import communication into;

Determine whether corresponding sender is described one or more appointment sender specified in sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, performing described required movement according to described instruction, and after this automatically deleting described instruction.

The non-transitory computer readable storage medium of 24. 1 kinds of one or more programs of storage, described one or more program comprises instruction, described instruction, when being performed by the electronic equipment with one or more processor, makes any one method in the method for described equipment execution according to claim 1-22.

25. 1 kinds of electronic equipments, comprise

One or more processor; And

Storer, described storer stores the instruction being used for operand word assistant, and described instruction is when being performed by described one or more processor, and make described processor executable operations, described operation comprises:

Phonetic entry is obtained from user;

In response to the described phonetic entry of acquisition, determine whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement;

After the described instruction of storage, obtain from corresponding sender and import communication into;

Determine whether corresponding sender is described one or more appointment sender specified in sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, performing described required movement according to described instruction, and after this automatically deleting described instruction.

26. 1 kinds of electronic equipments, comprise one or more processor and storer, described storer stores the one or more programs being used for being performed by described one or more processor, and described one or more program comprises the instruction of any one method in the method for performing according to claim 1-22.

27. 1 kinds of electronic equipments, comprising:

For obtaining the device of phonetic entry from user;

In response to the described phonetic entry of acquisition, for determining whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the device of the instruction of required movement;

After the described instruction of storage, be caught to can be used in obtaining from corresponding sender the device importing communication into;

For determining that whether corresponding sender is the device that in described one or more appointment sender one specifies sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, be caught to can be used in performing described required movement according to described instruction and the device of the after this described instruction of deletion automatically.

28. 1 kinds of electronic equipments, comprise the device of any one method in the method for performing according to claim 1-22.

29. 1 kinds, for the signal conditioning package used in the electronic device, comprising:

For obtaining the device of phonetic entry from user;

In response to the described phonetic entry of acquisition, for determining whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the device of the instruction of required movement;

After the described instruction of storage, be caught to can be used in obtaining from corresponding sender the device importing communication into;

For determining that whether corresponding sender is the device that in described one or more appointment sender one specifies sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, be caught to can be used in performing described required movement according to described instruction and the device of the after this described instruction of deletion automatically.

30. 1 kinds of signal conditioning packages for using in the electronic device, comprise the device of any one method in the method for performing according to claim 1-22.

Graphical user interface on 31. 1 kinds of electronic equipments, described electronic equipment has display, storer and the one or more processors for performing in described storer the one or more programs stored, and described graphical user interface comprises the user interface shown by any one method in the method according to claim 1-22.

32. 1 kinds of electronic equipments, comprising:

Sound reception unit, described sound reception unit is configured to obtain phonetic entry from user; And

Processing unit, described processing unit is coupled to described sound reception unit, and described processing unit is configured to:

In response to the described phonetic entry of acquisition, determine whether described phonetic entry comprises for follow-up import communication in response to receiving from one or more appointment sender into and perform the instruction of required movement;

After the described instruction of storage, obtain from corresponding sender and import communication into;

Determine whether corresponding sender is described one or more appointment sender specified in sender; And

When determining that corresponding sender is an appointment sender in described one or more appointment sender, performing described required movement according to described instruction, and after this automatically deleting described instruction.

33. electronic equipments according to claim 32, wherein determine whether described phonetic entry comprises instruction and comprise: determine whether described phonetic entry comprises for one or more appointment is follow-up imports communication type into and perform the instruction of required movement in response to receiving from one or more appointment sender;

Wherein said processing unit is configured to: whether import communication described in determining into is that follow-up of importing in communication type of described one or more appointment specifies and follow-uply imports communication type into; And

Wherein perform described required movement to comprise: determine corresponding sender be in described one or more appointment sender one specify sender and described in import into communication be described one or more appointment follow-up import in communication type one specify follow-up import communication type into time, perform described required movement according to described instruction, and after this automatically delete described instruction.

34. electronic equipments according to claim 33, the follow-up corresponding appointment of importing in communication type of wherein said one or more appointment is follow-up, and to import communication type into be in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

35. electronic equipments according to any one of claim 32-34, wherein said to import communication into be call, and wherein said required movement sends speech message.

36. electronic equipments according to claim 35, wherein said speech message be the record of voice to described user at least partially.

37. electronic equipments according to any one of claim 35-36, wherein said speech message be digitized version or the text-to-speech version of input text at least partially.

38. electronic equipments according to any one of claim 32-37, wherein said processing unit is configured to perform natural language processing to convert described phonetic entry to text to described phonetic entry, and wherein stores described instruction and comprise described instruction is stored as text.

39. electronic equipments according to any one of claim 32-34 and 38, wherein perform described required movement and comprise:

According to described instruction, write the response to corresponding sender; And

Described response is communicated to described sender.

40. according to electronic equipment according to claim 39, and wherein said communication is one in the following: call, voice mail, video call, SMS, MMS, VOIP calling or Email.

41. electronic equipments according to any one of claim 39-40, wherein said communication uses imports with described the identical means of communication that communicates into.

42. electronic equipments according to any one of claim 39-41, wherein said writing comprises according to described instruction and writes the response to corresponding sender based on contextual information at least in part.

43. electronic equipments according to claim 42, wherein said contextual information comprises one in the following: the positional information be associated with described user, pattern information or behavioural information.

44. electronic equipments according to any one of claim 32-43, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the list of the contact person be associated with the described user of described equipment, judge that whether corresponding sender is that in described one or more appointment sender one specifies sender.

45. electronic equipments according to any one of claim 32-43, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the caller ID be associated with corresponding sender, judge that whether corresponding sender is that in described one or more appointment sender one specifies sender.

46. electronic equipments according to any one of claim 32-43, wherein said processing unit is configured to identify the address be associated with the described sender of described communication; And

Wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the described address be associated with the corresponding sender of described communication, judge that whether described sender is that in described one or more appointment sender one specifies sender.

47. electronic equipments according to any one of claim 32-43, wherein determine that whether corresponding sender is that in described one or more appointment sender one specifies sender to comprise: based on the content importing communication described in receiving from corresponding sender into, judge that whether described sender is that in described one or more appointment sender one specifies sender.

48. electronic equipments according to any one of claim 32-34 and 38-47, wherein said to import communication into be in the following one: call, speech message, voice mail, video call, SMS, MMS, VOIP calling or Email.

49. electronic equipments according to any one of claim 32-34 and 38-47, wherein said action is selected from the group be made up of the following:

Speech message is sent to described sender;

SMS message is sent to described sender;

MMS or photo is sent to described sender;

Email is sent to described sender;

Event invitation is sent to described sender;

One or more calendar is shared with described sender; And

One or more custom attributes is shared with described sender.

50. electronic equipments according to any one of claim 32-49, wherein when further comment or the instruction not from the described user of described equipment, described execution and described deletion are performed automatically.

51. electronic equipments according to any one of claim 32-50, wherein perform described required movement and comprise:

Perform the first sub-action; And

Perform the second sub-action different from described first sub-action.

52. electronic equipments according to any one of claim 32-51, wherein said instruction specifies expiration period further; And

Described method comprises: determining that corresponding sender is not that in described one or more appointment sender one specifies sender and described expiration period past tense, automatically delete described instruction.

53. electronic equipments according to claim 52, wherein said corresponding sender is the first corresponding sender; And

Described processing unit is configured to:

Determining that the described first corresponding sender is not that in described one or more appointment sender one specifies sender and described expiration period not past tense, abandon deleting described instruction;

Obtain from the different from described first corresponding sender second corresponding sender and import communication into;

Determine whether the described second corresponding sender is described one or more appointment sender specified in sender; And

Determining that the described second corresponding sender is that in described one or more appointment sender one specifies sender and described expiration period not past tense, performing described required movement according to described instruction, and after this automatically deleting described instruction.