CN1643575A - Intelligent personal assistants - Google Patents
Intelligent personal assistants Download PDFInfo
- Publication number
- CN1643575A CN1643575A CNA038070065A CN03807006A CN1643575A CN 1643575 A CN1643575 A CN 1643575A CN A038070065 A CNA038070065 A CN A038070065A CN 03807006 A CN03807006 A CN 03807006A CN 1643575 A CN1643575 A CN 1643575A
- Authority
- CN
- China
- Prior art keywords
- user
- information
- disposed
- intelligent
- input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/004—Artificial life, i.e. computing arrangements simulating life
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
- G06F9/453—Help systems
Abstract
An intelligent social agent is an animated computer interface agent with social intelligence that has been developed for a given application or type of applications and a particular user population. The social intelligence of the agent comes from the ability of the agent to be appealing, affective, adaptive, and an appropriate when interacting with the user. An intelligent personal assistant is an implementation of an intelligent social agent that assists a user in operating a computing device and using application programs on a computing device.
Description
Technical field
The technology of interface proxy with the secondary computer system user the present invention relates to develop and uses a computer.
Background technology
Computer system can be used for finishing many tasks.The user of computer system can come auxiliary to user or the computer interface agency that carries out service for the user by information is provided.
Summary of the invention
In total one side, realize that a kind of intelligent personal assistants comprises: receive the input related with the user and with the related input of application program; Visit the user profile related with the user.Extract environmental information from the input that receives; And processing environment information and user profile, to produce adaptive response by intelligent personal assistants.
Each realizes comprising following one or more feature.For example, application program can be application program, recreational application programs or the recreation of personal information management application program, operational computations equipment.
The adaptive response of intelligent personal assistants can be associated with application program, recreational application programs or the recreation of personal information management application program, operational computations equipment.
On the other hand total, a kind of device of intelligent social agency of realizing comprises information extractor, adaptation engine and output generator.Information extractor is disposed at: visit the user profile related with the user, receive the input related with the user and extract environmental information from the input that receives.Adaptation engine is disposed at: from information extractor reception environment information and user profile, and processing environment information and user profile, to produce adaptive output.The output generator is disposed at: receive adaptive output, and the adaptive output among the expression intelligent social agency.
Each realizes comprising above-mentioned one or more feature and following one or more feature.For example, information extractor can be disposed at and receive physiological data or the application information related with the user.Information extractor can be disposed at: from the physiologic information related with the user, from the voice analysis information related with the user by the characteristics of speech sounds that extracts language content and analysis user, or, extract the information of relevant user's affective state always from user's language message.Extracting environmental information can also comprise: by using GPS, extract user's geographic position and the information of extracting based on user's geographic position.Extracting environmental information can comprise: extract about with the information of the information of the related applied environment of user or relevant user's diction.
The output generator can be the language generator, and adaptation engine can be disposed at the generation language performance, and the language generator can produce the language performance among the intelligent social agency.The output generator can be the emotion generator, and adaptation engine can be disposed at the generation facial expression, and the emotion generator can produce the facial expression among the intelligent social agency.The output generator can be the multi-mode generator, and it uses two patterns at least one, the adaptive output among the expression intelligent social agency.A pattern can be a language mode, and another pattern can be an emotion model.Adaptation engine can be disposed at facial expression and the language performance of generation by the expression in the intelligent social agency of multi-mode output generator.Adaptation engine can be disposed at the emotional expression that produces among the intelligent social agency.The output generator can be disposed at the emotional expression among the expression intelligent social agency.
Total another aspect, realize a kind of intelligent social agency, comprising: receive the input related with the user; Visit the user profile related with the user; Extract environmental information from the input that receives; And processing environment information and user profile, will be to produce by the adaptive output of intelligent social agency expression.
Each realizes comprising above-mentioned one or more feature and following one or more feature.For example, related with user input can comprise physiological data or the application information related with the user.Extracting environmental information can comprise from extracting the information of relevant user's affective state with related physiologic information, voice analysis information or the language message of user.Extract environmental information and can also comprise geographic position of extracting the user and the information of extracting based on user's geographic position.Extract environmental information can comprise extract about with the information of the information of the related applied environment of user or relevant user's diction.To can be language performance, facial expression or emotional expression by the adaptive output of intelligent social agency expression.
The realization of above-mentioned any technology can comprise be used to realize with the intelligent social agency's of the system interaction of user or other type method or process, computer-readable medium on computer program, system or device or mobile device.
Propose in one or more details accompanying drawing below of various realizations and the explanation.From these explanations and accompanying drawing and accessory rights requirement, further feature and advantage will be clear.
Description of drawings
Fig. 1 is the block diagram that is used for exploitation and uses intelligent social agency's (intelligent social agent) programmable system.
Fig. 2 is the block diagram of intelligent social agent operation computing equipment thereon.
Fig. 3 is the block diagram of the framework of the social intelligence engine of explanation.
Fig. 4 A and 4B are the process flow diagrams that is used to extract the process of user's emotion and psychological condition.
Fig. 5 is a process flow diagram of intelligent social being acted on behalf of the process of adaptive user and environment (context).
Fig. 6 is the process flow diagram of projection (cast) intelligent social agency's process.
Fig. 7-the 10th illustrates the block diagram of each side of the framework of intelligent personal assistants.
Same label is represented same element in each accompanying drawing.
Embodiment
With reference to Fig. 1, programmable system 100 is used to develop and uses intelligent social agency's programmable system 100 (for example to comprise various I/O (I/O) equipment, touch input device 107, personal digital assistant or " PDA " 108 and the mobile phone 109 of mouse 102, keyboard 103, display 104, voice recognition and speech synthesis apparatus 105, video camera 106, band stylus), can operate the computing machine 110 that is used for having CPU (central processing unit) (CPU) 120, I/O unit 130, storer 140 and data storage device 150 and communicate by letter.Data storage device 150 can be stored the executable instruction of machine, data (as the application data of configuration data or other type), and various programs such as operating system 152 and one or more application program 154 that is used to develop and use the intelligent social agency, all these can be handled by CPU 120.Each computer program can level process or Object-Oriented Programming Language realize, if or wish just with compilation or machine language realization; And in any situation, described language can be compiling or interpretative code.Data storage device 150 can be any type of nonvolatile memory, comprises for example semiconductor storage unit, as Erarable Programmable Read only Memory (EPROM), EEPROM (Electrically Erasable Programmable Read Only Memo) (EEPROM) and flush memory device; Disk such as internal hard drive and removable dish; Magneto-optic disk; With compact disk ROM (read-only memory) (CD-ROM).
System 100 can also comprise communication card or device 160 (for example, modulator-demodular unit and/or network adapter), is used to use communication link 175 (for example telephone wire, wireless network links, cable network link or cable system) and network 170 swap datas.Perhaps, can use universal system bus (USB) connector connected system 100, be used for and network 170 swap datas.Other example of system 100 can comprise handheld device, workstation, server, equipment or some combinations of these equipment that can respond in the mode of definition and execute instruction.Aforementioned any equipment can be replenished or comprised in ASIC by ASIC (application specific integrated circuit).
Though Fig. 1 illustrates PDA and the mobile phone peripherals as relative system 100, in some implementations, the functional of system 100 can directly be integrated in PDA or the mobile phone.
Fig. 2 explanation is used to comprise exemplary realization of the intelligent social agency 200 of the computing equipment of PDA 210, stylus 212 and intelligent social agency's 220 visual representing.Though Fig. 2 illustrates the personage of intelligent social agency as a style of speaking of animation, but intelligent social agency is not limited to such outward appearance, and for example can show as cartoon head, animal, the image from video or still image capturing, Drawing Object or as just sound.The social agency's of user's available parameter definition outward appearance.PDA can be for example from the obtainable iPAQ of COMPAQ
TMPocket PC.
Intelligent social agency 200 is the animation computer interface agencies with social intelligence, and it has been the exploitation of given application or equipment or potential user group.Agency's social intelligence becomes attracting (appealing) when coming from user interactions, (affective) of emotion arranged, the agency's of adaptive (adaptive) and suitable (appropriate) ability.The individual of establishment based target customer group and the intelligent social agency's of professionalism visual appearance, sound (voice) and individual character can help the intelligent social agency attractive to the targeted customer.With the influence of performance by face, voice (vocal) and language performance, can help the intelligent social agency to manifest has emotion to the targeted customer with the intelligent social proxy programming.The intelligent social proxy programming so that user, application and current environment are modified its behavior, can be helped the adaptive and suitable targeted customer of intelligent social agency.Mutual between intelligent social agency and the user can produce improved experience for the user when acting on behalf of assisted user operational computations equipment or computing equipment application program.
Fig. 3 illustrates the framework of social intelligence engine 300, and it can be attracting, that emotion is arranged, adaptive and suitable with user interactions the time that social intelligence engine 300 can make the intelligent social agency.The information that social intelligence engine 300 receives about user 305 comprised user profile (user profile) from user 305, and from the information of application program 310 receptions about application program 310.Social intelligence engine 300 generation behaviors and language (verbal) and non-language (nonverbal) are expressed and are used for the intelligent social agency.
The user can by say, input characters, use pointing device or the I/O equipment (as touch-screen or vision track equipment) that uses other type is mutual with social intelligence engine 300.Literal or voice (speech) can input receives as literal by the natural language processing system processing and by social intelligence engine.Voice will be discerned by speech recognition software, and can be handled by the characteristic of Voice analyzer, and the voice analysis device provides the abridged table of user's emotion and physiological status according to characteristic such as the range of pitch and the breathing of user's voice.
Relevant user's information can be received by social intelligence engine 300.Social intelligence engine 300 can receive personal characteristics's (as name, age, sex, race or nationality's information and preferred language) of relevant user, with relevant user's professionalism (as occupation, post and one or more member organization).The user profile that receives can comprise user profile or can be used to produce and the storage user profile by CPU (central processing unit) 120.
The non-linguistic information that receives from characteristic of Voice analyzer or natural language processing system can comprise voice prompting (vocal cue) (as fundamental note and the voice rate) from the user.Video camera or vision track equipment can provide the non-language data of relevant eyes of user focus, a direction and other body position information.Physical connection between user and the I/O equipment (as keyboard, mouse, handheld device or touch pad) can provide physiologic information (as the measurement of user's heart rate, blood pressure, breathing, body temperature and skin conductivity).GPS can provide the information in relevant user's geographic position.Other such environment knows that instrument can provide the additional information of relevant user's environment, one or more images of user's physical location are provided as video camera, it can be treated for environmental information, as the user is independent or in group, in buildings in office environment or outside in the park.
The information that social intelligence engine 300 also can receive about application program 310 from the application program 310 that operates on the computing machine 110.Receive from the information of application program 310 information extractor 320 by social intelligence engine 300.Information extractor 320 comprises language extraction apparatus 322, non-language extraction apparatus 324 and user environment extraction apparatus 326.
The language data of language extraction apparatus 322 process user input.The I/O equipment receiving data that this language extraction apparatus can use from the user, or can after processing, receive data (literal that produces by natural language processing system as original input) from the user.Language extraction apparatus 322 is caught language content, as order or the data by user input, is used for computing equipment or application program (as those related with computing machine 110).Language extraction apparatus 322 is also resolved language content to determine user's diction, as word, select grammer and grammatical style.
User environment extraction apparatus 326 is determined user's internal environment and external environment condition.User environment extraction apparatus 326 determines that according to user's physiological data and language data the user asks or carry out the pattern of action (can be described as internal environment).For example, for the user compare when speaking with normal style, when saying same words with very fast word speed, less breathing and very fast heart rate, but show the internal environment of the order indicating emergency of sales figures at special time period.User environment extraction apparatus 326 can be determined urgent internal environment from the language content of order, when comprising speech " soon " or " now " in order.
User environment extraction apparatus 326 is identified for the characteristic (can be described as user's external environment condition) of user environment.For example, GPS (being integrated in or being connected in computing machine 110) can be determined user's geographic position, can determine user's local weather condition, geologic aspects, culture and language from this geographic position.The noise level of user's environment for example can be determined by handling characteristic of Voice analyzer or the natural language processing system voice data, that be stored on the computing machine 110 that detect by the microphone that is integrated in or is connected in computing machine 110.By analyzing the image from video camera or vision track equipment, user environment extraction apparatus 326 can be determined other physics and social environment characteristic, as the user be independent or with other people together, be arranged in office environment or in park or automobile.
Applied environment extraction apparatus 328 is determined the information of relevant application environment.This information for example can comprise that the mutual frequency of the function of the result's of the importance of application program, the emergency that is associated with specific action, specific action level, the level of using or be used for the confidentiality of application's data, user and application program or application program, the level of complexity of application program, application program are used for personal use or employ environment setting, application program whether to be used for the level of the computing equipment resource of amusement and application program requirement.
Act on behalf of personality module 334 and receive user's basic abridged table and relevant user's dynamic abstract from machine study module 332.Perhaps, acting on behalf of personality module 334 can be from the basic abridged table of data storage device 150 calling parties or relevant user's dynamic abstract.Act on behalf of personality module 334 for can attracting and the intelligent social agency of suitable particular group creates outward appearance and sound (this can be described as projection (cast) intelligent social agency), and the intelligent social agency during with user interactions adaptive intelligent social agency be fit to this user and this user at the environment that changes (this can claim individualized intelligent is social act on behalf of).
Dynamic adaptation device module 336 receives the user's who adjusts basic abridged table and relevant user's dynamic abstract and the information that is received or compiled by information extractor 320 from machine study module 332.Dynamic adaptation device module 336 is also from acting on behalf of projection and the customized information that personality module 334 receives relevant intelligent social agency.
Dynamic adaptation device module 336 is determined intelligent social agency's action and behavior.Dynamic adaptation device module 336 can be used the language input from user and application environment, with one or more actions of determining that the intelligent social agency should carry out.For example, import that request " is checked my email message " and email application when not starting the user, the intelligent social agency starts email application and initialization e-mail applications function with the message that checks e-mails.Dynamic adaptation device module 336 can be used relevant user's the non-linguistic information and the environmental information of relevant user and application program, with the behavior that assists in ensuring that the intelligent social agency and the environment that is suitable for the user that moves.
For example, when machine learning module 332 indication users' internal environment is urgent, dynamic adaptation device module 336 can be adjusted the intelligent social agency, feasible agency has and seems serious facial expression and stop or suspending Nonvital Function (as receive large data files from network) or close unnecessary application program (as plotter program), to finish the emergency action of request as early as possible.
When machine learning module 332 indication human fatigues, dynamic adaptation device module 336 can be adjusted the intelligent social agency, makes the agency have the facial expression of loosening, and it is slower to speak, and uses the word of less syllable and the sentence of less word.
When machine learning module 332 indication users were glad or energetic, dynamic adaptation device module 336 can be adjusted the intelligent social agency, and having glad facial expression, and it is faster to speak.When using electronic commerce applications to place an order, dynamic adaptation device module 336 can make additional purchase or the renewal of intelligent social agency suggestion the user.
When machine learning module 332 indication users were dejected, dynamic adaptation device module 336 can be adjusted the intelligent social agency, having worried facial expression, and carried out less or only crucial suggestion.If machine learning module 332 indication users are dejected to the intelligent social agency, so dynamic adaptation device module 336 can make intelligent social agency apology and advisably interpretation problems what is and should how solves it.
Dynamic adaptation device module 336 can be adjusted the intelligent social agency, according to the user familiarity of current computer equipment, application program or function of application and the complexity of application program are turned round.For example, when application program complexity and user are unfamiliar with application program (for example, the user when for the first time using application program or the user when certain predetermined amount of time does not use application program), whether dynamic adaptation device module 336 can allow intelligent social agency inquiry user want to help, if and the user makes expression like this, the intelligent social agency just starts the help function that is used for application program.When uncomplicated or user was familiar with application program in application program, dynamic adaptation device module 336 did not generally allow the intelligent social agency offer help to the user.
With reference to Fig. 4 A, process 400A processor controls is extracted non-linguistic information and definite user's affective state.Process 400A starts from receiving relevant user's physiological status data (step 410A).The physiological status data can comprise autonomic nerves system (autonomic) data, as heart rate, blood pressure, respiratory rate, body temperature and skin conductivity.Physiological data can use the equipment that appends to user's finger or palm and can detect user's heart rate, respiratory rate and blood pressure from computing machine 110 to determine.
Processor is identified for the hypothesis (step 415A) of user's affective state according to the physiological data that receives by the physiology channel then.Processor can use the predetermined decision logic that specific physiological responses is relevant with affective state.As above relatively Fig. 3 is described, the heart rate of acceleration can with fear or angry related that and slow heart rate can be indicated relaxation state.
Second channel of the data that received by processor that is used for determining user's affective state is voice analysis data (step 420A), as the range of pitch in the user's voice, volume and breathing degree.For example, than user's basic model louder and faster voice can indicate user's happiness.Similarly, can indicate user's sadness than common lighter and slower voice.Processor is identified for the hypothesis (step 425A) of user's affective state according to the voice analysis data that receive by the characteristic of Voice channel then.
Being used for determining the 3rd channel of the data that received by processor of user's affective state, is user's the language content (step 430A) that discloses user's emotion.The example of such language content comprises phrase, as ", this is excellent " or " what? does file disappear? "Processor is identified for the hypothesis (step 435A) of user's affective state according to the language content that receives by the language channel then.
Processor solves any conflict and definite user's final affective state (step 440A) according to the data integration affective state hypothesis from physiology channel, characteristic of Voice channel and language channel then.Can finish conflict by predetermined decision logic solves.According to clear and definite rank, give affective state by each prediction of three channels with confidence factor to the concrete diagnosis of the affective state of the intrinsic predicted power of that channel of that particular emotion and appearance.Processor is eliminated fuzzy by comparison and integrated each confidence factor then.
Some realizations can receive physiological data, voice analysis data, language content or combination.When only receiving a kind of data, may not carry out integrated (step 440A).For example, when only receiving physiological data, execution in step 420A-440A not, and processor uses based on the user's of physiological data the affective state affective state as the user.Similarly, when only receiving the voice analysis data, process starts from receiving the voice analysis data, and not execution in step 410A, 415A and 430A-445A.Processor uses based on the user's of voice analysis data the affective state affective state as the user.
Similarly, with reference to Fig. 4 B, process 400B processor controls is extracted non-linguistic information and definite user's affective state.Processor receives relevant user's physiological data (step 410B), voice analysis data (step 420B) and indicates the language content (step 430B) of user's emotion, and is identified for the hypothesis ( step 415B, 425B and 435B) of user's affective state according to each data parallel.Processor solves the final affective state (step 440B) of any conflict and definite user, as described in relative Fig. 4 A according to the data integration affective state hypothesis from physiology channel, characteristic of Voice channel and language channel then.
With reference to Fig. 5, process 500 processor controls make the intelligent social agency adapt to user and environment.Process 500 can help the intelligent social agency suitably to move according to user and applied environment.
Process 500 starts from receiving (step 510) content and environmental information to computing machine 110 by processor from input-output apparatus (as voice recognition and speech synthesis apparatus, video camera, be connected the physiological detection equipment of user's finger).Content that receives and environmental information can be language message, non-linguistic information or the environmental informations from user or application program reception, maybe can be the information (as described in preceding relative Fig. 3) by the information extractor compiling.
The processor to access data memory device 150 then, determine the elemental user abridged table (step 515) of intelligent social agency the user mutual with it.The elemental user abridged table comprises personal characteristics's (as name, age, sex, race or nationality's information and preferred language) of relevant user, relevant user's professionalism (as occupation, post and one or more member organizations) and relevant user's non-linguistic information (as diction and physiology profile information).The elemental user profile information can receive during the registration process of the product that is used for the person in charge's (host) intelligent social agency, or is used for user's intelligent social agency's projection (casting) process reception by establishment, and is stored on the computing equipment.
Processor can be according to the environment and the content information (step 520) of elemental user profile information adjustment reception.For example, can receive sound instruction and " read email message now ".Usually, the sound instruction that word " now " is modified can produce the user environment pattern of " promptly ".Yet when elemental user profile information indication user generally made word " now " instruct as part, the user environment pattern can be changed into " normally ".
Content and environmental information that processor can receive by the affective state adjustment of determining the user.User's affective state can be determined from content and environmental information (as physiological data and voice analysis data).
Processor changes intelligent social agency (step 525) according to content of adjusting and environmental information.For example, processor can change intelligent social agency's diction and voice style, with diction and the voice style that more is similar to the user.
Elemental motion (step 530) in the processor executive utility then.For example, import that request " is checked my email message " and email application when being not activated the user, the intelligent social agency starts email application, and starts e-mail applications functional check email message (as described in preceding relative Fig. 3).
Processor is determined to comprise the suitable emotional expression (step 540) that is used for the intelligent social agency of facial expression and suitable language performance (step 535).
Processor produces the suitable language performance (step 545) that is used for the intelligent social agency.Suitable language performance comprises: based on the content and the content of environmental information, elemental user profile information or elemental user profile information and reception and language content combination, suitable and the suitable emotional semantic of environmental information that receive.
For example, can use speech coupling to act on behalf of the suitable emotion that express with emotion intension.This can finish by using the electronic dictionary that speech is related with affective state, association wherein as speech " extremely wonderful " is related with happiness, as with speech " delay " and dejected related etc.Processor is selected speech from the dictionary to user and fitness of environment.Similarly, processor can increase the speech number of use in language performance when user's affective state is happiness, if perhaps user's affective state is the sad speech that has less syllable with regard to speech number capable of reducing using or use.
Processor can send the language performance literal to the I/O equipment that is used for computer equipment, generally is display device.Processor can be voice with the language performance text conversion and export this voice.This can use literal to finish to speech convertor and voice operation demonstrator.
Simultaneously, processor produces the suitable emotion (step 550) of the facial expression that is used for the intelligent social agency.In addition, can select the facial expression given tacit consent to.The facial expression of acquiescence can be determined by role and the potential user group used, act on behalf of.Usually, intelligent social agency acquiescence can be friendly slightly, smile with happiness.
Facial emotional expression can be by revising the intelligent social agency the each several part of face finish with the performance emotion.For example, by showing that eyebrow raises up (for example crooked and high), eyebrow skin horizontal stretching, wrinkle down passes forehead, eyelid magnifies and the white of the eye visible, chin is opened and mouth takes it easy or uphold, can represent in surprise.
By show eyebrow raise up get together, forehead wrinkle tractive raises up in forehead center, upper eyelid and palpebra inferior stops (draw up), mouth magnifies and lip is nervous slightly or uphold and withdrawal, can represent to fear.Upwards shift upper lip onto by showing that upper lip improves, lower lip improves or lower lip reduces, wrinkle nose, cheek improve, occur line, eyelid under the palpebra inferior upwards pushes away but take it easy and eyebrow reduces, can square one's shoulders.By show eyebrow raise up get together, perpendicular line, palpebra inferior anxiety, upper eyelid anxiety occur between the eyebrow, eyes stare at firmly and eyes have a protrusion outward appearance, lip be not that tightly to force together be exactly nervous for square, nostril can enlarge, and can represent anger.By corners of the mouth withdrawal upwards, from the nose to the corners of the mouth on outer rim show that wrinkle, cheek improve, palpebra inferior shows that down wrinkle, palpebra inferior can raise up and take it easy, crow's foot stretches out from the tail of the eye, but emoticon.Be divided into the interior angle in triangle, upper eyelid and improve at last angle, the tractive corners of the mouth or one's lips quivering by the skin under stop eyebrow interior angle, the eyebrow, can represent sadness.
Processor produces the suitable emotion (step 555) of the language performance that is used for the intelligent social agency then.This can finish by revising the voice style from the baseline style of the voice that are used for intelligent social agency.The voice style can comprise that voice rate, tone are average, range of pitch, intensity, sound quality, tonal variations and pronunciation level.For example, in that voice rate is very fast, tone is on average very high, the non-constant width of range of pitch, voice intensity is normal, sound quality is irregular, tonal variations is normal and pronounce when accurate, the voice expression can be represented to fear.The voice style that can mean the particular emotion state is modified in the following table and proposes, and further describes at Murray I.R.﹠amp; Arnott, J.L. (1993), Toward thesimulation of emotion in synthetic speech:A review of the literature on humanvocal emotion, Journal of Acoustical Society of America, 93,1097-1108.
| Fear | Angry | Sad | Glad | Detest | |
| Voice rate | Very fast | Fast a little | Slow a little | Comparatively fast or slower | Very slow |
| Tone is average | Very high | Very high | Low a little | Very high | Very low |
| Range of pitch | Non-constant width | Non-constant width | Narrow a little | Non-constant width | Wide a little |
| Intensity | Normally | Higher | Lower | Higher | Lower |
| Sound quality | Irregular sound | The chest tone that band is breathed | Resonance | The shout that band is breathed | The chest tone of muttering |
| Tonal variations | Normally | Stiff stress | Distortion downwards | Level and smooth upwards distortion | Wide downward terminal distortion |
| Pronunciation | Accurately | Nervous | Ambiguous | Normally | Normally |
With reference to Fig. 6, process 600 processor controls produce the intelligent social agency to potential user group.This process (this can be described as projection intelligent social agency) can produce its outward appearance and sound attraction and be suitable for targeted customer's intelligent social agency.
Process 600 starts from being stored in user profile (step 605) in the elemental user abridged table by processor access.Be stored in user profile in the elemental user abridged table and can comprise relevant user's personal characteristics's (as name, age, sex, race or nationality's information and preferred language) and relevant user's professionalism (as occupation, post and one or more member organization).
Processor receives relevant intelligent social agency's role's information (step 610) to one or more concrete application programs.For example, the intelligent social agency can be used as and helps the agency, so that the function help information of relevant application program to be provided, perhaps can be used as the amusement player in the game application.
Processor use to attract rule then, further analyzes basic user profile, and selects to be used for to attract the intelligent social agency's of potential user group visual appearance (step 620).Processor can be used decision logic, and the concrete visual appearance that will be used for intelligent social agency is associated with concrete age group, occupation, sex is ethnic or cultural group.For example, decision logic can be based on similar attraction (age, individual character and the ethnic identity that promptly mate intelligent social agency and user).Seem the specialty speak the head can be more suitable for administrator user (as administrative execution president or financial executive chairman), and have extremely modern hair style speak the head can more attract the artist.
Processor is used and is fit to rule, further analyzes basic user profile, and revises intelligent social agency's projection (step 630).For example, male sex's intelligent social agency can be more suitable for technical theme, and women's intelligent social agency can be more suitable for fashion and decorate theme.
Processor presents the visual appearance (step 640) that is used for the intelligent social agency to the user then.Some realizations can allow user's modification intelligent social agency's attribute (as hair color, eye color and skin color), perhaps select from the several intelligent social agencies with different visual appearances.Some realize also can allowing the user to import figure drawing or image, to be used as intelligent social agency's visual appearance.
Processor will attract the elemental user abridged table (step 650) of rule application in storage, and will be fit to rule application is used for the intelligent social agency with selection in the elemental user abridged table of storage sound (step 660).This sound should attract the user, and is fit to by the sex of visual intelligent social agency expression (for example, the intelligent social agency with male sex's visual appearance has the male sex's sound, and has the sound that the intelligent social agency of women's visual appearance has the women).But the voice style characteristics of processor match user when being fit to intelligent social agency's sound (average, range of pitch and pronunciation) as voice rate, tone.
Processor presents the sound that is used for the intelligent social agency and selects (step 670).Some realizations can allow user's modification to be used for intelligent social agency's characteristics of speech sounds.
Processor is associated with particular user (step 680) with the intelligent social agency then.For example, processor can be associated with the intelligent social agent identifier intelligent social agency, storage intelligent social agent identifier and intelligent social agency's characteristic is in the data storage device 150 of computing machine 110, and storage intelligent social agent identifier and elemental user abridged table.Some realizations can be throwed one or more intelligent social agencies, to be fit to have user's group of similar individual or professionalism.
With reference to Fig. 7, intelligent social agency's realization is an intelligent personal assistants.Intelligent personal assistants with as the user interactions of the computing equipment of computing equipment 210, with computing equipment 210 and the use application program of assisting user in operation.Intelligent personal assistants helps the user management personal information of computing equipment, operational computations equipment 210 or operate in one or more application programs on the computing equipment, and use this computing equipment for amusement.
Intelligent personal assistants can be operated on mobile computing device, such equipment such as PDA, laptop computer or mobile phone or comprise the mixing apparatus of the function of related PDA, laptop computer or mobile phone.When intelligent personal assistants was operated on mobile computing device, intelligent personal assistants can be described as intelligent mobile personal assistant.Intelligent personal assistants also is operable in fixedly on the computing equipment, as desktop personal computer or workstation, and is operable in the system of network computing device, as described in relative Fig. 1.
Fig. 7 explanation is used for a realization of the framework 700 of intelligent personal assistants 730.Application program 710 comprises may operate at one or more application programs 725 of personal information management application program 715, one or more recreational application programs 720 and/or operational computations equipment on the computing equipment, as described in relative Fig. 1.
Intelligent personal assistants 730 uses social intelligence engine 735 mutual with user 740 and application program 710.Social intelligence engine 735 is substantially similar to the social intelligence engine 300 of Fig. 3.The information extractor 745 of intelligent personal assistants 730, with the described similar fashion of relative Fig. 3, receive from the information of relevant application program 710 and from relevant user's 740 information.
Intelligent personal assistants 730 uses adaptation engine 750 to handle the information of extracting, produce one or more responses (comprising language content and facial expression) then, so as to use language generator 755 and emotion generator 760, with the described similar fashion of relative Fig. 3, mutual with user 740.Intelligent personal assistants 730 also can produce one or more responses, so that operation operates in the one or more application programs 710 on the computing equipment 210, as described in relative Fig. 2-3 and Fig. 8-10.The response that produces can make intelligent personal assistants 730 can show when mutual attracting, that emotion is arranged, adaptive with user 740 and be fit to.User 740 is also mutual with one or more application program 710.
Fig. 8 explanation is used to realize help the framework 800 of the intelligent personal assistants of user management personal information.But intelligent personal assistants 810 assisted users 815 are as the assistant who works in all personal information management function of application.For the business users of using mobile computing device, intelligent personal assistants 810 may be used as the management assistant, helps user management appointment, email message and contacts list.With relative Fig. 3 and 7 described similar, intelligent personal assistants 810 uses social intelligence engine 825 mutual with user 815 and personal information management application program 820, and social intelligence engine 825 also comprises information extractor 830, adaptation engine 835, language generator 840 and emotion generator 845.
Personal information management application program 820 (also can be described as PIM) comprises e-mail function 850, calendar function 855, contact management function 860 and task list function 865 (also can be described as " (todo) that will do " tabulation).The personal information management application program can be the version of Microsoft Outlook that for example operate on PDA, Microsoft company, as Pocket Outlook .
Intelligent personal assistants 810 can be mutual with user 815 about e-mail function 850.For example, intelligent personal assistants 810 on weekdays beginning or when the user asks such action, can report the state of user's email account, as the quantity of unread message or have the quantity of the unread message of the emergency circumstance.Intelligent personal assistants 810 can be about unread message with emergency circumstance or when (intelligence and/or statistics according to exemplary electronic mail mode (pattern) monitor) is higher than typical case to the quantity of user's 815 unread messages, communicates by letter with user 815 with intense emotion more.The message that intelligent personal assistants 810 can notify user 815 to receive recently, and when the message that receives recently has the emergency circumstance, the communication of available more intense emotion.Intelligent personal assistants 810 can help user management message, as according to the deletion of user's exemplary message or file pattern or when the storage space of message meets or exceeds its restriction, suggestion deletion or file message, perhaps transmit message and give specific user or user's group according to user's exemplary message forward mode suggestion.
Intelligent personal assistants 810 can help the calendar 850 of user's 815 leading subscribers.For example, intelligent personal assistants 810 can be in the morning or the appointment to the arrival of he of user report in any time that the user wishes this day.Intelligent personal assistants 810 can be reminded the appointment of user's 815 arrivals in user's desired time, and how far the decision dating site is equipped with from user's present bit.If will be late or seem late to date users, intelligent personal assistants 810 can be thus with as louder remind him in a minute with the urgent mode that shows more deeply concernedly.For example, when the user does not need to advance the appointment that goes to, as business confab in the office at user place, and appointment important and urgent aspect be conventional appointment, intelligent personal assistants 810 can be with neutral emotion prompting user 815 these appointments with conventional acoustic tones and facial expression.When requiring the user to leave floor to advance time of appointment of the arrival that goes to appointment, intelligent personal assistants 810 can be with sound prompting user 815 these appointments with higher volume and more urgent emotion.
Intelligent personal assistants 810 can help user 815 to import appointment in calendar.For example, user 815 can use general or relative this appointment of speech word picture.The general description that intelligent personal assistants 810 will be dated is converted to the information that can import in calendar applications 860, and sends this information of order input in calendar.For example, the user can say " I have with Dr.Brown 41 appointment in next week ".Use social intelligence engine 825, intelligent personal assistants 810 can produce appropriate command and import appointment for calendar applications 860 in user's calendar.For example, intelligent personal assistants 810 can be understood the doctor that Dr.Brown is the user (may by carrying out the search in the contact data base 860), and must advance this doctor's office of user.Intelligent personal assistants 810 also can use the contact details in the contact management application program 860 to search the address, and can use map application to estimate the time of advancing and requiring to doctor's office, and determine the date of corresponding " next Thursday " from user's office address.Intelligent personal assistants 810 sends order to calendar applications then, with the appointment of the 1:00pm that imports the suitable date, and is created in the reminder message of enough time before the appointment, and this time is reserved the time of this doctor's that advances office to the user.
Intelligent personal assistants 810 also can help the contact 860 of user's 815 leading subscribers.For example, intelligent personal assistants 810 can add access customer 815 and said information to the new contact of intelligent personal assistants 810.For example, user 815 can say " my new doctor is the Dr.Brown of Oakdale ".Full name, address and the telephone number of Dr.Brown searched in the website of the insurance company of intelligent personal assistants 810 by using the user, and the doctor who accepts to pay from user's insurance carrying people is listed in this website.Intelligent personal assistants 810 sends order to contacts application 860 adding contact details then.Intelligent personal assistants 810 can be by adding access customer 815 input the new contact of cross reference contact, as also adding the contact details of Dr.Brown down, thereby help to organize this contacts list " doctor ".
Intelligent personal assistants 810 can help the task list application 865 of user's 815 leading subscribers.For example, intelligent personal assistants 810 can add the information that is used for new task, when the literal that may not watch computing equipment the user shows, as the user when driving a car, read task list and give the user, and remind user's task of short maturities.Intelligent personal assistants 810 can use the task 815 short maturitiess, that have higher important level of the sound prompting user with higher volume and more urgent emotion.
Some personal information management application programs can comprise sound Email and phone call functions (not shown).The sound Email that intelligent personal assistants 810 can help leading subscriber 815 to receive is as by broadcast message, storing message or report message state (for example having received how many new informations).When having passed through more time than the typical time of its sound email message of customer inspection, intelligent personal assistants 810 can use the sound prompting user 815 with higher volume and more urgent emotion also not play new information.
Intelligent personal assistants 810 can help user management user's call.By receiving and optionally handling the call that receives, intelligent personal assistants 810 can show as the virtual secretary that intelligent personal assistants 810 seems user 815.For example, busy and when not wanting to receive call the user, the calling that intelligent personal assistants 810 can not notify the user to arrive.Intelligent personal assistants 810 can be according to the precedence scheme of the people's of user's appointment tabulation, optionally notify the call of subscriber-related arrival, to converse with it if receive these people's call, user, if perhaps under the specified conditions of user's appointment, receive these people's call, for example even when the user is busy, the user will converse with it.
Intelligent personal assistants 810 can also be organized and present news and give user 815.Intelligent personal assistants 810 can use news sources and news category according to user's typical scenario.In addition or alternately, user 815 can select news sources and the classification that intelligent personal assistants 810 uses.
User 815 can select intelligent personal assistants 810 by its form that produces output, and whether such output such as intelligent personal assistants only produce voice output, and only both exported in literal output on display or voice and literal.User 815 can be by using phonetic entry or clicking mute button indication intelligent personal assistants 810 and only use literal output.
Fig. 9 illustrates the framework 900 of the intelligent personal assistants of the application in the computing equipment that assists user in operation.But intelligent personal assistants 910 assisted users 915 spread all over various application programs or function.Describe ground as relative Fig. 3 and 7, intelligent personal assistants 910 is mutual with the application program 920 in user 915 and the computing equipment, and application program 920 comprises the basic function and the application as enterprise's application that operates on the equipment of relevant equipment self.Intelligent personal assistants 910 uses social intelligence engine 945 similarly, and social intelligence engine 945 comprises information extractor 950, adaptation engine 955, language generator 960 and emotion generator 965.
Some examples of the basic function of relevant calculation equipment self are to watch battery status 925 in many other functions, open or close application program 930,935 and synchrodata 940.Intelligent personal assistants 910 is mutual about the battery status in the computing equipment 925 with user 915.For example, when being lower than 10% (or other user-defined threshold value) of battery capacity, intelligent personal assistants 910 can report that battery operation is at low electric weight at battery operation.Intelligent personal assistants 910 can be advised as making screen dim or close some application, and sends order to finish those functions when user 915 accepts a proposal.
Intelligent personal assistants 910 can be mutual with user 915, opens application program 930 functions and close application program 935 functions, switch application by use.For example, when user indication should be opened specific word processing document, because the user generally closes the specific electron form document when opening specific word processing document, so intelligent personal assistants 910 can be closed the specific electron form document and be opened specific word processing document.
Intelligent personal assistants 910 can with user interactions, the data 940 between synchronous two computing equipments.For example, intelligent personal assistants 910 can send order and duplicate personal information management to Desktop Computing equipment from the portable computing device as PDA.User 915 can ask synchronous each equipment and not specify what information synchronously of wanting.Intelligent personal assistants 910 can be synchronously based on the user's who keeps in touch personal information management typical scenario, suitable and synchronous on the table task list information, but do not duplicate the appointment information that only occupy among the PDA.
On the basic function of operational computations equipment self, intelligent personal assistants 910 can assist user in operation and operate in wide range of applications on the computing equipment.The enterprise's examples of applications that is used for intelligent personal assistants 910 is business report, budget management, project management, manufacturing monitoring, stock control, buying, sells, learns and training.
At mobile enterprise door (portal), by distinguishing priority and outstanding important and urgent information, intelligent personal assistants 910 can provide great help to user 915.Being used for intelligent social acts on behalf of the environment of applications define method of framework and instructs intelligent personal assistants 910 in this situation.For example, intelligent personal assistants 910 is by showing it or it is said to the user that the sale that can give prominence to limit priority descends and warns on screen.Selling the warning situation that descends, intelligent personal assistants 910 adaptive its dictions make it simple and direct and simple and clear, quicker in a minute, and as by the concern that seems of frowning a little.Intelligent personal assistants 910 can present business report to the user by voice or graphic presentation, as Sales Reports, procurement report with as the project status of production time line (timeline).Intelligent personal assistants 910 is understood any urgent or serious problem in outstanding or these situations of mark.Intelligent personal assistants 910 can present to managerial personnel with simple and flat-footed method and ratifies a motion, and makes the user can catch the information of most critical at once but not spends many steps by his discovery information.
Figure 10 illustrates the framework 1000 of the intelligent personal assistants of the computing equipment that helps the user to be used for amusement.The intelligent personal assistants that use is used for amusement can improve user and the mutual hope of intelligent personal assistants that is used for non-entertainment applications.But intelligent personal assistants 1010 assisted users 1015 spread all over various recreational application programs.Describe ground as relative Fig. 3 and 7, intelligent personal assistants 1010 is mutual with user 1015 and computing device entertainment program 1020, as by participating in recreation, the interesting plot amusement is provided and performs as performers.Intelligent personal assistants 1010 uses social intelligence engine 1030 similarly, and social intelligence engine 1030 comprises information extractor 1035, adaptation engine 1040, language generator 1045 and emotion generator 1050.
Intelligent personal assistants 1010 can be mutual based on the recreation and the user 1015 of computing equipment by participating in.For example, when playing games with the user, for example Card Games or other recreation based on computing equipment, as the animation car race game or the recreation of playing chess, intelligent personal assistants 1010 can serve as the participant.Intelligent personal assistants 1010 can be when helping user 1015 to be used for the computing equipment of amusement, mode and user interactions when helping the user about non-recreational application programs, more to exaggerate.For example, when playing games with the user, intelligent personal assistants 1010 can more be spoken up, and uses popular expression, laughs at, and often moves up and down eyebrow, and magnifies its eyes.When the user won competitive play to intelligent personal assistants 1010, intelligent personal assistants can be praised user 1015, and perhaps when the user was defeated by intelligent personal assistants, intelligent personal assistants can be comforted the user, flattered the user, perhaps discussed and how to improve.
By the interesting plot amusement is provided, as when the user drives a car by reading story or restating sport event to the user, perhaps when the user is unhappy or tired, tell funny stories to the user, intelligent personal assistants 1010 can serve as the amusement partner.As singing the music lyrics (this can be described as " lip is synchronous ") by showing, perhaps when showing as acting on behalf of of complete health, intelligent personal assistants 1010 dances amusement with music, and intelligent personal assistants 1010 can be used as performers' performance.
Various realizations can comprise the computer software on method or process, device or system or the computer media.Will be understood that: can carry out various modifications and do not break away from the spirit and scope of claim.For example, if carry out the step of disclosed technology with different order, if and/or each parts in disclosed system make up by different way and/or replaced or replenished by other parts, still can obtain useful result.
Claims (61)
1. computer implemented method that is used to realize intelligent personal assistants comprises:
Receive the input related with the user and with the related input of application program;
Visit the user profile related with the user;
Extract environmental information from the input that receives; And
Processing environment information and user profile are to produce adaptive response by intelligent personal assistants.
2. method according to claim 1, wherein:
Described application program is the personal information management application program, and
The adaptive response that is produced by intelligent personal assistants is associated with described personal information management application program.
3. method according to claim 1, wherein:
Described application program is the application program of operational computations equipment, and
The adaptive response that is produced by intelligent personal assistants is associated with the described computing equipment of operation.
4. method according to claim 1, wherein:
Described application program is a recreational application programs, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreational application programs.
5. method according to claim 4, wherein:
Described recreational application programs is recreation, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreation.
6. the signal of computer-readable medium or propagation has and embeds computer program on it, that be configured to realize intelligent personal assistants, and described medium comprises code segment, and this code segment is disposed at:
Receive the input related with the user and with the related input of application program;
Visit the user profile related with the user;
Extract environmental information from the input that receives; And
Processing environment information and user profile are to produce adaptive response by intelligent personal assistants.
7. medium according to claim 6, wherein:
Described application program is the personal information management application program, and
The adaptive response that is produced by intelligent personal assistants is associated with described personal information management application program.
8. medium according to claim 6, wherein:
Described application program is the application program of operational computations equipment, and
The adaptive response that is produced by intelligent personal assistants is associated with the described computing equipment of operation.
9. medium according to claim 6, wherein:
Described application program is a recreational application programs, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreational application programs.
10. medium according to claim 9, wherein:
Described recreational application programs is recreation, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreation.
11. a system that is used to realize intelligent personal assistants, described system comprise the processor that connects memory device and one or more input-output apparatus, wherein said processor is disposed at:
Receive the input related with the user and with the related input of application program;
Visit the user profile related with the user;
Extract environmental information from the input that receives; And
Processing environment information and user profile are to produce adaptive response by intelligent personal assistants.
12. system according to claim 11, wherein:
Described application program is the personal information management application program, and
The adaptive response that is produced by intelligent personal assistants is associated with described personal information management application program.
13. system according to claim 11, wherein:
Described application program is the application program of operational computations equipment, and
The adaptive response that is produced by intelligent personal assistants is associated with the described computing equipment of operation.
14. system according to claim 11, wherein:
Described application program is a recreational application programs, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreational application programs.
15. system according to claim 14, wherein:
Described recreational application programs is recreation, and
The adaptive response that is produced by intelligent personal assistants is associated with described recreation.
16. a device that is used to realize the intelligent social agency, described device comprises:
Information extractor, it is disposed at:
Visit the user profile related with the user,
Receive the input related with the user and
Extract environmental information from the input that receives;
Adaptation engine, it is disposed at:
From information extractor reception environment information and user profile, and processing environment information and user profile, to produce adaptive output; And
The output generator, it is disposed at:
Receive adaptive output from adaptation engine, and the adaptive output among the expression intelligent social agency.
17. device according to claim 16, wherein said input are the physiological datas related with the user, and described information extractor is disposed at this physiological data of reception.
18. device according to claim 16, wherein said input are the application informations related with the user, and described information extractor is disposed at the reception application information related with the user.
19. device according to claim 16, wherein said information extractor also are disposed at the information of extracting relevant user's affective state from the input that receives.
20. device according to claim 19, wherein said information extractor are disposed at according to the physiologic information related with the user, extract the information of relevant user's affective state.
21. device according to claim 19, the described information extractor that wherein is disposed at the information of the affective state that extracts relevant user is disposed at: by extracting the characteristics of speech sounds of language content and analysis user, according to the voice analysis information related, extract the information of relevant user's affective state with the user.
22. device according to claim 19, the described information extractor that wherein is disposed at the information of the affective state that extracts relevant user is disposed at: extract information based on the relevant user's of language message affective state from the input that receives.
23. device according to claim 16 wherein is disposed at the described information extractor of extracting environmental information and is disposed at:, extract user's geographic position by using GPS.
24. device according to claim 23 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: extracts the information based on user's geographic position.
25. device according to claim 16 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: extracts the information about the application content related with the user.
26. device according to claim 16 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: the information of extracting relevant user's diction from the input that receives.
27. device according to claim 16, wherein:
Described output generator is the language generator;
Be disposed at the described adaptation engine that produces adaptive output and be disposed at the generation language performance; And
Described language generator produces the language performance among the intelligent social agency.
28. device according to claim 16, wherein:
Described generator is the emotion generator;
Be disposed at the described adaptation engine that produces adaptive output and be disposed at the generation facial expression; And
Described emotion generator is represented the facial expression among the intelligent social agency.
29. device according to claim 16, wherein said output generator are multi-mode output generators, it uses first pattern and second pattern at least one, the adaptive output among the expression intelligent social agency.
30. device according to claim 29, wherein:
Described first pattern is a language mode;
Described second pattern is an emotion model;
Being disposed at the described adaptation engine that produces adaptive output is disposed at:
Produce facial expression and
Produce language performance; And
Described multi-mode output generator is represented facial expression and the language performance among the intelligent social agency.
31. device according to claim 16, wherein:
Described adaptation engine also is disposed at generation will be by the emotional expression of intelligent social agency expression; And
Described output generator is disposed at the emotional expression among the expression intelligent social agency.
32. one kind is used to realize and the intelligent social agency's of user interactions mobile device that described mobile device comprises:
Processor, connected storage and one or more input-output apparatus;
Be disposed at the social intelligence engine mutual with processor, described social intelligence engine comprises:
Information extractor, it is disposed at:
Visit the user profile related with the user,
Receive the input related with the user and
Extract environmental information from the input that receives;
Adaptation engine, it is disposed at:
From information extractor reception environment information and user profile, and processing environment information and user profile, to produce adaptive output; And
The output generator, it is disposed at:
Receive adaptive output from adaptation engine, and the adaptive output among the expression intelligent social agency.
33. mobile device according to claim 32, wherein said input are the physiological datas related with the user, and described information extractor is disposed at this physiological data of reception.
34. mobile device according to claim 32, wherein said input are the application informations related with the user, and described information extractor is disposed at this application information of reception.
35. mobile device according to claim 32, wherein said information extractor also are disposed at the information of extracting relevant user's affective state from the input that receives.
36. mobile device according to claim 35, wherein said information extractor are disposed at according to the physiologic information related with the user, extract the information of relevant user's affective state.
37. mobile device according to claim 35, the described information extractor that wherein is disposed at the information of the affective state that extracts relevant user is disposed at: by extracting the characteristics of speech sounds of language content and analysis user, according to the voice analysis information related, extract the information of relevant user's affective state from the input that receives with the user.
38. mobile device according to claim 35, the described information extractor that wherein is disposed at the information of the affective state that extracts relevant user is disposed at: extract information based on the relevant user's of language message affective state from the input that receives.
39. mobile device according to claim 32 wherein is disposed at the described information extractor of extracting environmental information and is disposed at:, extract user's geographic position by using GPS.
40. mobile device according to claim 35 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: extracts the information based on user's geographic position.
41. mobile device according to claim 32 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: extracts the information about the application content related with the user.
42. mobile device according to claim 32 wherein is disposed at the described information extractor of extracting environmental information and is disposed at: the information of extracting relevant user's diction from the input that receives.
43. mobile device according to claim 32, wherein:
Described output generator is the language generator;
Be disposed at the described adaptation engine that produces adaptive output and be disposed at the generation language performance; And
Described language generator produces the language performance among the intelligent social agency.
44. mobile device according to claim 32, wherein:
Described generator is the emotion generator;
Be disposed at the described adaptation engine that produces adaptive output and be disposed at the generation facial expression; And
Described emotion generator is represented the facial expression among the intelligent social agency.
45. mobile device according to claim 32, wherein said output generator are multi-mode output generators, it uses first pattern and second pattern at least one, the adaptive output among the expression intelligent social agency.
46. according to the described mobile device of claim 45, wherein:
Described first pattern is a language mode;
Described second pattern is an emotion model;
Being disposed at the described adaptation engine that produces adaptive output is disposed at:
Produce facial expression and
Produce language performance; And
Described multi-mode output generator is represented facial expression and the language performance among the intelligent social agency.
47. mobile device according to claim 32, wherein:
Described adaptation engine also is disposed at generation will be by the emotional expression of intelligent social agency expression; And
Described output generator is disposed at the emotional expression among the expression intelligent social agency.
48. a method that realizes the intelligent social agency, described method comprises:
Receive the input related with the user;
Visit the user profile related with the user;
Extract environmental information from the input that receives; And
Processing environment information and user profile will be by the adaptive outputs of intelligent social agency expression to produce.
49. according to the described method of claim 48, the wherein said input related with the user comprises the physiological data related with the user.
50. according to the described method of claim 48, the wherein said input related with the user comprises the application information related with the user.
51., wherein extract the information that environmental information comprises the affective state that extracts relevant user according to the described method of claim 48.
52. according to the described method of claim 51, the information of wherein extracting relevant user's affective state is based on the physiologic information related with the user.
53. according to the described method of claim 51, the information of wherein extracting relevant user's affective state is based on the voice analysis information related with the user.
54. according to the described method of claim 51, the information of wherein extracting relevant user's affective state is based on the language message from the user.
55., wherein extract environmental information and comprise the geographic position of extracting the user according to the described method of claim 48.
56., wherein extract environmental information and comprise the information of extraction based on user's geographic position according to the described method of claim 55.
57., wherein extract environmental information and comprise the information of extracting about the application content related with the user according to the described method of claim 48.
58., wherein extract the information that environmental information comprises the diction that extracts relevant user according to the described method of claim 48.
59. according to the described method of claim 48, wherein said adaptive output comprises will be by the language performance of intelligent social agency expression.
60. according to the described method of claim 48, wherein said adaptive output comprises will be by the facial expression of intelligent social agency expression.
61. according to the described method of claim 48, wherein said adaptive output comprises will be by the emotional expression of intelligent social agency expression.
Applications Claiming Priority (8)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US35934802P | 2002-02-26 | 2002-02-26 | |
| US60/359,348 | 2002-02-26 | ||
| US10/134,679 | 2002-04-30 | ||
| US10/134,679 US20030163311A1 (en) | 2002-02-26 | 2002-04-30 | Intelligent social agents |
| US10/158,213 US20030167167A1 (en) | 2002-02-26 | 2002-05-31 | Intelligent personal assistants |
| US10/158,213 | 2002-05-31 | ||
| US10/184,113 US20030187660A1 (en) | 2002-02-26 | 2002-06-28 | Intelligent social agent architecture |
| US10/184,113 | 2002-06-28 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1643575A true CN1643575A (en) | 2005-07-20 |
| CN100339885C CN100339885C (en) | 2007-09-26 |
Family
ID=27767911
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB038070065A Expired - Lifetime CN100339885C (en) | 2002-02-26 | 2003-02-26 | Intelligent personal assistants |
Country Status (4)
| Country | Link |
|---|---|
| EP (1) | EP1490864A4 (en) |
| CN (1) | CN100339885C (en) |
| AU (1) | AU2003225620A1 (en) |
| WO (1) | WO2003073417A2 (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2013071738A1 (en) * | 2011-11-14 | 2013-05-23 | Hou Wanchun | Personal dedicated living auxiliary equipment and method |
| CN103546503A (en) * | 2012-07-10 | 2014-01-29 | 百度在线网络技术（北京）有限公司 | Voice-based cloud social system, voice-based cloud social method and cloud analysis server |
| CN103543979A (en) * | 2012-07-17 | 2014-01-29 | 联想(北京)有限公司 | Voice outputting method, voice interaction method and electronic device |
| CN105893771A (en) * | 2016-04-15 | 2016-08-24 | 北京搜狗科技发展有限公司 | Information service method and device and device used for information services |
| US9491256B2 (en) | 2008-03-05 | 2016-11-08 | Sony Corporation | Method and device for personalizing a multimedia application |
| CN106486111A (en) * | 2016-10-14 | 2017-03-08 | 北京光年无限科技有限公司 | Many tts engines output word speed control method and system based on intelligent robot |
| CN107003997A (en) * | 2014-12-04 | 2017-08-01 | 微软技术许可有限责任公司 | Type of emotion for dialog interaction system is classified |
| CN107533564A (en) * | 2015-04-29 | 2018-01-02 | 微软技术许可有限责任公司 | Personalized greet is provided on digital assistants |
| CN109074809A (en) * | 2016-07-26 | 2018-12-21 | 索尼公司 | Information processing equipment, information processing method and program |
| CN109310353A (en) * | 2016-06-06 | 2019-02-05 | 微软技术许可有限责任公司 | Information is conveyed via computer implemented agency |
| CN112905284A (en) * | 2017-05-08 | 2021-06-04 | 谷歌有限责任公司 | Initiating sessions with automated agents via selectable graphical elements |
| CN115277951A (en) * | 2022-07-26 | 2022-11-01 | 云南电网有限责任公司信息中心 | Intelligent voice outbound method, device, equipment and medium |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE102004001801A1 (en) * | 2004-01-05 | 2005-07-28 | Deutsche Telekom Ag | System and process for the dialog between man and machine considers human emotion for its automatic answers or reaction |
| DE102010012427B4 (en) * | 2010-03-23 | 2014-04-24 | Zoobe Gmbh | Method for assigning speech characteristics to motion patterns |
| EP2672379A1 (en) * | 2012-06-06 | 2013-12-11 | BlackBerry Limited | Method and device for data entry |
| US9786296B2 (en) * | 2013-07-08 | 2017-10-10 | Qualcomm Incorporated | Method and apparatus for assigning keyword model to voice operated function |
| US10803850B2 (en) | 2014-09-08 | 2020-10-13 | Microsoft Technology Licensing, Llc | Voice generation with predetermined emotion type |
| CN105744090A (en) * | 2014-12-09 | 2016-07-06 | 阿里巴巴集团控股有限公司 | Voice information processing method and device |
| WO2016176375A1 (en) * | 2015-04-29 | 2016-11-03 | Microsoft Technology Licensing, Llc | Providing personalized greetings on a digital assistant |
| US9866927B2 (en) | 2016-04-22 | 2018-01-09 | Microsoft Technology Licensing, Llc | Identifying entities based on sensor data |
| US10945129B2 (en) | 2016-04-29 | 2021-03-09 | Microsoft Technology Licensing, Llc | Facilitating interaction among digital personal assistants |
| US10824932B2 (en) | 2016-04-29 | 2020-11-03 | Microsoft Technology Licensing, Llc | Context-aware digital personal assistant supporting multiple accounts |
| US10395652B2 (en) | 2016-09-20 | 2019-08-27 | Allstate Insurance Company | Personal information assistant computing system |
| WO2018157329A1 (en) | 2017-03-01 | 2018-09-07 | Microsoft Technology Licensing, Llc | Providing content |
| US10853717B2 (en) | 2017-04-11 | 2020-12-01 | Microsoft Technology Licensing, Llc | Creating a conversational chat bot of a specific person |
| US10643632B2 (en) | 2018-01-12 | 2020-05-05 | Wells Fargo Bank, N.A. | Automated voice assistant personality selector |
| US11509659B2 (en) | 2018-09-18 | 2022-11-22 | At&T Intellectual Property I, L.P. | Context-based automated task performance for user contacts |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP0247163B1 (en) * | 1985-11-27 | 1992-02-05 | Trustees Of Boston University | Pattern encoding system |
| US6021403A (en) * | 1996-07-19 | 2000-02-01 | Microsoft Corporation | Intelligent user assistance facility |
| US6185534B1 (en) * | 1998-03-23 | 2001-02-06 | Microsoft Corporation | Modeling emotion and personality in a computer user interface |
| US6151571A (en) * | 1999-08-31 | 2000-11-21 | Andersen Consulting | System, method and article of manufacture for detecting emotion in voice signals through analysis of a plurality of voice signal parameters |
-
2003
- 2003-02-26 EP EP03743263A patent/EP1490864A4/en not_active Ceased
- 2003-02-26 AU AU2003225620A patent/AU2003225620A1/en not_active Abandoned
- 2003-02-26 WO PCT/US2003/006218 patent/WO2003073417A2/en not_active Application Discontinuation
- 2003-02-26 CN CNB038070065A patent/CN100339885C/en not_active Expired - Lifetime
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9491256B2 (en) | 2008-03-05 | 2016-11-08 | Sony Corporation | Method and device for personalizing a multimedia application |
| WO2013071738A1 (en) * | 2011-11-14 | 2013-05-23 | Hou Wanchun | Personal dedicated living auxiliary equipment and method |
| CN103546503A (en) * | 2012-07-10 | 2014-01-29 | 百度在线网络技术（北京）有限公司 | Voice-based cloud social system, voice-based cloud social method and cloud analysis server |
| CN103546503B (en) * | 2012-07-10 | 2017-03-15 | 百度在线网络技术（北京）有限公司 | Voice-based cloud social intercourse system, method and cloud analysis server |
| CN103543979A (en) * | 2012-07-17 | 2014-01-29 | 联想(北京)有限公司 | Voice outputting method, voice interaction method and electronic device |
| CN107003997A (en) * | 2014-12-04 | 2017-08-01 | 微软技术许可有限责任公司 | Type of emotion for dialog interaction system is classified |
| US10515655B2 (en) | 2014-12-04 | 2019-12-24 | Microsoft Technology Licensing, Llc | Emotion type classification for interactive dialog system |
| CN107533564A (en) * | 2015-04-29 | 2018-01-02 | 微软技术许可有限责任公司 | Personalized greet is provided on digital assistants |
| CN107533564B (en) * | 2015-04-29 | 2021-03-26 | 微软技术许可有限责任公司 | Providing personalized greetings on a digital assistant |
| CN105893771A (en) * | 2016-04-15 | 2016-08-24 | 北京搜狗科技发展有限公司 | Information service method and device and device used for information services |
| CN109310353A (en) * | 2016-06-06 | 2019-02-05 | 微软技术许可有限责任公司 | Information is conveyed via computer implemented agency |
| CN109074809A (en) * | 2016-07-26 | 2018-12-21 | 索尼公司 | Information processing equipment, information processing method and program |
| CN109074809B (en) * | 2016-07-26 | 2020-06-23 | 索尼公司 | Information processing apparatus, information processing method, and computer-readable storage medium |
| CN106486111A (en) * | 2016-10-14 | 2017-03-08 | 北京光年无限科技有限公司 | Many tts engines output word speed control method and system based on intelligent robot |
| CN112905284A (en) * | 2017-05-08 | 2021-06-04 | 谷歌有限责任公司 | Initiating sessions with automated agents via selectable graphical elements |
| CN115277951A (en) * | 2022-07-26 | 2022-11-01 | 云南电网有限责任公司信息中心 | Intelligent voice outbound method, device, equipment and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| EP1490864A2 (en) | 2004-12-29 |
| WO2003073417A3 (en) | 2003-12-04 |
| WO2003073417A2 (en) | 2003-09-04 |
| EP1490864A4 (en) | 2006-03-15 |
| CN100339885C (en) | 2007-09-26 |
| AU2003225620A1 (en) | 2003-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN100339885C (en) | Intelligent personal assistants | |
| CN110688911B (en) | Video processing method, device, system, terminal equipment and storage medium | |
| Feine et al. | A taxonomy of social cues for conversational agents | |
| US20030167167A1 (en) | Intelligent personal assistants | |
| US20030187660A1 (en) | Intelligent social agent architecture | |
| CN109040471B (en) | Emotion prompting method and device, mobile terminal and storage medium | |
| Vinciarelli et al. | Bridging the gap between social animal and unsocial machine: A survey of social signal processing | |
| Brown et al. | (Im) politeness: Prosody and gesture | |
| US11646026B2 (en) | Information processing system, and information processing method | |
| US20140067397A1 (en) | Using emoticons for contextual text-to-speech expressivity | |
| JP2007272773A (en) | Interactive interface control system | |
| CN110493123A (en) | Instant communication method, device, equipment and storage medium | |
| CN109308178A (en) | A kind of voice drafting method and its terminal device | |
| CN112148850A (en) | Dynamic interaction method, server, electronic device and storage medium | |
| Herring et al. | Animoji performances | |
| KR20170135598A (en) | System and Method for Voice Conversation using Synthesized Virtual Voice of a Designated Person | |
| CN111063346A (en) | Cross-media star emotion accompany interaction system based on machine learning | |
| JP3595041B2 (en) | Speech synthesis system and speech synthesis method | |
| CN110781329A (en) | Image searching method and device, terminal equipment and storage medium | |
| CN114566187B (en) | Method of operating a system comprising an electronic device, electronic device and system thereof | |
| JPWO2019167848A1 (en) | Data conversion system, data conversion method and program | |
| JP2006330060A (en) | Speech synthesizer, speech processor, and program | |
| CN110795581B (en) | Image searching method and device, terminal equipment and storage medium | |
| CN109559760A (en) | A kind of sentiment analysis method and system based on voice messaging | |
| Fujita et al. | Virtual cognitive model for Miyazawa Kenji based on speech and facial images recognition. |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| CX01 | Expiry of patent term |
Granted publication date: 20070926 |
|
| CX01 | Expiry of patent term |