CN107000210A - Apparatus and method for providing lasting partner device - Google Patents

Apparatus and method for providing lasting partner device Download PDF

Info

Publication number
CN107000210A
CN107000210A CN201580049757.2A CN201580049757A CN107000210A CN 107000210 A CN107000210 A CN 107000210A CN 201580049757 A CN201580049757 A CN 201580049757A CN 107000210 A CN107000210 A CN 107000210A
Authority
CN
China
Prior art keywords
pcd
user
platform
ability
behavior
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201580049757.2A
Other languages
Chinese (zh)
Inventor
C.布里兹尔
A.米绍
F.拉伯格
J.L.罗斯
C.M.桑德
E.D.奎尔泽
F.法里迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jibo Inc
Original Assignee
Jibo Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to US201462024738P priority Critical
Priority to US62/024738 priority
Application filed by Jibo Inc filed Critical Jibo Inc
Priority to PCT/US2015/040581 priority patent/WO2016011159A1/en
Publication of CN107000210A publication Critical patent/CN107000210A/en
Pending legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00Manipulators not otherwise provided for
    • B25J11/0005Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
    • B25J11/001Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means with emotions simulating means
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1656Programme controls characterised by programming, planning systems for manipulators
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06NCOMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computer systems based on biological models
    • G06N3/004Artificial life, i.e. computers simulating life
    • G06N3/006Artificial life, i.e. computers simulating life based on simulated virtual individual or collective life forms, e.g. single "avatar", social simulations, virtual worlds or particle swarm optimisation
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/40Robotics, robotics mapping to robotics vision
    • G05B2219/40393Learn natural high level command, associate its template with a plan, sequence
    • GPHYSICS
    • G05CONTROLLING; REGULATING
    • G05BCONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00Program-control systems
    • G05B2219/30Nc systems
    • G05B2219/40Robotics, robotics mapping to robotics vision
    • G05B2219/40411Robot assists human in non-industrial environment like home or office

Abstract

One kind, which is used to develop, is used for lasting partner device(PCD)The development platform of technical ability include:Assets development library, with API(API), the API(API)Be configured to enable developers to carry out it is following at least one:It was found that, create, editor and access can be used for create technical ability one or more content assets;Representation aids external member, with one or more API, the one or more expression associated with the technical ability such as specified by the developer is received via the API, wherein the technical ability can be run in response to the input that at least one is defined by the PCD;Behavior editing machine, one or more behavior sequences for specifying the PCD for the technical ability;And technical ability deployment facility, the API with the runtime engine for the technical ability to be deployed to PCD.

Description

Apparatus and method for providing lasting partner device
The cross reference of related application
This application claims the rights and interests for the U.S. Provisional Patent Application sequence number 62/024738 submitted on July 15th, 2014.
All above-mentioned applications are fully incorporated in text by reference.
Technical field
Present application relates generally to lasting partner device.Especially, the application be related to for the device that provides partner device and Method, the partner device is suitable to be continuously present in the environment of people and provide to device with the user mutual of partner device Sensibility and/or received with the user on partner device and the application interacted of partner device, content, service or longitudinal data Collection association.
Background technology
Association area is described
Although the device of such as smart phone and tablet PC has increased ability(Such as networking features, fine definition are regarded Frequently, touch interface and application), this kind of device be confined in their ability with human user contact with, for example with from device Interaction provides the benefit of partnership and enhancing emotional experience.In the presence of the improvement device and correlation technique for providing partnership The need for system.
The content of the invention
This disclosure relates to which the method and system for providing partner device, the partner device is suitable to the ring for being continuously present in people There is provided in border and with the user mutual of partner device to the sensibility of device and/or with the user on partner device with Application, content, service or the longitudinal data of the interaction of partner device collect association.Device can be with related hardware, software and Other assemblies interaction is to provide the part of the system of rich interaction for extensive use, as further described herein.
According to plasticity and non-limiting example is illustrated, for developing opening for the technical ability for lasting partner device (PCD) Hair platform includes:Assets development library, with application programming interface (API), its be configured to enable developers to carry out it is following in It is at least one:It was found that, create, editor and access and can be used for creating the one or more content assets of technical ability that can be run by PCD; Representation aids group, with one or more API, through be received by it with as the associated one kind of the technical ability specified by developer or A variety of expression, wherein technical ability can be run in response to the input that at least one is defined by PCD;Behavior editing machine, for specifying use In the PCD of technical ability one or more behavior sequences;Facility is disposed with technical ability, with arriving runtime engine for disposing technical ability API is used to run technical ability.
According to plasticity and non-limiting example is illustrated, SDK (SDK) Develop ability is used for realizing Platform include being configured to the logic level module for being input to coded response that mapping receives and a kind of including being configured to detection Or the visual performance mould of a variety of visual performance events and the one or more visual performance events detected of notification logic level module Voice/the sound for the voice/sound that the perception level module of block, the sound for being configured to detection definition and notification logic level module are detected Sound identifier is moved with the one or more animations for being configured to generate the emotion/ownness for expressing definition with one or more are transmitted Sign in the expression engine of logic level module.
Brief description of the drawings
In figure(It is not necessarily drawn to scale)In, similar numeral can describe the substantially similar part in some views. Like numeral with different letter suffix can represent the different instances of substantially similar part.Figure generally through example rather than The mode of limitation is shown in the detailed description for some embodiments discussed in the literature.
Fig. 1 illustrates many views of the PCD according to exemplary and non-limiting example;
Fig. 2 illustrates the software architecture of the PCD according to exemplary and non-limiting example;
Fig. 3 diagrams are according to exemplary and non-limiting example psychology-social activity interactive module(PSIM)Framework;
Fig. 4 is illustrated according to exemplary and non-limiting example, shown by the Task Network of the interactive simple version of PCD greeting Network;
Fig. 5 illustrates the hardware structure of the PCD according to exemplary and non-limiting example;
Fig. 6 illustrates the mechanical framework of the PCD according to exemplary and non-limiting example;
Method of Fig. 7 diagrams according to exemplary and non-limiting example, for providing call answering and messaging service Flow chart;
Fig. 8 diagrams are relayed the flow chart of the method for story according to exemplary and non-limiting example by PCD;
Fig. 9 diagrams according to exemplary and non-limiting example, indicated by using PCD and/or influence user feeling state Method flow chart;
Figure 10 diagrams are realized the flow of the method for story performance or animation feature according to exemplary and non-limiting example by PCD Figure;
Side of Figure 11 diagrams according to exemplary and non-limiting example, for generating background story and being encoded to background story The flow chart of method;
Figure 12 diagrams are according to exemplary and non-limiting example, for accessing interaction data and user's need being solved using it The flow chart for the method asked;And
The method that Figure 13 diagrams adjust PCD behaviors according to exemplary and non-limiting example, for being inputted based on user Flow chart.
Figure 14 diagrams are according to exemplary and non-limiting example, and display circulation, lasting or semi-durable vision element show Example.
Figure 15 diagrams are according to exemplary and non-limiting example, and display circulation, lasting or semi-durable vision element show Example.
Figure 16 diagrams are according to exemplary and non-limiting example, and display circulation, lasting or semi-durable vision element show Example.
Figure 17 illustrates the exemplary and non-limiting example of the run time technical ability for PCD.
Figure 18 be realize using DK Develop abilities platform flow chart and various infrastructure components it is exemplary and unrestricted The explanation of property embodiment.
Figure 19 is the explanation for the exemplary and non-limiting example that can provide the user interface for creating assets.
Figure 20 is that developer can be allowed to see the visual chemical industry in the local sensing space (LPS) in PCD local sensing space The explanation of the exemplary and non-limiting screenshot capture of tool.
Figure 21 is the explanation of the screenshot capture of the behavior editing machine according to exemplary and non-limiting example.
Figure 22 is the explanation of the formal way of the establishment branching logic according to exemplary and non-limiting example.
Figure 23 is that thus selection logic can be added to the exemplary and unrestricted of behavior as independent variable (argument) The explanation of property embodiment.
Figure 24 is the explanation of the exemplary and non-limiting example of analogue window.
Figure 25 is the exemplary and non-limiting reality of the social the robot animation editing machine of social robot representation aids group Apply the explanation of example.
Figure 26 is the explanation of the exemplary and non-limiting example of PCD animations movement instrument.
Embodiment
According to exemplary and non-limiting example there is provided and describe have lasting partner device(PCD), for persistently depositing Be in the environment of people/user and with the user mutual of partner device.As used herein, " PCD " and " social robot " Can be with used interchangeably, except in addition in the case that context is indicated otherwise.As described more fully below, PCD is to lasting Social activity, which exists, provides completely different role, its by movement, figure, sound, light, smell and rich in expression.Enter one below Step introduces the concept for having " digital soul " supporter to PCD each embodiment.As used herein, " digital soul " refers to energy Enough using number format store, serve as input for determined by PCD and run action multiple attributes.As used herein , " environment " refers near user(It is enough to allow to observe user by PCD sensor)The physical environment of interior user.
The digital soul operates into social activity-emotion/interpersonal relationships sense supporter via interaction/interface to PCD in society Contacted with interaction and harmonious relation construction activities with user.As described more fully below, PCD 100 can be held to its user Row many kinds function.According to the exemplary and non-limiting example being described in detail below, PCD can(1)Promote and backer/ More meaningful participatory between user, physics insertion, the interaction of social positioning and(2)It may participate in holding for utilitarianism task OK, wherein PCD serves as assistant or provides the something of individual service, and it is including but not limited to user's offer useful information, help Scheduling, prompting, offer special services(For example serve as photographer)To help family's establishment/reservation/shared family's story and knowledge (For example, special formulation)Deng, and(3)Entertain user(For example, story, game, music and other media or content)And carry For accompanying and partnership.
According to exemplary and non-limiting example, PCD various functions can be completed via multiple operator schemes, these Operator scheme including but not limited to:
I. via interface of personalizing, the scope of different personality characters is alternatively expressed, it, which is included, to change with temporal adaptation to provide The speciality of kind partnership.
Ii. by talking warm user-centered interface, it can mail message and emotion.As described below, This interface can be showed emotion by many clues, emotion and individual character, and these clues include facial expression(Pass through animation or shifting It is dynamic), body movement, figure, sound, voice, color, light, smell and the like.
Iii. via across multiple sources(Sensor, data, the information from other devices, internet, GPS etc.)Obtain language Borderization longitudinal direction information makes PCD increasingly to its one or more users adjustment, adapts to and tune.
Iv. via adaptive self-configuration/self- recoverage more preferably to match user the need for/wish.
V. via the social activity and emotion details for considering particular condition and its user.
With reference to Fig. 1, it is illustrated that the PCD 100 of with good grounds exemplary and non-limiting example many views.As shown, PCD 100 includes many exemplary input/sensor devices, and it includes such as capacitance sensor 102.One or more electric capacity are passed Sensor 102 is operable to sensing physics social activity interaction(Its including but not limited to by comforting, embrace, touch and the like)And it is potential Serve as user interface in ground.PCD 100 can further include touch-screen 104 as such device, and it is configured to receive from user Input and for playing a part of graphic alphanumeric display from PCD 100 to user's output data.PCD 100 can further include one Individual or multiple filming apparatus 106, for receiving visual signature(It is including but not limited to still image and video)Input.PCD 100 can further include one or more control sticks 108, be inputted for being received from user.PCD 100 can further include one Or multiple loudspeakers 110, for sending or with exporting voice data otherwise.PCD 100 can further include one or many Individual microphone 112.
PCD software architectures
With reference to Fig. 2, it is illustrated that there is the block diagram for describing the software architecture 200 according to exemplary and non-limiting example.The software frame Structure 200 is adaptable to the technology of such as artificial intelligence, machine learning, and with the software and hardware system associated, it can make PCD 100 Experience can be provided for life, as can be by robotic embodiment and by being connect across wide variety of attachment means with people The emotional resonance role being in harmony.
According to exemplary and non-limiting example, one or more classifications are divided into the intelligence associated of PCD 100, its Can be by the social code coding of the mankind into machine.In certain embodiments, these one or more classifications can be PCD cognition- The basis of mood framework.One or more classifications may include but be not limited to psychology-social perceive, psychology-social study, psychology- Social activity interaction, psychology-social expression and the like.The psychology of intelligence-social activity, which perceives classification, may include the social clue of the mankind(Example Such as, vision, the sense of hearing, tactile)Integrated machine perceive, for supporting PCD 100 natural social interface and far field to interact.The heart Reason-social activity study classification may include algorithm, and PCD 100 can be by it by direct interaction and via the next free Hes of PCD 100 Identity, activity pattern, preference and the interest of people are learned in the data analysis for the multi-modal data that the device ecosystem is captured. PCD can record the speech samples of the people of the near field for entering it or far-field communication scope and be examined using speech recognition system The identity and personal data of the people of survey.In addition, PCD can detect the UUID that is broadcasted in the Discovery Channel of BLE enabled devices and Pair personal data associated with device users are decoded.PCD can be used obtain identity and personal data with from as Facebook, Twitter, LinkedIn or the like social network sites obtain additional personal information.PCD can declare in its near field or remote The display of the presence of the people detected in communication range and identity together with the personal profiles of the people of structure.
Psychology-social interactive class can make PCD 100 be able to carry out it is pre- answer formula (pro-active) decision-making process with Just support task and activity, and build and build technical ability-all pass through language with the trust of people and the harmonious relation of affectional tie With multi-modal behavior.The psychology of intelligence-social activity expression classification can enable PCD 100 for its multi-modal input editing " to show It is true to nature ", active content and by the layout (orchestra) of voice, movement, figure, sound and light come with being used as feelings People's contact of the role coordinated in sense.Framework 200 may include such module, and it corresponds to multi-modal machine cognition technology, language Sound identification, talking phonetic synthesis, and utilize cost-effective hardware module(That is, the common component of mobile device). As shown in Fig. 1, one or more software subsystems and these one or more subsystems are provided with PCD 100 It will be described in further detail below.
Psychology-social perception
PCD 100 psychology-social activity, which is perceived, may include:Auditory Perception, it can be used for manipulation phonetic entry;With visuo-spatial sense Know, it can be used for assessing the position of user, captures the emotion of user, the identity and posture that recognize user and maintenance and user Interaction.The array of microphone 202, one or more signal processing technologies can be used in PCD 100 Auditory Perception(Such as 204)With oneself Sound identification module 206 is moved to realize.In addition, Auditory Perception can be wanted with the unique sensation and processing of interactive social robot Component and technology by using being created for the mobile computing ecosystem is asked to realize.PCD 100 may include hardware and software, use In via the voice using microphone array 202 and the noise cancellation technique and third party's skill using signal processing module 204a Art scheme supports the multi-modal far field interaction to help automatic speech recognition module 206 and auditory scene analysis.
PCD 100, which can be configured to be adapted to, hears and understands what people saying in noisy environment.For doing so, Voice signal can be delivered to automatic speech recognizer(ASR)Signal processing module is passed through before module 206 204a.Voice signal is processed so that voice and static and dynamic background noise, echo, motor and other people even talked Isolate to improve ASR success rate.
According to exemplary and non-limiting example, PCD 100 can be configured to uses at least four MEMS in space configuration The array of microphone.It is also possible to use the algorithm based on sound arrival time(It is referred to herein as beamforming algorithm)With in spy Determine to isolate sound on direction.Using the placement of whole six microphone signals, direction vector and microphone, beamforming algorithm can Sound of the isolation from particular space source.Beamforming algorithm be able to can be provided on sound by allowing multiple wave beams simultaneously Multiple sources information.In addition, voice-non-voice detection algorithm can recognize speech source, and provide the space of loudspeaker Positioning.In certain embodiments, Wave beam forming information can with PCD 100 vision and awareness system it is integrated so as to choice direction with And the motor ability for rotating and being orientated.For example, 3D sensors can be used for the position and therefore of detection head part in the 3 d space Direction can be conveyed to beamforming algorithm, it can isolate it before the sound from sensing the feedback of position is transmitted into ASR modules 206 .
During operation, PCD 100 can generate sound by talking or manufacturing noise.Signal processing module 204a can match somebody with somebody Being set to prevents these sound from being fed back through microphone array 202 and entered in ASR modules 206.Made an uproar to remove loudspeaker Sound, signal processing module 204a can be used can be from the calculation that the signal for being fed to loudspeaker is deducted by microphone received signal Method.In order to reduce the motor noises that harmonic wave is abundant, PCD 100, which can be configured to, realizes mechanical means and signal processing technology.
In certain embodiments, PCD 100 can monitoring motor different mouths to solve from the generation of these parts of motor Noise.In this example, PCD 100, which can be configured to, is arranged on motor in elastomeric material, and it can absorb and can adopted by armature bearing The high-frequency produced with the form of birr.Motor may include brush, its can produce only motor rotate at high speed when it is obvious Hiss.Therefore, PCD 100 can show animation (animation) and movement to avoid hiss with relatively low speed. In addition, PCD 100, which can be configured to, realizes relatively low gear ratio and in addition by reducing motor speed to prevent hiss.It is logical Often, lower quality PWM drives(As those found in amateur's servomechanism)Toot the sound of high pitch can be produced.PCD 100 may be configured with the second best in quality PWM drivings to eliminate the motor noises of the part.Generally, the gear of motor can cause compared with The major part of the grinding sound of low pitch, its meter and motor noises.End gear drive (final gear drive) can undertake biography Most of torque in dynamic system, and so as to be the source of most of noise.PCD 100 can be configured to substitutes end with frictional drive Gear drive is held to minimize the noise source.In addition, PCD 100 can be configured to using signal processing technology so as to reduce by The noise of motor generation.In embodiment, microphone, which can be close to each motor and place, make it that noise signal can be from main microphon Signal in array 202 is deducted.
The audio-source of purification can be fed into ASR modules 206 by the output of PCD 100 audio pipeline, and it can be by voice It is converted into text and is possibly converting to the alternative competing words hypothesis rich in meaningful confidence level, such as using ASR n-best Output or word lattice.Voice(Word)Text representation then can be resolved intention to understand user and user offer information and Finally symbol is transformed into represent(It is semantic).ASR modules 206 can with normal quantity and corresponding to typical interpersonal communication distance away from From voice of the identification from user.In this example, the distance can be close to 5-6 feet or bigger, and this belongs to depending on numerous environment Property(Including ambient noise and voice quality).In this example, speech recognition scope should cover typical 12 feet × 15 feet rooms Between region.The signal for being fed to ASR modules 206 will be the result of microphone array beamforming algorithm and may originate from surrounding The acoustics angle of about +/- 30 degree of loudspeaker.Relatively narrow acoustics angle can allow actively to reduce the part of Background environmental noise and return Ring, it is the main cause of the accuracy of speech recognition of difference.In wherein voice signal too low scene, such as due to loudspeaker Very light too remotely from the sound of microphone or loudspeaker, PCD 100 can ask loudspeaker close to point with answering formula in advance(If for example, raised The distance of sound device is available as determined as 3D sensors)Or say more loud, or both have both at the same time.In some realities Apply in example, PCD 100 can be configured to using real-time embedded ASR solutions, and it can be with grammer and statistical language model (SLM)Support big vocabulary identification.In addition, acoustics ASR models can be used from acoustics equip data come train/tune so as to Improve phonetic recognization rate.
According to exemplary and non-limiting example, PCD 100 can be configured to including natural language processing layer, and it could be sandwiched in Between ASR modules 206 and PCD 100 interactive system.Natural language processing layer may include natural language understanding(NLU)Mould Block, it, which can take by the ASR texts generated and that text is assigned, looks like.In certain embodiments, NLU modules can configure For example expand Backus normal form (BNF) into form is adapted to(BNF)Mark, java language syntax forms(JSGF)Or speech recognition grammar Form(SRGF), it can be supported by Embedded Speech Recognition System device mentioned above.Because collecting increasing voiceband user, Traditional grammar gradually can be transformed into statistics grammer by PCD 100, and it can provide higher speech recognition and understanding performance, and allow Automaticdata driving is adapted to.
According to exemplary and non-limiting example, PCD 100 can be configured to for each interactive task using multi-modal Conversational system user-interface design principle carrys out design structure interactive stream(It is based upon the Task Network that PCD 100 brain is used Represent).Interactive stream may be designed in succession(For example, inputting one at a time)Or simultaneously(For example, input can be by their order of reception Independent process)Receive multi-modal input(For example, voice and touch)And generate multi-modal output(For example, voice message, PCD Movement, display icon and text).As example rather than as limitation, PCD 100 can inquire Yes/No problem, PCD's 100 Eyes become the question mark shape with the Yes/No icon that can be selected by one or more touch sensors.In embodiment, PCD 100, which may be adapted to processing, can express the natural language interaction of intention(For example, he!Let us is taken pictures!).In embodiment, hand over It can mutually be followed up using " orientation dialogue " mode.For example, after the identified intention taken pictures, PCD 100 can inquire orientation problem, For confirming just to have heard what or request additional information(For example you want to allow me to take pictures to you).
Visuo-spatial is perceived
According to exemplary and non-limiting example, PCD 100 can be configured to perceives sensing using one or more visuo-spatials Device(Such as RGB filming apparatus 212, depth filming apparatus 214 and other sensors)To receive 2D visions, 3D visions or sensing Motion or color.PCD 100 can be configured to the emotion perception for obtaining user in surrounding environment.For example, PCD 100 is detectable each The state that shows emotion of people.PCD 100 may include visuo-spatial perceive subsystem, for keep tracking user and environment when When physical state.The subsystem can be used as being total to that dynamic updates to the current state estimation of other internal software modules presentation user Data structure is enjoyed, is called local sensing space(LPS)208.LPS can be by the list centered on PCD 100 current location Combined in individual 3D coordinate systems it is multiple sensation inlet flows and build, and sensor can be used can count and his movement motion transform and It is registering in 3D.In embodiment, LPS 208 may be designed to maintain many ' rank ' information, each details for entering higher level And processing and key sensor input can be required.The ranks of LPS 208 may include:
People detects:People present in the detectable environment nearby of the rank.For example, PCD 100 can be used near sensor calculating The quantity of people.In embodiment, PCD 100 can be oriented using the visual movement queue in system.In addition, thermoelectricity is infrared (PIR)Sensing and the output of simple microphone integrated can be enabled the system to ' sleep in low-power to wake up microcontroller implementation Sleep ' in state, but still someone for entering room can be responded.This can be with visual movement clue and color segmentation models coupling To detect the presence of people.Detection can be combined with LPS 208.
People tracks:PCD 100 can be configured in 3D position people and therefore use such as vision, depth, motion, sound Actively mobile sensor determines the track of people by sound, color, feature &.For example, the combination that visual movement detection and 3D people detect Available for positioning user(Especially their head/face).In addition, LPS 208 may be adapted to include time model and other are defeated Enter to manipulate inaccessible and more synchronous people.In addition to motion and 3D clues, system can also be learned from image online(From turnover zone and 3D)Color segmentation model(Naive Bayesian)Adaptively to make user's face and hand and background separation and make multiple inputs As a result the people position detection that robust is provided system is combined with LPS 208 room and time filtering.
People recognizes:Vision sensor, hearing transducer or touch input for people ID can be used to recognize for PCD 100 Know or unknown people.In this example, one or more open-source OpenCV storehouses can be used for facial recognition modules.In addition, people tracks Information and motion detection can be combined to recognize for face detection the finite aggregate for the image district for being candidate.
Pose/Attitude Tracking:Vision sorter can be used in PCD 100(For example, face, body pose, skeleton tracking etc.)Or Mapping is touched to recognize everyone pose or posture.In embodiment, 3D data sets can be used for making this feature and PCD's 100 Sensor mode merges.In this example, open-source gesture recognition kit is adaptable to view-based access control model and the tracking of 3D visual signatures To accelerate self-defined gesture recognition.
Focus:PCD 100 can be configured to determination focal zone so that PCD 100 may point to or look at the focusing of determination Area.Various sensors are combined into the position/orientation collection for focus.For example, the estimated location of people can be in LPS 208 Generate focus position collection.These can be the most probable position of the estimation for people, together with the pass for specified location Note the confidence level of driving.Focus and direction set are defined the level by confidence level and produce the LPS 208 for being used by other modules The overall summary of data.PCD 100 these focuses and direction can be used select fixation object so as to directly to user addressing simultaneously And between a plurality of users seamlessly ' sight for overturning it '.In addition, this can allow the robots of PCD 100 to check relatively low-confidence Position is spent to confirm the presence of nearby users.
According to exemplary and non-limiting example, PCD 100 can be configured to be included activity estimation or can wrap in systems Containing more sensor mode, for by phonetic entry and the estimation of the affective state from phonetic-rhythm is tracked and known Not.The particle filter model based on grid for the input feature vector of processing can be used to combine from multiple defeated for LPS 208 The data entered.Must be by the further sensing on PCD parts and when not knowing significantly of action solution, particle filter when existing Ripple device can provide support for the expression of the robust On-line Estimation and multiple hypothesis situations of user's physical state.Particle filter skill Art can also allow association attributes naturally and feel that the mixture of input is combined into the individual probability of physically measurable amount User Status Explicit, closed mould of the model without requiring Joint Distribution.In addition, the particle filter based on grid can help to sit single 3D is merged in mark system(It is three-dimensional)And 2D(Vision)The input of sensing and implement space can be by only one in any specified time Object occupies this constraint.
According to exemplary and non-limiting example, PCD 100 can be configured to be distributed and heuristic including heuristic suggestion Transition model, it can help to time Capturing Models User Status, or even when PCD 100 can not directly not look at them also such as This.This can allow PCD 100 to use the multi-party conversation and can be easily in particle filter in turn naturally of speech and non-karst areas clue It is applicable in framework.Therefore, this statistical estimate that can allow to make robust by the way of having principle and heuristics focusing on people Combination.In addition, LPS 208 can learn prior probability distribution and will be adapted to wherein people from corridor, door can go out from repeated interaction ' focus ' in space that is existing and bypassing sales counter, and the spatial information can be used to be aligned for the most related of user automatically Position.Low-level image and signal transacting code can be customized and based on such as OpenCV, integrated vision kit(IVT), it is logical With the quality open source tool such as digital processing features and processor certain optimisation storehouse.
According to exemplary and non-limiting example, PCD 100 can be configured to the emotion that various ranks are recognized from video flowing, It is for example happy, angry, despise, detest, fearing, it is sad, puzzled, baffle with it is surprised.In embodiment, PCD 100 can be configured to Determine head position, sex, age and whether someone wears glasses, with facial hair etc..
According to exemplary and non-limiting example, audio input system is focused on user.In certain embodiments, PCD 100 can be configured to the motion that direction is for example moved, paid close attention to according to robot and estimation 3D is focused on come real-time update audio wave The direction of beam formation function.This can allow PCD 100 optionally to listen attentively to specific ' sensor ', wherein in the presence of related and active sound Frequency is inputted.This can pass through the integrated reliability to improve ASR and NLU functions with full 3D people sensing and focus.
Spatial Probability is learned
According to exemplary and non-limiting example, Spatial Probability can be used to learn technology to contribute to PCD 100 in user to enter Enter with he with when more successfully contact with.Over time, PCD 100 can remember the arrival of user and combine there is sequence And accumulate these statistical informations for designated room.This can give the prediction users of PCD 100 contact when entering room Rule, and PCD 100 thus can be enable to continue the holder that set period rotates sector and even guesses room.Example Such as, this feature can provide PCD 100 using limited prediction to support before PCD 100 can fully recognize that someone enters room As " he, Bill is you" as interaction ability.PCD 100 can most probable turn to direction in space and cause at one day The time sees someone in same time.
Psychology-social interaction
According to exemplary and non-limiting example, PCD 100 can be complete autonomous artificial image.PCD 100 can have feelings Sense, the target that himself may be selected(Based on user's input)And perform closed loop real-time control system realizing those targets so that User keeps happy and health.Psychology-social interactive module(PSIM)It is the top layer of Closed Loop Discrete time controlled system, it can locate Manage the output of sensor and to export and expressing selection action.Various support processes can concurrently be carried out on CPU, and be felt Input asynchronous can consign to decision-making block.It is " ticking(tick)" it is decision-making period, wherein PCD 100 accumulation sensor letter The purpose state of breath, current short-term memory/knowledge and task-driven can be combined to select new action and expression.
Fig. 3 A describe the framework of the PSIM 300 according to exemplary and non-limiting example.PSIM 300 core is to hold Line program 302, the operation of its layout other elements.Configuration processor 302 is responsible for regularly updating for PCD 100 brain.PSIM 300 each " ticking " may include the processing step for sending newer command movement towards psychology-social expression module using following manner Rapid collection.
Inside updates:
A. emotion updates
B. target selection
Input manipulation:
A. to from psychology-social asynchronous input sample for perceiving 304 and the blackboard 306 for being updated to decision-making module.
B. input may include the information such as people position, face ID samples, and the NLU speeches parsed form various use Family.
C. the fresh information for being updated to blackboard 306 can be only needed to behave like cache.
D. in addition, it may be desired to the capture information related to current task.
Inquiry manipulation:
A. the result from knowledge query operation is sampled as blackboard 306 from psychology-social knowledge base 308.
B. this collects the result for delaying to handle of inquiry operation to use in current decision.
Task Network 310:Thinking/renewal
A. configuration processor 302 can operation task network 310 " thinking " operation and make any required act in each level And decision-making.Active section point set in Task Network 310 can update during the process.
B. Task Network 310 acts as the flexible shape of the logic based on state machine of the sorter controller of robot interactive Formula.
Output manipulation:
A. the output for being loaded into the specific framework of blackboard 306 is transferred to psychology-social expression module 312.
According to exemplary and non-limiting example, configuration processor 302 may also provide the asynchronous of task in Task Network 310 The critical services sent.Any task in network 310 can be by asking asynchronous send to perform any computation-intensive Work and calculating is put off until concurrent background thread.This feature can allow the layout heavyweight of Task Network 310 to calculate and as dynamic The thing of work(As making network I/O slow or even blocking network I/O)Without " blocking " decision-making period or change PCD 100 certainly The reactivity of plan process.In certain embodiments, configuration processor 302 can send program operation, and it produces the new of Task Network 310 Section and they will dynamically be attached to execution tree with when product intelligent is ripe by planning ability come extended operation.Task Network 310 it is contemplated that to be concurrently layered finite state machine(CHFSM)Form.However, method is allowing used in the design of behavior tree Mankind designer and software engineer obtain immense success in terms of working together to create interactive experience in content pipeline.Appoint Business network design can realize clear, effective realization and the composition of task using traditional programming language.
Fig. 4 graphical task networks, it shows the simple version of PCD 100 greeting interaction.The framework of Task Network 310 makes Various expression, mobile, sensor operation and voice can be integrated in engine, and thus give designer the friendship to PCD 100 Mutually dynamic control completely.As shown, the tiny segment of network is active at any time during operation.It is used as content The part of establishment, visual task network representation can be used for communicating in both technology and design audient.In this example, PCD 100 pir sensor detects the people into region.PCD 100, which perceives PCD 100, can need to greet someone and start " to ask The fact that standby family " sequence." should greeting user ", sequence can make tracking act clue initialization and then say " hello ", and The tracking to user is updated when they are close.PCD 100 can keep updating the data input capturing the facial ID of user.At this In scene, ID shows that it is Jane, therefore PCD 100 is moved to next part of sequence, and wherein PCD 100 can form inspection Jane how the speech done and open he ASR/NLU processing window think that response is prepared.Once Jane says something, It is " good " or " bad " by speech classification that knowledge query, which can be used for, and PCD 100 suitable physics can be formed to Jane and voice is anti- His greeting should be completed.Network can pass on the concept how intelligence works.
Psychology-social expression
According to exemplary and non-limiting example, PCD 100 can be configured to including engine, and it can supplement PCD 100 social activity Property.For example, engine may include tag system, for changing voice output.Engine can allow the voice quality for controlling PCD 100. In this example, recording can be completed to control PCD 100 voice by voice-over actor.Engine may include for example for embedded dress The features such as the high-quality compacted voice file and simple pricing model put.In addition, PCD 100 may include:Animation engine, for pair Body joints rotation provides animation;Figure, shape, texture and color;LED lamplight, or mood coloring;Timing;With PCD's 100 In terms of any other expression.These animations can be exported with other expression such as audio cue, voice, smell.Animation draws Hold up then can at different rates, transformation and between curve playing animation all or part, while making it move with program Draw mixing in real time.The engine can flexible adaptation different PCD models, geometry and the free degree.
Dynamic alignment
According to exemplary and non-limiting example, PCD 100, which can be configured to, uses algorithm, and it can make PCD 100 in program Towards the point orientation in 3d space.PCD 100 eyes can seem to be fixed on a single point, and PCD 100 body can played Independent animation, or eyes can be leading, and body can be indicated and then in particular directions.In embodiment, it can be used for calculating The PCD enclosed geometry solver for checking target.Then the object pose is fed into multi objective mixed system, its may include pair plus Speed constraint, additivity mixing/layering and simulation VOR(Vestibulo-ocular reflex)Support.
Simulation
According to exemplary and non-limiting example, animation engine may include simulator, and it can play animation and program animation simultaneously And in fact mix them.Simulator can simulated feel input, such as face detection.In certain embodiments, it can be building up to The physical analogy of dummy model, so as to consider the current pumping limit of the quality of robot, the power of motor and robot (current draw limits)To verify and test animation.
Eyes
According to exemplary and non-limiting example, the person(For example, PCD 100 eyes)Figure represent that joint can be used Construct to allow it to make oneself to become and be shaped to different objects.Self-defined animation file can be used to make rainbow for eyes graphics engine Film becomes different shape, blinks, changes its color and change texture to allow comprehensive expression.
Figure
PCD API can support the display of in 2D scene graph styles interface figure, photo, animation, video and text.
Platform and the ecosystem
PCD 100 is the platform based on highly integrated, high-performance embedded linux system, and it is applied with mobile device " partner " The ecosystem, the rear end based on cloud and coupled with commercially available content and functional online shop.
PCD SDK
It is open and flexible flat to provide that PCD SDK can develop the Javascript and open language of community using modern times web Platform, thereon, third party developer can add low learning curve ability.All PCD applications for being created by PCD SDK, content and Service can be used for downloading using shop from PCD.PCD functions(It include TTS, sensory perception, NLU, animation and other)In it is complete Portion will be can use by PCD API.JavaScript platforms that the API is set up using node JS, at the top of V8, Chrome are opened Source JavaScript engine.Node JS uses event-based model, its quickly and efficiently and change into well robot volume Journey.Node JS is along with excessive out-of-the-box feature and can easily expand to additives.PCD API will be node JS Additives.Because additives is also easily removed or changed, developer can interact used method with PCD and can be controlled System.For example, developer can create outbound socket, but also limit the quantity of outbound connection.
Cloud framework
According to exemplary and non-limiting example, the complicated back-end platform based on cloud can be used for supporting PCD intelligence, retrieval Fresh content and enable a person to keep in touch with their family.The PCD devices of family can be via Wi-Fi connection into cloud PCD servers.Access to PCD Cloud Servers relies on the web communication protocols of high safety and encryption.Can be that can support active IOS, Android and HTML5 of PCD user, tender and kinsfolk develop various applications.Should using these movements and web It can be helped so as to provide on numerous devices always together with you and know how preferably to support all the time with, PCD 100 Your preference, demand and interest.With reference to Fig. 2, PCD 100 can be configured to the monitoring in cloud and may be such that families of the PCD 100 to him Unique data so that user to future PCD robots version and can protect the role and pass of their foundation with easily scalable System.For example, PCD server can be configured to collects data and from PCD robots and other attachment means in cloud storage 214 Computation measure is to allow machine learning algorithm to improve user model 216 and change PCD actor models 218.In addition, in cloud storage At 214 collect data can be used for analyze what PCD feature most can with user's sympathetic response, and understand making across the PCD ecosystems With pattern, so that sustained improvement product is supplied.
According to exemplary and non-limiting example, the back-end platform based on cloud can include Database Systems, for meaning In the storage and distribution of the data shared between numerous PCS.Back-end platform based on cloud can also trusteeship service apply with know Others(Such as voice ID is applied)PCD is supported with personal multi-modal data aspect is assembled by the interworking with social networks.
Server based on cloud
According to exemplary and non-limiting example, one or more PCD 100, which can be configured to, uses the web based on RESTful Service, communicated using compression JSON with the server back end based on cloud.
Security
According to exemplary and non-limiting example, zero configuration procotol can be used for checking identity together with OAUTH authentication models. In addition, apache shiro can provide the extra safety agreement around effect and license.All sensitive datas will pass through SSL Send.On the server side, configured using rigorous fire wall(Using OAUTH)It can be protected the data that obtain content token Shield.In addition, can need all call of the server based on cloud with effective content token.
Content delivery
According to exemplary and non-limiting example, using server A PI, it includes web services and calls to obtain for specifying The what be new of PCD devices.The web services can provide it is senior call, it is returned in succinct, compact taskpad is needed Examine message, alarm, update list(For example, shopping, prompting, check-in and the like)And other guide.PCD robots then can base The pending data represented in the inventory are retrieved in its current schedule opportunistic.In certain embodiments, PCD actuality It is that in cloud, this means that list, prompting, the master record of embedded and other application state are stored on PCD servers.In order to true Protect robot may have access to what be new, API can be called frequently and chance collect content(But use timely mode).
Work Process Management
According to exemplary and non-limiting example, it can be used and be offloaded to cloud and by the feature of not real-time returning result.This Can be closely consistent with the concept of the message queueing discussed above based on schedule.In addition, it can involve server architecture, it can Allow the request that is serviced on RESTful web services API and send work to application server.Amazon Simple Workflow(SWF)Or similar workflow can be used for realizing such system together with conventional message queuing system.
Update
According to exemplary and non-limiting example, it may be desired to which the content of renewal may include PCD 100 operating system nucleus, consolidate Part, hardware driver, V8 engines or buddy application.Can be by returning on the available type updated to updating for these contents Information and allow particular item ask web services and can use.Because PCD generally avoids upsetting in robot in which will need chance It can request that the User Activity updated when can quote renewal.The availability of certain form of renewal can be pushed to robot, Rather than updated by the periodic polling of PCD robots.
Record/measurement
According to exemplary and non-limiting example, PCD 100 can send log information to server.Server can be by the data It is stored in suitable container(SQL or NoSQL)In.Such as Hadoop(Amazon MapReduce)It can use with the instrument such as Splunk In analyze data.Measurement be able to can also be inquired about so that how report can interact with PCD 100 and how to make on people Run with PCD 100.Results of these analyses can be used on PCD how learning, interact and showing and also on not What feature can need carry out adjusting parameter in updating.
Machine learning
According to exemplary and non-limiting example, various training systems and feedback loop can be developed to allow PCD robots and base Improved in the systems stay of cloud.PCD robots can be used for the information of training machine learning algorithm.A certain amount of machine Study can occur in robot with, but in cloud, can assemble data to train grader from many sources.Service based on cloud Device can allow by sending the feelings that a number of data are correctly heard, seen or understood in low probability to mankind's encoder Content is cleared under condition and determines ground truth.Once creating new grader, they can be sent out by more new system discussed above Go out.The machine learning and training of grader/pre- gauge may span across supervision, no supervision or intensified learning method and ground truth Both more complicated mankind's codings.Training signal may include that PCD robots complete the task or explicit feedback produced by user(For example Speech, touch prompting, smiling face, posture etc.)Knowledge.Make the image from filming apparatus(It may include face and voice data) Aggregation can be used for the quality for improving those corresponding systems in cloud.
Remote exhibition is supported
According to exemplary and non-limiting example, remote exhibition feature can be used, it includes video session option.In addition, opening The security of user is ensured with the security model around video session.In addition, web applications and same mobile dress can be used Put application, its using effect, license and security infrastructure come protect end user in case video session ability without awarding Power is used.
Software infrastructure
The advanced abilities of PCD software systems are set up on robust and competent embedded Linux platform, the crucial storehouse of its use, Plate is supported, driver and other dependences to customize to provide our advanced software system clean and tidy, robust, reliable exploitation Environment.Top functional module is embodied as process in our embedded Linux system.PCD module basis facility it is special with Use JavaScript to support flexible content script, interaction and behavior for target, at the same based on language library support using C++ with C calculating taxation operation.It is set up on V8 JavaScript engines and successful node Node js, and it, which has, is encapsulated as C+ The critical extension and support in+module and storehouse.
Hardware system structure
Fig. 5 A illustrate PCD 100 hardware structure, can be engineered one or more abilities to support PCD 100 sensation, Motor, connectivity, power and calculating demand.In certain embodiments, PCD 100 one or more hardware elements are that core is hard The specialization and adaptation of part, it is used in high-end flat board and other mobile devices.However, the thing of shape, motion and sensor It is unique for PCD 100 that reason, which is realized and set,.PCD 100 overall physical structure can also refer to 3 ring Zeta types herein. The physical arrangement of PCD 100 such type can provide PCD 100 clean and tidy, controllable and attractive position.Implementing In example, structure can be from can be by vivid animation teacher be using passing on the concern and the principle of emotion to obtain.PCD 100 physical arrangement can The border of mechanically and electrically structure is defined based on the scope of three ring volumes, motion and required sensor placement.
According to exemplary and non-limiting example, PCD 100 can be configured to including for three mobile axles, one or Multiple stereoscopic vision filming apparatus 504, microphone array 506, touch-sensing ability 508 and the display such as LCD display 510 Device.Emotional expression and indication sensor can be supported for three mobile axles and the ability of user is accompanied using natural way. Stereoscopic vision filming apparatus 504 can be configured to support 3D positions and usertracking, be captured for providing video input, filming apparatus And the like.Microphone array 506 can support Wave beam forming audio input to maximize ASR performances.Touch-sensing ability 508 Alternative interaction can be achieved to cause PCD 100 as friend, or is used as the form of user interface.LCD display 510 can support emotion Expression and multidate information are explicit.It may also include environment LED lamplight.
According to exemplary and non-limiting example, hardware structure 500 can be configured to including electrical architecture, and it can be based on next Self-embedding formula control and robot space COTS processors and with coming at the high-end applications in self-moving device peace plate space Device is managed to combine.Embedded controller is responsible for motion control and low-level sensors aggregation, and most of software stack is in application processor Upper operation.Electric plates in product are separated by the function of being designed for V1 and this can provide mode to match robot Physical arrangement simultaneously mitigate to spread into larger design update a plate design change the need for.In some embodiments In, electrical structure may include filming apparatus interface board, and it can the integrated two low resolution MIPI shooting dresses based on movement-industry Module is put, system standard under unified central planning that these camera modules can support hardware synchronization so that capture images can oppose in time.It is vertical Body filming apparatus is designed to use continuation mode streamcast video.In addition, filming apparatus interface board can support single RGB applications to shoot Device, for shooting high resolution picture and video conference video quality.RGB application filming apparatus is designed for specific photograph Piece shooting, video capture and Video Applications.
According to exemplary and non-limiting example, hardware structure may include microphone interface plate, and it can carry microphone Array 506, audio frequency process and codec support 514 and the digital stream of audio are sent to main application processor 516.Come self Codec 514 audio output can loudspeaker 518 be in body independent position isolate for sound when find.
According to exemplary and non-limiting example, hardware structure may include body control panel 520, and it can be in body Entreat integrated in part and motor control, rudimentary body sensing, power management and system wake-up feature be provided PCD 100. As example rather than as limitation, body control panel 520 can be set up around industrial standard Cortex-M4F microcontroller platforms.Separately Outside, framework 500 may include application processor plate, and it can provide core on-chip system(SoC)Processor and make robot system Remainder link together.In embodiment, system in module can be used in plate(SoM)To make the time of exploitation earlier prototype With least cost.In certain embodiments, application processor plate can in order to reduce cost and simplify production and including SoC processing Device.The key interface of application processor plate may include to be used to support MIPI filming apparatus, display, radio communication and high-performance sound The interface of frequency.
According to exemplary and non-limiting example, hardware structure 500 can be configured to including power management plate 522, and it can Solve PCD 100 power requirement.Power management plate 522 may include power governor, battery charger and battery.Power adjusting Device can be configured to regulation input power so that the electric power that the one or more elements or plate of hardware structure 500 can house regulation is supplied Should.In addition, battery charger can configure battery pair charging while enabling PCD 100 to operate for a long time.Implementing In example, PCD 100 can have cradle/pedestal/frame folder, and it will cause PCD 100 to exist comprising wall plug and blind-mating connector Internal cell should be able to be charged when being placed on pedestal.
Mechanical framework
According to exemplary and non-limiting example, PCD 100 various features are supplied to user in the form of single assembly. Fig. 6 A illustrate PCD 100 exemplary design, and it can be configured to the software and hardware framework including needs to use close friend side Formula provides various features to user.PCD 100 mechanical framework optimizes for quiet graceful and expressivity, is directed to simultaneously The cost-effective charging of material.By carefully select optimal element from many mature markets and make they together be in for In PCD 100 unique combinations, unique apparatus is produced.As shown in fig. 6, mechanical framework describes various plates(Such as Mike Aerofoil, mainboard, cell panel, body control panel, filming apparatus plate)The placement of example location in PCD 100.In addition, There is provided to provide in one or more holes in PCD 100 design allows air flow to provide cooling effect so as to appropriate.
According to various exemplary and non-limiting example described below, PCD utilizes many biographies with processor communication Sensor carrys out sensing data.As described below, these sensor operations into obtain feel input all modes, processor via A series of programmable algorithms work to perform task to sensation input.In these tasks are met, PCD 100 utilizes storage In local storage(It forms PCD 100 part)In data and access long-range storage(For example at server or In cloud)Data, such as via wired or wireless communication pattern.Equally, PCD 100 utilizes various output devices, for example, touch Screen, loudspeaker, tactile element and the like to user's output information simultaneously participate in social interaction.In addition, data, sensing is described in detail Device and on partner device communication pattern operation and interoperability non-limiting disclosure can issue U. S. application Found in 2009/0055019 A1, its content is incorporated herein by reference.
Embodiment described herein be presented such applicable feature of partner device and functional novel and unobvious Embodiment, especially come realize between PCD 100 and user social activity interact.As known to a person skilled in the art, understand Various forms of sensing datas and technology can be used for assessing and social clue of the detection from physical environment.Such technology bag Include but be not limited to voice and the vision-based detection of speech recognition, eyes mobile tracking, mankind's posture, position, motion and the like.To the greatest extent Pipe is with reference to such technology description, and the disclosure is widely drawn towards comprising being obtained by PCD 100, handle and output data is realized Any and all method of features described herein and embodiment.
According to exemplary and non-limiting example, PCD 100 can be expressed as virtual presence in pure physical environment, for example When in mobile computing device(As mobile phone, PDA, wrist-watch etc.)During upper execution, or it can be expressed as mixed mode physical/virtual machine Device people.In certain embodiments, the source information for driving mixed mode, physics or Virtual PC D can be obtained, as it is identical Embodiment is the same.For example, such as can via GUI interface input and be stored in the source information in database can drive mechanical PCD with And the animation part of display(It forms a Virtual PC D part).In certain embodiments, source information includes a variety of sources, its Including the output from AI systems, from the output sensed in real time;Source animation software model;Movable information model and the like. In certain embodiments, data can be from pure virtual image(In Yuan Chu)The single source of behavior promote and then can be with Physics and Virtualization Mode are exported to physics PCD.Using which, PCD embodiment can be from pure physics to entirely virtual to leading Cross over gamut with relating to the mixed mode of some in both.PCD 100 possesses and is expressed as core roles, and it can be stored In cloud and can allow users with that physical unit done remembered and adhered to so that virtual role can remember physics What device there occurs and it reacted, and vice versa.Physics and virtual instance can be managed via cloud, with Just another is transferred to from one at the appropriate time, with dual experience or the like.
As shown, PCD 100 includes the design of general three-stage, and it is not included by three of the separation of substantially annulus completely not Same body segment.By making each body segment be rotated around ring, such as via internal motor(It is not shown), PCD 100 be configured to change it Shape realize the variously-shaped factor and track user and other objects with sensor 102,104,106,108,112. In various embodiments, PCD 100 attribute can be configured either statically or dynamically, and it includes but is not limited to the shape of touch screen 102, rich in table Specific rhythm quality and tonequality, numeral when the body movement of existing power, specifically rich expressive sound and memory, speech connect The details of mouth, full PCD 100 " face ", spectrum LED illumination element and the like.
According to exemplary and non-limiting example, PCD 100, which can be configured to, uses multi-model user interface, many of which Input and output can be effective simultaneously.The concurrent interface of so type can provide the Consumer's Experience of robust.In some embodiments In, user interface input or output one or more of can according to cause PCD 100 with respect to suboptimum operate environment and by Involve.Various patterns are operated to can help to be automatically stopped simultaneously(fail-safe)Consumer's Experience and ensure with interacting for device There is no loss of communications.
According to exemplary and non-limiting example, PCD 100 can be configured to the one or more inputs of processing so as to PCD 100 user provides rich experience.PCD 100 can be configured to the voice of identification user.For example, PCD 100 " is called out from speech recognition Awake word " and/or other mechanism are contacted with to reduce " false positive ".In certain embodiments, PCD 100 can be configured in N x M Identification voice in the near field range of foot, wherein N and M can be determined by the sound quality and detection sensitivity of PCD voice. In other embodiments, PCD 100 can be configured in the long-range scope more than N feet(Cover 15 feet of rooms of at least 12 feet x Between size region)Interior identification voice.In certain embodiments, PCD 100 can be configured to sound of the identification in addition to conversational language Sound.PCD can use voice signature database, and it is configured with the sound that PCD can be recognized and worked to it.PCD can be via The content of the database is shared in the communication for being either directly or based upon cloud with other PCD devices.As example rather than as limitation, except spoken language Sound beyond word may include correspond to break glass, doorbell, ring, people's tumble, steam whistle, gunslinging, audible alert and The sound of analog.In addition, PCD 100, which can be configured by, requires that user's identification and PCD 100 existing grader are mismatched Sound source carry out " learning " new sound.Device be able to can respond to multiple language.In certain embodiments, PCD 100 It can be configured to and the user outside near field range is responded with wake-up word.User can be needed to enter the visual field of device.
In certain embodiments, PCD 100 can have touch sensitive regions on its surface, involve due to any reason The touch area can be used during phonetic entry.Using these touch inputs, PCD 100 can inquire Yes/No problem or on screen Show option and be contemplated that touch of the user on screen is used as the input from user.In certain embodiments, PCD 100 Vision and movement can be used to distinguish a user with other, especially when two or more users are in the visual field.In addition, PCD 100 can explain total bone posture and movement, and some common postures near field range.These postures are than dress Control is put more towards social activity interaction orientation.In certain embodiments, PCD 100 can be configured to including filming apparatus to shoot photograph Piece and film.In embodiment, when user is in the preset range of filming apparatus, filming apparatus can be configured to shooting photo and Film.In addition, PCD 100 can be configured to support video conference(pop-ins).In addition, PCD 100 can be configured to including when shooting Device eliminates the pattern of " blood-shot eye illness " when being in picture mode.
In certain embodiments, PCD 100 can be configured to determination whether it picked up, carried, declined and the like.Separately Outside, PCD 100, which can be configured to, realizes magnetometer.In certain embodiments, PCD 100 can determine that ambient lighting level.In addition, PCD Display and accent light intensity level can be adjusted to suitable rank by 100 based on ambient light levels.In certain embodiments, PCD 100 can have the ability for coming estimation unit position using GPS.PCD 100 can determine that the relative position of premises.At some In embodiment, PCD 100 can be configured to including one or more passive IR motion detection sensors(PIR)To help total or far field Motion detection.In certain embodiments, PCD 100 may include that at least one thermistor carrys out the environment temperature of indicative for environments.
According to exemplary and non-limiting example, PCD 100 can be configured to be said using intelligence, natural sounding to user " sound " English.The tone that PCD 100 can be configured to the described voice of change carrys out emulation animation device affective state(Work as PCD 100 sound sadness when sad, etc.)In certain embodiments, PCD 100 can be configured to including at least one loudspeaker, It can play voice, hi-fi music and audio.In embodiment, PCD 100 can have multiple loudspeakers(One is directed to language Sound, one be directed to music), and/or for special audible signal and the extra loudspeaker of alarm.The loudspeaker for being exclusively used in voice can Disposed towards user and speech frequency response is tuned.Tuning can be responded to full range by being exclusively used in the loudspeaker of music.PCD 100 It can be configured to RGB, full motion display.In certain embodiments, the live image of display is diametrically can quilt (Shelter)About at least 4-1/2 ' '.In certain embodiments, PCD 100 can have minimum 3 one-movement-freedom-degrees, so as to allow ring 360 degree of sensors in border are covered and like mankind's posture and mobile scope(Rich expressive position)Both.PCD 100 can It is configured to make physics animation synchronous with sound, voice, accent light and display figure.The synchronization can be close enough so as to for people It is seamless that class, which is perceived,.In certain embodiments, PCD 100 can have designated area, and it can be notified and social friendship for environment Mutually both use accent light.According to device form, accent light can help to PCD 100 filming apparatus be in photo or The main body in photo is illuminated during film acquisition mode.In certain embodiments, PCD 100 can have filming apparatus flash lamp, its By the main body in the automatic illuminating photo when filming apparatus is in photo capture pattern.Illuminated in addition, completing main body, this for Accent light can be more preferable.In addition, PCD 100 can have the mould that " blood-shot eye illness " is eliminated when filming apparatus is in photo capture pattern Formula.
According to exemplary and non-limiting example, PCD 100 is recognizable and tracks user.In embodiment, PCD 100 can be noted that when people enters near field range.For example, near field range can be 10 feet.In another embodiment, PCD 100 can be noted that when people enters far-field range.For example, far-field range can be 10 feet.In some embodiments In, PCD 100 can be with video(Face recognition), depth filming apparatus(Framework characteristic is matched)And sound(Speech ID)Combination know Not up to 5 different users.In embodiment, " learning " routine by PCD 100 using come learn PCD 100 be possible to identification User.In certain embodiments, PCD 100 can be near field range with video, depth filming apparatus and auditory scene analysis Combination positions and tracked user in complete 360 degree.In certain embodiments, PCD 100 can in 10 feet of far-field range User is positioned and tracked in complete 360 degree.In certain embodiments, when user is near field range, PCD 100 can be maintained not Interior view with user relative to the position of its own.In certain embodiments, when user is farther from PCD 100, PCD 100 Can decline feature level.In embodiment, PCD 100 fully functioning can be in PCD 100 near field range to user It can use.In certain embodiments, PCD 100 can be configured to tracking user mood and response.In embodiment, PCD 100 can lead to Cross video analysis, skeleton tracking, phonetic-rhythm, user's vocabulary and spoken query(That is, device asks that " how do you do" and explain sound Should).
According to exemplary and non-limiting example, PCD 100 can be programmed with the social code of the mankind with by affective content It is mixed into its animation.Especially, the perception affective content expressed based on user, sequencing intelligently should be applied to PCD 100 to adopt The affective content of output is suitably adjusted with entirely autonomous mode.PCD 100 can be programmed to attempt to pass through voice, light, movement Improve the sensing mood of user with the combination of audio.In addition, PCD social activity codes can provide the energy that harmonious relation is set up with user Power, i.e. mirror image behavior, imitation head pose etc..
According to exemplary and non-limiting example, PCD 100 can be programmed to use society based on the user preference learned Handing-over is in harmony method and delivers the pre- internet content for answering formula to customize to user(It includes sports news and game, weather forecast, news Editing, the information on current event, etc.)And/or the preference of exploitation its own is used to share the information and data, as widening The mode of the potential interest of user.
PCD devices are programmed with Suitable content type and it is communicated to what each individual consumer that it is recognized was used The ability of both modes.
PCD devices be programmed with based on user preference and user to the reaction of the internet content of delivery and processing custom with The ability of content customization/delivery to individual consumer is improved and optimized to time.
PCD can be programmed to participate in the social dialogue with user to confirm that the information delivered is understood by user.
According to exemplary and non-limiting example, PCD 100 can be configured to management and the activity of monitoring user.At some In embodiment, communicator 122 can be created when user asks and stored if the trip gone shopping before them together with service The doing work of user, groceries or other lists can be conveyed to.In certain embodiments, in request, PCD 100 will can be arranged Table pushes user to(Via service), be used as text(SMS)Message is pushed to mobile phone, or movement or web applications by us. In some embodiments, user can be via the voice on PCD 100 or such using being made by servicing via mobile or web Request.PCD 100 can carry out managing listings with user mutual(That is, purchased/completion/item no longer needed is removed, is gone through based on user History provides suggestion to additional lists, etc.).PCD 100 can be inferred by being heard in being talked in environment and understanding key phrase Need to increase list(That is, device hears " our coffee has been used up " and asks user whether they want to add to debris row The coffee of table).
According to exemplary and non-limiting example, PCD 100 can be configured to provides carrying for user's generation in orthochronous Wake up or message.PCD 100 can be used for being provided for the condition that prompting is delivered in orthochronous.In embodiment, for prompting Condition may include real-time conditions, such as " you will see me for the first time tomorrow morning " or " next time my daughter herein " or even " afternoon next Tuesday, you saw me for the first time " and the like.Once meeting condition setting, PCD 100 can be contacted with user(Come from " checking " and body language/expression are perceived)And delivered and reminded using suitable voice and character.In certain embodiments, PCD 100 can analyze the mood content of offer and use the information to influence animation/light/delivery of the prompting.At other In embodiment, PCD 100 can be in PCD 100 by asking user whether they perform prompting action and the use that followed up after delivery prompting Family.
According to exemplary and non-limiting example, PCD 100 can monitor user's in the request that can be provided by user Lack.For example, user may be notified that PCD 100, when and why they move away from(For example, " I is taking a walk now ")With And the expected duration of activity causes PCD 100 to can ensure that user's frame in expectation/request is returned.If in addition, user Do not returned in regulation window, PCD 100 can notify the emergency contact due to the accident defined such as user.PCD 100 can notify emergency contact by text message and/or by Mobile solution.PCD 100 can be recognized when user returns In the presence of and follow-up activity(That is, how is the activity of asking, or the other problemses related to activity).The interaction of such type can be Realize that social activity is interacted between PCD 100 and user, and the information on user can also be collected for learning database.PCD 100 can show to check out/check-in time and current user state to such family/friend, as by user in order to which the purpose is known It is other.This can be realized by Mobile solution.PCD 100 can have more abilities for going deep into movement monitoring/medelling/report.
According to exemplary and non-limiting example, PCD 100 can be configured by one or more data cube computations and company It is connected to external network.In certain embodiments, PCD 100 may have access to the high bandwidth wireless data cube computation of robust, such as WiFi numbers According to connection.In embodiment, PCD 100 can be in two frequency bands of 2.4GHZ and 5GHZ with two stream MIMO configuration realizations of 2x2 802.1n WiFi specifications.In certain embodiments, PCD 100 may be connected to other blue-tooth devices(Medical sensor, audio are raised Sound device etc.).In embodiment, PCD 100 can be configured to customization its UUID with including multi-modal user data and and other BLE enables the devices of PCD 100 and shares multi-modal user data.In certain embodiments, PCD 100, which can have, arrives 3G/4G/LTE Or the connectivity of other cellular networks.
According to exemplary and non-limiting example, numerous devices of PCD 100 can be used from group networking technology using netted Network configuration to configure, to allow, share and communication is without the service based on cloud by immediate data.Alternatively, be in multiple PCD Between 100 share data can be uploaded and be stored in database/data center based on cloud, wherein it can be processed and Prepare to travel to numerous devices of PCD 100.Data, services based on cloud can with mesh network set combine with The devices of numerous PCD 100 in a plurality of locations not only provided part and central data store, shared but also offer distribution.
According to exemplary and non-limiting example, buddy application can be configured to be connected with PCD 100.In some embodiments In, but buddy application is on following platform available:IOS, Android and Web.Buddy application may include directly perceived and be easy to make User interface(UI), it can not need three interactions of superelevation to access feature or function.Buddy application can be provided user The access of PCD 100 virtual counterparts the user-accessible virtual counterparts to interact with true PCD 100.
In certain embodiments, user can access such as PCD 100 shopping list, active day by buddy application The information such as will.In addition, the longitudinal direction that the local User Activities of PCD 100 can be presented in buddy application to user is reported.In some implementations In example, buddy application can make user be connected to PCD 100 via video and audio.In addition, buddy application can asynchronously remind use The some conditions in family(For example, than late desired by check-in, there is big noise and local user without response etc. in local user).
In certain embodiments, it is allowed to management/deployment of a series of connectivity of devices or a series of control to devices Apply on web platforms but available.The UI of management application can enable keeper or buyer to hospital/tender, and it is needed Quickly to access detailed report, setting, deployment and/or tenability.In addition, group be able to can use management application come access across The information of PCD 100 management collection storage.Management application can the asynchronous prompting some conditions of keeper(For example, local user is than label Enter it is desired will evening, there is big noise and local user without response etc.).In addition, management application can be across its managing device Subset or all spread news and remind.
According to exemplary and non-limiting example, support console that PCD 100 monitored by personnel/support/can be allowed to examine Disconnected/one or more devices of deployment.Support console on web platforms but available.In embodiment, support that console can The List View of all deployment PCD devices is supported, it can be disposed setting, firmware and answered by unique sequence numbers, the owner, mechanism Recognized with version number or registration abnormity.In embodiment, console is supported to support interactive inquiry, wherein label includes sequence Row number, the owner, mechanism deployment setting, firmware and application version number or registration abnormity.In addition, supporting console to support device What is diagnosed calls and reports.
According to exemplary and non-limiting example, console is supported to help to dispose new firmware and software version(Push away Model).In addition, supporting console to help to dispose newer NLU, new opplication etc..Console is supported to support customer support feelings Scape, message are to the propagation of the subset or whole of deployment device to pass on the thing such as service downtime as planning.In some realities Apply in example, support console to need to support to access on multiple devices to measure, it includes(But it is exclusive to be following):With PCD 100 interaction the time spent in, across all application/services decompose time, for Voice & Video perception algorithm aggregation hit/ Miss measurement, operation of recording(For supporting data mining, etc.), recording exceptional, alert threshold(For example, supporting console should What abnormal level you is shouted in)And other.
According to exemplary and non-limiting example, PCD 100 may participate in teleconference.In certain embodiments, telecommunications Meeting can be via simpler UI(Utilize the body or touch screen 104 for touching PCD 100)Or activated via speech(For example may be used Initiated with many phrases, sound and the like)And start to perform.In one embodiment, it is necessary to there is no more than two PCD 100 touch to initiate teleconference.In certain embodiments, calling can also be sent out as the output of call scheduling/prompting feature Rise.Once being initiated, PCD 100 can play a part of phone, and it is received and exported using microphone 112 and loudspeaker 110 Voice data from user is connected using wifi simultaneously, the connection of bluetooth, phone or some of combine to influence telephone functionality.
Calling can be standard voice call or comprising video components.During such interaction, PCD 100 can be for meeting Play a part of photographer in the ends of PCD 100 of words.In certain embodiments, PCD 100 can be placed on desk or with enclosing Multiple users in room are occupied around PCD 1000(Such as family, its all can be during calling, mobile and activity)Other In the middle of social accumulation point.During calling, PCD 100 can point to filming apparatus 106 in desired location.In one embodiment, PCD 100 can make filming apparatus 106 remain pointing to talker/user using sound positioning and feature tracking.In other implementations In example, PCD 100 can be indicated by the people in room/user(For example, " PCD look at Ruby ").In other embodiments, it is remote People be able to can provide to track target via device, and PCD 100 autonomous will look at and track the target.In any scene In, if such as remote participant using smart phone, portable computer or can show other devices of video, shoot dress Put 106 receptions(It is used as input)It is presented to them.
Device can be it will be appreciated that and be responded with multiple language.It is defeated by receiving speech during such interaction Enter, input is converted via processor and export conversion output, PCD 100 can also be played for the people on the link other end and " translated The effect of member ".If there is available screen in the room with PCD 100(Such as TV, iPad and the like), PCD 100 can Send(For example via bluetooth or wifi)The audio for the participant to be shown on the TV screens and(If available)Video.Such as Fruit can use without other screens, and PCD 100 can relay the audio from remote participant, but long-distance video can be unavailable.At this In the example of sample, if the only relaying remote participants of PCD 100.In certain embodiments, PCD 100 user can be become it is lively and React, such as if remote participant suspends with persistently determining time quantum blink and overlook or if PCD 100 is sensed Remote participant very excitation time then does some dancings or " shake ".
In another embodiment, PCD 100 can be the incarnation of the people on the remote port of link.For example, in touch screen 104 The eyes of upper display or other areas become and render version(In addition to other embodiment, based on cartoon, image or video flowing).Can Storage is rendered and it may have access to for PCD 100.In other embodiments, PCD 100 can also be retrieved and associated with long-distance user And describe the data of long-distance user and imitate motion/non-karst areas clue of long-distance user to strengthen incarnation experience.
In certain embodiments, during calling, remotely-or locally participant can imply that the still image of participant, regard Frequency and audio clips and the filming apparatus views of PCD 100 or annotation(For example, " PCD remembers the numeral ")Storage.These marks Note item suitably by meta-tag and will be stored in PCD clouds.
According to other embodiment, PCD 100 can also aid in simulates remote interaction in request.For example, user can require The suggestion game of PCD 100, it will initiate to connect game mode(It is described more fully below)And it is recommended that game is until two ginsengs All agree to person.In another embodiment, user can also just talk about some things inquire PCD 100.As response, The common interest of the addressable alignment sessions participants of PCD 100 " informed PCD " databases, or talked about for enlightening them This event is that participant exploits PCD schedules(For example, " grandmother tells Ruby that a few days ago you and your friend enter the noon altogether Meal ").
Dispatch assistant
According to exemplary and non-limiting example, PCD 100 can be based on schedule availability, Special Day and/or in link The understanding that the other end is present(For example, " your mother is in now, and this day is her birthday, and you want to phone her ”)Carry out suggestion calling.The acceptable suggestion of user, PCD call applications are in PCD 100 and the PCD of remote participant in this case 100th, start between phone, intelligent apparatus or Skype accounts.User can also be by requiring that dispatch call connects later by PCD 100 It is recommended that, scheduling application in this case adds it to the schedule of user.
Call answering and message transmission
According to exemplary and non-limiting example, call answering and messaging functionality can be realized with PCD 100.This feature Other cellular connection devices will not called applied to the voice or video call and PCD 100 for being placed into PCD 100 Management service.With reference to Fig. 7, it is illustrated that there is exemplary and non-limiting example flow chart 700.As shown, in step 702 Place, when calling is placed into PCD 100, PCD 100 can declare caller to the people in room.If nobody in room, PCD 100 can check the schedule of user, and if it indicates that they stay out, at step 704, PCD 100 can be straight by calling Sending and receiving are sent to the Voice Mailbox associated with PCD 100.If conversely it indicates that they are in, PCD 100 will make at step 706 With compared with loud noise(Stroke, the tinkle of bells, cry)To obtain the attention of people in house.
Once PCD 100 obtains the attention of his user, at step 708, PCD 100 can declare caller and inquire Whether they, which want, is answered the call.At step 710, user can use simple touch interface or ideally with natural language interface make sound Should.If answer is affirmative, at step 712, PCD 100 is as described in synchronous message transmission feature multi-modal on demand Connect call like that.If answer is negative, at step 714, calling is sent to the Voice Mailboxs of PCD 100.
If caller is directed to Voice Mailbox, PCD 100 can greet them and require that they leave message.In some realities Apply in example, voice or voice/video(If caller is using kype or equivalent)Message can be recorded to be used for afterwards Date reset.
Once user return and PCD 100 detects them in a room again, PCD 100 can at step 716 incite somebody to action Message informing they(" you have message " is utilized in speech, or luminous cannon is utilized on non-karst areas(pompom), etc.)And Inquire whether they want to listen it.If it does, PCD 100 in TV/ flat boards/wait audio playback or can play audio/video Message, described above.
User can have to preserve the option of message later.He can tell PCD 100 again in special time or only " later " inquire, PCD 100 will again be inquired after predetermined time amount in this case.
If caller is that PCD 100 is unknown, PCD 100 can direct calls to Voice Mailbox and inform that user connects Receive from certain numeral unidentified calling, and if it is recorded if reset message.Then user may indicate that PCD 100 Effectively prevent the numeral advance from connection/Voice Mailbox.PCD 100 can also ask the user whether to wish synchronously or asynchronously Return to calling.If user receives, PCD 100 starts suitable messaging mode to complete user's request.In some realities Apply in example, PCD 100 can also provide callmanager functionality to the honeycomb of other in family or fixed telephone device.Again other In embodiment, PCD 100 can response calling and by session point out caller leave message, so as to play the part of the angle of personal assistant Color.
Story is connected to read
According to exemplary and non-limiting example, PCD 100 can read application so that remote participant energy comprising link story Enough " passing through " PCD 100 are to having PCD 100 local participant's reading stories in room.Reader can be by by Virtual PC D The simple of 110 guiding is interacted based on the interface of web or Android applications, by selecting story and reading its process. Reader can be read such as the word as the story prompted in Virtual PC D 110.In certain embodiments, the voice of reader will be by physics PCD 100 is reset to attentive listener, and wherein Predetermined filter allows reader to use and is difficult to put applied to the voice of reader Believe that noticeable mode " is dubbed " to role, there is no capability so to do also such even if him/her.Soundtrack and audio It can also insert in playback.The interface of reader can also show attentive listener " PCD's observes " video feed, and PCD 100 Its " photographer " ability can be used attentive listener is kept in video.
Physics PCD 100 can also be reacted when suitable with short animation to story, and PCD 100 eyes(Above Description)Different shape is become to support story element.The feature, which can be packaged in inside PCD call features, causes reader It can utilize on its session etc. to interrupt story with attentive listener.Stop reading stories using recognizable reader, and temporarily Stop feature, therefore reader and attentive listener can talk without being filtered.Alternatively, narrator can with pre-recorded story and It is dispatched to reset using story relay application described below later.
Hot line
According to exemplary and non-limiting example, user can be connect using PCD 100 via " PTT " or " intercom " type Mouth and " in network " member communication.This feature can be via the screen-icon in the single-tap on skin or PCD 100 or via simple Voice command " PCD 100, talk with mother " access.In certain embodiments, this feature is limited only to PCD to PCD times Words, and can be used only in the presence of two PCD 100 detect user on their end of link.
Story is relayed
With reference to Fig. 8, it is illustrated that there is exemplary and non-limiting example flow chart 800.As shown, at step 802, use Family/story narrator can record story to reset later for PCD 100 at any time.Event can be recorded using some modes Thing:
Pass through PCD 100:Their story is told PCD 100 by story narrator, and it, which records it, is used to reset.
Applied by the web interfaces of Virtual PC D 100 or Android:User is guided them by Virtual PC D 100 Story tell camera.They also have an opportunity to include more rich animation/audio/background sound in the story of these types It is happy.
Once story has been recorded, at step 804, scheduling preference that PCD 100 can be set according to narrator is replayed Story.Attentive listener hears the option of story by being given in scheduling time, and can receive, decline or reschedule story.
In embodiment, during story is described, PCD 100 can shoot attentive listener still photo with set rate.One Denier story is completed, and at step 806, PCD 100 can inquire attentive listener whether he/her wants message beaming back story narrator.Such as Fruit user receives, then at step 808, and PCD 100 can enter " asynchronous multi-modal message transmission " feature and compilation message simultaneously And send a message to the physics PCD 100 of narrator(If they have)Or via the web links of Virtual PC D 100.Incline Hearer can have an opportunity return message in comprising listen attentively to story himself/herself photo.
Photo/memory producer
According to exemplary and non-limiting example, PCD 100 can include photo/memory producer's feature, thus PCD 100 by The role of photographer is taken in event.There are two patterns to this:
PCD trap modes
In this mode, it is desirable to which the user in photo can stand together and say " PCD takes pictures to us ".PCD 100 is true Recognize and then make people/many individuals in camera image placed in the middle using speech clue(Using as " retrogressing ", " moving to left " etc. Clue).When they are positioned properly with respect to the pumping mechanism, PCD 100 tells them to remain stationary as, and then triggers smile using certain phrase (" smile ", etc.).PCD 100 can be used human facial expression recognition to inform that they are not laughing at and continuing to attempt to trigger smile.When All users in image are when laughing at, and auto-focusing and flash lamp can be used in PCD 100(If necessary)To shoot some photographs Piece.
Event photographer's pattern
In this mode, user may indicate that PCD 100 starts in special time(Or " present ", Schedule if desired), persistently make a reservation for Shoot the photo of event time quantum.PCD 100 run one's eyes around the room using the combination of sound position and face recognition and with Family defines the people that speed shot in room and hunts shadow photo.All photos produced can be locally stored in the memories of PCD 100.
Once producing photo, photo can have been uploaded to the clouds of PCD 100 and notify user by PCD 100.At this point, can be with Applied via PCD 100 or web interface accesses them, wherein Virtual PC D 100 bootable users, which pass through, deletes, editor, cuts out The process of photo such as cut.Then they will be emailed to user or post to facial book etc..In being somebody's turn to do for the application In " out-of-the-box " version, photo using being merely retained on the clouds of PCD 100 with permanently storing predetermined hold-time amount, wherein returning Shelves/meta-tag is used as " existing legacy for example described below(living legacy)" the supply of part Monthly Payment.
As described herein, PCD 100 strengthens interpersonal and social life so as to be operable to help.In one embodiment In, using or " app " can configure or install on PCD 100 and access and operate PCD 100 one or more interface units To realize doings.For example, PCD 100 may include factory install application, its operate into upon execution with user mutual with Set about shooting according to which PCD 100 and store one or more photos to receive one or more parameters.For example, user can be right PCD 100 is said " at least one photo that please shoot each independent individual in party ".As response, PCD 100 can make to come from It may have access to party guest's list of guest's list and set about shooting the photo of each guest.In one embodiment, PCD 100 Can remains stationary and individual pass through when inquired for their identity they, record instance and shoot individual photograph Piece.In another embodiment, PCD 100 can be interacted with guest and be required that they set PCD 100 before guest colony To shoot their photo.Within a period of time(The duration of such as party), PCD 100 meeting society according to user Friendship/activity(It includes compiling document for social event)The hope of aspect obtains one or more photos of party guest.
According to other one exemplary embodiments, PCD 100 can be read social clue and it reacted.For example, PCD 100 observable users, the demand for indicating more softly to speak to another person.As response, PCD 100 can reduce it and export speech Volume where communication.Similarly, PCD 100 can send the sound for indicating satisfaction when embracing or dabbing.In other implementations In example, PCD 100 can be sent or with exporting social clue otherwise.For example, PCD 100(Sense user appointment go it is late )Can in tense situation is seemed swing to accelerate the speed that user leaves.
Interactive schedule
According to exemplary and non-limiting example, PCD 100 may be configured with calendar system capture after work user and The business of family.PCD 100 can share schedule and make these days if other PCD 100 user provides license Those of journey table and other PCD 100 are integrated so that the whole big family with PCD 100 is possible to have in each family For everyone single unified schedule.
Item in the schedules of PCD 100 can use suitable information meta-tag, be initially about the name of kinsfolk, they To the how are you feeling today of appointment/event, date or date customizing messages(Holiday etc.)And the like.Enterable event type includes But be not limited to wakeup time, the meal time, appointment, remind, make a phone call, housework/yardwork etc..Note not every event Carry out scheduling events based on all must necessarily be located special time-can be inputted by sensor, for example " remind my tomorrow morning you first It is secondary to see that I takes my umbrella ".
The input of item to PCD 100 schedule can be completed using many modes.One embodiment is applied using Android Or web interface, wherein Virtual PC D 100 guiding user pass through process.Emoticon or other interfaces are to can be used for accusing at this moment Tell how are you feeling today of the users of PCD 100 to application/event.The figure description of schedule can be similar to Outlook in the pattern, from And allow user to see event/appointments of other network memberses.The schedules of PCD 100 can also have with Outlook in this aspect The feature of the similar elimination conflicting appointments done.
In certain embodiments, user may also be able to by natural language interface to schedule add-ins(" PCD, I am all again Two at 1 point in afternoons about dentist, ahead of time half an hour remind me ", or " PCD, dinner tonight is afternoon 5:30”).After PCD 100 User's sensation can be inquired after(If not passed on by user)(For example, " your how are you feeling today to the appointment”), so as to allow Suitable emotion meta-tag.
If event is reminded and tied down, PCD 100 can transmit prompting using one in two modes.If set for it The user for putting prompting exists in PCD 100 environment, and he will transmit prompting in person, and be reminded with speech, animation, facial expression etc. To complete.The affective content of facial expression can be obtained from the meta-tag of event, for example, inputted by emotion or user's speech.He Behavior can also be obtained from known contexts(For example, he always feels sleepy when waking up or always starved at the dinner hour).It is logical Crossing context and being suitable for the expression of different event makes it keep non-duplicate and pleasant and refresh by regular author content.
If there is reminding targeted user not together with the physics of PCD 100, PCD 100 can call them. In such example, if they are non-response to this, PCD 100 can make their phone send short messages with reminding.
List manager
According to exemplary and non-limiting example, PCD 100 may be configured with list manager feature.According to this feature, PCD 100 can create to do work list or Shopping List when user asks, and it can send out short if the travel that they go to shopping Believe to user.Feature can be initiated via simple touch interface or desirably through natural language interface by user.User could dictate that The list type to be made(For example, " groceries ", " clothes ", " doing work " or certain types of shop or shop name).PCD 100 can inquire that it is what to be initially in list, and user can be responded via spoken word so that PCD 100 adds to list Plus thing.In any later time, user can require that PCD 100 adds other to list.
According to some embodiments, PCD 100 can parse daily session determine should be to list add-ins.For example, If someone says in room " we do not have milk ", PCD 100 can be added to grocery list automatically.
When user, which is bound for PCD 100, maintains the targeted shop of list, user can ask PCD 100 by appropriate columns Table is sent short messages to them so that when they do shopping in shop, the list can be used them.In addition, if user is away from PCD 100 but near shop, they can ask to carry out transmission list by Android or web applications.
When they return(That is, PCD 100 next time sees use after they ask list to be sent short messages to them Family), how PCD 100 can inquire trip/whether user find all in list, if "Yes", PCD 100 will remove list And wait to it and add other.If "No", what PCD 100 will inquire after and not be purchased, and remove other from list .
In the case of list to be done work, user can tell PCD 110 " I has done this ", and the Xiang Kecong The list of storage is removed.
User can also ask the list by others' PCD generations to be sent short messages to them(Pending suitable license).For example, such as PCD 100 is given old father and mother by fruit adult, and the adult can inquire PCD 100 by by the PCD 100 of their father and mother The Shopping List of generation is sent to them so that they for their father and mother can buy groceries, or they themselves when being done shopping for It can ensure that they inquire PCD before having any required instrument etc. with regard to " doing work " list of mother in visit 100。
Know PCD
According to exemplary and non-limiting example, PCD 100 may be configured with " informed " feature.According to this feature, PCD 100 can User is set to understand the latest development of user's news, weather, physical culture interested etc..This feature can use simple touch in request Interface or ideally natural language interface are accessed(For example, " PCD 100 tells me the baseball achievement of last night ").
User can have the ability in some set of time " information session " of one day.This can pass through web or Mobile solution Interface is completed.Using this feature, 100 schedulable of PCD some times of Cheng Yi days relay some information.For example, user can Their PCD 100 is programmed with the supply news after user is clear-headed.If user says "Yes", the referable users of PCD 100 The information of request in his/her " bulletin between morning ".This may include that some team score/news, weather, professional paper title are returned Turn round and look at etc..PCD 100 can be started with these general introduction and in any point, user must seek knowledge it is more on particular item, and And PCD 100 will read whole news item.
News item can " PCDization ".Specifically, PCD 100 can be provided PCD 100 in the news of reading and be commented on and react. Such reaction can be because AI is produced and is related in context.
Mood, activity, environmental monitor
According to exemplary and non-limiting example, PCD 100 can be configured with the form of the application for PCD using 100 Mood, activity and environmental monitor.The application can have been bought PCD 100 people's purchase, for example, old father and mother.In purchase When buying, web interface or Android are applied and be can be used for accessing monitoring setting and state.The bootable users of Virtual PC D 100 pass through this Process.Some examples for the thing that can be monitored include(1)The environment temperature in room/house where PCD 100,(2)It is living It is dynamic(People per hour/the time # of day walking, the hour # etc. that does not sense people),(3)People/many personal moods in room:Base In the feedback from sensor(Facial expression, laugh frequency, the frequency using some word/phrases, etc.), it is expressed as limited selection Concentrate one and(4)PCD 100 can by ask whether take medicine or clearly see medication come monitor to therapeutic regimen according to From property.
The state of the monitor that can be set can check via application or web interface, or more than Alert Level In the case of(For example, too cold in room, nobody walks in threshold amount of time), then text can be sent to by PCD 100 Monitor user.In addition, PCD 100 can independently remind user by monitoring some conditions that user is set via application or web interface It is satisfied, for example, is trembled if too cold and open heating.
Mood ring
According to exemplary and non-limiting example, PCD 100 may be configured with mood ring feature.The mood ring feature is available PCD 100 sensor serves as indicator and the even influencer of mood/affective state of user.This feature can maintain to use The real-time logs of the affective state at family.The indicator can be based on human facial expression recognition, body temperature, dynamic eye, activity level and type, voice The rhythm, keyword are used and the simple technique even as PCD 100 inquires their how are you feeling todays of user.PCD 100 will taste Try user authentication technology(For example inquire)To correct his explanation and the more preferable emotion model of user made with the time.This may be used also Involve from the clouds of PCD 100 and learn data(Attested sensing data<->Affective state mapping from other users)'s " mass-rent(crowd sourcing)”.With reference to Fig. 9, it is illustrated that there is exemplary and non-limiting example flow chart 900.In step At 902, PCD 100 explains user's body/face/voice details to determine his affective state.PCD 100 can be accurate with the time User's body/face/voice details is really explained to determine affective state.
Once PCD 100 determines the affective state of user, this is returned other by him at step 904.This can be using perhaps Multimode is completed.For with positioning(For example, in hospital environment)Tender, PCD 100 can use illumination/face figure The combination of shape/posture indicates the mood of the people belonging to him so that tender can see at a glance the people that looks after it is sad/happy/ It is angry to wait and correspondingly intervene(Or intervene).
For not with the tender of positioning(For example, looking after the old father and mother still lived by oneself), PCD 100 can pass through Which data is presented from it and for which period in terms of customizable movement/web applications the affective state is provided.
Once setting up the understanding of user mood, at step 906, PCD 100 is attempted and is carried out the change of the mood. This can independently occur, and wherein PCD 100 attempts to make by telling a story/laughing at, take pity on, playing the process of game, emotion mirror image etc. Into the positive change of user feeling state.Alternatively, tender is reminding main users to be in Negative Affect state by PCD 100 When can indicate that PCD 100 is said/trial/and do some things by the Negative Affect mitigated in the particular context that they may know that Feelings.
Night-light
According to exemplary and non-limiting example, PCD 100 may be configured with night-light feature.According to this feature, if user exists Midnight wakes up, and PCD 100 may act as animation night-light.If meeting correct condition(For example, the time is at midnight, ambient light is very low, There is static and quiet or sleep noise for a long time, and then have suddenly mobile or talk), PCD 100 can be gently Ground wake up, make using the color for making us feeling at ease cannon luminous and may inquire after user whether OK.In certain embodiments, PCD 100 can enlighten the activity or application that can make us feeling at ease and help to make user fall asleep again.
CanThe randomized act of love
According to exemplary and non-limiting example, PCD 100 may be configured with lovely random behavior feature.According to this feature, PCD 100 is operable to say thing in the various times in a whole day using being designed as this pleasant or thought-provoking mode Feelings/ask questions.In one embodiment, the feature does not involve the natural language session with PCD 100 free form, and phase Instead, PCD 100 says the thing such as interesting, lovely, interesting as the material of thinking/session.
In certain embodiments, PCD 100 may have access to quotation, phrase, the joke of the internal or external positioning of PCD 100 etc. Database, it is created, maintains and refreshed every now and then.Data may be from such as weather, physical culture, news, RSS feeds, come from Other PCD 100 mass-rent and user profiles.By being marked and metatag and the individual users of PCD 100 is inclined to these bits Like compared with process, the fact that suitable or quotation can be transmitted to each individual PCD 100.
When PCD 100 determines to deliver lovely randomized act, PCD 100 may be connected to cloud, give ID and please Ask the position from data warehouse.Described above, server will make true and user preference, date/date/time, user Weather of residential area etc. matches to determine to consign to the optimum bit of the user.
In certain embodiments, this feature can play the form for taking simple problem, and wherein problem is specific to be enough more to hold Easily carry out response identification, and to it is such the problem of response can be used for help to set up the profile of the user, so that it is guaranteed that Orthochronous consigns to his/her PCD 100 more fitting positions.In other embodiments, user can be exclusively by simple Touch interface asks lovely action by natural language interface.In certain embodiments, this feature can using " like/no Like " user feedback collection is to enable algorithm becoming more preferable in terms of position interested is provided to the specific user.
DJ PCD
According to exemplary and non-limiting example, PCD 100 may be configured with DJ features.According to this feature, PCD 100 is operable It is characterized into music, dancing and the suggestion from PCD 100.If this feature can be using dry model operation.Such mould Formula or function can pass through simple touch interface(No more than 2 beats of action from starting expectation)Or pass through in other embodiments Natural language interface is accessed and controlled.Music can be locally stored or be received from external source.
When PCD 100 plays song using this feature, beat-tracking can be used by PCD 100 so that song is along with dancing Animation, light/color performance, facial expression etc..PCD 100 song selection may depend on which pattern selected, for example:
Jukebox pattern
In this mode, PCD 100 can play particular songs, artist or the special edition of user's selection.
Mood box pattern
In this mode, user asks the song of a certain mood.Mood metatag can be used to select song for PCD 100.User The feedback on the song similar to Pandora can be given, so as to allow the adjustment of PCD 100 weighting for further choosing Select.
Ambient music pattern
Once user selects the pattern, PCD 100 uses the information from web(Time in date, certain day of one week, one day, Calendar event, weather of outside etc.)And the information from sensor 102,104,106,108,112(For example, people in room Quantity/activity level, noise level etc.)To select the song to be played and play the volume that they are used, so as in a room Background scope.User can have control volume or skip the ability of song.In addition, user be able to can ask at any time Particular songs, and without departing from ambient music pattern.The song of request is can play, and user selects(Such as volume changes)Can Used in future selection weighting.
PCD likes
Although user can directly access the pattern in certain embodiments(" you like what kind of music, PCD”), PCD 100 once in a while can also insert one or more selections in song stream, or attempt to broadcast when initiating jukebox or Mood box patterns Put selection(In ambient music pattern, PCD 100 can not done that).PCD the selection of music can be based on coming from for regularly updating PCD 100, the list of company, by author or by for example from other PCD mass-rent song select create.PCD 100 likes Particular songs-such as PCD 100, which can be pulled out, from the specific PCD 100 in user network can declare that " your daughter exists always now Ask this song bent ", and then play daughter like song.
Dancing PCD
According to exemplary and non-limiting example, after using any mode playback song, PCD 100 can inquire how it does (And glad or sad response can be suitably made according to the response of user), or give user dancing on user How well fraction.PCD 100 can also capture the photo of dancing user and supply to upload to them into the PCD letters of user Shelves, social media website send them by Email.Various functions sexual norm includes:
Replicate you
In this mode, PCD 100 select the song to be played, and any use sound position/face/skeleton tracking with Vis/RGBD filming apparatus obtains user in visual field.When user follows music to dance, PCD 100 can attempt to imitate the dance of user Step.At the end of song, how PCD 100 performs it, or give user on user's in user action is replicated if being inquired The fraction that beat is kept how well.PCD 100 can also capture the photo of dancing user and supply they uploading to use The PCD profiles at family, social media website send them to user by Email.
Replicate PCD
In this mode, PCD 100 dances and user attempts to imitate dancing.If user does badly, the playback of music is again It is secondary to be affected.In certain embodiments, independent screen is shown for both user and PCD 100 mankind dancers to be imitated. Both user and PCD 100 dance therewith and then PCD 100 grades to both its own and user.
Dance therewith
In this mode, user plays the music from radio, iPod, singing, humming etc., and PCD 100 is attempted therewith Dance, so as to inquire that it is made how well at the end.
Story performance/animation
According to exemplary and non-limiting example, PCD 100 may be configured with story performance/animation feature.According to this feature, PCD 100 is operable to allow user to buy for the drama with PCD 100 interactive performance.With reference to Figure 10, it is illustrated that have and show The flow chart 1000 of plasticity and non-limiting example.At step 1002, drama can be bought completely and store it in use In the PCD cloud profiles at family, or they can rent Netflix styles.
The purchase of drama/scene can occur for example, by Android applications or web interface, and wherein Virtual PC D 100 can User is guided to pass through purchase and installation process.In certain embodiments, at step 1004, user may be selected them and wish performance Drama/scene.The selection and the control of feature when using it can be via simple touch interfaces(PCD 100 eyes Or body)Or realized via natural language interface.Once user selects drama, user can be inquired in Pan1006Chu, PCD 100 Whether hope is rehearsed or performed.
Regardless of the pattern of selection, at step 1008, PCD 100 can be by inquiring their would-be shapes of user As if no start in drama.After the very first time, PCD 100 will confirm the selection if selection drama again, and And user can change at any time.
Rehearsal pattern
Once user enter rehearsal pattern, at step 1010, PCD 100 can supply to perform drama so as to make user be familiar with play It is acute.If they already are familiar with, user can skip this.The performance dramas of PCD 100 such as are do want to user, in step 1012 Place, PCD 100 can protrude the lines for user role when user is readed over.
After this is readed over, at step 1014, PCD 100 can start to teach user's lines.For each lines, PCD 100 Prompting and lines can be declared, and word then is shown on touch screen 104 when user recites lines.Voice can be used in PCD 100 Identification attempts holding until user correctly repeats lines to determine whether user is correct.PCD 100 then can to Family supply is pointed out and allows them to repeat lines, so as to again attempt to until user can suitably repeat lines towards prompting. Then PCD 100 can move on to next lines.
Once user learns all lines, at step 1016, PCD 100 will be readed over using all promptings, so that Check correct lines as response and point out user if necessary.
Notice that originally prompting can take the form of figure, wherein eyes become to enlighten the shape of lines.This can be First time in terms of prompting attempts, and if user still can not remember lines, then PCD 100 can develop into speech and carry Show.
Performance pattern
Once user remembers that they wish all lines of role, at step 1018, they can enter performance mould Formula.In this mode, PCD 100 will carry out comprehensive performance of drama, so as to suspend the lines and such as to allow user to say them Fruit user stammers or forgotten, points out.PCD 100 will use whole tone during the performance, even during user's delivery lines Effect, background music, animation and lighting effects.In certain embodiments, after theatrical performances, PCD 100 can produce the card of drama Logical/animated versions, wherein including the speech audio of user during their lines and making itself and their institute's roles It is lip-sync(If possible).The cartoon is storable on PCD clouds, posted to social media website or by electronics postal Part, which issues user, to be used to sharing/being formed memory.In certain embodiments, PCD 100, which may be additionally configured to performance, has multiple participate in The drama of person, each plays the part of themselves role, and participant can be long-range(For example, in the other end of talk streams On).
Dancing PCD- shares
According to exemplary and non-limiting example, PCD 100 can be configured to using the extra of above-described dancing PCD applications Feature.In some embodiments of this feature, user can create customization dancing for PCD 100.This is come by mobile or web applications Create, so as to allow user to select song and select dance step to put together with music performance for PCD 100.User PCD 100 can be also allowed to select dance step so that collaboratively creating dancing with PCD 100.In certain embodiments, light effect can be increased Really/audio(" get off for example, PCD is said!”)And it is synchronous with dancing.In other embodiments, the dancings of PCD 100 can be transmitted Other PCD 100, show friend that Virtual PC D 100 played the part of, online preserve etc..User can also deduce by other PCD Other dancings of PCD 100 that 100 users are created.
The content that famous person produces
According to exemplary and non-limiting example, the content that this feature allows user to produce famous person is downloaded or is streamed to them PCD 100.Content is selected by web interface or Android applications, and wherein the bootable users of Virtual PC D 100 pass through interior Hold purchasing process.Content can be it is following in any one:
Advance voice
This may include to film《Mysterious science movie theatre 3000》The director of interior joke/performer's comment.All the elements can be inserted into film. Audio frequency watermark can be used for the media sync of the delivery and viewing for the PCD 100 for making content.
Real-time streams are broadcast
In this mode, PCD 100 can stream broadcast the content produced in real time in center by famous person/reviewer.Content creator Can also have real-time " manipulation " PCD 100 to supplement spoken word language to realize animation/light/color effects.In such example In, because content creator will watch event with user concurrent in theory and make comments in real time.This may include that politics is commented Comment is provided presidential speech, election coverage etc. by family, or the sportsman that user likes provides comment to sport event.
According to exemplary and non-limiting example, lasting partner device(PCD)100 are suitable to continue or almost constantly deposit It is in people or many personal environment.In one embodiment, people is that the various parameters data for recognizing people are obtained or right by PCD PCD can use the example of targeted people.As described more fully below, in addition to the ID of people, PCD 100, which can be recognized further, to be expert at For(Scheduling, routine, custom etc.), preference, attitude, target, the pattern on task dispatching.
Identification parameter data may be used in such as speech recognition, face recognition and the like, be passed using above-described Sensor 102,104,106,108, one or more of 112 recognizes the presence of people.Supplemental characteristic can be locally stored(For example exist In PCD 100 memory)Or it is remotely stored in the wired or wireless communications therewith of PCD 100(For example via bluetooth, wifi and class Like thing)Server on.Such supplemental characteristic can be manually entered in PCD 100 or server or can be by PCD 100 with the time Obtain or the part as initialization procedure,
For example, when the PCD 100 of other no initializtion is brought into user environment, the executable initialization code of user, thus Voice that PCD 100 is operated/interacted using it to obtain user, facial characteristics or the like(True letter related to other Breath)Example.In the home centers embodiment fully described below, if user forms user(It includes big family)Society Network is handed over then to may be present multiple.The data are storable in PCD 100 and equally can passed on by PCD 100 and deposited for outside Storage, such as at server.Other identification user data can be manually entered(Such as user name, user's birthday, eyes of user face Color, user's hair color, user's weight and the like), such as via the graphic user interface, speech interfaces or formation of server PCD 100 part.Once by the part input of supplemental characteristic or with being obtained otherwise by PCD 100, PCD 100 can Operate into and obtain other specification data in addition.For example, being initialized(It includes providing sample voice signature, for example, pass through Pre-determined text is recited to PCD 100)When, PCD 100 can autonomous operation be into the user for recognizing speech and obtains face recognition institute The facial characteristics needed.Maintain the lasting presence in user environment due to PCD 100, PCD 100 can be with time operation into obtaining Take the various parameters data at family.
In certain embodiments, during initializing, PCD 100 operates into the phase for obtaining the people on the ID beyond them Close information.As noted above, PCD 100 is operable to obtain background information, demographic information, hobby, contact details(Electricity Sub- mail, mobile phone etc.), interest, preference, individual character and the like.In such example, PCD 100 is operable to obtain for example The information based on text/GUI/ phonetic entries during " getting to know " interaction.In addition, PCD 100 is also operable to obtain contact Information and home centers(For example, old father and mother, children etc.)Personalizing parameters information, it can share between PCD 100 And directly input in PCD 100.In the various embodiments being described more fully below, PCD 100 operate into promotion family with The connection of big family.As described further below, daily information(It includes but is not limited to individual scheduling, event, mood and class Like thing)How PCD 100 can be interacted, recommended to user, Supply Operations, information provision provide important context.
According to exemplary and non-limiting example, context, the longitudinal data of the acquisitions of PCD 100 promote adaptive system System, it into adjusts its function with feature configuration increasingly for interest, preference and the use case of user.If for example, PCD 100 learns that user likes music, and it can download " music attribute " with it can be found that musical taste, broadcasting should from cloud automatically Class music and make informed music and recommend.
So, PCD 100 learns the life of user.PCD 100 can sense user in real world and it can be with Assemble data from other devices, technology, system, personal computing device, the ecological of personal electronic device for being connected to PCD 100. PCD 100 is collected from the longitudinal data to be learned people and enables it to learn user and make its own be configured to better adapt to With the activity pattern for matching the function that it can be provided.Importantly, PCD 100 learns your social activity/home mode, your life Hit(Your big family)Whom important people is, it learns and tracks your emotion/mood, and it learns important behavior pattern (When doing some things in your region), it learns your preference, hobby etc., it learn you wonder what, what please you, Deng.
As described more fully below, PCD 100 is configured to provide longitudinal data collection facility with user mutual, is used for Collect the data interacted of the user and PCD 100 on PCD 100.
According to exemplary and non-limiting example, PCD 100 is configured to obtain longitudinal data, and it includes holding with user Interaction long(Via involving vision, the sense of hearing and the interaction of touch sensor 102,104,106,108,112)One or more category Property.In each example, vision, the sense of hearing and tactile sensation can by PCD 100 from user perceive or with otherwise acquisition and User is conveyed to by PCD 100.For example, PCD 100 can include filming apparatus sensor 106, for obtaining vision letter from user Breath, it includes activity with user, the affective state data relevant with health.Equally, PCD 100 can include audio sensing Device 112, for obtaining audio-frequency information from user, its data for including obtaining from speech recognition, the data relevant with stress level with And the contextual information such as the identity for the entertainment medium that user is utilized.PCD 100 can further include touch sensor 102, for obtaining tactile data from user, it is included with being the touch of user or contact in the physical contacts of PCD 100(It is wrapped Include but be not limited to stroke and embrace PCD 100)Relevant data.In other embodiments, user it is also possible to use touch to browse PCD 100 touch screen interface.In other embodiments, the mobile phone that the position of PCD 100 or user for example can be carried via user To determine and provide location context-relevant information as input and provide service.
As noted, vision, the sense of hearing and tactile sensation can be conveyed to user by PCD 100.For example, audio output device can For exporting sound, alarm, music, phonetic order and the like and participating in and user conversation.Similarly, graphic elements can use In text and image are conveyed into user and conveying graph data is operated into, it includes the communication between PCD 100 and user An interactive part.It can be with use environment light and other clues(Its LED cannon).Haptic device 102 can be used for passing on The affective states of PCD 100 and various other data(It is included via for example vibrating)And interface/content of browsing apparatus.Device The different smells of situation, the mood of suitable user etc. can be sent.
Information can be assembled by being connected to PCD 100 different device.This can come from third party system(Medical treatment, family The data such as front yard safety), data of mobile device(Music playlist, photo, search history, schedule, contact list, video Lamp), desktop computer data(Inputted particularly by the doors of PCD 100).
, can be from various with the data and information that involve in the interacting of PCD 100 and user in addition to above-described sensor Data source obtains, is stored in them above and is output to them.In exemplary and non-limiting example, interaction data can be through By cloud data or other connection sexual norms(Bluetooth, etc.)And be stored on PCD 100 and passed between PCD 100 and user Send.In one embodiment, the access to the data of the cloud storage of user can be realized by PCD 100 to realize with PCD's 100 Interaction.For example, PCD 100 can search for internet, using application/service or access data from cloud-for example from cloud storage User scheduling and must be arrived at it triggering interaction use information.As an example, PCD 100 it may be noted that user is upper 9 points of noon, nearby there was breakfast appointment in dining room with friend.If PCD 100 notifies user to be stayed at home in first five minute of dating, PCD 100 can be by asking the user whether to get redy to start off via the speech of audio devices 110 with user mutual.In exemplary reality Apply in example, PCD 100 can be completed based on current GPS coordinate and the gps coordinate in dining room by independently carrying out running time calculating The heroic undertaking.Using which, PCD 100 can cause to the online or one or more algorithms of cloud data application of access to trigger Harmonious relation builds the action interacted between PCD 100 and user.People can be via social networks, real-time or asynchronous method (For example send text, set up real-time audio-visual connection, by other application/service(Facebook, twitter etc.)Connection And the like)To be communicated with PCD 100.Other examples include accessing amusement and the matchmaker of the user being stored in cloud by PCD 100 Body file, it includes but is not limited to itunes the and Netflix data that can be used for triggering interaction.
Using similar manner, according to other one exemplary embodiments, interaction data is storable near user environment or user In environment, such as on server or personal computer or mobile device, and it can be accessed by the user.PCD 100 can equally by Data storage is in cloud.In other embodiments, interaction data can be obtained via the sensor outside PCD 100.
According to exemplary and non-limiting example, activity log and device usage log can have been produced, for example, can have been stored On PCD 100, on server or in cloud, it, which can be used for promoting, interacts.Activity log can store user, PCD 100 or user The information record activity that both are participated in using interactive mode with PCD 100.For example, activity log is recordable to participate in chess game PCD 100 and user example.It can be stored with addition in such matching(From the enjoyment level of its deducibility user) The information of the affective state of period user.Using the data, such thing can be defined as under user's expectation how long by PCD 100 Chinese chess, from PCD 100 and user played chess over last time how long, user be desire to participate in the possibility of chess tournament And the like.Using similar manner, device usage log can be stored and maintained, its instruction user like when, how long and How to be interacted with PCD 100.As it will be apparent that both activity log and device usage log can be used for both increasing PCD The frequency interacted between 100 and user improves interactive quality again.
According to exemplary and non-limiting example, interaction data can be obtained via being manually entered.Such data can By user via input unit 102,104,106,108,112(It forms PCD 100 part)Directly input PCD 100 or Input computing device(Such as server, PDA, personal computer and the like)And transmit or with being communicated to PCD otherwise 100, such as via bluetooth or wifi/ clouds.In other embodiments, interaction data can be by PCD 100 via PCD 100 and user Between dialogue obtain.For example, PCD 100 may participate in the dialogue with user, it includes a series of problems, wherein user should Answer and be converted into text via the speech recognition software on PCD 100, on server or in cloud, as a result deposited as interaction data Storage.For GUI or the interaction based on touch is same.
According to exemplary and non-limiting example, interaction data can be via the sensor for being configured to recognize scent data 102nd, 104,106,108,112 produce.Equally, PCD 100, which can be configured to, sends smell.In still other embodiments, GPS Determining that equipment may be incorporated into PCD 100 with other positions strengthens interaction.For example, child user in family's self-driving travel or can spend a holiday In with his PCD 100.On the way, PCD 100 can determine that its geographical position, access internet to determine that nearby boundary mark is simultaneously And boundary mark participates in the dialogue with children through discussion, it is related to when and where.
In addition to discussion topic is understood fully using which, in certain embodiments, the result of such interaction can at that time or Long-range storage facility is sent in later time, is then so obtained from multiple users in the presence of the predefined security setting of basis Accumulation interaction data.Using which, the centralized data base of more desirable interactive mode can obtain number based on user property and PCD According to(For example, position)Statistical profile and develop.For example, in example before, its position can be defined as by PCD 100 On national square near Aero-Space museum and on National Museum of Natural History opposite.By accessing centralized data base simultaneously And age and the position of user are provided, it may be determined that other children matched with age of user profile tend to be interested in dinosaur.Cause This, PCD 100 begins participating in discussion dinosaur while directing the user to National Museum of Natural History.
According to exemplary and non-limiting example, PCD 100 can be at least partially based on the various physiology and physics of user Attribute and parameter modulate the aspect interacted with user.In certain embodiments, PCD 100 can be determined using eye tracking The direction of visual lines of user.Such information can be used for for example determining the interest of user or measure equivocal.Equally, use can be obtained The heart rate and respiratory rate at family.In still other embodiments, the colour of skin of user can be determined from vision sensor data and for doing The physics or affective state of clear user.Other behaviors category for the user that can be understood fully via sensor 102,104,106,108,112 Property include but is not limited to sound rule and select word.In other one exemplary embodiments, PCD 100 can understand fully and explain the physics of user Posture, for example, brandish or indicate, it can be subsequently used as the triggering of interaction.Equally, PCD 100 can assess and analyze the posture of user To determine whether user stands, it is hunchbacked, sway and the like.
According to various exemplary and non-limiting example, interacting between PCD 100 and user can be at least partially based on The user feeling or the state of mind or attribute of determination.For example, PCD 100 can determine that and record the speed of user's blink, user It is to understand fully whether user may be for example tight laughing at or bite his/her lip, user and send presence of laugh and the like , happiness, worry, happiness etc..Similarly, when user keeps relatively motionless and silence, PCD 100 in other quiet environment Whether the sight of observable user stares at the point in space and determines that user is in thinking or puzzled state.In other implementations again In example, user's posture can be explained in PCD 100(For example nod or shake the head), it is used as the consistent or inconsistent instruction of psychology.
According to exemplary and non-limiting example, user mutual via interface general property it is configurable and/or Coordinate into offer personification or based on inhuman PCD 100.In one embodiment, PCD 100 is configured to show that non-human is moved The characteristic of thing.By doing so it is possible, interacting between PCD 100 and user can be by imitating and/or amplifying user to particular animals Existing emotion hobby and strengthen.For example, PCD 100 by barking can imitate dog when operating into and passing on excitatory state.PCD 100 can be further equipped with the tail as adnexa, and it can be waved in response to user mutual.Equally, PCD 100 it is exportable with it is familiar The similar sound of cats " mew mew ".In addition to the real-time display at the interfaces of PCD 100, such interface property can anaplasia at any time Change to strengthen interaction by adjusting the aging course of user and the zoomorphisms of PCD 100.For example, the shapes of PCD 100 based on dog As the action of doggie can be imitated when obtaining first time and gradually maturation is provided on user in its behavior and interaction The relation of partial user and PCD images is developing this sensation.
As noted, in addition to based on animal or imaginary biological PCD characteristics, PCD 100 can be configured to offer and copy the mankind Anthropomorphic interface.Such mankind or " role " can be preconfigured, a certain combination of user's definable or both.This may include Play, wherein famous person, media personality or image can be presented in PCD 100(For example, Larry Bird, Jon Stewart, coming from Downton Abby image etc.)Habit and characteristic.Except residing on PCD 100, in addition to the outsides of PCD 100, PCD 100 angle Color or " digital soul " can be stored(For example in cloud), and can therefore be downloaded and on erecting device other PCD 100.This Other a little PCD can be figure(For example, its appearance is appeared in user's mobile device)Or enter another physics PCD 100(For example, new model).
PCD 100 role can also have synthesis or technological property.Therefore, PCD 100 plays the work for the technology of personalizing With, wherein it is seen that device PCD 100 has unique role of its own, without being an attempt to emulate the anything else existed, Such as people, animal, well-known image and the like.In certain embodiments, special role can be created for PCD 100, it can be with Change and change more preferably to be adapted to its user with the time.For example, because such technology sets up substantial connection and emotion, user PCD 100 the rhythm can with the time change with the rhythm of its user of the more closely mirror image rhythm of oneself.Except its user Buying makes PCD 100 personalized or customizes outside PCD 100 any cosmetic or virtual article, and PCD 100 can also change its figure Shape outward appearance is to be adapted to the hobby and preference of its user.
In an exemplary embodiment, PCD 100 digital soul defines PCD 100 characteristic and attribute and influence user Interactivity qualitative attribution between PCD 100.Although the digital soul divides from the interaction datas utilized of PCD 100 and information Pitch to participate in interacting with user, digital soul can change over time in the response with specific user is interacted.For example in order to point From user, each for example well-known historical personage can be based on using themselves PCD 100(Such as Albert Einstein)The digital soul of identical is installed.From that quarter is installed on two independent devices of PCD 100, each PCD 100 will Interact in different ways, this depends on being produced by PCD 100 and to the specific interactions of addressable user of PCD 100.Numeral Soul can never same physical form(Such as machine person form)Or digital form(For example, avatar)Ground uses many forms Embody.
According to exemplary and non-limiting example, PCD 100 provides machine learning facility, for the data based on collection To improve interactive quality.Algorithm for carrying out machine learning can be put down in PCD 100, in the calculating communicated with PCD 100 Occur on platform.In an exemplary embodiment, PCD 100 can using Correlation Criteria reflection be provided in order to user mutual training and Training.Association or " operated " conditioned reflex are focused on and come using fortifier in increase behavior.By the process, in behavior and right Association is formed between the consequence of the behavior.For example, PCD 100 quickly can wake up and jump out of bed in user(Such as with keeping quiet It is only relative)When send happiness sound.When user makes the obvious happy states of PCD 100 be associated with such action, PCD This between 100 and user is interacted faster gets up with time operation into excitation user.In another example, user is being observed When taking exercise, PCD 100 can send the sound or voice of encouragement.In such example, PCD 100 plays desired to user Action provides the effect of lasting positive reinforcement.
According to various one exemplary embodiments, PCD 100 can be when carrying out machine learning using known in the art a variety of One in analysis type, its include but is not limited to temporal mode modeling and identification, user preference modeling, tagsort, task/ Strategy modeling and intensified learning.
According to exemplary and non-limiting example, PCD 100 can use vision, audio, kinaesthesia or " VAK " model, use The interactive mode with user mutual is best suited in identification.PCD 100 is operable to determine the main learning style of user.Example Such as, if PCD 100 determine user use based on vision in the way of processing information, PCD 100 can be using chart or diagram, example Such as in graphic alphanumeric display 104(It forms PCD 100 part)On, by information conveyance to user.Equally, PCD 100 can The problem of issuing the user with and other promptings are operated into help them to maintain vigilance in acoustic environments.
Equally, if PCD 100 determine user use based on the sense of hearing in the way of processing information, PCD 100 can be brief Illustrate to start new interaction in the case of what is coming and what summary can be there occurs and terminated.Finally, if Processing information in the way of the determination users of PCD 100 are used based on kinaesthesia, PCD 100 is operable to via kinaesthesia and haptic interaction (It involves mobile and touched)Come and user mutual.For example, in order to make user get up and active in the morning, PCD 100 can join With activity, wherein PCD 100 asks embracing from user.In other embodiments, in order to prominent and strengthen social interaction Element, PCD 100 can send the smell relevant with interaction.
The ability for making PCD 100 be gone about in house is the importance as PCD 100.In operation, PCD 100 operate into give remote personnel be used for communicate physics embody and the social expression way of physics, its permission people " stays in them The stream of life in " rather than must stop and roll up face before screen(Modern video meeting).Therefore, 100 couples of PCD is random Interaction provides support, as user visits someone in their house.User can carry out other activities(Such as dishwashing Son etc.), and because PCD 100 can track user in room surrounding and still carry out dialogue in any case.Described above One exemplary embodiment in, PCD 100 be designed to across room carry its sensor and output etc..Include in terms of core technology
User can control PCD 100 filming apparatus view, and it can also contribute to by tracking and inversely being transported Move and be maintained on destination object to make its filming apparatus be maintained at target and automate this.
PCD 100 can be using the important non-karst areas clue of preservation(As eye contact)This mode is by your expression(Video Stream, figure etc.)It is rendered into screen.
PCD 100 can monitor the head pose of remote personnel, body gesture so that personnel have rich expressive entity In the presence of.PCD 100 can also produce the rich expressive body movement of its own to adapt to situation, such as posture mirror image and synchronization To set up harmonious relation.
PCD 100 can further trigger interesting animation and sound.Therefore user can attempt exactly to pass on yourself For you or interesting image.This reads actually useful for connection story, and wherein grandmother remotely can read together with her grandson Story is read, while different images are presented in story ession for telecommunication.
Whom PCD 100 can be traced in speech so that its sight/your filming apparatus view is automatically transferred to talker(Come Reduction must control the cognitive load in PCD 100 manually).
PCD 100, which can have, to be slided from main interface so that long-distance user can be safeguarded to the more or less direct of PCD 100 Control, and it can use independence to supplement.
PCD 100 can provide extensive visual field to user(Tunnel vision than other device offer/presentations is much better, because It must be stayed in for you before it).
By doing all these things, and PCD 100 can be placed on to different places around the house, remote personnel sense Feel that they can not only communicate till now, and can be with activity.In order to during sleep share story, in game room simultaneously And played together with grandson generation, remotely participate in Thanksgiving Dinner, help your daughter to be sitting in work top when cooking family's recipe at you It is first-class.It is supported without hands-on, therefore you feel all right as you elsewhere also there is actual physical social activity to exist.
According to exemplary and non-limiting example, PCD 100 can be configured to or suitable for using stable or balance mode peace Put user live and operate where typical environment a variety of surfaces on or around.For example, PCD 100 generally flat surface Friction pad can be included by friction pad manufacture or at least partly, it, which is operated into, prevents PCD 100 from sliding on a smooth surface.At other In embodiment, PCD 100 can or telescopic adnexa detachable using part, it can be manually or automatically deployment with by PCD 100 It is placed on uneven surface.In other embodiments, device can have hardware components, its enable it to walk about in the environment or Manipulating objects.It can show equipped with laser designator or projecting apparatus on outer surface or object.In such reality In example, PCD 100 can be included friction pad on or near the tip of adnexa and be skidded with further reduce.In still other embodiments In, PCD 100 can include one or more suction cups on PCD 100 outer surface or multiple surfaces to be used to temporarily be attached to table Face.In still other embodiments, PCD 100 can make PCD 100 fixed and/or outstanding in place comprising hook, ring and the like Hang PCD 100.
In other one exemplary embodiments, PCD 100, which is suitable to hand, to be carried.Specifically, PCD 100 is configured to weight and is less than 10kg and occupy no more than 4000cm3Volume.In addition, PCD 100 may include attached or detachable belt or handle for Used in PCD 100 is carried.
According to exemplary and non-limiting example, PCD 100 is configured to persistently perceive or can determine society via calculating The presence or appearance of intersection rope and socially exist.In this way, PCD 100 is operable to avoid the full cut-off phase.In some realities Apply in example, PCD 100 can be periodically into low power state or " sleep state " to retain electric power.By sleep state during, PCD 100 is operable to handle reduction input set to remind the presence of the social clues of PCD 100, and such as people or user enter PCD 100 nearby, the sound of people's voice and the like.When PCD 100 detects people or user that PCD 100 can interact with Presence when, PCD 100 can be transitioned into highly vigilant of pattern, more wherein in the input of the sensors of PCD 100 or be completely used for Receive and processing context data.
The ability for persistently perceiving social clue is kept to reduce the demand that PCD 100 is powered off or is powered manually forever.Because closing The ability closed and opened is the attribute associated with installations, and PCD 100 avoids the ability in complete power-down mode from playing increasing Plus PCD 100 is this perception of life partner.In certain embodiments, PCD 100 can be by sending white noise or imitating snoring Sound increase in sleep state.In such example, when user meets PCD 100 by chance, the sensing users' of PCD 100 In the presence of and for example, by with the noise waken up of instruction(For example yawn)Greet user and setting about be transitioned into highly vigilant of or on Power mode.The queue interacted started between PCD 100 and user is served as in such action.
According to exemplary and non-limiting example, PCD 100 be suitable to monitoring, tracking and characterize speech from user and Non-karst areas signal and clue.The example of such clue includes but is not limited to posture, direction of visual lines, selects word, sound rule, body appearance Gesture, facial expression, affective state, touch and the like.All such clues can by PCD 100 via sensor device 102, 104th, 106,108,112 capture.PCD 100 can be further configured to change and adjust its behavior effectively to imitate or mirror image The clue of capture.By doing so it is possible, PCD 100 by seem reflect user characteristic and the state of mind come increase PCD 100 with Harmonious relation between user.Such mirror image, which may be incorporated into PCD 100 personage or digital soul, to be used to be opened up for a long time by PCD 100 The existing characteristic can be temporary transient and for example in a period of time(It includes specific social interaction)Interior extension.
For example, if PCD 100, which detects user, periodically uses particular phrase, the phrase can be added to friendship by PCD 100 The corpus of mutual data when future is with user mutual by PCD 100 persistently to be used.Similarly, PCD 100 can be in real time or near Transient state speech and non-karst areas posture are imitated in real time.If for example, PCD 100 detects the additional finger of frequency upgrading of user's voice Show excitement word speed increase, PCD 100 can with increased word speed the frequency higher than normal frequency in speech with Family is interacted.
According to exemplary and non-limiting example, PCD 100 can via the part for forming PCD 100 various physics Display(It includes but is not limited to the body shape factor, physics movement, figure and sound)Show completely different role or numeral Soul.In one embodiment, PCD 100 can use rich expressive machinery.For example, under PCD 100 can be comprising may move Bar adnexa, it can be activated when the output via audio signal is talked.Such adnexa can be awarded and be enough according to circumstances Imitate many frees degree smiled or frowned.Similarly, PCD 100 may be configured with one or more " eye " accessories, and it can Change vision depth of exposure.Therefore, PCD 100 " can be opened wide in response to frightened, surprised, interested and the like to show Eyes " are expressed one's feelings.
According to exemplary and non-limiting example, PCD 100 can detect it in the posture in space or position with for example Change between screen pattern and one-piece pattern.If for example, PCD 100 includes the screen 104, PCD for being used for showing graphical information Screen can be lifted to user's face and arrive certain position by 100 in user(User can watch display from the position)When interior from Whatsoever state residing for it is converted to the pattern to screen output signal.
According to another embodiment, the one or more pressure sensors for forming PCD 100 part can detect user When PCD 100 is touched using social mode.For example, PCD 100 can undergo what pressure was used from more than pressure sensor Pattern determines user by comforting, stroke or pat PCD 100.The social contact mode of different detections may act as PCD 100 Trigger, for showing the social interactive behavior interacted encouraged or suppressed with user.
According to exemplary and non-limiting example, PCD 100 may be provided with accessory, the outward appearance for strengthening PCD 100 And sensation.Such accessory includes but is not limited to skin, outwardly and inwardly clothes, light, mask and the like.
Described above, PCD 100 role or digital soul can be from PCD 100 physically displayed bifurcateds.Including PCD The attribute of 100 roles can be stored as numerical data, and one can be transmitted and be communicated to the digital data for example via bluetooth or wifi Or other multiple computing devices, it includes but is not limited to server and personal computing device.In such context, individual's meter It can perform a series of any device of programmable steps using processor and the memory of storage to calculate device.In some realities Apply in example, PCD 100 digital soul can be transferred to consumer's accessory, such as wrist-watch or mobile phone.In such example, PCD 100 role can effectively and occasional transmission is to another device.In certain embodiments, in transmission, PCD 100 biography Defeated example can continue to sense the environment of user, participate in social interaction and retrieve and export interaction data.Such interaction data PCD 100 can be transferred in later time or uploads onto the server to retrieve for PCD 100 later.
According to exemplary and non-limiting example, PCD 100 can put on display visual pattern, and it is adjusted in response to social clue It is whole.For example, display 104 can send feux rouges in excitation time and send blue light in calmness.Equally, display 104 can be shown Animation paper scrap drops celebrates joyously to pass on, such as when user successfully completes task.In certain embodiments, for display Texture and animation can be at user option or can be programmed directly into PCD 100 or to the server communicated with PCD 100 Or in external device (ED).In still other embodiments, PCD 100 can send a series of beeping sounds and boatswain's pipe to express simulation feelings Sense.In certain embodiments, beeping sound and boatswain's pipe can imitate the pattern obtained from the speech of user with other spoken voices. In some examples, beeping sound, boatswain's pipe and other sense of hearings output may act as signing to the unique sense of hearings of PCD 100.In some realities Apply in example, can be in multiple PCD 100(For example form one group of " correlation " PCD 100 of analog family)It is upper to be signed using the identical sense of hearing Version, to indicate degree of relevancy.
In certain embodiments, PCD 100 may participate in the deformation between expression pattern to pass on emotion.For example, PCD 100 operable displays 104 have the mouth smiled or frowned to be transformed into from randomly or pseudo-randomly pattern or other figures Display, is used as the method for showing human emotion.
In other one exemplary embodiments, PCD 100 can scent of or pheromones are come the state of showing emotion.
According to yet another one exemplary embodiment, it can use and background story is provided with to the addressable forms of PCD 100, its It can be the basis with user mutual.So specified number is according to including one or more stories with reference to past event(It is existing true , also have imaginary), its formed PCD 100 before history a part.For example, PCD 100 can be provided with story, it can Produced via the voice of the passing thing in the life for informing PCD 100 and be conveyed to user.Such story can be in request Exported, can be triggered by interaction data by user.For example, PCD 100 can distinguish that today is the birthday of user from user data.As Response, PCD 100 can be triggered with the story related with the PCD 100 passing birthday to users to share.Including background story Data can be centrally stored and independently download to PCD 100 in request by user or by PCD 100.
Background story can be produced and stored by PCD 100 manufacturer and user can use in storage.With reference to Figure 11, Diagram has exemplary and non-limiting example flow chart 1100.In this example, at step 1102, manufacturer can receive pair Copied in PCD 100 and to physical culture, particularly baseball and the event of this background of the dog of Boston Boston Red Sox user-association interested The request of thing is used as input.As response, at step 1104, manufacturer or third party's background story supplier can produce substantially Background story.In this example, story may include that increase has particularly story(It is related to the details relevant with Boston Red Sox and is added to Baseball)Relatively general dog story.
In certain embodiments, at step 1106, background story can with will allow PCD 100 further real-time customization Variable encode.For example, background story can be for example:“Me and my brothers and sisters <for i==l to max_siblings, insert sibling_name[i]>Encoded in the false codes such as were raised in ... ".Adopt With which, when being read by PCD 100, readable story is to include other PCD 100(It is configured as relevant with PCD 100) Name.
According to exemplary and non-limiting example, PCD 100 can be provided with executable module or program, for managing The same of PCD 100 forms feature, thus encourages user to look after partner device.For example, being operable to module is formed to user's Born impulsion be used with via involve the behavior of sound associate with baby, figure, temperature and the like by starting and Child is looked after in the interaction of user.Expression negative emotion is operated into same module of forming(It is for example sad, lonely and/or dejected)Together When from user collect mitigate negative emotion action when, the harmonious relation between PCD 100 and user can be encouraged further. So, user is encouraged to interact to make PCD 100 feel happy with PCD 100.
According to exemplary and non-limiting example, PCD 100 may include such module, and it, which is configured to access, indicates to use Family attribute, PCD 100 user and PCD 100 interact and PCD 100 user environment interaction data.With reference to figure 1200, it is illustrated that have exemplary and non-limiting example flow chart.At step 1202, interaction data is accessed.Once it is tested Rope and storage, at step 1206, interaction data can be used for the need state for expecting user.Once need state is recognized, in step At rapid 1208, it can be used for pre- solving user's request with answering formula and not against the scheduling acted for execution.In some implementations In example, the physical appearance of user, posture and the like can form the basis for recognizing need state.In some instances, need Asking the identification of state can be supplemented by dispatching data, such as part including interaction data.For example, scheduling may indicate that and miss Meet the time that user takes this demand of antibiotic.What PCD 100 can be obtained partly from face analysis and voice modulation analysis Data understand fully the need state of user.
According to exemplary and non-limiting example, PCD 100 can be used as courier, for message to be relayed to from a people Another.Message includes but is not limited to the audio recording of sender's voice, the character relay messages of PCD 100, disappeared for strengthening Dancing/animation/sound clip of breath and song.
Message can be produced using various ways.In one embodiment, PCD 100 is presented as the application on intelligent apparatus. Sender can open application, and select message and the sound, scheduling associated etc..PCD 100 virtual instance can make in User roams in the process.In another embodiment, by the direct interaction with PCD 100, sender/user can be via letter One-touch interface or natural language interface indicate that PCD 100 tells another person some things in a certain future time.For example, with Family can be said " PCD, this morning, when my wife comes into kitchen, plays her certain song and tells she me to like her ".Hair The person of sending can also allow his/her lower voice of the records of PCD 100 for use as the part of message.In other embodiments, instead of sender's PCD 100 delivers message, and message can be delivered by the different PCD 100 of another position.In still another embodiment of the invention, use Message for example can be pushed away text and arrive the specific labels of PCD 100 by family/sender, and PCD 100 will say that this disappears to user/recipient Breath.Emoticon also can be inserted into message, so as to point out the recording animation/Sounds script to be carried out by PCD 100.Some are exemplary Emoticon is:
PCD 100 is accorded with(ticon) The meaning PCD 100 is accorded with The meaning
’) Blink oO Laugh a great ho-ho!
o( It is sad oD Cachinnate!!!!!
o) It is happy oX Pay no attention to and do not say
oB Rabbit can eat you! or Broken teeth PCD 100
op Hiss! od It is fabulous!
oP Ridicule(Capital Raspberries!) o[ Bloodsucker/naughtiness
o/ Uh ... it is uncertain ... puzzled o{ Short-tempered/badtempered old man
o* Cheek is kissed o# It is secret.It should not say!My face is very tight.
os Disagreeable PCD 100 {o / curious
ol Phoney(Or it is unconcerned) }o It is angry
o+ It is uncomfortable/to have eaten bad/sour thing o> Someone says with me
Table 1:Emoticon is defined
In addition, message can be scheduled with later, in specific date and time and in certain group situation(For example, " you are in first time Tuesday See someone " or " when someone is waking up Wednesday, them are given by the message ")It is lower to send.
In other embodiments, PCD 100 can be used for producing message to the user without PCD.Such message can be used The form of web links is produced, and can be used to deliver message comprising Virtual PC D 100, if as receiver has physics PCD As 100 physics PCD 100 will be done.
As it is thus apparent that PCD 100 can be configured to from more personal(The family of such as friend and user)Reception disappears Breath, wherein the message trigger action relevant with emotion specified in message.For example, people, which can send short messages message, gives user(Generation The sender of table message wishes to be embedded within via the emoticon of the emotions conveyed of PCD 100 or social action)The PCD of association 100.For example, if sender sends display to PCD 100, " message for the OX " that extremely misses you, PCD 100 can receive message When via VODER export " inbound message from Robert is shown ' extremely missing you ' " be simultaneously emitted by kissing sound, Corrugated lip or similar movement are shown on display.So, sender of the message can annotate their message to utilize PCD 100 The expression mode that can be passed through with user mutual.
With reference to Figure 14, it is illustrated that the exemplary and non-limiting example of example, wherein PCD 100 can be in the friendships with user Circulation, lasting or semi-durable vision element, such as eyes are shown during mutually using user interface.For example, following institute Show, in order to show question mark, including indicate that the vision element 1400 of the iris in ocular surface or the brighter circle of reflection can be in eye Eyeball deforms or is otherwise smoothly converted to question mark vision element 1400''' via mesopic vision element 1400', 1400'' When its position is moved to the bottom of question mark.Vision element causes high readability such as the ability of the described deformation with shown in.
With reference to Figure 15, it is illustrated that the exemplary and non-limiting example of example, wherein vision element 1500 (wherein eyes purport Be deformed into for eyes vision too complicated shape) " can blink " as shown it is more complicated to be converted into Shape 1500'.For example, as indicated, the vision element " blink " of eyes 1500 is with related variable of displays temperature or other weather Shape 1500'.
With reference to Figure 16, it is illustrated that the exemplary and non-limiting example of example, wherein can be from the table of ocular vision element Face region forms or dug out a mouthful symbol.In various embodiments, thus it is possible to vary the color of vision element is to strengthen shown table Reach.
According to various exemplary and non-limiting example, with the mobile dress of routine in such as smart phone and tablet personal computer The application for putting operation is compared, and PCD 100 can have and show " technical ability ".Just as the movement in such as iOS and Android is put down The application program run on platform is the same, and PCD 100 can support to dispose the ability of various new technical ability.PCD technical ability can be with Including JavaScript bags, draw together with various JavaScript API resource and configuration file can be called, and to execution Hold up fed information.As a result, inside and outside developer can be supported to develop the new technical ability for PCD 100.
As general principle, any new social robot technical ability can completely with writing in JavaScript, JavaScript is related to one group of the core component including the SDK (SDK) for developing new technical ability JavaScript API.However, in order to promote exploitation, one group of instrument, such as representation aids external member and behavior editing machine can be permitted Perhaps developer, which creates, is fed to configuration file in runtime engine, promotes simpler and faster skill development and previously makes Develop ability.
With reference to Figure 17, it is illustrated that the exemplary and non-limiting implementation of the platform of technical ability during for enabling operation for PCD 100 Example.As indicated, receive it is various input 1700, include but is not limited to the image from three-dimensional RGB filming apparatus, microphone array and Touch sensitive sensor.Input 1700 can be carried out via touch-screen.Input 1700 can form sensing processing module 1702 input, wherein performing processing to extract information from input data and classify to input data.Input can come from The application of software application outside device or device, such as web, Mobile solution, Internet of Things (IoT) device, household automatic device, Warning system etc..The example for the processing form that can be used in processing module is felt includes but is not limited to automatic speech recognition (ASR), emotion detection, face recognition (ID), people or Object tracking, Wave beam forming and touch recognition.Feel that the result of processing can To be forwarded to runtime engine 1704 as input.Runtime engine 1704 can operate the technical ability with application definition, alternatively connect Receive the additional input 1706 for being such as, but not limited to following form:One or more grammers, behavior tree, JavaScript, animation and Voice/sound.Runtime engine 1704 can similarly be received from kinsfolk's model 1708 and inputted.
Runtime engine 1704 can export the logical definition of the data, wherein technical ability for the input to form expression module 1710 Aspect is mapped to PCD 100 Expression element, including but not limited to animation (for example, movement of PCD various parts), figure (such as shown on screen, can be touch-screen, or above-mentioned eyes movement), illumination and voice or other sound, can be with PCD of each reflection as described in other places in the disclosure programmed in expression module 1710 pattern, state, mood, role Deng.Express module 1710 can to PCD 100 the output data of various nextport hardware component NextPorts 1712 and instruction with expression skill, including but It is not limited to audio output, display, illumination component and motion and enables motor.Output can include the device to outside PCD 100 Or the control signal or data of application, such as IoT devices, web applications, Mobile solution.
With reference to Figure 18, it is illustrated that for realizing using the flow chart of the platform of SDK Develop abilities and showing for various infrastructure components Plasticity and non-limiting example.As shown, logic level 1800 can communicate with perceiving level 1802.Perceiving level 1802 can be via Visual performance module 1804 detects various events, such as visual performance event, via the expression detection animation event of engine 1806, and Voice recognition event is detected via speech recognition device 1806.Logic level 1800 and perceive level 1802 between communication can be used for by The event of perception is converted to the technical ability of expression.
In consideration of it, some abilities can be provided via one group of JavaScript API.First, JavaScript API may have feels input for various types of.It there may be the JavaScript API for various expression outputs.Also There may be the JavaScript API for runtime engine 1704, it can then call other existing JavaScript API.There may be the JavaScript API of the information for being stored in various models (such as kinsfolk's model 1708). Runtime engine 1704 is using any one in these API, such as by extracting information via them with runtime engine 1704 Use.In embodiment, kinsfolk's model 1708 can be directly accessed without using the developer of runtime engine.Wherein, PCD 100 It can learn, such as using machine learning, information, behavior pattern, preference, use-case pattern etc., such as to allow PCD 100 to make it Itself it is adapted to and personalized to one or more users, its environment and its use pattern.This kind of data and result of this study It can be embodied in PCD 100 kinsfolk's model 1708.
Feel that input API can include all kinds, including automatic speech recognition (ASR) API, phonetic entry API, be used for Handle the API of other sound (for example, for music recognition, detection specific sound pattern etc.), for handling ultrasonic wave or sonar API, the API for handling electromagnetic energy (visible ray, radio signal, microwave, X-ray, infrared signal etc.), for image The API of processing, the API for handling chemical signal (for example, smog, carbon monoxide, smell etc.) and many other.Feel input API can be used for the directly input of sensor of the processing from PCD 100 or handle by other sensation input source (such as sensors Network, sensor of IOT devices etc.) collect and transmission sensing data.
Feel input on various, timestamp can be provided to allow to merge a variety of sensation input types.For example, Timestamp can be provided with speech recognition device, to allow to merge the voice of identification with other sensation inputs.ASR can be used for registering Various loudspeakers.Generally, voice tool external member can be provided for PCD 100 speech interface.
Additionally provide various feature trackings and personnel tracking API, touch API, emotion recognition API, expression output API, shifting Dynamic API, screen and eyes figure API, illumination API (for example, for LED), sound and Text To Speech (TTS) API, and It is various other.Sound and TTS API can allow PCD 100 to play audio file, be spoken from a string of texts.This can be The content of constant or string variable, the silence of any amount or their any combination.For example, developer can specify that for example with Issue orders:Speak (" beep.wav ", NAME, ":SIL 3sec ", " please to meet you for I "), cause to send beeping sound, lead to Cross and represent to say specific name with actual name filling NAME variables, quiet period is three seconds, is then greeted.Text can use SSML (SSML) is expressed.Plain text can be said according to conventional punctuate rule.In embodiment, there may be and cover Cover or be inserted into expression wave filter or the sound effect in the speech said.
PCD SDK can include uploading content assets (such as audio file) and set audio output (such as volume) The method of property.Social robot is configurable to play a variety of forms, such as .wav .mp3.Assets can be deposited Storage is for example stored in cloud or local computing de in various storehouses.PCD SDK can allow PCD for example by searching for because of spy Net or one or more websites search for the assets of appropriate content (such as music, video, animation).
One group of kinsfolk and utility program API can be provided, its as the data remotely stored front end, such as in cloud In.These API can also include the utility program that developer may wish to use (such as log recording).
One group of runtime engine API can be provided to enable and the interface of runtime engine 1704.Runtime engine 1704 can include Optional JavaScript components, it can be operated to the configuration file using some different instrument creations, for example but not It is limited to, behavior editing machine and representation aids external member.Runtime engine can also be multiplexed the data stored from kinsfolk, make again Developer is obtained to be easier to write technical ability.In embodiment, kinsfolk shop can also include extension PCD 100 physical energy The hardware accessories of power, such as projecting apparatus, for PCD 100 mobile foundation, executor, loudspeaker, and allow user to determine The decoration element of PCD 100 processed outward appearance.
Workflow can be followed to create new PCD technical ability, technical ability writing, mould is carried out since asset creation and then Intend, test and certification (this certification is provided in embodiment as the host enterprise of the method and system described by management).
With reference to Figure 19, it is illustrated that the exemplary and non-limiting example of the user interface for creating assets can be provided. Asset creation may relate to create the assets of technical ability.It may be not necessarily the first step, but in the flow for creating technical ability often It is a lasting task, wherein assets are improved or extended with technical ability development in itself.The assets that can be created Type includes animation, for example, new body and eyes animation is easily created using the special tool(s) in representation aids external member. Developer can also redesign body and eyes animation in " developer " part in PCD technical ability shop.In embodiment, open Originator can share their assets with consumer or other developers, such as in the technical ability shop for PCD 100 or for example In other environment of the door of developer.Assets can also include sound so that developer can use the sound that they like Editing machine creates themselves sound, as long as resource is the appropriate format with the characteristic suitably defined.Assets can include Utilize the Text To Speech assets of parameter tts system so that developer can create Text To Speech example, and use can be modulated Each attribute (such as " happy ") of voice annotates these examples.
Assets can include light visualization, such as to control the LED (such as on trunk) on PCD 100, this In the case of, developer can use representation aids external member to carry out regulation control.Note, developer can also reuse LED light and move Draw, for example, be also from " developer " part in PCD technical ability shop.
Assets can include input syntax.For the input syntax that management skill is recognized, developer can use voice work The various grammers that tool external member recognizes to provide them to wish.
Once developer has skilled qualification in order, developer can write technical ability with usage behavior editing machine in itself. Behavior editing machine enables logic control and feels the processing of input and the control of expression output.Although the major part of the step can be with Completed using simple editing machine, but SDK can allow to add direct JavaScript code to enable developer It is probably unique thing enough to make for certain skills, for example, exchange data with one or more proprietary REST API, etc..
Once technical ability is write out by (partly), developer can use PCD simulators to carry out the various aspects of exercise Using Skill, PCD simulators can in real time or near real-time occur.Simulator can support the triggering that fundamental sensation is inputted, and may be used also To be operated to the sensation input file that the early developer's logging mode for first passing through PCD is created.Input to simulator can be with From being physically entered to PCD 100, one or more sensors outside PCD 100 directly from simulator, or come From such as external device (ED) of IoT devices or such as web applications or the application of Mobile solution.Simulator will be defeated by WebGL figures Go out to support the part of expression system, and text with represent TTS export.Exploitation and simulation cycles can be real-time or near real-times , use WYSIWYG methods so that the change of technical ability is immediately visible on simulator and is compiled in response to the dynamic in simulator Volume.
Finally, developer may need to test technical ability of the PCD 100 with, because may not be supported more in simulator Complicated behavior (for example notifying).In addition to special experiment in real time, developer can be by creating via PCD logging mode The sensation input file built drives experiment again.In embodiment, input can be from external source in real time or near real-time ground streaming Transmission.
In addition, if developer wishes to enable other people to use and buy new technical ability, developer can be to for example SDK main frame submits technical ability to be authenticated.Various certification guides can be created, such as to encourage the behavior between different technical ability Uniformity, it is ensured that security, it is ensured that reliability etc..Once be certified, the technical ability can be placed in PCD shops for Family, other developers etc. access.In embodiment, developer can also be in upslides such as PCD 100 shop, the doors of developer Put resource (for example, animation, technical ability, sound etc.).
Various instruments can be deployed in SDK or be connected with SDK and be used.These can include allowing developer to check, manage The local aware space (for example, for recognizing people, tracking people, mood detection etc.) of solution and/or the social robot of experiment it is local Aware space (LPS) visualization tool.Instrument can include related to the voice in the voice tool external member of utility program each Kind of instrument is to create new syntax, and annotation Text To Speech output.In embodiment, instrument can be used in utterance Using wave filter or other sound or audio frequency effect.Instrument can include behavior editing machine, with allow developer for example by Determine the behavior tree (such as " brain ") of technical ability to create behavior.
Representation aids external member can include being used for one group of utility program for social robot creation expression output, and it can be with Including the animation simulation device for the animation behavior for simulating PCD 100.This can include HTML with webkit and interpreter or JavaScript, Google for example from belowTMV8 JS InterpreterTM.Standard weblication can be used Code strengthens behavior and screen graph.
When dry run can be provided environment as the various aspects for tempering technical ability instrument.
With reference to Figure 20, it is illustrated that the exemplary and non-limiting screenshot capture of local aware space (LPS) visualization tool, its Developer can be allowed to see PCD 100 local aware space, for example, seen by PCD 100 camera.It can be used for People in identification and tracking PCD 100 view.In embodiment, this can increase complexity and can include having for example The three-dimensional world of the element for other vision elements that virtual image and PCD 100 can be interacted.
Voice tool external member can include the instrument relevant with speaking with hearing (for example, " ear " instrument).This can be wrapped Include the various abilities for importing phrase and various types of grammers (such as words recognition, statistics) from storehouse, such as Yes/No language Method, Serial No., natural number, control (continue, stop, suspending), date and time, non-phrase chunking grammer, variable (such as $ Name) etc..These can use ASR, speech-to-text ability etc., and can be based on cloud or be embedded in upper 100 of PCD Body.Suite of tools can be included in the basic verification and debugging of the grammer with application logic in above-mentioned simulator.Tool cover Part can include the instrument for being used to develop NLU (natural language understanding) pattern for PCD 100.Resource can be with grammer in use device Compilation tool is created.Resource can include being used to collect the instrument (for example, as mechanical Turk) of data and for training The Machine learning tools of new model:For example for phrase chunking, by the people of voice identification or other voices or voice recognition or Understandability.Grammer can be issued is presented the output label debugged with logic for GUI.PCD 100 sensor bank can be used Feel resource and experiment grammer recognition performance in creating.The ASR actually said can be used to test whole technical ability.Phrase Identification grammer can be created, tests and adjust.
In behavior editing machine, when calling identifier, what developer can change the parameter of identifier is restricted group (example Such as, time-out, refusal etc.) and/or call readjustment (for example performing text-processing) to recognition result.
With reference to Figure 21, the screenshot capture of behavior editing machine is provided according to exemplary and non-limiting example.PCD behaviors are compiled Collecting device 2100 can enable developer/designer quickly to create new technical ability on PCD 100.Defined in this section Output file drives runtime engine 1704.The more details on behavior editing machine 2100 are provided below.
In embodiment, behavior authoring tools can include being designed to be easy to use, clear and definite, expansible and substantially WYSIWYG behavior tree creator.Behavior can include document living in itself.Each behavior can have description and annotation symbol. Behavior can be defined without being implemented.This allows the behavior that designer's " filling " there is no.
PCD systems of behavior can be made up of in its core the simple behavior of very low level.These low level behaviors can be with It is combined to create the complex behavior of higher level.The behavior of higher level can be manual coding, or relatively low by other The behavior composition of rank.This level is substantially unlimited.Although there is the gradient of complexity, behavior arrangement can be big Cause is divided into three ranks:(1) atom behavior (has the minimum behavior group of behaviour tree, generally includes to not necessarily rely on The behavior of PCD 100 function);(2) behavior based on PCD 100 (crosses over the behavior of PCD 100 whole ability groups, for example It is embodied in the various JavaScript API associated with social robot), (it can be hand for (3) compound senior behavior Work coding, be made up of in itself the behavior arrangement parameterized) and (4) skeleton behavior (be not present, do not fully achieve or its realize It is the behavior of separation).For example using machine learning method, for example, it can strengthen from PCD 100 experience learning behavior arrangement Study etc..Each function call (being for example embodied in JavaScript API) in social robot API can be expressed as Significant behavior.Skeleton behavior is inserted into for document purpose in behavior tree, and in realization later and operationally Binding.This allows to need the designer for the behavior that there is no to insert this " bindtype ", it include the behavior description and can Can result (failure, successfully etc.), and there is engineer's code after execution.If bindtype is present during playing back, So the type is bound to execution;Otherwise, PCD 100 or simulation it may be said that binding behavior title and its return type, and Continue in tree.Instrument can also support to perceive the definition of level the perception to develop complexity to handle approach.These perceive tree Output may be coupled to behavior etc..In addition, development platform and SDK support a set of to be available for the high-order categorization of perception that developer uses The multi-mode storehouse of module (multi-modal input/output module can be reused).
In the case of most of atom behaviors, behavior tree can be made up of these basic acts:BaseBehavior leaves Node;BaseDecorator behavior decorators;Parallel composition node;Sequence (and sequence variants) composite node;The compound section of selection Point;With random (and random change) composite node.Atom behavior may be almost the original letter to PCD JavaScript API Number is called, but is packaged as the behavior with appropriate sequential.They cross over whole API, thereby increases and it is possible to unusual low level.Some show Example includes:LookAt;LoadCompileClip;And PlayCompiledClip.The editing of compiling may have embedded thing Part.Behavior or decorator can monitor the event of some type, and in the definite moment execution logic of the event.This allows expression Close synchronization between output and higher level decision-making.Atom behavior can also include:PlayMp3;Listen; ListenTouch;With Blink (for example with blinkSpeed, interruptPreviousBlink=(true | False) related parameter.
Compound/senior behavior can be the senior behavior for combining other senior and/or rudimentary behaviors.These behaviors can be joined Numberization.Example can include:BeAttentive;TakeRandomPictures;BeHappy;With StreamCameraToScreen.Behavior can be goal-oriented, and such as change acts to realize the expected result with the world Or state.For example, in the case of Object tracking, target can track object and hold it in the ken.More complicated shows The behavior that example is search to find specific people or change PCD 100, for example, make one to smile.In embodiment, PCD 100 feelings Thread or emotion or emotional state can change PCD 100 behavior or behavior style.This target that can influence PCD or attention Priority ranking.This can also influence PCD 100 that what rule of thumb learns and how to learn.
The readability of behavior tree is important, particularly when tree change is big.A simple case statement is taken, according to voice Branch tree.The formal way of statement case statement is to create Select behaviors, and the behavior has child node, and thus it is by " selection " One child node is performed.Each child node is decorated with FailOnCondition, and it includes the logic of " selection " behavior. In form, it makes it difficult to automatically see and why can each filled relative to another one key element of selection without checking The logic of frilled organ.However, description field can be provided more contents by manual editing, but in selection logical sum describing word Not necessarily there is formal relationship between section.With reference to Figure 22, it is illustrated that create according to the branch for illustrating plasticity and non-limiting example The formal way of logic.Notice the code of the first and second decorators 2200,2202.Figure 22 illustrates formal relationship.
In PCD 100, there is common branches pattern.Some of them include:Branch based on grammer;Point based on touch Branch;With the branch of view-based access control model.
For most common branch, behavior instrument GUI can be visualized with Predigesting tree and be provided " description " between logic Formal relationship.This can be by realizing to behavior tree editing machine addition " Info " row, and Info row are by introspection basis The derived description of logic is filled automatically.Gui tool is known that the special Select behaviors meaning of referred to as " GrammarSelect " In the AD HOC presentation with GUI.Basic tree construction can with it is identical in Figure 22, but it can be with more readable side Formula is presented.
With reference to Figure 23, it is illustrated that exemplary and non-limiting example, it is possible thereby to which selection logic is added to as parameter Behavior is in itself.In this case, the parameter of addition can correspond to the String field of the grammer label returned, and should The value of parameter can be automatically placed in " Info " field.Addition into GrammarSelect each child node behavior Parameter value can be used for generation filling basis SucceedElseFail decorators correct code.
" common mode " for Multimodal interaction is known, and is that the monotype that is used for used in the past is interacted (Voice)Common mode differentiation.This is only in " sequential multimodal formula "(For example, both of which)In be real.However, machine People's behavior and man-machine interaction(HMI)With slightly different example.Although first is easier by behavior tree representation, dialogue " nesting " structure of frame makes itself preferably to provide nested " situation " sentence, or even more generally provides to being related to and have The expression of the recurrence digraph of conditional arc.It therefore, it can matching two to strengthen GrammarSelect, to increase HMI streams It is readable, it is allowed to set up complicated interaction.
In fact, any man-machine interaction can occur by this way.First, machine is configured to export something(Usual picture Those of animation+audio+texture), then the mankind input something(Typically voice or touch)Or some other processes returns pair In the significant event of interaction, and sequence is with additional output and inputs iteration.
Therefore, case statement above(GrammarSelect)Complete event example is expanded to if will cover simultaneously And can have general HMI selections, wherein label can be specified(It corresponds to event)And tag types(Grammer, vision, touch Touch).So above-mentioned will be:
HMI_InputSelect:
AnyBehavior1 voices:RANDOMPICTURE, is touched:AREA1
AnyBehavior2 voices:PLAYMUSIC, is touched:AREA2
AnyBehavior3 visions:TRACKINGFACELOST
It is OR with the label of CSV.In this illustration, behavior will respond AnyBehavior1 to someone, say " shooting with Machine picture " OR is touchedTouchAREA1, Behavior2 is to someone for response, says " broadcasting music " or touchesTouchArea 2, or response Behavior3, if vision system returns to TRACKINGFACELOST.
Improve HMI stream readable another way be by introduce be referred to as the basic act of such as " Speak " come Clearly see the text of the prompting in behavior tree canonical view.So, with reference to above example, if someone says RANDOMPICTURE, is put into AnyBehavior1Sequence:AnyBehavior1.
PCD 100 is said:" well, and I is now to clap your photo.Prepare
User returns to "Yes", processing Behavior Speech:It is or touchesTouch:YESAREA.
Then, PCD 100 initiates sequence, such as TakePictureBehavior.
If PCD 100 detects "No", for example, hear NoBehavior:NO feels " Touch:NOAREA, then use Family performs GoHomeBehavior and starts speech act:RobotSpeak is " good.Return to main screen ".
In this case, PCD Speak are to make the basic act of multiple promptings and corresponding animation randomization(In embodiment In, if the behavior of double-click, it can be seen that point out and animation, and behavior edit box will be popped up).It is important to key in This behavior, because UI designs can the write-in prompting when developer designs and applied.It is then possible to all promptings of automatic mining Behavior tree simultaneously creates list table for voice personnel, automatic for prompting establishment file name etc.(This will individually save it is substantial amounts of design and The skill development time).
The mode of interbehavior is expressed in the above example, and developer, which can be understood quickly, will occur what, Therefore this will represent to design and realize simultaneously.
On representing interaction, it is necessary to be important to note that, if interaction is deep using retraction tree(For example with perhaps Multinest round), then rapidly depleting horizontal real estate.Therefore, designer may define subsequent conversion encapsulation elsewhere Behavior in.Another readable problem of influence is that the exit criteria in nested statement is unclear., can in digraph is represented To place arc at any desired any point, and it is fully readable.In nested procedure, can generate make program with And the condition that other callers are exited.
The main window of behavior editing machine can be expansible and folding tree construction.This represents the tree construction of behavior.It is right Each behavior in the view, in embodiment, can drag, terminate, deleting, replicating, shearing, pasting, exchanging another row For, add or remove one or more decorations, up or lower section addition the brother of node and add child node (and apply it is any on State the brother of node or child node).
This top level view should information enough so that author can be best understood by tree is try to that what does.This meaning Taste, which, can include behavior and decorator title per a line, represent the small icon of behavior type and the description field of user's filling.
Each behavior can be parameterized with zero or more parameter.For example, SimplePlayAnimation behaviors may Need a parameter:Animation title.More complicated behavior usually requires more parameters.
Complex act, which can be created, as sub-line in behavior instrument is., can arbitrarily parameter beggar in embodiment Tree parameter and the top that they are graphically pushed up to complex act.
Each parameter of behavior can have " type " associated there.The type of parameter can allow behavior authoring tools User is helped to input the virtual value of each independent variable to graphically as much as possible.The embodiment of structure is inherited the following is type, It has the description as described in how instrument will help user's filling appropriate value to graphically:(1)CompiledClip:Editor compiles Translating editing can make developer enter Animation Editors, and it can be the editing machine based on timeline;(2)Character string:There is text This frame;(3)File:There is file selector:(4)Animation file:There is a file selector window, list available dynamic Draw, potentially include the animation that the animation and PCD of user's generation are created.It can also be shown to the link of animation authoring tools, with Making of cartoon at the scene;(5)Audio files:File selector is likely to occur, available mp3 files are listed;(6)Grammar file: File selector, lists available .raw or .grammar files;(7)Grammer text:Display is strong with syntax with being automatically performed The grammer syntax editing machine of tune;(8)TTS:There are TTS editing machines, be likely to be at preview mode;(9)JavaScript:Display JavaScript editing machines, such as Atom, the syntax with social robot API are emphasized to complete with possible code;(10)Ring Border variable:These are the variables important to PCD 100;(11)Numeral:There is digital frame, Min Max, default value;(12)Integer: There is integer choice box, Min Max, default value;(13)Boolean:There is true/false combo box or radio select button; (14)Array <Type>:The ability of the key element of type is added, subtracts, moves up or moved down in display;(15)Vector3d:Display (X, y, z)Frame;With(16)People:May be nearest, farthest, most well-known etc..
With the operation action trees of PCD 100, debugging web interface can show that the figure of tree is represented, prominent working as where it Front nodal point.Start, stop and button may can use in advance.In interval, instrument can allow to global monitored parameterses and row Introspection is carried out for parameter value.In addition, limited input interaction can keep available.This can include such as triggering phrase or by people It is placed near social robot, it can for example add the template knowledge on the people.In embodiment, developer is also Behavior model can be shared with other developers, for example, share sensorimotor technical ability or module.If for example, PCD 100 has Mobile basis, then can share navigation and mapping model between developer.Action logic class can be changed by developer, for example The variant of extension and offer functionally.
SDK instrument can include being used to manage the representation aids external member of the expression of social robot.Representation aids external member Central characteristics be analogue window.With reference to Figure 24, it is illustrated that the embodiment of analogue window, the front view in two of which screenshot capture Simulate PCD 100 animation.Top Front view 2 400 goes back the focus of simulation eye figure.Upper left in each screenshot capture Simulated screen figure 2402,2402'.The simulated view can be write with WebGL so that not need special instrument to simulate society Hand over the robot animation(In addition to the browser with current version, such as ChromeTM, it is currently running).This simulated view Itself need not be single automated tool;On the contrary, it can be able to will be caused with the view in insertion tool, the instrument The main frame of PCD platforms and other developers can create and test PCD animations(The animation of such as various technical ability).It can opened Called when originator wants real-time playback motion or animation or by " having stepped through " animation in order.Therefore, provided herein is A kind of emulation tool for being used to simulate the behavior of social robot, wherein identical code can be used for the imitative of social robot True and actual motion.
With reference to Figure 25, it is illustrated that the social the robot animation editing machine of social robot representation aids external member it is exemplary and non- Restricted embodiment.Use this instrument, developer can by by one or more social robot motions, screen graph, sound, The social the robot animation that text is constituted to speech action and illumination (such as main LED body illumination and function) pieces together.Figure 25 show that the conventional Animation Editors 2500 with the types being used together of PCD 100 can be adapted to.The key of Animation Editors is special Analogue window 2502, the Animation Editors 2504 that can include being used to play back social the robot animation are levied, in Animation Editors Developer/designer can be by assets(Motion, figure, sound/TTS, main LED body illumination or full animation)It is placed into timeline And in asset library 2506, wherein developer/designer can select existing assets to be included in timeline.Assets may be come From the hard disk drive of developer, or from PCD shops.This can be supported for the view for changing PCD 100, ratio, rotation Deng 3D views.In embodiment, editing machine can allow use can extend PCD virtual environment background or object, example Such as there is the embodiment for being used for simulating people, input is received from user interface.In embodiment, Animation Editors can have reversion Control and permit a user to the pattern that robot is posed and has the interface for being used to be set key frame based on the posture.With similar Mode, can be completed by touching to manipulate to the animation as the key elements based on screen of eyes, covering or Background factors, Then key frame is carried out to new direction/change.The variant of this method can also be implemented, for example, remembered using PCD 100 Record animation(Placeholder is final)Self-defined sound effect will greatly speed up the inventive processes of skill in design.In embodiment In, instrument can allow via the Animation Editors directly preview animation on the PCD 100 for be connected to editing machine.
In embodiment, the main frame of PCD platforms can be supported to import assets and create the ability of new assets." importing " and " establishment " ability can support various Asset Types as described herein.Moved for example, creating new element and can start social robot The Actions tool is drawn, and creates the speech instrument that new TTS phrases start social robot.
Creating new LED illumination scheme can specify via dialog box or illuminations.
In embodiment, one or more instruments can be embodied as web applications, such as ChromeTMWeb is applied.In reality Apply in example, given instrument can preserve social the robot animation in itself, such as with the unique document of such as .jba or .anim files Type, and save as social the robot animation projection file, such as .jbp file types.As PCD 100 is with new ability example Such as perception, physical capability, ability to express and new equipment(For example, augmented reality device)The evolution, this method such as connectivity New tool can be expanded to.
With reference to Figure 26, it is illustrated that the exemplary and non-limiting example of PCD Animation Editors 2500, it can for example pass through " New...Animation " is called to use from PCD Animation Editors 2500.In its core, there are arcuate positions and specify body Position(For example, in three partial robotics, passing through the radial position bottom of control machine people, middle and top).In fig. 26, One group of sliding block 2602 can be used for providing shift position.In embodiment, every group of position can also be timestamp so that complete shifting It is dynamic to be defined by the array of time/body-positional value.Remaining sliding block can be used for controlling the engagement in eye animation.In embodiment In, the new eyes animation of establishment can be separated with creating new Body Animation(Two animations are merged in the present embodiment).Finally, work Tool can also be supported to import texture file to control the outward appearance of eyes figure.The instrument can support the simulation with touch-screen to hand over Mutually.In embodiment, instrument can enable the various figures beyond eyes, such as interactive fiction animation.
PCD simulators not only can include above-mentioned analogue window, and can have be used for inject feel input interface/ Console.
In embodiment, the access based on key of pair network gateway associated with PCD 100 can allow developer Technical ability is installed in social robot to develop and test.Network gateway on PCD 100 can provide opening based on web Hair, debugging and visualization tool set, for user continue interacted with PCD 100 when PCD 100 technical ability operation when Between debug.
PCD 100 can be with associated long-range storage facility, such as PCD clouds, and it can include one group of trustship, be based on Web instrument and storage capacity, it supports the content creating for figure, body movement, sound and the animation of expression.Implementing In example, PCD 100 can have other to handle outside the venue, such as the study of speech recognition machine, navigation.This can include being used to make The instrument based on web of the behavior tree of the logic of technical ability is created with behavior treebank, and for strengthening " inserting for developer's technical ability The storehouse of part " content, such as common emotion animation, figure and sound.Interface can be it is expansible with other API(For example Home automation API etc.)Interface.
Method disclosed herein and system can solve various security considerations.For example, technical ability may need authorization token Access sensitive platform resource, such as video and input audio stream.Technical ability can be released as number by social robot shop " bag " of word signature, and can be verified during installation.Developer can obtain single software kit with appropriate key, make For a SDK part.
In embodiment, PCD SDK can include can be by such as ChromeTMThe group that the simple browser of browser is accessed Point, it supports routine web developing instruments, such as HTML5, CSS, JS and WebGL, and visualization painting canvas.In embodiment, example Such as ChromeTMThe open-source version of browser can be used for building desktop application and for simulator, development environment and phase Plug-in unit is closed, and run time is applied for PCD 100.This means PCD 100 code(Either it is used to develop, simulate Or used during operation)It can generally be run in the Conventional browsers with minimum revision, such as it is clear in mobile or PC to allow Look at preview technical ability on device.
SDK described in text can support various Asset Types, such as input syntax(Word identification language for example comprising pre-adjustment Method), graphic resource(Popular graphic resource for example for being shown on the screen of social robot);Sound(For example for The pop voice resource played on PCD 100 loudspeaker, carves the rhythm of PCD 100 voice, adds and filters to voice Device and other sound effects);Animation(Such as popular action binding, screen graph, sound and the language for being packaged into coordination animation Sound);With behavior tree(Such as developer can be merged into the fashion behavior tree example in technical ability).
PCD SDK can enable to manage feels input and control ability on a large scale, such as with local sensing space Related ability(Such as real-time 3D personage's tracking, people's identification by voice and/or face recognition and facial emotion are estimated);Into Picture(For example catch photo, covering image and compression image stream);Audio input(Such as positioning audio-source, selection audio signal beam Direction, and compression audio stream);Speech recognition(Such as identification of speaker identification, phrase and the use of phrase chunking grammer, Title identification, received pronunciation recognize and used self-defined phrase chunking grammer);Touch(For example detect to the face on graphical element Touch of the touch and detection in portion to the head of social robot);And control(For example using simplified IFTTT, have The complex behavior tree in JavaScript or built-in behaviors storehouse).
PCD SDK can also have the output and shared relevant various abilities with expression, for example, be related to movement(For example broadcast Put animation, the self-defined animation of creation, the self-defined animation of importing and programming and dynamic animation construction that social robot is created); Sound(For example play sound, the self-defined sound of importing, the self-defined sound of broadcasting and mixing that social robot is created(For example it is real When)Or blending sound);Voice output(For example play back the voice segments prerecorded, support correct name pronunciations, using text Text, the pre-recorded voice segments with reference to customization are played back to voice and are annotated using text to speech emotional).Illumination (For example control LED);Figure(For example perform figure or the self-defined figure of importing that social robot is created);Shared personalization Or technical ability(Run for example in single account on device, it is shared with other developers on other devices, and it is distributed to skill Can shop).
Exemplary there is provided for coordinating Internet of Things using PCD 100 with non-limiting example according to various(IOT) The method and system of the live show of device.
In certain embodiments, PCD 100 can have found type and the position of IOT devices, including loudspeaker, lamp automatically Deng.Then PCD 100 can control lamp and loudspeaker to strengthen live musical performance.PCD 100 can also learn through experience The preference of user, such as external device (ED)(Such as music apparatus, IOT devices)Personal settings and behavior.
Because cheap IOT devices become universal, them can be used in entertainment way.With space reflection, object The PCD 100 of detection and audio detection is ideally equipped to control these in phase with music, video and other entertainment mediums Device.Well-planned performance military order spectators are joyful.
There is business industry & solution to automatically control sound and illumination to strengthen drama and live musical performance.Similar is System is also used for enhancing Karaoke performance.The problem of existing business system be they costly, and it is correct to need professional knowledge Ground configures sound and lighting device.Controllable device generally aims at theater or auditorium Environment Design.These systems and device are in family In can not find.
There is provided herein properly programmed PCD 100, it can be with(1)It is automatic to find to include the IOT devices of lamp, loudspeaker etc. Type and position, and(2)These lamps, loudspeaker etc. are controlled, to strengthen live musical performance.
Consider that IOT lamps and loudspeaker have been arranged in such as kitchen and adjacent family room by family at home.As This family of the adopter of new technology can buy the personal PCD 100 that can be deployed in kitchen.As it, program is set A part, social robot can be found that type and the position of the IOT devices of family, and asks to access and control being permitted for they Can.If authorizing license, PCD 100 can provide to perform popular song.Then social robot uses the sound of oneself System and expression power physics animation strike up.Then, for the happiness of family, the IOT lamps of kitchen and family room start with Music is pulsed, and emphasizes music event.Then IOT loudspeakers are commenced play out, and strengthen stereo/spatial property of music.
By IOT devices and music(Or other)The ability that performance is coordinated strengthens PCD 100 perceived value.It can also make It is valuable in terms of automatically setting up and strengthening the self-organizing on-the-spot demonstration outside family to obtain PCD 100.
There is provided herein for adjusting the method for meeting between mankind participant or dialogue using PCD 100 and being System.In this kind of embodiment, appropriately designed PCD 100 may be used as meeting presider, to improve meeting and moving for talking with State property and validity.
Meeting generally as desired like that it is effective, and can dexterously adjust meeting individual not always can use 's.Successfully solve to cause the trial of the factor of suboptimum meeting that special training meeting or the shape using expert host is usually taken Formula.These methods can be effective, but they are expensive.
Unbred individual, which attempts to adjust meeting, generally to fail because the personal instruction provided colleague and suggestion have to Drag.
Generally, the purpose of meeting or dialogue is that the idea and opinion contributed in conference process by participant are discussed.Generally, Desired attendee there will be an opportunity to freely contribute.Make in view of these targets and expectation, optimal meeting or dialogue are all participants Go out preciousness and correlation contribution, and contribute all important ideas and opinion.
Many human factors may limit the success of meeting.For example, personal target and the phase for being not always directed to meeting Hope.In addition, person-to-person dynamic not always meets target and the expectation of meeting.Sometimes, the intention of the participant of meeting is bright The aobvious target for running counter to meeting.For example, it is intended to which the meeting that catalytic phase is mutually discussed may be kidnapped by participant, the target of the participant is Guiding is discussed in a certain direction.In other cases, person-to-person dynamic is probably hostile, causes discussion to concentrate on and moves State rather than expected theme.The successful minimum of meeting can also be made by unintentionally interrupting.For example, convictive expression participant Discussion may have been monopolized unintentionally, prevent other people from freely contributing.
It is many due to these limiting factors(If not most of)Meeting is suboptimum.In business environment, suboptimum, Poorly efficient meeting is probably the expensive wasting of resources.In the family, the talk of suboptimum is probably the unfortunate chance missed.
As described above the problem of is the result of natural human's tendency, and it still has because seldom accomplish to solve and Correct it.During personal typical educational, take a significant amount of time for reading, writing, arithmetic, science, art, music, business The guidance of industry etc..But for important skill, such as session, cooperation or persuasion(Rhetoric)There is provided seldom or indefinite guidance. Therefore, have an opportunity the efficiency for greatly improving cooperation (usually) and meeting (especially).
Research discloses the mankind and is more willing to receive and follows the guidance from social robot and advise rather than from another It is personal.Social robot can as meeting it is just, non-judgement property expert host.PCD living things feature recognition ability It can be allowed to be accurately tracked by and measure the degree of participation of each individual in a meeting.The information can be real-time as what is participated in Histogram is presented.Histogram can include:Everyone air time;It is person-to-person repeatedly;Everyone planning tone (It is positive/negative);Courtesy;Habit expression(Front and it is negative, encourage and disparage, it is insensitive);Cultural impoliteness;Personal mood state(Feelings Sense analysis);Gross energy changes with time;With main topic of discussion and subtopic.
During the entire process of meeting, PCD 100 can transcribe language content and it is associated with sociometry, to carry For the objective instrument for capturing both the validity for discussing and assessing meeting.
PCD 100 can be configured with dependent thresholds so that it can keep meeting normally to enter in session insertion OK.For example, robot can be inserted in a case where:Someone talks too much;Tone is too negative;Use unsuitable usual table Reach;Detection is insensitive;Integral energy is too low;And/or important theme is not resolved.
Abilities of the PCD 100 using it as just meeting presider with both social mirror images can help participant to realize two Individual important goal:It is more effectively carried out meeting and more effectively study cooperation and talks.
For example, meeting is can be with the environment of deployment techniques.Meeting participant can include coming from have various ditch air gratings Diverse discipline expert.In session by the case that talk participant controls, PCD hosts can be with(With non-judgment mode)It is in The real time histogram shown on now appropriate display, the display shows the relative air time of all participants.In addition, If using unsuitable expression, social robot can be with(Without judging)These expression are attributed to contribution participant, for example Via histogram.The energy and tone of meeting can also be measured and tracked in real time, and are compared with former effective meeting.Make For opportunity to study, the statistics that PCD 100 can be used to collect carrys out the effective and invalid meeting of comparison.
Thus, for example PCD 100 social robot can serve as the host of meeting, record and display relevant information, And the validity and dynamic of meeting are improved, this can be converted into the productivity of raising and preferably use resource.
It is also provided herein for organizing the network of robot agent with the movement of certified human identity and networking Distribute the method and system of information between device.
With the value volume and range of product increase of communication channel, " noise " that sender of the message and recipient have to compete for also increases Plus.In addition, new channel is usually specialized for specific message modes of delivery.Result is sender of the message has to decide on which is used Individual channel come make message deliver possibility and validity maximize.Similarly, message recipient, which has to decide on, " will watch " which It is individual(Which)Channel to receive message in time.These decisions are increasingly difficult to.
Now, the message from multiple electronic mail accounts can automatically be integrated by mail reader so that can be with Monitor multiple e-mail channels simultaneously.Equally, the text from multiple channels can be presented in an integrated fashion and disappear for mobile device Breath.However, message integrates the problem of can not solving " noise ".Bombard and receive by using the message all presented with model identical Person, it may make problem become even worse.
Social robot can play the effect of uniqueness in message communicating, because they can command notice, and Because the mankind distribute to the importance of the communication of mankind's hobby.When social robot is used as delivering message to recipient During channel, modes of delivery can be automatically selected by social robot so that the optimal degree of concern of message sink to recipient.
This can use the exclusive multifrequency nature of social robot to realize:(1)Social robot is physically present permission It attracts notice, i.e. motion, gaze-direction, " body language " by the expressivity clue of the congenital adaptation of the mankind;(2)With life The social robot of thing feature recognition ability can detect when the intended recipient of message is physically present, and can point out to connect Receipts person has maximally effective physics clue;With(3)The learning algorithm that social robot is used can use message content, in situation Hold the behavior history with recipient to make the optimal decision on how effectively to deliver message.
The social robot of such as PCD 100 networking and other devices such as mobile device and other network connection devices It can be used in method disclosed herein and system.When robot social using multiple networkings, by single social robot The information delivery advantage of offer is exaggerated.In home environment, the multiple PCD being distributed in the room/region in house can be assisted Its message is adjusted to deliver effort.Being physically present for multiple PCD adds robot and can deliver the window of message in whole family. PCD network can use its shared living things feature recognition ability to track the whereabouts of the intended recipient in whole family.By The learning algorithm of PCD Web vector graphic can generate the forecast model on recipient's movement and behavior, to determine which PCD generations Reason can most effectively deliver message.
This identical dynamically can apply to any physical location, and can apply to business, museum, library Deng.
The physical form of robot in PCD network can change.Network can be by fixed, movement, removable Dynamic, can rolling, can fly, PCD in the instrument board that is embedded in vehicle, being embedded in the utensil such as refrigerator Composition.
In addition, PCD " brain "(Its software, logic, learning algorithm, memory etc.)It can be replicated across a variety of devices, its In some there is the main body physically expressed, and some of them do not have, such as the softwares of PCD 100 be included in mobile phone or Flat board(Copy to mobile device)In the case of.
When PCD software is copied to mobile device, the device can serve as the complete cooperation of social robot network , the member realized completely, and with the mankind in social and/or technical network.The PCD situations of physical limit can be to delivering The degree that the task of message contributes depends on the function that it possesses, i.e., the PCD softwares included in typical smart mobile phone lead to Offer living things feature recognition, camera monitoring, speech recognition can be often provided by screen and even analog physical is represented.
The limited PCD situations of smart mobile phone usually can contribute the message formed completely, then can be by its in network His free PCD is delivered.
In the network of PCD situations, each situation can be operated as completely self-contained contributor.However, any given Situation can also act as the remote interface of another PCD situation on network(Remote control).The remote interface pattern can With intermittently movable, or situation can for good and all be configured to serve as the remote interface of another situation, such as PCD softwares wherein It is included in smart mobile phone or intelligent watch in the case of being used to provide remote access to unrestricted situation.
In embodiment, in home dwelling setting, it can be used by father and mother unfettered in kitchen(Full feature)Machine Device people unit creates message.Father and mother can create message by being conversed with PCD 100.
The message can be captured as V recording and as text transcription (such as from speech-to-text technology), And via text to voice(TTS)Delivering.Delivering is arranged in following a period of time, such as today after school.It is expected to receive Teenager person may stay out, but may be reached in expected Delivery time.In this illustration, teenager is going home after school, but Do not enter kitchen.The robot cell including flat board being embedded in by garage port in wall can recognize teenager when she reaches. Due to the unit including flat board and kitchen machine people unit, upstairs robot cell and teenager include iPod robot cell Networking, all four units cooperate with timely delivery information.For this message, modes of delivery preferably is via unfettered Robot cell, therefore flat unit is only referred to message and waits.", [teenager], you have information wait." juvenile possible Her room is gone to, kitchen and robot cell upstairs is bypassed.When Delivery time is reached, the network of robot cell can be with It is determined that, because teenager is not near free robot cell, next best mode of delivering message is via teenager Unit including iPod.Therefore, iPod units send prompt tone, and delivery information:" he, [teenager].There is a chocolate cake (brownie) you is waited in kitchen." when teenager enters kitchen finally, kitchen machine people's unit has appreciated that message Through delivering, and only provide the prompting of courtesy:", [teenager].If your all set that chocolate cake, it is in baking box In." PCD 100 can be with the source of the content of summary message, and message, for example " Carol, Jim have stayed one to disappear to you Breath.Today is about connecing the thing of children from football." this potentially contributes to Carol and decides when to listen information(Immediately or slightly Afterwards).
Therefore, the network of social robot can use living things feature recognition, track, be physically present(For example based on PCD Link between 100 and associated mobile device), non-language and/or social prompting and actively prompting it is no to deliver message It can then be lost in the noise of multiple crowded Message Channels.
In other embodiments, loudly listening to TV or play the video-game loudly played may be to neighbouring other people It is very irritated, because having different tastes in terms of audio pleasure is caused.In addition, the member of many families others stay up late Obtain more late.
The solution proposed is to support audience to use the mode from the earphone of social robot wireless receiving audio, institute To only have audience to hear him, and they can freely according to they want as loudly listen to, without appropriate Association.Variant can include bluetooth earphone, earphone beam, the mobile receiver with wired earphone(For example using local Wifi or indigo plant Tooth)Deng.
According to exemplary and non-limiting example, PCD 100 can have and the personal assistant on popular smart mobile phone Those similar prompting abilities.For example:" December 5, at 3 points in afternoon, reminded me to buy year anniversary present " " well, and I can remind You ".Prompting can occur repeatedly, to support the thing of medicine prompting.User can have is created as audio or video by prompting The option of record, in this case, PCD 100 may need to point out when recording and starting.PCD 100 can be created in message Summarized after building:For example, " OK, when I sees him, I will remind John [broadcasting audio] tomorrows.Prompting is only referred to timing Between PCD Jot special shape.
PCD 100 can in the family remind known people(Reminded for same, it is one or more)Thing.For example, " when you see Suzie, reminding her to do homework " or " 6 pm reminds father and mother to meet me from football training." such as Fruit provides prompting, if then he or she has social activity robotLink devices, should chain notice in social robot PCD The promoter of prompting.Remind the special shape for the PCD Jot for being only referred to fix time.In embodiment, link can be in PCD 100 Between mobile device.
If PCD 100 not remind by transmissibility, because target person is not reminded there, then can appear in the social activity of target On robotLink devices.If being not allocated to the social robotLink devices of target, PCD 100 can see mesh at it Message is shown immediately during mark personage.
According to exemplary and non-limiting example, PCD 100 can be into its catalogue other PCD send short essay This message or V recording, referred to herein as " Jots ".PCD Jot message can be editable, and PCD Jot are recorded It can before transmitting play back and record again.PCD 100 can confirm that PCD Jot have been successfully transmitted to sender.PCD 100 can safeguard " transmission " Jots files for each member of family, can browse and delete message by message.Hair The Jots sent can PCD link or PCD 100 on visible and/or editable.
PCD can maintain the list of PCD animations, " robotticons " is referred to herein as, similar to based on screen The expression used in device, for example, give message life or improve the vigor of message.Example potentially includes the lovely of " hello " and blinked " o0 " of eye or " uh-oh ".Social robotticons can be elaborate, and some special storehouses are available in PCD Buy in technical ability shop.Some PCD robotticons can be independent animation expression.Other are adapted to user video figure Picture/message it is integrated.PCD robots can include PCD expressive ability(LED, double knee jerks or other sound or sound effect Really, animation etc.)In any one.
If user's selection sends photo, the photo for example captured by PCT " snapshot " pattern, then PCD Jot abilities Available for being attached to photo.
For example, kinsfolk can always ask PCD 100 " prompting [coming from [people]] that me is played to me ", and PCD 100 can be responded by being commenced play out from the earliest prompting of the people.PCD screen may be indicated to remind and waited.Such as Fruit PCD sees PCD Jot intended recipient, if prompting was not watched within past six hours then, and remind when Between have arrived at now, then PCD 100 can provide broadcasting Jot.After message is checked, recipient, which can have, to be replied or turns The option of hair, then preserves or deletes message, or " pause " and replay message after user-defined time interval.Acquiescence Operation can preserve message.PCD can safeguard the PCD Jot of each member for the family that can be scrolled inbox.
In the case where there are multiple kinsfolks, incoming PCD Jot can carry the identifier of intended recipient.PCD 100 only can show message to intended recipient or other authorized users.For example, each member of family can have them certainly Oneself color, and " message " designator flashed in the color causes the kinsfolk to know that the message is to theirs. Example should adapt to there is a situation where that different messages wait different home member.Whether kinsfolk, which is authorized to, is checked another family The message of member can be configured via keeper.
PCD 100 can create pending list and Shopping List, and it can chain to check and can compile in PCD Volume.For example, user may can say " PCD, I needs Jenny registrations summer camp of side ", and PCD 100 can answer " I It with the addition of ' Jenny registrations summer camp of side ' and arrived your pending list.Or " PCD, addition butter to my Shopping List.” Can be that each kinsfolk or whole family create list.Each member of family can have list, and there may be Family's list.
PCD Jot may be overtime after without using a period of time.
PCD, which can have, engages in social activity and based on character(Emotion, the behavior of person model driving)Interaction, decision-making, with Lasting " Be " state of the tendency of user.The state can adjust PCD technical ability, based on experience and other input come it is personalized these The PCD behaviors of technical ability and performance are to specific user.
PCD 100 can have single different " power-off " posture, and cause some of its posture when it is closed Different animation sequences.PCD 100 can have single different " Asleep " posture (when PCD 100 is inserted into or in battery When being run on power supply), and after it obtains " sleep " order or if it determines to doze off at one's leisure, cause it Multiple different animation sequences of posture.PCD 100 can have a variety of different animations, and it corresponds to " wake-up " voice or tactile Order or other audio-visual events, or when it has been slept or closes<Opened at=48 hours/connect power supply.In embodiment, There may be different sleep patterns, such as PCD 100 is being waited but still with effective microphone and camera with suitable The sleep pattern waken up at that time.In another sleep pattern(It can be indicated by a certain prompting, such as LED indicator)In, PCD 100 Can have microphone and camera to close so that PCD 100 does not see or heard in sleep in this mode.In latter mode In, people may need to touch robot or wake up PCD 100 using the mode different from voice or vision input.
PCD 100 can have a variety of different animations, and it corresponds to voice or tactile " wake-up " order or or other audiovisual Event, when it has been slept or closes>Opened at=48 hours/connect power supply.
For example, PCD 100 can have a variety of wake up animation, it corresponds to voice or tactile " wake-up " is ordered, or at this At 11 points in evening ground time turns on the power after sleeping or closed more than 3 hours between 11 points of morning.
PCD 100 can have a variety of different " daydream " modes in sleep.These daydream states are likely to occur in about During 30% sleep stage, the duration was more than 15 minutes.PCD daydream can be interrupted, so as to order or by touching Screen makes it enter silence sleep state, diverts one's attention if the people in room has found that it has a dream.
PCD 100 can its power level be less than 20% when and behind every time e.g., from about 5% when successively decreasing it is oral and User is notified on screen.
When PCD 100 power supply switches between socket and battery, PCD 100 can notify user on screen.It It should be able to also answer a question, for example " whether you have insertedOr " you use battery" when pressing and keep its head During button on the back side, PCD 100 can be automatically turned on or closed.Short-press button, which is promoted, allows social robot to sleep.
PCD 100 can be configured to via(Voice or touch)Or only touch and waken up from sleep.If PCD 100 is opened Open but have neither part nor lot in actively interaction(That is, in the base station for being referred to herein as " Be " or " being " state), then PCD 100 can be Displaying passive cognitive animation when someone enters its circuit or sight or made a noise.If PCD 100 believes that the people wants ginseng With then these animations can cause idle actively cognitive.
If PCD 100 passively recognizes someone and thinks that the people wants to actively assist in, due to verbal order or Because the people is just deliberately walking towards PCD 100, it can show the active consciousness animation of " servicing you " type.
Whether PCD 100 can comment on it because exterior object covers his eyes and can't see it, if it is required Do the thing of any required sight.If the tappings on head independently of any kind of prompting of PCD 100, PCD 100 can It is idle actively cognitive to return to.In other embodiments, if PCD 100 is stroked or shown tender care for, or if it is oral Praise, then " happiness " animation can be presented in it, and returns to idle actively cognitive.
If the generally acknowledged member of PCD family recognizes that PCD 100 generally may be used in sight such as by voice ID Greeted with personal manner to the kinsfolk, although being not necessarily oral(This can depend on last of the kinsfolk The close relative of secondary observation).
If stranger arrives in sight or by speech detection, PCD can enter passive Cognitive Mode.If it is examined The interest from stranger is measured, it should introduce oneself, rather than repeated.PCD 100 can not inquire that who is another on one's own initiative One people, because " known kinsfolk " is managed by PCD household operation person.
If the generally acknowledged member of PCD family has unrecognized stranger, PCD 100 receives family in person first Member.If PCD 100 is made referrals to stranger by the kinsfolk, PCD 100 can not inquire that who is that this is another on one's own initiative Individual, because " known kinsfolk " is managed by the household operation person of social robot.
If the household operation person of social robot introduces social robot meets new people to meet, and keeper master Say that he should remember the new people dynamicly, social robot should occupy one in 16 ID grooves.If without available ID Groove, then PCD 100 can inquire keeper whether he or she wants to replace existing identified people.
When being required to learn new people, PCD 100 collects necessary vision and voice data, and can also advise pipe Reason person makes new people by PCD connectivity applications preferably to capture vision and audio sample, and study name pronunciations.
In certain embodiments, PCD 100 can have the greeting of diversified forms based on the time of one day.For example, " morning It is good " or " good night " or " you, which rise, is late "., can be continually but not total if PCD 100 knows the people that it is being greeted It is to come personalized using the name of the people.
If someone says goodbye to PCD 100, it can have a variety of farewell modes.If PCD 100 knows that the people says Goodbye, then it can personalization and the farewell of the name of the people.
PCD 100 can have some idle chat capabilities for building in this way so that they do not encourage not by The dialogue of constraint.These can include the voice for being intended to user response, or be designed to amusement user rather than calling response Simple small language.These voices can refer to known " family is true " defined in " family is true " tab, for example, wish house Someone in people's " happy birthday ".In embodiment, it can be shown on screen on PCD 100 expects what voice heard Visual cues, for example, point out PCD 100 user.Voice can also be the earth's core of the postcode based on specific PCD.Voice It is probably the topic pushed by design team from PCD clouds, for example " I dare not believe that Birdman sweeps across Oscar!" small language can Humorous, clever and with PCD the role of energy is consistent.Chatbot contents should also understand people from PCD memory and like and not like Vigorously what, it is told based on them, or it is reacted to what such as picture, song, joke thing collect from facial expression.
The problem of PCD 100 periodically can be designed to amusement to kinsfolk's inquiry.
PCD 100, which can have, represents unapprehended a variety of graceful modes, if can not although request repeats voice Understand user, then encourage user's forgiveness.
PCD 100 can have its separable gratifying specific behavior that express frequently, such as certain preference, it is frightened and Mood.
PCD 100 can have the multi-mode disambiguation example of definition, and it can be designed as the patient and width for causing user For this reason.
PCD 100 can have a variety of graceful modes and understand voice to express it, but can not satisfactorily observe or ring Should.
PCD 100 can show mode that is happy, occupied and not needing any auxiliary and recreate by oneself silently sometimes.
PCD 100, which can have, there are ways to show it during any delay event or in Core server regeneration period Between think deeply.
PCD 100, which can have, there are ways to warning user its WiFi connection disconnection, and WiFi has been reconnected. User can reactivate WiFi by the QR code for setting and being linked using PCD at any time.
PCD 100, which can have, allows user browse and input technical ability and basic setup and exit the basic of active technical ability Multi-modal navigation example.Senior setting may need to input by PCD Link.
PCD 100 can have the ability that its keeper comes out its " locking " that allows so that in the case of no password, In addition to its locked apology is notified, PCD 100 can not be used.
PCD 100 can show available WiFi network according to order.If WiFi connections are lost, PCD 100 Available WiFi network can be shown.PCD 100 can be provided on his screen the mode for inputting WiFi passwords.
PCD 100 can have each known member with the family to have visual correlation.For example, Jim is always blue, Jane is always pink, and mother is always green, father always purple.When PCD 100 is interacted with the member of family, the vision Scheme should be leading.The visual identifier can be used in PCD technical ability, to ensure that kinsfolk knows PCD 100 Recognize them.
PCD 100, which can be recognized, to be smiled and is responded in a similar way.
PCD 100 can play the picture from its PCD Snap photograph album in the case where sliding display pattern, when it is in Be If with user in picture, PCD 100 is it may be said that " you seem especially good in this ".Sometimes, PCD 100 can be looked into Its " oneself " photo, such as the first Macintosh or R2D2 or pinball machine is seen, but then also includes the photograph of his household frequently Piece.
PCD 100 can often show happiness without interaction.Rattled for example, it is played itself with, on its screen Picture is drawn as Mona Lisa as face with PCD 100.Over time, these technical ability may be developed(For example, from the moon Ball lander ASCII plays or stick figure starts, and then movees to more complicated game).In certain embodiments, PCD 100 can have pet, such as doggie, and its eye can become the ball that dog can obtain.PCD 100 can be with him dog Together passively back and forth.It may browse its technical ability, such as read recipe.It can dance to certain limited music libraries, practice Practise its action.Sometimes it is in nap.In certain embodiments, PCD 100 can be based on family's fact with medal come book Write the poem, for example Haikus.In other embodiments, PCD 100 can temper and give oneself and encourage.In other embodiments, PCD 100 can play musical instrument, watch interesting pipe clamp and response laugh, and color is performed by digital children's play, it is mobile so that Ball is moved through labyrinth and object for appreciation number is only.PCD 100 can have the photograph album and collect stamps of their own.
In certain embodiments, PCD 100 can be participated in and be shown the game based on table tennis, wherein from side to opposite side Mobile control user resist PCD 100 racket.
If PCD 100 is run using battery supply, the display remaining power life-span is there may be on its screen Icon.
If people praise PCD 100 in social situation rather than task situation, it can show " happiness/emotion " Animation.
When in group, PCD 100 can once with a personal contact.If they indicate to expect to converse with PCD 100, And the people that PCD 100 is currently being associated keeps silent or otherwise departed from, then it can only turn to and associate other people. In embodiment, PCD can use various non-languages and complementary social prompting comes while managing many people's interactions.
PCD 100 can have basic timer function.Such as " PCD, allows me to know, 15 minutes in the past ".
PCD 100 may can create tone to help user to determine being connected to via PCD links on PCD 100 phone The phone of loss of the position in the range of WiFi.Control whether someone can be not the device of oneself PCD chains answer the call on create Building the function of this tone can be configured by Administrator.
PCD 100 can have the stop watch function of those being similar in current smartphone.
PCD 100 can have onboard clock and can it is in office when notifying time in area, if requested.Have When, PCD 100 can be at least partially based on its participation level and what does to show the time, and other times may not show Show.PCD 100 can have alarm clock function.For example " social robot allows me to know when afternoon 3:30”.It may include Doze off function.PCD 100 can have several available alarm sounds, and each kinsfolk can set them preferred Alarm sound.If being not provided with preferred alarm sound, PCD 100 can select one.
PCD 100 can have built up multi-party interactive strategy, and it can change according to technical ability.
PCD 100 can have quick " demonstration spool ", if being required " showing off " its ability, it can show.
When being run into by voice ID and recognizing another PCD 100, PCD 100, which can have, to be specified but simple behavior choosing , if it is incorporated into another PCD 100 by kinsfolk.In embodiment, PCD 100 can have be designed for it is another The specific special behavior that one PCD 100 is interacted.
According to exemplary and non-limiting example, technical ability or behavior are given(Such as animation, voice)Can be based on Associated PCD 100 other attributes and different earth's surfaces are shown.For example, PCD 100 can be programmed or be adapted to, for example, pass through With interacting for time and user or colony, with some individual character, to undertake certain role, operated with AD HOC, with certain The level of mood, expression energy or fatigue is planted, certain effect etc. is played.PCD SDK can allow developer to indicate certain skills Or how its component should be changed based on any one in foregoing or foregoing any combinations.For example, PCD 100 can have There is " optimistic " individual character, in this case, compared with performing " introversion " PCD 100 of shorter, quieter version, it can be performed The speech act of longer, louder version.Similarly, " active " PCD 100 can carry out big movement, and " quiet " PCD 100 can carry out small movement when performing identical technical ability or behavior.Similarly, " fatigue " PCD 100 may show slow Motion, slow voice etc., be for example difficult to imply the time that children prepare sleep with discovering.Therefore, there is provided herein social activity Robot platform, it includes allowing the SDK of Develop ability and behavior, wherein can be according to the mould of the PCD 100 independently of technical ability Formula comes expression skill and behavior.In embodiment, PCD 100 may adapt to carry out different interactions from different people, for example Spoken with the adult different from children, while remaining in that clear, consistent role.
According to various embodiments, various technical ability can be provided.Important technical ability includes meeting technical ability(Including for the first time and with The enhanced video calling of meeting afterwards, such as robot), monitoring technical ability(Such as monitor the people in family and/or pet), photographer's skill Energy, technical ability of telling a story(It is mashed up with multimedia, for example allow user to be selected at branch point to influence risk map, based on many matchmakers Story that body surface is drilled etc.), play game skill, allow user use " witch mirror " technical ability as smart mirror of social activity robot, weather Technical ability, technical performance, or motion partner technical ability, the sound that interaction is moved with strengthening sports cast or sport information or activity such as to have fantasies of Happy technical ability, the technical ability worked together with recipe (being used as the intelligent interactive prompting device with background/animation effect), and coach Technical ability(Such as drug compliance, personal development, training).
Recognized for the ease of automated voice(Or other voice recognitions), method disclosed herein and system can enter traveling wave Beam is formed.Challenge is, it may be desirable to allow user for example by using for example " he, Buddy " " hot word " arouses social activity The attention of robot.If PCD 100 is present, it can be turned to(Or guide attention), send the sound of hot word.One carried out The mode of kind is to use Wave beam forming, wherein there is the wave beam for pointing to diverse location(Spatial filter or channel).In theory, correspondence In wave beam each spatial filter or channel from the channel acquisition sound, and seek to ignore other channels.Generally, people pass through Pickup has the wave beam of highest volume and assumes that highest volume wave beam is the wave beam spoken for people, in for example many mixers Carry out.Wave beam forming and utilization that methods and systems disclosed herein can be improved, for example, say hot word to pick up People wave beam.In embodiment, social robot platform disclosed herein can be for each wave beam or for wave beam Subset has the different instances of speech recognition device.Therefore, each speech recognition device is just bored in listening space.If device is for example In the group of four people, and a people say " he Buddy ", then device and then will see that someone calls note from the direction of the speaker Meaning.To achieve it, system and method can have the speech recognition device of the subset of each channel or channel.
It is desirable that people may want to the motion/orientation based on PCD to maintain the orientation of wave beam.Run Wave beam forming System can be from motor controller receive information, or the outside that can be inputted from such as gps system, vision system or vision is Receiving position of uniting or orientation, or from the position system in the environment of such as family, such as position based on IOT devices.For example, Motor controller is known that the angle that PCD 100 rotates PCD 100, and then PCD 100 may need to find its coordinate.This It can be redirected, or be realized by using other positions information by saying hot word again.Once position speaker Personage can be used to track, therefore PCD 100 can suitably be moved and rotated so that wave beam to be kept as speaker moves On the direction of speaker, and other perception mode can strengthen this point, for example, tracked by touching, pass through heat signature Deng.In embodiment, sound positioning and the integrated of visual cues can be used for for example determining face movement to determine by vision Which people attempts to speak to PCD 100.In embodiment, omnidirectional's " low resolution " vision system can also be disposed to detect room In motion, the camera of higher quality is then directed to speaker.
In other one exemplary embodiments, method disclosed herein and system can use tiling grammer as phrase chunking The part of technology.In order to carry out effective phrase chunking, can preferably have a short phrase, but according to must recognize how much Different phrases, the cost for setting up phrase chunking is higher.In order to be distinguished for example between ten contents, you have different different phrases More, it is higher that it becomes cost(Geometrically).In embodiment, method disclosed herein and system can resolve into phrase The different identifiers of operation, therefore be each small relatively low with cost simultaneously in different threads.It can introduce now A series of thing, because the concept of phrase chunking allows you to find the substantial block of speech of tool.For example, with phrase " he Buddy, I wants to take pictures and is sent to my younger sister ".In most cases, two blocks may be critically important:" taking pictures " and " it is sent to mine Younger sister ".According to a phrase chunking thread, another can be triggered, the phrase chunking identifier of modification.Identifier can be built Figure(The not exclusively figure of grammer, but actual identifier), the certain types of phrase of each identifier identification.Based on this Figure, identifier can be triggered by the appropriate female identifier for controlling its applicability and using.Therefore, there is provided herein with parallel Work multiple speech recognition devices automatic speech recognition system, institute's speech recognizer optionally according to figure arrangement with allow across The more phrase chunking of the phrase of wide scope.
Method described herein and system can be disposed by machine part or integrally, and the machine performs the meter on processor Calculation machine software, program code and/or instruction.Processor can be server, client, network infrastructure, mobile computing put down The part of platform, fixed calculating platform or other calculating platforms.Processor can be any kind of calculating or processing unit, its energy Enough execute program instructions, code, binary command and the like.Processor can be or including signal processor, digital processing Device, embeded processor, microprocessor or such as coprocessor(Math co-processor, graphics coprocessor, communication co-processor And the like)Etc. any version and the like, it can directly or indirectly promote the program code being stored thereon or program The execution of instruction.In addition, processor can realize the execution of multiple programs, thread and code.Thread can be performed simultaneously to strengthen The performance of processor and operate while promote application.By realizing, method described herein, program code, programmed instruction And the like can be realized in one or more threads.Thread can cause other threads, can distribute it associated with them excellent First level;Processor can be based on priority or based on the instruction provided in program code any other sequentially perform these lines Journey.Processor may include memory, and it is stored such as method, code, instruction and the program herein and described in other places.Processor can lead to Cross interface and carry out access storage media, it can be stored such as method, code and the instruction herein and described in other places.With relational processor For storage method, program, code, programmed instruction or other kinds of instruction(It can be calculated or processing unit is performed)Deposit Storage media may include but be not limited to CD-ROM, DVD, memory, hard disk, flash drive, RAM, ROM, cache and similar One or more of thing.
Processor may include one or more cores, and it can strengthen the speed and performance of multiprocessor.In embodiment, processing Device can be the two or more individual cores of combination(It is called chip)Dual core processor, four core processors, other chip-scale many places Manage device and the like.
Method described herein and system can by server, client, fire wall, gateway, hub, router or The machine part of computer software is performed on other such computers and/or networking hardware or is integrally disposed.Software program It can be associated with server, it may include file server, printing server, domain server, Internet Server, intranet service Device and other versions such as secondary server, master server, distributed server and the like.Server may include storage Device, processor, computer-readable medium, storage medium, port(Physics and virtual), communicator and can by wired or One or more of wireless medium access other servers, client, interface of machine and device and the like.As herein and The method of other places description, journey logic bomb can be performed by server.In addition, needed for the execution of method as described in this application Other devices wanted can be considered as a part for the infrastructure associated with server.
Server can provide the interface of other devices, its unrestrictedly include client, other servers, printer, Database server, printing server, file server, the communication server, distributed server and the like.In addition, the coupling Conjunction and/or connection can promote the long-range execution of program across a network.Some or all of networking in these devices can promote program Or method in one or more positions parallel processing without departing from scope.In addition, being attached to the device of server by interface In any one may include being capable of storage method, program, code and/or at least one storage medium of instruction.Central repository The programmed instruction to perform on differing devices can be provided.In this implementation, remote repositories may act as program code, instruction and The storage medium of program.
Software program can be with client associate, and it may include file client, Printing, domain client, internet Client, intranet client and other change shapes such as secondary client, primary client, distributed clients and the like Formula.Client may include memory, processor, computer-readable medium, storage medium, port(Physics and virtual), communication Device and can be by interface of wired or wireless medium access other clients, server, machine and device and the like It is one or more.Method, journey logic bomb as described in this paper and other places can be by client executings.In addition, such as in this application Described in method execution required for other devices can be considered a part with the infrastructure of client associate.
Client can provide the interface of other devices, its unrestrictedly include server, other clients, printer, Database server, printing server, file server, the communication server, distributed server and the like.In addition, the coupling Conjunction and/or connection can promote the long-range execution of program across a network.Some or all of networking in these devices can promote program Or method in one or more position parallel processings without departing from scope.In addition, being attached to by interface in the device of client Any one may include being capable of storage method, program, application program, code and/or at least one storage medium of instruction.Center Repository can provide the programmed instruction to perform on differing devices.In this implementation, remote repositories may act as program code, Instruction and the storage medium of program.
Method described herein and system can pass through network infrastructure portion or integral deployment.Network infrastructure can be wrapped Include such as computing device, server, router, hub, fire wall, client, personal computer, communicator, route dress Put and the element such as other active and passive device, module and/or parts as known in the art.Associated with network infrastructure Calculating and/or non-computational device may include in addition to miscellaneous part such as flash memory, buffer, storehouse, RAM, ROM and the like Storage medium.Herein and the process that describes of other places, method, program code, instruction can be by one in network infrastructure element Or multiple execution.
Method, program code and the instruction described herein with other places can be realized on the cellular network with multiple cells. Cellular network can be frequency division multiple access(FDMA)Network or CDMA(CDMA)Network.Cellular network may include mobile device, Cell site, base station, repeater, antenna, tower and the like.Subzone network can be GSM, GPRS, 3G, EVDO, mesh network Or other network types.
Herein and other places describe method, program code and instruction can on the mobile device or pass through its realize.Mobile dress Put may include guider, cell phone, mobile phone, mobile personal digital assistant, portable computer, palmtop computer, on Net sheet, pager, E-book reader, music player and the like.These devices may also include for example in addition to miscellaneous part The storage medium such as flash memory, buffer, RAM, ROM and one or more computing devices.Can fill the calculating associated with mobile device Put program code, method and the instruction for being able to carry out being stored thereon.Alternatively, mobile device can be configured to closes with other devices Make execute instruction.Mobile device can couple and be configured to the base station communication of configuration processor code with server.Mobile device It can be communicated on peer-to-peer network, mesh network or other communication networks.Program code is storable in the storage associated with server Computing device on medium and in embedded server is performed.Base station may include computing device and storage medium.Storage device It can store by the program code and instruction with the execution of the computing device of base station association.
Computer software, program code and/or instruction are storable on machine readable media or accessed thereon, and it can be wrapped Include:It is preserved for machine element, device and the recording medium of the persistently numerical data that some time intervals are calculated;It is referred to as random Access memory(RAM)Semiconductor memory;The mass storage more permanently stored, such as CD are typically used in, as Hard disk, tape, magnetic drum, the form of magnetic card and other kinds of magnetic storage;Processor register, cache memory, Volatile memory, nonvolatile memory;The optical memory such as CD, DVD;Such as flash memory(Such as USB rods or key)、 Floppy disk, tape, paper tape, card punch, independent ram disc, zip disk drive, removable mass storage, offline and the like Etc. removable medium;For example dynamic memory, static memory, read/writable memory device, alterable memory, read-only, random access, Sequential access, addressable point, addressable file, addressable content, Network Attached Storage, storage area networks, bar code, magnetic Property ink and the like wait other computer storages.
Method described herein and system can by physics and/or invisibles from a state transformation to another.Herein The method and system of description will can also represent the data of physics and/or invisibles from a state transformation to another.
The element for being described herein and describing that flow chart and block diagram in whole accompanying drawing include is implied between element Logic boundary.However, being put into practice according to software or hardware engineering, the element of description and its function can perform medium by computer In the machine of the processor with the programmed instruction for being able to carry out being stored thereon(As monolithic software configuration, as independent soft Part module or be taken as external routine, code, service etc., or any combination of these module)It is upper to realize, and own Such realization can be in the scope of the present disclosure.The example of such machine may include but can be not limited to personal digital assistant, just Take formula computer, personal computer, mobile phone, other hand-held computing devices, medical treatment device, wired or wireless communication device, change Can device, chip, calculator, satellite, tablet PC, e-book, gadget, electronic installation, the device with artificial intelligence, calculating dress Put, networked devices, server, router and the like.In addition, the element described in flow chart and block diagram or any other patrol Collecting part can realize on the machine for be able to carry out programmed instruction.So as to although figure and explanation above illustrates disclosed system Function aspects, for realize these function aspects software specific setting should not from these explanation infer, unless above and below Literary clear stipulaties are clearly visible in addition.Similarly, it can be appreciated that above-identified and the various steps of description can change, and walk Rapid order is adaptable to the application-specific of presently disclosed technology.All such changes and modification are intended to fall within the disclosure In the range of.In this way, the order of various steps show and/or illustrate should not be construed require that the specific execution of those steps is suitable Sequence, unless required by application-specific, or from context clear stipulaties or clearly visible in addition.
Method as described above and/or process and its step can be using the hardware, software or hardware for being suitable for application-specific Realized with any combinations of software.Hardware may include all-purpose computer and/or dedicated computing device or particular computing device or spy Determine the particular aspects or part of computing device.Process can be in one or more microprocessors, microcontroller, embedded microcontroller Device, programmable digital signal processor or other programmable devices are realized together with internally and/or externally memory.Process can be also Or replace included in application specific integrated circuit, programmable gate array, programmable logic array or any other can be configured to processing electricity In the combination of the device or device of subsignal.It can be further appreciated that one or more of process can be achieved as can be in machine The computer-executable code performed on computer-readable recording medium.
Structured programming language, the Object-Oriented Programming Language such as C++ such as C can be used in computer-executable code Or any other senior or low level programming language(Including assembler language, hardware description language and database programming language and technology) Create, the language can be stored, compiled or explained to run on one in device above, and in processor, processing The isomery combination or the combination of different hardware and software of device framework are able to carry out running on any other machine of programmed instruction.
So as to, in one aspect, above-described each method and its combination can be included in computer-executable code, The step of code performs its when being performed on one or more computing devices.In another aspect, method, which can be included in, holds Row its step system in, and can use the cross-device distribution of many modes, or it is functional can all be integrated into it is special, independence In device or other hardware.In another aspect, it may include for performing the instrument for the step of being associated with process described above Any one in above-described hardware and/or software.All such arrangements and combination are intended to and fall into the scope of the present disclosure It is interior.
Disclosed while characterized as method and system together with some preferred embodiments for being shown specifically and describing, to it Various modifications and improvements will be become apparent from for those skilled in that art.Therefore, spirit and scope described herein Not by example limitation above, and will it is allowed by law it is most wide in the sense that understand.
With reference to Figure 13, it is illustrated that have exemplary and flow chart and correlation method 1300 of non-limiting example.Method includes Lasting partner device is provided at step 1302(PCD).Method further comprise at step 1304 input from user from By posture, direction of visual lines, select word, sound rule, body gesture, facial expression, emotion clue and the speech for touching the group selection constituted With at least one in non-karst areas signal.Method further comprise at step 1306 adjust PCD behavior come mirror image speech and At least one in non-karst areas signal.
Development platform, storehouse, assets, PCD etc. all above-mentioned attributes can be extended to support other language and culture(This Ground).
All references are incorporated herein by reference.

Claims (45)

1. one kind, which is used to develop, is used for lasting partner device(PCD)Technical ability development platform, including:
Assets development library, with API(API), the API(API)It is configured to enable developers to Carry out it is following at least one:It was found that, create, editor and access can be used for create can be by the one of the PCD technical ability run Individual or multiple content assets;
Representation aids external member, with one or more API, is received and as described in being specified as the developer via the API The associated one or more expression of technical ability, wherein the technical ability can be by the PCD in response to the input that at least one is defined Operation;
Behavior editing machine, one or more behavior sequences for specifying the PCD for the technical ability;And
Technical ability disposes facility, with for the technical ability to be deployed into runtime engine to run the API of the technical ability.
2. development platform as claimed in claim 1, wherein the assets development library and the API of the representation aids external member In at least one include Javascript API.
3. development platform as claimed in claim 1, wherein the behavior editing machine shows described using graphic user interface One or more behavior sequences.
4. development platform as claimed in claim 3, wherein, the multiple order physical locations for representing the PCD by figure are used At least one behavior sequence is shown in the sequence.
5. development platform as claimed in claim 3, wherein being shown by representing the multi-mode expression of the PCD from multiple patterns Show at least one behavior sequence, the multiple pattern is selected from multiple order physical locations, the sound of the PCD by the PCD Sequence, the figure sequence shown by the PCD, sequence and the voice by the PCD by the PCD illuminating effects shown The group of sequence composition.
6. development platform as claimed in claim 3, wherein at least one behavior sequence include control signal to another device, In the input of at least one in software application and the software application developed from another device, for behavior and assets at least One.
7. development platform as claimed in claim 3, wherein specifying one or more of behavior sequences using digraph.
8. development platform as claimed in claim 7, wherein the digraph is recurrence digraph.
9. development platform as claimed in claim 1, is specified in mood, pattern and role wherein the representation aids external member is realized At least one, it is configured to modulate by PCD pairs based at least one in mood, pattern, user property model and role The expression of given technical ability.
10. development platform as claimed in claim 1, in addition to for handling can be captured by the PCD one or more Feel the sensation input processing module of input.
11. development platform as claimed in claim 10, wherein the sensation input module of the PCD has the local of definition Aware space, wherein the PCD is suitable for detect and track and wherein disposes the sensation input of the environmental correclation connection of the PCD.
12. development platform as claimed in claim 1, in addition to it is described for the code and simulation for operating to the skill development The analog module of at least one in PCD local aware space, wherein the analog module and the PCD can run it is identical Code.
13. development platform as claimed in claim 12, wherein, the PCD and the analog module potentially act as mutual defeated Enter source or output target.
14. development platform as claimed in claim 1, wherein the platform is suitable for delivering technical ability to PCD, the PCD is included At least one interface for receiving and running the technical ability.
15. development platform as claimed in claim 1, wherein the input of at least one definition includes numeral input and physics At least one in input.
16. development platform as claimed in claim 1, wherein the runtime engine is deployed on the PCD.
17. development platform as claimed in claim 1, wherein the runtime engine is deployed as a part for the development platform.
18. development platform as claimed in claim 1, wherein the runtime engine is deployed as the behavior for the PCD Simulator a part.
19. development platform as claimed in claim 1, wherein the assets development library includes one or more storehouse assets.
20. development platform as claimed in claim 1, wherein the representation aids external member includes being configured to temporarily coordinating described The timeline editing machine of PCD multi-model expression.
21. development platform as claimed in claim 20, wherein multi-model expression is selected from by voice, sound effect, sound The group that wave filter, physical motion, screen graph and LED illumination effect are constituted.
22. development platform as claimed in claim 1, including real time debugger/editing machine, it is used for supporting to be used for the simulation Device is exported and at least one in the resulting code at least one in the PCD and other assets change The live coding of WYSIWYG viewings.
23. one kind is used to use SDK(SDK)The platform of skill development is realized, including:
Logic level module, is configured to the input of reception being mapped to coding response;And
Level module is perceived, including:
Visual performance module, is configured to detect one or more visual performance events and notifies described to the logic level module One or more visual performance events detected;And
Voice/sound is recognized and Understanding Module, is configured to the sound of detection definition and is notified to detect to the logic level module Voice/sound;And
Engine is expressed, one or more animations of emotion/role state of generation expression definition are configured to, and will be one Or multiple animations are sent to the logic level module.
24. platform as claimed in claim 23, in addition to multimode data are collected and machine learning module, for realizing technical ability With the improvement of the automatic or manual guidance of at least one in the component of technical ability.
25. platform as claimed in claim 23, in addition to role module, the role module, which includes having, is based on mood, society The interactivity based on character with the attribute of task is handed over, it is used for the storage and the number of study that create the role for the PCD Word represent in one.
26. platform as claimed in claim 23, in addition to for the PCD user study and storage attribute at least The user property module of one.
27. platform as claimed in claim 23, wherein the animation is by selected from the physical location sequence by the PCD, institute The sound of stating PCD, the figure shown by the PCD, constitute by the PCD illuminating effects shown and by the voice of the PCD Group effect composition.
28. platform as claimed in claim 23, wherein the expression engine receive be selected from by pre-recorded audio, voice, Text, touch, the output speech of text formatting, animation, structure, the input of data from other devices and application.
29. platform as claimed in claim 23, wherein voice/sound identification and Understanding Module are received via selected under Arrange the input of the process generation of the group of composition:Phrase chunking, non-phrase chunking, non-speech sounds identification, automatic speech recognition, from The application of right language rule, for Data Collection and the new identification of training and understand model or improve the instrument of existing model, use The identification at family and the direction of sound source.
30. platform as claimed in claim 23, wherein the communication behaviour between the logic level module and perception level module Make that the technical ability for being converted to expression with internal event will be perceived.
31. platform as claimed in claim 23, wherein the traffic operation between the logic level module and the sensing module To provide input and receive from interface on another device, web services, Mobile solution, web applications, cloud computing resources, Yun Cun Store up at least one of resource, social media interface and the input of at least one in video conference interface.
32. platform as claimed in claim 23, wherein the communication between the logic level module and perception level module is grasped Make at least one in the role and the behavior to modulate the PCD.
33. platform as claimed in claim 32, wherein the communication is together with the contextual information including user property model Operation.
34. platform as claimed in claim 23, wherein the output speech of pre-recorded audio and text formatting is by text To voice and audio splicing/filter engine processing with the voice component of making of cartoon.
35. platform as claimed in claim 34, wherein the expression engine generation is associated with the voice component of the animation Sound effect.
36. platform as claimed in claim 23, wherein the visual performance module is configured to, execution is selected from by taking pictures, video is remembered Record, face and facial characteristics tracking, people's tracking, Object identifying, the position of the PCD, Emotion identification, gesture identification, gesture with The function of the group of track and face recognition composition.
37. platform as claimed in claim 23, wherein by the type of the PCD phonetic entries received by user via user Interface is defined.
38. platform as claimed in claim 23, in addition to in the technical ability repository to online shop and based on cloud extremely At least one for the PCD that the release module of few issue technical ability, wherein technical ability can be discovery, buy and download to user.
39. platform as claimed in claim 23, wherein one or more of visual performance events include social visual signature With at least one in mood visual signature.
40. platform as claimed in claim 38, wherein the mood visual signature includes depth characteristic, color characteristic, illumination Feature and infrared(IR)At least one in feature.
41. platform as claimed in claim 23, wherein the sound of the definition includes social characteristics and based on mood sound At least one in feature.
42. platform as claimed in claim 23, wherein the communication behaviour between the logic level module and perception level module Make to modulate user property model.
43. platform as claimed in claim 42, wherein, communication is grasped together with the contextual information including the actor model Make.
44. platform as claimed in claim 38, wherein downloading the technical ability from the cloud.
45. platform as claimed in claim 23, wherein input includes being used for the exploitation of behavior tree, the exploitation of storehouse assets and skill development Reusable multi-mode input-output module.
CN201580049757.2A 2014-07-15 2015-07-15 Apparatus and method for providing lasting partner device Pending CN107000210A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US201462024738P true 2014-07-15 2014-07-15
US62/024738 2014-07-15
PCT/US2015/040581 WO2016011159A1 (en) 2014-07-15 2015-07-15 Apparatus and methods for providing a persistent companion device

Publications (1)

Publication Number Publication Date
CN107000210A true CN107000210A (en) 2017-08-01

Family

ID=55079020

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201580049757.2A Pending CN107000210A (en) 2014-07-15 2015-07-15 Apparatus and method for providing lasting partner device

Country Status (2)

Country Link
CN (1) CN107000210A (en)
WO (1) WO2016011159A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107590718A (en) * 2017-08-31 2018-01-16 科大讯飞股份有限公司 Shared skill development platform and method, shared technical ability application platform and method
CN109166070A (en) * 2018-07-27 2019-01-08 深圳微盐传媒科技有限公司 Tool generation method and device in government system
CN109218402A (en) * 2018-08-02 2019-01-15 胡雄风 Hold cloud integrative solution
CN109358856A (en) * 2018-10-12 2019-02-19 四川长虹电器股份有限公司 A kind of voice technical ability dissemination method
CN109448427A (en) * 2018-11-09 2019-03-08 易的物联科技无锡有限公司 A kind of system of the wisdom parking management towards all kinds of parking lots
CN109902834A (en) * 2019-01-28 2019-06-18 北京怡凯智能技术有限公司 A kind of old man's company active interlocution robot of topic driving
CN110035166A (en) * 2019-03-20 2019-07-19 广州美术学院 A kind of interactive device being associated with multiple mobile phone terminals
CN110609620A (en) * 2019-09-05 2019-12-24 深圳追一科技有限公司 Human-computer interaction method and device based on virtual image and electronic equipment
US10740930B2 (en) 2018-11-07 2020-08-11 Love Good Color LLC Systems and methods for color selection and auditing
CN111968648A (en) * 2020-08-27 2020-11-20 北京字节跳动网络技术有限公司 Voice recognition method and device, readable medium and electronic equipment

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150314454A1 (en) 2013-03-15 2015-11-05 JIBO, Inc. Apparatus and methods for providing a persistent companion device
KR20160034243A (en) 2013-03-15 2016-03-29 지보, 인코포레이티드 Apparatus and methods for providing a persistent companion device
US10417346B2 (en) 2016-01-23 2019-09-17 Microsoft Technology Licensing, Llc Tool for facilitating the development of new language understanding scenarios
KR20190079669A (en) 2016-11-10 2019-07-05 워너 브로스. 엔터테인먼트 인크. Social robot with environmental control feature
CN107168221A (en) * 2017-04-11 2017-09-15 北京智能管家科技有限公司 Robot behavior expression and computer-readable medium
CN109635616A (en) * 2017-10-09 2019-04-16 阿里巴巴集团控股有限公司 Interactive approach and equipment

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6038493A (en) * 1996-09-26 2000-03-14 Interval Research Corporation Affect-based robot communication methods and systems
CN100436082C (en) * 2003-08-12 2008-11-26 株式会社国际电气通信基础技术研究所 Communication robot control system
KR20090001725A (en) * 2007-05-14 2009-01-09 주식회사 유진로봇 Hardware simulation method and the system for the robot drive
US20090055019A1 (en) * 2007-05-08 2009-02-26 Massachusetts Institute Of Technology Interactive systems employing robotic companions
CN102448678A (en) * 2009-05-26 2012-05-09 奥尔德巴伦机器人公司 System and method for editing and controlling the behavior of a movable robot
KR101190660B1 (en) * 2012-07-23 2012-10-15 (주) 퓨처로봇 Methods and apparatus of robot control scenario making
US20130066467A1 (en) * 2010-05-12 2013-03-14 Future Robot Co., Ltd. Service scenario editing apparatus for an intelligent robot, method for same, intelligent robot apparatus and service-providing method for an intelligent robot
US20140136302A1 (en) * 2011-05-25 2014-05-15 Se Kyong Song System and method for operating a smart service robot

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6038493A (en) * 1996-09-26 2000-03-14 Interval Research Corporation Affect-based robot communication methods and systems
CN100436082C (en) * 2003-08-12 2008-11-26 株式会社国际电气通信基础技术研究所 Communication robot control system
US20090055019A1 (en) * 2007-05-08 2009-02-26 Massachusetts Institute Of Technology Interactive systems employing robotic companions
KR20090001725A (en) * 2007-05-14 2009-01-09 주식회사 유진로봇 Hardware simulation method and the system for the robot drive
CN102448678A (en) * 2009-05-26 2012-05-09 奥尔德巴伦机器人公司 System and method for editing and controlling the behavior of a movable robot
US20130066467A1 (en) * 2010-05-12 2013-03-14 Future Robot Co., Ltd. Service scenario editing apparatus for an intelligent robot, method for same, intelligent robot apparatus and service-providing method for an intelligent robot
US20140136302A1 (en) * 2011-05-25 2014-05-15 Se Kyong Song System and method for operating a smart service robot
KR101190660B1 (en) * 2012-07-23 2012-10-15 (주) 퓨처로봇 Methods and apparatus of robot control scenario making

Cited By (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107590718A (en) * 2017-08-31 2018-01-16 科大讯飞股份有限公司 Shared skill development platform and method, shared technical ability application platform and method
CN109166070A (en) * 2018-07-27 2019-01-08 深圳微盐传媒科技有限公司 Tool generation method and device in government system
CN109166070B (en) * 2018-07-27 2021-07-20 深圳微盐传媒科技有限公司 Tool generation method and device in government affair system
CN109218402A (en) * 2018-08-02 2019-01-15 胡雄风 Hold cloud integrative solution
CN109358856A (en) * 2018-10-12 2019-02-19 四川长虹电器股份有限公司 A kind of voice technical ability dissemination method
CN109358856B (en) * 2018-10-12 2020-12-15 四川长虹电器股份有限公司 Voice skill publishing method
US10930027B2 (en) 2018-11-07 2021-02-23 Love Good Color LLC Systems and methods for color selection and auditing
US10740930B2 (en) 2018-11-07 2020-08-11 Love Good Color LLC Systems and methods for color selection and auditing
CN109448427A (en) * 2018-11-09 2019-03-08 易的物联科技无锡有限公司 A kind of system of the wisdom parking management towards all kinds of parking lots
CN109902834A (en) * 2019-01-28 2019-06-18 北京怡凯智能技术有限公司 A kind of old man's company active interlocution robot of topic driving
CN109902834B (en) * 2019-01-28 2021-02-05 北京怡凯智能技术有限公司 Topic-driven robot for active conversation accompanying old people
CN110035166A (en) * 2019-03-20 2019-07-19 广州美术学院 A kind of interactive device being associated with multiple mobile phone terminals
CN110609620A (en) * 2019-09-05 2019-12-24 深圳追一科技有限公司 Human-computer interaction method and device based on virtual image and electronic equipment
CN111968648A (en) * 2020-08-27 2020-11-20 北京字节跳动网络技术有限公司 Voice recognition method and device, readable medium and electronic equipment

Also Published As

Publication number Publication date
WO2016011159A9 (en) 2016-09-15
WO2016011159A1 (en) 2016-01-21

Similar Documents

Publication Publication Date Title
US10357881B2 (en) Multi-segment social robot
US20170206064A1 (en) Persistent companion device configuration and deployment platform
CN107000210A (en) Apparatus and method for providing lasting partner device
AU2014236686B2 (en) Apparatus and methods for providing a persistent companion device
JP2019521449A (en) Persistent Companion Device Configuration and Deployment Platform
US20170221483A1 (en) Electronic personal interactive device
JP2019523714A (en) Multi-interaction personality robot
US20150298315A1 (en) Methods and systems to facilitate child development through therapeutic robotics
TWI692717B (en) Image display device, topic selection method and program
Burden et al. Virtual humans: Today and tomorrow
Stefanidi et al. ParlAmI: a multimodal approach for programming intelligent environments
Calvo et al. Introduction to affective computing
Bonarini Communication in Human-Robot Interaction
Coursey et al. Living with harmony: a personal companion system by Realbotix™
Hanke et al. Embodied Ambient Intelligent Systems.
Smith Ok, google: designing information architecture for smart speakers
KR102306624B1 (en) Persistent companion device configuration and deployment platform
WO2018183812A1 (en) Persistent companion device configuration and deployment platform
US20210043208A1 (en) Generating response in conversation
WO2021007546A1 (en) Computing devices and systems for sending and receiving voice interactive digital gifts
US20200197818A1 (en) Development of virtual character in a learning game
EP3514783A1 (en) Contextual language learning device, system and method
Riccio Sophia Robot: An Emergent Ethnography
Rahman Creating a Sense of Presence in Remote Relationships: A concept of Calm Ambient artifact
WO2017152167A1 (en) Interactive toy device, and systems and methods of communication between the same and network devices

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1241803

Country of ref document: HK

WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20170801

WD01 Invention patent application deemed withdrawn after publication