CN109176535A - Exchange method and system based on intelligent robot - Google Patents
Exchange method and system based on intelligent robot Download PDFInfo
- Publication number
- CN109176535A CN109176535A CN201810777646.9A CN201810777646A CN109176535A CN 109176535 A CN109176535 A CN 109176535A CN 201810777646 A CN201810777646 A CN 201810777646A CN 109176535 A CN109176535 A CN 109176535A
- Authority
- CN
- China
- Prior art keywords
- user
- parameter
- situation
- modal
- intelligent robot
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
- B25J11/0005—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Manipulator (AREA)
Abstract
The present invention provides the exchange method based on intelligent robot, and it includes following steps: in the case where intelligent robot is in activation, obtaining multi-modal input data；It extracts in multi-modal input data, the situation of presence parameter in current scene；Situation of presence parameter is carried out to merge the dynamic life locus spectra to form user with historical circumstances parameter；Multi-modal interactive output data is generated based on dynamic life locus spectra.The present invention provides a kind of intelligent robot, intelligent robot has default image and preset attribute, can carry out multi-modal interaction with user.And, the present invention can also efficiently use the life map in the time generated in user's daily life, personage and place etc., significant topic recommendation is effectively expanded or carried out to user to interaction topic according to abundant develop and utilize to life map, interaction content is enriched, interactive accuracy is improved.
Description
Technical field
The present invention relates to artificial intelligence fields, specifically, being related to a kind of exchange method based on intelligent robot and being
System.
Background technique
The exploitation of robot multi-modal interactive system is dedicated to imitating human conversation, to attempt to imitate people between context
Interaction between class.But at present for, the exploitation of robot multi-modal interactive system relevant for intelligent robot is also not
It is too perfect, not yet occur carrying out the intelligent robot of multi-modal interaction, it is even more important that there is no for user in daily life
In the interactive product develop and useedd of the life map correlation such as generated time, personage and place.
Therefore, the present invention provides a kind of exchange method and system based on intelligent robot.
Summary of the invention
To solve the above problems, the present invention provides a kind of exchange method based on intelligent robot, the method includes
Following steps:
In the case where intelligent robot is in activation, multi-modal input data is obtained；
It extracts in the multi-modal input data, the situation of presence parameter in current scene；
The situation of presence parameter is carried out to merge the dynamic life locus spectra to form user with historical circumstances parameter；
Multi-modal interactive output data is generated based on the dynamic life locus spectra.
According to one embodiment of present invention, it extracts in the multi-modal input data, the situation of presence in current scene
The step of parameter, includes:
The geographical location for positioning current scene, determines the location parameters of current scene；
It captures and identifies the personage within the scope of current scene, determine the face of current scene；
The temporal information for obtaining current scene, determines the time parameter of current scene；
It is associated with the location parameters, personage's parameter and the time parameter, and is recorded as the current of current scene
Scene parameter.
According to one embodiment of present invention, the situation of presence parameter is carried out merging formation use with historical circumstances parameter
Include in the step of dynamic life locus spectra at family:
The parameter attribute of situation of presence parameter is judged, determines the classification of situation of presence parameter；
According to the classification of situation of presence parameter, situation of presence parameter is merged with historical circumstances parameter, forms user
Dynamic life locus spectra.
According to one embodiment of present invention, multi-modal interactive output data is generated based on the dynamic life locus spectra
The step of in include:
Determine the interaction topic of user Yu the intelligent robot；
The dynamic life locus spectra of traverse user is searched and interactive topic relevant user's life track；
It is multi-modal according to being generated to the interactive relevant user life track of topic and the multi-modal input data
Interaction output data, or update user life track.
According to one embodiment of present invention, in the step of determining interaction topic of the user with the intelligent robot, packet
Contain:
The multi-modal input data is parsed, the interaction for obtaining user is intended to, and decision exports friendship corresponding with the intention
Mutual topic；
Or,
Situation of presence parameter is parsed, situation of presence parameter is analyzed, decision interaction topic.
According to one embodiment of present invention, it carries out merging formation with historical circumstances parameter by the situation of presence parameter
Before the step of dynamic life locus spectra of user, also include:
The identity characteristic information for obtaining active user, judges the user property of active user, determines active user
Classification, wherein the classification of user includes: child user.
According to one embodiment of present invention, described when the user interacted with the intelligent robot is child user
Method also includes:
Determine the interaction topic for being suitble to children's interaction to be intended to；
Or,
Situation of presence parameter is parsed, situation of presence parameter and children's face parameter are analyzed, decision interaction topic.
According to one embodiment of present invention, when the user interacted with the intelligent robot includes child user, base
Include in the step of dynamic life locus spectra generates multi-modal interactive output data:
The multi-modal interactive output data is screened, the multi-modal interactive output data for being not suitable for child user is rejected.
According to another aspect of the present invention, a kind of program product is additionally provided, it includes any one of as above for executing
The series of instructions of the method and step.
According to another aspect of the present invention, a kind of interactive system based on intelligent robot, the system are additionally provided
Include:
Intelligent terminal, it includes camera, positioning device and time devices, for obtaining the situation of presence of current scene
Parameter and multi-modal input data, and have voice, emotion, the ability of expression and movement output；
Intelligent robot is mounted on the intelligent terminal, is carried out using as above described in any item methods multi-modal
Interaction；
Cloud brain, be used to carry out the multi-modal input data semantic understanding, visual identity, cognition calculate and
Affection computation exports multi-modal interactive output data with intelligent robot described in decision.
Exchange method and system provided by the invention based on intelligent robot provides a kind of intelligent robot, intelligent machine
Device people has default image and preset attribute, can carry out multi-modal interaction with user.Also, the present invention can also be effectively sharp
With the life map in time, personage and the place generated in user's daily life etc., according to the abundant exploitation to life map
With using effectively being expanded to interaction topic or carrying out significant topic recommendation to user, interaction content is enriched,
Improve interactive accuracy.
Other features and advantages of the present invention will be illustrated in the following description, also, partly becomes from specification
It obtains it is clear that understand through the implementation of the invention.The objectives and other advantages of the invention can be by specification, right
Specifically noted structure is achieved and obtained in claim and attached drawing.
Detailed description of the invention
Attached drawing is used to provide further understanding of the present invention, and constitutes part of specification, with reality of the invention
It applies example and is used together to explain the present invention, be not construed as limiting the invention.In the accompanying drawings:
Fig. 1 shows the interaction signal of the interactive system according to an embodiment of the invention based on intelligent robot
Figure；
Fig. 2 shows the structural block diagram of the interactive system according to an embodiment of the invention based on intelligent robot；
Fig. 3 shows the flow chart of the exchange method according to an embodiment of the invention based on intelligent robot；
Fig. 4 shows that the exchange method according to an embodiment of the invention based on intelligent robot extracts the situation of presence
The flow chart of parameter；
Fig. 5 shows that the exchange method according to an embodiment of the invention based on intelligent robot generates multi-modal friendship
The flow chart of mutual output data；
Fig. 6 shows another process of the exchange method according to an embodiment of the invention based on intelligent robot
Figure；And
Fig. 7 show it is according to an embodiment of the invention user, intelligent terminal and cloud brain between the parties
The flow chart communicated.
Specific embodiment
To make the object, technical solutions and advantages of the present invention clearer, the embodiment of the present invention is made below in conjunction with attached drawing
Further it is described in detail.
It is clear to state, it needs to carry out before embodiment as described below:
The intelligent robot that the present invention mentions can carry out multi-modal interaction with user.
Intelligent terminal includes camera, positioning device and time device, and the situation of presence for obtaining current scene is joined
Several and multi-modal input data, and have voice, emotion, the ability of expression and movement output；
Intelligent robot obtains multi-modal input data based on the hardware of the intelligent terminal, beyond the clouds the ability branch of brain
It holds down, semantic understanding, visual identity, cognition calculating, affection computation is carried out to multi-modal input data, to complete decision output
Process.
The cloud brain being previously mentioned is to provide the intelligent robot to carry out semantic understanding (language to the interaction demand of user
Semantic understanding, Action Semantic understanding, visual identity, affection computation, cognition calculate) processing capacity terminal, realize and user
Interaction, with the multi-modal interactive output data of the output of intelligent robot described in decision.
Each embodiment of the invention is described in detail with reference to the accompanying drawing.
Fig. 1 shows the interaction signal of the interactive system according to an embodiment of the invention based on intelligent robot
Figure.As shown in Figure 1, carrying out multi-modal interaction needs user 101, intelligent terminal 102, intelligent robot 103 and cloud brain
104.Wherein, the user 101 interacted with intelligent robot can be true people and another intelligent robot, another intelligent machine
Device people is similar with the interactive process of intelligent robot with the interactive process of intelligent robot with single people.Therefore, in Fig. 1 only
Show the multi-modal interactive process of user (people) Yu intelligent robot.
In addition, intelligent terminal 102 includes 1022 (substantially core processing of display area 1021 and hardware supported equipment
Device).Display area 1021 is used to show the image of intelligent robot 103, and hardware supported equipment 1022 and cloud brain 104 cooperate
It uses, for the data processing in interactive process.
The process interacted between intelligent robot and user 101 in Fig. 1 are as follows:
Interaction required early-stage preparations or condition have, and intelligent robot has natural language understanding, visual perception, touch
The AI abilities such as perception, language output, emotional facial expressions movement output.According to one embodiment of present invention, interactive in order to be promoted
Experience, intelligent robot can be indicated in predeterminable area after being activated.
Fig. 2 shows the structural frames of the interactive system based on intelligent robot according to another embodiment of the invention
Figure.As shown in Fig. 2, completing interaction needs user 101, intelligent terminal 102 and cloud brain 104.Wherein, intelligent terminal 102
Include man-machine interface 201, data processing unit 202, input/output unit 203 and interface unit 204.Cloud brain 104 wraps
Interface containing semantic understanding 1041, visual identity interface 1042, cognition calculate interface 1043 and affection computation interface 1044.
Interactive system provided by the invention based on intelligent robot includes intelligent terminal 102 and cloud brain 104.Intelligence
Energy robot 103 is run in intelligent terminal 102, and intelligent robot 103 has default image and preset attribute, in friendship
It can star voice, emotion, vision and sensing capability when mutual state.
In one embodiment, intelligent terminal 102 may include: man-machine interface 201, data processing unit 202, input it is defeated
Device 203 and interface unit 204 out.Wherein, man-machine interface 201 is shown in the predeterminable area of intelligent terminal 102 in fortune
The intelligent robot 103 of row state.
Data processing unit 202 generate in multi-modal interactive process with intelligent robot 103 for handling user 101
Data.Processor used can be data processing unit (Central Processing Unit, CPU), can also be it
His general processor, digital signal processor (Digital Signal Processor, DSP), specific integrated circuit
(Application Specific Integrated Circuit, ASIC), ready-made programmable gate array (Field-
Programmable Gate Array, FPGA) either other programmable logic device, discrete gate or transistor logic,
Discrete hardware components etc..General processor can be microprocessor or the processor is also possible to any conventional processor
Deng processor is the control centre of terminal, utilizes the various pieces of various interfaces and the entire terminal of connection.
It include memory in intelligent terminal 102, memory mainly includes storing program area and storage data area, wherein is deposited
Store up program area can application program needed for storage program area, at least one function (for example sound-playing function, image play function
Energy is equal) etc.；Storage data area can store according to intelligent terminal 102 use created data (such as audio data, browsing note
Record etc.) etc..In addition, memory may include high-speed random access memory, it can also include nonvolatile memory, such as firmly
Disk, memory, plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital,
SD) block, flash card (Flash Card), at least one disk memory, flush memory device or other volatile solid-states
Part.
Input/output unit 203 is used to obtain multi-modal interaction data and exports the output data in interactive process.It connects
Mouthful unit 204 is used to communicate with the expansion of cloud brain 104, and by with the interface in cloud brain 104, to fetching, to transfer cloud big
Intelligent robot ability in brain 104.The example of input/output unit includes the microphone for voice operating, scanner, takes the photograph
As head (not being related to the movement touched using the detection of visible or nonvisible wavelength) etc..Intelligent terminal 102 can be by mentioning above
To input equipment obtain multi-modal input data.
Cloud brain 104 include semantic understanding interface 1041, visual identity interface 1042, cognition calculate interface 1043 and
Affection computation interface 1044.The above interface is communicated with the expansion of interface unit 204 in intelligent terminal 102.Also, cloud is big
Brain 104 also includes and the corresponding semantic understanding logic of semantic understanding interface 1041, vision corresponding with visual identity interface 1042
Recognition logic and cognition calculate the corresponding cognition calculating logic of interface 1043 and emotion corresponding with affection computation interface 1044
Calculating logic.
As shown in Fig. 2, each ability interface calls corresponding logical process respectively in multi-modal data resolving.Below
For the explanation of each interface:
Semantic understanding interface 1041 receives the special sound instruction forwarded from interface unit 204, carries out voice knowledge to it
The other and natural language processing based on a large amount of corpus.
Visual identity interface 1042 can be calculated for human body, face, scene according to computer vision algorithms make, deep learning
Method etc. carries out video content detection, identification, tracking etc..Image is identified according to scheduled algorithm, the inspection of quantitative
Survey result.Have image preprocessing function, feature extraction functions, decision making function and concrete application function；
Wherein, image preprocessing function, which can be, carries out basic handling, including color sky to the vision collecting data of acquisition
Between conversion, edge extracting, image transformation and image threshold；
Feature extraction functions can extract the features such as the colour of skin of target, color, texture, movement and coordinate in image and believe
Breath；
Decision making function can be to characteristic information, is distributed to according to certain decision strategy and needs the specific of this feature information
Multi-modal output equipment or multi-modal output application, such as realize Face datection, human limbs identification, motion detection function.
Cognition calculates interface 1043, receives the multi-modal data forwarded from interface unit 204, and cognition calculates interface 1043
Data acquisition, identification and study are carried out to handle multi-modal data, to obtain user's portrait, knowledge mapping etc., to multimode
State output data carries out Rational Decision.
Affection computation interface 1044 receives the multi-modal data forwarded from interface unit 204, utilizes affection computation logic
(can be Emotion identification technology) calculates the current emotional state of user.Emotion identification technology is that one of affection computation is important
Component part, the content of Emotion identification research include facial expression, voice, behavior, text and physiological signal identification etc., are led to
Crossing the above content may determine that the emotional state of user.Emotion identification technology can be monitored only by vision Emotion identification technology
The emotional state of user can also monitor use in conjunction with by the way of using vision Emotion identification technology and sound Emotion identification technology
The emotional state at family, and be not limited thereto.In the present embodiment, it is preferred to use the two in conjunction with mode monitor mood.
Affection computation interface 1044 is to collect mankind face by using image capture device when carrying out vision Emotion identification
Portion's facial expression image is then converted into that data can be analyzed, the technologies such as image procossing is recycled to carry out the analysis of expression mood.Understand face
Expression, it usually needs the delicate variation of expression is detected, such as cheek muscle, mouth variation and choose eyebrow etc..
In addition, the interactive system provided by the invention based on intelligent robot can also cooperate a kind of program product, packet
Containing for executing the series of instructions for completing the exchange method step of intelligent robot.Program product can run computer and refer to
Enable, computer instruction includes computer program code, computer program code can for source code form, object identification code form,
Executable file or certain intermediate forms etc..
Program product may include: can carry computer program code any entity or device, recording medium, USB flash disk,
Mobile hard disk, magnetic disk, CD, computer storage, read-only memory (ROM, Read-Only Memory), random access memory
Device (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium etc..
It should be noted that the content that program product includes can be according to making laws in jurisdiction and patent practice is wanted
It asks and carries out increase and decrease appropriate, such as do not include electric carrier wave according to legislation and patent practice, program product in certain jurisdictions
Signal and telecommunication signal.
Fig. 3 shows the flow chart of the exchange method according to an embodiment of the invention based on intelligent robot.
In step S301, in the case where intelligent robot is in activation, multi-modal input data is obtained.Activation intelligence
There are many kinds of the modes of robot, in one embodiment, intelligent robot can be activated by way of key, can also led to
The mode for crossing voice wake-up activates intelligent robot, other can activate the mode of intelligent robot that can apply to the present invention
In, the present invention restricts not to this.
After intelligent robot activation, the multi-modal input data on periphery is obtained.The multi-modal input data on periphery includes
The data of user's input also include the data of surrounding enviroment.Multi-modal input data can be voice data, image/video data
And perception data etc..Configured with the related device for obtaining multi-modal input data on intelligent terminal 102.In one embodiment
In, multi-modal input data can be user's expression, voice data, gesture data, image data, video data, human face data,
Pupil iris information, light sensation information, finger print information and environmental information.
Then, in step s 302, extract in multi-modal input data, the situation of presence parameter in current scene.For
For the present invention, need to extract the situation of presence parameter in multi-modal input data.Fig. 4 shows a reality according to the present invention
The exchange method based on intelligent robot for applying example extracts the flow chart of situation of presence parameter.
As shown in figure 4, positioning the geographical location of current scene in step S401, the location parameters of current scene are determined.
There is positioning device in intelligent terminal 102, the geographical location of current scene can be positioned.The location information navigated to is recorded as working as
The location parameters of preceding scene.
Meanwhile in step S402, captures and identify the personage within the scope of current scene, determine the face of current scene.
In this step, it needs the camera on intelligent terminal 102 to capture the personage within the scope of current scene, the personage captured is believed
Breath compares screening to determine the face of current scene.
Meanwhile in step S403, the temporal information of current scene is obtained, determines the time parameter of current scene.Intelligence
Time device in terminal 102 can obtain the temporal information of current scene in real time, be recorded as the time parameter of current scene.
Finally, in step s 404, association location parameters, face and time parameter, and it is recorded as working as current scene
Preceding scene parameter.Location parameters, face and time parameter under current scene is associated, working as current scene is recorded as
Preceding scene parameter.
According to one embodiment of present invention, it after extracting the situation of presence parameter of current scene, needs to these feelings
Scape parameter is to arranging.Location parameters, face and time parameter are respectively divided into three groups, are amusement class parameter, work respectively
Make class parameter and family's class parameter.Location parameters include amusement class place, work class place and family's class place.Face packet
The class face containing amusement, work class face and family's class face.Time parameter includes amusement class time, work class time and family
Front yard class event.
In one embodiment, morning 9 point to 6 pm of the current time if it is Mon-Fri, current time ginseng
Number can be identified as the work class time.If current face is colleague, work class face can be identified as.If current
Place is unit mansion, then can be identified as work class place.
The standard for distinguishing amusement class parameter, work class parameter and family's class parameter can preset change, Ke Yigen
Depending on the lifestyle habits of user.In addition, when user is child user, by location parameters, face and time parameter
It is divided into three groups, respectively amusement class parameter, family's class parameter and study class parameter.
In one embodiment, morning 8 point to afternoon 4 point of the current time if it is Mon-Fri, current time ginseng
Number can be identified as learning the class time.If current face is classmate, can be identified as learning class face.If current
Place is school, then can be identified as learning class place.
Then, in step S303, situation of presence parameter is carried out to merge the dynamic to form user with historical circumstances parameter
Life locus spectra.Before this step, according to one embodiment of present invention, need to distinguish the identity of user.It obtains
The identity characteristic information for taking active user judges the user property of active user, determines the classification of active user,
In, the classification of user includes: child user.
After determining the identity type of user, need to merge current scene parameter with historical context parameter.?
Before fusion, need to identify the classification of situation of presence parameter.When user is general category adult user, identification situation of presence parameter is
Belong to amusement class, work class or family's class.When user is child user, identification situation of presence parameter is to belong to amusement class, learn
Practise class or family's class.Then the scene parameter after identification is merged with generic historical circumstances parameter, is generated new
User's life locus spectra under current class.
Finally, in step s 304, generating multi-modal interactive output data based on dynamic life locus spectra.User's is dynamic
Include the daily behavior of user in state life map, foundation can be provided to multi-modal interactive output data is generated.Fig. 5 is shown
Exchange method according to an embodiment of the invention based on intelligent robot generates the process of multi-modal interactive output data
Figure.
In step S501, the interaction topic of user and intelligent robot is determined.In one embodiment, solution can be passed through
Multi-modal input data is analysed, is intended to obtain the interaction of user, decision exports interactive topic corresponding with intention.Also it can parse
Situation of presence parameter analyzes situation of presence parameter, decision interaction topic.
According to one embodiment of present invention, it when the user interacted with intelligent robot is child user, determines and is suitble to
The interaction topic that children's interaction is intended to.Alternatively, parsing situation of presence parameter, to situation of presence parameter and children's face parameter into
Row analysis, decision interaction topic.
Then, in step S502, the dynamic life locus spectra of traverse user searches user relevant to interaction topic
Life track.
Finally, in step S503, according to user life track relevant to interaction topic and multi-modal input data
Multi-modal interactive output data is generated, or updates user life track.According to one embodiment of present invention, when with intelligent machine
When the user of device people interaction includes child user, multi-modal interactive output data is screened, rejects the multimode for being not suitable for child user
State interacts output data.
The interaction of user and intelligent robot is introduced below by an example:
User: I once goes to Shanxi and Shaanxi to travel, and leaves deep impression to the landscape of there.It is also exactly to eat
Thing it is also very much, especially wheaten food.
Intelligent robot: pair, the tourist resources in Shaanxi and Shanxi or very rich has many scenic spots and historical sites.
User: wheaten food carrys out word, and Shanxi and Shaanxi wheaten food type are abundant, and the mouthfeel of wheaten food is also very good.
Intelligent robot: yes, there are many wheaten food type in Shanxi, have plande noodles, hand-pulled noodles, seize pimple, push away nest nest etc..Shaanxi
Wheaten food have noodles with minced pork and winter bamboo shoots, the face biangbiang, the bold and vigorous face of oil, dip in the water surface etc..
User: right, I ate the bold and vigorous face of oil, especially nice.
Intelligent robot: having arrived the time having supper now, otherwise the face Na Jia that you and friend A were gone before going
Eat face in shop.
User:, I am about once friend A.
In the above dialogue, the experience traveled before is looked back in the place that user travels before having lifted first,
Landscape and wheaten food are referred to, the place Shaanxi and Shanxi that user and intelligent robot are traveled according to user immediately expand dialogue.
User refers to the wheaten food eaten, and according to the current time and before, the life track recommended user of user goes to eat intelligent robot
Face is eaten in shop, and user has promised the suggestion of intelligent robot.
In addition, intelligent robot can also be according to currently interactive place, knot in the interaction of intelligent robot and user
It closes the current time, actively proposes the topic of interaction, interacted with user's expansion.For example, active user is children, current time is
Summer noon, intelligent robot can be unfolded topic around ice cream with child user, interact with child user.
In one embodiment, when current interactive object is child user, the interaction words that suitable children's interaction is intended to are determined
Topic.Or parsing situation of presence parameter, situation of presence parameter and children's face parameter are analyzed, decision interaction topic.
When the user interacted with intelligent robot includes child user, multi-modal interactive output data is screened, rejects and is not suitable for children
The multi-modal interactive output data of user.
For child user, intelligent robot also includes study dialogue mode, can according to the study schedule before user,
The study schedule of supervisory user, consumer-oriented study.For example, intelligent robot can transfer before user study mistake topic note
Guidance is unfolded in record, the study current to user.
It should be noted that with the interaction of user there are many more mode, all users live relevant activity can be by
Record is the life locus spectra of user, is applied in interaction, the present invention restricts not to this.
Fig. 6 shows another process of the exchange method according to an embodiment of the invention based on intelligent robot
Figure.
As shown in fig. 6, in step s 601, intelligent terminal 102 is issued to cloud brain 104 and is requested.Later, in step
In S602, intelligent terminal 102 is constantly in the state for waiting cloud brain 104 to reply.During waiting, intelligent terminal
102 can carry out Clocked operation to returned data the time it takes.
In step S603, if the reply data not returned for a long time, for example, being more than scheduled time span
5S, then intelligent terminal 102 can select to carry out local reply, generate local common reply data.Then, in step s 604, defeated
The animation cooperated out with local common response, and voice playing equipment is called to carry out voice broadcasting.
Fig. 7 show it is according to an embodiment of the invention user, intelligent terminal and cloud brain between the parties
The flow chart communicated.
In order to realize the multi-modal interaction between intelligent terminal 102 and user 101, user 101, intelligent terminal 102 are needed
And communication connection is set up between cloud brain 104.This communication connection should be it is real-time, unobstructed, can guarantee to hand over
It is mutually impregnable.
In order to complete to interact, some conditions or premise are needed to have.These conditions or premise include intelligent terminal
Intelligent robot is loaded and run in 102, and intelligent terminal 102 has the hardware facility of perception and control function.Intelligence
Robot starting voice, emotion, vision and sensing capability when being in interaction mode.
After completing early-stage preparations, intelligent terminal 102 starts to interact with the expansion of user 101, firstly, being in intelligent robot
In the case where activation, multi-modal input data is obtained.At this point, two sides of expansion communication are intelligent terminal 102 and user 101 or intelligence
The direction of energy terminal 102 and surrounding enviroment, data transmitting is to be transmitted to intelligent terminal 102 from user 101 and surrounding enviroment.
Then, it extracts in multi-modal input data, the situation of presence parameter in current scene.It can in multi-modal input data
With the data comprising diversified forms, for example, may include text data, voice data, perception data in multi-modal input data
And action data etc..Intelligent terminal 102 can analyze multi-modal input data, extract in current scene and work as cause
Scape parameter.
Then, the situation of presence parameter is carried out merging the dynamic life trajectory diagram to form user with historical circumstances parameter
Spectrum.Intelligent terminal 102 can send to cloud brain 104 and request, and request cloud brain 104 merges situation of presence parameter and history feelings
Scape parameter.At this point, two sides of expansion communication are intelligent terminal 102 and cloud brain 104.
Finally, generating multi-modal interactive output data based on dynamic life locus spectra.Cloud brain 104 can be based on user
Dynamic life track generate multi-modal interactive output data.When intelligent terminal 102 receives the multimode of the transmission of cloud brain 104
After state interaction output data, intelligent terminal 102 can export multi-modal interactive output data by intelligent robot.At this point, expansion
Two sides of communication are intelligent terminal 102 and user 101.
Exchange method and system provided by the invention based on intelligent robot provides a kind of intelligent robot, intelligent machine
Device people has default image and preset attribute, can carry out multi-modal interaction with user.Also, the present invention can also be effectively sharp
With the life map in time, personage and the place generated in user's daily life etc., according to the abundant exploitation to life map
With using effectively being expanded to interaction topic or carrying out significant topic recommendation to user, interaction content is enriched,
Improve interactive accuracy.
It should be understood that disclosed embodiment of this invention is not limited to specific structure disclosed herein, processing step
Or material, and the equivalent substitute for these features that those of ordinary skill in the related art are understood should be extended to.It should also manage
Solution, term as used herein is used only for the purpose of describing specific embodiments, and is not intended to limit.
" one embodiment " or " embodiment " mentioned in specification means the special characteristic described in conjunction with the embodiments, structure
Or characteristic is included at least one embodiment of the present invention.Therefore, the phrase " reality that specification various places throughout occurs
Apply example " or " embodiment " the same embodiment might not be referred both to.
While it is disclosed that embodiment content as above but described only to facilitate understanding the present invention and adopting
Embodiment is not intended to limit the invention.Any those skilled in the art to which this invention pertains are not departing from this
Under the premise of the disclosed spirit and scope of invention, any modification and change can be made in the implementing form and in details,
But scope of patent protection of the invention, still should be subject to the scope of the claims as defined in the appended claims.
Claims (10)
1. a kind of exchange method based on intelligent robot, which is characterized in that the method comprises the steps of:
In the case where intelligent robot is in activation, multi-modal input data is obtained；
It extracts in the multi-modal input data, the situation of presence parameter in current scene；
The situation of presence parameter is carried out to merge the dynamic life locus spectra to form user with historical circumstances parameter；
Multi-modal interactive output data is generated based on the dynamic life locus spectra.
2. the method as described in claim 1, which is characterized in that extract in the multi-modal input data, in current scene
The step of situation of presence parameter, includes:
The geographical location for positioning current scene, determines the location parameters of current scene；
It captures and identifies the personage within the scope of current scene, determine the face of current scene；
The temporal information for obtaining current scene, determines the time parameter of current scene；
It is associated with the location parameters, the face and the time parameter, and is recorded as the situation of presence parameter of current scene.
3. method according to claim 1 or 2, which is characterized in that by the situation of presence parameter and historical circumstances parameter into
Include in the step of row fusion forms the dynamic life locus spectra of user:
The parameter attribute of situation of presence parameter is judged, determines the classification of situation of presence parameter；
According to the classification of situation of presence parameter, situation of presence parameter is merged with historical circumstances parameter, forms the dynamic of user
State life locus spectra.
4. method as claimed in any one of claims 1-3, which is characterized in that generated based on the dynamic life locus spectra
Include in the step of multi-modal interactive output data:
Determine the interaction topic of user Yu the intelligent robot；
The dynamic life locus spectra of traverse user is searched and interactive topic relevant user's life track；
Multi-modal interaction is generated according to the interactive relevant user life track of topic and the multi-modal input data
Output data, or update user life track.
5. method as claimed in claim 4, which is characterized in that determine the step of the interaction topic of user and the intelligent robot
In rapid, include:
The multi-modal input data is parsed, the interaction for obtaining user is intended to, decision output interaction words corresponding with the intention
Topic；
Or,
Situation of presence parameter is parsed, situation of presence parameter is analyzed, decision interaction topic.
6. method according to any one of claims 1 to 5, which is characterized in that by the situation of presence parameter and history feelings
Before scape parameter carries out the step of fusion forms the dynamic life locus spectra of user, also include:
The identity characteristic information for obtaining active user, judges the user property of active user, determines the class of active user
Not, wherein the classification of user includes: child user.
7. method as claimed in claim 6, which is characterized in that when the user interacted with the intelligent robot is child user
When, the method further includes:
Determine the interaction topic for being suitble to children's interaction to be intended to；
Or,
Situation of presence parameter is parsed, situation of presence parameter and children's face parameter are analyzed, decision interaction topic.
8. method as claimed in claim 6, which is characterized in that when the user interacted with the intelligent robot uses comprising children
When family, based on the dynamic life locus spectra generate multi-modal interactive output data the step of in include:
The multi-modal interactive output data is screened, the multi-modal interactive output data for being not suitable for child user is rejected.
9. a kind of program product, it includes for executing a series of of such as method and step of any of claims 1-8
Instruction.
10. a kind of interactive system based on intelligent robot, which is characterized in that the system includes:
Intelligent terminal, it includes camera, positioning device and time devices, for obtaining the situation of presence parameter of current scene
And multi-modal input data, and have voice, emotion, the ability of expression and movement output；
Intelligent robot is mounted on the intelligent terminal, is carried out using method such as of any of claims 1-8
Multi-modal interaction；
Cloud brain is used to carry out the multi-modal input data semantic understanding, visual identity, cognition calculating and emotion
It calculates, multi-modal interactive output data is exported with intelligent robot described in decision.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810777646.9A CN109176535B (en) | 2018-07-16 | 2018-07-16 | Interaction method and system based on intelligent robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810777646.9A CN109176535B (en) | 2018-07-16 | 2018-07-16 | Interaction method and system based on intelligent robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109176535A true CN109176535A (en) | 2019-01-11 |
| CN109176535B CN109176535B (en) | 2021-10-19 |
Family
ID=64936643
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810777646.9A Active CN109176535B (en) | 2018-07-16 | 2018-07-16 | Interaction method and system based on intelligent robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109176535B (en) |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008321A (en) * | 2019-03-07 | 2019-07-12 | 腾讯科技（深圳）有限公司 | Information interacting method and device, storage medium and electronic device |
| CN110000777A (en) * | 2019-03-12 | 2019-07-12 | 广东小天才科技有限公司 | Multihead display robot, multi-display method and device, readable storage medium storing program for executing |
| CN110109596A (en) * | 2019-05-08 | 2019-08-09 | 芋头科技（杭州）有限公司 | Recommended method, device and the controller and medium of interactive mode |
| CN110164249A (en) * | 2019-05-22 | 2019-08-23 | 重庆工业职业技术学院 | A kind of computer on-line study supervision auxiliary system |
| CN110347817A (en) * | 2019-07-15 | 2019-10-18 | 网易（杭州）网络有限公司 | Intelligent response method and device, storage medium, electronic equipment |
| CN110580516A (en) * | 2019-08-21 | 2019-12-17 | 厦门无常师教育科技有限公司 | interaction method and device based on intelligent robot |
| CN110640757A (en) * | 2019-09-23 | 2020-01-03 | 浙江树人学院（浙江树人大学） | Multi-mode interaction method applied to intelligent robot and intelligent robot system |
| CN110941774A (en) * | 2019-12-05 | 2020-03-31 | 深圳前海达闼云端智能科技有限公司 | Service recommendation method |
| CN111444873A (en) * | 2020-04-02 | 2020-07-24 | 北京迈格威科技有限公司 | Method and device for detecting authenticity of person in video, electronic device and storage medium |
| CN112235179A (en) * | 2020-08-29 | 2021-01-15 | 上海量明科技发展有限公司 | Method and device for processing topics in instant messaging and instant messaging tool |
| CN114428464A (en) * | 2021-12-29 | 2022-05-03 | 深圳市优必选科技股份有限公司 | Robot cluster control interaction method and device, terminal equipment and storage medium |
| CN116627261A (en) * | 2023-07-25 | 2023-08-22 | 安徽淘云科技股份有限公司 | Interaction method, device, storage medium and electronic equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001188783A (en) * | 1999-12-28 | 2001-07-10 | Sony Corp | Device and method for processing information and recording medium |
| KR20090090613A (en) * | 2008-02-21 | 2009-08-26 | 주식회사 케이티 | System and method for multimodal conversational mode image management |
| CN105867633A (en) * | 2016-04-26 | 2016-08-17 | 北京光年无限科技有限公司 | Intelligent robot oriented information processing method and system |
| CN106489148A (en) * | 2016-06-29 | 2017-03-08 | 深圳狗尾草智能科技有限公司 | A kind of intention scene recognition method that is drawn a portrait based on user and system |
| CN106537293A (en) * | 2016-06-29 | 2017-03-22 | 深圳狗尾草智能科技有限公司 | Method and system for generating robot interactive content, and robot |
| CN107885837A (en) * | 2017-11-09 | 2018-04-06 | 北京光年无限科技有限公司 | A kind of interaction output intent and intelligent robot for intelligent robot |
| CN108000526A (en) * | 2017-11-21 | 2018-05-08 | 北京光年无限科技有限公司 | Dialogue exchange method and system for intelligent robot |
| KR20180046649A (en) * | 2016-10-28 | 2018-05-09 | 한국과학기술연구원 | User intention detection system for initiation of interaction based on multi-modal perception and a method using the same |
| US20180166076A1 (en) * | 2016-12-14 | 2018-06-14 | Panasonic Intellectual Property Management Co., Ltd. | Voice interaction device, voice interaction method, voice interaction program, and robot |
-
2018
- 2018-07-16 CN CN201810777646.9A patent/CN109176535B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2001188783A (en) * | 1999-12-28 | 2001-07-10 | Sony Corp | Device and method for processing information and recording medium |
| KR20090090613A (en) * | 2008-02-21 | 2009-08-26 | 주식회사 케이티 | System and method for multimodal conversational mode image management |
| CN105867633A (en) * | 2016-04-26 | 2016-08-17 | 北京光年无限科技有限公司 | Intelligent robot oriented information processing method and system |
| CN106489148A (en) * | 2016-06-29 | 2017-03-08 | 深圳狗尾草智能科技有限公司 | A kind of intention scene recognition method that is drawn a portrait based on user and system |
| CN106537293A (en) * | 2016-06-29 | 2017-03-22 | 深圳狗尾草智能科技有限公司 | Method and system for generating robot interactive content, and robot |
| KR20180046649A (en) * | 2016-10-28 | 2018-05-09 | 한국과학기술연구원 | User intention detection system for initiation of interaction based on multi-modal perception and a method using the same |
| US20180166076A1 (en) * | 2016-12-14 | 2018-06-14 | Panasonic Intellectual Property Management Co., Ltd. | Voice interaction device, voice interaction method, voice interaction program, and robot |
| CN107885837A (en) * | 2017-11-09 | 2018-04-06 | 北京光年无限科技有限公司 | A kind of interaction output intent and intelligent robot for intelligent robot |
| CN108000526A (en) * | 2017-11-21 | 2018-05-08 | 北京光年无限科技有限公司 | Dialogue exchange method and system for intelligent robot |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110008321A (en) * | 2019-03-07 | 2019-07-12 | 腾讯科技（深圳）有限公司 | Information interacting method and device, storage medium and electronic device |
| CN110000777A (en) * | 2019-03-12 | 2019-07-12 | 广东小天才科技有限公司 | Multihead display robot, multi-display method and device, readable storage medium storing program for executing |
| CN110109596B (en) * | 2019-05-08 | 2021-11-16 | 芋头科技（杭州）有限公司 | Recommendation method and device of interaction mode, controller and medium |
| CN110109596A (en) * | 2019-05-08 | 2019-08-09 | 芋头科技（杭州）有限公司 | Recommended method, device and the controller and medium of interactive mode |
| CN110164249A (en) * | 2019-05-22 | 2019-08-23 | 重庆工业职业技术学院 | A kind of computer on-line study supervision auxiliary system |
| CN110347817A (en) * | 2019-07-15 | 2019-10-18 | 网易（杭州）网络有限公司 | Intelligent response method and device, storage medium, electronic equipment |
| CN110347817B (en) * | 2019-07-15 | 2022-03-18 | 网易（杭州）网络有限公司 | Intelligent response method and device, storage medium and electronic equipment |
| CN110580516A (en) * | 2019-08-21 | 2019-12-17 | 厦门无常师教育科技有限公司 | interaction method and device based on intelligent robot |
| CN110580516B (en) * | 2019-08-21 | 2021-11-09 | 厦门无常师教育科技有限公司 | Interaction method and device based on intelligent robot |
| CN110640757A (en) * | 2019-09-23 | 2020-01-03 | 浙江树人学院（浙江树人大学） | Multi-mode interaction method applied to intelligent robot and intelligent robot system |
| CN110941774A (en) * | 2019-12-05 | 2020-03-31 | 深圳前海达闼云端智能科技有限公司 | Service recommendation method |
| CN111444873A (en) * | 2020-04-02 | 2020-07-24 | 北京迈格威科技有限公司 | Method and device for detecting authenticity of person in video, electronic device and storage medium |
| CN111444873B (en) * | 2020-04-02 | 2023-12-12 | 北京迈格威科技有限公司 | Method and device for detecting authenticity of person in video, electronic equipment and storage medium |
| CN112235179B (en) * | 2020-08-29 | 2022-01-28 | 上海量明科技发展有限公司 | Method and device for processing topics in instant messaging and instant messaging tool |
| CN112235179A (en) * | 2020-08-29 | 2021-01-15 | 上海量明科技发展有限公司 | Method and device for processing topics in instant messaging and instant messaging tool |
| CN114428464A (en) * | 2021-12-29 | 2022-05-03 | 深圳市优必选科技股份有限公司 | Robot cluster control interaction method and device, terminal equipment and storage medium |
| CN114428464B (en) * | 2021-12-29 | 2023-12-15 | 深圳市优必选科技股份有限公司 | Robot cluster control interaction method and device, terminal equipment and storage medium |
| CN116627261A (en) * | 2023-07-25 | 2023-08-22 | 安徽淘云科技股份有限公司 | Interaction method, device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109176535B (en) | 2021-10-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109176535A (en) | Exchange method and system based on intelligent robot | |
| CN108000526B (en) | Dialogue interaction method and system for intelligent robot | |
| CN109522835A (en) | Children's book based on intelligent robot is read and exchange method and system | |
| CN110427472A (en) | The matched method, apparatus of intelligent customer service, terminal device and storage medium | |
| CN105843118B (en) | A kind of robot interactive method and robot system | |
| CN110110169A (en) | Man-machine interaction method and human-computer interaction device | |
| CN109271018A (en) | Exchange method and system based on visual human's behavioral standard | |
| CN109871450A (en) | Based on the multi-modal exchange method and system for drawing this reading | |
| CN107030691A (en) | A kind of data processing method and device for nursing robot | |
| CN110413841A (en) | Polymorphic exchange method, device, system, electronic equipment and storage medium | |
| CN107423398A (en) | Exchange method, device, storage medium and computer equipment | |
| CN105868827A (en) | Multi-mode interaction method for intelligent robot, and intelligent robot | |
| CN106528859A (en) | Data pushing system and method | |
| CN106625678A (en) | Robot expression control method and device | |
| CN107392783A (en) | Social contact method and device based on virtual reality | |
| CN109278051A (en) | Exchange method and system based on intelligent robot | |
| WO2018000259A1 (en) | Method and system for generating robot interaction content, and robot | |
| CN109343695A (en) | Exchange method and system based on visual human's behavioral standard | |
| CN109324688A (en) | Exchange method and system based on visual human's behavioral standard | |
| CN109948447A (en) | The discovery of personage's cyberrelationship and evolution rendering method based on video image identification | |
| CN107480766B (en) | Method and system for content generation for multi-modal virtual robots | |
| CN110532912A (en) | A kind of sign language interpreter implementation method and device | |
| CN107784355A (en) | The multi-modal interaction data processing method of visual human and system | |
| CN108416420A (en) | Limbs exchange method based on visual human and system | |
| CN108052250A (en) | Virtual idol deductive data processing method and system based on multi-modal interaction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |