Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
According to the robot interaction method and the robot provided by the embodiment of the invention, the robot actively interacts with a person in daily life to actively collect user information, and especially actively collect the information which is lack or needs to be confirmed in the current user characteristic information set, so that the improvement of user attributes is accelerated. The invention adopts a targeted active information inquiry and acquisition mode to efficiently perfect the characteristic information set of the user, establish the deep relation between the robot and the user and provide faster and more intimate user experience for subsequent human-computer interaction.
Referring to fig. 1, a block diagram of the interactive robot is shown.
The interactive robot 10 according to the present embodiment includes a processing unit 12, an audio acquiring module 20, an audio recognizing module 22, an image acquiring module 30, an image recognizing module 32, a responding module 40, and a transmitting and receiving unit 810. The interactive robot further includes a question answering module 40, a user information perfecting module 50, and an answering module 60.
The interactive robot 10 is wirelessly connected to the cloud server 100, and sends messages to the cloud server 100 and receives data from the cloud server 100. In an embodiment, the mobile terminal of the user is also connected to the cloud server 100 and establishes a contact with the robot owned by the user, so that the user can exchange data and information with the robot at home through the mobile terminal when the user goes out of the home.
The question-answering module 40 includes a communication scenario library 42 and establishes a feature information set corresponding to the user.
The question-answering module 40 acquires the field information of the user and calculates the user interaction parameters according to the field information;
the information refinement module 50 repeatedly determines the item to be supplemented and refines the characteristic information base. When the interaction parameters meet the requirements, the information completing module 50 queries and determines the items to be supplemented in the feature information set, and determines the communication scene information from the communication scene library 42 according to the items to be supplemented. The robot can issue questions to the user according to the communication scene information, wherein the communication scene information comprises the question scenes and the topics related to the items to be supplemented. For example, the item to be supplemented is a diet preference, and the questioning scene can be determined as a family scene of breakfast just getting up according to the current time such as 07:00, the number of people who exchange is one, and the questioning subject is weather or breakfast; or still taking the item to be supplemented as the dietary preference for example, according to the current environmental parameters, such as 13:00 noon and 35 ℃ temperature, the questioning scene can be determined as a noon family scene, the number of people to communicate one, and the questioning subject is weather experience or favorite beverage and the like according to the current temperature.
The user is actively asked by voice and/or image based on the relevant communication scene information. The information completing module 50 obtains the voice and/or image feedback information of the user, extracts the related content associated with the item to be supplemented in the feedback information, and stores the related content to the feature information set. The site information includes time, place, temperature, user voice information, user video information, and other communication conditions and environmental parameters set by the user. The interaction parameter indicates the suitability degree of human-computer interaction, for example, the range of the interaction parameter is 0-10, the interaction parameter value is more than 5, the interaction is suggested, and the optimal interaction opportunity is obtained when the interaction parameter is 10.
In this embodiment, the service provider may update the communication scenario library periodically through the cloud server 100.
Referring to fig. 9, a frame diagram of a robot interactive system is shown. The robot interaction system includes the cloud server 100 and a plurality of robots 10 connected to the cloud server 100. Wherein, each robot 10 can bind at least one user, and each user can bind at least one mobile terminal 15. For example, the bot 10-1 binds two users A1 and A2, user A1 binds the mobile terminal 15-1, the bot 10-2 binds one user B, and user B binds the mobile terminal 15-2. The robot 10 can upgrade the system and update the communication scene library 42 through the cloud server 100.
The response module 60 completes the response to the user question request based on the continuously self-perfected user characteristic information set, and realizes the purpose of providing the response closest to the question request with the least number of voice question-answering times. The response module 60 receives a request initiated by a user through voice and/or image, extracts associated content from the feature information set which is continuously improved according to the request, and responds to the request of the user after prejudging the associated content.
In specific implementation, the response module 60 extracts a matching keyword from the voice and/or image request, establishes a feature information classification relation table of the feature information set, extracts associated content with the closest classification relation from the feature information set according to the matching keyword, and determines communication scene information from the communication scene library according to the associated content to respond to the user request.
The items to be supplemented may be all content items related to the user's attributes. The following receives an updated and refined implementation of the items to be supplemented from three aspects of user habits and preferences, psychological attributes, and associated characters.
Referring to fig. 2, the information completing module 50 includes an inserting module 51, a testing module 53, an extracting module 55, and a determining module 57. When the item to be supplemented is the user habit and preference, the insertion module 51 inserts the question of the user habit and preference in the question-answering module chatting dialogue. When the item to be supplemented is a psychological attribute, the test module 53 obtains a psychological test question from the cloud server 100 and completes the psychological test question locally. When the item to be supplemented is the associated character information, the extraction module 55 obtains the associated character appearing in the user voice information by using a voice recognition technology, and extracts the associated character appearing in the video information by using a face recognition technology. The judgment module 57 judges the relevance of the associated person.
When the item to be supplemented is the user habit and preference, the information perfecting module 50 selects a chatting scene and a subject, and the inserting module 51 inserts a question of the user habit and preference in the question-answering module chatting conversation. The insertion module 51 obtains the feedback information of the user, extracts the related content associated with the habit and preference of the user in the feedback information, and stores the related content to the feature information set.
As an embodiment for supplementing the habit and preference of the user, the audio acquiring module 20 is a microphone for collecting the sound of the environment around the robot, and the image acquiring module 30 is a camera for capturing images. For example, after the robot is idle for a period of time or after determining that the current user may not be in a busy state through the environmental parameters of the user, for example, the user reads a book many times in the evening at the current time according to the history, the robot 10 or the mobile terminal 15 of the user may actively initiate a dialog to collect information according to the to-be-supplemented item which is absent or yet to be confirmed in the currently stored user attributes, which may take the following form but is not limited to the following form:
based on the chat scenario and topic, the robot can directly initiate a session, such as:
"host I want you well, you do nothing of I for a long time"
"what are you busy for the owner? Need me help you how? "
"will you be a severe haze in tomorrow, do not help you see the purifier? "
And the like.
The insertion module 51 interworks with some collections of user information, especially those to be complemented which are missing or yet to be confirmed in the user attributes, such as:
"what color you like? "
"do you like to eat chicken or beef? "
"do not want to take lunch break? "
And the like, and some habits and preferences of the user can be obtained in a natural way without causing the user to feel the objections.
From the perspective of user experience, the attributes of the user are never complete, and no matter how much information to be supplemented is collected, there is information that is required in the conversation.
As an embodiment of improving the psychological attribute of the user, the to-be-supplemented item is a psychological attribute, the information improvement module 50 further includes a test module 53, when the psychological attribute needs to be improved, the test module 53 initiates a request for obtaining a psychological test question to the cloud server 100, and receives the psychological test question selected by the cloud server 100 for the age, sex, and experience of the user. The testing module 53 displays the corresponding psychological test questions to the user through a display interface arranged by the robot, such as a touch display screen. The user can manually complete the psychological test question through the touch display screen or complete the psychological test in a voice interaction mode. The information completing module 50 sends the completed psychological test questions to the cloud server 100, and the cloud server 100 analyzes the returned psychological test answers to obtain an analysis result and sends the analysis result back to the requesting robot. The test module 53 receives the analysis result returned by the cloud server 100 for the psychological test, and stores the analysis result in the feature information set.
As an embodiment for improving the psychological attribute, the robot 10 downloads the psychological test questions for the user attribute from the cloud server 100, and actively presents the psychological test questions to the corresponding user of the robot through the touch display screen, or may make the test process vivid in combination with a voice manner, which may take the following forms but is not limited to the following forms:
"is this test said to be correct, do not want to try? "
'Wa' is woollen like my choice "
' o? How can you choose this? Not your style at all! "
' good score and high Wo, good Chongbai you! "
And the like. After the psychological question and answer is finished, the robot 10 provides the test answer to the cloud server 100 for analysis, and returns the psychological test analysis result of the user to the robot terminal, and stores the psychological test analysis result into the attribute corresponding to the user characteristic information set. In the psychological test process, the robot collects character information and preference information related to the psychological attributes of the user while the user obtains fun.
As an embodiment for perfecting the related persons, that is, when the item to be supplemented is related person information, the extracting module 55 obtains the daily voice information and video information of the user, extracts the related persons appearing in the voice information and stores the related persons into the feature information set, and extracts the related persons appearing in the video information and stores the related persons into the feature information set, which is the first step of establishing the related person file. The extracting module 55 needs to continuously obtain the daily voice information and video information of the user, extract a new associated person, and count the times of the stored associated persons.
After identifying and saving a plurality of related persons a plurality of times, the judging module 57 judges the relevance of the existing related persons to the user. In particular implementations, the relevance is based on a statistical number of occurrences of each associated person.
The information improvement module 50 also includes a anticipation module 59. The pre-judging module 59 counts the number of occurrences of all identified associated persons, and scans and counts the number of occurrences of each associated person, and compares the number with a set number threshold to judge whether the current associated person needs to perfect the feature information.
When a question is asked for the associated person of which the correlation exceeds a set threshold, the information perfection module 50 for completing the question asking for the associated person acquires the current site information of the user and generates a user interaction parameter; and when the interaction parameters are proper, determining communication scene information from the communication scene library according to the locked associated characters, and actively asking the user in combination with voice and images to improve the characteristic information of the locked associated characters.
The site information includes time, place, temperature, user voice information, user video information, and other communication conditions and environmental parameters set by the user.
As an embodiment of perfecting the associated people, a specific scenario is to interact with a user through photos or other private data. For example, when a user takes a photo, the AI program in the robot terminal searches the photo album, and recognizes each person in all photos in the photo album through the image acquisition module 30 and the image recognition module 32 in a face recognition or image recognition manner, the pre-judging module 59 counts the number of times each person appears, and it is assumed that the result in a certain search is as follows:
character A is known 75 times (who character A was previously known by this or other means)
Unknown character B30 times
Character C is known 22 times (who character C was previously known by this or other means)
Unknown figure D3 times
Unknown character E1 times
If the pre-judging module 59 finds that the occurrence frequency of a certain unknown person or some unknown persons exceeds a set frequency threshold, for example, 20 times, the unknown person is set as a person to be asked for questions, in this case, the unknown person B is set as the person to be asked for questions, the question and answer module 40 obtains the environmental parameters of the user and judges an appropriate time (for example, when the user is idle, when the user browses photos, or when the person to be asked for next time is taken again) to select an optional photo selected by the person to be asked for active conversation, for example:
"this beauty is good and beautiful, really star-like! Exactly who is this? "
"who is the handsome guy beside you? Family wants to know the relation between them "
While the screen display is as shown in figure 10.
According to the answer of the user, the content of the answer, such as name, relationship (such as wife, child, parent and the like) and face recognition characteristic value, is extracted and stored, and the content of the answer is saved to the corresponding characteristic information set of the user.
For the unknown persons not reaching the threshold value of the set times, if the degree of association between the unknown person D and the unknown person E in the example and the user is possibly small, the user does not make active questioning, but if the user actively mentions the unknown person not reaching the threshold value of the set times in the conversation process with the user, the unknown person is calibrated by the same method as above and added into the feature information set corresponding to the user.
Through the identification and question answering of the people in the photo for many times, the characteristic information and the associated information of a plurality of associated people related to the user can be determined, the understanding of the machine to the user is deepened, and the association relation storage of the invention can also develop more application scenes. In the embodiment of the invention, in a scene needing the photos, the user only needs to provide the requirements by voice, and then the interactive robot can directly extract the photos from the characteristic information set corresponding to the user and cut out the most suitable character image to submit, so that the user communication time is saved, and the human-computer interaction experience and the working efficiency of the user are improved.
Referring to fig. 3, an embodiment of the present invention further relates to a robot interaction method and a robot information collection method.
The robot information collection method comprises the following steps of user information collection:
the method comprises the following steps: establishing an exchange scene library and establishing a characteristic information set corresponding to a user;
step two: acquiring the field information of the user, and calculating user interaction parameters according to the field information; the site information includes time, place, temperature, user voice information, user video information, and other communication conditions and environmental parameters set by the user. The interaction parameter indicates the suitability degree of human-computer interaction, for example, the range of the interaction parameter is 0-10, the interaction parameter value is more than 5, namely, the interaction is suggested, and when the interaction parameter is 10, the optimal interaction opportunity is obtained;
step three: when the interactive parameters meet the requirements, inquiring and determining the items to be supplemented in the feature information set, determining related communication scene information from the communication scene library according to the items to be supplemented, and actively asking the user through voice and/or images based on the related communication scene information;
step four: acquiring voice and/or image feedback information of the user, extracting related content associated with an item to be supplemented in the feedback information, and storing the related content to the feature information set; extracting related content of the voice feedback information, which is related to an item to be supplemented, extracting related content of the video feedback information, which is related to the item to be supplemented, and judging the relevance of the related content and the item to be supplemented; in order to ensure the matching accuracy, establishing an associated classification table of the item to be supplemented; identifying feedback content from the voice and/or image feedback information; and determining whether the feedback content of the user is subject and can be stored according to the associated classification table of the item to be supplemented. And if the relevant content extracted from the feedback information is associated with the item to be supplemented, saving the relevant content to the feature information set.
Step five: and determining the next item to be supplemented, and repeating the step two to the step four. In the step, the robot repeatedly judges whether the current interaction parameters of the user meet the preset threshold value, and finds out the time suitable for questioning and exchanging to complete the next user attribute information to be completed and supplemented.
Preferably, the robot 10 updates the library of communication scenes periodically. Or the cloud server 100 updates the communication scene library periodically to provide a fine communication experience.
Referring to fig. 4, the robot interaction method means that the robot performs question asking and interaction with the user based on the continuously improved user feature information set, and the part of the work is performed by the response module 60. The method mainly comprises the following steps: receiving a request initiated by a user through voice and/or images; extracting the associated content from the continuously improved characteristic information set according to the request, and responding to the request of the user after prejudging the associated content.
In an embodiment, the method comprises the following steps:
step 202: the response module 60 establishes a feature information classification relation table of the feature information set;
step 204: receiving a request for a voice and/or an image of a user; extracting matching keywords from the voice and/or image request;
step 206: extracting the associated content with the closest classification relation from the feature information set which is continuously improved according to the matching keywords;
step 208: and determining the communication scene information from the communication scene library according to the associated content and then responding to the request of the user.
Referring to fig. 5, when the item to be supplemented is the habit and preference of the user, the processing procedure is as follows:
step 302: when the items to be supplemented are user habits and preferences, selecting chatting scenes and themes
Step 304: inserting questions about user habits and preferences in the chat session;
step 306: and acquiring feedback information of the user, extracting relevant content associated with user habits and preferences in the feedback information, and storing the relevant content to the characteristic information set.
Referring to fig. 6, when the item to be supplemented is a psychological attribute, the robot may locally store the psychological test question library, or may obtain the psychological test question from the cloud server. The psychological test questions are the most targeted psychological test questions selected according to the age, sex and experience of the user. And after the robot locally inquires or receives the psychological test questions, the psychological test questions are finished through a display interface or the user is asked by voice to finish the psychological test. Embodiments of cloud server analysis assignment of psychometric test questions are described below.
Step 402: when the item to be supplemented is the psychological attribute, acquiring a psychological test question from the cloud server; finishing the psychological test question through a display interface or asking questions to the user by adopting voice to finish the psychological test;
step 404: sending the completed psychological test to a cloud server;
step 406: and receiving an analysis result returned by the cloud server aiming at the psychological test, and storing the analysis result to the characteristic information set.
Referring to fig. 7, when the item to be supplemented is the associated character information, the robot identifies the associated character, and determining the associated character further includes:
step 502: extracting the associated persons appearing in the voice information, and storing the associated persons to the feature information set; extracting the associated persons appearing in the video information, and storing the associated persons to the characteristic information set;
step 504: judging the relevance of the associated person;
counting the occurrence times of all identified associated persons;
scanning the occurrence frequency of each associated person, comparing the occurrence frequency with a set frequency threshold value and judging whether the current associated person needs to perfect the characteristic information;
step 506: asking questions about related persons with the relevance exceeding a set threshold;
the step of inquiring the associated character comprises the steps of acquiring the site information of the user and generating user interaction parameters; when the interaction parameters are proper, determining the communication scene information from the communication scene library according to the locked associated characters, and
step 508: and actively sending a question to the user by combining voice and images to perfect the characteristic information of the locked associated person.
According to the robot interaction method and the robot provided by the embodiment of the invention, the robot actively interacts with a person in daily life to actively collect user information, and especially actively collect the information which is lack or needs to be confirmed in the current user characteristic information set, so that the improvement of user attributes is accelerated. According to the invention, the characteristic information set of the user is efficiently updated and perfected based on the mode that the robot actively asks and acquires information, the deep relation between the robot and the user is established, and faster and more intimate user experience is provided for subsequent human-computer interaction.
The robot provided by the embodiment of the invention is different from the working mode of the traditional robot, and the traditional robot mainly initiates a conversation by a human and asks a machine to answer. In the embodiment, the robot or the mobile terminal bound by the robot can select a proper communication opportunity in daily operation, actively interact with the user and collect various feature information and habit preference of the user, especially feature information which is lacked in user attributes or needs to be confirmed, through various question and answer modes such as chatting, photo communication combined with image recognition/face recognition, psychological test questions and the like, so that the purpose of self-accelerating and improving a user feature information set is achieved.
The robot realizes feedback processing which is most suitable for user requirements through minimum communication input based on a continuously self-perfected user characteristic information set, reduces the times of voice question answering or the number of information required to be filled by the user as much as possible, provides more intelligent and more careful service for the user, and enables the user to experience more upper floors.
Fig. 8 is a schematic diagram of a hardware structure of an electronic device 600 of a robot interaction method according to an embodiment of the present invention, where as shown in fig. 8, the electronic device 600 includes:
one or more processors 610, a memory 620, an audio data collector 630, a video data collector 640, a communication component 650, and a display unit 660, one processor 610 being taken as an example in fig. 8. The output of the audio data collector is the input of the audio identification module, and the output of the video data collector is the input of the video identification module. The memory 620 stores instructions executable by the at least one processor 610, and the instructions when executed by the at least one processor invoke data of the audio data collector and the video data collector to establish a connection with a cloud server through the communication component 650, so that the at least one processor can execute the robot interaction method.
The processor 610, the memory 620, the display unit 660 and the human-computer interaction unit 630 may be connected by a bus or other means, and fig. 8 illustrates the connection by the bus as an example.
The memory 620, as a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the robot interaction method in the embodiment of the present invention (for example, the insertion module 51, the test module 53, the extraction module 55, the judgment module 57, and the anticipation module 59 shown in fig. 2). The processor 610 executes various functional applications of the server and data processing by executing nonvolatile software programs, instructions, and modules stored in the memory 620, that is, implements the robot interaction method in the above-described method embodiment.
The memory 620 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the robot electronic device, and the like. Further, the memory 620 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, the memory 620 optionally includes memory located remotely from the processor 610, which may be connected to the robotically interacting electronic device through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, perform the robot interaction method in any of the above-described method embodiments, for example, perform the above-described method steps one to five in fig. 3, perform the above-described method steps 202 to 208 in fig. 4, and implement the functions of the insertion module 51, the test module 53, the extraction module 55, the judgment module 57, the anticipation module 59, and the like in fig. 2.
The product can execute the method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiment of the present invention.
Embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer-executable instructions for execution by one or more processors, for example, to perform the method steps one to five in fig. 3 described above, and to perform the method steps 202 to 208 in fig. 4 described above, so as to implement the functions of the insertion module 51, the test module 53, the extraction module 55, the judgment module 57, the prejudgment module 59, and the like in fig. 2.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware related to instructions of a computer program, which can be stored in a computer readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), or the like.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; within the idea of the invention, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.