WO2018157349A1

WO2018157349A1 - Method for interacting with robot, and interactive robot

Info

Publication number: WO2018157349A1
Application number: PCT/CN2017/075435
Authority: WO
Inventors: 张涛; 黄晓庆; 骆磊
Original assignee: 深圳前海达闼云端智能科技有限公司
Priority date: 2017-03-02
Filing date: 2017-03-02
Publication date: 2018-09-07
Also published as: CN107278302A; CN107278302B

Abstract

Disclosed is a method for interacting with a robot, the method comprising the following steps of collecting user information: acquiring site information about the user, and calculating user interaction parameters according to the site information; when the user interaction parameters satisfy requirements, querying and determining, from a user feature information set, items to be supplemented, determining, according to the items to be supplemented, related communication scene information from a communication scene library, and actively asking the user a question by means of voice and/or an image and based on the communication scene information related to the items to be supplemented; and acquiring voice and/or image feedback information about the user, extracting related contents, associated with the items to be supplemented, from the feedback information, and saving the related contents in the user feature information set.

Description

Robot interaction method and interactive robot

Technical field

The present application relates to the field of robot interaction, and in particular relates to a robot interaction method and an interaction robot.

Background technique

With the development of network transmission and big data technology and the improvement of hardware processing capabilities, more and more robots have entered people's family life. The current human-computer interaction methods are basically people asking for the answer. Although the answer methods are various and more intelligent, most of them are passively receiving the user's question information. There is no deep connection between the robot and the user.

For example, Chinese Patent Application No. 201610970633.4 discloses a robot human-computer interaction method and system, the robot human-computer interaction system includes: a first acquisition module for acquiring a laser signal; and a second acquisition module for acquiring a voice signal; An execution module, configured to activate different preset actions according to different laser receivers corresponding to the laser signal; and a second execution module, configured to perform a corresponding preset action and/or a corresponding preset according to the voice signal Set the voice.

The current human-computer interaction methods are basically people asking for a machine answer, and most of them are passive receiving user information. This passive interaction makes the user information that the robot lacks can only be provided by the user in the interaction by the robot repeatedly asking questions in the interaction. And some robot processes, such as booking a train ticket, even need more than ten items of information to complete the final scheduled task. However, the user experience is not good in the way of obtaining all the requirements through a question and answer in one interaction. The usable information stored by the robot is far from the user's needs, which may cause the user to abandon the use of voice interaction and switch back to the touch screen. . Therefore, how to effectively collect user information without affecting the user experience will be an urgent problem to be solved.

Therefore, the prior art robot interaction mode has yet to be improved.

Summary of the invention

The present application provides a robot interaction method and a robot, and the robot interaction method and the robot select an appropriate timing in the daily operation of the robot, and actively interact with the user through various interaction modes to collect various information and habit preferences of the user, and continuously self. The user feature information set is improved to support the user's subsequent question request, and the response that is closest to the question request is provided with a minimum number of voice questions and answers.

In a first aspect, an embodiment of the present application provides a robot interaction method, including the following user information collection steps:

Obtaining the site information of the user, and calculating user interaction parameters according to the site information;

When the user interaction parameter meets the requirements, the user feature information is queried and determined from the user profile information, and the related communication scenario information is determined from the communication scenario library according to the to-be-replenished item, and is based on the communication scenario information related to the item to be supplemented. Voice and/or image actively ask the user;

Acquiring the voice and/or image feedback information of the user, extracting related content associated with the item to be supplemented in the feedback information, and saving the related content to the user feature information set.

In a second aspect, the embodiment of the present application further provides an interactive robot, including: an audio acquisition module, an audio recognition module, an image acquisition module, and an image recognition module, and further includes a question answering module and a user information perfecting module:

The question and answer module includes an exchange scenario library and a feature information set;

The question and answer module is configured to obtain the site information of the user, and calculate user interaction parameters according to the site information;

The information perfecting module is configured to repeatedly determine the item to be supplemented and improve the feature information database, and is used to query and determine the item to be supplemented from the user characteristic information when the user interaction parameter meets the requirement, and exchange the item according to the to-be-replenished item. Determining related communication scene information in the scene library, and actively sending a question to the user through voice and/or image based on the communication scene information related to the item to be supplemented; acquiring the user's voice and/or image feedback information, and extracting the feedback information The related content associated with the item to be supplemented is saved to the user feature information set.

In a third aspect, the embodiment of the present application further provides an electronic device, including:

At least one processor; and,

a memory communicatively coupled to the at least one processor, a communication component, an audio data collector, and a video data collector; wherein

The memory stores instructions executable by the at least one processor, the instructions being invoked by the at least one processor to invoke data of the audio data collector and the video data collector, and establishing a connection with the cloud server through the communication component to enable the At least one processor is capable of performing the method as described above.

In a fourth aspect, the embodiment of the present application further provides a non-transitory computer readable storage medium, where the computer readable storage medium stores computer executable instructions for causing a computer to execute the above The method described.

In a fifth aspect, the embodiment of the present application further provides a computer program product, where the computer program product includes a computer program stored on a non-transitory computer readable storage medium, the computer program includes program instructions, when When the program instructions are executed by the computer, the computer is caused to perform the method as described above.

The beneficial effects of the present application are the robot interaction method and the robot provided by the embodiments of the present application, In the daily operation of the robot, the appropriate timing can be automatically selected. Through various interaction modes of the exchange scenario library, the user and the user interact actively to collect various information and habit preferences of the user, and continuously improve the user feature information collection to support the user. Subsequent question requests provide a response that is closest to the request with the least number of voice quiz.

DRAWINGS

The one or more embodiments are exemplified by the accompanying drawings in the accompanying drawings, and FIG. The figures in the drawings do not constitute a scale limitation unless otherwise stated.

1 is a system block diagram of an interactive robot provided by an embodiment of the present application;

2 is a subdivision module diagram of a user information perfecting module of an interactive robot provided by an embodiment of the present application;

3 is a main flowchart of a robot interaction method provided by an embodiment of the present application;

4 is a schematic diagram of a robot interaction method for a robot interaction method provided by an embodiment of the present application;

FIG. 5 is a flowchart of an implementation of a user interaction and preference in a robot interaction method according to an embodiment of the present application; FIG.

6 is a flowchart of an implementation of a psychological attribute in a robot interaction method according to an embodiment of the present application;

FIG. 7 is a flowchart of an implementation of a robot interaction method according to an embodiment of the present application;

8 is a schematic diagram of a hardware structure of an electronic device of a robot interaction method according to an embodiment of the present application;

9 is a schematic diagram of a framework of a robot interaction system provided by an embodiment of the present application;

FIG. 10 is a diagram showing an example of a related item to be added in a robot interaction method according to an embodiment of the present application.

detailed description

In order to make the objects, technical solutions, and advantages of the present application more comprehensible, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

The robot interaction method and the robot provided by the embodiments of the present application actively collect user information by actively interacting with humans in daily life, in particular, for the active collection of the current user feature information set that lacks or needs confirmation information, and accelerates the user attribute. perfect. This application uses targeted initiative The way of information inquiry and acquisition can effectively improve the user's feature information set, establish a deep connection between the robot and the user, and provide a faster and more intimate user experience for subsequent human-computer interaction.

Please refer to FIG. 1 , which is a block diagram of the interactive robot.

The interactive robot 10 according to this embodiment includes a processing unit 12, an audio acquisition module 20, an audio recognition module 22, an image acquisition module 30, an image recognition module 32, a response module 40, and a transmission and reception unit 810. The interactive robot further includes a question answering module 40, a user information perfecting module 50, and a response module 60.

The interactive robot 10 is wirelessly connected to the cloud server 100, transmits a message to the cloud server 100, and receives data from the cloud server 100. In an embodiment, the user's mobile terminal is also connected to the cloud server 100 and establishes the contact of the robot owned by the user, so that the user walks out of the outdoors to exchange data and information with the robot at home through the mobile terminal.

The question and answer module 40 includes an exchange scenario library 42 and establishes a feature information set corresponding to the user.

The question answering module 40 obtains the site information of the user, and calculates a user interaction parameter according to the site information;

The information refinement module 50 repeatedly determines the item to be supplemented and completes the feature information base. The information refinement module 50 queries and determines the to-be-replenished item in the feature information set when the interaction parameter meets the requirement, and determines the communication scenario information from the communication scenario library 42 according to the to-be-replenished item. The robot may query the user according to the communication scenario information, where the communication scenario information includes a questioning scene and a topic related to the supplementary item. For example, if the item to be supplemented is a dietary preference, the questioning scene can be determined according to the current time, such as 07:00, to determine the family scene just getting up for breakfast, the number of people communicating is one person, the subject of the question is weather or breakfast; or the supplementary item is still a dietary preference. For example, according to the current environmental parameters, such as 13:00 noon, the temperature is 35 degrees Celsius, the questioning scene can be determined as the noon family scene according to the current temperature, the number of people communicating is one person, and the subject is the weather feeling or favorite drink.

The user is actively queried by voice and/or image based on the related communication scene information. The information perfecting module 50 obtains the voice and/or image feedback information of the user, extracts related content associated with the item to be supplemented in the feedback information, and saves the related content to the feature information set. The site information includes time, location, temperature, user voice information, user video information, and other communication conditions set by the user, and environmental parameters. The interaction parameter indicates the appropriateness of the human-computer interaction. For example, the range of the interaction parameter is 0-10, the interaction parameter value greater than 5 is the suggested interaction, and the interaction parameter is 10 is the optimal interaction timing.

In this embodiment, the service provider can periodically update the exchange scenario library through the cloud server 100.

Please refer to FIG. 9, which is a schematic diagram of a robot interaction system framework. The robot interaction system includes the cloud server 100 and a plurality of robots 10 connected to the cloud server 100. Each robot 10 can be bound to at least one user, and each user can bind at least one mobile terminal 15. For example, the robot 10-1 binds two users A1 and A2, the user A1 is bound to the mobile terminal 15-1, the robot 10-2 is bound to one user B, and the user B is bound to the mobile terminal 15-2. The robot 10 can be upgraded through the cloud server 100 System and update exchange scenario library 42.

The response module 60 completes the response to the user's question request based on the continuously self-improved set of user feature information, and provides a response that is closest to the question request with a minimum number of voice questions and answers. The response module 60 receives a request initiated by the user through voice and/or image, and extracts the associated content from the continuously refined feature information set according to the request, and responds to the user's request after pre-judging the associated content.

In a specific implementation, the response module 60 extracts a matching keyword from the request for the voice and/or image, establishes a feature information classification relationship table of the feature information set, and extracts from the continuously improved feature information set according to the matching keyword. The related content whose classification relationship is closest, and the communication scene information is determined from the communication scene library according to the related content, and then the user's request is answered.

The to-be-replenished item can be all content items related to the user attribute. The following is an updated implementation method for receiving supplementary items from three aspects: user habits and preferences, psychological attributes, and related characters.

Referring to FIG. 2, the information perfecting module 50 includes an inserting module 51, a testing module 53, an extracting module 55, and a judging module 57. When the to-be-added item is user habit and preference, the plug-in module 51 inserts a questioning of the user's habits and preferences in the question-answer module chat session. When the item to be supplemented is a psychological attribute, the test module 53 acquires the psychological test question from the cloud server 100 and completes the psychological test question locally. When the to-be-replenished item is the associated person information, the extraction module 55 acquires the associated person appearing in the user's voice information by using the voice recognition technology, and uses the face recognition technology to extract the related person appearing in the video information. The determining module 57 determines the relevance of the associated person.

When the to-be-added item is user habit and preference, the information perfecting module 50 selects the gossip scene and the topic, and the inserting module 51 inserts a questioning of the user's habits and preferences in the question-answering module chat session. The insertion module 51 acquires feedback information of the user, extracts related content associated with user habits and preferences in the feedback information, and saves the related content to the feature information set.

As an embodiment of supplementing user habits and preferences, the audio acquisition module 20 is a microphone that collects ambient sounds of the robot, and the image acquisition module 30 cameras complete image capture through the camera. For example, after the robot is idle for a period of time, or through the user's environmental parameters, it is determined that the current user may not be in a busy state, for example, according to the history of the current time, the user reads the book multiple times at night, and the robot 10 or the user's mobile terminal 15 may be based on the current The information to be added in the stored user attributes or the items to be confirmed that need to be confirmed actively initiates a dialogue for information collection, which may take the following forms but is not limited to the following forms:

Based on the chat scene and theme, the robot can directly initiate a conversation, such as:

"Master, I miss you, you haven't ignored me for a long time~"

"What are you busy with? What do you need me to help you?"

"Tomorrow is a serious smog, do you want me to help you see the purifier?"

And so on.

In the dialog box, the insert module 51 intersperses some collections of user information, especially missing or still need to be confirmed in the user attribute, such as:

“What color do you like?”

“Do you like chicken or beef?”

"Would you like to take a lunch break?"

In other ways, the user's habits and preferences are obtained in a natural way without causing the user's dislike.

From the perspective of user experience, the attributes of the user will never be intact. No matter how much information to be added is collected, there must be information needed in the dialogue. In this embodiment, the chat scene and the theme can also pass through the cloud server. Make regular updates.

As an embodiment of perfecting the user's psychological attribute, the to-be-replenished item is a psychological attribute, and the information perfecting module 50 further includes a testing module 53 that initiates a request for obtaining a psychological test question to the cloud server 100 when the psychological attribute needs to be perfected. And receiving the psychology test question selected by the cloud server 100 for the user's age, gender, and experience. The test module 53 displays the corresponding psychological test questions to the user through a display interface set by the robot, such as a touch display screen. The user can manually complete the psychological test question through the touch display screen or complete the psychological test by using a voice interaction method. The information perfecting module 50 sends the completed psychological test question to the cloud server 100, and the cloud server 100 analyzes the returned psychological test answer, obtains the analysis result, and sends back the requested robot. The test module 53 receives the analysis result returned by the cloud server 100 for the psychological test, and saves the analysis result to the feature information set.

As an embodiment of perfecting the psychological attribute, the robot 10 downloads a psychological test question for the user attribute from the cloud server 100 and actively displays it to the corresponding user of the robot through the touch display screen, or may make the test process vivid in combination with the voice mode, which may adopt the following form: But not limited to the following form:

"This test is said to be very accurate. Do you want to try it?"

"Wow, the same as I chose~"

"Ah? How can I choose this? It's not your style at all!"

"The score is so good, I admire you!"

And so on. After the psychological question and answer, the robot 10 provides the test answer to the cloud server 100 for analysis, and returns the user psychological test analysis result to the robot terminal, and stores it in the attribute corresponding to the user feature information set. In the psychological testing process, while the user is having fun, the robot also collects personality information and preference information related to the user's psychological attributes.

As an embodiment of perfecting the associated person, that is, when the to-be-added item is related person information, the extracting module 55 acquires the daily voice information and the video information of the user, and extracts the related person appearing in the voice information to save the related person to the feature information. The set, and the associated person appearing in the extracted video information, save the related person to the feature information set, and the above is the first step of establishing the associated person file. The extraction module 55 also needs to continue to obtain the daily voice information and video information of the user, extract new related characters, and perform statistics on the number of saved related characters.

After identifying and storing a number of related characters multiple times, the determining module 57 determines the relevance of the existing associated person to the user. In the specific implementation, the correlation is based on the number of statistics of each associated person.

The information refinement module 50 also includes a pre-judgment module 59. The pre-judgment module 59 statistically counts the number of occurrences of all associated characters, and scans the number of occurrences of each associated person, and compares it with the set number of thresholds to determine whether the current associated person needs to perfect the feature information.

When the related person whose relevance exceeds the set threshold is asked, the information perfecting module 50 that completes the related person questioning acquires the current site information of the user, and generates a user interaction parameter; when the interactive parameter is appropriate, the related person is connected according to the locked The scene scene information is determined in the scene library, and the user is actively asked to combine the voice and the image to complete the feature information of the locked associated person.

The site information includes time, location, temperature, user voice information, user video information, and other communication conditions set by the user, as well as environmental parameters.

As an embodiment of perfecting the associated person, the specific scenario is to interact with the user through photos or other private data. For example, if the user takes a photo, the AI program in the robot terminal searches for the album, and the image recognition module 30 and the image recognition module 32 identify each character in all the photos of the album by means of face recognition or image recognition. Module 59 counts the number of occurrences of each character, assuming the results of a search are as follows:

Known person A 75 times (have been known by this way or other means who is the character A)

Unknown person B 30 times

Known person C 22 times (have been known by this way or other means who is the character C)

Unknown person D 3 times

Unknown person E 1 time

If the predictive module 59 finds that the number of occurrences of one or some unknown characters exceeds a set number of thresholds, such as 20 times, the unknown person is placed as a candidate to be questioned, in which case the unknown person B will be placed as a question to be asked. The candidate, the question and answer module 40 obtains the user environment parameters and determines the appropriate timing (such as when the user browses the photo when idle, or when the next time the question is selected to be photographed again), select one of the selected photos to initiate a dialogue, such as:

"This beautiful woman is so beautiful, there is a star fan! Who is this?"

"Who is this handsome guy next to you? People want to know him~~"

At the same time, the screen display is as shown in Figure 10.

According to the user's answer, the content of the response, such as a name, a relationship (such as a wife, a child, a parent, etc.) and a facial recognition feature value are extracted and stored, and the content of the response is saved to the feature information set corresponding to the user.

For unknown people who have not reached the set number threshold, the unknown person D and the unknown person E in this case may have less relevance to the user, so they will not actively ask questions first, but if the user is active during the dialogue with the user When an unknown person who does not reach the set number threshold is mentioned, the calibration is performed in the same manner as above, and is added to the feature information set corresponding to the user.

After many times of identifying and answering questions in the photo, the feature information and related information of a plurality of related characters related to the user can be determined, the machine's understanding of the user is deepened, and the association storage of the present application can also be developed. More application scenarios. In the embodiment of the present application, in a scene where a photo is required, the user only needs to voice the request, and the interactive robot can directly extract the photo from the feature information set corresponding to the user and cut out the most suitable character image submission, thereby saving user communication time. Improve user human interaction experience and work efficiency.

Referring to FIG. 3, the embodiment of the present application further relates to a robot interaction method and a robot information collection method.

The robot information collecting method includes the following user information collecting steps:

Step 1: Establish an exchange scenario library to establish a feature information set corresponding to the user;

Step 2: Obtain the site information of the user, and calculate user interaction parameters according to the site information; wherein the site information includes time, location, temperature, user voice information, user video information, and other communication conditions and environmental parameters set by the user, etc. . The interaction parameter indicates the appropriateness of the human-computer interaction. For example, the range of the interaction parameter is 0-10, the interaction parameter value is greater than 5 is the suggested interaction, and the interaction parameter is 10 is the optimal interaction timing;

Step 3: When the interaction parameter meets the requirements, query and determine the to-be-added item in the feature information set, determine relevant communication scenario information from the communication scenario library according to the to-be-replenished item, and pass the voice and/or based on the relevant communication scenario information. Or the image actively sends a question to the user;

Step 4: Acquire the voice and/or image feedback information of the user, extract relevant content associated with the item to be supplemented in the feedback information, and save the related content to the feature information set; wherein, the voice feedback information and the item to be supplemented are extracted. Correlating the related content, extracting relevant content associated with the item to be supplemented in the video feedback information, and determining the correlation between the related content and the item to be supplemented; to ensure the accuracy of the matching, establishing an associated classification table of the item to be supplemented; The feedback content is identified in the voice and/or image feedback information; determining whether the feedback content of the user is relevant and storable according to the to-be-added item association classification table. If mention The related content in the feedback information is associated with the to-be-replenished item, and the related content is saved to the feature information set.

Step 5: Determine the next to-be-replenished item and repeat steps 2 through 4. In this step, the robot repeatedly determines whether the current interaction parameter of the user meets the preset threshold, finds a suitable time for questioning and communication, and starts to improve the next user attribute information to be perfected and supplemented.

Preferably, the robot 10 periodically updates the exchange scene library. Or the cloud server 100 periodically updates the communication scene library to provide a delicate communication experience.

Please refer to FIG. 4 , wherein the robot interaction method refers to the robot completing the question and interaction with the user based on the continuously improved user feature information set, and the part of the work is completed by the response module 60 . The method mainly includes: receiving a request initiated by a user by using a voice and/or an image; extracting the related content from the continuously improved feature information set according to the request, and responding to the user's request after pre-judging the related content.

In an embodiment, the following steps are included:

Step 202: The response module 60 establishes a feature information classification relationship table of the feature information set.

Step 204: Receive a request for a voice and/or image of the user; extract a matching keyword from the request for the voice and/or image;

Step 206: Extract, according to the matching keyword, the related content whose classification relationship is closest from the continuously improved feature information set;

Step 208: Determine the communication scenario information from the communication scenario library according to the associated content, and then respond to the user's request.

Please refer to FIG. 5, when the to-be-added item is user habit and preference, the process is as follows:

Step 302: When the item to be supplemented is a user habit and preference, select a chat scene and a theme.

Step 304: Insert a question about the user's habits and preferences in the chat conversation;

Step 306: Acquire feedback information of the user, extract related content associated with user habits and preferences in the feedback information, and save the related content to the feature information set.

Please refer to FIG. 6. When the item to be supplemented is a psychological attribute, the robot can store the psychological test question bank locally, or obtain the psychological test question from the cloud server. The psychological test questions are the most targeted psychological test questions selected based on the user's age, gender, and experience. After the robot locally queries or receives the psychological test question, the psychological test question is completed through the display interface or the user is asked to complete the psychological test by using the voice. The following describes an embodiment of the Cloud Server Analysis Assignment Psychological Test Question.

Step 402: When the item to be supplemented is a psychological attribute, obtain a psychological test question from the cloud server; complete the psychological test question through the display interface or use the voice to ask the user to complete the psychological test;

Step 404: Send the completed psychological test to the cloud server;

Step 406: Receive an analysis result returned by the cloud server for the psychological test, and save the analysis result to the feature information set.

Referring to FIG. 7 , when the item to be supplemented is related person information, the robot recognizes the associated person, and the determined related person further includes:

Step 502: Extract an associated person appearing in the voice information, save the associated person to the feature information set, extract an associated person appearing in the video information, and save the associated person to the feature information set;

Step 504: Determine the relevance of the associated person.

The number of times all statistically identified related characters appear;

Scan the number of occurrences of each associated character, compare it with the set number of thresholds, and determine whether the current associated person needs to improve the feature information;

Step 506: Questioning an associated person whose relevance exceeds a set threshold;

The related person questioning step includes obtaining the site information of the user, and generating a user interaction parameter; when the interaction parameter is appropriate, determining the communication scene information from the communication scene library according to the locked associated person, and

Step 508: Actively ask the user to combine the voice and the image to complete the feature information of the locked associated person.

The robot interaction method and the robot provided by the embodiments of the present application actively collect user information by actively interacting with humans in daily life, in particular, for the active collection of the current user feature information set that lacks or needs confirmation information, and accelerates the user attribute. perfect. The application efficiently updates and perfects the user's feature information set based on the robot's active questioning and information acquisition, establishes a deep connection between the robot and the user, and provides a faster and more intimate user experience for subsequent human-computer interaction.

The robot provided by the embodiment of the present application is different from the traditional robot working mode. The traditional robot mainly initiates a dialogue by a person, and the person asks for a machine answer. In this embodiment, the mobile terminal bound by the robot or the robot can select an appropriate communication opportunity in daily operation, through various question and answer methods, such as chat, photo communication combined with image recognition/face recognition, psychological test questions, and the like. Actively interact with the user and collect various feature information and habitual preferences of the user, especially the feature information that is missing or still need to be confirmed in the user attribute, thereby achieving the purpose of self-accelerating and perfecting the user feature information set.

Based on the continuous self-improvement of the user feature information set, the robot provides feedback processing that best suits the user's needs with minimal AC input, minimizes the number of voice quizzes or reduces the number of information users need to fill in, and provides users with smarter and more intimate users. The service brings the user experience to the next level.

FIG. 8 is a schematic diagram of the hardware structure of the electronic device 600 according to the robot interaction method provided by the embodiment of the present application. As shown in FIG. 8 , the electronic device 600 includes:

One or more processors 610, a memory 620, an audio data collector 630, a video data collector 640, a communication component 650, and a display unit 660 are illustrated by one processor 610 in FIG. The sound The output of the frequency data collector is the input of the audio recognition module, and the output of the video data collector identifies the input of the module. The memory 620 stores instructions executable by the at least one processor 610, the instructions being invoked by the at least one processor to invoke data of the audio data collector and the video data collector, and the communication component 650 establishes a connection with the cloud server. To enable the at least one processor to execute the robot interaction method.

The processor 610, the memory 620, the display unit 660, and the human-machine interaction unit 630 may be connected by a bus or other means, and the connection by a bus is taken as an example in FIG.

The memory 620 is used as a non-volatile computer readable storage medium, and can be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions corresponding to the robot interaction method in the embodiment of the present application. / Module (for example, the plug-in module 51, the test module 53, the extracting module 55, the judging module 57, and the pre-judging module 59 shown in FIG. 2). The processor 610 executes various functional applications and data processing of the server by executing non-volatile software programs, instructions, and modules stored in the memory 620, that is, implementing the robot interaction method in the above method embodiments.

The memory 620 may include a storage program area and an storage data area, wherein the storage program area may store an operating system, an application required for at least one function; the storage data area may store data created according to usage of the robot electronic device, and the like. Moreover, memory 620 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 620 can optionally include memory remotely located relative to processor 610, which can be connected to the robotic interactive electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, perform the robot interaction method in any of the above method embodiments, for example, performing the above described FIG. Steps 1 to 5 of the method are performed, and the method steps 202 to 208 in FIG. 4 described above are executed to implement the functions of the inserting module 51, the testing module 53, the extracting module 55, the determining module 57, and the predicting module 59 in FIG. .

The above products can perform the methods provided by the embodiments of the present application, and have the corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided by the embodiments of the present application.

Embodiments of the present application provide a non-transitory computer readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the above The method steps 202 to 208 in FIG. 4 described above are performed, and the insertion module 51, the test module 53, the extraction module 55, the determination module 57, and the pre-judgment in FIG. 2 are implemented. The function of module 59 and the like.

The device embodiments described above are merely illustrative, wherein the described as separate components The units may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

Through the description of the above embodiments, those skilled in the art can clearly understand that the various embodiments can be implemented by means of software plus a general hardware platform, and of course, by hardware. A person skilled in the art can understand that all or part of the process of implementing the above embodiments can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. When executed, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; in the idea of the present application, the technical features in the above embodiments or different embodiments may also be combined. The steps may be carried out in any order, and there are many other variations of the various aspects of the present application as described above, which are not provided in the details for the sake of brevity; although the present application has been described in detail with reference to the foregoing embodiments, The skilled person should understand that the technical solutions described in the foregoing embodiments may be modified, or some of the technical features may be equivalently replaced; and the modifications or substitutions do not deviate from the embodiments of the present application. The scope of the technical solution.

Claims

A robot interaction method, comprising the following user information collection steps:

Obtaining the site information of the user, and calculating user interaction parameters according to the site information;

When the user interaction parameter meets the requirements, the user feature information is queried and determined from the user profile information, and the related communication scenario information is determined from the communication scenario library according to the to-be-replenished item, and is based on the communication scenario related to the item to be supplemented. Information is actively sent to the user by voice and/or image;

Acquiring the voice and/or image feedback information of the user, extracting related content associated with the item to be supplemented in the feedback information, and saving the related content to the user feature information set.
The method of claim 1 further comprising the step of:

Receiving a request initiated by a user via voice and/or image;

And extracting the related content from the feature information set according to the request, and responding to the request of the user after pre-judging the related content.
The method of claim 2 wherein:

Establishing a feature information classification relationship table of the feature information set;

Extracting a matching keyword from the request for the voice and/or image;

And extracting, according to the matching keyword, the related content whose classification relationship is closest from the continuously improved feature information set, and determining, according to the related content, the response scene and the theme and then responding to the request of the user.
The method according to any one of claims 1-3, wherein the to-be-replenished item is a user's habit and preference, and the communication scenario is specifically a chat-talking communication scenario, and the actively asking the user for the specific question includes:

Insert questions about user habits and preferences in the chat conversation;

Obtaining feedback information of the user, extracting related content in the feedback information associated with user habits and preferences, and saving the related content to the feature information set.
The method according to any one of claims 1-3, wherein when the item to be supplemented is a psychological attribute, the psychological test question is obtained from the cloud server, and the psychological test question is completed through the display interface or the voice is used. The user asks the question to complete the psychological test, including:

Sending the completed psychological test to the cloud server;

Receiving an analysis result returned by the cloud server for the psychological test;

The analysis result is saved to the feature information set.
The method according to any one of claims 1-3, wherein when the item to be supplemented is related person information, acquiring user voice information and video information, the method further includes:

Extracting an associated person appearing in the voice information, and saving the associated person to the feature information set;

Extracting an associated person appearing in the video information, and saving the associated person to the feature information set;

Determining the relevance of the associated person;

Questioning related people whose relevance exceeds a set threshold;

The associating person questioning step includes: acquiring the site information of the user, and generating a user interaction parameter; when the interaction parameter is appropriate, determining the communication scenario information from the exchange scenario library according to the locked associated person, and actively combining the voice with the user And the image is asked to complete the feature information of the locked associated person.
The method according to claim 6, wherein the step of determining the relevance of the associated person comprises:

The number of times all statistically identified related characters appear;

Scan the number of occurrences of each associated person, compare it with the set number of thresholds, and determine whether the current associated person needs to improve the feature information.
An interactive robot includes an audio acquisition module, an audio recognition module, an image acquisition module, and an image recognition module, and is characterized in that it further comprises a question and answer module and a user information perfecting module:

The question answering module includes an exchange scenario library and a user feature information set;

The question answering module is configured to acquire the site information of the user, and calculate a user interaction parameter according to the site information;

The information perfecting module is configured to repeatedly determine the item to be supplemented and improve the feature information database, and is used to query and determine the item to be supplemented from the user characteristic information when the user interaction parameter meets the requirement, and exchange the item according to the to-be-replenished item. Determining related communication scene information in the scene library, and actively sending a question to the user through voice and/or image based on the communication scene information related to the item to be supplemented;

Acquiring the voice and/or image feedback information of the user, extracting related content associated with the item to be supplemented in the feedback information, and saving the related content to the user feature information set.
The interactive robot according to claim 8, further comprising a response module, wherein the response module is configured to receive a request initiated by the user by voice and/or image, and continuously improve the feature information according to the request. The set extracts the associated content, and the user's request is answered after the associated content is predicted.
The interactive robot according to claim 9, wherein the response module is configured to establish a feature information classification relationship table of the feature information set for requesting from the voice and/or image. And extracting a matching keyword, and extracting the related content that is closest to the classification relationship from the continuously improved feature information set according to the matching keyword, and determining the communication scenario information from the communication scenario library according to the associated content. Respond to the request of the user.
The interactive robot according to any one of claims 8 to 10, wherein, when the item to be supplemented is user habit and preference, the information perfecting module further includes an inserting module, and the information perfecting module is configured to select a chat. a scenario and a theme, the insertion module is configured to insert a question for the user's habits and preferences in the question and answer module chat dialogue, and to obtain feedback information of the user, and extract the feedback information from the user's habits and preferences Related content, saving the related content to the feature information set.
The interactive robot according to any one of claims 8 to 10, wherein, when the item to be supplemented is a psychological attribute, the information perfecting module further comprises a testing module, wherein the testing module is configured to acquire a psychology from the cloud server. Testing the problem, completing the psychological test question through the display interface or using the voice to ask the user to complete the psychological test, the information perfecting module is further configured to send the completed psychological test to the cloud server, and receive the cloud The server saves the analysis result to the feature information set for the analysis result returned by the psychological test.
The interactive robot according to any one of claims 8 to 10, wherein when the item to be supplemented is related person information, the method further includes an extracting module and a determining module.

The extracting module is configured to acquire user voice information and video information, extract an associated person appearing in the voice information, save the related person to the feature information set, and extract an associated person appearing in the video information to save the related person to the Characteristic information set

The determining module is configured to determine a relevance of the associated person;

When the related person whose relevance exceeds the set threshold is asked, the information perfecting module for completing the related person questioning is further used to acquire the site information of the user and generate a user interaction parameter; when the interaction parameter is appropriate, according to the locked associated person The communication scene information is determined from the communication scene library, and the user is actively asked to combine the voice and the image to complete the feature information of the locked associated person.
The interactive robot according to claim 13, wherein the information perfecting module further comprises a pre-judging module, wherein the pre-judging module is configured to:

The number of times all statistically identified related characters appear;

Scan the number of occurrences of each associated person, compare it with the set number of thresholds, and determine whether the current associated person needs to improve the feature information.
An electronic device, comprising:

At least one processor; and,

a memory communicatively coupled to the at least one processor, a communication component, and an audio data collector And a video data collector; wherein

The memory stores instructions executable by the at least one processor, the instructions being invoked by the at least one processor to invoke data of an audio data collector and a video data collector, and establishing a connection with a cloud server through a communication component To enable the at least one processor to perform the method of any of claims 1-7.
A non-transitory computer readable storage medium, wherein the computer readable storage medium stores computer executable instructions for causing a computer to perform the method of any of claims 1-7 method.
A computer program product, comprising: a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when the program instructions are executed by a computer, The computer performs the method of any of claims 1-7.