CN111931897B - Interaction method, interaction device, electronic equipment and storage medium - Google Patents

Interaction method, interaction device, electronic equipment and storage medium Download PDF

Info

Publication number
CN111931897B
CN111931897B CN202010616400.0A CN202010616400A CN111931897B CN 111931897 B CN111931897 B CN 111931897B CN 202010616400 A CN202010616400 A CN 202010616400A CN 111931897 B CN111931897 B CN 111931897B
Authority
CN
China
Prior art keywords
user
interaction
value
willingness
feedback
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010616400.0A
Other languages
Chinese (zh)
Other versions
CN111931897A (en
Inventor
朱维峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Priority to CN202010616400.0A priority Critical patent/CN111931897B/en
Publication of CN111931897A publication Critical patent/CN111931897A/en
Priority to PCT/CN2021/099191 priority patent/WO2022001606A1/en
Application granted granted Critical
Publication of CN111931897B publication Critical patent/CN111931897B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Robotics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The embodiment of the application provides an interaction method, an interaction device, electronic equipment and a storage medium, which are applicable to man-machine interaction scenes in the field of Artificial Intelligence (AI), wherein the interaction method comprises the following steps: collecting an image set of a user; acquiring a user behavior parameter sequence according to an image set of a user; according to the user behavior parameter sequence, obtaining an interaction willingness value of a user through an interaction willingness value model; when the user meets the condition of active interaction according to the interaction wish value of the user, outputting interaction content according to the history interaction information of the user. According to the embodiment of the application, the interactive content can be output according to the historical interactive information of the user instead of outputting the same content to all users, so that different active interactive contents can be output for different users, further user experience is improved, and the adhesiveness with the users is improved. The electronic equipment in the embodiment of the application can be a robot, can realize the interaction of adults, children and the like in the aspects of entertainment, education and the like, and realizes personification accompanying.

Description

Interaction method, interaction device, electronic equipment and storage medium
Technical Field
The embodiment of the application relates to an artificial intelligence technology, in particular to an interaction method, an interaction device, electronic equipment and a storage medium.
Background
Robots are increasingly used as tools for interacting with users. For example, in a family life, a child may interact with a robot, such as to conduct a conversation, entertainment, or study, when the parent is unable to accompany the child.
Currently, when a user interacts with a robot, the user is required to actively initiate the interaction. For example, after the user speaks the voice, the robot detects the voice of the user, and then parses the voice of the user and performs a corresponding operation. In order to solve the problem of low user experience caused by passive response of the robot, the prior art also provides a technical scheme of active interaction with the user by the robot, and the robot can output fixed active interaction content to realize active interaction with the user.
However, in the mode of active interaction between the robot and the user in the prior art, the output active interaction content is single, and the user experience is low.
Disclosure of Invention
The embodiment of the application provides an interaction method, an interaction device, electronic equipment and a storage medium, which can output different active interaction contents aiming at different users, so that the user experience is improved, and the adhesiveness with the users is improved.
In a first aspect, an embodiment of the present application provides an interaction method, where the method may be applied to an intelligent device, or may also be applied to a chip in the intelligent device. The method is described below by taking an intelligent device as an example, in the method, an image set of a user can be acquired, a user behavior parameter sequence is acquired according to the image set of the user, and then an interaction willingness value of the user is obtained through an interaction willingness value model according to the user behavior parameter sequence, wherein the interaction willingness value is used for representing the interaction willingness of the user and the intelligent device. And when the user meets the condition of active interaction according to the interaction willingness value of the user, outputting the current interaction content according to the historical interaction information of the user.
It should be appreciated that the user's image set includes a plurality of images, and that the user behavior parameters may be obtained in each image, with the addition of the user behavior parameters in each image being a sequence of user behavior parameters. In one possible implementation, the sequence of user behavior parameters may be user behavior parameters in images arranged in time order of acquisition. In the embodiment of the present application, the user behavior parameter may represent the behavior of the user, and for example, the user behavior parameter may be any one of the following: the face angle of the user, the distance between the user and the intelligent equipment and the action of the user. The actions of the user may include facial actions, limb actions, and the like of the user.
According to the embodiment of the application, the user behavior parameter sequence can be input into the interaction willingness value model to obtain the interaction willingness value of the user. And if the interactive willingness value is smaller, the interactive willingness of the user and the intelligent equipment is weaker. In one possible implementation manner, the condition that the user meets the active interaction may be determined according to the interaction wish value and the interaction wish threshold of the user. For example, if the interaction wish value of the user is greater than the interaction wish threshold, determining that the user satisfies the condition of active interaction. Or if the interaction willingness value of the user is smaller than the interaction willingness threshold value, determining that the user meets the condition of active interaction. Wherein, the condition that the user satisfies the active interaction can be preset. It should be understood that the interaction willingness value model in the embodiment of the application may be a neural network model, where an interaction data sample and labeling information of the interaction data sample may be used as training data to train to obtain the interaction willingness value model, and the labeling information of the interaction data sample characterizes the interaction willingness value of the user in the interaction data sample.
The historical interaction information in the embodiment of the application can comprise the historical interaction content of the user and the intelligent equipment, wherein when the user meets the condition of active interaction, the current interaction content can be output according to the historical interaction information of the user. In view of the fact that the same content is not output to all users in the embodiment of the application, different active interaction contents can be output to different users in the embodiment of the application, the user experience can be improved, and the adhesiveness with the users is further improved. In addition, in the embodiment of the application, whether the user meets the condition of active interaction is determined according to the image set of the user instead of the single image, and the accuracy of determining that the user meets the condition of active interaction can be improved.
It should be noted that, in the embodiment of the present application, if the image in the image set includes a plurality of users, the current interactive content may be output according to the historical interaction information of the user with the largest interaction wish value. It should be noted that, the method for outputting the current interactive content in the embodiment of the present application is described below by taking "outputting the current interactive content according to the history interactive information of the user" as an example.
In order to save the electric quantity of the intelligent device, the intelligent device in the embodiment of the application can also start to collect the image set of the user when detecting that the surrounding user exists. That is, when the user is detected, the images of the user are acquired every preset time period to acquire the images of the preset number of users.
In an embodiment of the present application, in a possible implementation manner, the user behavior parameter sequence may include: and the face angle of the user and the distance between the user and the intelligent equipment in each image set are set. Correspondingly, in the embodiment of the application, the face angle of the user and the distance between the user and the intelligent device in each image can be obtained in the image set, and then the interaction wish value of the user is obtained through the interaction wish value model according to the face angle of the user and the distance between the user and the intelligent device in each image.
In the possible implementation manner, in order to more accurately determine whether the user has an active interaction wish, the embodiment of the application can further determine the interaction wish value of the user by combining the actions of the user in the image set. In the embodiment of the application, the face angle of the user and the distance between the user and the intelligent device in each image can be input into the interaction wish value model to obtain the initial interaction wish value of the user, and then the actions of the user are acquired in the image set. If the actions of the user comprise preset actions, a preset interaction willingness value is added on the basis of the initial interaction willingness value, and the interaction willingness value of the user is obtained; and if the action of the user does not comprise the preset action, taking the initial interaction willingness value as the interaction willingness value of the user.
By way of example, the preset action may be a hand-in action, a nodding action, or the like. In the embodiment of the application, on the basis of the interactive willingness value of the user according to the interactive willingness value model, if the preset action of the user is detected, the user can be determined to have stronger interactive willingness, and the initial interactive willingness value can be increased on the basis of the initial interactive willingness value.
The following describes how to output the current interactive content according to the historical interactive information of the user in the technical scheme of the application: in the embodiment of the application, the preference degree of the user for each interaction type can be obtained according to the historical interaction information of the user. And outputting the current interaction content according to the preference degree of the user for each interaction type. It should be understood that the degree of preference of the user for each interaction type refers to: the preference degree of the user for the interaction type of the interaction content indicates that the user likes or dislikes the interaction content of the interaction type.
The feedback tendency of the user for each interaction type can be obtained according to the feedback content of the user for each interaction type, and the feedback tendency comprises positive feedback, negative feedback and no feedback; and determining the preference degree of the user on each interaction type according to the initial preference degree of the user on each interaction type and the feedback tendency of the user on each interaction type. It should be noted that, in the embodiment of the present application, the positive feedback is that the user likes the interaction type, the negative feedback is that the user dislikes the interaction type, and no feedback indicates that the user does not respond to the interaction type, i.e. does not show a like or dislike of the interaction type. It should be understood that the feedback content in the embodiment of the present application may include any of the following: the user's voice, limb movements, facial movements.
When the intelligent device outputs the current interaction content, the intelligent device can output the current interaction of the interaction type with the highest preference degree of the user. In the embodiment of the application, in order to avoid that the content actively input by the intelligent device each time is the content of the interaction type with the highest preference degree of the user, traversing all types of content, determining the current target interaction type according to the historical interaction information of the user and the preference degree of the user on each interaction type; and randomly outputting the interactive content of the target interactive type. According to the method and the device for determining the target interaction type, the current target interaction type can be determined according to the interaction type of the interaction content output when the user actively interacts with the history of the user and the preference degree of the user for each interaction type.
It should be noted that for active interaction with a user, if the response time of the smart device is shorter, for example, the user notices the smart device slightly, the smart device may have an active interaction response, whereas if the response time of the smart device is longer, for example, the user observes the smart device for a while, the smart device does not react, and the smart device is not intelligent enough. Therefore, in the embodiment of the application, after the target interaction type is determined, the current interaction content can be output after a period of interval time, wherein the determination of the interval time is critical, and the user experience is directly influenced.
In the embodiment of the application, the interval duration before outputting the current interactive content can be determined according to the historical interactive information of the user, and then after the target interactive type is determined, the interval duration is spaced, and the current interactive content is output. It should be noted that in the embodiment of the present application, the initial interval duration may be preset, and the initial interval duration may be adjusted according to the historical interaction information between the user and the intelligent device. The feedback tendency of the user in each historical interaction can be obtained according to the feedback content of the user in each historical interaction, and the feedback tendency comprises positive feedback, negative feedback and no feedback; and determining the interval duration before outputting the interactive content according to the feedback tendency of the user in each time of the history interaction and the initial interval duration.
In the embodiment of the application, when the intelligent device actively interacts with the user, the current interaction content can be output by combining the emotion of the intelligent device. By way of example, if the emotion of the intelligent device is good, the intelligent device and the user are triggered to perform active interaction more easily, and if the emotion of the intelligent device is bad, the intelligent device and the user are difficult to trigger to perform active interaction, so that the intelligent device is more personified. If the emotion of the intelligent device is good, the interaction willingness threshold value is small, and the intelligent device and the user are easier to trigger to actively interact on the premise that the interaction willingness values of the users are the same. Similarly, if the emotion of the intelligent device is good, the interaction willingness threshold value is large, and on the premise that the interaction willingness values of the users are the same, the intelligent device is more difficult to trigger to actively interact with the users.
According to the historical interaction information of the user, the emotion value of the intelligent device can be obtained, the emotion type of the intelligent device is determined according to the emotion value of the intelligent device, and then a threshold value corresponding to the emotion type of the intelligent device is used as the interaction willingness threshold value. It should be noted that in the embodiment of the present application, an initial emotion value of the intelligent device may be preset, and then the initial emotion value may be adjusted according to the interaction history information of the user and the intelligent device. The intelligent device can acquire feedback trends of the user and the intelligent device according to feedback content of historical interaction of the user and the intelligent device, and further acquire emotion values of the intelligent device according to the feedback trends of the user and the intelligent device and the initial emotion values of the intelligent device so as to determine the interaction willingness threshold.
In a possible implementation manner, after outputting the current interactive content, the embodiment of the application may further acquire feedback content of the user based on the interactive content, and store the feedback content of the user based on the interactive content, so as to use the feedback content as the historical interactive information in the next interaction.
In a second aspect, an embodiment of the present application provides an interaction device, including: and the acquisition module is used for acquiring the image set of the user. The processing module is used for acquiring a user behavior parameter sequence according to the image set of the user; and obtaining an interaction willingness value of the user through an interaction willingness value model according to the user behavior parameter sequence, and outputting current interaction content according to historical interaction information of the user when the user meets the condition of active interaction according to the interaction willingness value of the user, wherein the interaction willingness value is used for representing the interaction willingness of the user and the intelligent equipment.
In one possible implementation manner, the processing module is specifically configured to determine that the user satisfies the condition of active interaction according to the interaction wish value and the interaction wish threshold of the user.
In one possible implementation, the set of images includes images of a preset number of users. The acquisition module is specifically used for acquiring images of the users at intervals of preset time periods when the users are detected, so as to acquire images of a preset number of users.
In one possible implementation, the sequence of user behavior parameters includes: the face angle of the user and the distance of the user from the intelligent device in each image in the image set.
The processing module is specifically used for acquiring the face angle of the user and the distance between the user and the intelligent equipment in each image in the image set; and obtaining an initial interaction willingness value of the user by inputting the face angle of the user in each image and the distance between the user and the intelligent equipment into the interaction willingness value model.
In one possible implementation manner, the processing module is specifically configured to input a face angle of a user in each image and a distance between the user and the intelligent device into the interaction willingness value model to obtain an initial interaction willingness value of the user; acquiring actions of a user in the image set; if the actions of the user comprise preset actions, adding a preset interaction wish value on the basis of the initial interaction wish value to obtain the interaction wish value of the user; and if the actions of the user do not comprise the preset actions, taking the initial interaction willingness value as the interaction willingness value of the user.
In one possible implementation manner, the processing module is specifically configured to output the current interaction content according to the historical interaction information of the user with the largest interaction wish value if the image in the image set includes a plurality of users.
In one possible implementation manner, the processing module is further configured to obtain an emotion value of the intelligent device according to the historical interaction information of the user; determining the emotion type of the intelligent equipment according to the emotion value of the intelligent equipment; and taking a threshold value corresponding to the emotion type of the intelligent equipment as an interaction willingness threshold value.
In one possible implementation manner, the processing module is specifically configured to obtain a feedback tendency of the user when interacting with the intelligent device according to feedback content of the user when interacting with the intelligent device, where the feedback tendency includes positive feedback, negative feedback and no feedback; and acquiring the emotion value of the intelligent device according to the feedback tendency of the user when the user interacts with the intelligent device and the initial emotion value of the intelligent device.
In one possible implementation manner, the processing module is specifically configured to obtain a preference degree of the user for each interaction type according to the historical interaction information of the user; and outputting the current interaction content according to the preference degree of the user for each interaction type.
In one possible implementation manner, the processing module is specifically configured to determine a current target interaction type according to the historical interaction information of the user and the preference degree of the user for each interaction type; and randomly outputting the interactive content of the target interactive type.
In one possible implementation, the historical interaction information further includes: the user feeds back content for each interaction type. The processing module is also used for obtaining feedback tendency of the user for each interaction type according to the feedback content of the user for each interaction type, wherein the feedback tendency comprises positive feedback, negative feedback and no feedback; and determining the preference degree of the user for each interaction type according to the initial preference degree of the user for each interaction type and the feedback tendency of the user for each interaction type.
In one possible implementation manner, the processing module is specifically configured to determine an interval duration before outputting the current interactive content according to the historical interaction information of the user; and outputting the current interactive content by spacing the spacing duration.
In one possible implementation, the historical interaction information of the user includes: the user has feedback content at each interaction. The processing module is specifically used for acquiring the feedback tendency of the user in each historical interaction according to the feedback content of the user in each historical interaction, wherein the feedback tendency comprises positive feedback, negative feedback and no feedback; and determining the interval duration before outputting the current interaction content according to the feedback tendency of the user in each interaction and the initial interval duration.
In one possible implementation, the feedback content includes any of the following: the user's voice, limb movements, facial movements.
In a possible implementation manner, the processing module is further configured to obtain feedback content of the user based on the current interactive content. And the storage module is used for storing the feedback content of the user based on the current interaction content.
In one possible implementation manner, the processing module is further configured to use an interaction data sample and labeling information of the interaction data sample as a training sample, train to obtain the interaction willingness value model, and characterize the interaction willingness value of the user in the interaction data sample by the labeling information of the interaction data sample.
The technical effects of the interaction device provided in the embodiment of the present application may refer to the technical effects of the method of the first aspect, which are not described herein.
In a third aspect, an embodiment of the present application provides an electronic device, which may be an intelligent device in the first aspect. The electronic device includes: a processor, a memory, a transceiver; the transceiver is coupled to the processor, and the processor controls the transceiving actions of the transceiver; wherein the memory is for storing computer executable program code, the program code comprising instructions; the instructions, when executed by a processor, cause the electronic device to perform the method as provided in the first aspect.
In a fourth aspect, embodiments of the present application provide an electronic device comprising means, modules or circuits for performing the method provided by the above possible designs of the first aspect. The electronic device may be an intelligent device, or may be a module applied to the intelligent device, for example, may be a chip applied to the intelligent device.
In a fifth aspect, an embodiment of the present application provides a chip on which a computer program is stored which, when executed by the chip, implements a method as provided in the first aspect.
In a sixth aspect, embodiments of the present application provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the method of the first aspect described above.
In a seventh aspect, embodiments of the present application provide a computer readable storage medium having instructions stored therein which, when run on a computer, cause the computer to perform the method of the first aspect described above.
The embodiment of the application provides an interaction method, an interaction device, electronic equipment and a storage medium, wherein the method comprises the following steps: collecting an image set of a user; acquiring a user behavior parameter sequence according to an image set of a user; according to the user behavior parameter sequence, obtaining an interaction willingness value of a user through an interaction willingness value model; when the condition that the user meets the active interaction is determined according to the interaction wish value of the user, outputting the current interaction content according to the historical interaction information of the user. In the embodiment of the application, on one hand, whether the user meets the condition of active interaction is determined according to the image set of the user instead of the single image, so that the accuracy of determining that the user meets the condition of active interaction can be improved. On the other hand, in the embodiment of the application, when the condition that the user meets the active interaction is determined, the current interaction content can be output according to the historical interaction information of the user. Instead of outputting the same content for all users, different active interaction contents can be output for different users in the embodiment of the application, so that user experience can be improved, and further, the adhesiveness with the users can be improved.
Drawings
Fig. 1 is a schematic view of a scenario where an interaction method provided by an embodiment of the present application is applicable;
FIG. 2 is a schematic diagram of an interaction;
FIG. 3 is a second interaction diagram;
fig. 4 is a schematic structural diagram of an intelligent device according to an embodiment of the present application;
FIG. 5 is a flow chart of an embodiment of an interaction method according to an embodiment of the present application;
FIG. 6 is a schematic diagram of acquiring training data according to an embodiment of the present application;
FIG. 7 is a first schematic diagram of interaction provided by an embodiment of the present application;
FIG. 8 is a second interaction diagram according to an embodiment of the present application;
FIG. 9 is a third interaction diagram according to an embodiment of the present application;
FIG. 10 is a flowchart illustrating another embodiment of an interaction method according to an embodiment of the present application;
FIG. 11 is a flowchart illustrating another embodiment of an interaction method according to an embodiment of the present application;
FIG. 12 is a flowchart of another embodiment of an interaction method according to an embodiment of the present application;
fig. 13 is a schematic structural diagram of an interaction device according to an embodiment of the present application.
Detailed Description
Fig. 1 is a schematic view of a scenario where the interaction method provided by the embodiment of the present application is applicable. As shown in fig. 1, the scene includes: user, smart device and server. The intelligent device can be an electronic device which interacts with a user, such as an intelligent sound box, a robot, a terminal device and the like. It should be understood that the intelligent device in the embodiment of the application can also interact with the server, so that the intelligent device can analyze the voice input by the user through the server, and in addition, the server can also feed back the interaction content, such as songs, stories and the like, to the intelligent device by adopting the database. In fig. 1, an intelligent device is taken as an example of a robot.
The terminal device in the embodiment of the present application may refer to a user device, an access terminal, a user unit, a subscriber station, a remote terminal, a mobile device, a user terminal, a wireless communication device, a user agent, or a user apparatus. The terminal device may be a mobile phone (mobile phone), a tablet (pad), a computer with a wireless transceiver function, a session initiation protocol (session initiation protocol, SIP) phone, a personal digital assistant (personal DIGITAL ASSISTANT, PDA), a handheld device with a wireless communication function, a computer or other processing device, a vehicle-mounted device, a wearable device, a Virtual Reality (VR) terminal device, an augmented reality (augmented reality, AR) terminal device, a wireless terminal in a smart home (smart home), a terminal device in a future 5G network or a terminal device in a future evolved public land mobile network (public land mobile network, PLMN), etc., which the embodiments of the present application are not limited.
Fig. 2 is a schematic diagram of the first interaction. When the user interacts with the intelligent device, the user actively initiates the interaction. For example, after the user speaks the wake-up word of the intelligent device, the user can wake up the intelligent device, and further, the user can speak the own requirements, such as "ask weather", "play song", "play video", "learn", and the like. As shown in fig. 2, the user speaks "how is today weather? When the method is used, the intelligent equipment can analyze the voice of the user to obtain the semantics of the user, and then feed back the user, such as reply of' Beijing weather cloudiness today, 18 ℃ -25 ℃. The interaction mode requires the user to actively initiate interaction, the intelligent device responds passively, the user asks for a sentence, and the intelligent device answers the sentence. The intelligent device is not intelligent enough, resulting in low user experience.
In order to solve the problem of passive response of the intelligent equipment, the technical scheme of active interaction of the intelligent equipment and a user is also provided. Fig. 3 is a second interaction diagram. As shown in fig. 3, the smart device may actively interact with the user, for example, the smart device may actively play the voice "hi", the host may relax the bar "for playing a song, and if the user replies" good ", the smart device starts playing the song. The intelligent device outputs fixed preset interaction content for different users so as to realize active interaction with the users. The user shown in fig. 3 is an adult, and the active interactive content played may be of interest to the adult user. However, if the user is a child or an old person, the voice cannot be used for inducing the interest of the user, and the user can also be used for inducing the dislike. Therefore, when the intelligent device actively interacts with the user, the output active interaction content is single, the active interaction content is disjointed with the behavior of the user, the user is not informed about the requirement of the user according to the use habit of the user, the interaction is initiated in a repeated similar mode, the user does not have the interest of continuing to interact with the robot after a period of time, the user experience is low, and the adhesion degree with the user is low.
In order to solve the technical problems, the embodiment of the application provides an interaction method, when the intelligent equipment determines that the user needs active interaction, the intelligent equipment can output the content of the active interaction by combining the historical interaction information of the user. Because different users interact with the smart device with different content. If a child user interacts with the intelligent device history, playing a fairy tale story in multiple demands; when teenager users interact with the intelligent equipment history, popular music is played in multiple demands; when the middle-aged user interacts with the intelligent device history, the news of the current affairs is played in multiple demands. According to the embodiment of the application, different interactive contents can be output aiming at different users when the users actively interact with the user, and the output contents are interesting contents of the user, so that the success probability of the active interaction with the user can be improved, and the adhesiveness with the user is further improved.
The following first describes the structure of the smart device in the embodiment of the present application. Fig. 4 is a schematic structural diagram of an intelligent device according to an embodiment of the present application. As shown in fig. 4, the smart device 400 may include: processor 401, memory 402, wireless communication module 403, audio module 404, microphone 405, sensor 406, camera 407, display 408, etc. It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the smart device 400. In other embodiments of the application, the smart device 400 may include more or less components than illustrated, or certain components may be combined, or certain components may be split, or different arrangements of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
The processor 401 may include one or more processing units, such as: the processor 401 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (IMAGE SIGNAL processor, ISP), a controller, a video codec, a digital signal processor (DIGITAL SIGNAL processor, DSP), a baseband processor, a display processing unit (display process unit, DPU), and/or a neural-network processor (neural-network processing unit, NPU), etc. Wherein the different processing units may be separate devices or may be integrated in one or more processors. In some embodiments, the smart device 400 may also include one or more processors 401. The processor may be a neural hub and a command center of the smart device 400, among others. The controller can generate operation control signals according to the instruction operation codes and the time sequence signals to finish the control of instruction fetching and instruction execution. A memory may also be provided in the processor 401 for storing instructions and data. In some embodiments, the memory in the processor 401 is a cache memory. The memory may hold instructions or data that has just been used or recycled by the processor 401. If the processor 401 needs to reuse the instruction or data, it can be called directly from the memory. This avoids duplicate accesses and reduces the latency of the processor 401, thereby improving the efficiency of the smart device 400.
In some embodiments, the processor 401 may include one or more interfaces. The interfaces may include an integrated circuit (inter-INTEGRATED CIRCUIT, I2C) interface, an integrated circuit built-in audio (inter-INTEGRATED CIRCUIT SOUND, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous receiver transmitter (universal asynchronous receiver/transmitter, UART) interface, a mobile industry processor interface (mobile industry processor interface, MIPI), a general-purpose input/output (GPIO) interface, a subscriber identity module (subscriber identity module, SIM) interface, and/or a universal serial bus (universal serial bus, USB) interface, among others. It should be understood that the connection relationship between the modules illustrated in the embodiment of the present application is only illustrative, and does not limit the structure of the smart device 400. In other embodiments of the present application, the smart device 400 may also use different interfacing manners, or a combination of multiple interfacing manners in the foregoing embodiments.
Memory 402 may be used to store one or more computer programs, including instructions. The processor 401 may cause the smart device 400 to perform the relevant actions in the embodiments described below by executing instructions stored in the memory 402. The memory 402 may include a stored program area and a stored data area. The storage program area can store an operating system; the storage area may also store one or more applications (e.g., gallery, contacts, etc.), and so forth. The storage data area may store data created during use of the smart device 400 (e.g., photos, contacts, etc.), and so on. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash memory (universal flash storage, UFS), and the like. In some embodiments, the processor 401 may cause the smart device 400 to perform various functional applications and data processing by executing instructions stored in the memory 402, and/or instructions stored in a memory provided in the processor 401.
The wireless communication function of the smart device 400 may be implemented by the wireless communication module 403. The wireless communication module 403 may provide solutions for wireless communication including wireless local area network (wireless local area networks, WLAN), bluetooth, global navigation satellite system (global navigation SATELLITE SYSTEM, GNSS), frequency modulation (frequency modulation, FM), NFC, infrared (IR), etc. applied on the smart device 400. The wireless communication module 403 may be one or more devices integrating at least one communication processing module. The wireless communication module 403 in the embodiment of the present application is configured to implement a transceiver function of an electronic device, for example, to implement communication with the server in fig. 1.
The smart device 400 may implement audio functions such as music playing, recording, etc. through an audio module 404, a microphone 405, etc. Wherein the audio module 404 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 404 may also be used to encode and decode audio signals. In some embodiments, the audio module 404 may be disposed in the processor 401, or a part of functional modules of the audio module 404 may be disposed in the processor 401. The smart device 400 may be provided with at least one microphone 405. In other embodiments, the smart device 400 may be provided with two microphones 405, and may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the smart device 400 may also be provided with three, four, or more microphones 405 to enable collection of sound signals, noise reduction, identification of sound sources, directional recording functions, etc.
The sensors 406 may include a pressure sensor 406A, a distance sensor 406B/proximity sensor 406C, and the like. The pressure sensor 406A is used for sensing a pressure signal, and can convert the pressure signal into an electrical signal. In some embodiments, a pressure sensor 406A may be disposed on the display 408, and the smart device 400 detects the touch operation intensity according to the pressure sensor 406A. A distance sensor 406B for measuring distance. The smart device 400 may measure the distance by infrared or laser. The proximity light sensor 406C may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The smart device 400 emits infrared light outwards through the light emitting diode. The smart device 400 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object in the vicinity of the smart device 400. When insufficient reflected light is detected, the smart device 400 may determine that there is no object in the vicinity of the smart device 400.
The smart device 400 may implement a photographing function through one or more cameras 407. In addition, the smart device 400 may implement display functionality through the display 408. The display 408 is used to display images, videos, and the like. The display 408 includes a display panel. The display panel may employ a Liquid Crystal Display (LCD) CRYSTAL DISPLAY, an organic light-emitting diode (OLED), an active-matrix organic LIGHT EMITTING diode (AMOLED), a flexible light-emitting diode (FLED), miniled, microLed, micro-oLed, a quantum dot LIGHT EMITTING diode (QLED), or the like. In some embodiments, the smart device 400 may include 1 or N displays 408, N being a positive integer greater than 1.
In the embodiment of the present application, the distance sensor 406B/proximity light sensor 406C may detect whether a user exists around the smart device, the camera may collect an image set of the user, and the processor 401 is configured to execute actions in S502-S504, S1002-S1006, and S1202-S1207 in the following embodiments. The interaction method in the following embodiments may be implemented based on the smart device shown in fig. 4, and specific technical solutions and technical effects refer to the relevant descriptions of the following embodiments. It should be understood that the configuration in fig. 4 is illustrated as an example of an intelligent device, and in the following embodiments, an execution body for executing the interaction method is illustrated as the intelligent device, and each module shown in fig. 4 is integrated in the intelligent device, so that functions of each module can be implemented.
The interaction method provided by the embodiment of the application is described below with reference to specific embodiments. The following embodiments may be combined with each other, and some embodiments may not be repeated for the same or similar concepts or processes. Fig. 5 is a flowchart of an embodiment of an interaction method according to an embodiment of the present application. As shown in fig. 5, the interaction method provided by the embodiment of the present application may include:
S501, collecting an image set of a user.
S502, acquiring a user behavior parameter sequence according to an image set of a user.
And S503, obtaining an interaction willingness value of the user through an interaction willingness value model according to the user behavior parameter sequence, wherein the interaction willingness value is used for representing the interaction willingness of the user and the intelligent equipment.
S504, when the condition that the user meets the active interaction is determined according to the interaction wish value of the user, outputting the current interaction content according to the historical interaction information of the user.
In S501, the intelligent device may collect images in real time, and if a user is identified in the collected images, the user may be tracked, and an image set of the user is collected. Optionally, in the embodiment of the present application, the intelligent device may use a user identification model to identify whether the image has a user. The user identification model is used for representing the corresponding relation between the characteristics of the user and the images, the user identification model can be a neural network model, and the training data can be images of different users. The intelligent device adopts the user identification model, so that whether a user exists in the acquired image or not can be determined, and when the user exists in the image, an image set of the user can be acquired.
The intelligent device can track the user in the image, and further collect a plurality of images of the user to obtain an image set of the user. Optionally, in the embodiment of the present application, a kalman filtering algorithm, a particle filtering algorithm, etc. may be used to track the user. Optionally, after the user is identified in the image, the embodiment of the application may continuously shoot a plurality of images without tracking the user so as to obtain an image set of the user.
The image set of the user may include a plurality of images of the user. The image in the image set of the user may include a face image of the user, a whole body image of the user, a half body image of the user, or the like. Therefore, the intelligent device can acquire the facial actions, limb actions and the like of the user according to the image set of the user.
It should be noted that, in order to save the power of the smart device, the smart device in the embodiment of the present application may also start to collect the image set of the user when detecting that the user exists around. The manner in which the smart device detects the presence of a user in the surroundings may be referred to above in relation to the distance sensor 306B/proximity light sensor 306C.
In the embodiment of the application, the image set of the user can comprise images of a preset number of users. When the intelligent device detects the user, the image of the user can be acquired once every preset time length so as to acquire images of a preset number of users. For example, the preset time period is 60ms, and the preset number is 5. When the intelligent device detects a user, an image of the user can be acquired, and then the images of the user are acquired at 120ms, 180ms, 240ms and 300ms, and 5 images are acquired, so that an image set of the user is obtained. It should be understood that the image in the image set in the embodiment of the present application is an image including the face and limbs of the user.
In the above S502, in the embodiment of the present application, after the image set of the user is collected, the intelligent device may obtain the user behavior parameter sequence according to the image set of the user. The image set comprises a plurality of images, and a set of user behavior parameters in each image is a user behavior parameter sequence.
The user behavior parameters may include any one of the following: the face angle of the user, the distance between the user and the intelligent equipment and the action of the user. The actions of the user may include facial actions, limb actions, and the like of the user.
In S503, in the embodiment of the present application, the interaction wish value of the user may be obtained through the interaction wish value model according to the user behavior parameter sequence. The interaction willingness value is used for representing interaction willingness of the user and the intelligent equipment. It should be understood that, if the interactive willingness value is larger, the interactive willingness of the user and the intelligent device is represented to be stronger, and if the interactive willingness value is smaller, the interactive willingness of the user and the intelligent device is represented to be weaker.
The construction process of the interactive willingness value model in the embodiment of the application is described first:
The method for constructing the interaction willingness value model by adopting training data is provided at present: in the intelligent equipment setting and public places, the interactive data of the user and the intelligent equipment is collected, and the interactive data is training data. On the one hand, interaction between the user and the intelligent equipment in the public place is unnatural, so that difference exists between the collected interaction data and the interaction data between the actual user and the intelligent equipment, and the accuracy of the interaction willingness value model is further affected. On the other hand, after the intelligent device collects the interaction data, staff is required to label the interaction data one by one, and more manpower is required to be consumed.
Fig. 6 is a schematic diagram of acquiring training data according to an embodiment of the present application. As shown in fig. 6, in the embodiment of the present application, an experiment site similar to a practical application process of a user, such as an experiment site similar to a home environment, may be preset, and an intelligent device may be set in the experiment site. The person to be collected, such as the child in fig. 6, is facing the smart device, and the data collector may be located opposite or on the side to be collected, so as to determine whether the user has an interactive intention by observing the actions, voices, etc. of the person to be collected. In the embodiment of the application, topics related to children can be preset, and the intelligent equipment collects contents such as voice, action and the like of the user in the process of interaction between the intelligent equipment and the collected person. It should be understood that, in order to correspond to the following, in the embodiment of the present application, the smart device may acquire the face angle of the user and the distance between the smart device. The data collector can mark the data as having interactive willingness on site if the data collector determines that the collected person has interactive willingness according to the feedback of the collected person, and can mark the data as having no interactive willingness on site if the data collector determines that the collected person has no interactive willingness.
In the embodiment of the application, a button can be preset on the intelligent device, and when the data collector presses the button for a long time, the intelligent device determines that the collected person has an interactive intention, so that the intelligent device can mark the collected data as 1, and the characterization data is the data of the collected person having the interactive intention. When the data collector presses the button for a short time or does not press the button, the intelligent device determines that the collected person does not have interaction wish, so that the intelligent device can mark the collected data as 0, and the characterization data is the data that the collected person does not have interaction wish. In the embodiment of the application, if the acquired person considers that the acquired person can judge whether the acquired person has interaction wish or not, the button can be pressed by the acquired person so as to trigger the intelligent device to mark the acquired data. Alternatively, if the volume of the intelligent device is not large, two data collectors may be provided, and the two data collectors may be located at positions around the intelligent device, and when the two data collectors consider that the collected people have interaction intention, the button may be pressed.
In the embodiment of the application, because the intelligent equipment collects data in the simulated experimental field, more real and accurate data can be collected, and the accuracy of the interaction willingness value model is further improved. In the embodiment of the application, the data (such as the user behavior parameters) collected by the intelligent equipment can be used as an interaction data sample, the labeling information (such as the labeling of 0 or 1) of the interaction data sample is used as a training sample, the interaction willingness value model is obtained through training, and the labeling information of the interaction data sample represents the interaction willingness value of the user in the interaction data sample.
In the embodiment of the application, a deep learning mode can be adopted, and the interactive willingness value model is obtained by training with training data, and the training process is not repeated in the embodiment of the application. It should be understood that the interaction wish value model in the embodiment of the application characterizes the mapping relation of the face angle, the distance from the intelligent device and the interaction wish value, that is, the face angle of the user in one image and the distance from the user to the intelligent device are input into the interaction wish value model, so that the interaction wish value of the user can be obtained.
In the embodiment of the application, the user behavior parameter sequence can be input into the interactive willingness value model, and the interactive willingness value of the user can be obtained.
In S504, in the embodiment of the present application, when it is determined that the user satisfies the condition of active interaction according to the interaction wish value of the user, the current interaction content may be output according to the historical interaction information of the user. Wherein, whether the user meets the condition of active interaction can be determined according to the interaction willingness value and the interaction willingness threshold value of the user. For example, if the interaction wish value of the user is greater than the interaction wish threshold, determining that the user satisfies the condition of active interaction. Or if the interaction willingness value of the user is smaller than the interaction willingness threshold value, determining that the user meets the condition of active interaction. The condition that the user satisfies the active interaction can be preset.
In the embodiment of the application, if the intelligent equipment determines that the user meets the condition of active interaction, the intelligent equipment can output the interaction content according to the historical interaction information of the user. The historical interaction information of the user may be: interactive voice, interactive actions and the like of the user in the process of historic interaction with the intelligent device, wherein the interactive actions can be facial actions or limb actions. Taking an example of a user interaction with the smart device, the smart device plays the voice "find Song A for you," the user speaks "not so, not good, and makes a" pan ". Accordingly, "the played voice of the intelligent device, and the voice and actions of the user" are historical interaction information in one interaction process. In this embodiment, the history interaction information of the user may be all the history interaction information of the user and the intelligent device, or may be all the history interaction information of the user and the intelligent device in a period of time, for example, the history interaction information of the user may be the history interaction information of the last month or the last week.
The intelligent device can store historical interaction information of a plurality of users and identification of each user. The identification of the user may be a facial image of the user, a morphological feature of the user, etc. After the intelligent device collects the image set of the user, if the image in the image set of the user comprises the face image of the user, the intelligent device can compare the face image with the stored face images of a plurality of users to determine the historical interaction information of the user. Or after the intelligent device collects the image set of the user, if the image in the image set of the user comprises limb actions of the user, the intelligent device can acquire the physical characteristics of the user in the image according to the image, and then compare the physical characteristics of the user with the stored physical characteristics of a plurality of users so as to determine the historical interaction information of the user.
The embodiment of the application can preset active interaction content with various interaction types, such as a calling interaction type, a puzzle interaction type, a music playing interaction type, a topic chat initiation interaction type, a dancing interaction type and the like. Multiple different content may also be included in the active interaction content for each interaction type. Active interaction content, such as the type of call interaction, may include: "hi, today weather is good, how does your mood? "," what's up, man ", or" today is a good day, let us relax a bar ", etc.
In one possible implementation, the smart device may output current interaction content of the user-preferred interaction type according to the user-preferred interaction type. Illustratively, if the user prefers to guess the type of puzzle interaction, a puzzle can be played. Wherein, the interaction type preferred by the user can be the interaction type with the most interaction types with the user.
In one possible implementation, the smart device may also play the current interaction content of the type of interaction that last interacted with the user. For example, if the type of interaction the user last interacted with the user is a play music interaction type, the current interaction content of the play music interaction type may be output.
Fig. 7 is a schematic diagram of interaction provided in an embodiment of the present application. For example, as shown in fig. 7, when the smart device detects that a surrounding user exists, the user is an adult, if the smart device determines that the user meets the condition of active interaction, the smart device may output the current interaction content of the interaction type preferred by the adult, such as outputting the voice "hi," how good the weather today, how good your mood is? ".
Fig. 8 is a second interaction diagram according to an embodiment of the present application. For example, as shown in fig. 8, when the smart device detects that a user exists around, the user is a child, if the smart device determines that the user meets the condition of active interaction, the smart device may output the current interaction content of the interaction type preferred by the child, such as playing a child song.
In the embodiment of the application, if the image in the image set collected by the intelligent device comprises a plurality of users, the intelligent device can output the current interaction content according to the history interaction information of the user with the largest interaction wish value after acquiring the interaction wish value of each user according to the mode.
Fig. 9 is a third interaction diagram according to an embodiment of the present application. Illustratively, as shown in FIG. 9, there are two users around the smart device, one a child and one an adult. After the intelligent device obtains the interaction wish value of the child and the adult according to the mode, the intelligent device determines that the interaction wish value of the child is larger, and then the intelligent device can output current interaction content, such as playing a baby song, according to the history interaction information of the child.
The interaction method provided by the embodiment of the application comprises the following steps: collecting an image set of a user; acquiring a user behavior parameter sequence according to an image set of a user; according to the user behavior parameter sequence, obtaining an interaction willingness value of a user through an interaction willingness value model; when the condition that the user meets the active interaction is determined according to the interaction wish value of the user, outputting the current interaction content according to the historical interaction information of the user. In the embodiment of the application, on one hand, whether the user meets the condition of active interaction is determined according to the image set of the user instead of the single image, so that the accuracy of determining that the user meets the condition of active interaction can be improved. On the other hand, when the condition that the user meets the active interaction is determined, the current interaction content can be output according to the historical interaction information of the user instead of outputting the same content for all users.
Based on the foregoing embodiments, how to obtain the interactive wish value of the user in the embodiment of the present application is described in detail below with reference to fig. 10. Fig. 10 is a flowchart of another embodiment of an interaction method according to an embodiment of the present application. As shown in fig. 10, the interaction method provided by the embodiment of the present application may include:
s1001, collecting an image set of a user.
S1002, in the image set, the face angle of the user and the distance between the user and the intelligent device in each image are obtained.
S1003, obtaining the interaction willingness value of the user through an interaction willingness value model according to the face angle of the user in each image and the distance between the user and the intelligent equipment.
S1004, according to the historical interaction information of the user, the preference degree of the user for each interaction type is obtained.
S1005, outputting the current interaction content according to the preference degree of the user for each interaction type.
The implementation in S1001 may refer to the related description in the foregoing embodiment S501, which is not described herein.
In S1002, the face angle of the user in the image refers to an included angle between the face angle in the image and the face of the user when the user faces the smart device. It should be understood that in the embodiment of the application, the images of the user with the face angles of the user being the angles can be stored in advance, and the intelligent device determines the face angle of the user in each image in the image set by comparing the face in the image set with the stored images of the user with the angles.
In addition, in the embodiment of the application, the pixel size of the head of the user when the user is at different distances from the intelligent device can be stored in advance, and then the distance between the user and the intelligent device in each image in the image set is determined according to the pixel size of the head of the user in the image set and the pixel size of the head of the user when the user is at different distances from the intelligent device. It should be understood that the manner of acquiring the face angle of the user and the distance between the user and the intelligent device in the image shown here is an example, and the embodiment of the application may also acquire the face angle of the user and the distance between the user and the intelligent device in the image according to other manners, which is not limited.
In the above S1003 embodiment of the present application, the face angle of the user in each image and the distance between the user and the intelligent device may be input into the interaction wish value model, so as to obtain the initial interaction wish value of the user.
The intelligent device can input the face angle of the user in each image and the distance between the user and the intelligent device into the interaction wish value model to obtain an initial interaction wish value of the user. Illustratively, the example is described in which the image set in the above embodiment includes 5 images, face angles of the user in the 5 images are 5 °,6 °,5 °,7 ° and 5 °, and distances between the user in the 5 images and the smart device are 600mm, 590mm, 610mm and 580mm, respectively, and in this embodiment, a vector x 0 of 1×10 of [5, 600,6, 600,5, 590,7, 610,5, 580] may be input into the interaction intent value model, and the vector x 0 represents [ angle, distance ] in the 5 images.
The interactive willingness value model in the embodiment of the application can calculate the initial interactive willingness value of the user by adopting three layers of the neural network with the small parameter number. Wherein, this interactive willingness value model includes: the first hidden layer, the second hidden layer and the output layer. Wherein, the first hidden layer has 10 neurons, the matrix calculation of the first layer is that x 0×H1+b1,H1 is 10×10, b 1 is a 1×10 vector, and then the tanh () calculation is performed to obtain a new 1×10 vector x 1. The second hidden layer has 6 neurons, the matrix calculation of the second layer is that x 1×H2+b2,H2 is 10×6, b 2 is a 1×10 vector, and a new 1×6 vector x 2 is obtained through tanh () calculation. The output layer, x 2×H3+b3,H3, is a 6 x 1 matrix and b 3 is a number. And the numerical value calculated through the tanh () is the final output value, namely the initial interaction wish value of the user. The initial interaction willingness value of the user is between 0 and 1, and the larger the initial interaction willingness value of the user is, the higher the tendency of the interaction willingness of the user and the intelligent equipment is represented.
In the embodiment of the application, the intelligent equipment can determine the initial interaction willingness value of the user by adopting the interaction willingness value model, and in order to more accurately determine whether the user has the active interaction willingness, the interaction willingness value of the user is further determined by combining the action of the user in the image. If the actions of the user in the images in the image set comprise the preset actions, the preset interaction wish value is added on the basis of the initial interaction wish value, and the interaction wish value of the user is obtained; and if the actions of the user do not comprise the preset actions, taking the initial interaction willingness value as the interaction willingness value of the user. The preset action may be a hand-engaging action, a nodding action, or a hook-comparing action. For example, the preset interaction willingness value is 0.2, the initial interaction willingness value of the user, which is obtained by the intelligent device through adopting the interaction willingness value model, is 0.7, and the interaction willingness value of the user is 0.7+0.2 and is 0.9 when the actions of the user in the image set include the action of the user. That is, in the embodiment of the application, the intelligent device acquires the initial interaction intention value of the user in a mode of adopting an interaction intention value model, and in addition, acquires the final interaction intention value of the user by combining the action of the user in the image of the image set.
In the above S1004, in the embodiment of the present application, the smart device may obtain the preference degree of the user for each interaction type according to the historical interaction information of the user. Wherein, the history interaction information further includes: the user feeds back content for each interaction type, which may be preset. Taking an interaction process of the user and the intelligent device as an example, the intelligent device plays a voice of ' for finding song A ' for you ', the user speaks ' not the voice, is inaudible ', and makes a motion of ' shaking head ', and accordingly, the feedback content of the user on the type of music interaction played is ' not the voice, is inaudible ', and the motion of ' shaking head '.
The intelligent device can store historical interaction information of the user, and the intelligent device can acquire feedback tendency of the user for each interaction type according to feedback content of the user for each interaction type. Feedback trends include positive feedback, negative feedback, and no feedback, among others. It should be understood that, in the embodiment of the present application, the positive feedback is that the user likes the interaction type, the negative feedback is that the user dislikes the interaction type, and no feedback indicates that the user does not respond to the interaction type, i.e. does not show the like or dislike of the interaction type.
For example, if the feedback content of the user on the music playing interaction type is an expression of "good hearing", "happy", a cheering action, etc., the music playing interaction type is characterized as favorite, and the feedback of the user is positive feedback. If the feedback content of the user for playing the music interaction type is "not the same, bad listening", "shaking head" and the like, the user does not like to play the music interaction type, and the feedback of the user is negative feedback.
It should be understood that, in the embodiment of the present application, the feedback content of the user includes any one of the following: user's voice, limb movements, facial movements. The intelligent device can determine whether the user is positively fed back, negatively fed back or without feedback according to the feedback content of the user. Examples of voice, limb actions and facial actions fed back by the user positively, examples of voice, limb actions and facial actions fed back by the user negatively, and examples of voice, limb actions and facial actions without feedback by the user can be stored in the intelligent device. It should be appreciated that when the voice, limb motion, or facial motion in the positive feedback example is included in the feedback content of the user, the feedback content of the user may be determined to be positive feedback, and when the voice, limb motion, or facial motion in the negative feedback example is included in the feedback content of the user, the feedback content of the user may be determined to be negative feedback.
The following table one is an example of positive feedback and negative feedback for a user:
List one
Content of forward feedback Content of negative feedback
And (3) voice: lovely, smart and playful " And (3) voice: do not want, get restless, quiet "
Limb movement: jumping, heart comparing and stroking intelligent equipment Limb movement: intelligent device capable of acting, swinging and patting in a specific ratio of X
Facial action: grin laugh and spit tongue Facial action: is to turn over the eyes, skim the mouth, turn over the white eyes
In the embodiment of the application, each interaction type is provided with an initial preference degree, and the initial preference degree is set according to big data fed back by a user or experience of staff. In the process of interaction between the intelligent device and the user, the initial preference degree of each interaction type can be adjusted according to the feedback tendency of the user on each interaction type, so that the preference degree of the current user on each interaction type is obtained.
Optionally, in the embodiment of the present application, the preference degree of the user for each interaction type may be adjusted once according to the feedback tendency of the smart device in 10 interaction processes of interaction with the user, where the preset preference degree δ=0.03 may be preset, and the addition and subtraction value of the preference degree of the user for each interaction type may be determined according to the feedback tendency of the user for each interaction type in 10 interaction processes of interaction with the smart device and the user. The method can increase delta for the interaction type with the largest forward feedback times, reduce the preference degree of other interaction types by delta/(N-1), and N is the number of the interaction types. It should be noted that the degree of preference for each interaction type has an upper and lower limit. When the preference degree of one interaction type is presumably beyond the upper limit, the preference degree of the interaction type is not continuously increased, and when the preference degree of the interaction type is reached to the lower limit, the preference degree of the interaction type is not continuously reduced, wherein the sum of the preference degrees of the interaction types is 1, so that the interaction content of all the interaction types can be triggered.
Exemplary types of interactions for the smart device include: the method comprises four active interaction modes of calling interaction type, puzzle-guessing interaction type, topic chat interaction type initiation and dancing interaction type, wherein the initial preference degrees of the four interaction types are respectively 0.4, 0.1 and 0.4, delta=0.03, and N is 4. The upper and lower limits of the preference degree of the calling interaction type, the puzzle interaction type, the topic chat initiation interaction type and the dancing interaction type are (0.5,0.3), (0.2,0.04), (0.2,0.04) and (0.5,0.3) respectively.
The last 10 times the user had feedback to the positive and negative categories of each category were (2, 1), (1, 0), (2, 2), respectively, and the last 5 times the user had feedback to the type of initiating topic chat interaction was (none, negative, positive). At the current moment, the user looks at the intelligent equipment to be in a fool, and the intelligent equipment calculates that the interactive willingness value of the user is always above 0.8, namely the user has interactive willingness. The smart device decides to initiate an active interaction and the user's preference for each current interaction type is [0.44,0.06,0.08,0.42] respectively.
The intelligent device actively initiates the interactive content of the topic chat interactive type to the user, says' hi, zhang three, how to always look at me. The user responds to the smart device and starts chat. After this interaction, a positive feedback is given by the user. Of the types of interactions of the user to initiate topic chat, 3 positive feedback occurs in 5 of only 10 interactions. Therefore, the preference degree of the calling interaction type, the puzzle-guessing interaction type, the topic chat interaction type and the dancing interaction type can be adjusted to be [0.43,0.05,0.07,0.45], the accumulated times of the topic chat interaction type are cleared, and the next time is counted from 0.
In addition, if the type of the calling interaction, the type of the puzzle interaction, the type of the topic chat interaction, and the preference degree of the type of the dancing interaction are respectively [0.43,0.04,0.07,0.46] and the preference degree of the type of the topic chat interaction is still to be initiated, at this time, the type of the puzzle interaction has reached the lower limit, so the probability of 0.01 which should be reduced here is subtracted by 0.01 in the preference degree of the type of the topic chat interaction, the total probability is maintained to be 1, and the updated preference degree is respectively [0.42,0.04,0.06,0.48].
In the foregoing S1005, in one possible implementation manner, in the embodiment of the present application, the intelligent device may output the current interaction content of the interaction type with the highest user preference, for example, if the interaction type with the highest user preference is the call interaction type, the intelligent device may output the active interaction content of the call interaction type, that is, the current interaction content.
In order to avoid that the content actively input by the intelligent device each time is the content of the interaction type with the highest preference degree of the user, the embodiment of the application can also output the interaction content of other interaction types. The intelligent device can determine the current target interaction type according to the historical interaction information of the user and the preference degree of the user for each interaction type. According to the method and the device for determining the target interaction type, the current target interaction type can be determined according to the interaction type of the interaction content output when the user actively interacts with the history of the user and the preference degree of the user for each interaction type.
Exemplary, the interaction types of the interaction content output during the user history active interaction are respectively: 2 times of calling interaction type, 1 time of puzzle interaction type, 2 times of music playing interaction type, 1 time of topic chat initiating interaction type and 1 time of dancing interaction type. According to the preference degree sum of 1, the intelligent device should include 4 times of calling interaction types, 4 times of riddle interaction types, 2 times of music playing interaction types, 1 time of topic initiating chat interaction types and 1 time of dancing interaction types when outputting 10 times of active interaction content. According to the interaction type of the output interaction content when the current user actively interacts with the history, the intelligent device can be determined to output when the intelligent device interacts with the user actively: 2 call interaction types and 1 puzzle interaction type. Therefore, the application can take the calling interaction type or the puzzle-guessing interaction type as the current target interaction type.
After determining the current target interaction type, the intelligent device can output the interaction content of the target interaction type randomly. It should be understood that, because each interaction type may correspond to a plurality of interaction contents, in the embodiment of the present application, the smart device may randomly output the interaction contents of the target interaction type, such as the smart device may output the current interaction contents of the call-in interaction type or the current interaction contents of the puzzle interaction type.
At present, when the intelligent device passively responds to a user, the quicker and better the response is, namely the shorter the response time is, the more intelligent the intelligent device is. For active interaction with a user, if the response time of the smart device is shorter, if the user notices the smart device slightly, the smart device may have an active interaction response, and if the response time of the smart device is longer, if the user observes the smart device for a while, the smart device does not react, and the smart device is not intelligent enough. Therefore, in the embodiment of the application, after the intelligent device determines the interactive content of the target interactive type which is randomly output according to the description, the current interactive content can be output after a period of interval time, wherein the determination of the interval time is critical, and the user experience is directly influenced.
In the embodiment of the application, the intelligent device can determine the interval duration before outputting the current interactive content according to the historical interactive information of the user, and then can output the current interactive content at intervals after determining the interactive content of the target interactive type which is randomly output. For example, if the intelligent device determines that the user meets the condition of active interaction and has determined to output the interaction content of the type of initiating the topic chat interaction, the interaction content of the type of initiating the topic chat interaction may be output after the interval duration.
The following describes how the intelligent device determines the interval duration before outputting the current interactive content:
Similar to the above determination of the preference degree of the user for each interaction type by the intelligent device, in the embodiment of the present application, the historical interaction information of the user includes: the intelligent device can acquire the feedback tendency of the user in each time of the history according to the feedback content of the user in each time of the history, and further determine the interval duration before outputting the current interaction content according to the feedback tendency of the user in each time of the history and the initial interval duration. Wherein, the feedback tendency, feedback content and the like of the user can be referred to the related description.
For example, the initial interval duration is 12s, after the intelligent device determines that the user meets the condition of active interaction, and has determined to output the interaction content of the type of initiating topic chat interaction, after waiting for 12s, the intelligent device outputs the interaction content of the type of initiating topic chat interaction (i.e. the current interaction content). In the embodiment of the application, when the user has 5 or more forward feedback in 10 interactions with the intelligent device, the initial time can be subtracted by 4s; if there are 3 to 5 forward feedbacks out of 10, the initial interval duration may be subtracted by 2s; if no more than 2 positive feedback is available in 10 times, the initial interval duration is not adjusted. When the user has negative feedback for 5 times or more in 10 interactions with the intelligent device, the initial interval duration can be added with 4s; if there are 3 to 5 negative feedback times in 10 times, the initial interval duration can be added with 2s; if no more than 2 negative feedback is available in 10 times, the initial interval duration is not adjusted. It should be noted that in the embodiment of the present application, the upper limit of the interval duration may be 8s, and the lower limit may be 8s.
For example, in the above example, 3 forward feedback occurs in approximately 10 interactions between the user and the smart device, so that the next time the user is determined to satisfy the condition of active interaction, and after the current interaction content has been determined, the smart device will start to perform active interaction with the user only at intervals of 10 s.
In the embodiment of the application, the intelligent device can adopt the interactive willingness value model to determine the initial interactive willingness value of the user, can accurately determine whether the user has interactive willingness, and further determines the interactive willingness value of the user by combining the actions of the user in the image set, thereby improving the accuracy of whether the user has active interactive willingness. In addition, in the embodiment of the application, the current target interaction type can be determined according to the preference degree of the user on each interaction type. In addition, when the current interactive content is output, the current interactive content can be output at intervals, so that the user does not feel that the interaction is very abrupt, the user does not feel that the interaction process is not intelligent enough, the interval is obtained according to the historical interaction information of the user and the intelligent device, the time for outputting the current interactive content is more accurate, and the user experience can be improved.
Fig. 11 is a flowchart of another embodiment of an interaction method according to an embodiment of the present application. As shown in fig. 11, in the embodiment of the present application, after executing the above S1005, the method may further include:
S1006, acquiring feedback content of the user based on the current interaction content, and storing the feedback content of the user based on the current interaction content.
After the intelligent device outputs the current interactive content, the intelligent device can acquire feedback content of the user based on the current interactive content and store the feedback content of the user based on the current interactive content. When the intelligent device and the user conduct the initiative textbook next time, the feedback content of the user based on the current interaction content can be used as the history interaction information, so that the preference degree of the user for each interaction type, the interval duration of outputting the current interaction content and the like can be updated.
In the embodiment of the application, the intelligent equipment continuously adopts the historical interaction information of the user and the intelligent equipment to update the preference degree of the user on each interaction type, the interval duration of outputting the current interaction content and the like, so that the intelligent equipment is more suitable for the habit of the user when the intelligent equipment performs active interaction, and the user experience can be further improved.
When the intelligent device plays voice, although emotion exists in the mood and expression, the emotion of the intelligent device is not gathered to interact in the process of interaction with a user. For example, as smart device is playing the voice "hi, today weather is good, how does your mood? While "when a user is able to use cool and dazzling speech, if he/she just beats the smart device during his/her interaction, his/her emotion should fall very low, but the current user speaks" put a happy song "and the smart device speaks" good "with a positive happy emotion. The front emotion and the back emotion of the intelligent equipment are not consistent, the intelligent equipment at present adopts positive emotion to interact with a user, the intelligent equipment is not enough anthropomorphic in expression, is more like equipment without emotion, and cannot accompany like people.
In the embodiment of the application, when the intelligent device actively interacts with the user, the current interaction content can be output by combining the emotion of the intelligent device. If the emotion of the intelligent device is good, the intelligent device is easier to trigger to actively interact with the user, and if the emotion of the intelligent device is bad, the intelligent device is harder to trigger to actively interact with the user. It should be noted that, in the embodiment of the present application, the emotion of the intelligent device affects the interaction willingness threshold in the above embodiment, where if the emotion of the intelligent device is good, the interaction willingness threshold is small, and on the premise that the interaction willingness values of the users are the same, the intelligent device is easier to trigger to actively interact with the users. Similarly, if the emotion of the intelligent device is good, the interaction willingness threshold value is large, and on the premise that the interaction willingness values of the users are the same, the intelligent device is more difficult to trigger to actively interact with the users.
Fig. 12 is a flowchart of another embodiment of an interaction method according to an embodiment of the present application. As shown in fig. 12, the interaction method provided by the embodiment of the present application may include:
S1201, an image set of the user is acquired.
S1202, acquiring face angles of users and distances from the users to the intelligent equipment in each image in the image set.
And S1203, inputting the face angle of the user in each image and the distance between the user and the intelligent device into an interaction willingness value model to obtain an initial interaction willingness value of the user.
S1204, acquiring the interaction wish value of the user according to the initial interaction wish value of the user and the actions of the user in the image set.
S1205, obtaining the emotion value of the intelligent equipment, and determining the emotion type of the intelligent equipment according to the emotion value of the intelligent equipment.
And S1206, taking a threshold value corresponding to the emotion type of the intelligent equipment as an interaction willingness threshold value.
S1207, according to the interaction willingness value and the interaction willingness threshold value of the user, if the user is determined to meet the condition of active interaction, outputting the current interaction content according to the preference degree of the user for each interaction type.
It should be understood that the implementation manners in S1201, S1202-S1204, S1207 in the embodiments of the present application may refer to the relevant descriptions in S501, S1002-S1003, S1004-S1005 in the above embodiments. The embodiment of the application does not limit the sequence of S1202-S1203 and S1205-S1206, and the two can be executed simultaneously.
In S1205, the interaction willingness threshold value for triggering the active interaction may be determined according to the emotion of the intelligent device. The intelligent device is provided with an initial emotion value, and the intelligent device can acquire feedback tendency of the user when interacting with the intelligent device according to feedback content of the user when interacting with the intelligent device, and further acquire the emotion value of the intelligent device according to the feedback tendency of the user when interacting with the intelligent device and the initial emotion value of the intelligent device.
For example, if the intelligent device has an initial emotion value at the beginning of each day, the intelligent device may obtain a feedback tendency of the user when interacting with the intelligent device on the same day according to the feedback content of the user when interacting with the intelligent device on the same day, and further obtain the emotion value of the intelligent device according to the feedback tendency of the user when interacting with the intelligent device on the same day and the initial emotion value of the intelligent device on the same day.
According to the embodiment of the application, the emotion value of the intelligent equipment can be set periodically, so that the initial emotion value of the intelligent equipment is different every day. For example, in the embodiment of the application, the initial emotion value of each day in each month of the intelligent device can be set with a period of one month, and the emotion values of each day of the intelligent device in one month are different. The feedback tendency of the user when interacting with the intelligent device on the same day is obtained, which is similar to the feedback tendency of the user when each interaction type or the feedback tendency of the user when each interaction is historic in the embodiment. Different, the feedback tendency of the user when interacting with the intelligent device on the same day is obtained in the embodiment of the application, and optionally, if the feedback tendency of the user when interacting with the intelligent device on the same day is positive feedback, a preset emotion value is added on the basis of the initial emotion value in the embodiment of the application; if the feedback tendency of the user when interacting with the intelligent device is negative feedback, reducing the preset emotion value on the basis of the initial emotion value so as to obtain the emotion value of the intelligent device; if the feedback trend of the user when interacting with the intelligent device is no feedback, the initial emotion value can be used as the emotion value of the intelligent device.
For example, the initial emotion value of the smart device is 1, the preset emotion value is 0.1, the user has 1 positive feedback and 2 negative feedback when interacting with the smart device on the same day, and the current emotion value of the smart device is 0.9.
In the embodiment of the application, the emotion types of the intelligent equipment can be classified into positive, calm and negative. Each emotion type corresponds to an emotion value range, and the intelligent device can determine the emotion type of the intelligent device according to the current emotion value. Illustratively, if the positive emotion value ranges from 0.7 to 1, the negative emotion value ranges from 0 to 0.3, and the rest emotion values are the emotion value ranges corresponding to calm. It is understood that the emotion value is equal to or greater than 0 and equal to or less than 1. The current emotion value of the intelligent device is 0.9, and the emotion type of the intelligent device can be determined to be positive.
In S1206, in the embodiment of the application, each emotion type of the intelligent device corresponds to a different threshold, namely, an interaction wish threshold. For example, the threshold corresponding to the emotion type of the intelligent device is 0.3 when the emotion type of the intelligent device is positive, the threshold corresponding to the emotion type of the intelligent device is negative, and the threshold corresponding to the emotion type of the intelligent device is 0.6 when the emotion type of the intelligent device is calm, and the threshold corresponding to the emotion type of the intelligent device is 0.4. If the emotion type of the intelligent device is forward, the threshold corresponding to the emotion type is 0.3 when the emotion type is forward, and the threshold can be used as the interaction wish threshold in the embodiment of the application.
If the interaction willingness value of the user is 0.5, the intelligent device and the user can be triggered to perform active interaction if the emotion type of the current intelligent device is positive or calm, but the intelligent device and the user cannot be triggered to perform active interaction if the emotion type of the intelligent device is negative. The intelligent device in the embodiment of the application has more personified expression, continuous emotion, and no active response user, brings accompany feeling to the user, and further improves user experience.
Fig. 13 is a schematic structural diagram of an interaction device according to an embodiment of the present application. The interaction means may be a server or a chip or a processor in a server or the like in the above embodiments. As shown in fig. 13, the interaction device includes: an acquisition module 1301, a processing module 1302 and a storage module 1303.
An acquisition module 1301 is configured to acquire an image set of a user.
A processing module 1302, configured to obtain a user behavior parameter sequence according to an image set of a user; and obtaining an interaction willingness value of the user through an interaction willingness value model according to the user behavior parameter sequence, and outputting current interaction content according to historical interaction information of the user when the user meets the condition of active interaction according to the interaction willingness value of the user, wherein the interaction willingness value is used for representing the interaction willingness of the user and the intelligent equipment.
In one possible implementation, the processing module 1302 is specifically configured to determine that the user satisfies the condition of active interaction according to the interaction intent value and the interaction intent threshold of the user.
In one possible implementation, the set of images includes images of a preset number of users. The acquisition module 1301 is specifically configured to acquire images of a user at intervals of a preset duration when the user is detected, so as to acquire images of a preset number of users.
In one possible implementation, the sequence of user behavior parameters includes: the face angle of the user and the distance of the user from the intelligent device in each image in the image set.
The processing module 1302 is specifically configured to obtain, in the image set, a face angle of a user and a distance between the user and the intelligent device in each image; and obtaining an initial interaction willingness value of the user by inputting the face angle of the user in each image and the distance between the user and the intelligent equipment into the interaction willingness value model.
In one possible implementation manner, the processing module 1302 is specifically configured to input the face angle of the user in each image and the distance between the user and the intelligent device into the interaction wish value model to obtain an initial interaction wish value of the user; acquiring actions of a user in the image set; if the actions of the user comprise preset actions, adding a preset interaction wish value on the basis of the initial interaction wish value to obtain the interaction wish value of the user; and if the actions of the user do not comprise the preset actions, taking the initial interaction willingness value as the interaction willingness value of the user.
In a possible implementation manner, the processing module 1302 is specifically configured to output the current interaction content according to the historical interaction information of the user with the largest interaction wish value if the image in the image set includes a plurality of users.
In one possible implementation, the processing module 1302 is further configured to obtain an emotion value of the intelligent device according to the historical interaction information of the user; determining the emotion type of the intelligent equipment according to the emotion value of the intelligent equipment; and taking a threshold value corresponding to the emotion type of the intelligent equipment as an interaction willingness threshold value.
In one possible implementation, the processing module 1302 is specifically configured to obtain, according to feedback content when the user interacts with the smart device, a feedback tendency when the user interacts with the smart device, where the feedback tendency includes positive feedback, negative feedback, and no feedback; and acquiring the emotion value of the intelligent device according to the feedback tendency of the user when the user interacts with the intelligent device and the initial emotion value of the intelligent device.
In one possible implementation, the processing module 1302 is specifically configured to obtain, according to the historical interaction information of the user, a preference degree of the user for each interaction type; and outputting the current interaction content according to the preference degree of the user for each interaction type.
In one possible implementation, the processing module 1302 is specifically configured to determine a current target interaction type according to the user history interaction information and the preference degree of the user for each interaction type; and randomly outputting the interactive content of the target interactive type.
In one possible implementation, the historical interaction information further includes: the user feeds back content for each interaction type. The processing module 1302 is further configured to obtain, for the feedback content of each interaction type, a feedback tendency of the user for each interaction type, where the feedback tendency includes positive feedback, negative feedback, and no feedback; and determining the preference degree of the user for each interaction type according to the initial preference degree of the user for each interaction type and the feedback tendency of the user for each interaction type.
In one possible implementation, the processing module 1302 is specifically configured to determine an interval duration before outputting the current interactive content according to the historical interaction information of the user; and outputting the current interactive content by spacing the spacing duration.
In one possible implementation, the historical interaction information of the user includes: the user has feedback content at each interaction. The processing module 1302 is specifically configured to obtain, according to feedback content of the user during each historical interaction, a feedback tendency of the user during each historical interaction, where the feedback tendency includes positive feedback, negative feedback, and no feedback; and determining the interval duration before outputting the current interaction content according to the feedback tendency of the user in each interaction and the initial interval duration.
In one possible implementation, the feedback content includes any of the following: the user's voice, limb movements, facial movements.
In a possible implementation manner, the processing module 1302 is further configured to obtain feedback content of the user based on the current interactive content. And the storage module 1303 is used for storing feedback content of the user based on the current interaction content.
In a possible implementation manner, the processing module 1302 is further configured to train the interaction willingness value model with the interaction data sample and the labeling information of the interaction data sample as training samples, where the labeling information of the interaction data sample characterizes the interaction willingness value of the user in the interaction data sample.
The interaction device provided by the embodiment of the application can execute the action of the server in the embodiment of the method, and the implementation principle and the technical effect are similar, and are not repeated here.
It should be noted that the above transceiver module may be actually implemented as a transceiver, or include a transmitter and a receiver. And the processing module can be realized in the form of software calling through the processing element; or in hardware. For example, the processing module may be a processing element that is set up separately, may be implemented in a chip of the above-mentioned apparatus, or may be stored in a memory of the above-mentioned apparatus in the form of program codes, and the functions of the above-mentioned processing module may be called and executed by a processing element of the above-mentioned apparatus. In addition, all or part of the modules can be integrated together or can be independently implemented. The processing element described herein may be an integrated circuit having signal processing capabilities. In implementation, each step of the above method or each module above may be implemented by an integrated logic circuit of hardware in a processor element or an instruction in a software form.
For example, the modules above may be one or more integrated circuits configured to implement the methods above, such as: one or more Application SPECIFIC INTEGRATED Circuits (ASICs), or one or more microprocessors (DIGITAL SIGNAL processors, DSPs), or one or more field programmable gate arrays (field programmable GATE ARRAY, FPGAs), etc. For another example, when a module above is implemented in the form of processing element scheduler code, the processing element may be a general purpose processor, such as a central processing unit (central processing unit, CPU) or other processor that may invoke the program code. For another example, the modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
The term "plurality" herein refers to two or more. The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone. In addition, the character "/" herein generally indicates that the front and rear associated objects are an "or" relationship; in the formula, the character "/" indicates that the front and rear associated objects are a "division" relationship.
It will be appreciated that the various numerical numbers referred to in the embodiments of the present application are merely for ease of description and are not intended to limit the scope of the embodiments of the present application.
It should be understood that, in the embodiment of the present application, the sequence number of each process does not mean that the execution sequence of each process should be determined by the function and the internal logic, and should not limit the implementation process of the embodiment of the present application.

Claims (18)

1. An interaction method, comprising:
Collecting an image set of a user;
acquiring a user behavior parameter sequence according to the image set of the user;
Obtaining an interaction willingness value of the user through an interaction willingness value model according to the user behavior parameter sequence, wherein the interaction willingness value is used for representing the interaction willingness of the user and intelligent equipment;
When the condition that the user meets the active interaction is determined according to the interaction wish value of the user, outputting current interaction content according to the historical interaction information of the user;
The determining that the user meets the condition of active interaction according to the interaction willingness value of the user comprises the following steps:
And determining that the user meets the condition of active interaction according to the interaction willingness value and the interaction willingness threshold value of the user, wherein the interaction willingness threshold value is related to a threshold value corresponding to the emotion type of the intelligent equipment, and the emotion type is determined based on the historical interaction information of the user.
2. The method of claim 1, wherein the set of images includes a preset number of images of the user, and wherein the acquiring the set of images of the user includes:
When the user is detected, the images of the user are acquired every other preset time so as to acquire the images of the preset number of users.
3. The method according to claim 1 or 2, wherein the sequence of user behavior parameters comprises: the face angle of the user and the distance between the user and the intelligent device in each image set are obtained by an interactive willingness value model according to the user behavior parameter sequence, and the method comprises the following steps:
Acquiring face angles of the users and the distance between the users and the intelligent equipment in each image in the image set;
And obtaining the interactive willingness value of the user through the interactive willingness value model according to the face angle of the user in each image and the distance between the user and the intelligent equipment.
4. The method according to claim 3, wherein the obtaining the interactive willingness value of the user through the interactive willingness value model according to the face angle of the user in each image and the distance between the user and the intelligent device comprises:
inputting the face angle of the user and the distance between the user and the intelligent equipment in each image into the interaction willingness value model to obtain an initial interaction willingness value of the user;
acquiring actions of the user in the image set;
if the actions of the user comprise preset actions, adding a preset interaction willingness value on the basis of the initial interaction willingness value to obtain the interaction willingness value of the user;
and if the action of the user does not comprise the preset action, taking the initial interaction willingness value as the interaction willingness value of the user.
5. The method according to any one of claims 1-2 and 4, wherein outputting the current interaction content according to the historical interaction information of the user comprises:
and if the images in the image set comprise a plurality of users, outputting the current interactive content according to the historical interactive information of the user with the largest interactive willingness value.
6. The method according to any one of claims 1-2, 4, wherein the method further comprises:
Acquiring an emotion value of the intelligent equipment according to the historical interaction information of the user;
determining the emotion type of the intelligent equipment according to the emotion value of the intelligent equipment;
and taking a threshold value corresponding to the emotion type of the intelligent equipment as the interaction willingness threshold value.
7. The method of claim 6, wherein the obtaining the emotion value of the smart device according to the historical interaction information of the user comprises:
according to the feedback content of the historical interaction of the user and the intelligent equipment, acquiring the feedback tendency of the interaction of the user and the intelligent equipment, wherein the feedback tendency comprises positive feedback, negative feedback and no feedback;
And acquiring the emotion value of the intelligent equipment according to the feedback tendency of the historical interaction of the user and the intelligent equipment and the initial emotion value of the intelligent equipment.
8. The method according to any one of claims 1-2 and 4, wherein outputting the current interaction content according to the historical interaction information of the user comprises:
acquiring the preference degree of the user for each interaction type according to the historical interaction information of the user;
and outputting the current interaction content according to the preference degree of the user for each interaction type.
9. The method of claim 8, wherein outputting the current interaction content according to the user's preference degree for each interaction type, comprises:
Determining a current target interaction type according to the historical interaction information of the user and the preference degree of the user for each interaction type;
and randomly outputting the interactive content of the target interactive type.
10. The method of claim 9, wherein the historical interaction information further comprises: the user feeds back the content of each interaction type; the step of obtaining the preference degree of the user for each interaction type according to the historical interaction information of the user comprises the following steps:
according to the feedback content of the user for each interaction type, acquiring the feedback tendency of the user for each interaction type, wherein the feedback tendency comprises positive feedback, negative feedback and no feedback;
and determining the preference degree of the user on each interaction type according to the initial preference degree of the user on each interaction type and the feedback tendency of the user on each interaction type.
11. The method of any of claims 1-2, 4, 7, 9-10, wherein the outputting the current interactive content comprises:
determining the interval duration before outputting the current interactive content according to the historical interactive information of the user;
and outputting the current interactive content by spacing the spacing duration.
12. The method of claim 10, wherein the user's historical interaction information comprises: the feedback content of the user in each interaction is historic; the determining the interval duration before outputting the current interactive content according to the historical interactive information of the user comprises the following steps:
According to the feedback content of the user in each historical interaction, the feedback tendency of the user in each historical interaction is obtained, wherein the feedback tendency comprises positive feedback, negative feedback and no feedback;
And determining the interval duration according to the feedback tendency of the user in each time of the history interaction and the initial interval duration.
13. The method according to claim 10 or 12, wherein the feedback content comprises any of the following: the user's voice, limb movements, facial movements.
14. The method of any one of claims 1-2, 4, 7, 9-10, 12, wherein the method further comprises:
acquiring feedback content of the user based on the interactive content;
and storing the feedback content of the user based on the interactive content.
15. The method of any one of claims 1-2, 4, 7, 9-10, 12, wherein the method further comprises:
And training the interaction data sample and the labeling information of the interaction data sample to obtain the interaction willingness value model, wherein the labeling information of the interaction data sample represents the interaction willingness value of a user in the interaction data sample.
16. An interactive apparatus, comprising:
The acquisition module acquires an image set of a user;
The processing module is used for acquiring a user behavior parameter sequence according to the image set of the user, obtaining an interaction willingness value of the user through an interaction willingness value model according to the user behavior parameter sequence, and outputting current interaction content according to the historical interaction information of the user when the user meets the condition of active interaction according to the interaction willingness value of the user, wherein the interaction willingness value is used for representing the interaction willingness of the user and intelligent equipment;
the processing module is further configured to:
And determining that the user meets the condition of active interaction according to the interaction willingness value and the interaction willingness threshold value of the user, wherein the interaction willingness threshold value is related to a threshold value corresponding to the emotion type of the intelligent equipment, and the emotion type is determined based on the historical interaction information of the user.
17. An electronic device, comprising: memory, processor, and transceiver;
the processor being operative to couple with the memory, read and execute instructions in the memory to implement the method of any one of claims 1-15;
The transceiver is coupled to the processor and is controlled by the processor to transmit and receive messages.
18. A computer-readable storage medium, characterized in that the computer storage medium stores computer instructions, which when executed by a computer, cause the computer to perform the method of any of claims 1-15.
CN202010616400.0A 2020-06-30 2020-06-30 Interaction method, interaction device, electronic equipment and storage medium Active CN111931897B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202010616400.0A CN111931897B (en) 2020-06-30 2020-06-30 Interaction method, interaction device, electronic equipment and storage medium
PCT/CN2021/099191 WO2022001606A1 (en) 2020-06-30 2021-06-09 Interaction method and apparatus, and electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010616400.0A CN111931897B (en) 2020-06-30 2020-06-30 Interaction method, interaction device, electronic equipment and storage medium

Publications (2)

Publication Number Publication Date
CN111931897A CN111931897A (en) 2020-11-13
CN111931897B true CN111931897B (en) 2024-06-28

Family

ID=73316779

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010616400.0A Active CN111931897B (en) 2020-06-30 2020-06-30 Interaction method, interaction device, electronic equipment and storage medium

Country Status (2)

Country Link
CN (1) CN111931897B (en)
WO (1) WO2022001606A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111931897B (en) * 2020-06-30 2024-06-28 华为技术有限公司 Interaction method, interaction device, electronic equipment and storage medium
CN112866066B (en) * 2021-01-07 2023-06-30 上海喜日电子科技有限公司 Interaction method, device, system, electronic equipment and storage medium
CN118494516A (en) * 2024-07-15 2024-08-16 比亚迪股份有限公司 Vehicle control method, cloud server, vehicle controller, client and vehicle

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868827A (en) * 2016-03-25 2016-08-17 北京光年无限科技有限公司 Multi-mode interaction method for intelligent robot, and intelligent robot

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108197115B (en) * 2018-01-26 2022-04-22 上海智臻智能网络科技股份有限公司 Intelligent interaction method and device, computer equipment and computer readable storage medium
CN108733209A (en) * 2018-03-21 2018-11-02 北京猎户星空科技有限公司 Man-machine interaction method, device, robot and storage medium
CN111931897B (en) * 2020-06-30 2024-06-28 华为技术有限公司 Interaction method, interaction device, electronic equipment and storage medium

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105868827A (en) * 2016-03-25 2016-08-17 北京光年无限科技有限公司 Multi-mode interaction method for intelligent robot, and intelligent robot

Also Published As

Publication number Publication date
WO2022001606A1 (en) 2022-01-06
CN111931897A (en) 2020-11-13

Similar Documents

Publication Publication Date Title
US11241789B2 (en) Data processing method for care-giving robot and apparatus
CN111931897B (en) Interaction method, interaction device, electronic equipment and storage medium
CN113986016B (en) Intelligent assistant
US11858118B2 (en) Robot, server, and human-machine interaction method
CN110609620B (en) Human-computer interaction method and device based on virtual image and electronic equipment
US20190206401A1 (en) System and method for intelligent initiation of a man-machine dialogue based on multi-modal sensory inputs
US8321221B2 (en) Speech communication system and method, and robot apparatus
CN112204564A (en) System and method for speech understanding via integrated audio and visual based speech recognition
CN109313935B (en) Information processing system, storage medium, and information processing method
US11074491B2 (en) Emotionally intelligent companion device
CN106774845B (en) intelligent interaction method, device and terminal equipment
WO2022121592A1 (en) Livestreaming interaction method and apparatus
JP2021068404A (en) Facial expression generation system for avatar and facial expression generation method for avatar
CN112651334A (en) Robot video interaction method and system
US11403289B2 (en) Systems and methods to facilitate bi-directional artificial intelligence communications
WO2023006033A1 (en) Speech interaction method, electronic device, and medium
CN112752159A (en) Interaction method and related device
Zhen-Tao et al. Communication atmosphere in humans and robots interaction based on the concept of fuzzy atmosfield generated by emotional states of humans and robots
CN110718119A (en) Educational ability support method and system based on wearable intelligent equipment special for children
US20230230293A1 (en) Method and system for virtual intelligence user interaction
US20240169319A1 (en) Information processing system, activity sensor, and non-transitory recording medium
CN117056822A (en) Sign language identification method and electronic equipment
CN117953902A (en) Man-machine interaction method, device and equipment
CN117765952A (en) Man-machine interaction method, device and equipment
JP2022006610A (en) Social capacity generation device, social capacity generation method, and communication robot

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant