TWI778477B - Interaction methods, apparatuses thereof, electronic devices and computer readable storage media - Google Patents

Abstract

The present disclosure provides interaction methods, apparatuses thereof, electronic devices and computer readable storage media. A method includes: receiving a first message from a client; obtaining, based on an instruction content included in the first message, driving data matching the instruction content; controlling a display interface of the client to play a response animation of an interactive object by using the driving data.

Description

Interactive method, device, electronic device and storage medium

The present disclosure relates to the field of computer technologies, and in particular, to an interaction method, an apparatus, an electronic device, and a storage medium.

With the rapid development of the Internet, live broadcasting has become an important information dissemination method. Due to the different time periods for different audiences to watch online live broadcasts, live broadcasters cannot live broadcast 24 hours a day to meet the needs of different audiences. This problem can be solved by using digital people for live broadcast, however, the interactive technology between the digital human anchor and the audience needs to be researched and developed.

According to an aspect of the present disclosure, an interaction method is provided, the method comprising: receiving a first message from a client; acquiring driving data matching the indication content based on indication content included in the first message; using the The driving data controls the display interface of the client to play the response animation of the interactive object.

With reference to any of the implementation manners provided in the present disclosure, the acquiring driving data matching the indication content based on the indication content included in the first message includes: acquiring response content for the indication content, the response content A response text is included; based on at least one target text included in the response text, the control parameters of the set action of the interactive object matching the target text are obtained.

With reference to any of the implementation manners provided in the present disclosure, the acquiring driving data matching the indication content based on the indication content included in the first message includes: acquiring response content for the indication content, the response content Including a phoneme sequence; obtaining the control parameters of the interactive object matching the phoneme sequence.

With reference to any of the embodiments provided in the present disclosure, the control parameters of the interactive object include a gesture control vector of at least one local area, and the acquiring the control parameters of the interactive object matching the phoneme sequence includes: controlling the phoneme sequence Perform feature coding to obtain a first coding sequence corresponding to the phoneme sequence; obtain a feature coding corresponding to at least one phoneme according to the first coding sequence; obtain at least one local area of the interactive object corresponding to the feature coding Attitude control vector.

With reference to any implementation manner provided in the present disclosure, the method further includes: sending instruction information including the response content to the client, so that the client displays the response content based on the instruction information.

With reference to any implementation manner provided in the present disclosure, the controlling the client to play the response animation of the interactive object on the display interface by using the driving data includes: sending the driving data of the interactive object to the the client, so that the client generates a response animation according to the driving data; controls the client to play the response animation in the display interface; or, based on the driving data, adjusts the virtual model parameters of the interactive object; With the adjusted virtual model parameters, a rendering engine is used to generate a response animation of the interactive object, and the response animation is sent to the client.

According to an aspect of the present disclosure, an interaction method is provided, the method comprising: in response to a user input operation from a client, sending a first message including indicating content to a server; In response to the second message, the response animation of the interactive object is played on the display interface of the client.

With reference to any implementation manner provided in the present disclosure, the indication content includes text content; the method further includes: displaying the text content in the client, and/or playing an audio file corresponding to the text content.

With reference to any implementation manner provided by the present disclosure, the displaying the text content in the client includes: generating bullet screen information of the text content; displaying the bullet screen information in a display interface of the client .

With reference to any implementation manner provided in the present disclosure, the second message includes a response text for the indicated content; the method further includes: displaying the response text in a display interface of the client, and/or, Determine and play the audio file corresponding to the response text.

With reference to any implementation manner provided in the present disclosure, the second message includes the driving data of the interactive object; the second message based on the response of the server to the first message is displayed on the client Playing the response animation of the interactive object in the interface includes: adjusting the virtual model parameters of the interactive object based on the driving data; based on the adjusted virtual model parameters, using a rendering engine to generate the response animation of the interactive object, and Displayed in the display interface of the client; wherein the driving data includes control parameters for the interactive object that match the phoneme sequence corresponding to the response text, and/or, are the same as those contained in the response text The at least one target text matches the control parameter for the set action of the interactive object.

With reference to any implementation manner provided by the present disclosure, the second message includes a response animation made by the interactive object to the indicated content.

With reference to any of the embodiments provided in the present disclosure, the user's input operation includes: the user follows the limb operation picture displayed in the display interface to make a corresponding human body gesture; in response to the user's input operation from the client , acquiring a user behavior image including the human body posture; identifying the human body posture information in the user behavior image; and driving the interactive object displayed on the display interface to respond based on the human body posture information.

With reference to any of the embodiments provided in the present disclosure, the driving the interactive object displayed on the display interface to respond based on the human body posture information includes: determining the relationship between the human body posture information and the human body posture in the limb operation screen. Matching degree; based on the matching degree, the interactive objects displayed on the display interface are driven to respond.

With reference to any of the implementation manners provided in the present disclosure, the driving the interactive object to respond based on the matching degree includes: instructing the interactive object displayed on the display interface to respond when the matching degree reaches a set condition A first response is issued, wherein the first response includes a body movement and/or voice prompt showing qualified posture; and displaying the next body operation screen; in the case that the matching degree does not reach the set condition, instruct the display interface The displayed interactive object makes a second response, wherein the second response includes displaying a body movement and/or a voice prompt of an unqualified posture; and keeping the current body operation screen displayed.

According to an aspect of the present disclosure, an interactive device is provided, the device includes: a receiving unit, configured to receive a first message from a client; indicating that the content matches; a driving unit, configured to use the driving data to control the display interface of the client to play the response animation of the interactive object.

With reference to any of the embodiments provided in the present disclosure, the obtaining unit is specifically configured to: obtain response content for the indication content, where the response content includes a response text; based on at least one target text included in the response text, Obtain the control parameters of the set action of the interactive object matching the target text.

With reference to any of the implementation manners provided in the present disclosure, the obtaining unit is configured to, based on the response content for the indication content obtained in the response text, the response content including a phoneme sequence; obtain a phoneme sequence matching the phoneme sequence. Control parameters of the interactive object.

With reference to any of the embodiments provided in the present disclosure, the control parameter of the interactive object includes a gesture control vector of at least one local area, and when the acquiring unit acquires the second control parameter of the interactive object matching the phoneme sequence, it is used for : perform feature coding on the phoneme sequence to obtain a first coding sequence corresponding to the phoneme sequence; obtain a feature coding corresponding to at least one phoneme according to the first coding sequence; obtain the interactive object corresponding to the feature coding The pose control vector of at least one local region of .

With reference to any of the implementation manners provided in the present disclosure, the apparatus further includes a sending unit, configured to send instruction information including a response content to the instruction content to the client, so that the client can base on the instruction information The content of the answer is displayed.

With reference to any implementation manner provided in the present disclosure, the driving unit is configured to: send the driving data of the interactive object to the client, so that the client generates a response animation according to the driving data; control the client Play the response animation in the display interface; or, based on the driving data, adjust the two-dimensional or three-dimensional virtual model parameters of the interactive object; based on the adjusted two-dimensional or three-dimensional virtual model parameters, use a rendering engine to generate the The response animation of the interactive object, and the response animation is sent to the client.

According to an aspect of the present disclosure, an interactive device is provided, the device includes: a sending unit, configured to send a first message including indicating content to a server in response to a user input operation from a client; a playing unit, configured to based on The second message that the server responds to the first message plays the response animation of the interactive object in the display interface of the client.

With reference to any implementation manner provided in the present disclosure, the indication content includes text content; the apparatus further includes a first display unit, configured to display the text content in the display interface of the client, and/or, determine and Play the audio file corresponding to the text content.

With reference to any of the implementation manners provided in the present disclosure, when the first display unit is used to display the text content in the client, it is specifically configured to: generate bullet screen information of the text content; The barrage information is displayed on the display interface of the terminal.

With reference to any implementation manner provided in the present disclosure, the second message includes a response text to the indicated content; the apparatus further includes a second display unit, configured to display the response in a display interface of the client text, and/or, determine and play the audio file corresponding to the response text.

With reference to any implementation manner provided in the present disclosure, the second message includes driving data of the interactive object; the playing unit is configured to: adjust the virtual model parameters of the interactive object based on the driving data; based on the adjustment After the virtual model parameters, the rendering engine is used to generate the response animation of the interactive object, and display it in the display interface of the client; wherein, the driving data includes matching with the phoneme sequence corresponding to the response text for the indicated content The control parameters for the interactive object, and/or the control parameters for the setting action of the interactive object that match at least one target text contained in the response text.

With reference to any implementation manner provided by the present disclosure, the second message includes a response animation made by the interactive object to the indicated content.

With reference to any of the embodiments provided in the present disclosure, the user's input operation includes that the user follows the limb operation picture displayed in the display interface to make a corresponding human body gesture; the generating unit is configured to: obtain the information including: The user behavior image of the human body posture; recognize the human body posture information in the user behavior image, and drive the interactive object displayed on the display interface to respond based on the human body posture information.

With reference to any of the embodiments provided in the present disclosure, the generating unit is specifically configured to: determine the degree of matching between the human body posture information and the human posture in the limb operation screen; and, based on the matching degree, drive the display interface to display The interactive object responds.

With reference to any implementation manner provided in the present disclosure, the generating unit is specifically configured to: instruct the interactive object displayed on the display interface to make a first response when the matching degree reaches a set condition, wherein the first response The response includes displaying body movements and/or voice prompts with qualified posture; and displaying the next body operation screen; in the case that the matching degree does not meet the set condition, instructing the interactive object displayed on the display interface to make a second response, Wherein, the second response includes displaying body movements and/or voice prompts of unqualified postures; and keeping the current body operation screen displayed.

According to an aspect of the present disclosure, an electronic device is provided, the device includes a memory and a processor, where the memory is used for storing computer instructions executable on the processor, and the processor is used for executing the computer instructions The interaction method proposed by any of the embodiments of the present disclosure is implemented at the same time.

According to an aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the interactive method provided by any embodiment of the present disclosure is implemented.

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

The term "and/or" in this article is only a relationship to describe related objects, which means that there can be three relationships, for example, A and/or B, which can mean that A exists alone, A and B exist at the same time, and B exists alone. three conditions. In addition, the term "at least one" herein refers to any combination of any one of a plurality or at least two of a plurality, for example, including at least one of A, B, and C, and may mean including those composed of A, B, and C. Any one or more elements selected in the collection.

Using several people as anchors, live broadcasts can be performed at any time period, and 24-hour uninterrupted live broadcasts can be achieved, meeting the different needs of different audiences for the time to watch live broadcasts. As the interactive objects of the users during the live broadcast, how to give timely feedback to the questions raised by the users and how to interact with the users in a vivid and natural way are urgent problems to be solved.

In view of this, the present disclosure provides an interactive solution that can be applied to any scenario involving interaction with virtual interactive objects, such as webcasting.

The interaction method proposed in the embodiments of the present disclosure may be applied to a terminal device or a server. The terminal device may be, for example, an electronic device with a client installed, such as a mobile phone or a tablet computer. The present disclosure does not limit the form of the terminal device. The client is, for example, a live video client, including a live video client, a somatosensory interactive client, and the like. The server can be any server that can provide processing capabilities for interactive objects.

Interactive objects can be any kind of interactive objects that can interact with users, which can be virtual characters, virtual animals, virtual objects, cartoon images, and other virtual images that can realize interactive functions. Interactive objects can be based on two-dimensional It can also be constructed based on a 3D virtual model, and interactive objects are obtained by rendering a 2D or 3D virtual model. The user may be a real person, a robot, or other smart devices. The interaction mode between the interactive object and the user may be an active interaction mode or a passive interaction mode.

Exemplarily, in a live video broadcast scenario, the display interface of the client can display animations of interactive objects, and the user can perform input operations in the client of the terminal device, such as inputting text, inputting voice, triggering actions, triggering buttons, etc. , to interact with interactive objects.

FIG. 1 is a flowchart illustrating an interaction method according to at least one embodiment of the present disclosure, and the interaction method can be applied to a server side. As shown in FIG. 1 , the method includes steps 101 to 103 .

In step 101, a first message from the client is received.

Exemplarily, the indication content carried in the first message may include information input by the user through an input operation performed by the client, and the user's input operation includes an input text operation, a voice input operation, an action trigger operation, and a keystroke. trigger actions, etc. The input information can be sent by the client to the server; or when the client sends the input information to the server, the input information can be directly displayed on the client. The form of the indication content carried in the first message includes, but is not limited to, text, voice, image (eg, facial expression, motion image), video, and the like. The specific form of the first message is related to the application scenario. For example, in a live video broadcast scenario, the client may be a client that supports the function of watching live video, and the first message may be collected by the client and sent after the user enters text content on the display interface. The first message carries The indication content is, for example, the input text content, and the indication content can be displayed in the display interface in the form of a bullet screen; another example, in the somatosensory interaction scenario, the first message can be collected on the client side. User behavior After the image is sent, the indication content carried in the first message is, for example, the collected user behavior image. Of course, in the specific implementation, the present disclosure does not limit the sending mechanism of the first message and the form of the indication content carried in the first message.

In step 102, based on the indication content included in the first message, drive data matching the indication content is acquired.

Exemplarily, the driving data includes one or more of voice-driven data, expression-driven data, and action-driven data. In an implementation manner, the drive data may be pre-stored in the server or other associated service servers, and after receiving the first message from the client Search from the associated service server to obtain the driving data matching the indicated content. In another implementation manner, the driving data may be generated according to the indication content, for example, by inputting the indication content into a pre-trained deep learning model to predict the driving data corresponding to the indication content.

In step 103, using the driving data, the display interface of the client is controlled to play the response animation of the interactive object.

In the embodiment of the present disclosure, the interactive object is obtained by rendering a virtual model, such as a two-dimensional or three-dimensional virtual model. The virtual model may be custom generated or obtained by converting an image or video of a character. The embodiments of the present disclosure do not limit the manner of generating the virtual model.

The response animation can be generated according to the drive data, and by controlling the display interface of the client, such as a live video interface, to play the response animation of the interactive object, the response of the interactive object to the first message from the client can be displayed, which is Responses include outputting a piece of language, and/or making some actions, expressions, etc.

In the embodiment of the present disclosure, the server receives the first message from the client, obtains matching driving data according to the indication content contained in the first message, and uses the driving data to control the playback on the display interface of the client The response animation of the interactive object displays the response of the interactive object, so that the interactive object can give timely feedback to the user's instruction content, so as to realize the timely interaction with the user.

FIG. 2 is an exemplary illustration of the application of the interaction method proposed by at least one embodiment of the present disclosure to a live broadcast process. As shown in FIG. 2 , the interactive object is a three-dimensional virtual character with an image of a doctor. In the display interface of the client, the process of performing the live broadcast of the 3D avatar as the host can be displayed, and the user on the client can input the instruction content in the display interface to send the first message carrying the instruction content. Correspondingly, After receiving the first message from the client, the server can identify the instruction content, such as "how to wash hands", and then obtain matching drive data according to the instruction content, and can control the display of the client according to the drive data. The 3D avatar's response to the instruction "How to wash your hands". For example, the three-dimensional avatar is controlled to output a voice corresponding to "how to wash hands", and at the same time make actions and/or expressions matching the output voice.

In some embodiments, the indicative content includes textual content. The response content for the indicated content may be acquired in the following manner: based on a natural language processing (Natural Language Processing, NLP) algorithm, the language intent expressed by the text content is identified, and the response content matching the language intent is acquired.

In some embodiments, the text content may be processed using a pre-trained neural network model for natural language processing, such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (Recurrent Neural Networks) Network, RNN), Long Short Term Memory network (Long Short Term Memory network, LTSM) and so on. By inputting the text content included in the first message into the neural network model, and by classifying the language intention represented by the text content, the language intention category expressed by the text content is determined.

Since the text content included in the first message may contain multiple layers of meanings, the natural language processing algorithm can be used to identify the actual intention that the user wants to express, so that the content that the user really wants to obtain can be directly fed back, which improves the User's interactive experience.

In some embodiments, according to the language intent, the response content that matches the language intent and conforms to the language intent may be searched from a preset database. Further, the server may generate a response content based on the response content. Driving data for making the interactive object express the content of the response. Wherein, the database may be deployed in the server or in the cloud, which is not limited in the present disclosure.

In the case of identifying the linguistic intent, the server may extract parameters, ie entities, related to the linguistic intent from the textual content. For example, entities can be determined by means of systematic word segmentation, information extraction, etc. In the data corresponding to the language intent classification, the entity can further determine the response text that conforms to the language intent. Those skilled in the art should understand that the above manner is only used for example, and other manners may also be used to obtain the response text matching the language intent, which is not limited in the present disclosure.

In some embodiments, the server may generate voice-driven data according to the response content, where the voice-driven data includes, for example, a phoneme sequence corresponding to the response text included in the response content. By generating the voice corresponding to the phoneme sequence and controlling the client to output the voice, the interactive object can be made to output the voice that expresses the content represented by the response text.

In some embodiments, the server may generate action-driven data according to the response content, so that the interactive object performs an action expressing the response content.

In an example, when the response content includes response text, the action-driven data may be generated according to the response content in the following manner: based on at least one target text included in the response text, obtain a match with the target text The control parameter of the set action of the interactive object.

The target text may be set keywords, keywords, key sentences, and the like. Taking the keyword "washing hands" as an example, if the response text includes "washing hands", it can be determined that the response text includes the target text. A matching set action can be set for each target text in advance, and each set action can be realized by a set of control parameter sequences, such as the displacement of multiple skeleton points to form a set of control parameters, and the control parameters formed by multiple sets of control parameters. A sequence is used to adjust the model parameters of the interactive object, so that the interactive object can perform the set action.

In the embodiment of the present disclosure, by making the interactive object respond to the first message in the form of action, the user can obtain an intuitive and vivid response to the first message, and the interactive experience of the user is improved.

In some embodiments, the voice information corresponding to the target text can be determined; the time information for outputting the voice information can be obtained; the execution time of the set action corresponding to the target text can be determined according to the time information; according to the execution time , and control the interactive object to execute the set action with the control parameter corresponding to the target text.

In the case of controlling the client to output the voice according to the phoneme sequence corresponding to the response text, the time information of outputting the voice corresponding to the target text may be determined, such as the time to start outputting the voice corresponding to the target text, the end of the output The time and duration of the output. The execution time of the set action corresponding to the target text can be determined according to the time information, and within the execution time, or within a certain range of the execution time, the interactive object is controlled to execute with the control parameters corresponding to the target text the setting action.

In the embodiment of the present disclosure, for each target text, the duration of outputting the corresponding voice is consistent with or similar to the duration of the control action according to the corresponding control parameter, so that the interactive object outputs the corresponding value of the target text. The voice and the time of the action are matched, so that the voice of the interactive object and the action are synchronized and coordinated, so that the user can feel that the interactive object responds during the live broadcast, which improves the interaction between the user and the host during the live broadcast. An interactive experience.

In some embodiments, gesture-driven data may be generated according to the response text, so that the client can display the gesture of the interactive object that matches the voice corresponding to the response text, such as making corresponding expressions and actions .

In an example, the response content may also include a phoneme sequence, or, in the case where the response content includes a response text, a phoneme sequence corresponding to the response text may also be extracted, and after the response content including the phoneme sequence is obtained, you can obtain the corresponding phoneme sequence. The phoneme sequence matches the control parameters for the interactive object. Wherein, the control parameter of the interactive object includes a gesture control vector of at least one local area, and the obtaining the control parameter for the interactive object that matches the phoneme sequence includes: performing feature encoding on the phoneme sequence to obtain the obtained phoneme sequence. The first coding sequence corresponding to the phoneme sequence; according to the first coding sequence, obtain the feature code corresponding to at least one phoneme; obtain the gesture control vector of at least one local area of the interactive object corresponding to the feature code.

In some embodiments, by controlling the client to play the voice corresponding to the response text and causing the client to display a response animation of the gesture of the interactive object that matches the voice, the response of the interactive object is made more responsive. Anthropomorphic, more vivid and natural, which enhances the user's interactive experience.

In an embodiment where the control parameter of the interactive object includes a gesture control vector of at least one local area, the gesture control vector can be obtained in the following manner.

First, feature encoding is performed on the phoneme sequence corresponding to the response text to obtain a coding sequence corresponding to the phoneme sequence. Here, in order to distinguish it from the coding sequence mentioned later, the coding sequence corresponding to the phoneme sequence of the text data is referred to as the first coding sequence.

For multiple phonemes included in the phoneme sequence, a subcoding sequence corresponding to each phoneme is generated.

In an example, it is detected whether there is a first phoneme corresponding to each time point, and the first phoneme is any one of the plurality of phonemes; the encoding value at the time point corresponding to the first phoneme is set as For the first numerical value, the coding value at the time point without the first phoneme is set as the second numerical value, and the sub-coding sequence corresponding to the first phoneme can be obtained after assigning the coding value at each time point. For example, the code value may be set to 1 at the time point when the first phoneme is present, and the code value may be set to 0 at the time point when the first phoneme is absent. Those skilled in the art should understand that the above setting of the encoding value is only an example, and the encoding value may also be set to other values, which is not limited in the present disclosure.

After that, a first coding sequence corresponding to the phoneme sequence is obtained according to the sub-coding sequences corresponding to the plurality of phonemes respectively.

In an example, for the sub-coding sequence corresponding to the first phoneme, a Gaussian filter may be used to perform a Gaussian convolution operation on the continuous values of the first phoneme in time, so as to filter the matrix corresponding to the feature code and smooth the The action of the mouth region transition at each phoneme transition.

FIG. 3 is a flowchart of a method for obtaining an attitude control vector provided by at least one embodiment of the present disclosure. As shown in FIG. 3 , the phoneme sequence 310 includes phonemes j, i1, j, and ie4 (for the sake of brevity, only part of the phonemes are shown), and for each phoneme j, i1, and ie4, the corresponding sub-phones are obtained respectively. Coding sequences 321, 322, 323. In each sub-encoding sequence, the corresponding encoding value is the first numerical value (for example, 1) at the time when the phoneme is present (second (s) is used as the time unit in FIG. 3 ), and at the time when the phoneme is absent ( In FIG. 3 , the corresponding coded value on the second (s) is the second value (for example, 0). Taking the sub-coding sequence 321 as an example, when the phoneme sequence 310 has phoneme j, the value of the sub-coding sequence 321 is the first value, and when there is no phone j, the value of the sub-coding sequence 321 is the second value. All sub-coding sequences constitute the first coding sequence 320 .

Next, according to the first coding sequence, a feature code corresponding to at least one phoneme is acquired.

According to the coding values of the sub-coding sequences 321, 322, 323 corresponding to phonemes j, i1, and ie4 respectively, and the durations of the corresponding phonemes in the three sub-coding sequences, that is, the duration of j in the sub-coding sequence 321, The duration of i1 in the sub-coding sequence 322 and the duration of ie4 in the sub-coding sequence 323 can obtain the characteristic information of the sub-coding sequences 321 , 322 and 323 .

In an example, a Gaussian convolution operation can be performed on the continuous values of phonemes j, i1, ie4 in time in the sub-coding sequences 321, 322, 323 using a Gaussian filter, so as to smooth the feature code, and obtain a smoothed 330 of the first coding sequence. That is, the Gaussian convolution operation is performed on the temporally continuous values of 0-1 of the phoneme through the Gaussian filter, so that the coding values in each coding sequence are from the second numerical value to the first numerical value or from the first numerical value to the second numerical value. The change phase becomes smooth. For example, in addition to 0 and 1, the values of the coding sequence also show intermediate state values, such as 0.2, 0.3, etc., and the gesture control vector obtained according to the values of these intermediate states makes the interactive characters excessively move and change their expressions more smoothly , Naturally, it improves the interactive experience of the target object.

In some embodiments, the feature code corresponding to at least one phoneme may be obtained by performing a sliding window on the first code sequence. Wherein, the first coding sequence may be a coding sequence after Gaussian convolution operation.

With a time window of a set length and a set step size, a sliding window is performed on the coding sequence, and the feature coding in the time window is used as the feature coding of the corresponding at least one phoneme, and after the sliding window is completed, according to the obtained multiple feature codes, the second code sequence can be obtained. As shown in FIG. 3 , by sliding a time window of a set length on the first encoding sequence 320 or the smoothed first encoding sequence 330, feature encoding 1, feature encoding 2, and feature encoding 3 are obtained respectively, and so on. After traversing the first coding sequence, the feature codes 1, 2, 3, . . . , M are obtained, thereby obtaining the second coding sequence 340 . Wherein, M is a positive integer, and its value is determined according to the length of the first coding sequence, the length of the time window, and the step size of the time window sliding.

According to the feature codes 1, 2, 3, ..., M, corresponding attitude control vectors 1, 2, 3, ..., M can be obtained respectively, thereby obtaining a sequence 350 of attitude control vectors.

The sequence 350 of gesture control vectors is temporally aligned with the second encoding sequence 340, and since each encoded feature in the second encoding sequence is obtained from at least one phoneme in the phoneme sequence, the sequence of gesture control vectors Each feature vector in 350 is also obtained from at least one phoneme in the phoneme sequence. While playing the phoneme sequence corresponding to the text data, the interactive object is driven to act according to the sequence of the gesture control vector, that is, the interactive object can be driven to emit the sound corresponding to the text content, and the sound can be synchronized with the sound. The action gives the target object the feeling that the interactive object is talking, which improves the interactive experience of the target object.

Assuming that the encoding feature starts to be output at the set time of the first time window, the gesture control vector before the set time can be set to the default value, that is, when the phoneme sequence is just started to be played, the interactive object can be set to the default value. After the set time, the interactive object starts to use the sequence of gesture control vectors obtained according to the first coding sequence to drive the interactive object to make an action. Taking Figure 3 as an example, the encoding feature 1 starts to be output at time t0, and before time t0 corresponds to the default attitude control vector.

In some embodiments, when the time interval between phonemes in the phoneme sequence is greater than a set threshold, the interactive object is driven to act according to the set gesture control vector of the local area. That is, when the interactive character pauses for a long time, the interactive object is driven to perform the set action. For example, when the output sound pauses for a long time, the interactive character can be made to smile or make a slight body swing, so as to avoid the interactive character standing blankly when the pause is long, which makes the interactive object speak. Natural and smooth, improve the interactive experience of the target object.

In some embodiments, for at least one target text included in the response text, a control parameter for a set action of an interactive object that matches the at least one target text is obtained, so as to drive the interactive object to execute the Set action; for the response content other than the at least one target text, the control parameters of the interactive object can be obtained according to the phoneme corresponding to the response content, so as to drive the interactive object to make a pronunciation corresponding to the response content Matching gestures, such as expressions and movements.

Taking the live broadcast process shown in Figure 2 as an example, when the received first message contains the text content "how to wash hands", through the natural language processing algorithm, it can be recognized that the user's language intention is "consult how to wash hands". By retrieving in the preset database, the content corresponding to the answer on how to wash hands can be obtained, and the content can be used as the answer text. By generating action-driven data, voice-driven data, and gesture-driven data according to the response text, the interactive object can make expressions and actions that match the pronunciation while answering the question "how to wash hands" through voice. And at the same time use body movements to demonstrate how to wash hands.

In some embodiments, instruction information including the response text may also be sent to the client, so that the client displays the response text based on the instruction information.

For example, for a response text to the question "how to wash your hands", the instruction message can be displayed in text form on the client by sending instruction information containing the response text to the client, so that the Users can more accurately receive the information conveyed by interactive objects.

In some embodiments, the virtual model corresponding to the interactive object (the virtual model can be either a two-dimensional virtual model or a three-dimensional virtual model) can be stored in the client. In this case, the driving data of the interactive object may be sent to the client, so that the client generates a response animation according to the driving data; and the client is controlled to play the response animation. For example, the client can be controlled to adjust the virtual model parameters of the interactive object according to the control parameters included in the driving data; and based on the adjusted virtual model parameters, the rendering engine is used to generate the response animation of the interactive object, and The response animation is played in response to the first message. When the virtual model is a two-dimensional virtual model, the virtual model parameters are two-dimensional virtual model parameters, and when the virtual model is a three-dimensional virtual model, the virtual model parameters are three-dimensional virtual model parameters. For another example, the server may determine a control command for controlling the response mode of the interactive object based on the driving data, and send the control command to the client, so that the client displays the responding interactive object based on the control command screen.

In the case where the data volume of the virtual model of the interactive object is small and the performance of the client is not high, the client can generate a response animation according to the drive data by sending the drive data to the client , so that the screen of the responding interactive object can be displayed conveniently and flexibly.

In some embodiments, the virtual model corresponding to the interactive object is stored on the server or the cloud. In this case, the virtual model parameters of the interactive object can be adjusted based on the driving data; based on the adjusted virtual model parameters, a rendering engine is used to generate a response animation of the interactive object, and send it to the client In the response animation, the action or expression of the interactive object is displayed in the response animation. By sending the response animation to the client to realize the response of the interactive object, it is possible to avoid the freeze caused by the client's rendering, and can display the high-quality response animation on the client, which improves the user's interactive experience.

FIG. 4 is a flowchart illustrating another interaction method according to at least one embodiment of the present disclosure. This interactive method can be applied to the client. The method includes steps 401-402.

In step 401, in response to a user input operation from the client, a first message including the indication content is sent to the server.

Exemplarily, the user input operation includes an input text operation, a voice input operation, an action-triggered operation, a key-triggered operation, etc. In response to the user input operation, the first message sent to the server, the first message carrying The indicated content includes but is not limited to one or more of text, voice, images (such as facial expressions, motion images), and video. For example, in a live video broadcast scenario, the client may be a client that supports the function of watching live video, and the first message may be collected by the client and sent after the user enters text content on the display interface. The first message carries For example, the instruction content is input text content, and the instruction content can be displayed in the display interface in the form of a bullet screen. For another example, in a somatosensory interaction scenario, the first message may be sent after the client terminal collects the user behavior image, and the indication content carried in the first message is, for example, the collected user behavior image. Of course, in the specific implementation, the present disclosure does not limit the sending mechanism of the first message and the form of the indication content carried in the first message.

In step 402, based on the second message that the server responds to the first message, the response animation of the interactive object is played on the display interface of the client.

The second message is generated by the server in response to the instruction content included in the first message, and is used for causing the client to display an interactive object responding to the instruction content.

In the embodiment of the present disclosure, the interactive object is obtained by rendering a virtual model, such as a two-dimensional or three-dimensional virtual model. The virtual model may be custom generated or obtained by converting an image or video of a character. The embodiments of the present disclosure do not limit the manner of generating the virtual model.

In an embodiment of the present disclosure, by sending a first message including an indication content to a server according to a user input operation, an interactive object is displayed in the client based on the second message responded by the server in response to the first message The response to the instruction content enables the interactive object to give timely feedback to the user's instruction content, thereby realizing timely interaction with the user.

In some embodiments, the indication content includes text content; the method further includes: displaying the text content on a display interface of the client, and/or determining and playing an audio file corresponding to the text content. That is, the text content input by the user can be displayed on the client terminal; the audio file corresponding to the text content can also be played on the client terminal, and the voice corresponding to the text content can be output.

In some embodiments, the displaying the text content in the client includes: generating bullet screen information of the text content; and displaying the bullet screen information in a display interface of the client.

In the live video broadcast scenario, for the text content input by the user, corresponding bullet screen information can be generated, and the bullet screen information can be displayed on the display interface of the client. Taking FIG. 2 as an example, when the user inputs "how to wash hands" in the live interactive interface of the client, the display interface can display the barrage information "how to wash hands" corresponding to the text content.

In some embodiments, the second message includes a response text for the indicated content; the method further includes: displaying the response text on a display interface of the client, and/or determining and playing the response text. The audio file corresponding to the response text.

The response text indicating the content may be obtained by: identifying the language intent expressed by the text content, and searching for a response text matching the language intent from a preset database. For the specific method, refer to the above-mentioned embodiments, and details are not repeated here.

Taking the live video broadcast scene as an example, the response text for replying to the user's bullet screen information can also be displayed on the display interface in the form of bullet screen information; and the audio file corresponding to the response text can be played on the display interface, that is, The voice corresponding to the response text is output, so that the user's bullet screen information can be answered accurately and intuitively, and the user's interactive experience can be improved.

In some embodiments, the second message includes control parameters of the interactive object that match the phoneme sequence corresponding to the response text, and/or matches at least one target text included in the response text The control parameters of the setting action of the interactive object; the second message based on the server's response to the first message, playing the response animation of the interactive object in the display interface of the client, including: Based on the driving data, the virtual model parameters of the interactive object are adjusted; based on the adjusted virtual model parameters, a rendering engine is used to generate a response animation of the interactive object, and display it on the display interface of the client. Wherein, generating control parameters of the interactive object matching the phoneme sequence corresponding to the response text, and generating control parameters of the setting action of the interactive object matching at least one target text included in the response text For the specific method, refer to the above-mentioned embodiments, and details are not repeated here.

In the case that the data volume of the virtual model of the interactive object is small and the performance of the client is not high, the client obtains the driving data, and generates a response animation according to the driving data, so that the rendering can be conveniently and flexibly A screen showing the interactive object that responded.

In some embodiments, the second message further includes a response animation made by the interactive object to the indicated content; the second message based on the server's response to the first message, in the Playing the response animation of the interactive object in the display interface of the client includes: displaying the response animation in the display interface of the client.

In some embodiments, the virtual model corresponding to the interactive object is stored on the server or the cloud. In this case, the response animation can be generated on the server side or in the cloud. For the specific manner of generating the response animation, reference may be made to the foregoing embodiments, and details are not described herein again.

By sending the response animation to the client to realize the response of the interactive object, it is possible to avoid the freeze caused by the client's rendering, and can display the high-quality response animation on the client, which improves the user's interactive experience.

In some embodiments, the user's input operation includes that the user follows the limb operation picture displayed in the display interface to make a corresponding human gesture; in this case, in response to the user's input operation from the client , the method further includes: acquiring a user behavior image including the human body posture; recognizing the human body posture information in the user behavior image, and driving the interactive objects displayed on the display interface based on the human body posture information respond.

In some embodiments, driving the interactive object displayed on the display interface to respond based on the human body posture information includes: determining a degree of matching between the human body posture information and the human body posture in the limb operation screen; based on The matching degree drives the interactive objects displayed on the display interface to respond.

In some embodiments, the driving the interactive object to respond based on the matching degree includes: instructing the interactive object displayed on the display interface to make a first response when the matching degree reaches a set condition , wherein the first response includes displaying a qualified body movement and/or voice prompt; and displaying the next body operation screen; in the case where the matching degree does not reach the set condition, instructing the interactive object displayed on the display interface A second response is made, wherein the second response includes displaying the body movements and/or voice prompts of unqualified postures; and keeping the current body operation screen displayed.

Exemplarily, the following are some embodiments in which the embodiments of the present disclosure are applied in the scenario of a live video broadcast platform:

In some embodiments, the received first message from the client is the user bullet screen text transmitted by the live broadcast platform.

In some embodiments, after analyzing the intention of the bullet screen by a natural language processing algorithm, a corresponding answer is obtained, and then the content of the answer is broadcast through the interactive object. In addition, the action corresponding to the content of the answer can also be displayed through the interactive object.

In some embodiments, the natural language processing capability of the client is directly integrated, and natural language processing is performed on the indication content included in the first message to obtain a response text that matches the language intention of the indication content and conforms to the language intention , and directly provide the text corresponding to the output response text to the interactive object for playback.

In some embodiments, the interactive object may mimic the user's speech. For example, for the voice input by the user through the client, by converting the voice into text, obtaining the voice features of the user according to the voice, and outputting the voice corresponding to the text based on the voice features, an interactive object can be realized. Mimic the user's speech.

In some embodiments, the interactive object can also be displayed on the page according to the content returned by natural language processing, and can be displayed according to the pre-designed content to be displayed and the UI content in an interactive manner, so that the display of the response content is more eye-catching and attractive. user's attention.

In the above embodiment, real-time interaction can be broadcast live. During the live broadcast, the user can interact with interactive objects in real time to obtain feedback. It can also broadcast live continuously, and can also automatically produce video content, which is a new way of live TV.

Exemplarily, the interactive object can be represented as a digital human in three-dimensional form. The digital human combines the artificial intelligence (AI) simulation animation generation ability with the natural language understanding ability, and can communicate with the user like a real person. The digital person can generate the corresponding mouth shape, expression, eyes and body movements according to the answer content, and finally output high-quality, audio-video synchronized voice and multi-dimensional animation content, and present the complete digital person image to the user naturally.

In some embodiments, content service libraries in different fields of knowledge can be quickly connected and efficiently applied to more industries. At the same time, digital human images in various styles such as hyper-realism and cartoons can also be provided according to the needs of different scenarios. , gesture recognition and other AI technologies to intelligently interact with users. For example, hyper-realistic digital people can create intelligent front desks for banks, business halls, and service halls, and communicate with customers in a real and effective manner, improving service quality and customer satisfaction.

In some embodiments, cartoon-style figures can be used in interesting interaction-oriented scenarios, such as smart guides in offline supermarkets, or smart coaches, virtual teachers, etc., to attract customers, stimulate interest, strengthen teaching effect, etc.

At least one embodiment of the present disclosure also provides an interactive device, which can be applied to a server. As shown in FIG. 5 , the apparatus 50 includes: a receiving unit 501, configured to receive a first message from a client; and an obtaining unit 502, configured to obtain, based on the indication content included in the first message, matching the indication content The driving unit 503 is configured to use the driving data to control the display interface of the client to play the response animation of the interactive object.

In some embodiments, the obtaining unit 502 is configured to: obtain a response content for the indicated content, where the response content includes a response text; and, based on at least one target text included in the response text, obtain a response to the target text The control parameter of the set action of the matching interactive object.

In some embodiments, the obtaining unit 502 is configured to: obtain response content for the indicated content, where the response content includes a phoneme sequence; and obtain a control parameter of the interactive object matching the phoneme sequence.

In some embodiments, the control parameter of the interactive object includes a gesture control vector of at least one local area, and when the obtaining unit 502 obtains the control parameter of the interactive object matching the phoneme sequence, it is used for: controlling the phoneme performing feature encoding on the sequence to obtain a first encoding sequence corresponding to the phoneme sequence; obtaining a feature encoding corresponding to at least one phoneme according to the first encoding sequence; obtaining at least one local area of the interactive object corresponding to the feature encoding The attitude control vector of .

In some embodiments, the method further includes a sending unit, configured to send instruction information including response content to the instruction content to the client, so that the client displays the response based on the instruction information content.

In some embodiments, the driving unit 503 is configured to: send the driving data of the interactive object to the client, so that the client can generate a response animation according to the driving data; control the client to play in the display interface the response animation; or, based on the driving data, adjust the two-dimensional or three-dimensional virtual model parameters of the interactive object; based on the adjusted two-dimensional or three-dimensional virtual model parameters, use a rendering engine to generate the interactive object's response animation , and send the response animation to the client.

At least one embodiment of the present disclosure further provides another interactive device, which can be applied to a client. As shown in FIG. 6 , the apparatus 60 includes: a sending unit 601 for sending a first message including indicating content to a server in response to a user input operation from a client; a playing unit 602 for sending, based on the server The second message that the browser responds to the first message plays the response animation of the interactive object on the display interface of the client. The interactive objects are rendered through virtual models such as two-dimensional or three-dimensional virtual models.

In some embodiments, the indication content includes text content; the apparatus further includes a first display unit, configured to display the text content in a display interface of the client, and/or determine and play the text The audio file corresponding to the content.

In some embodiments, when the first display unit is used to display the text content in the client, it is specifically configured to: generate bullet screen information of the text content; in the display interface of the client Display the barrage information.

In some embodiments, the second message includes a response text to the indicated content; the apparatus further includes a second display unit configured to display the response text in a display interface of the client, and/or Or, determine and play the audio file corresponding to the response text.

In some embodiments, the second message includes driving data of the interactive object; the playing unit 602 is configured to: adjust the virtual model parameters of the interactive object based on the driving data; Model parameters, using a rendering engine to generate a response animation of the interactive object, and display it in the display interface of the client; wherein, the driving data includes a phoneme sequence that matches the phoneme sequence corresponding to the response text for the indicated content. The control parameter of the interactive object, and/or the control parameter of the set action of the interactive object that matches at least one target text contained in the response text.

In some embodiments, the second message includes a response animation of the interactive object to the indicated content.

In some embodiments, the user's input operation includes that the user follows the limb operation picture displayed in the display interface to make a corresponding human body posture; the generating unit 601 is configured to: acquire a data including the human body posture User behavior image; recognize the human body posture information in the user behavior image, and drive the interactive object displayed on the display interface to respond based on the human body posture information.

In some embodiments, the generating unit 601 is specifically configured to: determine the degree of matching between the human body posture information and the human posture in the body manipulation screen; and, based on the degree of matching, drive the interactive object displayed on the display interface to respond .

In some embodiments, the generating unit 601 is specifically configured to: instruct the interactive object displayed on the display interface to make a first response when the matching degree reaches a set condition, wherein the first response includes that the display posture is qualified and display the next body operation screen; if the matching degree does not meet the set condition, instruct the interactive object displayed on the display interface to make a second response, wherein the second Responses include displaying body movements and/or voice prompts for unqualified postures; and keeping the current body manipulation screen displayed.

At least one embodiment of the present disclosure further provides an electronic device. As shown in FIG. 7 , the electronic device 70 includes a memory 701 and a processor 702, where the memory 701 is used to store computer instructions that can be executed on the processor 702, The processor 702 is configured to implement the interaction method described in the embodiment of the present disclosure relating to the server when executing the computer instructions.

At least one embodiment of the present specification further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by the processor 701, the interactive method described in the server-related embodiment of the present disclosure is implemented.

At least one embodiment of the present disclosure also provides an electronic device. As shown in FIG. 8 , the electronic device 80 includes a memory 801 and a processor 802, where the memory 801 is used to store computer instructions that can be executed on the processor 802, The processor 802 is configured to implement the interaction method described in the embodiments of the present disclosure involving the client when executing the computer instructions.

At least one embodiment of the present specification further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by the processor 802, the interactive method described in the embodiments of the present disclosure involving the client is implemented.

As will be appreciated by one skilled in the art, one or more embodiments of this specification may be provided as a method, system or computer program product. Accordingly, one or more embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may be implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) having computer-usable program code embodied therein in the form of a computer program product.

Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data processing device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for related parts, please refer to the partial description of the method embodiment.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the acts or steps recited in the claims may be performed in an order different from that of the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Embodiments of the subject matter and functional operations described in this specification can be implemented in digital electronic circuits, in tangible embodiment of computer software or firmware, in computer hardware including the structures disclosed in this specification and their structural equivalents, or A combination of one or more of them. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, ie, one or more of computer program instructions encoded on a tangible, non-transitory program carrier for execution by, or to control the operation of, a data processing apparatus. Multiple mods. Alternatively or additionally, program instructions may be encoded on artificially generated propagating signals, such as machine-generated electrical, optical or electromagnetic signals, which are generated to encode and transmit information to suitable receiver devices for data retrieval. The processing device executes. The computer storage medium can be a machine-readable storage device, a machine-readable storage substrate, a random or sequential access memory device, or a combination of one or more of these.

The processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform corresponding functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, eg, an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

Computers suitable for the execution of a computer program include, for example, general and/or special purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit will receive instructions and data from read-only memory and/or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include, or be operably coupled to, such mass storage devices to receive data therefrom or to include, or be operatively coupled to, one or more mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks, for storing data. Send data to it, or both. However, the computer does not have to have such a device. Furthermore, the computer may be embedded in another device, such as a mobile phone, personal digital assistant (PDA), mobile audio or video player, game console, global positioning system (GPS) receiver, or a universal serial bus (USB ) flash memory drives for portable storage devices, to name a few.

Computer-readable media suitable for storage of computer program instructions and data include all forms of non-volatile memory, media, and memory devices including, for example, semiconductor memory devices (eg, EPROM, EEPROM, and flash memory devices), magnetic disks (such as internal hard disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. The processor and memory may be supplemented by or incorporated in special purpose logic circuitry.

While this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or what may be claimed, but rather are used primarily to describe features of specific embodiments of particular inventions. Certain features that are described in this specification in multiple embodiments can also be implemented in combination in a single embodiment. On the other hand, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may function as described above in certain combinations and even be originally claimed as such, one or more features from a claimed combination may in some cases be removed from the combination and the claimed A protected combination may point to a subcombination or a variation of a subcombination.

Similarly, although operations in the figures are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or sequentially, or that all illustrated operations be performed, to achieve the desired results . In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above-described embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems may generally be integrated together in a single software product , or packaged into multiple software products.

Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claim may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous.

The above descriptions are only some examples of one or more embodiments of this specification, and are not intended to limit one or more embodiments of this specification. All within the spirit and principles of one or more embodiments of this specification, all Any modification, equivalent replacement, improvement, etc. made should be included within the protection scope of one or more embodiments of the present specification.

101: Steps of receiving the first message from the client 102: based on the indication content included in the first message, a step of acquiring drive data matching the indication content 103: the step of controlling the display interface of the client to play the response animation of the interactive object using the driving data 401: In response to the user input operation from the client, the step of sending a first message including the indicated content to the server 402: Based on the second message that the server responds to the first message, the step of playing the response animation of the interactive object on the display interface of the client 501: Receiving unit 502: Get unit 503: Drive unit 50: Interactive installations 601: sending unit 602: playback unit 60: Interactive installations 702: Processor 701: Memory 70: Electronic Equipment

FIG. 1 is a flowchart illustrating an interaction method according to at least one embodiment of the present disclosure. FIG. 2 is a schematic diagram illustrating that the interaction method proposed by at least one embodiment of the present disclosure is applied to a live broadcast process. FIG. 3 is a flowchart of a method for obtaining an attitude control vector provided by at least one embodiment of the present disclosure. FIG. 4 is a flowchart illustrating another interaction method according to at least one embodiment of the present disclosure. FIG. 5 is a schematic structural diagram of an interactive device according to at least one embodiment of the present disclosure. FIG. 6 is a schematic structural diagram of another interactive device according to at least one embodiment of the present disclosure. FIG. 7 is a schematic structural diagram of an electronic device according to at least one embodiment of the present disclosure. FIG. 8 is a schematic structural diagram of another electronic device according to at least one embodiment of the present disclosure.

101: Steps of receiving the first message from the client

102: based on the indication content included in the first message, a step of acquiring drive data matching the indication content

103: the step of controlling the display interface of the client to play the response animation of the interactive object using the driving data

Claims (12)

An interaction method, comprising: receiving a first message from a client; obtaining driving data matching the indicated content based on the indication content included in the first message; and controlling the display interface playback of the client by using the driving data The response animation of the interactive object; wherein, acquiring the driving data matching the indication content based on the indication content included in the first message includes: acquiring response content for the indication content, where the response content includes phonemes sequence, and obtain the control parameters of the interactive object matching the phoneme sequence; wherein, the control parameters of the interactive object include gesture control vectors of at least one local area, and obtaining the interactive object matching the phoneme sequence The control parameters include: performing feature encoding on the phoneme sequence to obtain a first encoding sequence corresponding to the phoneme sequence; obtaining a feature encoding corresponding to at least one phoneme according to the first encoding sequence; obtaining the corresponding feature encoding The gesture control vector of at least one local area of the interactive object. The interaction method according to claim 1, wherein, based on the indication content included in the first message, acquiring the driving data matching the indication content, further comprising: Acquire response content for the indicated content, the response content includes a response text, and based on at least one target text included in the response text, acquire control parameters for setting actions of interactive objects matching the target text. The interaction method according to claim 1, wherein the using the driving data to control the client to play the response animation of the interactive object in the display interface comprises: sending the driving data of the interactive object to a the client, so that the client generates a response animation according to the driving data; controls the client to play the response animation in the display interface; or, based on the driving data, adjusts the virtual model parameters of the interactive object ; Based on the adjusted virtual model parameters, use a rendering engine to generate a response animation of the interactive object, and send the response animation to the client. An interaction method, comprising: in response to a user input operation from a client, sending a first message including indicating content to a server; based on a second message responded by the server to the first message, in the client The display interface plays the response animation of the interactive object; wherein, the second message includes the driving data of the interactive object, and the driving data includes the control for the interactive object that matches the phoneme sequence corresponding to the response text parameters, the control parameters of the interactive object include the attitude control vector of at least one local area, and the attitude control vector is obtained by the following methods: performing feature encoding on the phoneme sequence, and obtaining the first number corresponding to the phoneme sequence. a coding sequence; obtaining a feature code corresponding to at least one phoneme according to the first coding sequence; obtaining a gesture control vector of at least one local area of the interactive object corresponding to the feature code. The interaction method according to claim 4, wherein the indication content includes text content; the method further comprises: displaying the text content in the client, and/or playing audio corresponding to the text content wherein, the displaying the text content in the client includes: generating bullet screen information of the text content; and displaying the bullet screen information on a display interface of the client. The interaction method according to claim 4, wherein the second message includes a response text for the indicated content; the method further comprises: displaying the response text on a display interface of the client, and/ Or, determine and play the audio file corresponding to the response text. The interaction method according to any one of claim 4 to 6, wherein the response animation of the interactive object is played in the display interface of the client based on the second message that the server responds to the first message, The method includes: adjusting the virtual model parameters of the interactive object based on the driving data; using a rendering engine to generate a response animation of the interactive object based on the adjusted virtual model parameters, and displaying it on the display interface of the client; wherein , the drive data also includes at least A target text matching control parameter for the set action of the interactive object. The interactive method according to claim 4, wherein the user's input operation includes: the user follows the limb operation picture displayed in the display interface to make a corresponding human gesture; input operation, the method further includes: acquiring a user behavior image including the human body posture; recognizing the human body posture information in the user behavior image; based on the human body posture information, driving the display interface to display The interactive object responds. The interaction method according to claim 8, wherein driving the interactive object displayed on the display interface to respond based on the human body posture information comprises: determining the human body posture information and the human body in the limb operation screen The matching degree of the gesture; based on the matching degree, the interactive object displayed on the display interface is driven to respond. The interaction method according to claim 9, wherein the driving the interactive object to respond based on the matching degree comprises: instructing the interaction displayed on the display interface when the matching degree reaches a set condition The object makes a first response, wherein the first response includes displaying qualified body movements and/or voice prompts; and displaying the next body operation screen; in the case that the matching degree does not meet the set condition, indicating the The interactive object displayed on the display interface makes a second response, wherein the second response includes a display gesture Unqualified body movements and/or voice prompts; and keeping the current body operation screen displayed. An electronic device, the device includes a memory and a processor, the memory is used to store computer instructions executable on the processor, and the processor is used to implement request items 1 to 1 when executing the computer instructions The interaction method according to any one of 3, or the processor is configured to implement the interaction method according to any one of request items 4 to 10 when executing the computer instructions. A computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, realizes the interactive method described in any one of claim 1 to 3, or when the computer program is executed by the processor. Implement the interaction method described in any one of claim items 4 to 10.

Images