CN112804546B

CN112804546B - Interaction method, device, equipment and storage medium based on live broadcast

Info

Publication number: CN112804546B
Application number: CN202110018400.5A
Authority: CN
Inventors: 邱理权
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-01-07
Filing date: 2021-01-07
Publication date: 2022-10-21
Anticipated expiration: 2041-01-07
Also published as: CN112804546A

Abstract

The application provides a live broadcast-based interaction method, a live broadcast-based interaction device, live broadcast-based interaction equipment and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: when a first interaction instruction of an audience account number is not received, a face area is covered with a first facial image in a live broadcast picture, after the first interaction instruction of the audience account number is received, the first interaction instruction is responded, the first facial area of the face is displayed in the live broadcast picture, the first facial area is at least a part of facial area covered by the first facial image, at the moment, the first facial image covering the face area is completely or partially uncovered, the audience can see more areas of the face area, therefore, the interaction between a main broadcast and the audience in the live video broadcast process is not unidirectional any more, the audience can obtain corresponding feedback after triggering the first interaction instruction, the interaction efficiency of a user is improved, and the live video broadcast watching experience of the audience is also improved.

Description

Interaction method, device, equipment and storage medium based on live broadcast

Technical Field

The present application relates to the field of computer technologies, and in particular, to a live broadcast-based interaction method, apparatus, device, and storage medium.

Background

With the continuous development and progress of computer technology and internet technology, live video is receiving more and more attention as a new information transmission mode. When video live broadcasting is carried out, video data are collected through camera equipment erected on site, and then the collected video data are uploaded to a server, so that the uploaded video data are distributed to a network through the server to be watched by audiences.

In the related technology, the anchor acquires the portrait of the anchor through the camera equipment, and then uploads the live video stream including the anchor portrait to the server, so that the audience can acquire the live video stream from the server to watch the live content of the anchor. The audience can issue comments or give a virtual gift to the anchor in the process of watching the live video so as to realize interaction with the anchor.

However, in the related art, the interaction mode that the audience publishes the comments or presents the virtual gifts to the anchor is usually unidirectional, that is, the audience does not necessarily get feedback after publishing the comments or presenting the virtual gifts to the anchor, the interaction effect between the anchor and the audience in the live video broadcasting process is affected, the user interaction efficiency is low, and the experience of the audience watching the live video is poor.

Disclosure of Invention

The embodiment of the application provides an interaction method, device, equipment and storage medium based on live broadcast, and feedback is given to interaction operation of audiences in a video live broadcast process, so that the interaction efficiency of users is improved, and the experience of watching the video live broadcast by the audiences is also improved. The technical scheme is as follows:

in one aspect, a live broadcast-based interaction method is provided, including:

displaying a live broadcast picture of a live broadcast room, wherein a face part in the live broadcast picture is covered with a first mask image;

receiving a first interaction instruction of any audience account number in the live broadcast room;

and responding to the first interaction instruction, and displaying a first face area of the face in the live broadcast picture, wherein the first face area is at least part of the face area covered by the first facial image.

In another aspect, a live broadcast-based interaction method is provided, including:

displaying a live broadcast picture of a live broadcast room, wherein a third face area with a face is displayed in the live broadcast picture;

receiving a second interaction instruction of any audience account number in the live broadcast room;

and responding to the second interaction instruction, and displaying a third facial image at a position corresponding to a third face area of the face in the live broadcast picture.

displaying a live broadcast picture of a live broadcast room, wherein a first mask image covers a face part of a human face in the live broadcast picture;

and responding to the operation implemented by the target audience account number in the live broadcast room, and displaying a first face area of the face in the live broadcast picture, wherein the first face area is at least part of the face area covered by the first facial image.

In another aspect, a live broadcast-based interaction device is provided, including:

the first display module is used for displaying a live broadcast picture of a live broadcast room, and a first mask image covers a face part of a human face in the live broadcast picture;

the receiving module is used for receiving a first interaction instruction of any audience account in the live broadcast room;

the first display module is further configured to display a first face area of the face of the human face in the live broadcast picture in response to the first interaction instruction received by the receiving module, where the first face area is at least a part of a face area covered by the first mask image.

In one possible implementation, the apparatus further includes: an image processing module;

the image processing module is used for performing texture fitting on the first facial mask image and the face in the image based on a first facial key point of the image and a second facial key point of the first facial mask image for the acquired image;

the first display module is used for displaying the face part of the human face subjected to texture fitting by the image processing module in the live broadcast picture;

the first face key points are used for identifying the outline and the five sense organs of the face in the image, and the second face key points are used for identifying the outline and the five sense organs of the face in the image of the first facial mask.

In a possible implementation manner, the image processing module is configured to perform the following processing:

for the acquired image, acquiring the first facial key point of the image;

mapping the second facial keypoints of the first mask image to the first facial keypoints of the image;

and according to the mapping result, performing texture fitting on the first facial tool image and the face part in the image.

In a possible implementation manner, the first display module is configured to display an entire region of the face in the live view.

In a possible implementation manner, the first display module is configured to display the acquired image in the live view.

In a possible implementation, the first mask image comprises at least two sub-mask images, different ones of which are associated with different interaction instructions;

the first display module is used for displaying a face area covered by a target sub mask image on the face of the human face in the live broadcast picture, and the target sub mask image is a sub mask image associated with the first interaction instruction.

In a possible implementation manner, the first presentation module is configured to perform the following processing:

performing texture fitting on the sub-mask images except the target sub-mask image in the first mask image and the face part in the acquired image based on the first face key point of the acquired image and the third face key point of the first mask image;

displaying the face after texture fitting in the live broadcast picture;

the first facial key point is used for identifying the outline and the five sense organs of the face in the acquired image, and the third facial key point is used for identifying the outline and the five sense organs of the face in the sub-mask image except the target sub-mask image in the first facial mask image.

In a possible implementation manner, the first display module is further configured to display, in response to the first interaction instruction, a second facial image at a position corresponding to a second face area of the face of the human face in the live view.

In a possible implementation manner, the first interaction instruction is triggered by an interaction operation of the audience account, the interaction operation is a virtual gift presenting operation, a approval operation, or a comment making operation, the virtual gift presenting operation for presenting different virtual gifts corresponds to different first interaction instructions, and the different first interaction instructions correspond to different face regions of the face.

the second display module is used for displaying a live broadcast picture of a live broadcast room, and a first mask image covers the face part of a human face in the live broadcast picture;

the second display module is further configured to display a first face area of the face of the human face in the live broadcast picture in response to an operation performed by the target audience account in the live broadcast room, where the first face area is at least a part of a face area covered by the first facial image.

the third display module is used for displaying a live broadcast picture of a live broadcast room, and a third face area with a face is displayed in the live broadcast picture;

the instruction receiving module is used for receiving a second interaction instruction of any viewer account in the live broadcast room;

the third display module is further configured to display a third accessory image at a position corresponding to a third face area of the face in the live broadcast picture in response to the second interaction instruction received by the instruction receiving module.

In another aspect, a computer device is provided that includes a processor and a memory having stored therein at least one instruction that is sandwiched and executed by the processor to perform operations performed by a live based interaction method as described above.

In another aspect, a computer-readable storage medium is provided, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the operations performed by the live broadcast-based interactive method as described above.

In another aspect, a computer program product or a computer program is provided, where the computer program product or the computer program includes program code stored in a computer-readable storage medium, and a processor of a computer device reads the program code from the computer-readable storage medium, and executes the program code, so that the computer device performs the operations performed by the interaction method for live video.

The technical scheme provided by the embodiment of the application brings the beneficial effects that:

when not receiving first interactive instruction, the face covers there is first utensil image in the live broadcast picture, spectator can't see the whole region of face this moment, after receiving the first interactive instruction from spectator's account number, show in the live broadcast picture before by the first face region that first utensil image covered, spectator can see more regions of face this moment, consequently, the interaction between anchor and spectator is no longer unidirectional among the live broadcast in-process of video, spectator can obtain corresponding feedback through triggering first interactive instruction, thereby improve user interaction efficiency, also promote spectator and watch the live experience of video.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the description below are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.

FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present application;

fig. 2 is a flowchart of a live broadcast-based interaction method provided in an embodiment of the present application;

fig. 3 is a flowchart of another live broadcast-based interaction method provided in an embodiment of the present application;

fig. 4 is a flowchart of another live broadcast-based interaction method provided in an embodiment of the present application;

fig. 5 is a signaling interaction diagram of a live broadcast-based interaction method according to an embodiment of the present application;

fig. 6 is a schematic diagram of a live view displayed by an anchor terminal according to an embodiment of the present application;

fig. 7 is a schematic view of a live view displayed by another anchor terminal according to an embodiment of the present application;

FIG. 8 is a schematic diagram of a first face keypoint provided by an embodiment of the present application;

FIG. 9 is a schematic diagram of a second face keypoint provided by an embodiment of the present application;

fig. 10 is a schematic diagram of a live view displayed by a viewer terminal according to an embodiment of the present application;

fig. 11 is a schematic diagram of another live view displayed by a viewer terminal according to an embodiment of the present application;

fig. 12 is a signaling interaction diagram of another live broadcast-based interaction method provided in an embodiment of the present application;

fig. 13 is a schematic diagram of a first mask image according to an embodiment of the present disclosure;

FIG. 14 is a schematic view of another first mask image provided in an embodiment of the present application;

fig. 15 is a schematic view of a live view displayed by another anchor terminal according to an embodiment of the present application;

fig. 16 is a schematic diagram of a live view displayed by another anchor terminal according to an embodiment of the present application;

fig. 17 is a schematic view of a live view displayed on a viewer terminal according to another embodiment of the present application;

FIG. 18 is a schematic view of a live view displayed on a spectator terminal according to an embodiment of the present application;

fig. 19 is a signaling interaction diagram of another live broadcast-based interaction method provided in the embodiment of the present application;

fig. 20 is a schematic diagram of a live view displayed by a anchor terminal according to an embodiment of the present application;

fig. 21 is a schematic diagram of a live view displayed by a viewer terminal according to an embodiment of the present application;

fig. 22 is a signaling interaction diagram of still another live broadcast-based interaction method according to an embodiment of the present application;

fig. 23 is a schematic view of a live view including a third widget image displayed by an anchor terminal according to an embodiment of the present application;

fig. 24 is a schematic diagram of a live view including a third widget image displayed by an audience terminal according to an embodiment of the present application;

fig. 25 is a schematic diagram of a live broadcast-based interaction device according to an embodiment of the present application;

fig. 26 is a schematic diagram of another live-based interaction apparatus provided in the embodiment of the present application;

fig. 27 is a schematic diagram of another live-based interaction apparatus provided in the embodiment of the present application;

fig. 28 is a schematic structural diagram of a terminal according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some, not all embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without any inventive step are within the scope of protection of the present application.

The terms "first," "second," and the like, in this application, are used for distinguishing between similar items and items that have substantially the same function or similar functionality, and it should be understood that "first," "second," and "nth" do not have any logical or temporal dependency, nor do they define a quantity or order of execution.

Artificial Intelligence (AI) is a theory, method, technique and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human Intelligence, perceive the environment, acquire knowledge and use the knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technique of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a manner similar to human intelligence. Artificial intelligence is the research of the design principle and the implementation method of various intelligent machines, so that the machines have the functions of perception, reasoning and decision making.

The artificial intelligence technology is a comprehensive subject and relates to the field of extensive technology, namely the technology of a hardware level and the technology of a software level. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

Fig. 1 is a schematic diagram of an implementation environment of a live broadcast-based interaction method provided in an embodiment of the present application, and referring to fig. 1, the implementation environment may include: anchor terminal 101, server 102, and viewer terminal 103.

Server 102 is connected directly or indirectly to anchor terminal 101 and viewer terminal 103, respectively, via wired or wireless communication. The anchor terminal 101 and the audience terminal 103 may each be, but are not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart audio, a smart watch, and the like.

The anchor terminal 101 has installed and operated a live application program for use in live broadcast of the anchor, and the anchor terminal 101 displays a live frame so that the anchor can set up a live room through the anchor terminal 101 for live broadcast. The anchor terminal 101 may display the acquired video data through a live frame, and may also encode the video data and send the encoded video data to the server 102. Anchor terminal 101 may refer broadly to one of a plurality of anchor terminals, and this embodiment is illustrated only with anchor terminal 101. Those skilled in the art will appreciate that the number of the anchor terminals may be greater or smaller, for example, the number of the anchor terminals may be only one, or the number of the anchor terminals may be tens or hundreds, or greater, and the number of the anchor terminals and the device types are not limited in the embodiments of the present application.

The server 102 may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud server, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a Content Delivery Network (CDN), a big data and artificial intelligence platform, and the like. The server 102 may encode based on the audio frame and the video frame uploaded by the anchor terminal 101 to obtain video data, may receive a video data acquisition request from each of the viewer terminals 103, and further may distribute the obtained video data to each of the viewer terminals 103 in response to the video data acquisition request, so that each of the viewer terminals 103 performs live broadcast based on the received video data. One skilled in the art will appreciate that the number of servers may be greater or less, and the present application is not limited thereto. Of course, the server 102 may also include other functional servers to provide more comprehensive and diverse services.

The viewer terminal 103 may have a live application installed and running thereon for viewing a live broadcast by a viewer, and the viewer terminal 103 may display a live broadcast picture so that the viewer can view the live broadcast through the viewer terminal 103. The audience terminal 103 may send the video data acquisition instruction to the server 102, receive video data returned by the server 102, decode the received video data to obtain an audio frame and a video frame, and perform live broadcast based on the audio frame and the video frame obtained by decoding. The spectator terminal 103 may be broadly referred to as one of a plurality of spectator terminals, and the embodiment is illustrated only with the spectator terminal 103. Those skilled in the art will appreciate that the number of the foregoing spectator terminals may be greater or smaller, for example, the number of the foregoing spectator terminals may be only one, or the number of the foregoing spectator terminals may be several tens or several hundreds, or greater, and the number of the spectator terminals and the type of the device are not limited in the embodiment of the present application.

It should be noted that, except for special descriptions, the anchor terminal related to the following embodiments of the live broadcast-based interaction method may be the anchor terminal 101, the server related to the following embodiments of the live broadcast-based interaction method may be the server 102, and the viewer terminal related to the following embodiments of the live broadcast-based interaction method may be the viewer terminal 103.

Fig. 2 is a flowchart of a live broadcast-based interaction method provided in an embodiment of the present application, where the method is applied to a anchor terminal, and referring to fig. 2, the method includes the following steps:

201. the anchor terminal displays a live broadcast picture of the live broadcast room, wherein a face part of a human face in the live broadcast picture is covered with a first mask image.

And the anchor terminal displays the live broadcast picture of the live broadcast room based on the acquired image. The camera shooting equipment transmits the collected images to the anchor terminal, and the anchor terminal processes the images after acquiring the images.

The anchor terminal may present the live view in any of the following ways.

In one possible implementation, the anchor terminal shows a live view in which the face of the person is covered with the first mask image based on the processed image. Based on the above, the anchor terminal also sends video data to the server based on the processed image, so that the video data is sent to the audience terminals through the server, and the terminals and the anchor terminal can see the same live broadcast picture.

In another possible implementation manner, the anchor terminal sends video data to the server based on the processed image, and the server distributes the video data to the anchor terminal and the audience terminal, at this time, the anchor terminal receives the video data of the live broadcast room from the server, decodes the received video data to obtain an audio frame and a video frame, and then displays a live broadcast picture of the live broadcast room based on the audio frame and the video frame.

202. The anchor terminal receives a first interaction instruction of any audience account number in a live broadcast room.

And the audience terminal responds to the triggering of the audience account number and generates a first interaction instruction. And after the audience terminal sends the first interaction instruction to the server, the server sends the first interaction instruction to the anchor terminal, and the anchor terminal receives the first interaction instruction from the server.

203. The anchor terminal responds to the first interaction instruction, a first face area of the face of the user is displayed in the live broadcast picture, and the first face area is at least part of the face area covered by the first facial image.

The anchor terminal is provided with at least one mask image, the first mask image is a mask image which is associated with the first interaction instruction, and after the first mask image is attached to the texture of the face of the person in the image, all or part of the area of the face of the person is covered by the first mask image, so that the covered area of the face of the person is not visible. For example, the first mask image is a Beijing opera facial makeup image, a dance mask image, a cartoon character face image, an animal face image, or the like.

According to the scheme provided by the embodiment of the application, before the first interaction instruction is received, the face part of the human face in the live broadcast picture is covered by the first mask image, at the moment, the audience cannot see the whole area of the face part of the human face, after the first interaction instruction is received, all or part of the face part of the human face covered by the first mask image is displayed in the live broadcast picture, at the moment, more regions of the face of the person can be seen by the audience, so that the interaction between the anchor and the audience is not unidirectional any more in the live video process, and the audience can obtain corresponding feedback by triggering a first interaction instruction, so that the interaction efficiency of the user is improved, and the live video watching experience of the audience is also improved.

Fig. 3 is a flowchart of a live broadcast-based interaction method provided in an embodiment of the present application, where the method is applied to a viewer terminal, and referring to fig. 3, the method includes the following steps:

301. and the audience terminal displays a live broadcast picture of the live broadcast room, wherein the face part in the live broadcast picture is covered with a first mask image.

The spectator terminal obtains video data from the server, shows the live broadcast picture based on the video data, and the face of the human face in the live broadcast picture covers has the first mask image, and the region covered by the first mask image is invisible.

It should be noted that, regarding the relevant content of the first mask image, reference is made to step 203 in the foregoing embodiment.

302. And the audience terminal responds to the operation implemented by the target audience account number in the live broadcast room, and displays a first face area of the face in the live broadcast picture, wherein the first face area is at least part of the face area covered by the first face image.

The method includes that a spectator terminal responds to an operation performed by a target spectator account, displays a first face region of a face in a live broadcast picture, and the operation performed by the spectator account is touch operation, voice input operation or gesture operation.

According to the scheme provided by the embodiment of the application, when the operation implemented by the target audience account number is not received, in the live broadcast picture displayed by the audience terminal, the first face area of the face of the person is covered by the first face image, at the moment, the audience cannot see the first face area in the live broadcast picture, after the operation implemented by the target audience account number is received, in the live broadcast picture displayed by the audience terminal, the first face area with the face of the person is displayed, at the moment, the first face image covering the first face area is uncovered, the audience can see more areas of the face of the person, therefore, the visible area of the face of the person in the live broadcast picture can be changed in response to the operation implemented by the audience, so that the audience can obtain corresponding feedback by implementing interactive operation, the user interaction efficiency is improved, and the experience of the audience for watching live broadcast video is also improved.

In the embodiment shown in fig. 2, a certain area on the anchor face in the original live view is covered with a mask image, and after the viewer triggers the first interaction command, the anchor face in the live view is no longer covered with the previous mask image, in another embodiment, see fig. 4, the anchor face in the original live view is not covered with a mask image, and after the viewer triggers the first interaction command, a certain area on the anchor face in the live view is covered with a mask image. Fig. 4 is a flowchart of a live broadcast-based interaction method provided in an embodiment of the present application, where the method is applied to a anchor terminal, and referring to fig. 4, the method includes the following steps:

401. and the anchor terminal displays a live broadcast picture of the live broadcast room, wherein a third face area with the face of the human face is displayed in the live broadcast picture.

And the third face area of the face of the human face is the whole area or partial area of the face of the human face. And when the third face area is the whole area of the face, the live broadcast picture displayed by the anchor terminal comprises the complete face appearance of the anchor. When the third face area is a partial area of the face of the human face, the area of the face of the human face except the third face area is covered, and the live broadcast picture displayed by the anchor terminal comprises the face appearance of the anchor part.

402. And the anchor terminal receives a second interaction instruction of any audience account number in the live broadcast room.

And the audience terminal responds to the triggering of the audience account number and generates a second interaction instruction. And after the audience terminal sends the second interaction instruction to the server, the server sends the second interaction instruction to the anchor terminal, and the anchor terminal receives the second interaction instruction from the server.

403. And the anchor terminal responds to the second interaction instruction, and displays a third facial image at a position corresponding to a third face area of the face in the live broadcast picture.

At least one mask image is provided on the anchor terminal, and the third mask image is a mask image in which the second interaction instruction is associated. And after the third mask image is attached to the texture of the face of the human face in the image, covering all or part of the area of the face of the human face by the third mask image, so that the covered area of the face of the human face is invisible. For example, the third mask image is a Beijing opera facial makeup image, a dance mask image, a cartoon character face image, an animal face image, or the like.

According to the scheme, before the second interaction instruction is received, the third face area with the face is displayed in the live broadcast picture, at the moment, the audience can see the third face area, after the second interaction instruction is received, the third face area is covered by the third face image in the live broadcast picture, at the moment, the audience cannot see the third face area, therefore, the interaction between the anchor and the audience is not unidirectional in the live video process, the audience can obtain corresponding feedback by triggering the second interaction instruction, the interaction efficiency of a user is improved, and the experience of watching live video by the audience is also improved.

Fig. 5 is a signaling interaction diagram of a live broadcast-based interaction method according to an embodiment of the present application, where the embodiment of the present application takes an interaction among a host terminal, a server, and a viewer terminal as an example, and with reference to fig. 5, the method includes:

501. the anchor terminal acquires an image and displays the acquired image in a live broadcast picture of a live broadcast room.

The anchor terminal is provided with and runs a live broadcast application program or an application program with a live broadcast function, and the anchor performs live broadcast through the application program.

After the anchor terminal starts the application program, the anchor terminal acquires an image acquired by the camera equipment, wherein the image comprises the face of the anchor. And after the anchor terminal acquires the image, displaying the acquired image through a live broadcast picture of the live broadcast room.

Fig. 6 is a schematic view of a live view displayed by a anchor terminal, where fig. 6 is a schematic view of a live view displayed by an anchor terminal, and an image of a face of an anchor is displayed in the live view.

502. The anchor terminal presents a mask selection button in the function list, which is used to present a list of candidate mask images.

The live broadcast room provides a function list, a plurality of function buttons including a mask selection button are displayed in the function list, when a user triggers the mask selection button, a candidate mask image list can be displayed in a live broadcast picture, and the candidate mask image list comprises at least one candidate mask image.

503. The anchor terminal presents a candidate mask image list in response to a trigger operation on a mask selection button, the candidate mask image list including at least one candidate mask image.

The candidate mask image list includes a plurality of mask labels, one mask label corresponding to each candidate mask image.

Referring to the live broadcast screen shown in fig. 6, a makeup button 601 serving as a mask selection button is displayed on the live broadcast screen, the anchor broadcasts triggers the makeup button 601 in the live broadcast screen, and the anchor broadcasts terminals respond to the trigger operation of the anchor broadcasts on the makeup button 601 to display a candidate mask image list, wherein the candidate mask image list includes mask labels corresponding to a plurality of candidate mask images such as a Beijing opera facial makeup 602, a glittering and translucent snow 603, a Xinjingyin color 604, a watercolor 605, a Yuanhui 606 and the like.

While the list of candidate mask images is illustrative of one possible implementation, in some embodiments, there are other ways for the user to select a mask image, such as including a thumbnail of at least one candidate mask image in the list of candidate mask images, or presenting a mask selection button on a live view to provide a candidate mask image instead of presenting the mask selection button in a list of functions, which is not limited in this application.

Based on the number of times that the user uses the candidate mask images, the mask labels in the candidate mask image list are sorted accordingly, so that the image labels corresponding to the frequently-used candidate mask images are arranged at the front of the list.

504. The anchor terminal responds to the fact that any candidate mask image is selected, the selected candidate mask image is used as a first mask image, and the acquired image is processed to enable the face part in the image to cover the first mask image.

And after the anchor terminal displays the candidate mask image list, responding to the selection operation of the anchor on any candidate mask image, and taking the selected candidate mask image as a first mask image. After the first facial appliance image is determined, the anchor terminal performs texture fitting on the first facial appliance image and the face part of the human face in the acquired image, so that the first facial appliance image covers a first face area of the face part of the human face in the image.

Referring to fig. 6, in response to that a Beijing opera facial makeup 602 is selected, the anchor terminal uses the Beijing opera facial makeup 602 as a first facial image, performs texture fitting on the Beijing opera facial makeup 602 and a face in the acquired image, covers the face with the Beijing opera facial makeup 602, and refers to fig. 7 for the face covered with the Beijing opera facial makeup 602, where fig. 7 is a schematic diagram of a live broadcast picture displayed by the anchor terminal according to an embodiment of the present disclosure.

In a possible implementation manner, for an acquired image, the live broadcast terminal performs texture fitting on the first facial part image and the face part in the image based on a first facial key point of the image and a second facial key point of the first facial part image so as to obtain an image in which a first facial part area of the face part is covered with the first facial part image. The first face key points are used for identifying the outline and the five sense organs of the face in the image, and the second face key points are used for identifying the outline and the five sense organs of the face in the image of the first facial mask.

Because the first face key point identifies the outline and the five sense organs of the face in the image, the second face key point identifies the outline and the five sense organs of the face in the first mask image, and the anchor terminal enables the first mask image to be matched with the face in the image based on the first face key point and the second face key point, so that the texture fitting of the first mask image and the face is accurately carried out, the effect of wearing the mask by the anchor is simulated, and the effect of watching live broadcast by final audiences is ensured.

Referring to fig. 8, fig. 8 is a schematic diagram of first face key points provided in an embodiment of the present application, where the first face key points identify outlines of faces and relative position relationships of eyes, a nose, a mouth, eyebrows, and the like in an image. Second face keypoints referring to fig. 9, where fig. 9 is a schematic diagram of second face keypoints provided in an embodiment of the present application, where the second face keypoints identify the outline of a face in a mask image and the relative positional relationships between the eyes, nose, and mouth.

Optionally, when the anchor terminal texture-fits the first mask image to the face of the person based on the first face key point and the second face key point, the anchor terminal is implemented by the following steps S1 to S4:

s1, acquiring a first face key point of the acquired image.

Aiming at the acquired image, the anchor terminal acquires a first face key point of the image based on an artificial intelligence technology. For example, a key point recognition model is trained in advance, and the anchor terminal obtains a first facial key point of the image by applying the key point recognition model.

And S2, determining a second mask key point of the first mask image.

Second facial key points of the first facial image are predetermined and stored in a live application installed on the anchor terminal, and the anchor terminal reads the second facial key points of the first facial image.

And S3, mapping the second face key points to the first face key points according to the labels of the face key points.

The first face key points and the second face key points are provided with corresponding labels, the face key points with different labels are used for identifying different positions on the face, and the face key points with each label are used for identifying specific positions on the face. Referring to fig. 8 and 9, the facial keypoints labeled 1-7 are used to identify the face contour, the facial keypoints labeled 8-13 are used to identify the mouth position, and the keypoints labeled 14-21 are used to identify the eyes position.

The anchor terminal maps each second face keypoint to a first face keypoint having the same label as it based on the label of the face keypoint. Referring to fig. 8 and 9, the anchor terminal maps the second face keypoints, labeled 1-28 in fig. 9, sequentially to the first face keypoints, labeled 1-28 in fig. 8.

And S4, performing texture fitting on the first mask image and the face part of the human face in the image according to the mapping result.

The anchor terminal combines a plurality of first key point groups based on the labels of the first face key points according to a preset key point combination rule, wherein each first key point group comprises three first face key points. The anchor terminal divides the face part in the acquired image based on each first key point group to acquire a plurality of triangular first image blocks, so that three first face part key points included in each first key point group are three vertexes of the corresponding first image blocks.

And the anchor terminal combines a plurality of second key point groups according to the key point combination rule and based on the labels of the second face key points, wherein each second key point group comprises three second face key points. The anchor terminal divides the first mask image based on each second key point group to obtain a plurality of triangular second image blocks, so that three second face key points included in each second key point group are three vertexes of the corresponding second image blocks, and no overlapping area exists between the second image blocks.

And the anchor terminal respectively determines the first image block associated with each second image block according to the mapping result, and three vertexes of the second image blocks are respectively mapped with the three vertexes of the associated first image blocks. And for each second image block, the anchor terminal carries out zooming deformation on the second image block, so that the zoomed deformed second image block has the same shape and size as the associated first image block, and the second image block after the edge zooming is in texture joint with the associated first image block, so that each vertex of the second image block is coincided with the mapped vertex on the first image block.

For example, the three second face keypoints denoted by

reference numerals

1, 2, and 14 in fig. 9 are combined into a second keypoint group, and based on the second keypoint group, the triangle in fig. 9 with the three second face keypoints denoted by

reference numerals

1, 2, and 14 as vertices is regarded as a second tile 1. Accordingly, the three first face keypoints denoted by

reference numerals

1, 2, and 14 in fig. 8 are combined into a first keypoint group 1, and then, based on the first keypoint group, a triangle in fig. 8 with the three first face keypoints denoted by

reference numerals

1, 2, and 14 as vertices is taken as a first tile 1. The second picture block 1 is subjected to zooming deformation, so that the second picture block 1 and the first picture block 1 have the same shape and size, and the zoomed and deformed second picture block 1 is subjected to texture fitting with the first picture block 1, so that

vertexes

1, 2 and 14 on the second picture block 1 are coincided with

vertexes

1, 2 and 14 on the first picture block 1 respectively.

The anchor terminal divides the first mask image into a plurality of second image blocks according to the mapping result between the first face key points and the second face key points, and carries out texture laminating with the corresponding region of the face of the human face after the second image blocks are zoomed and deformed, so that the face of the anchor has the effect of wearing the mask after the first mask image is laminated to the face of the human face in the image, and the authenticity of the effect of wearing the mask is improved.

It should be noted that the above-mentioned manner of performing texture fitting is only one possible implementation manner, and in some embodiments, there are other manners to perform texture fitting on the first mask image and the face of the human face in the acquired image, for example, the first mask image is divided into a plurality of quadrilateral image blocks based on the second facial key point, and then the quadrilateral image blocks are subjected to texture fitting on the face of the human face, and the like, which is not limited in this application.

505. The anchor terminal displays the face covered with the first mask image in the live broadcast picture.

And the anchor terminal displays the acquired image in a live broadcast picture after acquiring the image of the face covered by the first mask image.

The processed image displayed by the anchor terminal in the live broadcast picture can be seen in fig. 7, and the image of the face covered by the Beijing opera facial makeup is displayed by the anchor terminal in the live broadcast picture.

506. And the anchor terminal sends the video data to the server based on the processed image.

The anchor terminal processes the acquired image to obtain an image of the face covered with the first mask image, and then sends video data to the server based on the obtained image, wherein in the image in the video data, the face is covered with the first mask image.

And the anchor terminal processes the acquired image, so that the face part of the face in the processed image is covered with the first mask image. And the anchor terminal encodes the processed image to obtain video data, and sends the video data to the server in a wired or wireless communication mode. When encoding an image, the anchor terminal uses a common encoding format, such as H265, H264, and the like. A long connection (Websocket) is established between the anchor terminal and the server so as to realize the communication between the anchor terminal and the server.

It should be noted that, step 505 and step 506 may be executed synchronously or in the current order, or step 506 may be executed first and then step 505 is executed.

507. The server receives video data from the anchor terminal.

508. The server transmits the received video data to the viewer terminal.

And after receiving the video data from the anchor terminal, the server transmits the received video data to the audience terminal in a wired or wireless communication mode. The audience terminal is a terminal logged in with an audience account in a live broadcast room.

509. The viewer terminal receives video data from the server.

510. And the audience terminal displays a live broadcast picture based on the received video data, and the face part covered by the first mask image is displayed in the live broadcast picture.

After receiving the video data from the server, the audience terminal displays a live broadcast picture based on the received video data, so that the audience can see live broadcast content of the anchor through the audience terminal. In the live broadcast picture displayed by the audience terminal, the face of the human face is covered with the first mask image, and at the moment, the audience cannot see the whole area of the face of the main broadcast.

Referring to fig. 10, fig. 10 is a schematic view of a live broadcast displayed by a spectator terminal, where a human face and a face covered with a Beijing opera facial makeup image are displayed on the live broadcast in an embodiment of the present application.

511. The audience terminal responds to the virtual gift giving operation of the audience in the live broadcast room and sends a first interaction instruction to the server.

The audience terminal displays a virtual gift presenting button in a live broadcast picture, and the virtual gift presenting button is used for enabling the audience to realize virtual gift presenting operation. And the anchor terminal responds to the triggering operation of the audience on the virtual gift presenting button in the process of displaying the live broadcast picture, generates a first interaction instruction and sends the generated first interaction instruction to the server.

It should be noted that, for example, the audience terminal sends the first interaction instruction to the server in response to the virtual gift presenting operation, in some embodiments, the audience terminal sends the first interaction instruction to the server in response to other types of operations, or the audience terminal sends the corresponding first interaction instruction to the server in response to the virtual gift presenting operation for presenting different virtual gifts. For example, the audience terminal responds to a praise operation or a comment making operation, and sends a first interaction instruction to the server, the praise operation is used for praise of the live content of the anchor, and the comment making operation is used for making comment characters and expressions which can be seen by the anchor and each audience.

The audience terminal responds to virtual gift presenting operation, approval operation or comment issuing operation and sends a corresponding first interaction instruction to the server, or the audience terminal responds to virtual gift presenting operation for presenting different virtual gifts and sends a corresponding first interaction instruction to the server, so that the audience can trigger the first interaction instruction through various interaction operations, and the interestingness of watching live video by the audience can be improved.

Referring to the live view shown in fig. 10, a comment box 1001, a virtual gift giving button 1002, and a like button 1003 are displayed on the live view. After a viewer triggers the comment frame 1001, the viewer inputs comment characters and emoticons in the comment frame 1001, the viewer terminal responds to the triggering operation of the viewer on the comment frame 1001 and sends the comment characters and the emoticons input by the viewer to the server, and the server forwards the received comment characters and the emoticons to the anchor terminal and other viewer terminals, so that the anchor terminal and each viewer terminal display the received comment characters and the emoticons in a live broadcast picture. After the virtual gift giving button 1002 is triggered by the audience, the audience terminal sends a virtual gift giving instruction to the server in response to the triggering operation of the virtual gift giving button 1002 by the audience, and the server gives a virtual gift to the anchor account registered by the anchor terminal in response to the received virtual gift giving instruction. After the audience triggers the approval button 1003, the audience terminal responds to the triggering of the approval button 1003, and sends an approval instruction to the server, and the server forwards the received approval instruction to the anchor terminal and each audience terminal, so that the anchor terminal and each audience terminal display the approval effect on the anchor in the mean picture.

512. The server receives a first interaction instruction from the audience terminal.

513. And the server sends the received first interaction instruction to the anchor terminal.

514. And the anchor terminal responds to the first interaction instruction and stops processing the acquired image.

And the anchor terminal responds to a first interaction instruction from the server and stops processing the acquired image so as to display the whole area of the face in the live broadcast picture.

515. The anchor terminal displays a live broadcast picture based on the acquired image, and the whole area with the face of the human face is displayed in the live broadcast picture.

Starting from the acquired current frame image, the anchor terminal displays the image in a live broadcast picture, so that the whole area of the face of the anchor is displayed in the live broadcast picture.

Before the anchor terminal does not receive a first interaction instruction, the obtained image is processed, so that the face part in the image is covered with the first mask image, the image is displayed in a live broadcast picture, the face part in the live broadcast picture is covered with the first mask image, and the whole area of the face part cannot be seen. After the anchor terminal receives the first interaction instruction, the anchor terminal displays the acquired image in the live broadcast picture from the current moment when the first interaction instruction is received, at the moment, the face part of the face in the live broadcast picture is not covered by the first mask image any more, and the whole area of the face part of the face can be seen.

Because audience terminals are the same as the live broadcast picture displayed by the anchor terminal, audience only can see the partial region of the face in the live broadcast picture before executing interactive operation, and after executing interactive operation, the whole region of the face can be seen in the live broadcast picture, therefore, the live broadcast picture of the audience terminals is changed based on the interactive operation executed by the audience, so that the audience can see the real face of the anchor after executing the interactive operation, the interest of live video is improved, the interaction efficiency of users can be improved, and the experience of watching live video by the audience is also improved.

The anchor terminal responds to a live broadcast picture displayed by the first interaction instruction, and the live broadcast picture is shown in fig. 6, wherein the face and the face in the live broadcast picture are not covered by the mask image and are the same as the live broadcast picture when the anchor terminal does not select the mask image to cover.

516. The anchor terminal transmits video data to the server based on the acquired image.

It should be noted that, step 514 and step 515 may be executed synchronously or according to the current order, or step 515 is executed first and then step 514 is executed.

517. The server receives video data from the anchor terminal.

518. The server transmits the received video data to the viewer terminal.

519. The viewer terminal receives video data from the server.

520. And the audience terminal displays a live broadcast picture based on the received video data, and the live broadcast picture displays the whole area with the face.

After receiving the video data from the server, the audience terminal displays the live broadcast picture based on the received video data so as to display the whole face area in the live broadcast picture. Because the face and the face of the image in the video data received by the audience terminal are not covered by the mask image, the face and the face in the live broadcast picture displayed by the audience terminal are not covered by the mask image, and at the moment, the audience can see the whole area of the face and the face.

Referring to fig. 11, fig. 11 is a schematic view of another live view displayed by a spectator terminal, where an entire area of a face of a main player is displayed in the live view.

In the embodiment of the application, before the audience presents the virtual gift to the anchor, the face of the anchor face in the live broadcast picture is covered by the first face image, the audience cannot see the real face of the anchor, after the audience presents the virtual gift to the anchor, all areas with the face of the anchor face are displayed in the live broadcast picture, and the audience can see the real face of the anchor, so that the interaction between the anchor and the audience in the live video broadcasting process is not unidirectional any more, and the audience can obtain corresponding feedback by presenting the virtual gift, thereby improving the interaction efficiency of users and also improving the experience of the audience in watching live video.

In some embodiments, the terminal can also respond to the first interaction instruction, and display a special effect animation from a shielded state to a non-shielded state in the first face region of the face in the live broadcast picture, that is, a process that the face is off the mask, so that the display mode can provide a process that the face appears, viewers can have more real and visual viewing experience, and the visual experience is greatly improved.

In the embodiment shown in fig. 5, a certain area of the anchor face in the original live view is covered with a mask image, and after the viewer triggers the first interaction instruction, the anchor face in the live view is no longer covered with the previous mask image, so that the whole area of the anchor face is displayed in the live view. Fig. 12 is a signaling interaction diagram of another live broadcast-based interaction method according to an embodiment of the present application, where the embodiment of the present application takes an interaction among an anchor terminal, a server, and a viewer terminal as an example, and with reference to fig. 12, the method includes:

1201. the anchor terminal acquires an image and displays the acquired image in a live broadcast picture of a live broadcast room.

1202. The anchor terminal presents a mask selection button in the function list, which is used to present a list of candidate mask images.

1203. The anchor terminal presents a list of candidate mask images including at least one candidate mask image in response to a trigger operation of a mask selection button.

1204. The anchor terminal responds to that any candidate mask image is selected, the selected candidate mask image is used as a first mask image, the acquired image is processed based on the first mask image, the face part of the face in the image covers the first mask image, and the first mask image comprises at least two sub-mask images.

It should be noted that the first mask image includes at least two sub-mask images, and different sub-mask images have a common image area, or different sub-mask image portions include a common image area.

Fig. 13 is a schematic view of a first mask image provided in this embodiment of the present application, and fig. 14 is a schematic view of another first mask image provided in this embodiment of the present application. Referring to fig. 13, the first mask image is composed of a sub-mask image 1301, a sub-mask image 1302, and a sub-mask image 1303. Referring to fig. 14, the first mask image is composed of a sub-mask image 1401 and a sub-mask image 1402.

1205. The anchor terminal displays the face covered with the first mask image in the live broadcast picture.

1206. And the anchor terminal sends video data to the server based on the processed image.

1207. The server receives video data from the anchor terminal.

1208. The server transmits the received video data to the viewer terminal.

1209. The viewer terminal receives video data from the server.

1210. And the audience terminal displays a live broadcast picture based on the received video data, and the face part covered by the first mask image is displayed in the live broadcast picture.

It should be noted that, the above steps 1201-1210 refer to the steps 501-510 in the foregoing embodiment.

1211. The audience terminal sends a first interaction instruction associated with the target sub-mask to the server in response to a virtual gift giving operation of the audience in the live broadcast room.

The audience terminal responds to different interaction operations of audiences to generate different first interaction instructions, and the different first interaction instructions are associated with different sub-masks in the first mask image.

In a possible implementation manner, the triggering operation performed by the audience on the audience terminal is a virtual gift giving operation, a commenting operation or a commenting operation, the audience terminal generates an interaction instruction 1 in response to the virtual gift giving operation, the audience terminal generates an interaction instruction 2 in response to the commenting operation, and the audience terminal generates an interaction instruction 3 in response to the commenting operation. Referring to fig. 13, an interactive instruction 1 is associated with a sub-mask image 1301 included in the first mask image, an interactive instruction 2 is associated with a sub-mask image 1302 included in the first mask image, and an interactive instruction 3 is associated with a sub-mask image 1303 included in the first mask image.

The audience terminal responds to the virtual gift giving operation, sends the first interaction instruction to the server after taking the interaction instruction 1 as the first interaction instruction, and at the moment, the sub-mask image 1301 in the first mask image is the target sub-mask image.

In another possible implementation manner, the triggering operation performed by the viewer on the viewer terminal is a virtual gift giving operation, and the viewer terminal generates different interaction instructions in response to the virtual gift giving operation for different virtual gifts, wherein the different interaction instructions are associated with different sub-mask images in the first mask image. By associating the presented virtual gifts with the sub-mask images, when different virtual gifts are presented by audiences, different sub-mask images covering the face of the person can be uncovered to see different areas of the anchor face, so that the diversity and interestingness of interaction are increased.

For example, the viewer 1 presents the virtual gift 1 to the anchor, the viewer terminal generates the interactive instruction 4 in response to an operation of the viewer 1 presenting the virtual gift 1, the viewer 2 presents the virtual gift 2 to the anchor, and the viewer terminal generates the interactive instruction 5 in response to an operation of the viewer 2 presenting the virtual gift 2. Referring to fig. 14, a sub-mask image 1401 is associated with the interactive instruction 4 and a sub-mask image 1402 is associated with the interactive instruction 5.

1212. The server receives a first interaction instruction from the audience terminal.

1213. And the server sends the received first interaction instruction to the anchor terminal.

It should be noted that, the steps 1211 to 1213 refer to the steps 511 to 513 in the foregoing embodiment.

1214. The anchor terminal responds to the first interaction instruction and determines a target sub-mask image in the first mask image, wherein the target sub-mask image is associated with the first interaction instruction.

And the anchor terminal responds to the received first interaction instruction, and determines a target sub-mask image associated with the first interaction instruction from the sub-mask images included in the first mask image.

1215. And processing the acquired image based on the sub-mask image except the target sub-mask image in the first mask image by the anchor terminal to enable the face and the face in the image not to be covered by the target sub-mask image.

The anchor terminal determines a third face key point based on the target sub-mask image, wherein the third face key point is used for identifying the outline and the five sense organs of the face in the sub-mask image except the target sub-mask image in the first mask image.

And the anchor terminal performs texture fitting on the sub-mask images except the target sub-mask image in the first mask image and the face of the human in the image based on the first face key point of the image and the third face key point of the first mask image from the acquired current frame image, so that the face of the human in the image is not covered by the target sub-mask image.

In response to receiving the interaction instruction 1, the anchor terminal determines the sub-mask image 1301 as a target sub-mask image, performs texture fitting on the mask image except the sub-mask image 1301 in the first mask image and the face of the image in the video data, and obtains an image in which the face is covered by the sub-mask image 1302 and the sub-mask image 1303. In response to the received interaction instruction 4, referring to fig. 14, the anchor terminal determines the sub-mask image 1401 as a target sub-mask image, performs texture fitting on the mask image except the sub-mask image 1401 in the first mask image and the face of the image in the video data, and obtains an image in which the face of the included face is covered by the sub-mask image 1402.

It should be noted that, the step 504 in the foregoing embodiment is referred to in the process of texture-fitting each of the sub-mask images except for the target sub-mask image to the face of the human face.

1216. And the anchor terminal displays the live broadcast picture, and the face of the human face in the live broadcast picture is not covered by the target sub-mask image.

And after the anchor terminal acquires the image of the face of the person which is not covered by the target sub-mask image, displaying the acquired image in a live broadcast picture.

The anchor terminal responds to the received interactive instruction 1 to process images in the video data, a live broadcast picture of the anchor terminal is shown in fig. 15, and fig. 15 applies to a schematic diagram of a live broadcast picture displayed by the anchor terminal provided by an embodiment. The anchor terminal responds to the received interaction instruction 4 to process the image in the video data, the live broadcast picture of the anchor terminal is shown in fig. 16, and fig. 16 is a schematic view of the live broadcast picture displayed by another anchor terminal provided by the embodiment of the application.

1217. And the anchor terminal sends video data to the server based on the image of the face not covered by the target sub-mask image.

1218. The server receives video data from the anchor terminal.

1219. The server transmits the received video data to the viewer terminal.

1220. The viewer terminal receives video data from the server.

It should be noted that steps 1217-1220 refer to steps 516-519 of the previous embodiment.

1221. And the audience terminal displays a live broadcast picture based on the received video data, wherein the face of the human face in the live broadcast picture is not covered by the target sub-mask image.

After receiving the video data from the server, the audience terminal displays a live broadcast picture based on the image in the video data, wherein the face of the human face in the live broadcast picture is not covered by the target sub-mask image, and the face of the human face is covered by other sub-mask images except the target sub-mask image.

In the live broadcast picture, the sub-mask image 1301 in the first mask image does not cover the face of a human face, and at this time, the audience can see the face of the head of the main broadcast. In response to the operation of the audience 2 giving the virtual gift 1 to the anchor broadcast, when the audience terminal sends the interaction instruction 4 to the server, the live broadcast picture displayed by the audience terminal refers to fig. 18, where fig. 18 is a schematic view of another live broadcast picture displayed by the audience terminal according to the embodiment of the present application, in the live broadcast picture, the sub-mask image 1401 in the first mask image does not cover the face of the human face, and at this time, the audience can see the face of the left face of the anchor broadcast.

In some embodiments, the terminal can also respond to the first interaction instruction, and display a special effect animation from the shielded state to the non-shielded state in the corresponding face area of the face in the live broadcast picture, that is, a process of taking off part of the mask from the face.

In the embodiment of the application, before the audience presents the virtual gift to the anchor, the face of the anchor face in the live broadcast picture is covered by the first mask image, the audience cannot see the area covered by the first mask image of the face of the anchor face, after the audience presents the virtual gift to the anchor, the target sub-mask image included by the first mask image in the live broadcast picture does not cover the face of the anchor face any more, the audience can see the anchor face covered by the target sub-mask image, therefore, the interaction between the anchor and the audience in the video live broadcast process is not unidirectional any more, and the audience can obtain corresponding feedback by presenting the virtual gift, so that the interaction efficiency of the user is improved, and the experience of the audience watching the video live broadcast is also improved.

In the embodiment shown in fig. 12, the anchor face in the original live broadcast screen is covered with the first mask image, and after the viewer triggers the first interaction instruction, a certain partial area of the anchor face previously covered with the first mask image is displayed in the live broadcast screen, and in another embodiment, referring to fig. 19, a certain area of the anchor face in the original live broadcast screen is covered with a mask image, and after the viewer performs the virtual gift giving operation, the anchor face in the live broadcast screen is not covered with the previous mask image, but a mask image is added to a different area of the anchor face. Fig. 19 is a signaling interaction diagram of another live broadcast-based interaction method according to an embodiment of the present application, where the embodiment of the present application takes an interaction among a anchor terminal, a server, and a viewer terminal as an example, and with reference to fig. 19, the method includes:

1901. the anchor terminal acquires an image and displays the acquired image in a live frame of a live broadcasting room.

1902. The anchor terminal presents a mask selection button in a function list, the mask selection button being used to present a list of candidate mask images.

1903. The anchor terminal presents a list of candidate mask images including at least one candidate mask image in response to a trigger operation of a mask selection button.

1904. The anchor terminal responds to the fact that any candidate mask image is selected, the selected candidate mask image is used as a first mask image, the acquired image is processed based on the first mask image, and a first face area of the face of the person in the image is covered by the first mask image.

The anchor terminal responds to the trigger operation of the anchor, processes the acquired image, so that the face part in the image is covered with a first mask image, namely a first face area of the face part in the image is covered by the first mask image, and a second face area of the face part in the image is partially or completely visible.

For example, the first facial image is the right half of a Beijing opera facial makeup, the second facial image is the left half of the Beijing opera facial makeup, the anchor terminal responds to the trigger operation of the anchor and processes the image in the video data, the right half of the face of the human face in the image is covered by the first facial image, the left half of the face of the human face in the image is visible, and the processed image can be referred to as the image of the face of the human face in FIG. 16.

1905. The anchor terminal displays the face covered with the first mask image in the live broadcast picture.

The anchor terminal displays a live broadcast picture based on an image of the face part covered by the first mask image, the live broadcast picture refers to fig. 16, the appearance of the right half face of the face part in the live broadcast picture is invisible due to the fact that the face part is covered by the first mask image, and the appearance of the left half face of the face part is visible.

1906. And the anchor terminal sends the video data to the server based on the processed image.

1907. The server receives video data from the anchor terminal.

1908. The server transmits the received video data to the viewer terminal.

1909. The viewer terminal receives video data from the server.

1910. And the audience terminal displays a live broadcast picture based on the received video data, and the face part covering the first mask image is displayed in the live broadcast picture.

The audience terminal displays a live broadcast picture based on the received video data, and a first face area of the face of the audience is covered by a first face image in the live broadcast picture.

In the live broadcast picture displayed by the audience terminal, referring to fig. 18, the appearance of the right half face of the face in the live broadcast picture is invisible because of being covered by the first mask image, and the appearance of the left half face of the face is visible.

1911. The audience terminal responds to the virtual gift presentation operation of the audience in the live broadcast room and sends a first interaction instruction to the server.

1912. The server receives a first interaction instruction from the audience terminal.

1913. And the server sends the received first interaction instruction to the anchor terminal.

1914. And the anchor terminal responds to the first interaction instruction and stops processing the acquired image based on the first facial instrument image.

It should be noted that, in the above steps 1901 to 1914, reference is made to steps 501 to 514 in the previous embodiment.

1915. And the anchor terminal processes the image in the acquired video data based on the second facial mask image associated with the first interaction instruction, so that a second face area of the face of the person in the image is covered by the second facial mask image.

And the anchor terminal responds to the received first interaction instruction, and carries out texture fitting on the second facial mask image and the face part of the human face in the image from the obtained current frame image so as to obtain an image of which the second face area of the face part is covered by the second facial mask image.

Fig. 20 is a schematic view of a live broadcast picture displayed by the anchor terminal according to an embodiment of the present application, where the face of the human face covered with the second facial image is visible, and the face of the right half of the face of the human face is invisible due to being covered with the second facial image.

It should be noted that, the second mask image and the face of the human face are texture-fitted, reference is made to the description of fitting the first mask image and the face of the human face in the foregoing embodiment, and specific reference is made to step 504 in the foregoing embodiment.

1916. The anchor terminal displays the live broadcast picture, and the face part of the human face in the live broadcast picture is covered with a second facial instrument image.

And after the anchor terminal acquires the image of which the face part is covered by the second facial tool image, displaying the acquired image in a live broadcast picture.

The live view shown by the anchor terminal can be seen in fig. 20.

1917. And the anchor terminal sends video data to the server based on the image of the face covered by the second mask image.

1918. The server receives video data from the anchor terminal.

1919. The server transmits the received video data to the viewer terminal.

1920. The viewer terminal receives video data from the server.

It should be noted that steps 1917-1920 are described with reference to steps 516-519 of the previous embodiment.

1921. And the audience terminal displays the live broadcast picture based on the received video data, and the face part of the human face in the live broadcast picture is covered by the second mask image.

And after receiving the video data from the server, the audience terminal displays the live broadcast picture based on the image in the video data, and a second face area of the face in the live broadcast picture is covered by a second facial image.

Fig. 21 is a schematic view of a live broadcast picture displayed by a spectator terminal, where in the live broadcast picture, the appearance of the left half face of the face of a person is invisible because the face of the person is covered by a second facial image, and the appearance of the right half face of the person is visible.

In the embodiment of the application, before the audience presents the virtual gift to the anchor, the first face area of the face of the audience in the live broadcast picture is covered by the first facial mask image, the audience cannot see the first face area, but can see the second face area of the face of the audience, after the audience presents the virtual gift to the anchor, the second face area in the live broadcast picture is covered by the second facial mask image, the audience cannot see the second face area, but can see the first face area, therefore, the interaction between the anchor and the audience in the live video broadcast process is not unidirectional any more, the audience can obtain corresponding feedback by presenting the virtual gift, thereby improving the interaction efficiency of users, and also improving the experience of the audience watching the live video broadcast.

In the embodiments shown in fig. 5, 12 and 19, a certain area on the anchor face in the original live broadcast screen is covered with a mask image, and after the viewer triggers the first interaction instruction, all or a part of the area on the anchor face previously covered by the first mask image is displayed in the live broadcast screen, and in other embodiments, referring to fig. 22, the anchor face in the original live broadcast screen is not covered by the mask image, and after the viewer triggers the second interaction instruction, a certain area on the anchor face in the live broadcast screen is covered with a mask image. Fig. 22 is a signaling interaction diagram of another live broadcast-based interaction method according to an embodiment of the present application, where the interaction between the anchor terminal, the server, and the viewer terminal is taken as an example for explanation, and referring to fig. 22, the method includes:

2201. the anchor terminal acquires an image and displays the acquired image in a live frame of a live broadcasting room.

It should be noted that, in step 2201, reference is made to step 501 in the foregoing embodiment.

2202. And the anchor terminal sends video data to the server based on the acquired image.

It should be noted that, step 2201 and step 2202 may be executed synchronously or in the current order, or step 2202 is executed first and then step 2201 is executed.

2203. The server receives video data from the anchor terminal.

2204. The server transmits the received video data to the viewer terminal.

2205. The viewer terminal receives video data from the server.

2206. And the audience terminal displays a live broadcast picture based on the received video data, and the live broadcast picture displays the whole area with the face.

It should be noted that steps 2202-2206 refer to steps 516-520 in the previous embodiment.

The live broadcast displayed by the audience terminal is shown in fig. 11, and the whole area of the face is displayed in the live broadcast.

2207. And the audience terminal responds to the virtual gift giving operation of the audience in the live broadcast room and sends a second interaction instruction to the server.

It should be noted that, the viewer terminal sends the second interaction instruction to the server, refer to the foregoing embodiment in which the viewer terminal sends the first interaction instruction to the server, and specifically refer to step 510 in the foregoing embodiment.

2208. The server receives a second interaction instruction from the audience terminal.

2209. And the server sends the received second interaction instruction to the anchor terminal.

It should be noted that steps 2208-2209 refer to steps 512-513 in the previous embodiment.

2210. And the anchor terminal responds to the second interaction instruction and processes the acquired image, so that the face part in the image covers the third facial image.

The anchor terminal is provided with at least one mask image, the third mask image is a mask image which is associated with the second interaction instruction, and after the third mask image is attached to the face texture in the image, all or part of the face is covered by the third mask image, so that the covered area of the face is invisible. For example, the first mask image is a Beijing opera facial makeup image, a dance mask image, a cartoon character face image, an animal face image, or the like.

And the anchor terminal responds to the second interaction instruction, and carries out texture fitting on the third facial image and the face in the acquired image, so that the face in the image is covered with the third facial image.

Wherein, the human face that covers the third utensil image can refer to fig. 23, and fig. 23 is the schematic diagram of the live broadcast picture that anchor terminal station that this application embodiment provided shows, and the human face is covered by the third utensil image in this live broadcast picture, and the third utensil image is pig nose pattern.

It should be noted that, texture fitting is performed on the third facial image and the face in the acquired image, so that the face in the image is covered with the third facial image, reference is made to the description of performing texture fitting on the first facial image and the face in the acquired image in the foregoing embodiment, and specifically refer to step 504 in the foregoing embodiment.

2211. And the anchor terminal displays the face covered with the third mask image in the live broadcast picture.

And the anchor terminal responds to the second interaction instruction, displays the face covered with the third widget image in the live broadcast picture, the displayed live broadcast picture can refer to fig. 23, and the nose area of the face in the live broadcast picture is covered by the third widget image.

2212. And the anchor terminal sends video data to the server based on the processed image.

2213. The server receives video data from the anchor terminal.

2214. The server transmits the received video data to the viewer terminal.

2215. The viewer terminal receives video data from the server.

It should be noted that steps 2212-2215 refer to steps 506-509 in the foregoing embodiments.

2216. And the audience terminal displays a live broadcast picture based on the received video data, and the face part covered by the third mask image is displayed in the live broadcast picture.

And after receiving the video data from the server, the audience terminal displays the live broadcast picture based on the image in the video data, and the face part of the human face in the live broadcast picture is covered by the third mask image.

Fig. 24 is a schematic view of a live view displayed by a spectator terminal, where a nose area of a face of a person in the live view is covered by a third widget image according to an embodiment of the present application.

In the embodiment of the application, before spectator carries out interactive operation, the whole region that includes the human face in the live broadcast picture that spectator terminal showed, spectator can see the complete facial appearance of anchor now, after spectator carries out interactive operation, in the live broadcast picture that spectator terminal showed, human face covers there is third utensil image, the subregion of anchor human face is covered by third utensil this moment, make spectator's interactive operation can obtain the feedback on the one hand, improve user interaction efficiency, on the other hand spectator can cover the pernicious mask image on the facial image of anchor, help promoting the interest of live video, consequently, can promote spectator's live video experience.

In any of the above embodiments, the terminal can also respond to the received interactive instruction, and display the special effect animation from the shielded to the non-shielded special effect animation or from the non-shielded to the shielded special effect animation in the corresponding face area of the face in the live broadcast picture, so that the audience has more real and intuitive viewing experience, and the visual experience is greatly improved.

It should be noted that all the optional technical solutions in the foregoing method embodiments may be combined arbitrarily to form an optional embodiment of the present application, and details are not described herein again.

Fig. 25 is a block diagram of a live broadcast-based interaction device provided in an embodiment of the present application, and referring to fig. 25, the device includes:

the first display module 2501 is configured to display a live broadcast picture in a live broadcast room, where a face part in the live broadcast picture is covered with a first mask image;

a receiving module 2502, configured to receive a first interaction instruction of any viewer account in a live broadcast room;

the first display module 2501 is further configured to display, in response to the first interaction instruction received by the receiving module 2502, a first face region of a face of a person in a live broadcast, where the first face region is at least a part of a face region covered by the first facial image.

The device that this application embodiment provided, through receiving the first interactive instruction that comes from audience account, before receiving first interactive instruction, show in the live broadcast picture and cover the face that has first face utensil image, after receiving first interactive instruction, show in the live broadcast picture before by the partial or whole face region that first face utensil image covered, make spectator can be through interactive operation, uncover the facial mask of anchor in the live broadcast picture, can see the face of anchor, consequently the interaction between anchor and spectator is no longer unidirectional among the live video in-process, spectator can obtain corresponding feedback through triggering first interactive instruction, thereby improve user interaction efficiency, also promote spectator and watch the live experience of video.

the image processing module is used for performing texture fitting on the first facial feature image and the face part of the human face in the image based on the first facial key point of the image and the second facial key point of the first facial feature image for the acquired image;

the first display module 2501 is configured to display a face and a face after texture fitting by the image processing module in a live broadcast picture;

In one possible implementation, the image processing module is configured to perform the following processing:

acquiring a first face key point of an acquired image;

mapping second facial keypoints of the first mask image to first facial keypoints of the image;

and according to the mapping result, performing texture fitting on the first mask image and the face part of the human face in the image.

In a possible implementation manner, the first presenting module 2501 is configured to present an entire region of the face of the human in a live view.

In a possible implementation manner, the first displaying module 2501 is configured to display the acquired image in a live view.

In one possible implementation, the first mask image comprises at least two sub-mask images, different sub-mask images being associated with different interaction instructions;

the first display module 2501 is configured to display, in a live broadcast, a face area of the face covered by a target sub-mask image, where the target sub-mask image is a sub-mask image associated with the first interaction instruction.

In a possible implementation manner, the first presentation module 2501 is configured to perform the following processing:

performing texture fitting on the sub-mask images except the target sub-mask image in the first mask image and the face part in the acquired image based on the first face key point and the third face key point of the first mask image;

displaying the face after texture fitting in a live broadcast picture;

the first face key point is used for identifying the outline and the five sense organs of the face in the acquired image, and the third face key point is used for identifying the outline and the five sense organs of the face in the sub-mask image except the target sub-mask image in the first mask image.

In a possible implementation manner, the first displaying module 2501 is further configured to display, in response to the first interaction instruction, a second facial image at a corresponding position of a second face area of the face of the human face in the live view.

In a possible implementation manner, the first interaction instruction is triggered by an interaction operation of the audience account, the interaction operation is a virtual gift giving operation, a praise operation or a comment making operation, the virtual gift giving operation for giving different virtual gifts corresponds to different first interaction instructions, and the different first interaction instructions correspond to different face regions of the face of the person.

In a possible implementation manner, the first display module 2501 is further configured to display, in response to the first interaction instruction, a special effect animation from being blocked to not being blocked of a first face region of the face of the human face in the live view.

It should be noted that, the apparatus provided in this embodiment of the present application is applied to a anchor terminal, and each module in this embodiment of the present application may perform the processing performed by the anchor terminal in each method embodiment described above, so that specific processing procedures of each module may refer to the description in each method embodiment described above, and are not described again here.

Fig. 26 is a block diagram of a live broadcast-based interaction device provided in an embodiment of the present application, and referring to fig. 26, the device includes:

the second display module 2601 is configured to display a live broadcast picture in a live broadcast room, where a face in the live broadcast picture is covered with a first mask image;

the second presenting module 2601 is further configured to present, in response to an operation performed by the target audience account in the live broadcast, a first face area of the face of the person in the live broadcast, where the first face area is at least a part of a face area covered by the first mask image.

In the embodiment of the application, when the target audience account number does not implement the operation, the live broadcast picture shows the face and face covered with the first mask image, after the target audience account number implements the operation, the part or the whole face area covered by the first mask image in the live broadcast picture enables audiences to uncover the face mask of the main broadcast through the interactive operation, the face of the main broadcast can be seen, the interactive operation of the audiences in the video live broadcast process can be fed back, the user interaction efficiency is improved, and the experience of watching the video live broadcast by the audiences is also improved.

It should be noted that, the apparatus provided in this embodiment of the present application is applied to a viewer terminal, and each module in this embodiment of the present application may perform processing performed by the viewer terminal in each method embodiment described above, so specific processing procedures of each module may refer to descriptions in each method embodiment described above, and are not described herein again.

Fig. 27 is a block diagram of another live broadcast-based interaction device provided in an embodiment of the present application, and referring to fig. 27, the device includes:

a third display module 2701, configured to display a live broadcast picture of a live broadcast room, where a third face area with a face is displayed in the live broadcast picture;

the instruction receiving module 2702 is configured to receive a second interaction instruction of any viewer account in the live broadcast room;

the third displaying module 2701 is further configured to display a third facial image at a position corresponding to a third face area of the face of the person in the live broadcast picture in response to the second interaction instruction received by the instruction receiving module 2702.

In the embodiment of the application, through receiving the second interactive instruction from the audience account number, before receiving the second interactive instruction, show the whole region of human face in the live broadcast picture, audience can see the whole region of anchor face at this moment, after receiving the second interactive instruction, show the human face that covers and have the third utensil image in the live broadcast picture, audience can only see the part region of anchor face or can not see anchor face at this moment, consequently, the interaction between anchor and audience is no longer unidirectional in the live video process, audience can obtain corresponding feedback through triggering the second interactive instruction, thereby improve user interaction efficiency, also promote audience and watch the live experience of video.

An embodiment of the present application provides a computer device, configured to execute the methods provided in the foregoing embodiments, where the computer device may be implemented as a terminal, and a structure of the terminal is described below:

fig. 28 is a schematic structural diagram of a terminal according to an embodiment of the present application. The terminal 2800 may be: a smartphone, a tablet computer, a laptop computer, or a desktop computer. Terminal 2800 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and so on.

Generally, terminal 2800 includes: one or more processors 2801 and one or more memories 2802.

Processor 2801 may include one or more processing cores such as 4-core processors, 8-core processors, and so forth. The processor 2801 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 2801 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also referred to as a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 2801 may be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content that the display screen needs to display. In some embodiments, the processor 2801 may further include an AI (Artificial Intelligence) processor for processing computational operations related to machine learning.

Memory 2802 may include one or more computer-readable storage media, which may be non-transitory. Memory 2802 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 2802 is used to store at least one computer program for execution by processor 2801 to implement the live based interaction methods provided by method embodiments herein.

In some embodiments, terminal 2800 may also optionally include: a peripheral interface 2803 and at least one peripheral. The processor 2801, memory 2802, and peripheral interface 2803 may be connected by buses or signal lines. Various peripheral devices may be connected to peripheral interface 2803 by buses, signal lines, or circuit boards. Specifically, the peripheral device includes: at least one of a radio frequency circuit 2804, a display screen 2805, a camera assembly 2806, an audio circuit 2807, and a power supply 2808.

Peripheral interface 2803 can be used to connect at least one peripheral associated with I/O (Input/Output) to processor 2801 and memory 2802. In some embodiments, processor 2801, memory 2802, and peripheral interface 2803 are integrated on the same chip or circuit board; in some other embodiments, any one or both of the processor 2801, the memory 2802, and the peripheral interface 2803 may be implemented on separate chips or circuit boards, which are not limited in this embodiment.

The Radio Frequency circuit 2804 is used to receive and transmit RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuit 2804 communicates with a communication network and other communication devices by electromagnetic signals. The radio frequency circuit 2804 converts an electric signal into an electromagnetic signal for transmission, or converts a received electromagnetic signal into an electric signal. Optionally, the radio frequency circuit 2804 comprises: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth.

The display screen 2805 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When display screen 2805 is a touch display screen, display screen 2805 also has the ability to capture touch signals on or over the surface of display screen 2805. The touch signal may be input to the processor 2801 as a control signal for processing. At this point, the display 2805 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard.

The camera assembly 2806 is used to capture images or video. Optionally, camera head assembly 2806 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of a terminal, and a rear camera is disposed at a rear surface of the terminal.

The audio circuit 2807 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 2801 for processing or inputting the electric signals to the radio frequency circuit 2804 for realizing voice communication.

Power supply 2808 is used to provide power to various components within terminal 2800. Power supply 2808 can be alternating current, direct current, disposable batteries, or rechargeable batteries.

In some embodiments, terminal 2800 also includes one or more sensors 2809. The one or more sensors 2809 include, but are not limited to: acceleration sensor 2810, gyro sensor 2811, pressure sensor 2812, optical sensor 2813, and proximity sensor 2814.

The acceleration sensor 2810 can detect the magnitude of acceleration on three coordinate axes of a coordinate system established with the terminal 2800.

The gyro sensor 2811 may acquire a body direction and a rotation angle of the terminal 2800, and the gyro sensor 2811 may acquire a 3D motion of the user on the terminal 2800 in cooperation with the acceleration sensor 2810.

Pressure sensors 2812 may be disposed on side frames of terminal 2800 and/or underneath display 2805. When the pressure sensor 2812 is disposed in a side frame of the terminal 2800, a user's grip signal on the terminal 2800 can be detected, and the processor 2801 performs right-left hand recognition or shortcut operation based on the grip signal acquired by the pressure sensor 2812. When the pressure sensor 2812 is disposed below the display screen 2805, the processor 2801 controls operability controls on the UI interface in accordance with a user's pressure operation on the display screen 2805.

Optical sensor 2813 is used to collect ambient light intensity. In one embodiment, processor 2801 may control the display brightness of display screen 2805 based on the ambient light intensity collected by optical sensor 2813.

Proximity sensor 2814 is used to gather the distance between the user and the front face of terminal 2800.

Those skilled in the art will appreciate that the architecture shown in fig. 28 does not constitute a limitation on terminal 2800, and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components may be used.

In an exemplary embodiment, a computer-readable storage medium, such as a memory including a computer program, is also provided, the computer program being executable by a processor to perform the live-based interaction method of the above embodiments. For example, the computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, and the like.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only exemplary of the present application and should not be taken as limiting, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A live-based interaction method, performed by a anchor client, comprising:

displaying a live broadcast picture of a live broadcast room, wherein a face of a main broadcast in the live broadcast picture is covered with a first mask image, the first mask image comprises at least two sub-mask images, different sub-mask images are associated with different interaction instructions, and adjacent sub-mask images comprise a public image area;

receiving a first interaction instruction of any audience account in a live broadcast room, wherein the first interaction instruction is triggered by interaction operation of the audience account, the interaction operation is virtual gift presenting operation, approval operation or comment issuing operation, different interaction operations correspond to different first interaction instructions, the virtual gift presenting operation for presenting different virtual gifts corresponds to different first interaction instructions, and the different first interaction instructions correspond to different face regions of the face;

responding to the first interaction instruction, displaying a face area covered by a target sub-mask image on the face of the human face in the live broadcast picture, and continuously covering other sub-mask images except the target sub-mask image on the face of the human face in the first mask image, wherein the target sub-mask image is the sub-mask image associated with the first interaction instruction.

2. The method of claim 1, wherein the displaying the live view of the live room comprises:

performing texture fitting on the first facial tool image and the face part in the image based on a first facial key point of the image and a second facial key point of the first facial tool image for the acquired image;

displaying the face after texture fitting in the live broadcast picture;

the first face key points are used for identifying the outline and the five sense organs of the face in the image, and the second face key points are used for identifying the outline and the five sense organs of the face in the first face image.

3. The method according to claim 2, wherein for the acquired image, texture fitting the first facial tool image to the face part of the human face in the image based on a first facial key point of the image and a second facial key point of the first facial tool image comprises:

for the acquired image, acquiring the first facial key point of the image;

and according to the mapping result, performing texture fitting on the first mask image and the face part in the image.

4. The method of claim 1, further comprising:

and responding to the first interaction instruction, and displaying the whole area of the face in the live broadcast picture.

5. The method according to claim 1, wherein the displaying, in the live view, a face area covered by a target sub-mask image on the face of the human face comprises:

displaying the face after texture fitting in the live broadcast picture;

6. The method of claim 1, further comprising:

and responding to the first interaction instruction, and displaying a second facial image at a position corresponding to a second face area of the face of the human face in the live broadcast picture.

7. The method according to any one of claims 1 to 6,

in response to the first interaction instruction, displaying a special effect animation of a first face area of the face from being blocked to not being blocked in the live broadcast picture, wherein the first face area is at least part of the face area covered by the first facial image.

8. A live-based interaction method, performed by a viewer client, comprising:

in response to an operation performed by a target audience account in the live broadcast room, displaying a first face area covered by a target sub-mask image on the face of the human face in the live broadcast picture, where the first face area is at least a part of the face area covered by the first face image, and other sub-mask images except the target sub-mask image in the first face image continue to be covered on the face of the human face, the target sub-mask image is a sub-mask image associated with a first interaction instruction, the first interaction instruction is generated by an anchor client according to the operation performed by the target audience account, the operation is a virtual gift giving operation, a commenting operation or a commenting operation, different operations correspond to different first interaction instructions, and the different first interaction instructions are used for giving the virtual gift giving operations of different virtual gifts corresponding to different first interaction instructions, and different first interaction instructions correspond to different areas of the face of the human face.

9. A live broadcast-based interaction device, comprising:

the first display module is used for displaying a live broadcast picture of a live broadcast room, wherein a face of a main broadcast in the live broadcast picture is covered with a first mask image, the first mask image comprises at least two sub-mask images, different sub-mask images are associated with different interaction instructions, and adjacent sub-mask images comprise a public image area;

the receiving module is used for receiving a first interaction instruction of any audience account in a live broadcast room, the first interaction instruction is triggered by interaction operation of the audience account, the interaction operation is virtual gift presenting operation, approval operation or comment issuing operation, different interaction operations correspond to different first interaction instructions, the virtual gift presenting operation for presenting different virtual gifts corresponds to different first interaction instructions, and the different first interaction instructions correspond to different face regions of the face;

the first display module is further used for responding to the first interaction instruction received by the receiving module, displaying a face area covered by a target sub-mask image on the face of the human face in the live broadcast picture, continuously covering other sub-mask images except the target sub-mask image on the face of the human face in the first mask image, and enabling the target sub-mask image to be a sub-mask image associated with the first interaction instruction.

10. A live broadcast-based interaction device, comprising:

the second display module is used for displaying a live broadcast picture of a live broadcast room, wherein a face of a main broadcast in the live broadcast picture is covered with a first mask image, the first mask image comprises at least two sub-mask images, different sub-mask images are associated with different interaction instructions, and adjacent sub-mask images comprise a public image area;

the second display module is further configured to display, in response to an operation performed by a target audience account in the live broadcast room, a first face area covered by a target sub-mask image on the face of the human face in the live broadcast picture, where the first face area is at least a part of a face area covered by the first face image, and other sub-mask images except the target sub-mask image in the first face image continue to cover the face of the human face, the target sub-mask image is a sub-mask image associated with a first interaction instruction, the first interaction instruction is generated by the main broadcast client according to the operation performed by the target audience account, the operation is a virtual gift giving operation, a approval operation, or a comment issuing operation, different operations correspond to different first interaction instructions, and the different operations are used for giving different virtual gifts, the virtual gift giving operations of different virtual gifts correspond to different first interaction instructions, and the different first interaction instructions correspond to different face areas of the human face.

11. A computer device comprising a processor and a memory, the memory having stored therein at least one instruction that is loaded and executed by the processor to implement the live based interaction method of any one of claims 1 to 7 or to implement the live based interaction method of claim 8.

12. A computer-readable storage medium having stored therein at least one instruction, which is loaded and executed by a processor to implement the live-based interaction method of any one of claims 1 to 7 or the live-based interaction method of claim 8.