CN113438493B

CN113438493B - Popup animation generation method, server, system and storage medium

Info

Publication number: CN113438493B
Application number: CN202110642219.1A
Authority: CN
Inventors: 查彬
Original assignee: Guangzhou Cubesili Information Technology Co Ltd
Current assignee: Guangzhou Cubesili Information Technology Co Ltd
Priority date: 2018-05-30
Filing date: 2018-05-30
Publication date: 2023-06-13
Anticipated expiration: 2038-05-30
Also published as: CN108769775B; CN108769775A; CN113438493A

Abstract

The invention relates to a popup animation generation method, a server, a system and a storage medium, wherein the method comprises the following steps: receiving a video stream sent by a host client in a live broadcast process; generating a popup window pushed to the audience client under the condition that user state information of the audience client which is in the same network live channel with the anchor client meets a preset trigger condition; after receiving the response of the audience client to the popup window, carrying out face recognition on the image frames in the video stream, and extracting a first person image according to the recognized face; and according to the actions of the characters in the first character image and/or the second character image acquired in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display. The method and the device are beneficial to improving the interaction effect between the anchor client and the audience client.

Description

Popup animation generation method, server, system and storage medium

The application is a divisional application with the application date of 2018, 5 month and 30 days, the application number of CN 2018105430729 and the invention and creation name of a data processing method and device in network live broadcast and a network live broadcast system.

Technical Field

The present invention relates to the field of network live broadcasting technologies, and in particular, to a popup animation generating method, a server, a system, and a storage medium.

Background

Webcast is an emerging form of webcast system that generally includes an anchor audience client, a server, and an audience client, as shown in fig. 1. The audience client can record the audio and the video, then upload the recorded audio and video to the server, and then distribute the recorded audio and video to the audience client for display by the server. Meanwhile, the audience client can forward the text, expression and other information sent by the user to the anchor audience client for display through the server. Currently, in a live webcast scene, popup windows are sometimes pushed to a live audience client or an audience client, so that the purposes of improving interestingness and promoting interaction between the live audience client and the audience client are achieved. However, the conventional pushing manner has poor interaction improving effect.

Disclosure of Invention

Based on the above, the invention provides a popup animation generation method, a server, a system and a storage medium, which are used for solving the technical problem of poor interaction improving effect in the prior art.

According to a first aspect of an embodiment of the present invention, a popup animation generation method is provided, which is applied to a network live broadcast server in a network live broadcast system; the method comprises the following steps:

receiving a video stream sent by a host client in a live broadcast process;

generating a popup window pushed to the audience client under the condition that user state information of the audience client which is in the same network live channel with the anchor client meets a preset trigger condition;

after receiving the response of the audience client to the popup window, carrying out face recognition on the image frames in the video stream, and extracting a first person image according to the recognized face;

according to actions of characters in the first character image and/or a second character image obtained in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display; the second character image is a pre-stored character image used for representing a user of the audience client, and comprises a cartoon character image, a character image uploaded in advance by the audience client or a character image generated according to a pre-acquired face image; the relative positions of the first person image and the second person image are set according to the action of the second person image.

Optionally, the user status information at least includes: the user logs in the duration information of the network live channel; the triggering conditions include at least: the time length of the user logging in the network live channel reaches a preset logging time threshold;

or, the user status information at least includes: operating state information for operating each control on a live interface of the network live channel by a user; the triggering conditions include at least: receiving operation state information of a user on a preset control on a live broadcast interface;

or, the user status information at least includes: the number of times the user sends a message on the network live channel; the triggering conditions include at least: the number of times the user sends the message on the network live channel is the first time.

Optionally, the first person image is a face image of a host user corresponding to the host client; and/or, the second person image further comprises at least one of: the account image of the audience client, the character image of the audience user corresponding to the audience client, or the character image selected according to the gender information corresponding to the audience client.

Optionally, the popup animation further comprises popup background images;

the generating the popup animation sent to the audience client for display further comprises:

selecting a target popup background image from a plurality of popup background images according to a preset selection mode, or generating the target popup background image according to the current time, place and/or weather;

and rendering the second person image and the first person image by using the target popup background image.

Optionally, the preset selection manner includes at least one of the following: randomly selecting, selecting according to the current time, place and/or weather;

wherein different popup background images have different colors, patterns and/or animation effects.

Optionally, the popup includes an example picture indicating the popup action and a payment control;

the response of the spectator client to the popup includes a control operation instruction for the payment control; the control operation execution instruction is used for executing the payment operation.

Optionally, the method further comprises: transmitting the popup animation to the anchor client and/or other audience clients in the same network live channel; the playing time of playing the popup animation at the live client is preset time; and/or the number of times of playing the popup animation at the live client does not exceed a preset number of times.

Optionally, the method further comprises: updating the pushing mode of the popup window according to the operation information of each audience client to the popup window; and/or updating the content of the popup animation according to the operation information of each audience client to the popup animation.

Optionally, the operation information includes at least one of: responding to the popup, shielding the popup, forwarding the popup link and clicking the popup animation;

the updated pushing mode of the popup window comprises pushing the popup window meeting at least any one of the following conditions to a user: the most number of times the popup animation is praised and/or the most number of times the popup link is forwarded;

the updated content of the popup animation comprises the common characteristics of the popup animation with the most front response of the audience client; the positive response includes any one of: the popup animation is purchased, the popup animation is endorsed and/or the popup link is forwarded.

According to a second aspect of an embodiment of the present invention, there is provided a webcast server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any one of the methods of the first aspect when executing the program.

According to a third aspect of an embodiment of the present invention, there is provided a live webcast system, including a live webcast server, a hosting client, and a viewer client;

the anchor client is used for sending a video stream to the network live broadcast server in the live broadcast process;

the network live broadcast server is used for receiving video streams sent by the anchor client in the live broadcast process; generating a popup window pushed to the audience client under the condition that user state information of the audience client which is in the same network live channel with the anchor client meets a preset trigger condition;

the audience client is used for displaying the popup window and sending a response to the popup window to the network live broadcast server;

the network live broadcast server is further used for carrying out face recognition on the image frames in the video stream after receiving the response of the audience client to the popup window, and extracting a first person image according to the recognized face; according to actions of characters in the first character image and/or a second character image obtained in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display; the second character image is a pre-stored character image used for representing a user of the audience client, and comprises a cartoon character image, a character image uploaded in advance by the audience client or a character image generated according to a pre-acquired face image; the relative positions of the first person image and the second person image are set according to the action of the second person image;

The spectator client is also configured to display the popup animation.

According to a fourth aspect of embodiments of the present invention, there is provided a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the embodiments described above.

By applying the scheme of the embodiment of the invention, the popup window can be pushed to the audience client only when the user state of the audience client meets the triggering condition, so that the interference to the user when the popup window is pushed is reduced; and the first person image in the popup animation is extracted from the video stream in the live broadcast process, and the positions of two different persons in the popup animation are set according to the actions of the persons in the first person image and/or the second person image, so that different popups can be generated in different live broadcast rooms, the popup animation generation mode is more flexible, and the interaction effect between the anchor client and the audience client is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention as claimed.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and together with the description, serve to explain the principles of the invention.

Fig. 1 is a system architecture diagram of a webcast system of one embodiment.

Fig. 2 is a flow chart of a method for pushing a popup window in a live webcast according to an embodiment of the present invention.

Fig. 3 is a schematic diagram of a live room of an embodiment.

FIG. 4 is a schematic diagram of a pop-up window animation according to one embodiment of the invention.

Fig. 5 is a schematic view of a pop-up window according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a popup animation played in accordance with one embodiment of the present invention.

Fig. 7 is a flowchart of a method for pushing a popup window in a live webcast according to another embodiment of the present invention.

FIG. 8A is a flowchart of a method for generating a popup animation according to an embodiment of the present invention.

FIG. 8B is a flowchart of another popup animation generation method, according to one embodiment of the present invention.

Fig. 9 is a flowchart of a data processing method in network live broadcast according to an embodiment of the present invention.

Fig. 10 is a block diagram of a data processing apparatus in network broadcast according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numbers in different drawings refer to the same or similar elements, unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the invention. Rather, they are merely examples of apparatus and methods consistent with aspects of the invention as detailed in the accompanying claims.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any or all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used herein to describe various information, these information should not be limited by these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the invention. The word "if" as used herein may be interpreted as "at … …" or "at … …" or "responsive to a determination", depending on the context.

As shown in fig. 1, a system architecture diagram of a webcast system 100 of one embodiment may include a hosting client 101, a webcast server 102, and a spectator client 103, the hosting client 101 interacting with the spectator client 103 through the webcast server 102. The anchor client 101 refers to an end that sends a live webcast video, and is generally a client used by an anchor user in live webcast. The viewer client 103 refers to a client employed by a viewer (i.e., a live viewer user) receiving and viewing a live video, typically in a live video. The anchor client 101 and the viewer client 103 may interact after joining the same live room (i.e., a webcast channel). To facilitate interaction between the anchor client 101 and the audience client 103, the webcast server 102 generates and pushes a pop to the audience client 103. The spectator client 103 may respond to the popup to purchase the popup animation and may also send the purchased popup animation to the anchor client 101 for play display. The anchor client 101 may also send feedback information to the viewer client 101 after displaying the popup animation. Wherein the response may be to perform a purchase operation.

It should be noted that the audience client 103 and the anchor client 101 are functionally divided only, and in practical applications, the functions of the audience client 103 and the anchor client 101 may be performed by the same client at different times. Thus, the same client can be used as the viewer client 103 when watching a live webcast, and as the anchor client 101 when publishing live video.

It should be noted that the system architecture in fig. 1 is only an exemplary architecture, and is not meant to limit the present invention. The scheme of the invention can also be applied to other forms of network living broadcast systems, and the description of the scheme is omitted.

In one embodiment, the present invention provides a method of pushing a pop in a webcast that may be performed by a webcast server 102 in a webcast system 100 shown in fig. 1. As shown in fig. 2, the method may include the steps of:

in step 201, user state information sent by a viewer client is received.

In step 202, if the user status information meets a preset trigger condition, pushing a popup window to the audience client;

in step 203, after receiving the response of the spectator client to the popup, sending the popup animation to the spectator client for display;

The popup animation comprises a first person image and a second person image, and the relative positions of the first person image and the second person image are set according to the action of the second person image.

Wherein, the popup may include an example picture of the popup animation for showing the user with the approximate content of the popup animation. For example, the popup animation can be a wall-clattering animation, and then an example picture of the wall-clattering animation can be included in the popup, and an example wall-clattering action is made by two persons in the picture, so that a user can more intuitively know the content of the popup animation. In one embodiment, the same popup may be employed for the popups sent to the various spectator clients; in another embodiment, different popups may also be sent to different clients, which respond to the different popups to obtain different popup animations.

In this embodiment, the viewer client may be the viewer client 103 in the webcast system 100 shown in fig. 1. The user status information may be used by the webcast server 102 to determine whether the user has a potential intent to receive a pop-up. Any one of the following information may be employed as the user status information:

(1) The user logs in the login state information of the network live channel. The login status information may be duration information of a user logging in a live network channel. Each live webcast channel has a channel ID for distinguishing the channel from other channels, and it can be determined which channel the user joins according to the channel ID joined by the user's client. Taking the example of joining a live broadcast room, it can be determined which live broadcast room the user joins according to the room number of the live broadcast room the client joins. After a user enters a certain network live channel, timing can be started; after the user exits the live webchannel, the timer may be stopped to obtain the duration information of the user logging into the live webchannel.

(2) Operating state information of a user on a live interface of a network live channel. Wherein, the live interface of the network live channel specifically refers to an interface displayed on the client of the audience. As shown in fig. 3, in a live room scenario, the interface 300 may include a video display area 301 thereon for displaying video sent by the anchor client 101; a message display area 302, configured to display historical messages sent by each user (including the user itself and other users) in the live broadcast room; and a control area 303, wherein the control area 303 comprises at least one control 303a, and each control 303a realizes different functions, and the controls comprise, but are not limited to, a message control for inputting a message, a gift control for sending a gift, a setting control for setting system functions, an exit control for exiting a living room, and the like. The positional layout of the respective areas is not limited to that shown in fig. 3. The user can operate each control, and different operation states can be obtained by executing different operations.

(3) Communication status information of the user on the network live channel. The communication status information may be the number of times the user transmits a message on the network live channel, for example, when the user transmits a message on the network channel for the first time, the corresponding communication status information is the status of the first transmission of the message. The times of sending the messages on different network live channels by the user are counted respectively. Taking the live room scenario as an example, assuming that a user first sends a message in the live room with number 11111 and then sends a message in the live room with number 22222, the message sent by the user in the live room with number 22222 is still sent for the first time, and the message sent in the live room with number 22222 is not considered to be sent for the second time because the user has already sent a message in the live room with number 11111.

Corresponding to the above three kinds of user state information, the following three kinds of trigger conditions may be set, respectively:

(1) A login status information trigger condition is set based on the login status information. Specifically, the login state information triggering condition may be that a duration of a user logging in a live network channel reaches a preset login time threshold. The login time threshold may be set according to actual requirements, for example, may be set to 5 minutes, or 10 minutes, or set to other durations; the method can be dynamically set according to the historical time length of logging in the network live channel by the user.

In one embodiment, the duration of the user logging into the live webchannel may be cleared upon monitoring that the user is logged out of the live webchannel. Thus, the user can recalculate the time length when exiting and logging in the same network live channel again, and the logging time length of entering the same network live channel twice is not accumulated.

(2) The operation state information trigger condition is preset based on the operation state information. Specifically, the operation state information triggering condition may be that operation state information of a preset control on the live interface is received by a user. In one embodiment, the preset control may be a list control; correspondingly, the operation state information is: expansion operation information and retraction operation information of the list control. The operation state information triggering conditions are further as follows: the expanding operation information is received first, then the collapsing operation information is received, and the objects in the list are not received. In this embodiment, the list control may be used to expand a list, which may include one or more objects, and when the list control is operated, the list may be expanded or collapsed. When the list is expanded, the objects in the list may be manipulated.

Taking a live room scenario as an example, the list control may be a gift control, where the gift control may be used to expand a gift list, where each object in the gift list is a gift. The gift list may be expanded or collapsed when the gift control is operated. When the gift list is expanded, the gift in the gift list may be operated, which may include a gift operation, a preview gift operation, a purchase gift operation, and the like. In performing the gift sending operation, if a gift already exists in the audience client 103, the gift may be transmitted to the anchor client 101 through the audience client 103 via the webcast server 102; if no gift exists in the spectator client 103, the spectator client 103 may be prompted for purchase information and the gift may be sent to the anchor client 101 via the webcast server 102 after the spectator client 103 has purchased successfully. In the above scenario, the operation state information triggering condition is further: the spectator client 103 first expands the gift list, but does not send the gift to the anchor client 101, and then packs up the gift list.

(3) The communication state information trigger condition is preset based on the communication state information. The communication status information may include the number of times the user sends a message on the network live channel, and the corresponding triggering condition of the communication status information may be that the number of times the user sends a message on the network live channel is the first time.

According to the embodiment, the popup is pushed based on the trigger condition set by the login state information, and the popup is pushed only when one of the login state information trigger condition, the operation state information trigger condition or the communication state information trigger condition is met, so that the interference to a user when the popup is pushed is reduced, and the user experience is improved.

In one embodiment, to further reduce interference to the user, a switch function may also be provided that may be used to enable or disable the push window function. Therefore, when the popup window is pushed to the client side of the audience, whether the function of pushing the popup window is in an enabled state or not can be judged; if yes, pushing the popup window to the audience client.

In one embodiment, the popup window can be closed by the user independently, so that the user experience is further improved. Specifically, if a user's instruction for closing the pop-up window is received, the function of pushing the pop-up window may be set to a disabled state within a preset first time period; wherein the disabled state is used to prevent pushing of the popup to the viewer client. The first predetermined period of time may be 1 day, 5 days, 7 days, etc. For example, when the webcast server 102 receives a close instruction sent by the viewer client 103, the function of pushing the popup may be set to a disabled state within 7 days, and the popup is not pushed to the viewer client 103 for 7 days.

In one embodiment, if the number of times of receiving a closing instruction sent by a user on a live network channel reaches a preset popup closing number threshold, when the user is on the live network channel, setting a popup pushing function to a disabled state. Taking a live broadcast room scene as an example, if the popup windows are closed for 2 times continuously in the same live broadcast room, a user does not push the popup windows any more in the live broadcast room.

In one embodiment, in a preset second period of time, if the number of times of pushing the popup window to the viewer client reaches a preset threshold of number of times of pushing, the function of pushing the popup window is set to a disabled state. Through setting up the upper limit of the number of times of pushing the popup window to every user, can further reduce the interference to the user, avoid reducing user experience because of repeated pushing. In this embodiment, the second period of time may be 1 day; the preset push number threshold may be 2 times. That is, if a certain user has been pushed 2 shots during the same day, then no shots are pushed to him.

In one embodiment, as shown in FIG. 4, a pop-up animation 400 includes a first person image 401, a second person image 402, and optionally, a pop-up background image 403. Wherein the first character image 401 may be a anchor image and the second character image 402 may be a pre-stored character image, which may be used to represent the character of the audience user. The pop-up background image 403 may be a heart-shaped icon. The pop-up window animation 400 may be a wall-clattering animation. The animation may be a VR (Virtual Reality) animation or an AR (Augmented Reality ) animation to enhance the user's immersion.

In an actual application scenario, the animation in the popup animation 400 may be a "wall-clattering anchor" animation, the first person image 401 may be an anchor image, the second person image 402 may be an image of a live audience user, and the popup background image may be a heart icon. In the animation, the audience presents wall-clattering action to the anchor.

In one embodiment, as shown in FIG. 5, an example picture 501 of a popup animation 400 is displayed in a popup 500 and includes a payment control 502; wherein the payment control 502 is used for a user to perform a payment operation. Further, a close control 503 for closing the pop-up window may also be included on the pop-up window 500. The close control 503 may employ a native button to prevent the popup content from being unloaded due to network problems, affecting the user experience.

The payment control 502 in the pop-up window 500 may be a payment button, and text information, such as "buy and send" may also be displayed on the payment control 502 to prompt the user for operation. Some text content may be included in the pop-up window 500, and the text content may include the subject of the pop-up window (e.g., 1-element wall-clattering anchor), and may also include other content, such as: "only 1 yuan can be used for wall-clattering anchor, the success rate of the sister is 100%", and "99.9% of vermicelli is used for wall-clattering anchor.

When receiving a control operation instruction of a user to the payment control 502, the network live broadcast server 102 can respond to the control operation instruction to detect account balance parameters of the user; the control operation execution instruction is used for executing payment operation; and if the account balance parameter is greater than a preset balance parameter threshold, sending the popup animation to the anchor client for display. Wherein the anchor client may be the anchor client 101 shown in the network architecture of fig. 1.

When a control operation instruction (e.g., clicking on a control) of the payment control 502 by the user is received, the user's account balance may be detected. If the account balance is enough to purchase the popup animation, the popup animation can be directly sent; if the account balance is insufficient, the purchase information is sent to the user, and the user is prompted to pay or recharge. After the pop-up animation is successfully transmitted, an animation effect can be presented on the anchor client 101 and/or other audience clients of the same living room, and the dynamics of the gift presentation can be displayed on the message display area 302 of the living room.

When a user stays in a living broadcast room for a long time or tries to give a gift or sends a message in the living broadcast room, the user can be generally considered to have a willingness or tendency to purchase a special effect of 1-membered wall-dong anchor, so that the user with the purchasing tendency can be screened out more accurately by setting the three triggering conditions, and the popup window is pushed only for the user, so that the interference to the user without the purchasing tendency is reduced; on the other hand, the information interaction amount between the network live broadcast server and the client is reduced.

In one embodiment, after successful delivery of the animation, a prompt may also be sent to the anchor client 101 via the webcast server, prompting the anchor client 101 to send a feedback message to the audience client 103. For example, in the example of "1-element wall-clattering anchor" described above, after the audience client 103 sends the "1-element wall-clattering anchor" animation to the anchor client 101, prompt information (e.g., text prompt information and/or audio prompt information, etc.) may be sent to the anchor client 101 through the webcast server 102, and the anchor client 101 sends an answer message to the audience client 103. In this way, interactions between the anchor client 101 and the audience client 103 may be improved. Meanwhile, the 1-element wall-clattering anchor animation can be played at the anchor client 101, the playing time length can be set to be a preset value, for example, 3 seconds, the situation that the animation blocks the display interface of the anchor client 101 for a long time is avoided, and the interference to the anchor client 101 is reduced. Further, the number of times of playing the animation at the anchor client 101 may set an upper limit of times, and when the number of times of playing reaches the upper limit of times, the animation is not played any more, so as to further reduce the interference to the anchor client 101. An animation 600 played by the anchor 101 of one embodiment is shown in fig. 6, and includes a prompt 601 and an animation 602. The prompt message 601 may be text message, and the text message may be, for example, "XXX gives the first time to the loved you, and gives a response in comparison with the heart.

In one embodiment, an upper limit on the number of times the user purchases the animation may also be set, for example, the upper limit may be set to 1. A user tag may be established, and for users who have not purchased an animation, a first tag may be set; for users who have purchased animations, a second tab may be set. The first tag and the second tag may be represented in the form of binary bits, for example, the first tag may be represented by binary bit "1"; the second tag may be represented by a binary bit "0". This representation has a small data size. Other label forms may be employed, as the invention is not limited in this regard.

In one embodiment, after the audience client performs the preset operation on the popup, the webcast server 102 may also push the popup to the associated client of the audience client; the associated client is a client which is logged in by an account associated with a viewer account, and the viewer account is an account which is logged in to the viewer client; the preset operation comprises any one of the following steps: responding to a pop, approving a pop animation, or forwarding a pop link for viewing the pop and/or the pop animation.

The associated client may be a client of a friend, where the friend refers to an account in a friend list of the audience account. Responding to a popup may refer to the viewer client purchasing a popup animation. The praise popup animation may be that the viewer client 103 operates a praise plug-in the popup animation. The forwarding popup link may be that the viewer client 103 forwards the link including the popup and/or popup animation to a friend client or to a third party client (e.g., QQ, weChat, etc.).

In one embodiment, the webcast server 102 may also send the most heavily praised popup animation to the spectator client. The most praised popup animation may be considered the most popular popup animation, and pushing such popup animation to the spectator client 103 may increase the probability that the spectator client will respond to the popup.

In one embodiment, the webcast server 102 may also push popup windows to other viewer clients in the living room after the viewer clients perform preset operations; the preset operation comprises any one of the following steps: responding to a pop, approving a pop animation, or forwarding a pop link. Because viewers entering the same living room generally have relatively similar preferences, pushing the popup to other viewer clients 103 after one of the viewer clients 103 performs a preset operation can increase the probability that the viewer client 103 will respond to the popup.

In one embodiment, the webcast server may also push the popup window in the landing page advertisement. The login page advertisement is advertisement information displayed on the interface of the client during the process of logging in the client 103 (generally, during a period of time after inputting an account number and clicking a "login" control). Typically, such advertising information will be displayed for a period of 3 to 5 seconds. By pushing the popup window in the time period, excessive interference to the client is avoided, and the probability of the response of the client 103 to the popup window can be improved.

In one embodiment, after the audience client sends out the popup, the webcast server 102 may also push the popup animation to the anchor client and/or other audience clients in the same living room for display.

Another embodiment of the present invention also provides a method for pushing a popup window in a live webcast, which may be performed by the viewer client 103 in the live webcast system 100 shown in fig. 1. As shown in fig. 7, the method may include the following steps 701 and 702:

step 701: and monitoring user state information sent by the audience client and sending the user state information to the live webcast server.

Step 702: receiving a popup window pushed by the network live broadcast server and sending a response to the popup window; pushing the popup window when the user state information meets a preset triggering condition;

step 703: displaying the popup animation sent by the network live broadcast server after receiving the response on a display interface; the popup animation comprises a first person image and a second person image, and the relative positions of the first person image and the second person image are set according to the action of the second person image.

In one embodiment, the method further comprises: and if the display time of the popup animation reaches a preset display time threshold, clearing the popup animation on the display interface.

The popup animation sent in the method of pushing popups in a live webcast described above may be generated by the live webcast server 102 based on a particular manner. After sending the popup animation to the viewer client 103 based on the method for pushing the popup in the live webcast, operation information (for example, responding to the popup, shielding the popup, forwarding the popup link, approving the popup animation, etc.) of the viewer client on the popup and/or the popup animation can also be obtained, so as to be used for improving and updating contents of the popup animation, pushing modes of the popup, etc. in a subsequent maintenance process.

In one embodiment, an embodiment of the present invention provides a popup animation generation method that may be performed by a webcast server 102 in a webcast system 100 shown in fig. 1.

As shown in fig. 8A, the method includes:

step 81: and receiving the video stream sent by the anchor client in the live broadcast process.

Step 82: and generating a popup window pushed to the audience client under the condition that the user state information of the audience client which is in the same network live channel with the anchor client meets the preset trigger condition.

Step 83: and after receiving the response of the audience client to the popup window, carrying out face recognition on the image frames in the video stream, and extracting a first person image according to the recognized face.

Step 84: and according to the actions of the characters in the first character image and/or the second character image acquired in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display.

As shown in fig. 8B, the method may include the steps of:

step 801: receiving a video stream sent by a host client in a live broadcast process;

Step 802: extracting a first person image from the video stream;

step 803: the second person image acquired in advance is subjected to image superposition with the first person image, and a popup animation is generated; the position relation of the second person image and the first person image in the popup animation is set according to actions of the first person image and/or the person in the second person image, and the popup animation is used for being sent to a client of a spectator to be displayed.

According to the embodiment, the first person image is extracted from the video stream sent by the anchor client in the live broadcast process, the popup animation is generated according to the first person image and the second person image which is acquired in advance, and the position relationship between the second person image and the first person image in the popup animation is set according to the actions of the persons in the second person image, so that different popups can be generated in different live broadcast rooms, the popup animation generation mode is more flexible, and the interaction effect between the anchor client and the audience client is improved.

In step 801, the webcast server 102 may receive a video stream sent by a hosting client during a live broadcast, where the video stream may include a first person image, and of course, may include some background images. The first person image includes at least a face image of a person. In one embodiment, the first persona image is a persona image of a host user.

In step 802, the webcast server 102 may extract a first person image from a video stream. During extraction, the webcast server 102 may perform face recognition on an image frame of the video stream, and when a face is recognized, acquire a first person image in the image frame, and stop performing an operation of extracting the first person image. Since most of image frames in a live scene generally include person images, the first person image can be extracted more quickly by this way of extracting the first person image. Further, after the first person image is extracted, whether the face image in the first person image is a face front image or a face side image can be identified, and if the face image is the face side image, the first person image is extracted again until the face front image is extracted.

In step 803, the pre-acquired second person image may be a pre-stored person image, for example, a cartoon person image or the like; or may be a character image uploaded in advance by the viewer client 103; or a person image generated from a face image acquired in advance. The term "pre-image" as used herein means that the second person image is acquired before the image is superimposed, that is, before the image is superimposed. The positional relationship between the second person image and the first person image in the popup animation is set according to the actions of the person in the second person image, for example, the actions may be "wall clattering" actions, "heart comparing" actions, and the like. Taking the wall-clattering action as an example, the relative positions of the second person image and the first person image which are overlapped in the popup animation meet the following conditions: the lips of the person in the second person image are located in the area where the face of the person in the first person image is located, and at least one palm of the person in the second person image is located on one side of the head of the person in the first person image.

The generated popup animation may be available for transmission to the viewer client 103 for display. Further, the spectator client 103 may also send the popup animation to the anchor client 101 for display. In practical applications, the client 103 of the audience can purchase the popup animation first, and then send the popup animation to the client 101 of the anchor for display through the live webcast server 102 after the purchase is successful. After the audience client 103 has purchased successfully, the webcast server 102 may also push the popup to other audience clients (e.g., the audience clients of the same living room, or associated clients of the audience clients purchasing the popup) for display.

In one embodiment, the second person image may include an account header of the viewer's client. After the crowd client 103 joins the living room, the webcast server 102 may obtain an account header of the audience client 103. Further, the account head portrait can be further subjected to face recognition, and if the face image is recognized, a second person image is generated according to the face image.

In another embodiment, the second person image may also be a person image uploaded by the viewer client 103. If there is a history of the audience client 103 uploading the character image in the webcast server 102, the character image may be saved in the database for a period of time. Before image superimposition, the webcast server 102 may first search the database for a person image, and if so, generate a second person image based on the found person image.

In one embodiment, the webcast server 102 may also obtain user information characterizing the gender of the live audience user; and selecting a second character image according to the user information. The user information may be obtained from information submitted by the viewer client 103 at registration. The second person images of different genres may be stored in the database in advance. If the sex of the live audience user is male, selecting a second character image with the sex being male; if the sex of the live audience user is female, a second character image with the sex female is selected.

In one embodiment, the popup animation further includes a popup background image. The pop-up background image is an image for setting off or rendering the second person image and the first person image. Some specific patterns and/or animation effects may be included in the pop-up background image. In practical applications, a plurality of popup background images may be pre-generated and stored, and different popup background images may have different colors, patterns, animation effects, and/or the like.

When generating the popup animation, one from a plurality of popup background images is selected. The selection mode can be random selection or can be selected according to the current time, place and/or weather and other factors. For example, if the current time is a time period corresponding to daytime, (e.g., 6:00-18:00 of the day), selecting a popup background image of light tone; if the current time is mid-autumn, a popup window background image including a "moon" icon may be selected. For another example, if the current location is Guangzhou, a bullet window background image may be selected that includes a pattern of Guangzhou landmarks (e.g., guangzhou towers). For another example, if the current weather is a sunny day, a popup window background image including "sun" may be selected. In addition, the popup background images may also be generated directly from the current time, place, and/or weather. In a manner similar to the embodiments described above.

In one embodiment, the webcast server 102 may also obtain character information sent by the viewer client on a public chat channel in the live room; and inserting the character information into the popup animation. Further, the webcast server 102 may perform semantic analysis on the character information to screen out character information having a preset meaning, and then insert the screened character information into the popup animation. Before insertion, the character information can be further rendered to improve the visual perception of the user. The resolution of the characters may also be set according to the resolution of the popup animation in order to match the characters to the popup animation.

In one embodiment, the webcast server 102 may also insert audio in the popup to play the audio simultaneously as the popup is pushed to the viewer client 103. The audio inserted into the pop-up window may be pre-stored audio, or may be audio uploaded by the viewer client 103, for example, may be audio uploaded by the viewer client 103 and including live viewer user voice. Similarly, speech in audio may also be semantically recognized prior to insertion of the audio.

In one embodiment, the pop-up animation may be a wall-dong animation of the first person image of the second person image. That is, the character corresponding to the second character image forces the character corresponding to the first character image to the wall, and makes a clattering sound by one hand or leaning against the wall, so that the character can escape without any position. The popup animation may be another animation, for example, an animation in which a character corresponding to the second character image and a character corresponding to the first character image perform a "heart rate" motion.

In one embodiment, the second person image and the first person image may both be 3D images. If the second person image and the first person image acquired by the webcast server 102 are 2D images, the 2D images may also be converted into 3D images, and then a pop-up animation may be generated through the 3D images.

According to the embodiment of the invention, different types of popup animation can be generated according to actual needs, and the generation modes are flexible and various.

In one embodiment, the present invention further provides a popup animation, which may be generated based on the popup animation generation method in any of the above embodiments. Further, the generated popup animation may be transmitted to the viewer client 103 by the webcast server 102 when a preset condition is satisfied. For example, a popup may be pushed to the viewer client 103 first, and if a response (e.g., a purchase operation) to the popup by the viewer client is received, a popup animation is sent.

In one embodiment, as shown in fig. 9, the present invention further provides a data processing method in live webcasting, which may be performed by the live webcasting server 102 in fig. 1, and the method includes:

step 901: respectively obtaining feedback times of receiving feedback of each audience client by the popup animation in a historical time period; the popup animation is sent to the audience client based on the response of the audience client to the popup in the network live broadcast process;

Step 902: and updating the popup animation sent to the audience client according to the feedback times.

According to the scheme of the embodiment, the popup animation pushed to the audience client is updated according to the feedback of the audience client to the popup animation in the historical time period, so that popup animation which is generally interested or not interested by the audience client can be timely found, and the most popular popup animation is timely pushed to the audience client.

In step 901, the popup animation may be sent to the viewer client during the live webcast based on the method for pushing a popup in the live webcast in any of the embodiments, and the popup animation may be generated based on the popup animation generation method in any of the embodiments. The historical time period can be preset (for example, one week, one month) or can be dynamically set according to actual requirements.

In step 902, an update period may be preset, and the push popup animation in the update period is updated through the feedback times in the historical time period. For example, the number of feedback in the past month may be obtained to update the push pop-up animation in the future week. In one embodiment, the update period may also be set to the same duration as the historical period.

In one embodiment, the feedback number includes: the number of times the popup animation is praised and/or the number of times the popup link is forwarded. The popup pushed to the viewer client may be updated to a popup that satisfies at least any of the following conditions: the number of times the popup animation is endorsed is the greatest and/or the number of times the popup link is forwarded is the greatest. The more times, the more audience clients 103 that are interested in the popup animation are illustrated, the greater the probability that the subsequently pushed popup receives the audience client response, and therefore, the push proportion of such popup animation can be improved. Accordingly, the fewer the number of times, the more spectator clients 103 that are not interested in the popup animation, the smaller the probability that the subsequently pushed popup receives the spectator client response, and thus, the sending proportion of such popup animations may be reduced, or such popup animations may not be sent, or such popup animations may be redesigned.

Further, after the feedback times are obtained, the clustering analysis may be further performed on each feature of the popup animation according to the feedback times, so as to obtain a feature common to the popup animation having the most positive response received, and send the popup animation having the feature to the audience client 103. Wherein, the positive response refers to the response that the popup animation is purchased, the popup animation is prayed and/or the popup link is forwarded, etc. In this way, the popup animation and design popup animation features may be updated according to the preferences of the viewer client to increase the acceptance of the popup animation.

In one embodiment, the step of updating the popup animation sent to the spectator client 103 based on the number of feedback comprises: the popup animation transmitted to the viewer client 103 is updated to a popup animation satisfying at least any one of the following conditions: the number of endorsements is the greatest and/or the number of popup links is the greatest. The present embodiment may update the popup animation push and design popup animation features according to the preferences of the viewer client 103 to increase the acceptance of the popup animation.

In one embodiment, the feedback times are updated after receiving feedback information of the popup animation from the viewer client 103. By the method, accurate feedback times can be obtained so as to update the push popup animation later.

In one embodiment, feedback information of the popup animation of the audience client is pushed to the associated client; the associated client is a client which is logged in by an account associated with a viewer account, and the viewer account is an account which is logged in to the viewer client. In one embodiment, the associated client may be a friend client. Friends refer to the account numbers in the friend list of the viewer account numbers.

In one embodiment, the viewer client feedback information for the popup animation is pushed in the public talk channel and/or in the live room closed captioning. In this way, the probability that the popup is noticed by the viewer client 103 within the living room can be increased, thereby increasing the probability that the popup is responded to.

In one embodiment, the purchase information includes a link to purchase a pop-up animation. Thus, the client side of the audience can conveniently purchase the popup animation.

In one embodiment, if the spectator client meets a popup disable condition, the function of pushing popup to the spectator client is set to be in a disabled state; wherein, the bullet window forbidden condition is: and the number of times that the audience client closes the popup window reaches a preset number of times threshold. Thus, on one hand, the data interaction between the audience client 103 and the network live broadcast server 102 is reduced, and resources are saved; on the other hand, the interference to the audience client 103 is reduced, and the user experience is improved.

In one embodiment, if the audience client side meets the popup prohibition condition, the permission of the audience client side to purchase the popup animation is set to be in a purchase prohibition state; wherein, the popup forbidden condition is: and the audience client purchases the popup animation in a preset time period, or the number of times that the audience client purchases the popup animation reaches a preset purchase number threshold. In this embodiment, the time interval for the viewer client 103 to purchase the popup animation, or the upper limit for the viewer client 103 to purchase the popup animation may be set. Therefore, a large number of network water armies can be avoided from being subjected to bill-brushing, and meanwhile, the accuracy of statistical data can be guaranteed.

In one embodiment, if the client side meets the benefit receiving condition, the authority of the client side to receive the benefit is set to be in an on state; wherein, the welfare collection conditions are: the spectator client purchases the popup animation and the spectator client does not receive the benefit. The benefit in this embodiment may be an on-line benefit or an off-line benefit, where the on-line benefit may be a gift package, and the off-line benefit may be a peripheral product of the network live platform. In this way, the cost performance of the popup animation may be increased, thereby increasing the probability of the spectator client 103 responding to the popup animation.

In one embodiment, the popup includes information characterizing the number of times the popup animation was purchased. In this embodiment, when the popup window is pushed, information indicating the number of times the popup window is purchased may be displayed in the popup window at the same time. In this way, the viewer client can learn the information that the popup animation was purchased, thereby increasing the probability that the popup animation was purchased.

In one embodiment, as shown in fig. 10, the present invention further provides a data processing apparatus in network live broadcast, which may include:

the obtaining module 1001 is configured to obtain feedback times of the popup animation receiving feedback from each viewer client in the historical time period respectively; the popup animation is sent to the audience client based on the response of the audience client to the popup in the network live broadcast process;

And an updating module 1002, configured to update the popup animation sent to the audience client according to the feedback times.

Specific details of the implementation process of the functions and roles of each module in the device are shown in the implementation process of corresponding steps in the data processing method in the network live broadcast, and are not repeated here.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The apparatus embodiments described above are merely illustrative, wherein the modules illustrated as separate components may or may not be physically separate, and the components shown as modules may or may not be physical, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purposes of the present invention. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

In one embodiment, the present invention further provides a live webcast system, including: the audience client and the webcast server. The viewer client in the network living broadcast system is configured to perform the method performed by the viewer client 103 in the method of any of the above embodiments; the webcast server is configured to perform the method performed by the webcast server 102 in the method of any of the embodiments described above.

For example, the spectator client is configured to send feedback on the popup animation to the webcast server; and the network live broadcast server is used for transmitting the popup animation to the audience client based on the response of the audience client to the popup during the network live broadcast process, respectively acquiring the feedback times of the popup animation in the historical time period, which are fed back by each audience client, and updating the popup animation transmitted to the audience client according to the feedback times. Further, the network live broadcast system further includes: a host client; the network live broadcast server is also used for sending the popup animation to the anchor client for display.

In an embodiment, the present invention further provides a live webcast system, including: the audience client and the webcast server.

the network live broadcast server is further used for carrying out face recognition on the image frames in the video stream after receiving the response of the audience client to the popup window, and extracting a first person image according to the recognized face; according to actions of characters in the first character image and/or a second character image obtained in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display;

the spectator client is also configured to display the popup animation.

In one embodiment, the present invention also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of the method of any of the embodiments described above.

In one embodiment, the present invention further provides a network live broadcast server, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the steps of the method described in any of the above embodiments when the processor executes the program.

The present application may take the form of a computer program product embodied on one or more storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) having program code embodied therein. Computer-usable storage media include both permanent and non-permanent, removable and non-removable media, and information storage may be implemented by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to: phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, may be used to store information that may be accessed by the computing device.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This disclosure is intended to cover any adaptations, uses, or adaptations of the disclosure following the general principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

The foregoing description of the preferred embodiments of the present disclosure is not intended to limit the disclosure, but rather to cover all modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present disclosure.

Claims

1. A popup animation generation method is characterized by being applied to a network live broadcast server in a network live broadcast system; the method comprises the following steps:

receiving a video stream sent by a host client in a live broadcast process;

according to actions of characters in the first character image and/or a second character image obtained in advance, overlapping the second character image and the first character image to corresponding positions, and generating a popup animation which is sent to the audience client for display;

Updating the content of the popup animation according to the operation information of each audience client to the popup animation; wherein the operation information includes at least one of: responding to the popup, shielding the popup, forwarding the popup link and clicking the popup animation; the content of the updated popup animation includes features common to the popup animation that received the most positive response of the spectator client, the positive response including: the popup animation is purchased, the popup animation is praised and/or the popup link is forwarded; after updating the content of the popup animation, the popup pushed to the audience client comprises an updated popup animation example picture, wherein the example picture is used for displaying the content of the updated popup animation to a user.

2. The method of claim 1, wherein the step of determining the position of the substrate comprises,

the user status information includes at least: the user logs in the duration information of the network live channel; the triggering conditions include at least: the time length of the user logging in the network live channel reaches a preset logging time threshold;

3. The method of claim 1, wherein the first person image is a face image of a anchor user corresponding to the anchor client;

and/or the second person image comprises at least one of: and presetting a character image, an account image of the audience client, a character image of an audience user corresponding to the audience client or a character image selected according to gender information corresponding to the audience client.

4. The method of claim 1, wherein the pop-up animation further comprises a pop-up background image;

5. The method of claim 4, wherein the preset selection pattern comprises at least one of: randomly selecting, selecting according to the current time, place and/or weather;

6. The method of claim 1, wherein the popup window further comprises a payment control;

7. The method as recited in claim 1, further comprising:

transmitting the popup animation to the anchor client and/or other audience clients in the same network live channel; the playing time of playing the popup animation at the live client is preset time; and/or the number of times of playing the popup animation at the live client does not exceed a preset number of times.

8. The method according to claim 1 or 7, further comprising:

and updating the pushing mode of the popup according to the operation information of each audience client to the popup.

9. The method of claim 8, wherein the updated manner of pushing the popup window includes pushing the popup window satisfying at least any one of the following conditions to the user: the number of times the popup animation is endorsed is the greatest and/or the number of times the popup link is forwarded is the greatest.

10. A live webcast server comprising a memory, a processor and a computer program stored on the memory and executable on the processor, said processor implementing the steps of the method according to any of claims 1 to 9 when said program is executed.

11. The network live broadcast system is characterized by comprising a network live broadcast server, a host broadcast client and a spectator client;

the network live broadcast server is used for receiving video streams sent by the anchor client in the live broadcast process; and generating a popup window pushed to the audience client under the condition that the user state information of the audience client in the same network live channel as the anchor client meets a preset trigger condition;

the audience client is also used for displaying the popup animation;

the network live broadcast server is also used for updating the content of the popup animation according to the operation information of each audience client to the popup animation; wherein the operation information includes at least one of: responding to the popup, shielding the popup, forwarding the popup link and clicking the popup animation; the content of the updated popup animation includes features common to the popup animation that received the most positive response of the spectator client, the positive response including: the popup animation is purchased, the popup animation is praised and/or the popup link is forwarded; after updating the content of the popup animation, the popup pushed to the audience client comprises an updated popup animation example picture, wherein the example picture is used for displaying the content of the updated popup animation to a user.

12. A computer readable storage medium, characterized in that a computer program is stored thereon, which program, when being executed by a processor, implements the steps of the method according to any of claims 1 to 9.