CN113448475A

CN113448475A - Interaction control method and device for virtual live broadcast room, storage medium and electronic equipment

Info

Publication number: CN113448475A
Application number: CN202110735130.XA
Authority: CN
Inventors: 庄宇轩
Original assignee: Guangzhou Boguan Information Technology Co Ltd
Current assignee: Guangzhou Boguan Information Technology Co Ltd
Priority date: 2021-06-30
Filing date: 2021-06-30
Publication date: 2021-09-28
Anticipated expiration: 2041-06-30
Also published as: CN113448475B

Abstract

The disclosure provides an interaction control method of a virtual live broadcast room, an interaction control device of the virtual live broadcast room, a computer readable storage medium and electronic equipment, and belongs to the technical field of live broadcast. The method is applied to an anchor client, and comprises the following steps: receiving an interaction instruction and a current frame image sent by a live audience client through a live platform server; sending the interaction instruction and the current frame image to an anchor server so that the anchor server determines a feedback event of the interaction instruction according to the current frame image; receiving the feedback event sent by the anchor server and generating a target video stream of the feedback event; and sending the target video stream to the live audience client through the live platform server so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream. The method and the device can improve the interactive experience of the live broadcast user.

Description

Interaction control method and device for virtual live broadcast room, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of live broadcast technologies, and in particular, to an interaction control method for a virtual live broadcast room, an interaction control apparatus for a virtual live broadcast room, a computer-readable storage medium, and an electronic device.

Background

With the development of internet technology and the like, live broadcast is widely popular among the public as a real-time interactive social contact mode.

At present, a user can watch live video of a main broadcast in a live broadcast room and interact with the main broadcast by means of sending comments and the like. In this way, the interaction mode between the user and the live broadcast room is single, and the association degree between the interaction content and the live broadcast content is low, so that a strong connection between the user and the anchor cannot be established. Therefore, how to effectively improve the diversity of the interaction modes between the user and the live broadcast room and strengthen the connection between the user and the live broadcast room becomes a problem to be solved in the prior art.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure provides an interaction control method for a virtual live broadcast room, an interaction control device for a virtual live broadcast room, a computer-readable storage medium, and an electronic device, thereby improving, at least to a certain extent, the problem in the prior art that the interaction mode between a user and a live broadcast room is relatively single.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.

According to a first aspect of the present disclosure, there is provided an interaction control method for a virtual live broadcast room, applied to an anchor client, the method including: receiving an interaction instruction and a current frame image sent by a live broadcast audience client through a live broadcast platform server, wherein the interaction instruction is used for indicating a virtual live broadcast room to perform effect display, and the current frame image is a live broadcast picture displayed in a live broadcast interface at the current moment intercepted by the live broadcast audience client; sending the interaction instruction and the current frame image to an anchor server so that the anchor server determines a feedback event of the interaction instruction according to the current frame image; receiving the feedback event sent by the anchor server, and generating a target video stream of the feedback event; and sending the target video stream to the live audience client through the live platform server so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream.

In an exemplary embodiment of the present disclosure, the feedback event includes a feedback event of a target virtual object corresponding to the interaction instruction, and when receiving the feedback event sent by the anchor server and generating a target video stream of the feedback event, the method includes: and generating a target video stream of the feedback event of the target virtual object according to the feedback event of the target virtual object.

In an exemplary embodiment of the present disclosure, the target virtual object includes a target commodity, and the method includes, when generating the target video stream of the feedback event: and acquiring commodity data of the target commodity, and generating a target video stream of a feedback event of the target commodity according to the commodity data, wherein the feedback event of the target commodity comprises any one of a commodity purchasing event, a commodity information browsing event and a commodity special effect display event.

In an exemplary embodiment of the present disclosure, the method further comprises: receiving transaction result data aiming at the target commodity sent by the live broadcast platform server; and generating a new video stream according to the transaction result data, and sending the new video stream to the live broadcast audience client through the live broadcast platform server so that the live broadcast audience client plays the new video stream.

In an exemplary embodiment of the present disclosure, the generating the target video stream of the feedback event includes: obtaining an effect template of the feedback event; extracting a keying picture of a anchor from a real scene picture containing the anchor and shot by a camera; and synthesizing the image matting picture and the effect template to generate the target video stream.

In an exemplary embodiment of the present disclosure, when the target video stream is sent to the live viewer client through the live platform server, so that the live viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream, the method further includes: determining a feedback type of the feedback event, wherein the feedback type comprises a single execution event and a continuous execution event; and controlling the virtual live broadcast room to display the effect according to the feedback type of the feedback event.

In an exemplary embodiment of the present disclosure, the controlling the virtual live broadcast room to perform effect display according to the feedback type of the feedback event includes: when the feedback type of the feedback event is the single execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, and stopping the effect display of the virtual live broadcast room when the target video stream is played; when the feedback type of the feedback event is the continuous execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, and determining the feedback progress of the feedback event; when the feedback progress of the feedback event meets a preset ending condition, ending the effect display of the virtual live broadcast room; and when the feedback progress of the feedback event is determined not to meet the preset ending condition, maintaining the effect display of the virtual live broadcast room.

According to a second aspect of the present disclosure, an interaction control method for a virtual live broadcast room is provided, which is applied to a main broadcast server, and the method includes: receiving an interaction instruction and a current frame image sent by a anchor client; determining a feedback event of the interaction instruction according to the current frame image; and sending the feedback event to the anchor client so that the anchor client generates a target video stream of the feedback event.

In an exemplary embodiment of the present disclosure, the determining a feedback event of the interaction instruction according to the current frame image includes: and analyzing the interaction instruction, determining a target virtual object corresponding to the interaction instruction according to the object information obtained after analysis, and determining a feedback event of the target virtual object.

In an exemplary embodiment of the present disclosure, when the virtual object includes a commodity to be sold, and a target virtual object corresponding to the interaction instruction is determined according to object information obtained after parsing, and a feedback event of the target virtual object is determined, the method includes: determining the commodity to be sold corresponding to the interaction instruction according to the object information to obtain a target commodity corresponding to the interaction instruction; and determining a feedback event of the target commodity.

In an exemplary embodiment of the disclosure, when determining a feedback event of the interaction instruction, the method further includes: and acquiring a trigger instruction of a feedback event of the target virtual object, and determining the feedback event corresponding to the trigger instruction as the feedback event of the target virtual object when the trigger instruction is matched with the interaction instruction.

According to a third aspect of the present disclosure, an interaction control method for a virtual live broadcast room is applied to a live broadcast viewer client, and a graphical user interface is provided through a display screen of the live broadcast viewer client, where the graphical user interface includes a live broadcast interface, and the live broadcast interface includes a picture display area for displaying a live broadcast picture received by the live broadcast viewer client, and the method includes: responding to the interaction operation of the audience aiming at the live broadcast picture, generating an interaction instruction, and intercepting the live broadcast picture displayed in the live broadcast interface at the current moment to obtain a current frame image; sending the interaction instruction and the current frame image to an anchor client through a live broadcast platform server so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction; receiving the target video stream sent by the anchor client through the live broadcast platform server; and playing the target video stream in the picture display area, and controlling the virtual live broadcast room to display the effect.

In an exemplary embodiment of the present disclosure, in generating the interaction instruction, the method includes: and responding to the interactive operation of the audience aiming at the live broadcast picture, determining a virtual object corresponding to the operation position of the interactive operation in the current frame image as a target virtual object, and generating an interactive instruction about the target virtual object.

In an exemplary embodiment of the present disclosure, the target video stream includes a video stream generated by the anchor client according to a feedback event of a target virtual object corresponding to the interaction instruction, and when controlling the virtual live broadcast room to perform effect display, the method includes: and controlling the target virtual object to perform effect display.

In an exemplary embodiment of the present disclosure, the controlling the target virtual object to perform an effect display while playing the target video stream includes: when the target video stream is played, controlling the virtual live broadcast room to execute picture display operation corresponding to the feedback event of the target commodity; the screen display operation includes any one of: controlling the target commodity to carry out special effect display; controlling the live broadcast interface to jump to a detail page of the target commodity; and adding an order for purchasing the target commodity in the order to be purchased of the live audience client or providing a purchase interface of the target commodity.

In an exemplary embodiment of the present disclosure, the method further comprises: in response to the order placing operation of the target commodity by the audience, stopping the display of the target commodity by the virtual live broadcast room, and generating transaction data of the target commodity; sending the transaction data to the live broadcast platform server to enable the live broadcast platform server to carry out settlement processing, and returning transaction result data obtained after settlement processing to the anchor client; and receiving the transaction result data aiming at the target commodity sent by the anchor client and a new video stream generated by the anchor client according to the transaction result data, and playing the new video stream.

According to a fourth aspect of the present disclosure, an interaction control apparatus of a virtual live broadcast room is provided, which is applied to an anchor client, and the apparatus includes: the receiving module is used for receiving an interaction instruction and a current frame image which are sent by a live audience client through a live platform server; the first sending module is used for sending the interaction instruction and the current frame image to an anchor server so that the anchor server can determine a feedback event of the interaction instruction according to the current frame image; the generating module is used for receiving the feedback event sent by the anchor server and generating a target video stream of the feedback event; and the second sending module is used for sending the target video stream to the live audience client so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream.

In an exemplary embodiment of the present disclosure, the feedback event includes a feedback event of a target virtual object corresponding to the interaction instruction, and when the feedback event sent by the anchor server is received and a target video stream of the feedback event is generated, the generating module is configured to generate the target video stream of the feedback event related to the target virtual object according to the feedback event of the target virtual object.

In an exemplary embodiment of the disclosure, the target virtual object includes a target commodity, and when a target video stream of the feedback event is generated, the generating module is further configured to obtain commodity data of the target commodity, and generate the target video stream of the feedback event of the target commodity according to the commodity data, where the feedback event of the target commodity includes any one of a commodity purchase event, a commodity information browsing event, and a commodity special effect display event.

In an exemplary embodiment of the present disclosure, the generating module is further configured to receive transaction result data, which is sent by the live broadcast platform server and is for the target commodity; and generating a new video stream according to the transaction result data, and sending the new video stream to the live broadcast audience client through the live broadcast platform server so that the live broadcast audience client plays the new video stream.

In an exemplary embodiment of the present disclosure, the generating module is configured to obtain an effect template of the feedback event, extract a matting picture of a anchor from a real scene picture including the anchor shot by a camera, and synthesize the matting picture and the effect template to generate the target video stream.

In an exemplary embodiment of the present disclosure, when the target video stream is sent to the live viewer client through the live platform server, so that the live viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream, the second sending module is configured to determine a feedback type of the feedback event, where the feedback type includes a single execution event and a continuous execution event, and control the virtual live broadcast room to perform effect display according to the feedback type of the feedback event.

In an exemplary embodiment of the disclosure, the second sending module is further configured to, when the feedback type of the feedback event is the single execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, and when the playing of the target video stream is finished, stopping the presentation of the effect to the virtual live broadcast room, and when the feedback type of the feedback event is the continuous execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, determining the feedback progress of the feedback event, ending the effect display of the virtual live broadcast room when the feedback progress of the feedback event is determined to meet the preset ending condition, and when the feedback progress of the feedback event is determined not to meet the preset ending condition, maintaining the effect display of the virtual live broadcast room.

According to a fifth aspect of the present disclosure, an interaction control apparatus of a virtual live broadcast room is provided, which is applied to an anchor server, and the apparatus includes: the receiving module is used for receiving an interaction instruction and a current frame image sent by an anchor client, wherein the interaction instruction is used for indicating a virtual live broadcast room to perform effect display, and the current frame image is a live broadcast picture displayed in a live broadcast interface at the current moment intercepted by a live broadcast audience client; the determining module is used for determining a feedback event of the interaction instruction according to the current frame image; and the sending module is used for sending the feedback event to the anchor client so that the anchor client generates a target video stream of the feedback event.

In an exemplary embodiment of the present disclosure, the determining module is configured to analyze the interaction instruction, determine a target virtual object corresponding to the interaction instruction according to object information obtained after the analysis, and determine a feedback event of the target virtual object.

In an exemplary embodiment of the present disclosure, the virtual object includes a commodity to be sold, and when the target virtual object corresponding to the interaction instruction is determined according to the object information obtained after the analysis and the feedback event of the target virtual object is determined, the determining module is further configured to determine the commodity to be sold corresponding to the interaction instruction according to the object information to obtain the target commodity corresponding to the interaction instruction; and determining a feedback event of the target commodity.

In an exemplary embodiment of the disclosure, when determining a feedback event of the interaction instruction, the sending module is configured to obtain a trigger instruction of the feedback event of the target virtual object, and when the trigger instruction matches the interaction instruction, determine the feedback event corresponding to the trigger instruction as the feedback event of the target virtual object.

According to a sixth aspect of the present disclosure, an interaction control device of a virtual live broadcast room is provided, which is applied to a live broadcast viewer client, and provides a graphical user interface through a display screen of the live broadcast viewer client, where the graphical user interface includes a live broadcast interface, and the live broadcast interface includes a picture display area for displaying a live broadcast picture received by the live broadcast viewer client, and the device includes: the generation module is used for responding to the selection operation of the audience aiming at the live broadcast picture, generating an interaction instruction, and intercepting the live broadcast picture displayed in the live broadcast interface at the current moment to obtain a current frame image; the sending module is used for sending the interaction instruction and the current frame image to an anchor client through a live broadcast platform server so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction; the receiving module is used for receiving the target video stream sent by the anchor client through the live platform server; and the playing module is used for playing the target video stream in the picture display area and controlling the virtual live broadcast room to display the effect.

In an exemplary embodiment of the disclosure, when the interactive instruction is generated, the generating module is configured to respond to an interactive operation of a viewer on the live broadcast screen, determine a virtual object corresponding to an operation position of the interactive operation in the current frame image as a target virtual object, and generate the interactive instruction about the target virtual object.

In an exemplary embodiment of the present disclosure, the target video stream includes a video stream generated by the anchor client according to a feedback event of a target virtual object corresponding to the interaction instruction, and when controlling the virtual live broadcast room to perform effect display, the play module is configured to control the target virtual object to perform effect display.

In an exemplary embodiment of the present disclosure, the target virtual object includes a target commodity, and the playing module is further configured to control the virtual live broadcast room to execute a screen display operation corresponding to a feedback event of the target commodity when the target video stream is played, where the screen display operation includes any one of: controlling the target commodity to carry out special effect display; controlling the live broadcast interface to jump to a detail page of the target commodity; and adding an order for purchasing the target commodity in the order to be purchased of the live audience client or providing a purchase interface of the target commodity.

In an exemplary embodiment of the disclosure, the playing module is further configured to, in response to an order placing operation of a viewer on the target commodity, stop display of the target commodity in the virtual live broadcast room, generate transaction data of the target commodity, send the transaction data to the live broadcast platform server, enable the live broadcast platform server to perform settlement processing, return transaction result data obtained after the settlement processing to the anchor client, receive the transaction result data for the target commodity sent by the anchor client and a new video stream generated by the anchor client according to the transaction result data, and play the new video stream.

According to a seventh aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing any one of the above-mentioned interaction control methods for a virtual live broadcast.

According to an eighth aspect of the present disclosure, there is provided an electronic apparatus comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the above interaction control methods of the virtual live broadcast room via execution of the executable instructions.

The present disclosure has the following beneficial effects:

according to the interaction control method of the virtual live broadcast room, the interaction control device of the virtual live broadcast room, the computer-readable storage medium and the electronic device in the exemplary embodiment, the anchor client can receive the interaction instruction and the current frame image of the live broadcast audience client, the anchor server can determine the feedback event of the interaction instruction according to the current frame image, the anchor client can generate the target video stream of the feedback event, and the target video stream is sent to the live broadcast audience client, so that the live broadcast audience client can control the virtual live broadcast room to perform effect display when playing the target video stream. On one hand, the generated target video stream is sent to the live broadcast audience client side to be played, and the virtual live broadcast room is controlled to carry out effect display, so that the interaction modes of audiences and the live broadcast room are enriched, various interaction requirements of the audiences are met, the contact between the audiences and the live broadcast room is enhanced, and the payment conversion of users is facilitated; on the other hand, by determining the current frame image and determining the feedback event of the interactive instruction according to the current frame image, the interactive instruction can be ensured to correspond to the live broadcast image when the user performs interactive operation, the problem that the interactive instruction and the live broadcast image are not synchronous due to network delay and the like is avoided, and the accuracy of determining the feedback event of the interactive instruction is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is apparent that the drawings in the following description are only some embodiments of the present disclosure, and that other drawings can be obtained from those drawings without inventive effort for a person skilled in the art.

Fig. 1 is a system architecture diagram illustrating an operating environment of an interaction control method of a virtual live broadcast room in the present exemplary embodiment;

fig. 2 shows a flowchart of an interaction control method of a virtual live room performed by an anchor client in the present exemplary embodiment;

fig. 3A and 3B are schematic diagrams of a virtual live room in the present exemplary embodiment;

FIG. 4 is a diagram illustrating a live interface in the exemplary embodiment;

FIG. 5 shows a schematic view of an article for sale in the present exemplary embodiment;

FIG. 6 illustrates a sub-flow diagram of an interaction control method in the exemplary embodiment;

fig. 7A and 7B are diagrams illustrating a live interactive effect in the present exemplary embodiment;

fig. 8A and 8B are diagrams illustrating another live interactive effect in the present exemplary embodiment;

FIG. 9 illustrates a sub-flowchart of another interaction control method in the exemplary embodiment;

fig. 10 is a flowchart illustrating an interaction control method of a virtual live broadcast room performed by a anchor server in the present exemplary embodiment;

FIG. 11 is a schematic diagram of an interaction region in the exemplary embodiment;

FIG. 12 is a sub-flowchart illustrating another interactive control method in the present exemplary embodiment;

FIG. 13 is a schematic view showing one target commodity in the present exemplary embodiment;

fig. 14A and 14B are diagrams showing a purchase of a target commodity in the present exemplary embodiment;

FIG. 15 is an interaction flow diagram illustrating a method of interaction control in the exemplary embodiment;

fig. 16 is a block diagram showing a configuration of an interaction control apparatus of a virtual live broadcast room in the present exemplary embodiment;

fig. 17 is a block diagram showing the configuration of an interaction control apparatus of another virtual live room in the present exemplary embodiment;

fig. 18 is a block diagram showing a configuration of an interaction control apparatus of a virtual live room according to still another exemplary embodiment;

FIG. 19 illustrates a computer-readable storage medium for implementing the above-described method in the present exemplary embodiment;

fig. 20 shows an electronic device for implementing the above method in the present exemplary embodiment.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In view of the foregoing problems, exemplary embodiments of the present disclosure first provide an interaction control method of a virtual live broadcast. Fig. 1 shows a system architecture diagram of an environment in which the method operates. As shown in fig. 1, the system architecture 100 may include: a live viewer client 110, a live platform service 120, an anchor client 130, and an anchor service 140. The live viewer client 110 represents a terminal device, such as a smart phone, a tablet computer, a personal computer, an intelligent vehicle-mounted device, etc., through which a viewer watches live video in a live platform; the live platform server 120 represents a background service system of a live platform, and can be used for establishing interaction between the live viewer client 110 and the anchor client 130; the anchor client 130 refers to a terminal device for live broadcast by an anchor, including but not limited to a smart phone, a tablet computer, and a personal computer; the anchor server 140 may be configured to provide a relevant live service for the anchor client 130, for example, may execute business logic of the anchor client 130 during live broadcasting to control the anchor client 130 to perform corresponding screen display or implement a specific function, such as filtering comment content. The live audience client 110 and the live platform server 120 may perform information interaction via a network, for example, an audience may perform an interactive operation on the live audience client 110, and the live platform server 120 responds to the interactive operation of the live audience client 110 and sends corresponding special effect information to the live audience client 110 for displaying. The anchor client 130 and the anchor server 140 may also interact through a network or a physical connection, for example, the anchor client 130 may connect to the anchor server 140 through the network and send a request for setting the content of the anchor client display screen during live broadcast, such as a request for setting a live broadcast screen and an interactive area split screen display, so that the anchor server 140 may set the display screen of the anchor client in response to the request.

It should be noted that, in the present exemplary embodiment, the number of each device in fig. 1 is not limited, for example, any number of live viewer clients 110 and anchor clients 130 may be provided according to implementation needs, and both the live platform server 120 and the anchor server 140 may be a cluster formed by multiple servers.

The interaction control method of the virtual live broadcast room in the exemplary embodiment is specifically described below with the anchor client 130, the anchor server 140, and the live viewer client 110 as execution subjects.

Fig. 2 shows a flow of an interaction control method of a virtual live broadcast room performed by the anchor client 130, which may include the following steps S210 to S240:

and S210, receiving an interaction instruction and a current frame image sent by a live audience client through a live platform server.

The interaction instruction can be used for indicating the virtual live broadcast room to display corresponding effects. For example, the interaction instruction may be used to instruct the virtual live broadcast room to switch the display view angle, or trigger a certain special effect of a certain virtual element in the virtual live broadcast room or the virtual live broadcast room, and the like; the current frame image may be a live view displayed in a live interface at the current time captured by a live viewer client.

In this exemplary embodiment, the virtual live broadcast room refers to that a scene in the live broadcast room is a virtual scene, the virtual scene may be a virtual picture rendered by an Engine of a UE (universal Engine), and a user may interact with a virtual element in the virtual scene. During live broadcasting, the anchor can select a desired scene from a pre-generated virtual scene template, and can adjust the broadcast data of the virtual scene, such as adding scene elements, cutting size, setting color, and the like. For example, before the live broadcast is started, the anchor is broadcast in front of the green screen, and the live broadcast platform server can switch the background of the anchor into the virtual scene selected by the anchor and cut the green screen range according to the setting requirements of the anchor. As shown in fig. 3A, the virtual live broadcast room presents a picture obtained by combining the anchor and the virtual scene, and at the same time, the anchor can select the slider at the bottom of the picture to drag the selected slider to adjust the cropping range of the green screen, as shown in fig. 3B, the cropping range of the green screen changes along with the movement of the slider at the bottom of the picture. After the anchor selects the virtual scene and the broadcast data of the virtual scene, the live broadcast can be started, and the composite picture of the anchor and the set virtual scene is displayed in the live broadcast picture. By the method, the anchor can adjust the synthesis range of the virtual scene and the picture parameters of the live broadcast picture according to the requirements, so that the flexibility of synthesizing the live broadcast picture can be enhanced, the picture types of the live broadcast picture can be greatly enriched, and the watching experience of a user is improved.

When the audience watches the live video on the live audience client, the interactive operation of the virtual live broadcast room can be triggered through touch operation or gesture operation and the like, and the interactive instruction generated by the live audience client in response to the interactive operation is sent to the live broadcast platform server and sent to the anchor client through the live broadcast platform server.

And S220, sending the interaction instruction and the current frame image to a main broadcasting service end so that the main broadcasting service end determines a feedback event of the interaction instruction according to the current frame image.

The feedback event may be a response event of the received interaction instruction, for example, a display event of a live view after the live viewer client receives the interaction instruction, or a special effect display event of a virtual object in a virtual live broadcast room.

When the anchor client receives the interaction instruction and the current frame image sent by the live audience client, the interaction instruction and the current frame image can be sent to the anchor server, so that the anchor server can determine the feedback event of the interaction instruction according to the current frame image.

And step S230, receiving a feedback event sent by the anchor server and generating a target video stream of the feedback event.

The target video stream may be a video stream that presents a certain special effect presentation or picture presentation generated according to the feedback event indicated by the interaction instruction. After the anchor server determines a feedback event of the interaction instruction, the feedback event may be sent to the anchor client, and the anchor client may generate a target video stream of the feedback event, for example, the anchor client may execute the UE instance according to the feedback event to render an anchor picture, so as to generate the target video stream.

In an alternative embodiment, the feedback event may include a feedback event of a target virtual object corresponding to the interaction instruction, and thus, in step S230, the following method may be performed:

and generating a target video stream of the feedback event about the target virtual object according to the feedback event of the target virtual object.

The target virtual object may be a virtual character or an item in the virtual live broadcast room indicated by the interaction instruction, for example, a "desk lamp" in the virtual live broadcast room shown in fig. 4.

When the feedback event is the feedback event of the target virtual object corresponding to the interaction instruction, it indicates that the audience needs to trigger special effect display or picture display on the target virtual object, so that a target video stream related to the feedback event of the target virtual object can be generated according to the feedback event of the target virtual object.

Further, in the present exemplary embodiment, the virtual live broadcast room may be used for performing live broadcast shopping, for example, a virtual scene in the virtual live broadcast room may display articles to be sold, and the viewer may select the articles to be sold while watching the live broadcast, so as to trigger special effect display or purchase of the articles to be sold, for example, as shown in fig. 5, the articles to be sold may be displayed in a live broadcast screen display area 510 in the live broadcast interface, and each article to be sold may be displayed on a "virtual shelf," the user may select the articles to be sold on the "virtual shelf," interact with the selected articles, and the public screen display area 520 may display comments, a pop-up screen, and the like of the viewer. Based on this, in an alternative embodiment, the target virtual object may also include the target commodity, that is, the interactive operation triggered by the viewer through the live viewer client may be an interactive operation for the target commodity, and thus, when generating the target video stream of the feedback event, the following method may be performed:

and acquiring commodity data of the target commodity, and generating a target video stream of a feedback event of the target commodity according to the commodity data.

The feedback event of the target commodity can include any one of a commodity purchasing event, a commodity information browsing event and a commodity special effect displaying event.

The anchor client can acquire commodity data of the target commodity, such as detailed data of the target commodity, a thumbnail of the target commodity, information of price, inventory and the like, and then generate a target video stream of a feedback event of the target commodity according to the commodity data. For example, when the feedback event is a commodity special effect display event, the anchor client may generate a target video stream representing a special effect of the target commodity according to the thumbnail in the commodity data, for example, the special effect of the target commodity may be that the thumbnail of the target commodity is alternately displayed in an enlarging and reducing manner.

By the method, the target video stream of the target virtual object can be generated, compared with the existing interactive modes such as comment sending, barrage sending and the like, the interactive mode of live broadcast pictures can be greatly enriched, and the interest of a user in watching live broadcast is improved.

And S240, sending the target video stream to the live audience client through the live platform server so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream.

Specifically, the anchor client may generate a target video stream by operating the UE instance, and send the target video stream to the live viewer client through the live platform server, for example, the live platform server may send the target video stream to the client according to a live viewer client ID (Identification) and a user ID that generate the interaction instruction, so as to play the target video stream, and complete the effect display of the virtual live room.

Compared with the live broadcast mode of the anchor in the real scene, the target video stream can present richer live broadcast pictures due to the fact that the image matting picture and the virtual scene of the anchor are fused, and the audience can interact with the virtual elements in the virtual scene, so that the relevance between the audience and the virtual live broadcast can be increased, and the payment conversion of the user is facilitated.

In an optional implementation manner, when the target virtual object is a target commodity, the target video stream may also be sent to the live broadcast audience client through the live broadcast platform server, so that when the live broadcast audience client plays the target video stream, the virtual live broadcast room is controlled to execute a picture display operation corresponding to a feedback event of the target commodity.

The screen display operation may be an operation of controlling a live broadcast screen to display, which is generated by executing a feedback event, and may include any one of controlling a target commodity to perform special effect display, controlling a live broadcast interface to jump to a detail page of the target commodity, adding an order for purchasing the target commodity in an order to be purchased at a live broadcast audience client, and providing a purchase interface for the target commodity.

For example, when the target video stream is sent to the live viewer client through the live platform server, the live viewer client may play the target video stream, and control the target commodity in the virtual live broadcast room to perform special effect display or picture display, for example, the live broadcast picture may be switched to a window display, and the live broadcast picture may jump to a detail page of the target commodity.

When playing the target video stream related to the target commodity, the viewer may select to perform corresponding operations on the target commodity through the live viewer client, for example, the viewer may add an order for purchasing the target commodity in an order to be purchased of the live viewer client and perform settlement to purchase the target commodity. Thus, in an alternative embodiment, the following method may also be performed:

receiving transaction result data aiming at a target commodity sent by a live broadcast platform server;

and generating a new video stream according to the transaction result data, and sending the new video stream to the live audience client through the live platform server so that the live audience client plays the new video stream.

The transaction result data may be used to indicate whether the viewer's transaction for the target good was successful, and may also include other information such as the transaction amount, and the current remaining inventory amount.

After receiving the transaction result data sent by the live broadcast platform server, the anchor client can determine whether the transaction of the target commodity is successful, the current stock of the target commodity and the like according to the transaction result data, so that a new live broadcast video stream can be generated, and the commodities to be sold which can be purchased by the audience at the current moment can be displayed in the new live broadcast video stream, and the target commodity which is already transacted is not displayed.

By the method, the commodity information can be displayed in the virtual live broadcast room, so that audiences can interact with target commodities in the virtual live broadcast room, and the method has important significance for promoting commodity transaction and improving live broadcast interaction and interestingness.

Further, as described above, the live view presented by the virtual live room may be a view synthesized by the anchor and the virtual scene. Thus, in an alternative embodiment, as shown in fig. 6, the target video stream of the feedback event may be generated through the following steps S610 to S630:

step S610, an effect template of the feedback event is obtained.

The effect template of the feedback event may be a fixed template for enabling the virtual live broadcast room to present a display effect corresponding to the feedback event.

And step S620, extracting a keying picture of the anchor from the real scene picture containing the anchor shot by the camera.

The anchor is broadcast in front of the green screen, the real scene picture shot by the camera is a picture formed by the anchor and the green screen environment, and the picture outside the anchor is removed by identifying the picture of the area where the anchor is located in the real scene picture, so that the image scratching picture of the anchor can be obtained. In addition, when shooting through the camera, the anchor can also adjust parameters such as the angle of the camera, so that the shot real scene picture shows the expected effect.

Step S630, the matting picture and the effect template are synthesized to generate a target video stream.

And overlapping and mixing the keying picture of the anchor and the effect template of the feedback event to obtain a target video stream after the anchor and the virtual scene are synthesized.

In an alternative embodiment, the feedback event may include two types, i.e., a single execution event or a continuous execution event, according to the duration of execution of the feedback event, etc. The single execution event may include execution for 1 time, or a feedback event whose execution time is less than a preset threshold, for example, the feedback event may be a feedback event that controls turning on or off of lights; the continuous execution event may include a feedback event with an execution time greater than or equal to a preset threshold, for example, a feedback event for making a special effect presentation in a virtual live broadcast. The preset threshold may be set according to actual requirements, for example, may be set to 5 seconds, 10 seconds, and the like, and this exemplary embodiment is not particularly limited in this regard. Based on this, in step S240, the following method may also be performed:

determining a feedback type of a feedback event;

and controlling the virtual live broadcast room to display the effect according to the feedback type of the feedback event.

Specifically, when the client plays the target video stream, the server may control the virtual live broadcast room to perform effect display within a period of time according to the feedback type of the feedback event. For example, referring to fig. 7A, when it is determined that the feedback event is a "desk lamp" turned on at the upper right corner according to the interaction instruction, and it can be determined that the feedback event is a single execution event according to the preset event rule, the "desk lamp" can be controlled to be turned off according to the event rule, the picture effect shown in fig. 7B is presented, and the picture effect is maintained until the next time the user turns on the "desk lamp". For another example, referring to fig. 8A, when the user double-clicks a "book" or shakes the mobile phone, the anchor server may determine that the feedback event is a "run out of a cat" triggering the egg-shaped effect according to the received interaction instruction, and may determine that the feedback event of the interaction instruction is a continuous execution event according to a preset event rule, then as shown in fig. 8B, the virtual live broadcast room may be controlled to present a video frame of the "run out of a cat" in the virtual scene, and when the event execution is finished, the virtual scene in the live broadcast frame is displayed as a scene before the execution of the feedback event.

Through the mode, the effect display of the virtual live broadcast room can be controlled according to the feedback type of the feedback event, the visual effect of the live broadcast room is enriched, and the watching experience of a user is improved.

Further, in an optional implementation manner, when controlling the virtual live broadcast room to perform effect display according to the feedback type of the feedback event, as shown in fig. 9, the following steps S910 to S940 may also be performed:

step S910, when the feedback type of the feedback event is a single execution event, monitoring the playing progress of the target video stream, and when the playing of the target video stream is completed, stopping the effect display on the virtual live broadcast room.

When the feedback event is a single execution event, the anchor client can monitor the playing progress of the target video stream through the live broadcast platform server when the target video stream is played, and when the target video stream is played, the feedback event is executed completely, and the effect display of the virtual live broadcast room can be stopped.

Step S920, when the feedback type of the feedback event is a continuous execution event, monitoring the playing progress of the target video stream, and determining the feedback progress of the feedback event.

When the feedback event is a continuous execution event and the target video stream is played, the playing progress of the target video stream of the live audience client can be monitored in real time through the live platform server, and the feedback progress of the feedback event corresponding to the target video stream is determined according to the playing progress, for example, the execution duration of the feedback event can be detected.

In step S930, when it is determined that the feedback progress of the feedback event satisfies the preset end condition, the effect display on the virtual live broadcast room is ended.

And step S940, when the feedback progress of the feedback event is determined not to meet the preset ending condition, the effect display of the virtual live broadcast room is kept.

The preset ending condition may be set according to the specific event content of each feedback event, and the like. For example, the duration of the special effect display reaches a preset time, or the execution of an action event is ended.

When the feedback progress of the feedback event meets the preset ending condition, the interactive feedback is ended, so that the effect display of the virtual live broadcast room can be ended; on the contrary, when the feedback progress of the feedback event does not meet the preset ending condition, the interactive feedback is still in progress, and the effect display of the virtual live broadcast room can be continuously kept. By the method, the feedback progress of the feedback event can be monitored in real time, so that when the feedback is finished, the live broadcast picture can be switched back to the conventional picture in time, and in the conventional picture, the virtual scene of the virtual live broadcast room is displayed as the scene before the display of the triggerless effect.

Through the interaction control method of the virtual live broadcast room executed by the anchor client 130, the interaction instruction and the current frame image sent by the live broadcast audience client through the live broadcast platform server can be received, the interaction instruction and the current frame image are sent to the anchor server to determine the feedback event of the interaction instruction, then the target video stream is generated according to the feedback event, and the target video stream is sent to the live broadcast audience client through the live broadcast platform server, so that the live broadcast audience client controls the virtual live broadcast room to display the effect when playing the target video stream, the interaction mode of the virtual live broadcast room is greatly enriched, and the user renewal and conversion are facilitated.

Fig. 10 shows a flow of an interaction control method of a virtual live broadcast room performed by the anchor server 140, and may include the following steps S1010 to S1030:

and S1010, receiving an interaction instruction and a current frame image sent by the anchor client.

And S1020, determining a feedback event of the interaction instruction according to the current frame image.

After receiving the interaction instruction and the current frame image, the anchor server can determine a feedback event corresponding to the interaction operation when the live audience client generates the interaction instruction according to the current frame image. For example, the anchor server may analyze the interaction instruction to obtain operation content, such as an operation position and operation time, of an interaction operation in the current frame image, which is generated when the live viewer client generates the interaction instruction, and further determine a feedback event of the interaction instruction, such as which effect display needs to be performed in the virtual live room, and a display duration. For example, referring to fig. 11, when watching a live video, a viewer may slide a screen left and right in a sliding region to generate an interaction instruction for controlling a virtual live broadcast room to perform view angle transformation, and the anchor server may determine, according to the interaction instruction, at what angle and size the virtual scene in the virtual live broadcast room needs to be presented according to the viewing requirement of the viewer.

Further, the display screen of the virtual live broadcast room may be composed of a plurality of virtual elements, and the interactive operation of the viewer on the virtual live broadcast room may be performed with respect to the target virtual object. Therefore, step S1020 may also be implemented by:

and analyzing the interaction instruction, determining a target virtual object corresponding to the interaction instruction according to the object information obtained after analysis, and determining a feedback event of the target virtual object.

The target virtual object may be a virtual object that the viewer wants to perform an interactive operation, may be a virtual object corresponding to an operation position in the current frame image when the live viewer client triggers the interactive operation, and may be a virtual object at a click position in a live view of the viewer, for example.

After receiving the interaction instruction, the anchor server may analyze the interaction instruction to obtain object information, where the object information may be information of a target virtual object determined by an operation position of the interaction operation in a currently displayed live broadcast picture when the live broadcast viewer client receives the interaction operation of the viewer. The anchor server side can determine a target virtual object corresponding to the interaction instruction according to the object information and determine a feedback event of the target virtual object.

Specifically, when determining a feedback event of the target virtual object, the anchor server may detect whether a triggerable feedback event exists in the target virtual object, and if the triggerable feedback event does not exist, may determine that the current interaction operation is an invalid operation, otherwise, when the triggerable feedback event exists in the target virtual object, may further determine the feedback event indicated by the interaction instruction, for example, may determine one or more feedback events indicated by the interaction instruction in the multiple feedback events of the target virtual object.

In an alternative embodiment, the virtual object in the virtual live broadcast room may include an article for sale, and thus, in step S1020, the following method may be further performed:

and determining the commodity to be sold corresponding to the interaction instruction according to the object information to obtain a target commodity corresponding to the interaction instruction, and determining a feedback event of the target commodity. The feedback event of the target commodity can include any one of a commodity purchasing event, a commodity information browsing event and a commodity special effect displaying event. For example, the anchor server may determine a commodity to be sold to which the interaction instruction points according to the information of the virtual object included in the object information, determine the commodity to be sold as a target commodity corresponding to the interaction instruction, and further determine a feedback event corresponding to the interaction instruction according to the event information included in the interaction instruction, so as to obtain the feedback event of the target commodity.

Further, in order to determine a feedback event corresponding to the interaction instruction in the multiple feedback events of the target virtual object, in an optional implementation manner, when the feedback event of the interaction instruction is determined, the anchor server may further obtain a trigger instruction of the feedback event of the target virtual object, and when the trigger instruction is matched with the interaction instruction, determine the feedback event corresponding to the trigger instruction as the feedback event of the target virtual object.

Specifically, the anchor server may obtain a trigger instruction of a feedback event of the target virtual object according to a preset interaction feedback rule, and match the trigger instruction with the interaction instruction to determine the feedback event corresponding to the trigger instruction matched with the interaction instruction as the feedback event of the target virtual object.

And step S1030, sending the feedback event to the anchor client so that the anchor client generates a target video stream of the feedback event.

After the feedback event is determined, the anchor server side can send the feedback event to the anchor client side, so that the anchor client side can render a live broadcast picture through an engine of the UE to generate a target video stream.

By the interaction control method of the virtual live broadcast room executed by the anchor server, the feedback event of the interaction instruction can be determined by the anchor server, so that the anchor client can generate the target video stream according to the feedback event.

The present exemplary embodiment further provides a method for controlling interaction in a virtual live broadcast room performed by the live viewer client 110, where a graphical user interface may be provided through a display screen of the live viewer client, the graphical user interface may include a live broadcast interface, and the live broadcast interface may include a picture display area for displaying a live broadcast picture received by the live viewer client. The live interface can be an audience interface of a main broadcast displayed in the graphical user interface during live broadcast, and the picture display area can be all or a partial area in the live interface and can be used for displaying a live picture. As shown in fig. 11, a live view screen can be displayed in the screen display area, and a barrage, comments, gifts, and the like transmitted by viewers, a main broadcast, and the like during live view can be displayed in the public screen display area.

Fig. 12 shows a flow of an interaction control method of a virtual live room performed by the live viewer client 110, which may include the following steps S1210 to S1240:

and S1210, responding to the interaction operation of the audience aiming at the live broadcast picture, generating an interaction instruction, and intercepting the live broadcast picture displayed in the live broadcast interface at the current moment to obtain a current frame image.

When the audience carries out interactive operation on the live broadcast picture at the live broadcast audience client, the live broadcast audience client can generate an interactive instruction and intercept the live broadcast picture at the current moment to obtain a current frame image. The interaction instruction can include information of a virtual object corresponding to an operation position when the audience performs interaction operation, time of the interaction operation, information such as an account number and an address of a live audience client and the like. By the method, the playing time of the current frame image can be ensured to be consistent with the generation time of the interactive instruction, and the misjudgment operation that the interactive instruction is regarded as an invalid instruction due to the fact that the interactive instruction is asynchronous with the current frame image is avoided.

And S1220, sending the interaction instruction and the current frame image to the anchor client through the live broadcast platform server so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction.

The live broadcast audience client side can send the interaction instruction and the current frame image to the live broadcast platform server side, the live broadcast platform server side forwards the interaction instruction and the current frame image to the anchor client side, the anchor client side can send the received interaction instruction and the current frame image to the anchor server side, the anchor server side can determine a feedback event of the interaction instruction according to the current frame image, and the anchor client side can generate a target video stream according to the feedback event of the interaction instruction.

And step S1230, receiving a target video stream sent by the anchor client through the live broadcast platform server.

When the anchor client generates the target video stream according to the interaction instruction and the current frame image, the target video stream can be sent to the live audience client through the live platform server according to the account number, address and other information of the live audience client.

And S1240, playing the target video stream in the picture display area and controlling the virtual live broadcast room to display the effect.

In this exemplary embodiment, the screen display area may display a live screen, and the screen display area may be a whole area in the live interface or a partial area in the live interface. When the target video stream is played in the picture display area, the live broadcast audience client can control the live broadcast picture to carry out corresponding special effect or picture display.

In an optional implementation manner, the target video stream may include a video stream generated by the anchor client according to a feedback event of the target virtual object corresponding to the interaction instruction, and when controlling the virtual live broadcast room to perform effect display, the target virtual object may also be controlled to perform effect display.

As described above, the virtual object may include a commodity to be sold, and therefore, in an alternative embodiment, the target virtual object may include a target commodity, and when the target video stream is played, the virtual live broadcast room may be further controlled to perform a screen display operation corresponding to a feedback event of the target commodity. The screen display operation may include any one of controlling the target commodity to perform special effect display, controlling the live broadcast interface to jump to a detail page of the target commodity, adding an order for purchasing the target commodity in an order to be purchased at the live broadcast audience client, or providing a purchase interface of the target commodity.

For example, referring to fig. 13, when the feedback event of the target commodity is a commodity special effect display event, when the target video stream is played, the live broadcast viewer client may control the commodity identifier of the target commodity to be displayed in a dark color, and at the same time, the transparency of other commodities to be sold may be reduced to be displayed, or the target commodity may also present a corresponding animation special effect according to the commodity content, for example, when the target commodity is cat food, the live broadcast viewer client may play an animation picture that a kitten eats the cat food. For another example, when the feedback event of the target commodity is a commodity purchase event, and when the target video stream is played, as shown in fig. 14A, a function area added to the shopping cart may appear at the bottom of the live broadcast interface, and further when an operation instruction that the target commodity is dragged to the function area added to the shopping cart by the viewer is received, as shown in fig. 14B, the target commodity may be controlled to move along a track dragged by the viewer, and when the target commodity is dragged to the function area added to the shopping cart, a settlement function is triggered, that is, a settlement page may be skipped, so that the viewer completes a purchase operation.

By the method, the interaction mode of the virtual live broadcast room can be enriched, the requirement of interaction between audiences and the anchor is met, and the user experience is improved.

In an alternative embodiment, the following method may also be performed:

responding to the ordering operation of the target commodity by the audience, stopping displaying the target commodity by the virtual live broadcast room, and generating transaction data of the target commodity;

the transaction data are sent to a live broadcast platform server so that the live broadcast platform server can carry out settlement processing, and transaction result data obtained after settlement processing are returned to the anchor client;

and receiving the transaction result data aiming at the target commodity sent by the anchor client and a new video stream generated by the anchor client according to the transaction result data, and playing the new video stream.

For example, when the audience triggers the ordering operation of the target commodity, the virtual live broadcast room can cancel the display of the target commodity, and meanwhile, the transaction data of the target commodity can be generated and sent to the live broadcast platform server, so that the live broadcast platform server performs settlement processing to generate transaction result data. And then, the live broadcast platform server side can send the transaction result data to the anchor client side, and the anchor client side can generate a new video stream after knowing the transaction processing result and send the new video stream to the live broadcast audience client side, so that the live broadcast audience client side can play the new video stream. In the new video stream, the target item is no longer displayed, but a new item for sale may be displayed.

Fig. 15 is an interaction diagram illustrating an interaction control method of a virtual live broadcast room in the exemplary embodiment, and as shown in the figure, the method may include the following steps:

in step S1501, the anchor enters the virtual broadcast flow. Specifically, the anchor can be broadcast in front of the green screen, a virtual scene in the live broadcast platform is selected, the image matting picture of the anchor and the virtual scene are synthesized through the live broadcast platform, and the anchor enters a virtual broadcast starting state.

In step S1502, the anchor starts live broadcasting.

In the live broadcast process, the anchor client can shoot video pictures through a camera and the like and send the video pictures to the live broadcast platform server for processing, if special effects are added, live broadcast video streams are generated, and live broadcast audience clients can watch the live broadcast video streams through network connection with the live broadcast platform server.

In step S1503, the viewer enters the virtual live broadcast room of the anchor for watching.

Step S1504, the audience clicks or drags the display area of the live broadcast picture to trigger the interaction operation, and the live broadcast audience client 110 generates an interaction instruction; meanwhile, the live broadcast audience client can intercept the live broadcast picture at the current moment to obtain the current frame image.

In step S1505, the live viewer client 110 sends the interaction instruction and the current frame image to the anchor client 130.

Specifically, the live viewer client 110 may send the interaction instruction and the current frame image to the anchor client 130 through the live platform server 120.

In step S1506, the anchor client 130 receives the interaction instruction and the current frame image, and sends the interaction instruction and the current frame image to the anchor server 140.

In step S1507, the anchor server 140 determines a feedback event according to the current frame image and the interaction instruction.

In step S1508, the anchor server 140 sends the feedback event to the anchor client 130 so that the anchor client 130 can generate a target video stream of the feedback event.

In step S1509, the anchor client 130 generates a target video stream of the feedback event by performing UE instance rendering video pictures.

In step S1510, the anchor client 130 sends the target video stream to the live viewer client 110 through the live platform service 120.

In step S1511, the live viewer client 110 plays the target video stream. And in the playing process, displaying in the virtual live broadcast room according to the effect indicated by the interactive instruction.

In step S1512, the anchor client 130 determines the feedback type of the feedback event. That is, the anchor client 130 determines whether the feedback event is a single execution event or a continuous execution event.

In step S1513, the anchor client 130 monitors the playing progress of the target video stream through the live platform server 120, and determines whether the feedback progress is over. For example, when it is determined that the target video stream is played, if the feedback event is a single execution event, step S1514 is executed, the anchor client 130 generates and transmits a normal live video stream, if the feedback event is a continuous feedback event, it is further determined whether the feedback progress is finished, after the feedback progress is finished, step S1514 is executed, the anchor client 130 generates and transmits a normal live video stream, and in the live video stream, a virtual scene in a virtual live broadcast room is displayed as a scene before an interaction instruction is not triggered; when it is determined that the feedback progress is not over, step S1510 is performed to continue transmitting the target video stream for playing.

In step S1514, the anchor client 130 generates a live video stream and sends it to the live viewer client 110. At this time, the virtual scene of the virtual live broadcast room in the live broadcast video stream is consistent with the scene before the interactive instruction is not triggered.

In step S1515, the live viewer client 110 plays the live video stream.

In summary, according to the interaction control method of the virtual live broadcast room in the exemplary embodiment, the anchor client receives the interaction instruction and the current frame image of the live broadcast viewer client, determines the feedback event of the interaction instruction according to the current frame image through the anchor server, generates the target video stream of the feedback event through the anchor client, and sends the target video stream to the live broadcast viewer client, so that the live broadcast viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream. On one hand, the generated target video stream is sent to the live broadcast audience client side to be played, and the virtual live broadcast room is controlled to carry out effect display, so that the interaction modes of audiences and the live broadcast room are enriched, various interaction requirements of the audiences are met, the contact between the audiences and the live broadcast room is enhanced, and the payment conversion of users is facilitated; on the other hand, by determining the current frame image and determining the feedback event of the interactive instruction according to the current frame image, the interactive instruction can be ensured to correspond to the live broadcast image when the user performs interactive operation, the problem that the interactive instruction and the live broadcast image are not synchronous due to network delay and the like is avoided, and the accuracy of determining the feedback event of the interactive instruction is improved.

Further, the present exemplary embodiment also provides an interaction control apparatus of a virtual live broadcast, which may be applied to an anchor client, and as shown in fig. 16, an interaction control apparatus 1600 of a virtual live broadcast may include: the receiving module 1610 is configured to receive an interaction instruction and a current frame image sent by a live viewer client through a live platform server; the first sending module 1620, configured to send the interaction instruction and the current frame image to the anchor server, so that the anchor server determines a feedback event of the interaction instruction according to the current frame image; a generating module 1630, configured to receive a feedback event sent by the anchor server, and generate a target video stream of the feedback event; the second sending module 1640 may be configured to send the target video stream to the live viewer client, so that the live viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream.

In an exemplary embodiment of the disclosure, the feedback event includes a feedback event of a target virtual object corresponding to the interaction instruction, and when the feedback event sent by the anchor server is received and a target video stream of the feedback event is generated, the generating module 1630 may be configured to generate the target video stream of the feedback event related to the target virtual object according to the feedback event of the target virtual object.

In an exemplary embodiment of the disclosure, the target virtual object includes a target commodity, and when a target video stream of a feedback event is generated, the generating module 1630 may further be configured to obtain commodity data of the target commodity, and generate the target video stream of the feedback event of the target commodity according to the commodity data, where the feedback event of the target commodity includes any one of a commodity purchase event, a commodity information browsing event, and a commodity special effect display event.

In an exemplary embodiment of the present disclosure, the generating module 1630 may further be configured to receive transaction result data for a target commodity, which is sent by a live platform server; and generating a new video stream according to the transaction result data, and sending the new video stream to the live audience client through the live platform server so as to enable the live audience client to play the new video stream.

In an exemplary embodiment of the disclosure, the generating module 1630 may be configured to obtain an effect template of a feedback event, extract a matting picture of a anchor from a real scene picture including the anchor captured by a camera, and synthesize the matting picture and the effect template to generate a target video stream.

In an exemplary embodiment of the present disclosure, when the target video stream is sent to the live viewer client through the live platform server, so that the live viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream, the second sending module 1640 may be configured to determine a feedback type of the feedback event, where the feedback type includes a single execution event and a continuous execution event, and control the virtual live broadcast room to perform effect display according to the feedback type of the feedback event.

In an exemplary embodiment of the disclosure, the second sending module 1640 may be further configured to monitor, when the feedback type of the feedback event is a single execution event, the playing progress of the target video stream through the live broadcast platform server, and stop the effect display on the virtual live broadcast room when the playing of the target video stream is completed, and when the feedback type of the feedback event is a continuous execution event, monitor, through the live broadcast platform server, the playing progress of the target video stream, and determine the feedback progress of the feedback event, when it is determined that the feedback progress of the feedback event satisfies a preset end condition, end the effect display on the virtual live broadcast room, and when it is determined that the feedback progress of the feedback event does not satisfy the preset end condition, keep the effect display on the virtual live broadcast room.

The present exemplary embodiment further provides an interaction control apparatus of a virtual live broadcast room, which may be applied to a main broadcast server, and as shown in fig. 17, an interaction control apparatus 1700 of a virtual live broadcast room may include: a receiving module 1710, configured to receive an interaction instruction and a current frame image sent by an anchor client, where the interaction instruction may be used to instruct a virtual live broadcast room to perform effect display, and the current frame image is a live broadcast picture displayed in a live broadcast interface at a current time captured by a live broadcast viewer client; a determining module 1720, configured to determine a feedback event of the interaction instruction according to the current frame image; the sending module 1730 may be configured to send the feedback event to the anchor client, so that the anchor client generates a target video stream of the feedback event.

In an exemplary embodiment of the disclosure, the determining module 1720 may be configured to analyze the interaction instruction, determine a target virtual object corresponding to the interaction instruction according to object information obtained through the analysis, and determine a feedback event of the target virtual object.

In an exemplary embodiment of the present disclosure, the virtual object may include a commodity to be sold, and when the target virtual object corresponding to the interaction instruction is determined according to the object information obtained after the analysis and the feedback event of the target virtual object is determined, the determining module 1720 may be further configured to determine the commodity to be sold corresponding to the interaction instruction according to the object information to obtain the target commodity corresponding to the interaction instruction; and determining a feedback event of the target commodity.

In an exemplary embodiment of the disclosure, when determining a feedback event of an interaction instruction, the sending module 1730 may be configured to obtain a trigger instruction of the feedback event of the target virtual object, and when the trigger instruction matches the interaction instruction, determine the feedback event corresponding to the trigger instruction as the feedback event of the target virtual object.

The exemplary embodiment further provides an interaction control device of a virtual live broadcast room, which can be applied to a live broadcast viewer client, and a graphical user interface is provided through a display screen of the live broadcast viewer client, where the graphical user interface includes a live broadcast interface, and a picture display area for displaying a live broadcast picture received by the live broadcast viewer client is included in the live broadcast interface, as shown in fig. 18, an interaction control device 1800 of the virtual live broadcast room may include: the generating module 1810 may be configured to respond to a selection operation of a viewer on a live broadcast picture, generate an interaction instruction, and intercept a live broadcast picture displayed in a live broadcast interface at a current moment to obtain a current frame image; the sending module 1820 may be configured to send the interaction instruction and the current frame image to the anchor client through the live broadcast platform server, so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction; the receiving module 1830 may be configured to receive a target video stream sent by the anchor client through the live platform server; the playing module 1840 may be configured to play the target video stream in the screen display area, and control the virtual live broadcast room to perform effect display.

In an exemplary embodiment of the disclosure, when generating the interaction instruction, the generating module 1810 may be configured to determine, in response to an interaction operation of a viewer on a live broadcast screen, a virtual object corresponding to an operation position of the interaction operation in a current frame image as a target virtual object, and generate the interaction instruction about the target virtual object.

In an exemplary embodiment of the disclosure, the target video stream includes a video stream generated by the anchor client according to a feedback event of the target virtual object corresponding to the interaction instruction, and when controlling the virtual live broadcast room to perform effect display, the playing module 1840 may be configured to control the target virtual object to perform the effect display.

In an exemplary embodiment of the disclosure, the target virtual object may include a target product, and the playing module 1840 may be further configured to control the virtual live broadcast room to execute a screen display operation corresponding to a feedback event of the target product when the target video stream is played, where the screen display operation includes any one of: controlling the target commodity to carry out special effect display; controlling the live broadcast interface to jump to a detail page of the target commodity; and adding an order for purchasing the target commodity in the to-be-purchased order of the live audience client or providing a purchase interface of the target commodity.

In an exemplary embodiment of the disclosure, the playing module 1840 may be further configured to, in response to an order placing operation of a target commodity by a viewer, stop display of the target commodity in a virtual live broadcast room, generate transaction data of the target commodity, send the transaction data to a live broadcast platform server, so that the live broadcast platform server performs settlement processing, return the transaction result data obtained after the settlement processing to an anchor client, receive the transaction result data for the target commodity sent by the anchor client and a new video stream generated by the anchor client according to the transaction result data, and play the new video stream.

The specific details of each module in the above apparatus have been described in detail in the method section, and details of an undisclosed scheme may refer to the method section, and thus are not described again.

As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or program product. Accordingly, various aspects of the present disclosure may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the disclosure described in the above-mentioned "exemplary methods" section of this specification, when the program product is run on the terminal device.

Referring to fig. 19, a program product 1900 for implementing the above method according to an exemplary embodiment of the present disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

Program product 1900 may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).

Exemplary embodiments of the present disclosure also provide an electronic device capable of implementing the above method, which may be any one of the live viewer client 110, the live platform server 120, the anchor client 130 and the anchor server 140 shown in fig. 1. An electronic device 2000 according to this exemplary embodiment of the present disclosure is described below with reference to fig. 20. The electronic device 2000 shown in fig. 20 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 20, the electronic device 2000 may be embodied in the form of a general purpose computing device. The components of the electronic device 2000 may include, but are not limited to: the at least one processing unit 2010, the at least one memory unit 2020, the bus 2030 connecting the various system components including the memory unit 2020 and the processing unit 2010, and the display unit 2040.

Where the storage unit 2020 stores program code, the program code may be executed by the processing unit 2010 such that the processing unit 2010 performs the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned "exemplary methods" section of this specification. For example, processing unit 2010 may perform the method steps shown in fig. 2, 6, 9-10, 12, and 15, and/or the like.

The storage unit 2020 may include readable media in the form of volatile storage units, such as a random access memory unit (RAM)2021 and/or a cache memory unit 2022, and may further include a read only memory unit (ROM) 2023.

The storage unit 2020 may also include a program/utility 2024 having a set (at least one) of program modules 2025, such program modules 2025 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.

Bus 2030 may be one or more of any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 2000 may also communicate with one or more external devices 2100 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 2000, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 2000 to communicate with one or more other computing devices. Such communication may occur over an input/output (I/O) interface 2050. Also, the electronic device 2000 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) through the network adapter 2060. As shown, the network adapter 2060 communicates with the other modules of the electronic device 2000 via the bus 2030. It should be appreciated that although not shown, other hardware and/or software modules may be used in conjunction with the electronic device 2000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.

It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit, according to exemplary embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.

Furthermore, the above-described figures are merely schematic illustrations of processes included in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.

Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the exemplary embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to make a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the exemplary embodiments of the present disclosure.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

Claims

1. An interaction control method of a virtual live broadcast room is applied to an anchor client, and is characterized in that the method comprises the following steps:

receiving an interaction instruction and a current frame image sent by a live broadcast audience client through a live broadcast platform server, wherein the interaction instruction is used for indicating a virtual live broadcast room to perform effect display, and the current frame image is a live broadcast picture displayed in a live broadcast interface at the current moment intercepted by the live broadcast audience client;

sending the interaction instruction and the current frame image to an anchor server so that the anchor server determines a feedback event of the interaction instruction according to the current frame image;

receiving the feedback event sent by the anchor server, and generating a target video stream of the feedback event;

and sending the target video stream to the live audience client through the live platform server so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream.

2. The method according to claim 1, wherein the feedback event includes a feedback event of a target virtual object corresponding to the interaction instruction, and when receiving the feedback event sent by the anchor server and generating a target video stream of the feedback event, the method includes:

and generating a target video stream of the feedback event of the target virtual object according to the feedback event of the target virtual object.

3. The method of claim 2, wherein the target virtual object comprises a target commodity, and wherein in generating the target video stream of the feedback event, the method comprises:

and acquiring commodity data of the target commodity, and generating a target video stream of a feedback event of the target commodity according to the commodity data, wherein the feedback event of the target commodity comprises any one of a commodity purchasing event, a commodity information browsing event and a commodity special effect display event.

4. The method of claim 3, further comprising:

receiving transaction result data aiming at the target commodity sent by the live broadcast platform server;

and generating a new video stream according to the transaction result data, and sending the new video stream to the live broadcast audience client through the live broadcast platform server so that the live broadcast audience client plays the new video stream.

5. The method of claim 1, wherein the generating the target video stream of the feedback event comprises:

obtaining an effect template of the feedback event;

extracting a keying picture of a anchor from a real scene picture containing the anchor and shot by a camera;

and synthesizing the image matting picture and the effect template to generate the target video stream.

6. The method of claim 1, wherein when the target video stream is sent to the live viewer client through the live platform server, so that the live viewer client controls the virtual live broadcast room to perform effect display when playing the target video stream, the method further comprises:

determining a feedback type of the feedback event, wherein the feedback type comprises a single execution event and a continuous execution event;

7. The method according to claim 6, wherein the controlling the virtual live broadcast room to perform effect exhibition according to the feedback type of the feedback event comprises:

when the feedback type of the feedback event is the single execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, and stopping the effect display of the virtual live broadcast room when the target video stream is played; and

when the feedback type of the feedback event is the continuous execution event, monitoring the playing progress of the target video stream through the live broadcast platform server, and determining the feedback progress of the feedback event;

when the feedback progress of the feedback event meets a preset ending condition, ending the effect display of the virtual live broadcast room;

and when the feedback progress of the feedback event is determined not to meet the preset ending condition, maintaining the effect display of the virtual live broadcast room.

8. An interaction control method of a virtual live broadcast room is applied to a main broadcast server side, and is characterized in that the method comprises the following steps:

receiving an interaction instruction and a current frame image sent by a anchor client;

determining a feedback event of the interaction instruction according to the current frame image;

and sending the feedback event to the anchor client so that the anchor client generates a target video stream of the feedback event.

9. The method of claim 8, wherein the determining the feedback event of the interaction instruction according to the current frame image comprises:

10. The method according to claim 9, wherein the virtual object includes a commodity for sale, and when a target virtual object corresponding to the interaction instruction is determined according to the object information obtained after the analysis and a feedback event of the target virtual object is determined, the method includes:

determining the commodity to be sold corresponding to the interaction instruction according to the object information to obtain a target commodity corresponding to the interaction instruction;

and determining a feedback event of the target commodity.

11. The method of claim 9, wherein in determining the feedback event for the interaction instruction, the method further comprises:

and acquiring a trigger instruction of a feedback event of the target virtual object, and determining the feedback event corresponding to the trigger instruction as the feedback event of the target virtual object when the trigger instruction is matched with the interaction instruction.

12. An interactive control method of a virtual live broadcast room is applied to a live broadcast audience client, a graphical user interface is provided through a display screen of the live broadcast audience client, the graphical user interface comprises a live broadcast interface, wherein the live broadcast interface comprises a picture display area for displaying a live broadcast picture received by the live broadcast audience client, and the method is characterized by comprising the following steps of:

responding to the interaction operation of the audience aiming at the live broadcast picture, generating an interaction instruction, and intercepting the live broadcast picture displayed in the live broadcast interface at the current moment to obtain a current frame image;

sending the interaction instruction and the current frame image to an anchor client through a live broadcast platform server so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction; and

receiving the target video stream sent by the anchor client through the live broadcast platform server;

and playing the target video stream in the picture display area, and controlling the virtual live broadcast room to display the effect.

13. The method of claim 12, wherein in generating the interaction instruction, the method comprises:

and responding to the interactive operation of the audience aiming at the live broadcast picture, determining a virtual object corresponding to the operation position of the interactive operation in the current frame image as a target virtual object, and generating an interactive instruction about the target virtual object.

14. The method according to claim 12, wherein the target video stream includes a video stream generated by the anchor client according to a feedback event of a target virtual object corresponding to the interaction instruction, and when controlling the virtual live broadcast room to perform effect display, the method includes:

and controlling the target virtual object to perform effect display.

15. The method of claim 14, wherein the target virtual object comprises a target commodity, and wherein controlling the target virtual object to perform effect presentation while playing the target video stream comprises:

when the target video stream is played, controlling the virtual live broadcast room to execute picture display operation corresponding to the feedback event of the target commodity;

the screen display operation includes any one of:

controlling the target commodity to carry out special effect display;

controlling the live broadcast interface to jump to a detail page of the target commodity;

and adding an order for purchasing the target commodity in the order to be purchased of the live audience client or providing a purchase interface of the target commodity.

16. The method of claim 15, further comprising:

in response to the order placing operation of the target commodity by the audience, stopping the display of the target commodity by the virtual live broadcast room, and generating transaction data of the target commodity;

sending the transaction data to the live broadcast platform server to enable the live broadcast platform server to carry out settlement processing, and returning transaction result data obtained after settlement processing to the anchor client;

17. An interaction control device of a virtual live broadcast room is applied to an anchor client, and is characterized by comprising:

the receiving module is used for receiving an interaction instruction and a current frame image which are sent by a live audience client through a live platform server;

the first sending module is used for sending the interaction instruction and the current frame image to an anchor server so that the anchor server can determine a feedback event of the interaction instruction according to the current frame image;

the generating module is used for receiving the feedback event sent by the anchor server and generating a target video stream of the feedback event;

and the second sending module is used for sending the target video stream to the live audience client so as to control the virtual live broadcast room to perform effect display when the live audience client plays the target video stream.

18. An interaction control device of a virtual live broadcast room is applied to a main broadcast server side, and is characterized by comprising the following components:

the receiving module is used for receiving an interaction instruction and a current frame image sent by an anchor client, wherein the interaction instruction is used for indicating a virtual live broadcast room to perform effect display, and the current frame image is a live broadcast picture displayed in a live broadcast interface at the current moment intercepted by a live broadcast audience client;

the determining module is used for determining a feedback event of the interaction instruction according to the current frame image;

and the sending module is used for sending the feedback event to the anchor client so that the anchor client generates a target video stream of the feedback event.

19. The utility model provides an interactive control device of virtual live broadcast room, is applied to live broadcast audience client, through the display screen of live broadcast audience client provides a graphical user interface, graphical user interface includes the live broadcast interface, wherein, including showing in the live broadcast interface the picture display area of the live broadcast picture that the live broadcast audience client received, its characterized in that, the device includes:

the generation module is used for responding to the selection operation of the audience aiming at the live broadcast picture, generating an interaction instruction, and intercepting the live broadcast picture displayed in the live broadcast interface at the current moment to obtain a current frame image;

the sending module is used for sending the interaction instruction and the current frame image to an anchor client through a live broadcast platform server so that the anchor client generates a target video stream of a feedback event corresponding to the interaction instruction; and

the receiving module is used for receiving the target video stream sent by the anchor client through the live platform server;

and the playing module is used for playing the target video stream in the picture display area and controlling the virtual live broadcast room to display the effect.

20. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1-16.

21. An electronic device, comprising:

a processor; and

a memory for storing executable instructions of the processor;

wherein the processor is configured to perform the method of any of claims 1-16 via execution of the executable instructions.