CN116567274A

CN116567274A - Live broadcast interaction method, device and system

Info

Publication number: CN116567274A
Application number: CN202210102164.XA
Authority: CN
Inventors: 陈曦
Original assignee: Huawei Cloud Computing Technologies Co Ltd
Current assignee: Huawei Cloud Computing Technologies Co Ltd
Priority date: 2022-01-27
Filing date: 2022-01-27
Publication date: 2023-08-08
Also published as: WO2023142756A1

Abstract

The application provides a live interaction method, a live interaction device and a live interaction system, wherein the method comprises the following steps: the cloud computing platform fuses the image of the anchor and the virtual object to obtain a first live broadcast picture, the first live broadcast picture is sent to the terminal equipment of the audience and the terminal equipment of the anchor for display, then when the terminal equipment of the audience or the terminal equipment of the anchor receives a first operation of the user on the image of the anchor or the virtual object in the first live broadcast picture, the first operation is processed into a first operation instruction, the first operation instruction is sent to the cloud computing platform, the cloud computing platform obtains a second live broadcast picture according to the first operation instruction and the first live broadcast picture, and after the cloud computing platform obtains the second live broadcast picture, the second live broadcast picture is sent to the terminal equipment of the audience and the terminal equipment of the anchor for display. The method can optimize live interaction effect, provide various interaction modes for the anchor and the audience, and optimize interaction experience between the anchor and the audience.

Description

Live broadcast interaction method, device and system

Technical Field

The application relates to the technical field of live broadcasting, in particular to a live broadcasting interaction method, device and system.

Background

With the rapid development of internet technology, live broadcasting is now in deep in daily work and life of people, and various live broadcasting can bring diversified information to people and provide fresh entertainment experience.

However, in the current live broadcast, the interaction mode between the host and the audience is relatively single, and in general, the host can only introduce the real-life articles to the audience, and the audience can only interact with the host by sending a barrage, praying, commenting or giving a virtual gift (such as a virtual rocket or a virtual flower, etc.), so that the interaction mode is not ideal for the host and the audience.

Disclosure of Invention

The live broadcast interaction method, device and system can optimize live broadcast interaction effect, provide various interaction modes for the anchor and the audience, and optimize interaction experience between the anchor and the audience.

In a first aspect, a live interaction method is provided, applied to a cloud computing platform, and the method includes: firstly, fusing an image of a main broadcasting with a virtual object to obtain a first direct broadcasting picture, then, transmitting the first direct broadcasting picture to a terminal device of a spectator and the terminal device of the main broadcasting, then, receiving a first operation instruction of the image of the main broadcasting or the virtual object in the first direct broadcasting picture, finally, obtaining a second direct broadcasting picture according to the first operation instruction and the first direct broadcasting picture, and transmitting the second direct broadcasting picture to the terminal device of the spectator and the terminal device of the main broadcasting.

Wherein, the host is a virtual person or a real person, that is, the image of the host can be a virtual character image or a real person image of the host, specifically, when the image of the host is a virtual character image, the image of the host can be a virtual three-dimensional character model; the virtual object may be any virtual three-dimensional model such as a virtual building, a virtual stage, a virtual animal, a virtual plant, a virtual table, a virtual chessboard, a virtual golf ball, etc., and the virtual object may be used to realize an interactive game between a spectator and a host, such as a chess game or a ball game, etc., which is not particularly limited herein.

The first operation instruction is an instruction obtained by processing a first operation, and the first operation is an operation of a spectator on an image or a virtual object of a host in the first direct-broadcasting picture, or an operation of the host on the image or the virtual object of the host in the first direct-broadcasting picture. It should be understood that, when the first operation is an operation performed by the viewer, the first operation instruction is an instruction obtained by processing the first operation by the terminal device of the viewer, and the terminal device of the viewer sends the first operation instruction to the cloud computing platform after obtaining the first operation instruction; when the first operation is performed by the anchor, the first operation instruction is an instruction obtained by processing the first operation by the anchor terminal equipment, and the anchor terminal equipment sends the first operation instruction to the cloud computing platform after obtaining the first operation instruction.

The first operation instruction may be an instruction for changing the avatar of the main cast in the first direct cast picture, or an instruction for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, a color of a point on the virtual object in the first direct cast picture, or an instruction for adding the avatar of the real object in the first direct cast picture, which is not particularly limited herein.

When the first operation instruction is used for changing the image of the anchor in the first on-line screen, the first operation instruction may be specifically used for adding an ornament (such as a hairpin and glasses) to the image of the anchor, changing the hairstyle of the anchor (such as changing straight hair into curly hair), changing the garment of the anchor (such as changing one-piece dress into western-style clothes), or the like, where in a specific implementation, the ornament, the hairstyle, or the garment may be a virtual object originally included in the first on-line screen. When the character of the anchor is a virtual three-dimensional character model, the first operation instruction may be used to change one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the character of the anchor, for example, the character of the anchor is located in a middle position of the screen in the first on-screen, the first operation instruction is an instruction to move the character of the anchor to a lower left corner of the first on-screen, and for example, the character of the anchor is a standing posture in the first on-screen, and the first operation instruction is an instruction to make the character of the anchor sit down in the first on-screen.

When the first operation instruction is used to change one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the virtual object in the first on-screen, the first operation instruction may be used to change a position of the virtual object in the first on-screen, change a speed at which the virtual object moves in the first on-screen, change a color at which the virtual object is displayed in the first on-screen, or the like, for example, assuming that the virtual object is a chess piece, the first operation instruction may be an instruction for changing a position of the chess piece, and for example, assuming that the virtual object is a golf ball, the first operation instruction may be an instruction for letting the golf ball fly up at a certain angle and speed.

When the first operation instruction is used to add the image of the real object in the first direct-play picture, the first operation instruction may be specifically used to add the image of the real cap, the image of the real plant, the image of the real animal, etc. in the first direct-play picture, which are not specifically limited herein.

According to the live broadcast interaction method, various interaction operations can be carried out on the image or virtual object of the anchor in the live broadcast picture by the audience or the anchor, and the live broadcast picture is updated based on the interaction operations carried out by the audience or the anchor, namely, the live broadcast picture seen by the audience and the anchor can be changed along with various interaction operations carried out by the audience or the anchor by the method, so that various interaction modes can be provided for the anchor and the audience by the scheme, and interaction experience between the anchor and the audience is optimized.

In addition, in the scheme, the image of the anchor and the virtual object are fused together to obtain the live broadcast picture, and compared with the live broadcast picture obtained by simply superposing two-dimensional images in the prior art, the stereoscopic display effect of the live broadcast picture is better, and the displayed content is more natural and coordinated.

In a possible implementation manner, the live interaction method provided in the first aspect further includes the following steps: firstly, receiving a second operation instruction of a host computer image or a virtual object in a second live broadcast picture, then obtaining a third live broadcast picture according to the second operation instruction and the second live broadcast picture, and finally, sending the third live broadcast picture to terminal equipment of a spectator and terminal equipment of the host computer.

The second operation instruction is an instruction obtained by processing the second operation, when the first operation is an operation performed by a spectator, the second operation is an operation performed by a host on a host image or a virtual object in a second live broadcast picture, the second operation instruction is obtained by processing the second operation by a terminal device of the host, and the terminal device of the host sends the second operation instruction to the cloud computing platform after obtaining the second operation instruction; when the first operation is an operation performed by the anchor, the second operation is an operation performed by the audience on the anchor image or the virtual object in the second live broadcast picture, the second operation instruction is obtained by processing the second operation by the terminal equipment of the audience, and the terminal equipment of the audience sends the second operation instruction to the cloud computing platform after obtaining the second operation instruction.

The second operation instruction may be an instruction for changing the avatar of the main cast in the first direct cast picture, or an instruction for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, a color of a point on the virtual object in the first direct cast picture, or an instruction for adding the avatar of the real object in the first direct cast picture, which is not particularly limited herein.

It can be seen that after the live broadcast picture updated based on the operation of the opposite side is seen by the live broadcast and the audience, the live broadcast picture can be updated again by operating the image or the virtual object of the live broadcast in the updated live broadcast picture, that is, the live broadcast and the audience can interact by operating the image or the virtual object of the live broadcast in the live broadcast picture for multiple times.

In one possible implementation manner, the fusing of the image and the virtual object of the anchor may be specifically implemented in the following manner, so as to obtain a first direct-play picture: and after fusing the three-dimensional data of the anchor and the three-dimensional data of the virtual object, processing the fused three-dimensional data to obtain a two-dimensional first direct broadcasting picture.

Wherein, the three-dimensional data of the anchor refers to the data of the three-dimensional model constituting the anchor, and the three-dimensional data of the virtual object refers to the data of the three-dimensional model constituting the virtual object.

In a specific implementation, the virtual objects may be divided into virtual scenes and virtual props, where the virtual scenes may be used as scenes where the image of the host is located when the host is live, for example, virtual game scenes, virtual life scenes, or virtual work scenes, and the virtual props may be used as props that interact with the audience when the host is live, for example, virtual chessboard, virtual chess pieces, virtual golf balls, virtual football, virtual ornaments, and the like.

The three-dimensional data of the anchor and the three-dimensional data of the virtual object are fused, and it can be understood that the three-dimensional model corresponding to the image of the anchor and the virtual prop are placed at the proper positions in the virtual scene.

In a specific implementation, the fused three-dimensional data may be subjected to rendering processing to obtain a two-dimensional first direct-play picture, where the rendering method may be a rasterization rendering method, a ray tracing rendering method, or a method in which the rasterization rendering method and the ray tracing rendering method are mixed, which is not limited herein specifically.

In the live broadcast picture obtained by implementing the implementation mode, the image of the anchor and the virtual object are fused together, and compared with the live broadcast picture obtained by simply superposing two-dimensional images in the prior art, the live broadcast picture has better stereoscopic display effect of the image of the anchor and the virtual object and is more natural and coordinated.

In one possible implementation manner, the fusing of the image and the virtual object of the anchor may be specifically implemented in the following manner, so as to obtain a first direct-play picture: firstly, acquiring a two-dimensional image of a host, and then fusing the two-dimensional image of the host and the two-dimensional image of a virtual object to obtain a two-dimensional first direct broadcasting picture.

The two-dimensional image of the anchor can be a live video which is uploaded in real time by the anchor and comprises a plurality of frames of continuous images, then, character recognition is carried out on each frame of image, and after the anchor is recognized, the two-dimensional image of the anchor is scratched out of each frame of image; the two-dimensional image of the virtual object may be obtained by performing rendering processing on three-dimensional data of the virtual object, and the rendering method may be a rasterization rendering method, a ray tracing rendering method, or a method in which the rasterization rendering method and the ray tracing rendering method are mixed, which is not particularly limited herein.

In the specific implementation, the two-dimensional image of the anchor and the two-dimensional image of the virtual object can be subjected to virtual-real fusion through the augmented reality technology, so that a first live broadcast picture is obtained, in the live broadcast picture, the image of the anchor and the virtual object are fused together, the three-dimensional display effect is good, and the picture is natural and coordinated.

In a second aspect, a live interaction method is provided, applied to a terminal device of a viewer or a terminal device of a host, and the method includes: first, a first live view including a character of a main cast and a virtual object is displayed, and then, a second live view obtained according to a first operation instruction for the character of the main cast or the virtual object in the first live view and the first live view is displayed.

In a possible implementation manner, the live interaction method provided in the second aspect further includes the following steps: and displaying a third live broadcast picture, wherein the third live broadcast picture is obtained according to a second operation instruction of the image or the virtual object of the host broadcast in the second live broadcast picture and the second live broadcast picture.

In one possible implementation manner, the first operation instruction is an instruction obtained by processing the first operation, and the second operation instruction is an instruction obtained by processing the second operation; the first operation is an operation of audience on the image or virtual object of the anchor in the first live broadcast picture, and the second operation is an operation of the anchor on the image or virtual object of the anchor in the second live broadcast picture; or, the first operation is an operation of the anchor on the anchor image or virtual object in the first live broadcast picture, and the second operation is an operation of the audience on the anchor image or virtual object in the second live broadcast picture.

In one possible implementation, the anchor is a real or virtual person.

In one possible implementation manner, the first direct broadcast picture is a two-dimensional picture obtained by fusing three-dimensional data of a host and three-dimensional data of a virtual object and processing the fused three-dimensional data.

In one possible implementation, the first direct broadcast picture is obtained by fusing a two-dimensional image of the anchor and a two-dimensional image of the virtual object.

In one possible implementation, the first operating instruction is for changing the avatar of the anchor; or, the first operation instruction is used for changing one or more of coordinate values, moving speed, acceleration, offset angle, moving direction and color of the point on the virtual object; or, the first operation instruction is used for adding the image of the real object in the first direct broadcast picture.

In one possible implementation, the first operating instructions are used to add decorations, change hairstyles or change clothing to the image of the anchor.

In one possible implementation, the virtual object is used to implement an interactive game between spectators and a host player, such as a chess game or a ball game, etc.

In a third aspect, a live broadcast interaction method is provided, and is applied to a live broadcast system, where the live broadcast system includes a cloud computing platform, a terminal device of a viewer, and a terminal device of a host, and the method includes: the cloud computing platform fuses the image of the anchor and the virtual object to obtain a first live broadcast picture, the first live broadcast picture is sent to the terminal equipment of the audience and the terminal equipment of the anchor for display, then the terminal equipment of the audience or the terminal equipment of the anchor sends a first operation instruction for the image of the anchor or the virtual object in the first live broadcast picture to the cloud computing platform, the cloud computing platform obtains a second live broadcast picture according to the first operation instruction and the first live broadcast picture after receiving the first operation instruction, and finally the cloud computing platform sends the second live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor for display.

In a fourth aspect, a live interaction device is provided, applied to a cloud computing platform, where the device includes modules for performing the method provided in the first aspect or any one of the possible implementations of the first aspect.

In a fifth aspect, a live interaction device is provided, for application to a terminal device of a host or a terminal device of a viewer, the device comprising means for performing the method provided in the second aspect or any one of the possible implementations of the second aspect.

In a sixth aspect, a live broadcast system is provided, where the live broadcast system includes the live broadcast interaction device described in the fourth aspect and the interaction device described in the fifth aspect.

In a seventh aspect, a cloud computing platform is provided that includes one or more computing devices, each computing device including a processor and a memory; the processor of the one or more computing devices is configured to execute the instructions stored by the memory of the one or more computing devices, causing the one or more computing devices to perform the steps of: firstly, fusing an image of a main broadcasting with a virtual object to obtain a first direct broadcasting picture, then, transmitting the first direct broadcasting picture to a terminal device of a spectator and the terminal device of the main broadcasting, then, receiving a first operation instruction of the image of the main broadcasting or the virtual object in the first direct broadcasting picture, finally, obtaining a second direct broadcasting picture according to the first operation instruction and the first direct broadcasting picture, and transmitting the second direct broadcasting picture to the terminal device of the spectator and the terminal device of the main broadcasting.

An eighth aspect provides a terminal device comprising a processor and a memory; the processor is configured to execute the instructions stored in the memory, and cause the terminal device to perform the following steps: first, a first live view including a character of a main cast and a virtual object is displayed, and then, a second live view obtained according to a first operation instruction for the character of the main cast or the virtual object in the first live view and the first live view is displayed.

In a ninth aspect, a computer readable storage medium is provided, the computer readable storage medium storing instructions for implementing the method provided by any one of the possible implementations of the first to third aspects.

In a tenth aspect, there is provided a computer program product comprising a computer program which, when read and executed by a computing device, causes the computing device to perform the method as provided by any one of the possible implementations of the first to third aspects above.

Drawings

Fig. 1 is a schematic structural diagram of a live broadcast system according to the present application;

fig. 2 is a schematic diagram of a live broadcast screen according to the present application;

Fig. 3 is an interaction schematic diagram of a live interaction method provided in the present application;

FIG. 4 is a schematic diagram of a fusion of three-dimensional data of a host and three-dimensional data of a virtual object provided herein;

FIG. 5 is a flow diagram of a rasterized rendering method provided herein;

FIG. 6 is a schematic diagram of a process of transformation of a vertex shader provided herein;

FIG. 7 is a schematic diagram of a tessellation technique provided herein;

fig. 8A is a schematic diagram of a live view provided in the present application;

fig. 8B is a schematic diagram of a live view provided in the present application;

fig. 9 is a schematic structural diagram of a live interaction device provided in the present application;

fig. 10 is a schematic structural diagram of another live interaction device provided in the present application;

fig. 11 is a schematic structural diagram of a terminal device provided in the present application;

FIG. 12 is a schematic diagram of a cloud computing platform provided herein;

fig. 13 is a schematic structural diagram of a computing device provided herein.

Detailed Description

The technical solutions provided in the present application will be described below with reference to the accompanying drawings.

In order to make the technical solution provided in the present application clearer, before specifically describing the technical solution provided in the present application, explanation of related terms is first performed.

Virtual object: referring to objects that are not present in the real world, virtual objects are virtual three-dimensional models created in advance in the virtual world that can be used to reflect objects in the real world in the virtual world.

Virtual scene: the virtual reality scene can be simulated by a computer by utilizing a three-dimensional virtual reality scene technology, can be a half-simulated half-fictional three-dimensional environment scene, and can be a pure fictional three-dimensional environment scene. The three-dimensional virtual reality scene technology is a computer simulation system capable of creating and experiencing a virtual world, a three-dimensional simulation scene of a real scene is generated by using a computer, and the three-dimensional virtual reality scene technology is a system simulation of interactive three-dimensional dynamic vision and entity behaviors of multi-source information fusion. Virtual scenes include any real scene that exists in real life, including any scene that can be perceived through somatosensory, such as visual, auditory, etc., and are simulated by computer technology.

Augmented reality (augmented reality, AR): the technology is a technology which skillfully fuses virtual information with the real world, and a plurality of technical means such as multimedia, three-dimensional modeling, real-time tracking and registering, intelligent interaction, sensing and the like are widely applied, after the virtual information such as characters, images, three-dimensional models, music, videos and the like generated by a computer are simulated, the virtual information is applied to the real world, and the two kinds of information are mutually complemented, so that the enhancement of the real world is realized, and more abundant information than the real world can be obtained. The image of the real scene and the image of the virtual object can be subjected to virtual-real fusion through the augmented reality technology, and the two-dimensional image obtained after fusion contains both the content of the real scene and the virtual object.

Rendering: refers to the process of generating images from a model in software, where the model is a description of a three-dimensional object in a well-defined language or data structure, including geometry, viewpoint, texture, and illumination information. The image is a digital image or a bitmap image. Rendering this term is similar to "artist rendering of a scene," and in addition, rendering is also used to describe "the process of computing effects in a video editing file to generate a final video output.

The application scenario related to the embodiment of the present application is briefly described below.

The present application mainly relates to live scenes, see fig. 1, and fig. 1 is a schematic structural diagram of a live broadcast system related to the present application, a host may implement live broadcast through a live broadcast system 100, and a viewer may view content of the host live broadcast through the live broadcast system 100, and in addition, the host and the viewer may interact through the live broadcast system 100. As shown in fig. 1, live broadcast system 100 includes a terminal device 110, a network device 120, and a cloud computing platform 130.

The terminal device 110 may also be referred to as a mobile terminal or a user terminal, and may be an electronic device with a live application installed, such as a personal computer, a smart phone, a tablet computer, a notebook computer, a palm top computer, a mobile internet device (mobile internet device, MID), a wearable device (e.g., a smart watch, a smart bracelet, a pedometer, etc.), and the like, which are not particularly limited herein. Through the live application, the terminal device 110 may support live broadcasting by a host and viewing of live broadcasting by a viewer. Optionally, the terminal device 110 may further have various client applications, such as a shopping application, a search application, an audio playing application, and the like, which are not specifically limited herein.

For convenience of description and distinction, in fig. 1, a terminal device of a main cast is represented by a terminal device 110A, and a terminal device of a viewer is represented by a terminal device 110B. It should be understood that, in addition to the anchor terminal 110A supporting live broadcasting, the anchor may also be a spectator, who views live broadcasting of other anchors through the terminal 110A, and the spectator terminal 110B may also be a spectator, who views live broadcasting through the terminal 110B, in addition to supporting live broadcasting of spectators.

Network device 120 is configured to communicate data between terminal device 110 and cloud computing platform 130 via a communication network of any communication mechanism/communication standard. The communication network may be a wide area network, a local area network, a point-to-point connection, or any combination thereof.

The cloud computing platform 130 may be an independent server, a server cluster or a distributed system formed by a plurality of servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, a content distribution network, basic cloud computing services such as big data and an artificial intelligence platform. Taking the example that the cloud computing platform 130 is a cloud-based server in fig. 1, the cloud computing platform 130 may include a plurality of cloud computing nodes, where each cloud computing node includes, from bottom to top, hardware, a virtualization service, and a live application server. Wherein, the liquid crystal display device comprises a liquid crystal display device,

The hardware includes computing resources, storage resources, and network resources. The computing resources may employ heterogeneous computing architectures, for example, a central processing unit (central processing unit, CPU) +graphics processor (graphics processing unit, GPU) architecture, a cpu+ai chip, a cpu+gpu+ai chip architecture, and the like, which are not particularly limited herein. The storage resources may include memory and the like. Here, the computing resource may be divided into a plurality of computing unit resources, the storage resource may be divided into a plurality of storage unit resources, and the network resource may be divided into a plurality of network unit resources. Therefore, the image processing platform can be freely combined on the basis of unit resources according to the resource requirements of users, so that the users can provide the resources according to the requirements of the users. For example, the computing resource may be divided into computing unit resources of 5u and the storage resource may be divided into storage unit resources of 10G, and then the combination of computing resource and storage resource may be 5u+10g,5u+20g,5u+30u, …,10u+10g,10u+20g,10u+30u, ….

The virtualization service is a service for constructing resources of a plurality of physical hosts into a uniform resource pool through a virtualization technology, and flexibly isolating mutually independent resources according to the needs of users so as to run application programs of the users.

The virtualization service may include a Virtual Machine (VM) service, a bare metal (bare metal server, BMS) service, and a container (container) service. The VM service may be a service that virtualizes a Virtual Machine (VM) resource pool on a plurality of physical hosts through a virtualization technology to provide a user with a VM for use as needed. The BMS service is a service for virtualizing a BMS resource pool on a plurality of physical hosts to provide a user with BMS for use on demand. A container service is a service that virtualizes a pool of container resources on multiple physical hosts to provide users with containers for use on demand. A VM is a virtual computer that is modeled, i.e., a computer that is logically. The BMS is elastically telescopic high-performance computing service, has no difference between computing performance and traditional physical machines, and has the characteristic of safe physical isolation. The container is a kernel virtualization technology, and can provide lightweight virtualization to achieve the purpose of isolating user space, processes and resources. It should be understood that the VM service, the BMS service, and the container service in the above-mentioned virtualization service are merely specific examples, and the virtualization service may be other lightweight or heavy-weight virtualization services in practical applications, which are not specifically limited herein.

The live application server may be used to invoke hardware to implement live services, such as providing live video recording, injection, transcoding, etc. services for the anchor, and providing live video distribution services for the audience.

Specifically, the live broadcast application server may receive, through the network device 120, live broadcast video sent by the live broadcast application client on the anchor terminal device 110A, then transcode, store, and other services the live broadcast video, and when the live broadcast application server may also receive, through the network device 120, a live broadcast watching request sent by the viewer terminal device 110B, then find a corresponding live broadcast video according to the live broadcast watching request, and finally send the found live broadcast video to the viewer terminal device 110B through the network device 120, where the live broadcast video is displayed to the viewer by the live broadcast application client on the viewer terminal device 110B. It can be seen that the live application client is equivalent to an intermediary between the user (referred to as the host or viewer) and the live application server, which are collectively referred to as the rendering application.

In a specific implementation, the live application server and the live application client may be provided by a live application provider. For example, a live application developer installs a live application server on a cloud computing platform 130 provided by a cloud service provider, and the live application developer provides a live application client to a user for downloading via the internet and installs on the user's terminal device 110. Optionally, the live application server and the live application client may also be provided by a cloud service provider.

In a specific implementation, a cloud vendor may use a live broadcast service provided by the cloud computing platform 130 as a cloud service, after a live broadcast platform purchases the cloud service provided by the cloud vendor, a host who registers an account number in the live broadcast platform may use the cloud service to conduct live broadcast through the live broadcast platform, a viewer who registers the account number in the live broadcast platform may use the cloud service to watch live broadcast through the live broadcast platform, and the host and the viewer may use the cloud service to interact.

It should be understood that the live broadcast system 100 shown in fig. 1 is merely provided as a specific example, and in practical applications, the live broadcast system 100 may include any number of terminal devices 110, network devices 120, and cloud computing platforms 130, which are not specifically limited herein.

However, in the current live broadcast, the interaction mode between the host and the audience is relatively single, and in general, the host can only introduce real-life articles to the audience, and the audience can only interact with the host by sending a barrage, praying, commenting or giving a virtual gift, wherein the virtual gift is a two-dimensional image material. As shown in FIG. 2, audience A sent a barrage "hello-! "interact with the anchor Lisa, audience B sent a bullet screen" your live is very beautiful-! "interact with an anchor Lisa".

Moreover, when the audience interacts with the anchor by presenting the virtual gift, the virtual gift is simply and vividly superimposed on the existing live broadcast picture, and the display effect is abrupt.

With the increasing aesthetic level and entertainment demands of people, people generally expect better live interaction effects and diversified interactions in the live broadcast process, and the interaction manner is not ideal for the anchor and audience.

In order to solve the problems, the live interaction method, the live interaction device and the live interaction system can optimize live interaction effect, and provide various interaction modes for the anchor and the audience, so that interaction experience between the anchor and the audience is optimized.

The live interaction method, the live interaction device and the related equipment provided by the application are respectively described in detail below with reference to corresponding drawings.

Referring to fig. 3, fig. 3 is an interaction schematic diagram of a live interaction method provided in the present application, where the live interaction method may be applied to the live interaction system 100 shown in fig. 1, specifically, as shown in fig. 3, the live interaction method provided in the present application may be executed by the live interaction system 100 shown in fig. 1, and may include the following steps:

S301: the cloud computing platform 130 merges the image of the anchor and the virtual object to obtain a first direct-play picture.

In a specific embodiment of the present application, the virtual objects may be divided into virtual scenes and virtual props, where the virtual scenes may be used as scenes where the image of the host is located when the host is live broadcast, for example, virtual game scenes, virtual living scenes, or virtual working scenes; the virtual prop can be used as a prop for interaction between the host and audience when the host broadcasts live, such as a virtual chessboard, virtual chessman, virtual golf ball, virtual football, virtual ornament, etc.

In a specific embodiment of the present application, the cloud computing platform 130 may fuse the image of the anchor and the virtual object in any one of the following manners to obtain the first direct-play picture:

in mode 1, when the image of the anchor is a virtual three-dimensional character model, the cloud computing platform 130 may fuse the three-dimensional data of the anchor and the three-dimensional data of the virtual object to obtain fused three-dimensional data, and then process the fused three-dimensional data to obtain a two-dimensional first direct-play image.

From the above, the virtual object includes a virtual scene and a virtual prop, so that the three-dimensional data of the anchor and the three-dimensional data of the virtual object are fused, which can be understood as that the three-dimensional model corresponding to the image of the anchor and the three-dimensional model corresponding to the virtual prop are placed at the appropriate positions in the virtual scene.

For example, assuming that the virtual scene is a virtual room in which a chess game can be played, the virtual prop includes a virtual chessboard and virtual chess pieces, the image of the host, the virtual chessboard and the virtual chess pieces can be placed in the middle of the virtual room, wherein the image of the host is behind the virtual chessboard, and the virtual chess pieces are on the virtual chessboard, as shown in fig. 4.

In a specific implementation, the cloud computing platform 130 may perform rendering processing on the fused three-dimensional data to obtain a two-dimensional first direct-play image, and the rendering method may be a rasterization rendering method, a ray tracing rendering method, or a method of mixing the rasterization rendering method and the ray tracing rendering method, which is not limited herein specifically.

The process of rendering the first direct-broadcast picture by the cloud computing platform 130 will be described in detail below taking the example that the cloud computing platform 130 performs rendering by using a rasterization rendering method.

Referring to fig. 5, fig. 5 is a schematic diagram of a rasterization rendering method provided in the present application, and as shown in fig. 5, the rasterization rendering method provided in the present application generally includes an application stage, a geometry stage, and a rasterization stage. Wherein, the liquid crystal display device comprises a liquid crystal display device,

the application stage comprises the following steps: the method mainly comprises three tasks: (1) Preparing scene data such as a virtual scene and three-dimensional model information, illumination information and the like in the virtual scene; (2) To improve rendering performance, a coarse-grain culling (culling) operation is typically performed to cull out objects that are not visible in the scene, so that the objects do not need to be handed over to the geometry stage process; (3) The rendering state of each three-dimensional model, such as the material used, texture, etc., is set. The output of the application stage is the geometric information required by rendering, namely, the rendering primitive, wherein each rendering primitive contains all vertex data corresponding to the primitive, the rendering primitive can be a point, a line, a triangle and the like, and the rendering primitive is transmitted to the geometric stage.

Geometric stage: typically include multiple stages of vertex specification (vertex specification), vertex shader (vertex shader), primitive assembly, tessellation technique (tessellation), geometry shader (geometry shader), vertex post-processing (vertex post-processing), primitive assembly (primitive assembly), rasterization (rasterization), fragment shader (fragment shader), and pixel-by-pixel processing (per-sample operations), among others.

Vertex specifications are typically used to obtain vertex data. Wherein, the vertex data is generated according to the virtual scene and the three-dimensional model in the virtual scene, the vertex data comprises three-dimensional coordinates of the vertex, and the vertex data can also comprise normal vector of the vertex, color of the vertex and the like. Vertices may be points on a three-dimensional model, e.g., where two sides of a polygon in the three-dimensional model intersect, common endpoints of two sides in the three-dimensional model, and so forth.

Vertex shaders are typically used to transform the three-dimensional coordinates of vertices from model space (object space) to screen space (screen/image space). As shown in fig. 6, the process of transformation may be: the model space is transformed into world space (world space), the view space is transformed into view space (view space), the view space is transformed into nominal projection space (normalized projection space), and the nominal projection space is transformed into screen space. The visual space comprises a visual cone, wherein the space in the visual cone is a space which can be seen from the view angle of a user, and the space outside the visual cone is a space which cannot be seen from the view angle of the user.

Tessellation techniques are used to substantially increase the number of vertices in a three-dimensional model. As shown in fig. 7, it is assumed that the three-dimensional model includes three vertices constituting triangles, and that there are three vertices in the three-dimensional model before tessellation is performed, as shown on the left side of fig. 7. After tessellation, the number of vertices in the three-dimensional model changes from three to six, as shown on the right side of FIG. 7. It can be seen that the three-dimensional model appears rough and stiff before tessellation and appears realistic and vivid after tessellation.

The geometry shader is used to transform one or more vertices in the three-dimensional model into disparate primitives (primitives) to generate more vertices.

Vertex post-processing is used to clip (clip) primitives, i.e., if the primitive is partially outside the viewing cone and partially inside the viewing cone, then the portion of the primitive outside the viewing cone needs to be clipped, leaving only the portion inside the viewing cone.

Primitive assembly is typically used to assemble vertices in a three-dimensional model into geometric primitives, which will produce a series of triangles, line segments, and points. The assembled line segments may include individual line segments, line segments that are joined end-to-end but not eventually closed, line segments that are joined end-to-end and eventually sealed closed, and the like. The assembled triangles may include individual triangles, strings of linear continuous triangles, continuous triangles in a fan shape, and the like. At this stage, culling, i.e. removal of objects not visible from the scene, may also be performed. Here, culling may include visual cone culling (frame culling), visual mouth culling (occlusion culling).

The rasterization stage includes rasterization, fragment shader (fragment shader), and pixel-by-pixel processing (per-sample operation).

Rasterization is a process of converting vertex data into primitives, which has the effect of converting a graph into an image consisting of individual grids, and is characterized in that each element corresponds to a pixel in a frame buffer. Thus, the first part of the rasterization works: determining which integer grid areas in window coordinates are occupied by the graphic elements; the second part works: a color value and a depth value are assigned to each region. The rasterization process produces fragments (fragments) in which each point on the two-dimensional image contains color, depth, and texture data, which point and associated information is called a fragment.

The fragment shader is used to compute the final color output of the pixel.

The pixel-by-pixel processing includes depth testing and transparency processing. It will be appreciated that if we draw an object closer to it and then draw an object farther from it, then the object farther from it will cover the object nearer to it because of the later drawing, which is not desirable, whereas depth testing is actually recording the distance (drawing coordinates) of the pixel points from the camera in the 3D world, the greater the depth value (Z value) of each pixel point (drawn on the screen) stored in the depth buffer, the farther from the camera, so that after having depth buffered, the order of drawing objects is less important, and can be displayed normally in terms of distance (Z value).

In the above scheme, the processing sequence of the rasterization rendering method is as follows: vertex shaders, tessellation techniques, geometry shaders, post-vertex processing (including clipping), primitive assembly (including culling), rasterization, fragment shaders, and pixel-by-pixel processing are examples, and in practical applications, the processing order of the rasterization rendering method may vary and is not specifically limited herein.

Through the rendering operation, a two-dimensional first direct-broadcasting picture can be obtained, and in the obtained first direct-broadcasting picture, the image of the anchor and the three-dimensional display effect of the virtual object are good, and the pictures are natural and coordinated.

In mode 2, when the image of the anchor is a real person image, the cloud computing platform 130 may fuse the two-dimensional image of the virtual object and the two-dimensional image of the anchor, so as to obtain a two-dimensional first direct broadcast picture.

The two-dimensional image of the anchor may be a two-dimensional image of the anchor, which is obtained by the cloud computing platform 130 receiving live video including multiple frames of continuous images uploaded in real time by the anchor, and then performing person recognition on each frame of image, and after recognizing the anchor, matting out the anchor from each frame of image.

In a specific implementation, when each frame of image uploaded by the live host is received, the cloud computing platform 130 extracts the two-dimensional image of the live host from the image, and then performs virtual-real fusion on the extracted two-dimensional image of the live host and the two-dimensional image of the virtual object to obtain a two-dimensional first direct-broadcasting picture, wherein in the obtained first direct-broadcasting picture, the three-dimensional display effect of the image of the live host and the three-dimensional display effect of the virtual object are better, and the pictures are more natural and coordinated.

In a specific implementation, the cloud computing platform 130 may perform rendering processing on three-dimensional data of the virtual object to obtain a two-dimensional image of the virtual object, where the rendering method may be a rasterization rendering method, a ray tracing rendering method, or a method of mixing the rasterization rendering method and the ray tracing rendering method, which is not specifically limited herein; the cloud computing platform 130 may perform virtual-real fusion on the two-dimensional image of the live person and the two-dimensional image of the virtual object through the augmented reality technology, so as to obtain a first direct-play picture.

It can be seen that, in the first direct-broadcasting picture obtained in the above mode 1 or mode 2, the image of the anchor and the virtual prop are fused into a part of the virtual scene, and compared with the direct-broadcasting picture obtained by simply superposing two-dimensional images in the prior art, the three-dimensional display effect of the picture is better, and the picture is more natural and coordinated.

It should be understood that the above-listed two ways of fusing the image of the anchor with the virtual object to obtain the first direct-play picture are merely examples, and other ways of fusing the image of the anchor with the virtual object to obtain the first direct-play picture are also included in the scope of protection of the present application, which should not be considered as specific limitation herein.

S302: the cloud computing platform 130 transmits the first direct-cast picture to the viewer's terminal device 110B.

In a specific implementation, the live broadcast application server on the cloud computing platform 130 may send the first live broadcast picture to the live broadcast application client on the terminal device 110B of the audience through the network, and the live broadcast application client displays the first live broadcast picture to the audience.

S303: the cloud computing platform 130 sends the first direct-cast picture to the anchor's terminal device 110A.

In a specific implementation, the live broadcast application server on the cloud computing platform 130 may send the first live broadcast picture to the live broadcast application client on the anchor terminal device 110A through the network, and the live broadcast application client displays the first live broadcast picture to the anchor.

S304: the viewer's terminal device 110B displays the first direct-play picture.

S305: the anchor terminal device 110A displays a first direct-play screen.

S306: the viewer's terminal device 110B or the anchor's terminal device 110A receives a first operation of its corresponding user on the anchor's character or virtual object in the first direct-play screen.

Wherein the first operation is an operation for changing the image of the anchor in the first anchor screen, which is input to the terminal device held by the audience or anchor while watching the first anchor screen, or, an operation for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the virtual object in the first direct-play screen, or an operation for adding an avatar of the real object in the first direct-play screen is not particularly limited herein. In fig. 3, taking the first operation as an operation of the viewer and the terminal device receiving the first operation as the terminal device 110B of the viewer as an example, it should be understood that fig. 3 is only an example and should not be taken as a specific limitation.

When the first operation is an operation for changing the image of the anchor in the first on-air screen, the first operation may specifically be used to add an ornament (such as a hairpin, glasses) to the image of the anchor, change the hairstyle of the anchor (such as changing hair into curls), change the garment of the anchor (such as changing one dress into a western-style suit), or the like, and in a specific implementation, the ornament, hairstyle, or garment may be a virtual object originally included in the first on-air screen. When the character of the anchor is a virtual three-dimensional character model, the first operation may be used to change one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the character of the anchor, for example, the character of the anchor is located in a middle position of the screen in the first anchor screen, the first operation is an operation for moving the character of the anchor to a lower left corner of the first anchor screen, and for example, the character of the anchor is a standing posture in the first anchor screen, and the first operation is an operation for setting the character of the anchor in the first anchor screen.

When the first operation is an operation for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the virtual object in the first direct-play screen, the first operation may be specifically used for changing a position of the virtual object in the first direct-play screen, changing a speed at which the virtual object moves in the first direct-play screen, or changing a color of the virtual object displayed in the first direct-play screen, or the like, for example, assuming that the virtual object is a chess piece, the first operation is an operation for changing a position of the chess piece, and for example, assuming that the virtual object is a golf ball, the first operation is an operation for letting the golf ball fly up at a certain angle and speed.

In the case where the first operation is an operation for adding the avatar of the real object in the first direct-broadcasting picture, the first operation may be specifically used for adding the avatar of the real cap, the avatar of the real plant, the avatar of the real animal, etc. in the first direct-broadcasting picture, none of which is specifically limited herein.

In particular implementations, the viewer/anchor may input the first operation to the viewer's terminal device 110B/anchor's terminal device 110A through input components including, but not limited to: a keyboard, a mouse, a touch screen, a touch pad, an audio input device, etc., taking an input component as the touch screen as an example, the first operation may be a sliding operation performed by the viewer on the touch screen of the terminal device 110B of the viewer using a finger or a stylus, and the sliding operation may be used to change the position of the virtual object in the first direct broadcast picture.

S307: the terminal device 110B of the viewer or the terminal device 110A of the anchor receives the first operation, and processes the first operation as a first operation instruction for the anchor character or the virtual object in the first direct broadcast screen.

In fig. 3, the first operation is treated as a first operation instruction by the terminal device 110B of the viewer as an example, and it should be understood that fig. 3 is merely an example and should not be considered as being particularly limited.

Since the first operation instruction is obtained by processing the first operation, the first operation instruction corresponds to the first operation, and when the first operation is an operation for changing the avatar of the anchor in the first on-the-fly screen, the first operation instruction is an instruction for changing the avatar of the anchor in the first on-the-fly screen, and when the first operation is an operation for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the virtual object in the first on-the-fly screen, the first operation instruction is an instruction for changing one or more of a coordinate value, a moving speed, an acceleration, an offset angle, a moving direction, and a color of a point on the virtual object in the first on-the-fly screen, and when the first operation is an operation for adding the avatar of the real object in the first on-the-fly screen, the first operation instruction is an instruction for adding the avatar of the real object in the first on-the on-fly screen.

In a specific implementation, a viewer or a host may scan a real object in advance to obtain scan data of the real object, then input the scan data of the real object into a corresponding terminal device, when an image of the real object needs to be added in a first direct broadcast picture, a first operation performed by the viewer or the host may be to click on the scan data of the real object stored in the terminal device, and after receiving the first operation, the terminal device 110B of the viewer or the terminal device 110A of the host processes the first operation, so as to obtain a first operation instruction for adding the image of the real object in the first direct broadcast picture, where the first operation instruction may carry the scan data of the real object; alternatively, the spectator or the anchor may construct a three-dimensional model of the real object in advance, then input three-dimensional data of the real object (data indicating the three-dimensional model of the real object) to the corresponding terminal device, and when the image of the real object needs to be added in the first on-air screen, the first operation performed by the spectator or the anchor may be clicking on the three-dimensional data of the real object stored in the terminal device, and after receiving the first operation, the terminal device 110B of the spectator or the terminal device 110A of the anchor processes the first operation, so as to obtain a first operation instruction for adding the image of the real object in the first on-air screen, where the first operation instruction may carry the three-dimensional data of the real object.

S308: the terminal device of the audience terminal device 110B or the anchor terminal device 110A that obtains the first operation instruction sends the first operation instruction to the cloud computing platform 130.

In a specific implementation, the first operation instruction may be sent to the cloud computing platform 130 by the live application client on the terminal device 110B of the audience or the terminal device 110A of the anchor through the network. In fig. 3, taking the example that the terminal device 110B of the viewer transmits the first operation instruction to the cloud computing platform 130, it should be understood that fig. 3 is only an example and should not be considered as being particularly limited.

S309: the cloud computing platform 130 obtains a second live broadcast picture according to the first operation instruction and the first live broadcast picture.

Specifically, when the first operation instruction is used to change the image of the anchor or change one or more of coordinate values, moving speed, acceleration, offset angle, moving direction and color of a point on the virtual object, the cloud computing platform 130 may acquire physical information of the image of the anchor or the virtual object in the first direct broadcast picture affected by the first operation instruction, and then update the first direct broadcast picture according to the acquired physical information, so as to obtain a second direct broadcast picture; when the first operation instruction is used for adding the image of the real object, if the first operation instruction carries the scanning data of the real object, the cloud computing platform 130 may construct three-dimensional data of the real object based on the scanning data of the real object, then update the first direct broadcast picture according to the three-dimensional data of the real object, so as to obtain the second direct broadcast picture, and if the first operation instruction carries the three-dimensional data of the real object, the cloud computing platform 130 may directly update the first direct broadcast picture according to the three-dimensional data of the real object.

Wherein, the physical information of the image or the virtual object of the anchor after being influenced by the first operation instruction may include any one or more of the following: the image of the anchor or the coordinate value of the point on the virtual object, the speed of the point movement, the acceleration of the point movement, the offset angle of the point movement, the direction of the point movement, the color of the point, etc., after being affected by the first operation instruction. It should be understood that the above-listed physical information is by way of example only and should not be construed as being particularly limiting.

Taking the first operation instruction as an example for moving the position of the virtual chess piece in the first direct play picture, taking the physical information of the virtual object affected by the first operation instruction as the coordinate value of each point on the virtual chess piece after being moved, taking the first operation instruction as an example for enabling the virtual golf ball in the first direct play picture to fly at a certain offset angle and speed, taking the physical information of the virtual object affected by the first operation instruction as the offset angle and speed for enabling each point on the virtual golf ball to fly, taking the first operation instruction as an example for changing the color of the clothes of the image of the main player in the first direct play picture, and taking the physical information of the image of the main player affected by the first operation instruction as the color information of each point on the clothes of the image of the main player.

Referring to fig. 8A and 8B, which are exemplarily shown in the present application, fig. 8A is a first direct broadcast screen 800A, fig. 8B is a second direct broadcast screen 800B, and assuming that a first operation instruction is used to move the chess pieces 8001 in the first direct broadcast screen 800A from a first row of a second grid on the chessboard to a third row of a fifth grid on the chessboard, it can be seen that in the second direct broadcast screen 800B, the chess pieces 8001 have been moved to a position satisfying the user's requirement.

In a specific embodiment of the present application, the second live broadcast picture is obtained by updating the first live broadcast picture according to the obtained physical information, which may be the image or the virtual object of the anchor in the virtual scene corresponding to the first live broadcast picture is updated according to the obtained physical information, and then rendering is performed on the three-dimensional data of the virtual scene, the three-dimensional data of the virtual object in the virtual scene, and the image of the anchor, so as to obtain the two-dimensional second live broadcast picture. In a specific embodiment of the present application, the updating of the first live broadcast picture according to the three-dimensional data of the real object to obtain the second live broadcast picture may be performed by adding the three-dimensional data of the real object to the virtual scene corresponding to the first live broadcast picture, and then performing rendering processing on the three-dimensional data of the virtual scene, the three-dimensional data of the virtual object in the virtual scene, the image of the anchor in the virtual scene, and the three-dimensional data of the real object added to the virtual scene, so as to obtain the two-dimensional second live broadcast picture.

S310: the cloud computing platform 130 transmits the second live view to the terminal device 110B of the viewer.

S311: the cloud computing platform 130 sends the second live view to the anchor's terminal device 110A.

S312: the terminal device 110B of the viewer displays a second live view.

S313: the anchor terminal device 110A displays a second live view.

The execution of S302 to S303 may be performed in parallel or sequentially in any order, the execution of S304 to S305 may be performed in parallel or sequentially in any order, the execution of S310 to S311 may be performed in parallel or sequentially in any order, and the execution of S312 to S313 may be performed in parallel or sequentially in any order, which is not particularly limited herein.

In a specific embodiment of the present application, if the first operation is an operation performed by the audience, when the terminal device 110B of the audience and the terminal device 110A of the anchor display the second live broadcast picture, the terminal device 110A of the anchor may receive the second operation of the anchor on the image or the virtual object in the second live broadcast picture, and then the terminal device 110A of the anchor may process the second operation into a second operation instruction for the image or the virtual object of the anchor in the second live broadcast picture, and send the second operation instruction to the cloud computing platform 130, and the cloud computing platform 130 obtains a third live broadcast picture according to the second operation instruction and the second live broadcast picture, and after obtaining the third live broadcast picture, the cloud computing platform 130 may send the third live broadcast picture to the terminal device 110B of the audience and the terminal device 110A of the anchor for display.

If the first operation is an operation performed by the anchor, when the terminal device 110B of the audience and the terminal device 110A of the anchor display the second live broadcast picture, the terminal device 110B of the audience may receive the second operation of the audience on the anchor image or the virtual object in the second live broadcast picture, then the terminal device 110B of the audience may process the second operation into a second operation instruction for the anchor image or the virtual object in the second live broadcast picture, and send the second operation instruction to the cloud computing platform 130, the cloud computing platform 130 obtains a third live broadcast picture according to the second operation instruction and the second live broadcast picture, and after obtaining the third live broadcast picture, the cloud computing platform 130 may send the third live broadcast picture to the terminal device 110B of the audience and the terminal device 110A of the anchor for display.

The definition of the second operation is similar to that of the first operation in the embodiment shown in fig. 3, the definition of the second operation instruction is similar to that of the first operation instruction in the embodiment shown in fig. 3, the process of obtaining the third live view by the cloud computing platform 130 according to the second operation instruction and the second live view is similar to that of obtaining the second live view by the cloud computing platform 130 according to the first operation instruction and the first live view in step S309 in the embodiment shown in fig. 3, and for brevity of the description, please refer to the relevant contents in the embodiment shown in fig. 3.

It can be seen that in the live broadcast interaction method provided by the application, after the live broadcast and the audience see the live broadcast picture updated based on the operation of the other party, the live broadcast picture can be updated again by operating the image or the virtual object of the live broadcast in the updated live broadcast picture, that is, the live broadcast interaction method provided by the application can realize interaction of the live broadcast and the audience operating the image or the virtual object of the live broadcast in the live broadcast picture for a plurality of times.

In a possible embodiment, if the first operation is an operation performed by the anchor, the anchor's terminal device 110A may also obtain, when receiving the first operation of the anchor, limb motion information when the anchor performs the first operation, optionally, facial expression information when the anchor performs the first operation, and then, when sending a first operation instruction to the cloud computing platform 130, the anchor's terminal device 110A may also send, to the cloud computing platform 130, the limb motion information and the facial expression information when the anchor performs the first operation, and the cloud computing platform 130 obtains the second live broadcast picture according to the first operation instruction, the limb motion information and the facial expression information when the anchor performs the first operation, and the first live broadcast picture.

Similarly, if the second operation is an operation performed by the anchor, the anchor's terminal device 110A may also obtain the limb motion information when the anchor performs the second operation when receiving the second operation of the anchor, optionally may also obtain the facial expression information when the anchor performs the second operation, and then, when sending the second operation instruction to the cloud computing platform 130, the anchor's terminal device 110A may also send the limb motion information and the facial expression information when the anchor performs the second operation to the cloud computing platform 130, and the cloud computing platform 130 obtains the third live broadcast picture according to the second operation instruction, the limb motion information and the facial expression information when the anchor performs the second operation, and the second live broadcast picture.

Specifically, the cloud computing platform 130 may update the first direct broadcast picture according to the first operation instruction, the limb motion information and the facial expression information when the anchor performs the first operation, so as to obtain the second direct broadcast picture; the cloud computing platform 130 may update the second live broadcast picture according to the second operation instruction, the limb motion information and the facial expression information when the anchor performs the second operation, so as to obtain a third live broadcast picture; the process of updating the first direct broadcast picture by the cloud computing platform 130 according to the first operation instruction/the second operation instruction may refer to the related description in step S309, and the cloud computing platform 130 updates the first direct broadcast picture/the second direct broadcast picture according to the limb motion information and the facial expression information of the anchor, which may be understood that the limb motion and the facial expression of the anchor in the first direct broadcast picture/the second direct broadcast picture are driven to operate according to the limb motion information and the facial expression information of the anchor, and thus the limb motion and the facial expression of the anchor in the updated direct broadcast picture may be changed compared with the limb motion and the facial expression of the anchor in the direct broadcast picture before the update.

In a specific implementation, the terminal device 110A of the anchor may capture an image of the anchor through a camera, where the image includes limb motion information and facial expression information of the anchor, or when the anchor wears a wearing device capable of collecting the limb motion information of the anchor, the limb motion information of the anchor may also be collected through the wearing device, which is not limited herein specifically.

In a possible embodiment, the terminal device 110A of the anchor may also collect the limb motion information and the facial expression information of the anchor in real time, and then send the collected limb motion information and facial expression information of the anchor to the cloud computing platform 130 in real time, and the cloud computing platform 130 updates the limb motion information and facial expression information of the image of the anchor in the live broadcast picture in real time according to the limb motion information and the facial expression information of the anchor, which is not limited herein.

In a specific implementation, when the cloud computing platform 130 sends live broadcast pictures to the terminal device 110B of the audience and the terminal device 110A of the anchor, the live broadcast pictures may be transmitted by using a real-time streaming protocol (real time streaming protocol, RTSP), a web real-time communication (webreal-time communication, abbreviated as WebRTC) protocol, or other real-time transmission protocols.

In summary, the live broadcast interaction method provided by the application can realize that the audience or the anchor carries out various interaction operations on the image or the virtual object of the anchor in the live broadcast picture, and updates the live broadcast picture based on the interaction operations carried out by the audience or the anchor, that is, the live broadcast picture seen by the audience and the anchor can be changed along with various interaction operations carried out by the audience or the anchor by the method, so that various interaction modes can be provided for the anchor and the audience by the scheme, and interaction experience between the anchor and the audience is optimized.

It can be further seen that, in the live broadcast interaction method provided by the application, the image of the anchor and the virtual object are fused together to obtain the live broadcast picture, and when the cloud computing platform 130 updates the live broadcast picture according to the operation instruction, the image of the anchor and the virtual object in the updated live broadcast picture are also fused together, and compared with the live broadcast picture obtained by simply superposing two-dimensional images in the prior art, the stereoscopic display effect of the live broadcast picture is better, and the displayed content is more natural and coordinated.

The live broadcast interaction method provided by the application is explained in detail above, and based on the same inventive concept, the live broadcast interaction device provided by the application is introduced next.

The present application provides two live broadcast interaction devices, one of which can be applied to a terminal device 110 shown in fig. 1, such as 110A and 110B, and the other of which can be applied to a cloud computing platform 130 shown in fig. 1, and when the live broadcast interaction device provided by the present application is applied to the terminal device 110 and/or applied to the cloud computing platform 130, the live broadcast system 100 shown in fig. 1 can be enabled to optimize a live broadcast interaction effect between a host and a viewer, provide various interaction modes for the host and the viewer, and optimize an interaction experience between the host and the viewer. It should be understood that the unit modules in the live broadcast interaction device provided by the application can have multiple divisions, and each module can be a software module, a hardware module, or a software module and a hardware module, which are not limited in the application.

Referring to fig. 9, fig. 9 is a schematic structural diagram of a live interaction device 900 applied to a terminal device 110, where the device 900 is shown in an exemplary manner, and the device 900 includes: a display module 910, a receiving module 920, a processing module 930, and a sending module 940.

The display module 910 is configured to display a first direct-play screen, where the first direct-play screen includes an image of a host and a virtual object.

The receiving module 920 is configured to receive a first operation on an avatar or a virtual object of a host in a first direct broadcast screen.

The processing module 930 is configured to process the first operation as a first operation instruction for an image or a virtual object of a host in the first direct broadcast screen.

The sending module 940 is configured to send the first operation instruction to the cloud computing platform 130, and the cloud computing platform 130 obtains a second live broadcast picture according to the first operation instruction and the first live broadcast picture.

The receiving module 920 is configured to receive the second live broadcast picture sent by the cloud computing platform 130.

And a display module 910, configured to display a second live broadcast picture.

In one possible implementation, the receiving module 920 is configured to receive a second operation on the image or the virtual object of the anchor in the first live view, the processing module 930 is configured to process the second operation into a second operation instruction on the image or the virtual object of the anchor in the second live view, the sending module 940 is configured to send the second operation instruction to the cloud computing platform 130, the cloud computing platform 130 obtains a third live view according to the second operation instruction and the second live view, the receiving module 920 is configured to receive the third live view sent by the cloud computing platform 130, and the display module 910 is configured to display the third live view.

In one possible implementation, the first operation is an operation of a spectator, the second operation is an operation of a anchor, or the first operation is an operation of an anchor, and the second operation is an operation of a spectator. It should be understood that when the first operation is an operation by the audience, the terminal device 110 to which the apparatus 900 may be applied is the terminal device 110B of the audience, and when the first operation is an operation by the anchor, the terminal device 110 to which the apparatus 900 may be applied is the terminal device 110A of the anchor.

In one possible implementation, the anchor is a real or virtual person.

In one possible implementation manner, the first direct broadcast picture is a two-dimensional direct broadcast picture obtained by fusing three-dimensional data of a host and three-dimensional data of a virtual object and processing the fused three-dimensional data.

In one possible implementation, the first operating instructions are for adding decorations, changing hairstyles or changing clothing to the image of the anchor.

Referring to fig. 10, fig. 10 is a schematic structural diagram of a live interaction device 1000 applied to a cloud computing platform 130, where the device 1000 includes: a processing module 1010, a transmitting module 1020, and a receiving module 1030.

And a processing module 1010, configured to fuse the image of the anchor and the virtual object to obtain a first direct-play picture.

A transmitting module 1020, configured to transmit the first direct broadcast picture to the terminal device 110B of the audience and the terminal device 110A of the anchor.

The receiving module 1030 is configured to receive a first operation instruction for an avatar or a virtual object of a host in a first direct broadcast screen.

And the processing module 1010 is configured to obtain a second live broadcast picture according to the first operation instruction and the first live broadcast picture.

A sending module 1020, configured to send the second live broadcast picture to the terminal device 110B and the anchor terminal device 110A.

In one possible implementation, the receiving module 1030 is configured to receive a second operation instruction for a avatar or a virtual object of the anchor in the second live broadcast picture; the processing module 1010 is configured to obtain a third live broadcast picture according to the second operation instruction and the second live broadcast picture; a sending module 1020, configured to send the third live broadcast picture to the terminal device 110B and the anchor terminal device 110A.

In one possible implementation manner, the first operation instruction is an instruction obtained by processing the first operation, and the second operation instruction is an instruction obtained by processing the second operation; wherein the first operation is an operation of a spectator on a live image or a virtual object in a first live picture, and the second operation is an operation of a live image or a virtual object in a second live picture; alternatively, the first operation is an operation of the anchor on the anchor character or virtual object in the first live view, and the second operation is an operation of the viewer on the anchor character or virtual object in the second live view.

In one possible implementation, the anchor is a real or virtual person.

In one possible implementation manner, the processing module 1010 may specifically implement fusing the image of the anchor and the virtual object to obtain the first direct-play picture in the following manner: and after fusing the three-dimensional data of the anchor and the three-dimensional data of the virtual object, processing the fused three-dimensional data to obtain a two-dimensional first direct broadcasting picture.

In one possible implementation manner, the processing module 1010 may specifically implement fusing the image of the anchor and the virtual object to obtain the first direct-play picture in the following manner: firstly, acquiring a two-dimensional image of a host, and then fusing the two-dimensional image of the host and the two-dimensional image of a virtual object to obtain a two-dimensional first direct broadcasting picture.

Specifically, the specific implementation of the foregoing live interaction apparatus 900 to perform various operations may refer to the relevant descriptions in the steps performed by the anchor terminal device 110A or the viewer terminal device 110B in the foregoing live interaction method embodiment, and the specific implementation of the foregoing live interaction apparatus 1000 to perform various operations may refer to the relevant descriptions in the steps performed by the cloud computing platform 130 in the foregoing live interaction method embodiment, which are not repeated herein for brevity of the description.

In summary, the live broadcast interaction device (such as the device 900 shown in fig. 9 and the device 1000 shown in fig. 10) provided by the present application can implement various interaction operations of the audience or the host on the image or the virtual object of the host in the live broadcast image, and update the live broadcast image based on the interaction operations performed by the audience or the host, that is, by the method, the live broadcast image seen by the audience and the host becomes diversified along with the various interaction operations performed by the audience or the host, so that the scheme can implement various interaction modes provided for the host and the audience, and optimize the interaction experience between the host and the audience.

In addition, the live broadcast interaction device 1000 provided by the application fuses the image and the virtual object of the live broadcast together to obtain the live broadcast picture, and when the live broadcast interaction device 1000 updates the live broadcast picture according to the operation instruction, the image and the virtual object of the live broadcast in the updated live broadcast picture are fused together, compared with the live broadcast picture obtained by simply superposing two-dimensional images in the prior art, the three-dimensional display effect of the live broadcast picture is better, and the displayed content is more natural and coordinated.

The present application further provides a terminal device 110, referring to fig. 11, fig. 11 is a schematic structural diagram of the terminal device 110 provided in the present application, as shown in fig. 11, where the terminal device 110 includes: processor 1110, memory 1120, and communication interface 1130, wherein processor 1110, memory 1120, and communication interface 1130 may be interconnected by bus 1140.

The processor 1110 may read the program code (including instructions) stored in the memory 1120, and execute the program code stored in the memory 1120, so that the terminal device 110 performs the steps performed by the anchor terminal device 110A or the viewer terminal device 110B in the live interaction method provided in the present application shown in fig. 3, or causes the terminal device 110 to deploy the live interaction apparatus 900.

The processor 1110 may have a variety of specific implementations, such as a CPU, or a combination of a CPU and hardware chips. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (programmable logic device, PLD), or a combination thereof. The PLD may be a complex programmable logic device (complex programmable logic device, CPLD), a field-programmable gate array (field-programmable gate array, FPGA), general-purpose array logic (generic array logic, GAL), or any combination thereof. Processor 1110 executes various types of digitally stored instructions, such as software or firmware programs stored in memory 1120, which enable terminal device 110 to provide a variety of services.

Memory 1120 is used for storing program codes and is controlled to be executed by processor 1110. The program code may comprise one or more software modules, which may be the software modules provided in the embodiment of fig. 9, such as the display module 910, the receiving module 920, the processing module 930, and the transmitting module 940.

Memory 1120 may include volatile memory (RAM), such as random access memory (random access memory); the memory 1120 may also include a non-volatile memory (ROM), such as a read-only memory (ROM), a flash memory (flash memory), a Hard Disk Drive (HDD), or a Solid State Drive (SSD); the memory 1120 may also include a combination of the above.

The communication interface 1130 may be a wired interface (e.g., an ethernet interface, a fiber optic interface, other types of interfaces (e.g., infiniBand interface)) or a wireless interface (e.g., a cellular network interface or using a wireless local area network interface) for communicating with other computing devices or apparatus. The communication interface 1130 may employ a family of protocols over the transmission control protocol/internet protocol (transmission control protocol/internet protocol, TCP/IP), such as the remote function call (remote function call, RFC) protocol, the simple object access protocol (simple object access protocol, SOAP) protocol, the simple network management protocol (simple network management protocol, SNMP) protocol, the common object request broker architecture (common object request broker architecture, CORBA) protocol, and the distributed protocol, among others.

The bus 1140 may be a peripheral component interconnect express (peripheral component interconnect express, PCIe) bus, an extended industry standard architecture (extended industry standard architecture, EISA) bus, a unified bus (Ubus or UB), a computer quick link (compute express link, CXL), a cache coherent interconnect protocol (cache coherent interconnect for accelerators, CCIX), or the like. The bus 1140 may be divided into an address bus, a data bus, a control bus, and the like. The bus 1140 may include a power bus, a control bus, a status signal bus, and the like in addition to a data bus. But for clarity of illustration, the various buses are labeled as bus 1140 in the figures. For ease of illustration, only one thick line is shown in FIG. 11, but not only one bus or one type of bus.

The specific implementation process of the above terminal device 110 for executing the steps executed by the terminal device 110A or the terminal device 110B in the live broadcast interaction method provided in the present application shown in fig. 3 is described in the above method embodiment, and will not be described herein again.

It should be understood that terminal device 110 is only one example provided for embodiments of the present application, and that terminal device 110 may have more or fewer components than shown in fig. 11, may combine two or more components, or may have different configuration implementations of the components.

The present application also provides a cloud computing platform 130, where the cloud computing platform 130 may be implemented by a computing device cluster arrangement including one or more computing devices 1200 shown in fig. 12 to perform the steps performed by the cloud computing platform 130 in the live interaction method shown in fig. 3.

Specifically, in the case where the cloud computing platform 130 includes only one computing device 1200, all modules in the live interaction apparatus 1000 shown in fig. 10 may be deployed in the one computing device 1200: a processing module 1010, a transmitting module 1020, and a receiving module 1030.

Where the cloud computing platform 130 includes a plurality of computing devices 1200, each computing device 1200 of the plurality of computing devices 1200 may be used to deploy a portion of the modules in the live interaction apparatus 1000 shown in fig. 10, or two or more computing devices 1200 of the plurality of computing devices 1200 may be used together to deploy one or more modules in the live interaction apparatus 1000 shown in fig. 10.

Taking the example that the cloud computing platform 130 includes two computing devices 1200A and 1200B, where the two computing devices are used to deploy the live interaction apparatus 1000 shown in fig. 10, the computing device 1200A may be used to deploy the processing module 1010, the computing device 1200B may be used to deploy the transmitting module 1020 and the receiving module 1030, or the computing device 1200A is used to deploy the transmitting module 1020 and the receiving module 1030, and the computing device 1200A is used to co-deploy the processing module 1010 with the computing device 1200B.

Specifically, each computing device 1200 in the cloud computing platform 130 may include a processor 1210, a memory 1220, and a communication interface 1230, where the memory 1220 in one or more computing devices 1200 in the cloud computing platform 130 may store the same code (may also be referred to as an instruction or a program instruction, etc.) corresponding to the method executed by the cloud computing platform 130 in the live interaction method provided in the application, and the processor 1210 may read the code from the memory 1220 and execute the code to implement the method executed by the cloud computing platform 130 in the live interaction method provided in the application, and the communication interface 1230 may be used to implement communication between each computing device 1200 and other devices.

In some possible implementations, each computing device 1200 in the cloud computing platform 130 may also communicate with other device connections over a network. Wherein the network may be a wide area network or a local area network, etc.

Taking the example that all the modules of the live interaction device 1000 are deployed on one computing device 1200, the computing device 1200 provided in the present application is described in detail with reference to fig. 13.

Referring to fig. 13, a computing device 1200 includes: processor 1210, memory 1220, and communication interface 1230, wherein processor 1210, memory 1220, and communication interface 1230 may be interconnected by bus 1240. Wherein, the liquid crystal display device comprises a liquid crystal display device,

The processor 1210 may read the program code stored in the memory 1220, and execute the program code stored in the memory 1220, so that the cloud computing platform 130 performs the steps performed by the cloud computing platform 130 in the live interaction method provided by the present application shown in fig. 3, or the computing device 1200 deploys the live interaction apparatus 1000.

Processor 1210 may have various specific implementations, for example, processor 1210 may be a CPU or GPU, and processor 1210 may also be a single-core processor or a multi-core processor. Processor 1210 may be formed from a combination of a CPU and hardware chips. The hardware chip may be an ASIC, a PLD, or a combination thereof. The PLD may be CPLD, FPGA, GAL or any combination thereof. Processor 1210 may also be implemented as a separate logic device, such as an FPGA or DSP, using built-in processing logic.

The memory 1220 is used for storing program codes and is controlled to be executed by the processor 1210. The program code may comprise software modules provided in the embodiment of fig. 10: a processing module 1010, a transmitting module 1020, and a receiving module 1030.

In practical applications, the memory 1220 may be a nonvolatile memory, such as ROM, PROM, EPROM, EEPROM or flash memory. Memory 1220 may also be volatile memory, which may be RAM, which acts as external cache.

The communication interface 1230 may be a wired interface (e.g., an ethernet interface) or a wireless interface (e.g., a cellular network interface or using a wireless local area network interface) for communicating with other computing nodes or devices. When the communication interface 1230 is a wired interface, the communication interface 1230 may employ a protocol family over TCP/IP, such as RFC protocol, SOAP protocol, SNMP protocol, CORBA protocol, distributed protocol, and the like.

Bus 1240 may be a PCIe bus, EISA bus, UB, CXL, CCIX, etc. The bus 1240 may be classified into an address bus, a data bus, a control bus, and the like. The bus 1240 may include a power bus, a control bus, a status signal bus, and the like in addition to the data bus. But for clarity of illustration, the various buses are labeled as bus 1240 in the drawing. For ease of illustration, only one thick line is shown in fig. 13, but not only one bus or one type of bus.

The above-mentioned computing device 1200 is used for executing the steps executed by the cloud computing platform 130 in the live interaction method provided in the present application shown in fig. 3, and detailed implementation procedures of the steps are described in the above-mentioned method embodiment, which is not repeated here.

It should be understood that computing device 1200 is only one example provided herein, and that computing device 1200 may have more or fewer components than shown in fig. 13, may combine two or more components, or may have different configuration implementations of the components.

The present application further provides a live broadcast system, where the live broadcast system may include the live broadcast interaction device 900 shown in fig. 9 and the live broadcast interaction device 1000 shown in fig. 10, or include the terminal device 110 shown in fig. 11 and the cloud computing platform 130 shown in fig. 12.

The present application also provides a computer readable storage medium, where instructions are stored, where the instructions can implement some or all of the steps of the live interaction method described in the foregoing embodiments when executed.

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to the related descriptions of other embodiments.

In the above embodiments, it may be implemented in whole or in part by software, hardware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium, etc.

The foregoing is merely a specific embodiment of the present application. Variations and alternatives will occur to those skilled in the art from the detailed description provided herein and are intended to be included within the scope of the present application.

Claims

1. A live interaction method, characterized by being applied to a cloud computing platform, the method comprising:

fusing the image of the anchor and the virtual object to obtain a first direct-broadcasting picture;

transmitting the first direct broadcast picture to terminal equipment of a spectator and terminal equipment of the anchor;

receiving a first operation instruction of a host image or a virtual object in the first direct broadcast picture;

obtaining a second live broadcast picture according to the first operation instruction and the first live broadcast picture;

and sending the second live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor.

2. The method according to claim 1, wherein the method further comprises:

receiving a second operation instruction of a host image or a virtual object in the second live broadcast picture;

obtaining a third live broadcast picture according to the second operation instruction and the second live broadcast picture;

and sending the third live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor.

3. The method of claim 2, wherein the first operation instruction is an instruction that is a result of processing a first operation, and the second operation instruction is an instruction that is a result of processing a second operation;

wherein the first operation is an operation of the audience on the avatar or virtual object of the anchor in the first live broadcast picture, and the second operation is an operation of the anchor on the avatar or virtual object of the anchor in the second live broadcast picture; alternatively, the first operation is an operation of the anchor on an anchor character or virtual object in the first live view, and the second operation is an operation of the viewer on an anchor character or virtual object in the second live view.

4. A method according to any one of claims 1 to 3, wherein the anchor is a real person or a virtual person.

5. The method according to any one of claims 1 to 4, wherein the fusing the avatar and the virtual object of the anchor to obtain the first direct-play picture includes:

and after fusing the three-dimensional data of the anchor and the three-dimensional data of the virtual object, processing the fused three-dimensional data to obtain a two-dimensional first direct broadcasting picture.

6. The method according to any one of claims 1 to 4, wherein the fusing the avatar and the virtual object of the anchor to obtain the first direct-play picture includes:

acquiring a two-dimensional image of the anchor;

and fusing the two-dimensional image of the anchor and the two-dimensional image of the virtual object to obtain a two-dimensional first direct broadcasting picture.

7. The method of any one of claims 1 to 6, wherein the first operating instruction is for changing the avatar of the anchor; or, the first operation instruction is used for changing one or more of coordinate values, moving speed, acceleration, offset angle, moving direction and color of the point on the virtual object; or the first operation instruction is used for adding the image of the real object in the first direct broadcast picture.

8. The method of claim 7, wherein the first operating instructions are for adding decorations, changing hairstyles, or changing clothing to the character of the anchor.

9. The method of any one of claims 1 to 8, wherein the virtual object is used to enable an interactive game between the spectator and the host.

10. A live interaction method, applied to a terminal device of a host or a terminal device of a viewer, the method comprising:

displaying a first direct-play picture, wherein the first direct-play picture comprises the image of the anchor and a virtual object;

and displaying a second live broadcast picture, wherein the second live broadcast picture is obtained according to a first operation instruction of a main broadcasting image or a virtual object in the first live broadcast picture and the first live broadcast picture.

11. The method according to claim 10, wherein the method further comprises:

and displaying a third live broadcast picture, wherein the third live broadcast picture is obtained according to a second operation instruction of the image or the virtual object of the host broadcast in the second live broadcast picture and the second live broadcast picture.

12. The method of claim 11, wherein the first operation instruction is an instruction that is a result of processing a first operation, and the second operation instruction is an instruction that is a result of processing a second operation;

13. The method according to any one of claims 10 to 12, wherein the anchor is a real person or a virtual person.

14. The method according to any one of claims 10 to 13, wherein the first direct broadcast picture is a two-dimensional picture obtained by fusing three-dimensional data of the anchor and three-dimensional data of the virtual object and processing the fused three-dimensional data.

15. The method according to any one of claims 10 to 13, wherein the first on-air screen is obtained by fusing a two-dimensional image of the anchor and a two-dimensional image of the virtual object.

16. The method of any one of claims 10 to 15, wherein the first operating instruction is for changing the avatar of the anchor; or, the first operation instruction is used for changing one or more of coordinate values, moving speed, acceleration, offset angle, moving direction and color of the point on the virtual object; or the first operation instruction is used for adding the image of the real object in the first direct broadcast picture.

17. The method of claim 16, wherein the first operating instructions are for adding decorations, changing hairstyles, or changing clothing to the character of the anchor.

18. The method of any one of claims 10 to 17, wherein the virtual object is used to enable an interactive game between the spectator and the host.

19. The live broadcast interaction method is characterized by being applied to a live broadcast system, wherein the live broadcast system comprises a cloud computing platform, terminal equipment of audiences and terminal equipment of anchor, and the method comprises the following steps:

the cloud computing platform fuses the image of the anchor and the virtual object to obtain a first direct-broadcasting picture;

the cloud computing platform sends the first direct-broadcasting picture to the terminal equipment of the audience and the terminal equipment of the anchor;

the terminal equipment of the audience and the terminal equipment of the anchor display the first direct-broadcasting picture;

the terminal equipment of the audience or the terminal equipment of the anchor sends a first operation instruction for the image or the virtual object of the anchor in the first direct broadcasting picture to the cloud computing platform;

the cloud computing platform obtains a second live broadcast picture according to the first operation instruction and the first live broadcast picture;

the cloud computing platform sends the second live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor;

And the terminal equipment of the audience and the terminal equipment of the anchor display the second live broadcast picture.

20. A live interaction device, characterized by being applied to a cloud computing platform, the device comprising:

the processing module is used for fusing the image of the anchor and the virtual object to obtain a first direct-broadcasting picture;

the sending module is used for sending the first direct-broadcasting picture to the terminal equipment of the audience and the terminal equipment of the anchor;

the receiving module is used for receiving a first operation instruction of the image or the virtual object of the anchor in the first direct broadcasting picture;

the processing module is used for obtaining a second live broadcast picture according to the first operation instruction and the first live broadcast picture;

and the sending module is used for sending the second live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor.

21. A live interaction device, applied to a terminal device of a host or a terminal device of a viewer, the device comprising:

the display module is used for displaying a first direct-play picture, wherein the first direct-play picture comprises the image of the anchor and a virtual object;

the display module is used for displaying a second live broadcast picture, wherein the second live broadcast picture is obtained according to a first operation instruction of a main broadcasting image or a virtual object in the first live broadcast picture and the first live broadcast picture.

22. A cloud computing platform, the cloud computing platform comprising one or more computing devices, each computing device comprising a processor and a memory; the processor of the one or more computing devices is configured to execute instructions stored in the memory of the one or more computing devices, causing the one or more computing devices to perform the steps of:

23. A terminal device, the device comprising a memory and a processor; the memory is configured to store instructions, and the processor is configured to execute the instructions stored in the memory, so that the terminal device performs the following steps:

displaying a first direct-play picture, wherein the first direct-play picture comprises an image of a host player and a virtual object;

24. A live broadcast system, the live broadcast system comprising: the system comprises a cloud computing platform, terminal equipment of audiences and terminal equipment of anchor;

the cloud computing platform is used for fusing the image of the anchor and the virtual object to obtain a first direct-broadcasting picture;

the cloud computing platform is used for sending the first direct-broadcasting picture to the terminal equipment of the audience and the terminal equipment of the anchor;

the terminal equipment of the audience and the terminal equipment of the anchor are used for displaying the first direct-broadcasting picture;

the terminal equipment of the audience or the terminal equipment of the anchor is used for sending a first operation instruction for the image or the virtual object of the anchor in the first direct broadcasting picture to the cloud computing platform;

the cloud computing platform is used for obtaining a second live broadcast picture according to the first operation instruction and the first live broadcast picture;

the cloud computing platform is used for sending the second live broadcast picture to the terminal equipment of the audience and the terminal equipment of the anchor;

And the terminal equipment of the audience and the terminal equipment of the anchor are used for displaying the second live broadcast picture.

25. A computer readable storage medium comprising instructions for implementing the method of any one of claims 1 to 19.