WO2019057194A1

WO2019057194A1 - Linked microphone-based live streaming method and system

Info

Publication number: WO2019057194A1
Application number: PCT/CN2018/107278
Authority: WO
Inventors: 伏英娜; 金宇林
Original assignee: 迈吉客科技（北京）有限公司
Priority date: 2017-09-25
Filing date: 2018-09-25
Publication date: 2019-03-28
Also published as: CN107750014A; CN107750014B

Abstract

Provided are a linked microphone-based live streaming method and system, which solve the problem whereby the amount of interaction information cannot be effectively controlled due to the lack of an optimization means for an interaction object. The method comprises: in the process of establishing linked microphone-based live streaming, respectively setting a local real object at a host end and a guest end, wherein at least one of the host end and the guest end is provided with a local virtual object, which synchronously changes with the local real object; in the process of linked microphone-based live streaming, at least one of the host end and the guest end transporting the local virtual object, instead of the local real object, to a host scene, and the transported local virtual object receiving control data of the corresponding local real object to form a synchronous change; and converting an interactive change process formed in the host scene into streaming media, and broadcasting same live to an audience. By converting a video stream with a large data amount for a real object into action expression parameters, to be transmitted in real time, with a small data amount, and a virtual object, to be transmitted in quasi-real time, of an interaction host, the high-quality bandwidth pressure on a real-time data transmission system is relieved.

Description

Continuous wheat live method and system

The present application claims the priority of the Chinese application filed on Sep. 25, 2017, which is hereby incorporated by reference.

Technical field

The present invention relates to the field of software communication technologies, and in particular, to a live wheat live broadcast method and system.

Background of the invention

The network social platform built on the Internet technology has gradually formed a kind of continuous live broadcast platform with the development of technology. The live broadcast platform needs to establish real-time information interaction between the anchor end, the guest end and the audience end. FIG. 1 is a schematic diagram of a data transmission architecture in a prior art live broadcast technology. As shown in FIG. 1 , the server-side live streaming mode and the client live streaming mode using the real-time network transmission system and the distributed content network are included, and the information distribution for the anchor scene can be adopted by a CDN network (Content Delivery Network). The information is cached synchronously to the CDN network node close to the viewer, ensuring that the information is published in real time for a large audience. For the interaction between the anchor and the guest in the scene of the anchor, the real-time network transmission system, such as the SD-RTN network (Software-Defined RealTime Net work software defines the real-time transmission network), realizes the fast routing and real-time transmission of the interaction data between the anchor end and the guest end, Ensure the real-time quality of the interaction (ie QoS quality of service level). Generally, the anchor end initiates an interaction process with the guest end. In the interaction process, the interaction information between the anchor end and the guest end is formed by the general audio and video encoder to form an information encapsulation and transmitting party, and the other party forms a message synthesis through the universal audio and video decoder, at the anchor end or guest. The end forms an interactive information presentation scene. The presentation scene of the anchor side forms a standard media stream to be transmitted to the streaming server in real time. Usually, the streaming server is set in the CDN network and distributed to the viewer through the streaming server. The main real-time transmission technology scheme adopted by the interaction process between the anchor end and the guest end in the prior art also includes a multi-channel technical solution based on the RTMP protocol (RealTime Message Protocol) and WebRTC (Web Real-Time Communication). P2P technology solution for webpage real-time communication), the media stream can be transmitted to the viewer based on RTMP or HLS (HTTP Live Streaming), the stream format can be FLV (FLASH VIDEO), etc. The audio and video codec in the interactive scene can be H.26x, etc. Coding technology. The prior art can basically satisfy the real-time transmission control of the audio and video interaction information of all parties in the live broadcast in a simple scene with limited resolution.

The data bandwidth consumed by the Lianmai live broadcast platform belongs to a high quality of service level. Because the interactive objects in the live broadcast scenario lack effective technical means to achieve interactive information fusion and optimization, the amount of interactive audio and video data between the anchor end and the guest end may be transmitted in real time. Sexual effects. As the video clarity of the live broadcast between the anchor end and the guest end is developed to a high resolution, the network bandwidth on the planned route is continuously and rapidly consumed, and the data burst traffic at the edge of the backbone network is increased, resulting in anchors, guests and viewers. The real-time bandwidth control method gradually fails, which greatly affects the smoothness of the live broadcast process.

Due to the limitation of real-time bandwidth quality, the type, number and resolution of interactive objects in the live broadcast scene are also affected. It is difficult to uniformly schedule and consume only limited resources in a layered overlay manner, resulting in a between the anchor and the guest. The interaction mode is rigid, and there is more redundant information and invalid information.

Summary of the invention

In view of this, the embodiment of the present invention provides a method and system for continuously broadcasting a live wheat, which is used to solve the technical problem that the interactive object lacks an interactive object in the interaction scenario of the continuous live broadcast platform, and the amount of interactive information cannot be effectively controlled.

The Lianmai live broadcast method of the present invention is applied to the interactive process of Lianmai live broadcast, including:

In the process of establishing the live broadcast of the connected wheat, the local real object is set on the anchor end and the guest end respectively, and at least one end of the anchor end and the guest end is configured to set a local end that changes synchronously with the real object of the local end. Virtual object

In the process of the live broadcast, at least one end of the anchor end and the guest end transfers the local virtual object to the local real scene instead of the local real object, and the transmitted the original Receiving, by the end virtual object, the action control data of the corresponding real object of the local end to form a synchronous change;

Converting the interaction change process formed in the anchor scene into streaming media and broadcasting to the viewer.

In an embodiment of the present invention, in the process of establishing the live broadcast of the continuous wheat, the local real object is respectively set on the anchor end and the guest end, and at least one end of the anchor end and the guest end is set and The local virtual object whose local real object synchronizes changes includes:

Establishing a coordinate space on the anchor end;

Establishing the local real object in the coordinate space to form a control reference point;

Obtaining a virtual object and a corresponding model control point to form the local virtual object;

Establishing a mapping rule between the model control point and the control reference point;

The change of the control reference point is collected, and the action control data is formed according to the mapping rule.

Establishing a coordinate space associated with the anchor end at the guest end;

Collecting changes of the control reference point, and forming action control data of the local virtual object according to a mapping rule.

In an embodiment of the present invention, in the process of the live broadcast, at least one end of the anchor end and the guest end transfers the local virtual object to the host real object instead of the local real object. In the scenario, the synchronized change of the action control data of the local virtual object that is transmitted by the local virtual object is:

The anchor end uses the local virtual object to replace the local real object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

The anchor end sends the real object of the local end to the guest end;

The anchor end receives the real object of the guest end;

The anchor end interacts with the local virtual object in the anchor scene and the real object in the guest end.

The anchor end uses the local real object in the anchor scene;

Sending, by the anchor end, the real object of the local end to the guest end;

The anchor end receives the virtual object of the guest end;

The anchor end interacts with the local real object in the anchor scene and the virtual object of the guest end.

Sending, by the anchor end, the local virtual object to the guest end;

The anchor end receives the virtual object of the guest end;

The anchor end interacts with the local virtual object in the anchor scene and the virtual object of the guest end.

The anchor end adopts the local real object in the anchor scene;

Sending, by the anchor end, the local virtual object to a plurality of the guest terminals;

The anchor end receives a real object of each of the guest terminals;

The anchor end interacts with the real object of the local end and the real object of the guest end in the anchor scene.

The anchor end adopts the local virtual object in the anchor scene;

The anchor end receives a virtual object of each of the guest ends;

In an embodiment of the present invention, the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

On the physical link between the anchor end and the guest end, the audio and video data of the local real object during the interaction is transmitted to the local virtual object in parallel with the motion control data.

On the physical link between the anchor end and the guest end, the audio and video data of the local real object during the interaction is transmitted to the local virtual object in synchronization with the motion control data.

The Lianmai live broadcast system of the present invention comprises:

The object association module is configured to set a local real object on the anchor end and the guest end respectively in the process of establishing the live broadcast of the connected wheat, and at least one end of the anchor end and the guest end is set with the local end The local virtual object whose real object synchronizes changes;

An object interaction module, configured to transmit, by the at least one end of the anchor end object and the guest end, the local virtual object to the host real scene in the process of the live broadcast The transmitted local virtual object receives the corresponding action control data of the local real object to form a synchronous change;

The scenario publishing module is configured to convert the interaction change process formed in the anchor scenario into streaming media and broadcast to the viewer.

The Lianmai live broadcast system of the present invention comprises: a processor and a memory, wherein:

The memory is used to store the step program of the above-mentioned continuous live broadcast method;

The processor is configured to execute a step program of the dual-number continuous live broadcast method.

The Lianmai live broadcast method and system of the present invention separately and virtualize the interaction main body of the anchor end and the guest end, parameterize the action expression of each interaction main body, and configure the real object or the virtual object of each interaction main body in the same scene space. The continuous action formed by the interaction subject in the interaction forms a control parameter to control the virtual object of the interaction subject to make an accurate interaction action, and the coherent interaction between the anchor end and the guest end in a continuous time form a real-time media stream through the same scene space. Publish live broadcasts. The continuous live broadcast method of the embodiment of the present invention converts a large data volume video stream of a real object that needs to be transmitted in real time between the anchor end and the guest end into a small data amount motion expression parameter that needs only real-time transmission and a virtual body of the cross-real-time transmission interactive body. The object greatly reduces the bandwidth requirement of real-time data transmission, reduces the high-quality bandwidth pressure on the real-time data transmission system, and enables the real-time service quality of the data transmission system to be further improved, in order to further form real-time multiple guest and host-side real-time The interactive interaction between the live broadcast and the live broadcast between the anchor and the limited audience provides an important technical guarantee. The Lianmai live broadcast method and system of the invention simultaneously enrich the expression form of the interactive subject, so that the visual expression type and the expression image of the single interactive subject can be greatly enriched, so that the scene content of the Lianmai live broadcast is flexible and changeable. Improve the user experience of the audience.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of a data transmission architecture in a prior art live broadcast technology.

FIG. 2 is a flowchart of a live broadcast method according to an embodiment of the present invention.

FIG. 3 is a flowchart of pre-processing of a live broadcast method of a continuous wheat according to an embodiment of the present invention.

FIG. 4 is a flow chart (1) of an interactive process of a continuous live broadcast method of a continuous live broadcast of a live wheat system according to an embodiment of the present invention.

FIG. 5 is a flowchart (2) of an interaction process of a live broadcast method of a continuous live wheat stream in an embodiment of the present invention.

FIG. 6 is a flowchart (3) of an interactive process of a continuous live broadcast method of a continuous live broadcast of a live wheat system according to an embodiment of the present invention.

FIG. 7 is a flowchart (4) of an interactive process of a continuous live broadcast method of a continuous live broadcast of wheat in an embodiment of the present invention.

FIG. 8 is a flowchart (5) of an interaction process of a continuous live broadcast method and a live broadcast of a live beer according to an embodiment of the present invention.

FIG. 9 is a schematic structural diagram of a continuous live broadcast system according to an embodiment of the present invention.

Mode for carrying out the invention

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

The step numbers in the drawings are only used as reference numerals for the steps, and do not indicate the order of execution.

FIG. 2 is a flowchart of a method for live broadcast of a continuous wheat according to an embodiment of the present invention. As shown in Figure 2, it includes:

Step 100: In the process of establishing the live broadcast of the continuous wheat, the local real object is set on the anchor end and the guest end respectively, and at least one end of the anchor end and the guest end sets the local virtual object that changes synchronously with the real object of the local end. Real objects include real people, real static images formed by objects, and real dynamic images, such as static expressions and fixed limb gestures, dynamic expressions, and continuous limb movements. A virtual object is a three-dimensional model or a two-dimensional model formed by stereo modeling, including a texture map. The virtual object may be a model formed from a model or a creative work formed by a real anchor object.

Step 200: In the process of the live broadcast of the live broadcast, at least one end of the anchor end and the guest end transmits the local virtual object to the local real scene instead of the local real object, and the transmitted local virtual object receives the corresponding local real The object's motion control data forms a synchronous change. The anchor scene is effectively controlled by the anchor end, so that the anchor scene can form a complicated scene schedule, so that the anchor scene is used as a connected scene or a connected scene. For example, a panoramic lianmai scene formed by scene scheduling, a lianmai scene in a picture-in-picture, a lianmai scene in a plurality of picture-in-pictures, and the like.

Step 300: Convert the interaction change process formed in the anchor scene into streaming media, and broadcast it to the viewer.

The Lianmai live broadcast method of the embodiment of the present invention separately objects (real object) and virtualized (virtual object) of the interaction main body (the main object and the guest) of the anchor end and the guest end, and parameterizes the action expression of each interaction main body (using the reference point) And the change record action of the reference point), the real object or the virtual object of each interaction body is configured in the same scene space (an anchor scene), and the control parameters are formed by using the continuous motion expression formed by the interaction body in the interaction to control the virtual body of the interaction body The object makes an accurate interaction, and through the same scene space, the coherent interaction between the anchor end and the guest end in a continuous time forms a real-time media stream for publication and live broadcast. The continuous live broadcast method of the embodiment of the present invention converts a large data volume video stream of a real object that needs to be transmitted in real time between the anchor end and the guest end into a small data amount motion expression parameter that needs only real-time transmission and a virtual body of the cross-real-time transmission interactive body. The object greatly reduces the bandwidth requirement of real-time data transmission, reduces the high-quality bandwidth pressure on the real-time data transmission system, and enables the real-time service quality of the data transmission system to be further improved, in order to further form real-time multiple guest and host-side real-time The interactive interaction between the live broadcast and the live broadcast between the anchor and the limited audience provides an important technical guarantee.

FIG. 3 is a flowchart of pre-processing of a live broadcast method of a continuous wheat according to an embodiment of the present invention. As shown in FIG. 3, in an embodiment of the present invention, in the process of establishing a live broadcast in the step 100, the method includes:

Step 110: Establish a coordinate space on the anchor end. Correspondingly establish the relevant coordinate space on the guest side. The related coordinate basis makes the real object or the virtual object at one end can be reasonably displayed in the coordinate space of the opposite end. With proper adjustment and setting, the real object or the virtual object at both ends can form the required scene in the coordinate space of one end.

The same live space can be provided by the universal live broadcast framework running on the anchor side and the guest side, as well as the proportions, positions and associated controls of the objects in the scene composition.

In an embodiment of the present invention, in the step 100, at least one end of the anchor end and the guest end is configured to set a local virtual object that is synchronously changed with the action of the real object of the local end, and includes:

Step 120: Establish a local real object in the coordinate space of the anchor end to form a control reference point. Usually it refers to the image of the anchor itself on the anchor end, and the image of the guest itself is obtained on the guest side. The specific image can be the whole body or part of the body of the anchor itself or the guest itself. Obtaining a real object can be obtained by using a camera for video capture. Through the image recognition, a control reference point can be formed in the body or the body, and the local motion of the reference point with the body or the body can be controlled, and the position change of the control reference point can reflect the action change of the real object.

Step 130: Acquire a virtual object and a corresponding model control point to form a local virtual object. The virtual object may be a stereo or planar modeling model formed according to the anchor itself, or a stereo or planar modeling model formed according to a literary work or other real object. The model includes model vertices and material maps with adjustable shapes. By adjusting the relative positions of the model vertices, the corresponding deformation changes of the model local shape and the material map can be formed. The model vertices can be used as model control points for changing the shape of the model. The obtained virtual objects can be several, corresponding to different avatars.

Step 140: Establish a mapping rule between the model control point and the control reference point. The mapping between the model control points of the real object and the control reference points of each virtual object may be one-to-one, one-to-many or many-to-one.

Step 150: Collect a change of the control reference point of the real object of the local end, and form action control data of the local virtual object according to the mapping rule.

Steps 120 to 150 can also be applied to the guest end. The guest end can also form the action control data between the local virtual objects corresponding to the real object of the local end, just like the anchor end.

The action change of the real object of the local end is collected, and the action control data of the local virtual object is formed according to the mapping rule. The change of the action of the real object is obtained by the position change of the control reference point in the coordinate space itself or the guest itself, and the relative position change of the model control point of the local virtual object is changed according to the selected local virtual object and the corresponding mapping rule. The corresponding action is synchronously changed, and the local virtual object changes synchronously with the action of the real object of the local end.

Through the existing high-speed data transmission system, a physical link between the route-optimized anchor and the guest end can be established, and the physical link is used for the interaction of audio and video objects between the anchor end and the guest end, as well as real objects, virtual objects, and motion control. Real-time transmission of data.

FIG. 4 is a flow chart (1) of an interactive process of a continuous live broadcast method of a continuous live broadcast of a live wheat system according to an embodiment of the present invention. As shown in FIG. 4, in an embodiment of the present invention, the interaction process formed by the at least one end of the host and the guest end transmitting the local virtual object to the local real scene in the step 200 includes:

Step 210: The anchor end uses the local virtual object to replace the local real object in the anchor scene.

Step 211: The anchor end initiates a connection request to the guest end, and accepts the guest end response.

Step 212: The anchor end sends the real object of the local end to the guest end.

Step 213: The anchor end receives the real object of the guest end.

Step 214: The anchor end passes the local virtual object in the anchor scene and the real object in the guest end

Interact.

Correspondingly, on the guest side, the guest side interacts with the real object on the host side and the real object on the anchor side in the guest scene.

In this embodiment, the anchor end uses a virtual object on the local end to replace the corresponding real object in the anchor scene, so that the anchor image and style of the anchor end can be combined with the characteristics of the local virtual object to form a richer live broadcast scenario, and the live broadcast to the viewer is increased. Appeal. The interaction between the anchor end and the guest end interacts with the real objects at both ends, and the service quality of the real-time network transmission system (for example, SD-RTN network) can be used to express rich interactive emotions and physical movement details, and the virtual object is used for the anchor in the live broadcast to the viewer. While ensuring the quality of interaction, the demand for live broadcast bandwidth of the viewer is effectively reduced, and the real-time performance of live broadcast to a small-scale audience is ensured.

As shown in FIG. 5, in an embodiment of the present invention, the interaction process formed by the at least one end of the host and the guest end transmitting the local virtual object to the local real scene in the step 200 includes:

Step 220: The anchor end uses the local real object in the anchor scene.

Step 221: The anchor end initiates a connection request to the guest end, and accepts the guest end response.

Step 222: The anchor end sends the real object of the local end to the guest end.

Step 223: The anchor end receives the virtual object of the guest end.

Step 224: The anchor end interacts with the local real object in the anchor scene and the virtual object on the guest end.

Correspondingly, on the guest side, the guest side interacts with the real virtual object in the guest scene and the real object on the anchor side.

In this embodiment, in the anchor scene of the anchor end, the real object of the local end is used to form an interaction with the real object of the guest end, and the real object of the anchor end is used to induce the guest to generate a richer anchor image and style in the guest scene of the guest end, which can make The guest's characteristics set a richer live broadcast scenario, increasing the attraction of the anchor scene to the viewer's live broadcast. The interaction between the anchor end and the guest end interacts with the virtual object through the real object between the two ends, and can convert part of the video data exchanged between the opposite ends into interactive motion control data, thereby effectively reducing the requirement for the quality of service level of the real-time network transmission system, and satisfying The bandwidth requirement of the uplink and downlink imbalance or the unbalanced data volume during the interaction between the guest and the anchor ensures the real-time of the interaction process.

FIG. 6 is a flowchart (3) of an interactive process of a continuous live broadcast method of a continuous live broadcast of a live wheat system according to an embodiment of the present invention. As shown in FIG. 6 , in an embodiment of the present invention, the interaction process formed by the at least one end of the host and the guest end transmitting the local virtual object to the local real scene in the step 200 includes:

Step 230: The anchor end uses the local virtual object to replace the local real object in the anchor scene.

Step 231: The anchor end initiates a connection request to the guest end, and accepts the guest end response.

Step 232: The anchor end sends the local virtual object to the guest end.

Step 233: The anchor end receives the virtual object of the guest end.

Step 234: The anchor end interacts with the local virtual object in the anchor scene and the virtual object of the guest end.

Correspondingly, on the guest side, the guest side interacts with the local real object in the guest scene or the virtual object on the local end and the virtual object on the anchor side.

In this embodiment, the anchor end uses a virtual object of the local end to replace the corresponding real object in the anchor scene, and uses the virtual object of the guest end to replace the real object of the corresponding guest end, so that the image and style of the anchor and the guest can be combined with the local virtual The object's people set features to create a richer live broadcast scenario, increasing the appeal to viewers. The interaction between the anchor end and the guest end can make full use of the service quality of the real-time network transmission system to express rich interactive emotions and physical movement details through the interaction of the virtual objects at the two ends. In the live broadcast to the audience, the anchor and the guest can use the virtual object as much as possible. The video data exchanged between the real objects is converted into interactive motion control data, which effectively reduces the requirement of the quality of service level of the real-time network transmission system, and can meet the real-time game interaction of high resolution and high frame rate between the anchor and the guest.

FIG. 7 is a flowchart (4) of an interactive process of a continuous live broadcast method of a continuous live broadcast of wheat in an embodiment of the present invention. As shown in FIG. 7 , in an embodiment of the present invention, at least one end of the anchor end and the guest end transmits the local virtual object to the anchor real object in the anchor scene to form a combination with the interaction process. The interaction process includes:

Step 240: The anchor end uses the local real object in the anchor scene.

Step 241: The anchor end initiates a connection request to the guest end, and accepts the guest end response.

Step 242: The anchor sends the local virtual object to several guest terminals.

Step 243: The anchor end receives the real object of each guest.

Step 244: The real end end passes the real object of the local end in the anchor scene and the realities of several guest ends.

Objects interact.

Correspondingly, on each guest side, the guest side interacts with the real object on the local end and the virtual object on the anchor side in the guest scene.

In this embodiment, in the anchor scene of the anchor end, the real object of the local end is used to form an interaction with the real object of each guest end, and the virtual object of the anchor end is used to induce a guest to generate a richer anchor image and style in the guest scene of each guest. The guest's personality can be set to form a richer live broadcast scenario, and the attraction of the anchor scene to the live broadcast of the viewer is increased. The interaction between the anchor end and each guest end interacts with the real object through the virtual object between the two ends, and can convert part of the video data exchanged between the opposite ends into interactive action control data, thereby effectively reducing the service quality of the anchor network to the real-time network transmission system in the group live broadcast. The requirement of the level satisfies the bandwidth requirement of the uplink and downlink imbalance or the unbalanced data volume during the interaction between the guest and the anchor, and ensures the real-time of the interaction process.

FIG. 8 is a flowchart (5) of an interaction process of a continuous live broadcast method and a live broadcast of a live beer according to an embodiment of the present invention. As shown in FIG. 8 , in an embodiment of the present invention, in step 200, at least one end of the anchor end and the guest end transfers the local virtual object to the local real scene instead of the local real object to form an interaction process, including:

Step 250: The anchor end adopts the local virtual object in the anchor scene.

Step 251: The anchor end initiates a connection request to the guest end, and accepts the guest end response.

Step 252: The anchor sends the local virtual object to several guest terminals.

Step 253: The anchor end receives the virtual object of each guest.

Step 254: The anchor end interacts with the local virtual object in the anchor scene and the virtual objects of the guest end.

Correspondingly, on each guest side, the guest side interacts with the local virtual object in the guest scene and the virtual object on the anchor side.

In this embodiment, the anchor end uses a virtual object of the local end to replace the corresponding real object in the anchor scene, and uses the virtual object of the guest end to replace the real object of the corresponding guest end, so that the image and style of the anchor and the guest can be combined with the local virtual The object's people set features to create a richer live broadcast scenario, increasing the appeal to viewers. The interaction between the anchor end and the guest end can make full use of the real-time network transmission system service quality to ensure the expression of rich interactive emotions and physical movement details through the real-time network transmission system service quality. In the live broadcast to the audience, the anchor and the guest try to adopt the virtual object as much as possible to the opposite end. The video data exchanged between real objects is converted into interactive action control data, which effectively reduces the requirement of the quality of service level of the real-time network transmission system, and can realize real-time interaction between the anchor and more guests in the existing high-speed data transmission system.

Some real objects will further include real sounds, such as native sounds. The virtual object will further include virtual sounds, including emotions expressed in terms of vocabulary, phrases, and sentences. In an embodiment of the present invention, the action change data of the local virtual object that is transmitted in step 200 and received by the local virtual object is synchronously changed.

On the physical link between the anchor end and the guest end, the audio and video data of the local real object in the interaction process is transmitted to the local virtual object in parallel with the motion control data.

The parallel transmission to the virtual object means that the audio and video data in the interaction process is encapsulated to form audio and video package data on the real object side, and the action control data formed on the real object side is encapsulated to form control package data, audio and video package data and control. The encapsulated data is transmitted under different transport protocols under different encapsulation protocols. For example, the audio and video package data is encapsulated by the H.264 protocol, and the control package data is encapsulated by using RTMP or other protocols. This method can be used to adapt to the delay caused by route redirection when the loan service level is low.

In an embodiment of the present invention, the action change data of the local virtual object that is transmitted in step 200 and received by the local virtual object is synchronously changed.

On the physical link between the anchor end and the guest end, the audio and video data of the local real object in the interaction process is transmitted to the local virtual object in synchronization with the motion control data.

The synchronous transmission to the virtual object means that the audio and video data in the interaction process and the motion control data formed on the real object side are uniformly encapsulated on the real object side to form unified encapsulated data, and the audio and video 20 data and the motion control data adopt a unified encapsulation protocol. Transfer under the control of the same transport protocol. For example, the H.264 protocol is used to encapsulate audio and video data, and the reserved byte encapsulation action control data in the H.264 protocol encapsulation format is utilized, and the unified encapsulation data formed can fully utilize the error correction control mechanism of the H.264 protocol to ensure Data integrity.

FIG. 9 is a schematic structural diagram of a continuous live broadcast system according to an embodiment of the present invention. As shown in FIG. 9, the object association module 400 is configured to set a local real object on the anchor end and the guest end in the process of establishing a live broadcast of the live wheat, and at least one end of the anchor end and the guest end is set to be true with the local end. The local virtual object whose object is synchronized.

The object interaction module 500 is configured to transmit, at least one end of the host virtual object and the local real object to the anchor scene in the process of the live broadcast of the live broadcast, and the transmitted local virtual object receives the corresponding The motion control data of the real object of the local end forms a synchronous change.

The scenario publishing module 600 is configured to convert the interaction change process formed in the anchor scenario into streaming media and broadcast to the viewer.

The object association module 400 includes:

The scenario planning sub-unit 410 is configured to establish a coordinate space on the anchor end.

The real object obtaining unit 420 is configured to establish a local real object in a coordinate space of the anchor end to form a control reference point.

The virtual object obtaining unit 430 is configured to acquire the virtual object and the corresponding model control point to form a local virtual object.

Object association unit 440: used to establish a mapping rule between the model control point and the control reference point.

The data collection form 450 is used to collect the change of the control reference point of the real object on the local end, and form the action control data of the local virtual object according to the mapping rule.

The object interaction module 500 includes:

The real object replacement unit 511 is configured to replace the local real object in the anchor scene by using the local virtual object by the anchor end.

The real object placement unit 512 is configured to use the local real object in the anchor scene in the anchor scene.

The virtual object placement unit 513 is configured to use the local virtual object in the anchor scene in the anchor scene.

Also includes:

The real object unit 521 is sent to the anchor end to send the local real object to the guest end.

The first sending virtual object unit 522 is configured to send the local virtual object to the guest end by the anchor end.

The second sending virtual object unit 523 is configured to send the local virtual object to the guest end by the anchor end.

Also includes:

The continuous live broadcast establishing unit 530 is configured to initiate a connection request to the guest end by the anchor end, and accept the guest end response.

Also includes:

The first receiving real object unit 541 is configured to receive the real object of the guest end on the anchor end.

The first receiving virtual object unit 542 is configured to receive the virtual object of the guest end on the anchor end.

The second receiving real object unit 543 is configured to receive the real object of each guest end by the anchor end.

The second receiving virtual object unit 544 is configured to receive the virtual object of each guest end by the anchor end.

Also includes:

The first scene interaction unit 551 is configured to perform interaction between the local virtual object in the anchor scene and the real object in the guest end.

The second scenario interaction unit 552 is configured to perform interaction between the local real object in the anchor scene and the virtual object on the guest end in the anchor scene.

The third scene interaction unit 553 is configured to perform interaction between the local virtual object in the anchor scene and the virtual object in the guest end.

The fourth scenario interaction unit 554 is configured to perform interaction between the local real object in the anchor scene and the real objects in the guest end in the anchor scene.

The fifth scene interaction unit 555 is configured to perform interaction between the local virtual object in the anchor scene and the virtual objects in the guest end in the anchor scene.

Also includes:

The first transmission unit 561 is configured to transmit, in the physical link between the anchor end and the guest end, the audio and video data of the local real object in the interaction process and the motion control data are transmitted to the local virtual object in parallel.

The second transmission unit 562 is configured to perform, on the physical link between the anchor end and the guest end, the audio and video data of the local real object in the interaction process and the motion control data are synchronously transmitted to the local virtual object.

The Lianmai live broadcast system of the embodiment of the present invention includes a processor and a memory, wherein:

The memory is used to store the programmed devices and units in the continuous live broadcast system of the embodiment of the present invention, or to store the programmed steps in the continuous live broadcast method of the embodiment of the present invention.

A program for executing a program of each device and unit in a memory, or a step.

The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions, etc., which are within the spirit and principles of the present invention, should be included in the scope of the present invention. within.

Claims

A method for direct live broadcast of wheat, which is applied to the interactive process of continuous live wheat, characterized in that it comprises:

In the process of establishing the live broadcast of the connected wheat, the local real object is set on the anchor end and the guest end respectively, and at least one end of the anchor end and the guest end is configured to set a local end that changes synchronously with the real object of the local end. Virtual object

In the process of the live broadcast, at least one end of the anchor end and the guest end transfers the local virtual object to the local real scene instead of the local real object, and the transmitted the original Receiving, by the end virtual object, the action control data of the corresponding real object of the local end to form a synchronous change;

Converting the interaction change process formed in the anchor scene into streaming media and broadcasting to the viewer.
The live broadcast method according to claim 1, wherein in the process of establishing the live broadcast of the continuous wheat, the local real object is set on the anchor end and the guest end, respectively, the anchor end and the guest The local virtual object whose at least one end of the terminal is set to be synchronized with the real object of the local end includes:

Establishing a coordinate space on the anchor end;

Establishing the local real object in the coordinate space to form a control reference point;

Obtaining a virtual object and a corresponding model control point to form the local virtual object;

Establishing a mapping rule between the model control point and the control reference point;

The change of the control reference point is collected, and the action control data is formed according to the mapping rule.
The live broadcast method according to claim 1, wherein in the process of establishing the live broadcast of the continuous wheat, the local real object is set on the anchor end and the guest end, respectively, the anchor end and the guest At least one end of the end is configured to set a local virtual object that changes synchronously with the real object of the local end:

Establishing a coordinate space associated with the anchor end at the guest end;

Establishing the local real object in the coordinate space to form a control reference point;

Obtaining a virtual object and a corresponding model control point to form the local virtual object;

Establishing a mapping rule between the model control point and the control reference point;

Collecting changes of the control reference point, and forming action control data of the local virtual object according to a mapping rule.
The method of claim 1 , wherein in the process of the live broadcast, at least one end of the anchor end and the guest end replaces the local virtual object The local real object is transmitted to the anchor scene, and the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

The anchor end uses the local virtual object to replace the local real object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

The anchor end sends the real object of the local end to the guest end;

The anchor end receives the real object of the guest end;

The anchor end interacts with the local virtual object in the anchor scene and the real object of the guest end.
The method of claim 1 , wherein in the process of the live broadcast, at least one end of the anchor end and the guest end replaces the local virtual object The local real object is transmitted to the anchor scene, and the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

The anchor end uses the local real object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

Sending, by the anchor end, the real object of the local end to the guest end;

The anchor end receives the virtual object of the guest end;

The anchor end interacts with the local real object in the anchor scene and the virtual object of the guest end.
The method of claim 1 , wherein in the process of the live broadcast, at least one end of the anchor end and the guest end replaces the local virtual object The local real object is transmitted to the anchor scene, and the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

The anchor end uses the local virtual object to replace the local real object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

Sending, by the anchor end, the local virtual object to the guest end;

The anchor end receives the virtual object of the guest end;

The anchor end interacts with the local virtual object in the anchor scene and the virtual object of the guest end.
The method of claim 1 , wherein in the process of the live broadcast, at least one end of the anchor end and the guest end replaces the local virtual object The local real object is transmitted to the anchor scene, and the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

The anchor end adopts the local real object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

Sending, by the anchor end, the local virtual object to a plurality of the guest terminals;

The anchor end receives a real object of each of the guest terminals;

The anchor end interacts with the real object of the local end and the real object of the guest end in the anchor scene.
The method of claim 1 , wherein in the process of the live broadcast, at least one end of the anchor end and the guest end replaces the local virtual object The local real object is transmitted to the anchor scene, and the synchronized change of the motion control data of the local real object that is received by the local virtual object is:

The anchor end adopts the local virtual object in the anchor scene;

The anchor end initiates a connection request to the guest end, and accepts the guest end response;

Sending, by the anchor end, the local virtual object to a plurality of the guest terminals;

The anchor end receives a virtual object of each of the guest ends;

The anchor end interacts with the local virtual object in the anchor scene and the virtual object of the guest end.
The continuous live broadcast method according to claim 1, wherein the received action control data of the local virtual object that is received by the local virtual object forms a synchronous change comprising:

On the physical link between the anchor end and the guest end, the audio and video data of the local real object during the interaction is transmitted to the local virtual object in parallel with the motion control data.
The continuous live broadcast method according to claim 1, wherein the received action control data of the local virtual object that is received by the local virtual object forms a synchronous change comprising:

On the physical link between the anchor end and the guest end, the audio and video data of the local real object during the interaction is transmitted to the local virtual object in synchronization with the motion control data.
A continuous live wheat system, including:

An object association module, configured to set a local real object on the anchor end and the guest end respectively in the process of establishing the live broadcast of the connected wheat, wherein at least one end of the anchor end and the guest end is set to be true with the local end The local virtual object whose object is synchronously changed;

An object interaction module, configured to transmit, by the at least one end of the anchor end object and the guest end, the local virtual object to the host real scene in the process of the live broadcast The transmitted local virtual object receives the corresponding action control data of the local real object to form a synchronous change;

The scenario publishing module is configured to convert the interaction change process formed in the anchor scenario into streaming media and broadcast to the viewer.
A continuous live wheat system, including a processor and a memory, wherein:

The memory is configured to store a step program of the continuous live broadcast method according to any one of claims 1 to 10;

The processor is configured to execute the step program of the continuous live broadcast method according to any one of claims 1 to 10.