CN106792228B

CN106792228B - Live broadcast interaction method and system

Info

Publication number: CN106792228B
Application number: CN201611142174.7A
Authority: CN
Inventors: 卓康志; 冯皓; 陈哲祥; 林伟鹏; 林剑宇
Original assignee: Fujian Kaimi Network Science & Technology Co ltd; Fujian Star Net Communication Co Ltd
Current assignee: Fujian Kaimi Network Science & Technology Co ltd; Fujian Star Net Communication Co Ltd
Priority date: 2016-12-12
Filing date: 2016-12-12
Publication date: 2020-10-13
Anticipated expiration: 2036-12-12
Also published as: CN106792228A

Abstract

The invention provides a live broadcast interaction method and system. The method comprises the following steps: receiving real object order information aiming at a real object from a terminal for receiving live broadcast content in a digital audio-visual unit of a digital audio-visual place, generating real object information to be paid based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the real object information to be paid. The embodiment of the invention can realize presenting the real goods and improve the interactivity. The embodiment of the invention can broadcast AR content directly by combining the camera and can introduce AR direct broadcasting into KTV places. The implementation mode of the invention can also improve the safety of mobile payment and reduce the cost, and can also realize cloud configuration, and the upgrading and maintenance are very convenient.

Description

Live broadcast interaction method and system

Technical Field

The invention belongs to the technical field of mobile live broadcast, and particularly relates to a live broadcast interaction method and system.

Background

The network live broadcast absorbs and continues the advantages of the internet, the online live broadcast is carried out by utilizing a video mode, talent and skill exhibition, product exhibition, related meetings, background introduction, scheme evaluation, online investigation, conversation interview, online training and other contents can be released to the internet on site, and the popularization effect of the activity site is enhanced by utilizing the characteristics of intuition, quickness, good expression form, rich contents, strong interactivity, unlimited region, divisible audience and the like of the internet. After the live broadcast is finished, the live broadcast can also continuously provide rebroadcast and on-demand for readers at any time, effectively prolongs the live broadcast time and space, and exerts the maximum value of the live broadcast content.

At present, an independent signal acquisition device (audio + video) is usually erected on site, and then uploaded to a server through a network, and released to a website for people to watch. The existing live broadcast system mainly comprises: the method comprises the steps that video data of a camera acquired by a mobile phone and audio data acquired by an audio module are processed by beauty software, then audio and video synthesis is pushed to a (content delivery Network (CDN) platform, online users watch live content sources on the platform, and meanwhile, the online users interact with an initiating live user through a barrage, a message bar, a virtual prop and a virtual gift

However, the existing live broadcast software can only present virtual props and gifts in the user interaction environment, and cannot present actual supermarket articles or actual articles.

In addition, the existing live video broadcast basically only has functions of beautifying and the like, and the cool and dazzling video content cannot be live broadcast out by combining an enhanced display (AR) function. In addition, the existing live broadcast mainly utilizes a mobile phone, is limited by the focal length of a camera of the mobile phone, has limited pictures, cannot meet live broadcast of multiple people, and is often limited by the capacity of a battery of the mobile phone. In addition, the camera of the mobile phone is easy to shake in the shooting process, and the shot video is easy to be unclear; and the mobile phone camera is at the removal shooting in-process, and network signal is not good, causes the live broadcast card easily to pause.

Disclosure of Invention

The embodiment of the invention provides a live broadcast interaction method and a live broadcast interaction system, so that real objects are presented, and the interactivity is improved.

The technical scheme of the embodiment of the invention is as follows:

a live interaction method, the method comprising:

receiving real object order information aiming at a real object from a terminal for receiving live broadcast content in a digital audio-visual unit of a digital audio-visual place, generating real object information to be paid based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the real object information to be paid.

In one embodiment, the method further comprises:

determining live broadcast content based on a mobile terminal, and sending the live broadcast content to a cloud end so as to generate a play address of the live broadcast content; or

Determining live broadcast content based on the box song requesting terminal, and sending the live broadcast content to a cloud end so as to generate a playing address of the live broadcast content by the cloud end.

In one embodiment, the determining live content based on the mobile terminal includes:

when a live user selects the mobile terminal as a live recording source on the mobile terminal, enabling the mobile terminal to record the video of the live user as live content; or, when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the mobile terminal, enabling the camera to record the video of the live user, and performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content; or when a live broadcast user selects the mobile terminal and a camera arranged in a digital audio-visual unit of a digital audio-visual place to be used as a live broadcast recording source on the mobile terminal, enabling the mobile terminal to record the video of the live broadcast user, enabling the camera to record the video of the live broadcast user, and performing augmented reality processing on the video recorded by the mobile terminal, the video of the live broadcast user and a preset virtual scene model to be used as live broadcast content; or when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the mobile terminal, enabling the camera to record the video of the live user, and taking the video of the live user as the live content;

or

The determining of the live broadcast content based on the box song requesting terminal comprises: when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the box song ordering terminal, enabling the camera to record the video of the live user, and taking the video of the live user as live content; or when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the box song ordering terminal, enabling the camera to record the video of the live user, and performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content.

In one embodiment, said receiving live content in a digital audiovisual unit of a digital audiovisual venue comprises:

the terminal is connected to a live broadcast page through WeChat to receive the live broadcast content; and/or

And the terminal receives the live broadcast content through a preset application program.

In one embodiment, the physical object comprises at least one of:

wine; a beverage; snacks; fresh flowers; fruits; cooking; a toy; a garment; an ornament.

In one embodiment, the method further comprises:

receiving order information from a mobile terminal bound to a digital audiovisual unit of a digital audiovisual venue via a payment management platform in the same local area network as a set-top box in the digital audiovisual unit of the digital audiovisual venue;

sending the order information to a gateway server through a local area network via the payment management platform, sending the order information to a cloud server by the gateway server via the internet, and sending the order information to a payment platform by the cloud server via the internet;

receiving information to be paid from a gateway server through a local area network via the payment management platform, wherein the information to be paid is generated by the payment platform based on order information, is sent to a cloud server via the internet, and is sent to the gateway server via the internet by the cloud server;

displaying the information to be paid so as to execute payment by the mobile terminal based on the information to be paid;

or, the method further comprises: the payment platform returns a payment result to the cloud server through the Internet; the cloud server sends the payment result to the gateway server through the Internet; the gateway server sends a payment result to the payment management platform through the local area network;

or, the method further comprises: when a new payment platform is added, the cloud server further establishes internet connection with the new payment platform; the cloud server selects a target payment platform based on a preset strategy and sends the order information to the target payment platform; or when the payment platform is replaced by another payment platform, the cloud server further establishes internet connection with the other payment platform; the cloud server sends the order information to the other payment platform via the internet; or when the payment platform is deleted, the cloud server deletes the internet connection with the payment platform.

In one embodiment, the receiving order information via a payment management platform in a local area network includes:

receiving order information from the set top box via the local area network; wherein: the order information is triggered by a user at a mobile terminal and is sent to the set top box by the mobile terminal through the local area network; or, receiving order information from a payment server via the local area network; wherein: the order information is triggered on the payment server by a manager;

or

The number of the payment servers is one or more; and/or the information to be paid comprises a payment code in a two-dimensional code form or a bar code form or a bank-type payment link.

In one embodiment, the method further comprises:

recording the video or the picture of the real object;

and displaying the live broadcast content and the video or the picture of the real object in a digital audio-visual unit of the digital audio-visual place.

In one embodiment, the method further comprises: updating a first object into the virtual scene model, and updating interactive content into the virtual scene model according to an interactive instruction when the interactive instruction is received so as to obtain the live broadcast content;

wherein the first object comprises at least one of: a real person object; a pet; a plant.

In one embodiment, updating the first object into the virtual scene model comprises: acquiring signals of more than one camera device in real time, and acquiring more than one first image data; extracting more than one first object from each first image data according to a preset condition, and updating the more than one first object into the virtual scene model in real time;

the interactive content further comprises content data of a first material, the first material comprising: text material, image material, sound material or combination of image material and sound material; or, the virtual scene model is a 3D virtual stage.

A live interactive system, comprising:

the cloud server is used for generating a playing address based on live broadcast content in a digital audio-visual unit of a digital audio-visual place;

and the payment platform is used for receiving the real object order information aiming at the real object from the terminal for watching the live broadcast content based on the playing address, generating the information to be paid for the real object based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the information to be paid for the real object.

In one embodiment, further comprising:

the mobile terminal is used for recording a video of a live user as live content or performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content when the live user selects the mobile terminal as a live recording source on the mobile terminal, and sending the live content to the cloud server;

the camera is arranged in a digital audio-visual unit of the digital audio-visual place and used for recording a video of a live user as live content when the live user selects the camera as a live recording source on the mobile terminal, or performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content and sending the live content to a cloud server;

and the box song ordering terminal is used for enabling the camera to record the video of the live user and using the video of the live user as live content or performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content when the live user selects the camera as a live recording source on the box song ordering terminal.

In one embodiment, further comprising a set-top box located in a digital audiovisual unit of the digital audiovisual venue;

the camera or the mobile terminal is further configured to record the video or the picture of the real object and send the video or the picture of the real object to the set top box;

the set top box is connected with the display unit and is used for sending the video or the picture of the real object to the display unit;

and the display unit is used for displaying the live broadcast content and the video or the picture of the real object.

In one embodiment, the box is arranged in a digital audiovisual venue comprising at least one of: karaoke OK venue; a night club; a bar; a hotel; a multimedia conference room; an internet bar; a shadow bar; a billiard chamber; and/or

The digital audiovisual unit includes a box.

It can be seen from the foregoing technical solutions that, in the embodiments of the present invention, real object order information for a real object is received from a terminal that receives live content in a digital audiovisual unit in a digital audiovisual place, information to be paid for the real object is generated based on the real object order information, and the real object is distributed to the digital audiovisual unit in the digital audiovisual place after the terminal completes payment based on the information to be paid for the real object. The embodiment of the invention can realize the presentation of real articles in the live broadcast process, thereby improving the interactivity of the live broadcast.

The embodiment of the invention can live the virtual scene model content by combining the camera and introduce AR live broadcast into a digital audio-visual place.

In addition, the implementation mode of the invention can also improve the safety of mobile payment and reduce the cost, and can also realize cloud configuration, thereby being very convenient for upgrading and maintaining. Especially, when the number of the payment servers in the payment management platform is multiple, the cost benefit of the embodiment of the invention is more obvious. In the embodiment of the invention, when the payment platform changes, expands or upgrades the payment mode, the corresponding configuration work can be completed at the cloud end without any changes of other servers such as a ordering terminal and a payment server, and the upgrading and maintenance are greatly simplified.

In addition, in the embodiment of the invention, the transaction sensitive data of each transaction place can be managed in a centralized manner on the cloud server, so that the data security is ensured, and a key basis is provided for big data mining.

In addition, the invention can adopt the camera arranged in the digital audio-visual unit of the digital audio-visual place as the live broadcast recording source, overcomes the defect that the camera of the mobile phone is easy to shake in the shooting process, and overcomes the defect that the network signal of the camera of the mobile phone is not good and the live broadcast is easy to cause blockage.

Drawings

Fig. 1 is a flowchart of a live broadcast interaction method according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a gift delivery process according to an embodiment of the present invention.

Fig. 3 is a block diagram of a live interactive system according to an embodiment of the present invention.

Fig. 4 is a schematic diagram of a live interactive element according to an embodiment of the present invention.

Fig. 5 is an exemplary block diagram of a mobile payment system according to an embodiment of the present invention.

Fig. 6 is an exemplary block diagram of a mobile payment system at a digital audiovisual location according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the accompanying drawings.

For simplicity and clarity of description, the invention will be described below by describing several representative embodiments. Numerous details of the embodiments are set forth to provide an understanding of the principles of the invention. It will be apparent, however, that the invention may be practiced without these specific details. Some embodiments are not described in detail, but rather are merely provided as frameworks, in order to avoid unnecessarily obscuring aspects of the invention. Hereinafter, "including" means "including but not limited to", "according to … …" means "at least according to … …, but not limited to … … only". In view of the language convention of chinese, the following description, when it does not specifically state the number of a component, means that the component may be one or more, or may be understood as at least one.

In the embodiment of the invention, the real object order information aiming at the real object is received from the terminal for receiving the live broadcast content in the digital audio-visual unit of the digital audio-visual place, the information to be paid for the real object is generated based on the real object order information, and the real object is distributed to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the information to be paid for the real object.

In one embodiment, a mobile terminal (e.g., a mobile phone) may be bound to a digital audiovisual unit of a digital audiovisual place, a user opens a live broadcast function through a client in the mobile terminal, and in the live broadcast function, a video file shot by a camera of the live broadcast mobile terminal may be selected, and a video file shot by a camera arranged in a live broadcast box (the video file may be a video of a virtual stage) may also be selected.

In one embodiment, the user may also determine live content based on the box ordering terminal. When a live user selects a camera arranged in a digital audio-visual unit of a digital audio-visual place as a live recording source on a box song ordering terminal, enabling the camera to record a video of the live user, and taking the video of the live user as live content; or when the live broadcast user selects a camera arranged in a digital audio-visual unit of a digital audio-visual place as a live broadcast recording source on the box song ordering terminal, enabling the camera to record the video of the live broadcast user, and performing augmented reality processing on the video of the live broadcast user and a preset virtual scene model to serve as the live broadcast content.

And after determining the live broadcast content, sending the live broadcast content to a CDM live broadcast channel of the cloud server, and generating a live broadcast watching path. Then, the user directly shares the live broadcast watching path with others, and others can watch the live broadcast through the live broadcast path, or the user puts the live broadcast path on a client platform, and others actively search the live broadcast path to watch the live broadcast.

Live broadcast page can have bullet screen button, virtual gift, virtual stage property or place online supermarket entry on. Others can buy the bullet screen, the virtual gift, the virtual prop and the like on the live broadcast page. After the box is purchased, the barrage button, the virtual gift and the virtual prop are sequentially sent to the box set-top box through the cloud server and the gateway server, and then the box set-top box controls the barrage, the virtual gift, the virtual prop and the like to be displayed on a display screen of the box. When the physical goods need to be delivered, the user enters the on-line supermarket from the entrance of the on-line supermarket to purchase the physical goods, the physical order information (shopping information) comprises box information, the account of the delivered person and the like, and after payment, the physical goods are delivered to the corresponding box by the place.

Fig. 1 is a flowchart of a live broadcast interaction method according to an embodiment of the present invention. In the flow shown in fig. 1, a method for determining live content is described by taking a mobile terminal as an example. The embodiment of the invention can also determine the live broadcast content through the box song requesting terminal or other terminals. In fig. 1, a box will be described as an example of a digital viewing unit in a digital viewing location.

As shown in fig. 1, the method includes:

step 101: and binding the mobile terminal with a digital audio-visual unit of the digital audio-visual place.

Here, the mobile terminal may be bound to a digital audiovisual unit of the digital audiovisual place by scanning the two-dimensional code. For example, a two-dimensional code is posted in a digital audiovisual unit of a digital audiovisual place, and a camera of the mobile terminal scans the two-dimensional code and is bound to the digital audiovisual unit of the digital audiovisual place.

Here, the digital audiovisual place includes a karaoke place; a night club; a bar; a hotel; a multimedia conference room; an internet bar; a shadow bar; billiard chambers, and the like. The digital audiovisual unit includes a box in a digital audiovisual venue.

Step 102: the live broadcast content is determined based on the mobile terminal, and the live broadcast content is sent to the cloud end so that a playing address of the live broadcast content can be generated by the cloud end.

Here, when the live user selects the mobile terminal as the live recording source on the mobile terminal, the mobile terminal is enabled to record the video of the live user as live content.

When a live user selects a camera arranged in a digital audio-visual unit of a digital audio-visual place as a live recording source on a mobile terminal, enabling the camera to record a video of the live user, and performing Augmented Reality (AR) processing on the video of the live user and a preset virtual scene model to serve as live content.

When a live broadcast user selects the mobile terminal and a camera arranged in a digital audio-visual unit of a digital audio-visual place to be used as a live broadcast recording source on the mobile terminal, the mobile terminal is enabled to record a video of the live broadcast user, the camera is enabled to record the video of the live broadcast user, and the video recorded by the mobile terminal, the video of the live broadcast user and a preset virtual scene model are subjected to augmented reality processing to be used as live broadcast content.

When a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the mobile terminal, enabling the camera to record a video of the live user, and taking the video of the live user as live content;

in other embodiments, the live content may also be determined by the box-based song requesting terminal, specifically: when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the box song ordering terminal, enabling the camera to record the video of the live user, and taking the video of the live user as the live content. Or when a live user selects a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live recording source on the box song ordering terminal, enabling the camera to record the video of the live user, and performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content.

Moreover, after the cloud generates the playing address of the live content, the playing address of the live content can be sent to the mobile terminal, and then the mobile terminal directly shares the playing address of the live content with others. Or after the cloud generates the playing address of the live content, the playing address of the live content can be sent to the mobile terminal, the mobile terminal places the playing address of the live content on the client platform, and other people actively search the playing address of the live content on the client platform to watch the live content. Or after the cloud generates the playing address of the live broadcast content, the playing address of the live broadcast content is directly placed on the client platform, and other people actively search the playing address of the live broadcast content on the client platform to watch the live broadcast.

Step 103: and receiving real object order information aiming at the real object from a terminal for watching the live broadcast content based on the broadcast address of the live broadcast content, generating real object information to be paid based on the real object order information, and distributing the real object to a digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the real object information to be paid.

In one embodiment, the terminal connects to the live page through the WeChat (e.g., views the live content through a fifth generation hypertext markup language page of the WeChat) to receive the live content; and/or the terminal receives the live content through a preset application program.

In one embodiment, the physical object comprises at least one of: wine; a beverage; snacks; fresh flowers; fruits; cooking; a toy; a garment; ornaments, and the like. In this practical operation, the real object is not limited to the above types, and as long as the real object can be purchased in an online supermarket, the real object can be used as a present.

For example, the live user binds the mobile terminal with the digital audiovisual unit of the digital audiovisual place. And the live user Zhang III selects a camera arranged in a digital audio-visual unit of the digital audio-visual site as a live recording source on the mobile terminal, the camera records the talent performance video of the live user Zhang III, and the talent performance video of the live user is processed with a preset virtual scene model (such as a stage background) to be used as live content in an Augmented Reality (AR) mode. The camera sends the live broadcast content to the cloud end so that the cloud end can generate a playing address of the live broadcast content. The terminal li four watches the live broadcast content through the broadcast address of the live broadcast content, and is very much interested in the performance content of the live broadcast user zhang san. Lie four intends to deliver a three-entity gift as a reward. And sending real object order information aiming at the real object by the Li IV, generating real object information to be paid by the payment platform based on the real object order information, and distributing the real object to the live broadcast user in the digital audio-visual unit of the digital audio-visual place by three times when the Li IV completes payment based on the real object information to be paid.

While the foregoing detailed description describes embodiments of physical categories and viewing live content, those skilled in the art will appreciate that this description is exemplary only and is not intended to limit the scope of the present invention.

In one embodiment, the method further comprises: recording a video or picture of a real object; and displaying the live broadcast content and the video or the picture of the real object in a digital audio-visual unit of the digital audio-visual place.

In specific implementation, a live broadcast user can scan a specific two-dimensional code on a song ordering table through a K meter App or a WeChat on a mobile terminal so as to bind a box. Live broadcast users can select to use a professional camera or a mobile terminal with a camera in a box to carry out live broadcast. Use professional camera live, can select the AR function, go out virtual stage, scene live. When a user initiates a live broadcast, the user may actively share the live H5 page with the off-site audience through WeChat. And the user outside the field also uses the K meter App to find the corresponding live broadcast in the hot live broadcast hall list. Off-site viewers can send barrages, virtual gifts, virtual props, or actual supermarket items to interact with the live broadcast. The live broadcast system and the KTV song-ordering system are seamlessly combined. The pop-up screen, the virtual gift and the real gift sent by the off-site user are synchronously displayed in cool and dazzling mode on a big box television of the KTV, and if the off-site user sends the gift of an actual supermarket, the off-line supermarket system can automatically output the corresponding actual gift and send the gift to a live broadcaster by a waiter.

The embodiment of the invention preferably also provides a fused virtual scene interaction method. In one embodiment, the method further comprises: updating a first object into the virtual scene model, and updating interactive content into the virtual scene model according to an interactive instruction when the interactive instruction is received so as to obtain the live broadcast content; wherein the first object includes: a physical or virtual gift.

In one embodiment, updating the first object into the virtual scene model comprises: acquiring signals of more than one camera device in real time, and acquiring more than one first image data; extracting more than one first object from each first image data according to a preset condition, and updating the more than one first object into a virtual scene in real time; the interactive content further comprises content data of a first material, the first material comprising: text material, image material, sound material or combination of image material and sound material; or, the virtual scene model is a 3D virtual stage.

Therefore, the embodiment of the invention also provides a method for interaction of the fused virtual scene, which can be applied to various requirements of network live broadcast or MTV (maximum Transmission video) production and the like. The method for interaction of the fusion virtual scene updates the first object into the virtual scene, and updates the interaction content into the virtual scene according to the interaction instruction when the interaction instruction is received, so as to obtain the image data.

Specifically, the fused virtual scene interaction comprises the following steps:

the first step is as follows: the method comprises the steps of acquiring signals of more than one camera device in real time, and acquiring more than one first image data.

The second step is that: more than one first object is extracted from each first image data according to a preset condition. The first object may be different specific objects as required in different embodiments, for example, the first object may be a real man anchor, may be a pet animal, etc., may also be a plant, etc.; the number of the first objects may be single or 2 or more. Depending on these actual requirements, different algorithms and settings may be used to efficiently extract the first object in the first data image. The following is exemplified by an embodiment of an algorithm that extracts the first object specifically.

In one embodiment, the first image data includes a first object that is a character anchor, and the background of the anchor is a solid background. The specific steps of extracting the first object in the first image data are as follows: the GPU compares the color value of each pixel in the first image data with a preset threshold value; and if the color value of the pixel is within the preset threshold value, setting the Alpha channel of the pixel point to be zero, namely displaying the background as transparent color, and extracting the object.

Since the background is a pure color, the present embodiment uses a chroma key method to perform matting. The preset threshold is a color value of a background color, for example, if the background color is green, the threshold of the RGB color value of the preset pixel point is (0 ± 10, 255-10, 0 ± 10). The background color can be selected from green or blue, and the background with two colors can be simultaneously arranged at the shooting place for the selection of the anchor. When the host sings a song on clothes with a large contrast to green, the background of green can be selected. In the process of extracting an object (portrait), because the difference between clothes worn by a broadcaster and background color is larger, after the color value of each pixel in an image is compared with a preset threshold value, the color value of a pixel point of a background part is within the preset threshold value, an Alpha channel of the pixel point of the background part is set to be zero, namely, the background is displayed to be transparent; and the pixel points of the portrait part are not in the preset threshold value, and the portrait part is reserved, so that the portrait is extracted from the image.

In a specific embodiment, the GPU on the equipment can be used for carrying out matting processing operation, so that the CPU time is not occupied, and the system speed is increased; the GPU is special hardware for processing the image, so that the operation time of pixels with different sizes is the same, for example, the operation time of pixels with 8 bits, 16 bits and 32 bits is the same, and the operation time of the pixels can be greatly saved; the normal CPU will prolong the processing time with the increase of the pixel size, so the human image extraction speed of the embodiment is greatly improved. The difference points enable the embodiment to be realized by adopting the embedded device with the GPU, even if the performance of the CPU in the embedded scheme is weak, the scheme of the embedded device can still realize smooth display by applying the scheme of the embodiment, because if the CPU is used to extract the first object from the first image data, the CPU needs to read the video acquired by the camera device and perform the processing such as image matting, the CPU is too heavy to carry out smooth display. And this embodiment is applied to in the embedded scheme, carries out in putting into GPU with above-mentioned matting processing, has both alleviateed CPU's burden, can not cause the influence to GPU's operation simultaneously.

The third step: and receiving an interaction instruction sent from the first terminal. In different embodiments, the first terminal sends the interaction instruction through a computer network, where the computer network may be an Internet network or a local area network, and may be connected by a wired network, a WiFi network, a 3G/4G mobile communication network, a bluetooth network, or a ZigBee network. The first terminal can be a PC, mobile communication equipment such as a mobile phone and a tablet personal computer, and wearable equipment such as a smart watch, a smart bracelet and smart glasses. In some embodiments, the first terminal may further include a short-range control device such as a remote controller, and the first terminal may generate an infrared signal, a radio wave, and the like to transmit a corresponding interaction instruction.

The fourth step: and updating more than one first object into the virtual scene in real time, and updating or switching the virtual scene according to the interaction instruction to obtain second image data. In an embodiment, the virtual scene comprises a computer simulated virtual reality scene or a real shot video scene, etc. Still further, embodiments may also provide virtual scenes, such as 3D virtual reality scenes or 3D video scenes, in conjunction with newly developed 3D image technology.

The 3D virtual reality scene technology is a computer simulation system capable of creating and experiencing a virtual world, a 3D simulation scene of a real scene is generated by a computer, and the system simulation is an interactive three-dimensional dynamic scene with multi-source information fusion and system simulation of entity behaviors. The virtual scene comprises any actual scene existing in real life, including any scene which can be sensed through body feeling, such as vision, hearing and the like, and is realized in a simulated mode through computer technology. One application of the 3D virtual reality scene is a 3D virtual stage, and the 3D virtual stage simulates a real stage through a computer technology, so that a stage effect with strong stereoscopic impression and reality is realized. The method can be realized through a 3D virtual stage, and the anchor object which is not on the stage in reality performs the scene effect on various stages. When the 3D video is taken, two cameras are used for simulating parallax errors of left and right eyes, two films are respectively taken, then the two films are simultaneously projected on a screen, and the left eye image can be seen only by the left eye of a viewer and the right eye image can be seen only by the right eye of the viewer. And finally, after the two images are overlapped by the brain, a picture with a stereoscopic depth sense can be seen, namely the 3D video.

In embodiments where the interaction is in a different virtual scene, the interaction instructions may include different content, and in some embodiments the interaction instructions include a command to update the first material into the virtual scene. The method specifically comprises the following steps: and updating the first object to a virtual scene in real time, and updating the first material to the virtual scene according to the interaction instruction, so as to obtain the second image data. The first material may be image material, sound material, or a combination of image and sound material. Taking live webcast as an example, the first material includes virtual gifts, praise, background sound, applause and the like, audiences of live webcast can send interactive instructions of the virtual gifts such as flowers and the like to the main webcast through mobile phones, and the sent gifts are reflected in virtual scenes in the form of flower pictures. The audience of the network live broadcast can also send the clapping interactive instruction to the anchor broadcast through the mobile phone, and the clapping interactive instruction is played in a clapping mode. The first material may be system preset for user selection and in some embodiments the interactive instructions may include content data for the first material in addition to a command to update the first material into the virtual scene. For example, the audience uploads an interactive instruction for presenting the virtual gift through the mobile terminal, the interactive instruction also comprises a picture of the presented virtual gift, and the picture of the gift is updated to the virtual scene after the interactive instruction is received. Therefore, when the audience sends the interaction instruction, the audience can select an interaction mode and can customize the content data of the first material according to own preference, such as favorite picture material, sound material or material combining pictures and sound.

In some embodiments, the interactive instruction further includes a command to change a virtual scene shot, where the command to change the virtual scene shot includes switching a view angle of the virtual scene shot, changing a focal length of the virtual scene shot, and performing local blurring on the virtual scene. By switching the visual angle of the virtual scene lens, the pictures of the virtual scene can be simulated to be watched from different visual angles; by changing the focal length of the lens of the virtual scene, the picture of the virtual scene can be zoomed in and zoomed out; and the partial blurring processing is carried out on the virtual scene, so that the part of the picture which is not subjected to the blurring processing in the virtual scene can be highlighted. Through the command of changing the virtual scene lens, the interaction degree and the interestingness of audiences can be greatly improved.

Different from the existing live broadcast interaction, the interactive content is directly superposed on the surface layer of the image or the video, so that the superposed interactive content floats on the surface of the virtual scene in visual effect, so that the visual effect of the interactive content is very abrupt and is difficult to be integrated with the virtual scene. In the above embodiment, the interactive content is updated into the virtual scene at the same time as the first object is updated into the virtual scene, wherein the first object, the interactive content and the virtual scene are rendered and imaged together, so that the interactive content and the first object can be naturally and harmoniously fused in the virtual scene, thereby having a good visual effect. In an embodiment, the interactive content may also be a 3D interactive model obtained through 3D modeling, and the 3D interactive model is rendered in real time with the first object and the virtual scene, so that the 3D interactive model is naturally displayed in the virtual scene, and if the interactive content is a donation flower, the donation flower can be stereoscopically displayed in the virtual scene; if the interactive content is praise, praise information can be displayed on a virtual screen in the virtual scene. For example, when the virtual scene is a stage, the interactive content is a flower donation, and by adopting the method of the invention, the donated flower can be displayed in the stage and becomes a part of the stage, and the display effect is good, unlike the prior art that the flower donated only floats on the surface of the interface.

In one embodiment, a signal of a camera device is acquired in real time, and a first image data is acquired, and meanwhile, a signal of a microphone is acquired in real time, and a first sound data is acquired;

and updating the first object into the virtual scene in real time, and updating the first sound into the virtual scene in real time to obtain second image data. Taking live webcasting as an example, the first sound data is a description of a webcast or a singing sound, or a mixed sound of a singing sound of the webcast and a song accompaniment. The first sound is updated to the virtual scene in real time, and meanwhile, the updated second image data is displayed on the display terminal in real time. Therefore, not only can the sound of the network anchor be heard, but also the picture (combination of the portrait and the virtual scene) synchronous with the sound can be seen at the display terminal, and the effect of the virtual stage is realized.

In the above embodiment, after the second image data is obtained, the second image data is displayed by the display device, and by displaying the second image data on the display device, the user can see the video in which the first object and the virtual scene are combined. When the second image data is displayed, the picture of the second image data can be coded firstly, and the image data can be displayed smoothly in real time on the display device through coding processing. In the prior art, the original picture is not processed generally, and the data volume of the original picture is large, so that the prior art does not provide a technology for displaying the picture synthesized by the portrait and the virtual scene on a client in real time. In this embodiment, the picture of the updated second image data is encoded first, and the size of the picture can be greatly reduced through the encoding operation.

For example: in the case of a resolution of 720P, the size of 1 frame video is 1.31MByte, and 1 second video is 30 frames, so that the size of 1 second video in the conventional video is: 30 × 1.31 ═ 39.3 MByte; after the picture is coded, the bit rate is set to be 4Mbit and the size of the video in 1 second is set to be 4Mbit even under the resolution of 720P, and the video in 1 second is 0.5MByte because 1Byte is 8 bits; compared with the existing video, the encoded video data is greatly reduced, so that the encoded video data can be smoothly transmitted on the network, and the audio and video data can be smoothly displayed on a client.

In some embodiments, after the second image data is obtained, the second image data is stored and recorded. The stored second image data can be uploaded to a gateway server, the gateway server uploads the received image data to a cloud server, and the cloud server receives the image data and generates a sharing address. Through the steps, sharing of the second image data is achieved. And logging in the sharing address through terminal equipment (such as mobile phones, computers, tablets and other electronic equipment with a display screen), so that the audio and video data can be directly played or the second image data can be downloaded.

The obtained second image data can be played on a local display device and can also be played in real time at a network end. The method specifically comprises the following steps:

the network client acquires the second image data through a real-time streaming protocol, and decodes and displays a picture of video data in the second image data, wherein the picture content can be a picture rendered by a 3D scene; the audio data is decoded and played back by an audio playback device (e.g., a speaker). The real-time streaming protocol may be the RTSP protocol. The image data in the second image data is encoded in advance, and the video data can be smoothly played by the client through the image data encoding operation.

Details of the implementation of the blended virtual scene interaction are described in detail above. Those skilled in the art will appreciate that the above-described interaction may not be used or other interaction may be used, and the present invention is not limited thereto.

The actual gift-offering process of the embodiment of the invention is described in detail below.

As shown in fig. 2, the method includes:

step 201: off-site viewing live user a views live using App or wechat H5 pages.

Step 202: and (3) watching the live broadcast user A off site to select a real object and a gift delivery object.

Step 203: and the off-site watching live broadcast user A realizes payment through payment modes such as WeChat or Paibao and the like.

Step 204: and after successful payment, the supermarket management system of the place delivers the goods.

Step 205: and the mobile phone App or H5 interface informs the song ordering system of the commodity of the supermarket management system in the place after the television shows that the commodity pays successfully.

Step 206: the gift-obtaining person receives the physical gift delivered by the supermarket and sent by the waiter.

Therefore, in the embodiment of the invention, the real object can be conveniently and quickly sent in the live broadcasting process to increase the interaction, the interaction is not limited to virtual props and gifts, and meanwhile, the AR content is live broadcasted by combining a professional camera, so that the live broadcasting can be introduced into a KTV place, and the KTV live broadcasting era is opened.

The embodiment of the invention also provides a live broadcast interactive system. The method comprises the following steps: the cloud server is used for generating a playing address based on live broadcast content in a digital audio-visual unit of a digital audio-visual place; and the payment platform is used for receiving the real object order information aiming at the real object from the terminal for watching the live broadcast content based on the playing address, generating the information to be paid for the real object based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the information to be paid for the real object.

In one embodiment, the live interactive system further comprises:

As shown in fig. 3, the live interactive system includes:

the mobile terminal is used for being bound with a digital audio-visual unit of a digital audio-visual place, when a live broadcast user selects the mobile terminal as a live broadcast recording source on the mobile terminal, recording a video of the live broadcast user as live broadcast content, and sending the live broadcast content to the cloud server;

the system comprises a camera arranged in a digital audio-visual unit of a digital audio-visual place and a cloud server, wherein the camera is used for recording a video of a live user when the live user selects the camera as a live recording source on a mobile terminal, performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as live content, and sending the live content to the cloud server;

the cloud server is used for generating a playing address of the live broadcast content based on the live broadcast content provided by the mobile terminal or the camera;

and the payment platform is used for receiving the real object order information aiming at the real object from the terminal for watching the live broadcast content based on the broadcast address of the live broadcast content, generating the information to be paid for the real object based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the information to be paid for the real object.

In one embodiment, the live interactive system further comprises a set-top box located in a digital audiovisual unit of the digital audiovisual venue;

the camera or the mobile terminal is also used for recording the video or the picture of the real object and sending the video or the picture of the real object to the set top box; the set-top box is connected with the display unit and is used for sending the video or the picture of the real object to the display unit; and the display unit is used for displaying live broadcast content and videos or pictures of real objects.

In one embodiment, the box is arranged in a digital audiovisual venue comprising at least one of: karaoke OK venue; a night club; a bar; a hotel; a multimedia conference room; an internet bar; a shadow bar; billiard chambers, and the like.

Fig. 4 is a schematic diagram of a live interactive element according to an embodiment of the present invention. In the live broadcasting process, the states of entering and exiting from the live broadcasting channel of the user can be displayed, and the processes of virtual gift delivery and physical gift delivery can also be displayed.

Besides the live interaction, the embodiment of the invention also discloses a mobile payment method.

Mobile Payment (Mobile Payment), also called Mobile Payment, refers to a commercial transaction that both parties of a transaction use Mobile terminal equipment as a carrier for some goods or services and are realized through a Mobile communication network. The mobile terminal used for mobile payment may be a mobile phone, a Personal Digital Assistant (PDA), a mobile Personal Computer (PC), or the like. Mobile payment is currently widely used in various industries. When payment is carried out after the purchase order, the user can carry out checkout operation in the modes of cash, card payment or third-party platform payment and the like. During payment, it is often necessary for a payment terminal (e.g., a card swipe terminal such as a card swipe) to have a stable connection to the internet. If the payment server cannot be normally connected to the network, the payment cannot be completed. However, when there are many payment servers, connecting each payment server to the internet causes a cost problem. Moreover, when a large number of payments are settled simultaneously, the plurality of payment servers put little stress on the load and bandwidth of the network, and even possibly cause payment timeout due to network congestion. In addition, directly exposing the payment server to the internet may also lead to security risk issues.

At present, there are two main ways for mobile payment in digital audiovisual places: in the mode (1), the mobile terminal directly purchases products on a network platform through the Internet, and the mobile terminal directly pays on a payment platform of a third party. In the method (1), the digital audiovisual place cannot obtain the consumer data of the user, and cannot perform operations such as large data analysis. And in the mode (2), the mobile terminal directly pays to a payment management platform in the digital audio-visual place, and the payment management platform is in butt joint with a payment platform of a third party. However, when new payment methods are introduced, the payment management platform must be updated. Because the payment management platform is arranged in the place, the payment management platform in the place has to perform corresponding operation after the payment mode is increased or changed every time, which is extremely inconvenient.

In view of the defect that the mobile payment can be realized only by directly connecting the payment server with the internet in the prior art, the embodiment of the invention provides a novel mobile payment method. The payment management platform can realize mobile payment without being directly connected with the internet, so that the safety of mobile payment is improved, and the cost is reduced. The mobile payment system provided by the embodiment of the invention comprises: the system comprises a payment management platform, a gateway server, a cloud server and a payment platform, wherein the payment management platform is arranged in a local area network; the payment management platform is used for receiving the order information through the local area network and sending the order information to the gateway server through the local area network; the gateway server is used for sending the order information to the cloud server through the Internet; the cloud server is used for sending the order information to the payment platform through the Internet; the payment platform is used for generating information to be paid based on the order information and sending the information to be paid to the cloud server through the Internet; the cloud server is also used for sending the information to be paid to the gateway server through the Internet; the gateway server is also used for sending information to be paid to the payment management platform through the local area network; and the payment management platform is also used for displaying the information to be paid so as to execute payment by the user based on the information to be paid. Preferably, the payment management platform comprises an ordering terminal, a set-top box and/or a payment server.

Correspondingly, the live interaction method shown in fig. 1 further includes:

and displaying the information to be paid so as to execute payment by the mobile terminal based on the information to be paid.

In one embodiment, the live interaction method shown in fig. 1 further includes:

the payment platform returns a payment result to the cloud server through the Internet;

the cloud server sends the payment result to the gateway server through the Internet;

and the gateway server sends the payment result to the payment management platform through the local area network.

when a new payment platform is added, the cloud server further establishes internet connection with the new payment platform; the cloud server selects a target payment platform based on a preset strategy and sends the order information to the target payment platform; or

When the payment platform is replaced by another payment platform, the cloud server further establishes internet connection with the another payment platform; the cloud server sends the order information to the other payment platform via the internet; or

And when the payment platform is deleted, the cloud server deletes the internet connection with the payment platform.

In one embodiment, receiving order information via a payment management platform in a local area network comprises:

receiving order information from the set top box via the local area network; wherein: the order information is triggered by a user at a mobile terminal and is sent to the set top box by the mobile terminal through the local area network; or, receiving order information from a payment server via the local area network; wherein: the order information is triggered by the manager on the payment server.

In one embodiment, the live interaction method shown in fig. 1 further includes the number of the payment servers is one or more; and/or the information to be paid comprises a payment code in a two-dimensional code form or a bar code form or a bank-type payment link.

As shown in fig. 5, the system includes: the payment system comprises an ordering terminal 501, a payment server 502, a gateway server 503, a cloud server 504 and a payment platform 505; wherein the ordering terminal 501, the payment server 502 and the gateway server 503 are arranged in the same local area network, which is arranged in the mobile payment location. The ordering terminal 501 and the payment server 502 are communicated to form a payment management platform.

The number of the order placing terminals 501 may be plural, and a plurality of users may generate respective order information on the respective order placing terminals 501.

In one embodiment, one or more ordering terminals 501 correspond to one payment server 502. Similarly, the number of the payment servers 502 may be plural, and one or more payment servers 102 correspond to one gateway server 503.

Specifically, the method comprises the following steps:

an order issuing terminal 501 for sending order information to a corresponding payment server 502 via a local area network;

a payment server 502, configured to receive order information from the order placing terminal 501 via a local area network, and send the order information to the gateway server 503 via the local area network;

a gateway server 503, configured to send order information to the cloud server 504 via the internet;

a cloud server 504 for sending the order information 504 to a payment platform 505 via the internet;

the payment platform 505 is configured to generate information to be paid based on the order information, and send the information to be paid to the cloud server 504 via the internet;

the cloud server 504 is further used for sending the information to be paid to the gateway server 503 via the internet;

the gateway server 503 is further configured to send information to be paid to the payment server 502 which forwards the order information via the local area network;

the payment server 502 is further configured to present information to be paid, so that the user performs payment based on the information to be paid.

Specifically, the order information generated by the order placing terminal 501 may include a transaction time, a name of an order item, the number of order items, and the like. The order information can further carry merchant information such as place name, place code, box name, box ID, gateway server and the like. In one embodiment, the information to be paid may be implemented as a payment code or a bank-like payment link. The user scans the payment code through the handheld mobile terminal to confirm the order and execute payment, or clicks a payment link of a triggered bank type to confirm the order and execute payment. Furthermore, payment platform 505 may be a payment platform developed by the merchant itself, or any third party payment platform, such as a paypal payment platform, a WeChat payment platform, a Unionpay payment platform, and so on.

As can be seen, in the system architecture shown in fig. 5, the payment server obtains order information based on the local area network, and obtains information to be paid from the cloud through the gateway server. The payment server no longer needs to have internet networking capability, thereby reducing hardware costs. Especially, when the number of the payment servers is plural, the cost effectiveness of the embodiment of the present invention is more significant.

In addition, in the embodiment of the invention, the payment server is not exposed in the internet any more, so that various security risk problems can be avoided.

Embodiments of the present invention will now be described in detail with reference to the following examples, which are intended to be exemplary of embodiments of the present invention. According to the embodiment of the invention, a payment server (such as a cash register server) in a digital audiovisual place can complete payment under the condition that the network cannot be connected.

As can be seen from fig. 6, in the local area network of the digital audiovisual place, there are arranged a digital audiovisual terminal 601, a set-top box 602, a gateway server 603, and a payment server 604. The gateway server 603 has an internet connection with the cloud server 604, and the cloud server 604 is connected with the payment platform 605.

For example, the digital audiovisual terminal 601 may be a song order screen within a digital audiovisual unit implemented as a digital audiovisual venue, or the like. After a user places an order at the digital audiovisual terminal 601, the set top box 602 receives an order request from the digital audiovisual terminal 601 based on the lan connection, associates and packages merchant information such as a place name, a place code, a box name, a box ID, a gateway server number and the like with the order information, and then sends the packaged order information to the payment server 604 based on the lan. The payment server 604 receives the order information and transmits the order information to the gateway server 603 based on the local area network. The gateway server 603 sends the order information to the cloud server 604 through the internet, and after receiving the order information, the cloud server 604 decapsulates the encapsulated order information and sends the decapsulated order information to the corresponding payment platform 605 through the internet. The payment platform 605 returns a payment code corresponding to the order information to the cloud server 604 through the internet, and the cloud server 604 packages the payment code and transmits the payment code to the gateway server 603 based on the internet. The gateway service 603 pushes the payment code to the payment server 604 over the local area network. The user makes a payment for the payment code presented on the payment server 604 through the mobile terminal 606. After the payment is completed, the payment result is returned to the cloud server 304 by the corresponding payment platform 605, the cloud server 604 transmits the payment result to the gateway server 603 and the mobile terminal 606 of the user, and the gateway server 603 transmits the payment result to the payment server 604, so that the whole payment process is completed.

When the payment platform 605 is changed, the change can be realized by changing the payment platform interface of the cloud server without changing the payment server 604 or the gateway server 603, so that the configuration work is simple.

Moreover, when the user a selects to initiate the live broadcast through the mobile terminal 606, the mobile terminal 606 records the talent performance video of the user a and transmits the live broadcast content to the cloud server 604. The cloud server 604 generates a broadcast address of the live content. Terminal B on the payment terminal 607 watches the live content through the broadcast address of the live content and likes the performance content of the live user a very much. User B intends to present a triple gift as a reward. The user B sends out physical order information for the physical object, the payment platform 605 generates physical information to be paid based on the physical order information, and distributes the physical object to the user a of the mobile terminal 606 in the digital audiovisual unit of the digital audiovisual place after the user B completes payment based on the physical information to be paid.

While embodiments of the present invention have been described above by way of example in a digital audiovisual field, those skilled in the art will appreciate that this description is merely exemplary and is not intended to limit the scope of embodiments of the present invention.

In summary, in the embodiment of the present invention, the mobile terminal is bound to the digital audiovisual unit of the digital audiovisual place; determining live broadcast content based on the mobile terminal, and sending the live broadcast content to a cloud end so as to generate a play address of the live broadcast content; the method comprises the steps of receiving real object order information aiming at a real object from a terminal for watching the live broadcast content based on the broadcast address of the live broadcast content, generating real object information to be paid based on the real object order information, and distributing the real object to a digital audio-visual unit of a digital audio-visual place after the terminal completes payment based on the real object information to be paid. The embodiment of the invention can realize presenting the real goods and improve the interactivity.

The embodiment of the invention can broadcast AR content directly by combining the camera and can introduce AR direct broadcasting into KTV places.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A live interactive method, comprising:

receiving real object order information aiming at a real object from a terminal for receiving live broadcast content in a digital audio-visual unit of a digital audio-visual place, generating real object information to be paid based on the real object order information, and distributing the real object to the digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the real object information to be paid; the method further comprises the following steps:

binding the mobile terminal with a digital audio-visual unit of a digital audio-visual place;

determining live broadcast content taking the mobile terminal as a live broadcast recording source or taking a camera arranged in a digital audio-visual unit of a digital audio-visual place as the live broadcast recording source based on the mobile terminal, and sending the live broadcast content to a cloud so as to generate a playing address of the live broadcast content by the cloud; or

Determining live broadcast content taking a camera arranged in a digital audio-visual unit of the digital audio-visual place as a live broadcast recording source based on a box song requesting terminal, and sending the live broadcast content to a cloud terminal so as to generate a playing address of the live broadcast content by the cloud terminal;

the method further comprises the following steps: receiving order information from a mobile terminal bound to a digital audiovisual unit of a digital audiovisual venue via a payment management platform in the same local area network as a set-top box in the digital audiovisual unit of the digital audiovisual venue without internet networking capability; sending the order information to a gateway server through a local area network via the payment management platform, sending the order information to a cloud server by the gateway server via the internet, and sending the order information to a payment platform by the cloud server via the internet; receiving information to be paid from a gateway server through a local area network via the payment management platform, wherein the information to be paid is generated by the payment platform based on order information, is sent to a cloud server via the internet, and is sent to the gateway server via the internet by the cloud server; and displaying the information to be paid so as to execute payment by the mobile terminal based on the information to be paid.

2. The live interaction method of claim 1,

the determining live content based on the mobile terminal includes:

or

3. The live interaction method of claim 1, wherein receiving live content in a digital audiovisual unit of a digital audiovisual venue comprises:

4. A live interaction method as claimed in claim 1, wherein said real objects comprise at least one of:

5. The live interaction method of claim 1, further comprising:

the method further comprises the following steps: the payment platform returns a payment result to the cloud server through the Internet; the cloud server sends the payment result to the gateway server through the Internet; the gateway server sends a payment result to the payment management platform through the local area network;

or the method further comprises:

when a new payment platform is added, the cloud server further establishes internet connection with the new payment platform; the cloud server selects a target payment platform based on a preset strategy and sends the order information to the target payment platform; or when the payment platform is replaced by another payment platform, the cloud server further establishes internet connection with the other payment platform; the cloud server sends the order information to the other payment platform via the internet; or when the payment platform is deleted, the cloud server deletes the internet connection with the payment platform.

6. The live interaction method of claim 1, wherein receiving order information via a payment management platform in a local area network comprises:

or

7. The live interaction method of claim 1, further comprising:

recording the video or the picture of the real object;

8. The live interaction method of claim 3,

the method further comprises the following steps: updating a first object into a virtual scene model, and updating interactive content into the virtual scene model according to an interactive instruction when the interactive instruction is received so as to obtain the live broadcast content;

9. The live interaction method of claim 8,

updating the first object into the virtual scene model comprises: acquiring signals of more than one camera device in real time, and acquiring more than one first image data; extracting more than one first object from each first image data according to a preset condition, and updating the more than one first object into the virtual scene model in real time;

10. A live interactive system, comprising:

the payment platform is used for receiving real object order information aiming at a real object from a terminal for watching live broadcast content based on the playing address, generating real object information to be paid based on the real object order information, and distributing the real object to a digital audio-visual unit of the digital audio-visual place after the terminal completes payment based on the real object information to be paid;

further comprising:

the mobile terminal is bound with a digital audio-visual unit of a digital audio-visual place and used for recording a video of a live broadcast user as live broadcast content or performing augmented reality processing on the video of the live broadcast user and a preset virtual scene model to serve as the live broadcast content when the live broadcast user selects the mobile terminal as a live broadcast recording source on the mobile terminal, and sending the live broadcast content to a cloud server;

the box song ordering terminal is used for enabling the camera to record the video of the live user and using the video of the live user as live content or performing augmented reality processing on the video of the live user and a preset virtual scene model to serve as the live content when the live user selects the camera as a live recording source on the box song ordering terminal;

a payment management platform in the same local area network as a set-top box in a digital audiovisual unit of a digital audiovisual venue, without internet networking capability, to: receiving order information from a mobile terminal bound to a digital audiovisual unit of the digital audiovisual venue; sending the order information to a gateway server through a local area network, sending the order information to a cloud server by the gateway server through the Internet, and sending the order information to a payment platform by the cloud server through the Internet; receiving information to be paid from a gateway server through a local area network, wherein the information to be paid is generated by a payment platform based on order information, is sent to a cloud server through the Internet, and is sent to the gateway server through the Internet by the cloud server; and displaying the information to be paid so as to execute payment by the mobile terminal based on the information to be paid.

11. The live interaction system of claim 10, further comprising a set-top box located in a digital audiovisual unit of the digital audiovisual venue;

12. A live interaction system as claimed in claim 10 or 11, wherein said digital audiovisual venue comprises at least one of: karaoke OK venue; a night club; a bar; a hotel; a multimedia conference room; an internet bar; a shadow bar; a billiard chamber; and/or

The digital audiovisual unit includes a box.