CN116074544A - Multi-platform live broadcast method, system, equipment and medium - Google Patents
Multi-platform live broadcast method, system, equipment and medium Download PDFInfo
- Publication number
- CN116074544A CN116074544A CN202211426714.XA CN202211426714A CN116074544A CN 116074544 A CN116074544 A CN 116074544A CN 202211426714 A CN202211426714 A CN 202211426714A CN 116074544 A CN116074544 A CN 116074544A
- Authority
- CN
- China
- Prior art keywords
- live broadcast
- audio
- data
- platform
- push
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 238000012545 processing Methods 0.000 claims abstract description 15
- 230000005540 biological transmission Effects 0.000 claims description 64
- 238000004590 computer program Methods 0.000 claims description 18
- 238000004806 packaging method and process Methods 0.000 claims description 14
- 238000009877 rendering Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 230000036461 convulsion Effects 0.000 description 5
- 238000007726 management method Methods 0.000 description 5
- 239000000463 material Substances 0.000 description 4
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 241000251468 Actinopterygii Species 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- KRTSDMXIXPKRQR-AATRIKPKSA-N monocrotophos Chemical compound CNC(=O)\C=C(/C)OP(=O)(OC)OC KRTSDMXIXPKRQR-AATRIKPKSA-N 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
- H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- General Engineering & Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The invention discloses a multi-platform live broadcast method, a multi-platform live broadcast system, multi-platform live broadcast equipment and a computer storage medium, and relates to the live broadcast field, wherein the method is applied to plug-flow application and comprises the following steps: determining the resolution and the coding format supported by a target live broadcast platform to be pushed; creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing; and sending the encoded push data to a target live broadcast platform server based on the push module. Detecting the resolution and the coding format supported by a target live broadcast platform to be pushed, and if no encoder meeting the conditions exists, creating an encoder; if a compliant encoder exists, media data is retrieved from the compliant encoder. For different live broadcast platforms with the same parameter requirements, hardware consumption is reduced by improving the multiplexing rate of the encoder, and the method is power-saving and environment-friendly. Therefore, the technical problems of high memory occupation, high power consumption and easy hardware loss of the multi-platform live broadcast method are solved.
Description
Technical Field
The present invention relates to the field of live broadcast, and in particular, to a multi-platform live broadcast method, a multi-platform live broadcast system, a multi-platform live broadcast device, and a computer storage medium.
Background
Live broadcast technology has been rapidly developed in recent years, and a live broadcast function of one host broadcast on several platforms simultaneously becomes a requirement. At present, a method for realizing multi-platform live broadcast is mainly realized through cloud service, and the idea is that content of live broadcast of a host broadcast terminal is sent to a content distribution service node of a cloud terminal and then forwarded to a cloud mobile phone, and the cloud mobile phone is pushed to a third-party live broadcast platform as virtual equipment. In this way, each push uses a separate encoder, which increases CPU and memory consumption. The other mode is to independently operate apps of different live broadcasting platforms on different mobile phones to conduct live broadcasting, and the mode is high in cost and troublesome to operate.
Disclosure of Invention
The invention mainly aims to provide a multi-platform live broadcast method, a multi-platform live broadcast system, multi-platform live broadcast equipment and a computer storage medium, and aims to solve the technical problems of high memory occupation, high power consumption and easy hardware loss of the multi-platform live broadcast method in the prior art.
In order to achieve the above purpose, the present invention provides a multi-platform live broadcast method applied to plug-flow application, comprising the following steps:
determining the resolution and the coding format supported by a target live broadcast platform to be pushed;
creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing;
and sending the encoded push data to a target live broadcast platform server based on the push module.
Optionally, before the step of creating the corresponding encoder according to the resolution and the encoding format to perform encoding processing, the method further includes:
receiving live broadcast audio and video data packaged and sent by a network transmission sending end of a network camera based on a network transmission receiving end;
unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and transmitting the plug flow data to a specified live broadcast platform server based on a plug flow module.
Further, after the step of unpacking the live audio and video data, the method further includes:
and performing video decoding and video rendering on the coded video obtained by unpacking the live audio and video data, and performing audio decoding and audio playing on the coded audio obtained by unpacking the live audio and video data.
Further, before the step of encapsulating the unpacked live audio and video data and the custom data added by the anchor into push data, the method further includes:
acquiring rendered video, played audio, and anchor added custom data
Further, the step of the push application sending the push data to a specified live platform server based on a push module includes:
and based on the push live interface layer, determining whether the push data is transmitted to a specified live broadcast platform server through a preset transmission protocol push module or a preset third-party live broadcast SDK.
In addition, in order to achieve the above object, the present invention further provides a multi-platform live broadcast system, including: the network camera, push application and live broadcast platform;
the network camera comprises a network transmission receiving end;
the plug flow application comprises a network transmission receiving end and a plug flow module;
the push application receives live broadcast audio and video data packaged and sent by the network transmission sending end of the network camera based on the network transmission receiving end;
and the push application unpacks the live broadcast audio and video data, packages unpacked live broadcast audio and video data and host-added custom data into push data, and sends the push data to a specified live broadcast platform server based on the push module.
Further, the network camera further includes:
the lens module is used for collecting original video data;
the video coding module is used for coding the original video data to obtain coded video data;
the microphone array is used for collecting original audio data;
the audio coding module is used for coding the original audio data to obtain coded audio data;
the network transmission transmitting end is used for packaging the coded video data and the coded audio data into live audio and video data and transmitting the live audio and video data to the network transmission receiving end of the plug-flow application.
Further, the network transmission receiving end of the push application is used for receiving live audio and video data packaged and sent by the network transmission sending end of the network camera, and unpacking the live audio and video data into coded video data and coded audio data;
the plug-flow application further comprises:
the video decoding module is used for decoding the coded video data into original video data and performing video rendering;
the audio decoding module is used for decoding the encoded audio data into original audio data and playing the audio;
and the multiplexer is used for packaging the unpacked live broadcast audio and video data and the custom data added by the anchor into push stream data.
Further, the push module comprises a push live broadcast interface layer, a preset transmission protocol push module and a preset third-party live broadcast SDK;
and the push application is used for determining whether to send push data to a specified live broadcast platform server through the preset transmission protocol push module or the third-party live broadcast SDK based on the push live broadcast interface layer.
In addition, to achieve the above object, the present invention further provides a multi-platform live broadcast apparatus, including: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the multi-platform live method as claimed in any one of the preceding claims.
In addition, to achieve the above object, the present invention further provides a computer storage medium, on which a computer program is stored, which when executed by a processor, implements the steps of the multi-platform live broadcast method as set forth in any one of the above.
The multi-platform live broadcast method, the multi-platform live broadcast system, the multi-platform live broadcast equipment and the computer storage medium provided by the invention are used for determining the resolution and the coding format supported by the target live broadcast platform to be pushed; creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing; and sending the encoded push data to a target live broadcast platform server based on the push module.
In the application, detecting the resolution and the coding format supported by a target live broadcast platform to be promoted, and if no encoder meeting the conditions exists, creating an encoder; if a compliant encoder exists, media data is retrieved from the compliant encoder. For different live broadcast platforms with the same parameter requirements, the encoder can be reused, the utilization rate of the CPU and the memory is reduced, the hardware consumption is reduced by improving the multiplexing rate of the encoder, and the method is power-saving and environment-friendly. Therefore, the technical problems of high memory occupation, high power consumption and easy hardware loss of the multi-platform live broadcast method are solved.
Drawings
Fig. 1 is a schematic structural diagram of hardware operation of a terminal device according to an embodiment of the present invention;
FIG. 2 is a flow chart of an embodiment of a multi-platform live method of the present invention;
FIG. 3 is a schematic diagram of a plug flow module according to an embodiment of the multi-platform live broadcast system of the present invention;
FIG. 4 is a schematic diagram of a conventional plug flow of an embodiment of a multi-platform live broadcast system according to the present invention;
FIG. 5 is a current plug flow diagram of one embodiment of a multi-platform live broadcast system of the present invention;
FIG. 6 is a system diagram of one embodiment of a multi-platform live broadcast system of the present invention;
fig. 7 is a schematic diagram of a push live interface layer of an embodiment of a multi-platform live system according to the present invention;
fig. 8 is a schematic diagram of a multi-platform live broadcast device according to the present invention.
The achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
As shown in fig. 1, fig. 1 is a schematic structural diagram of a hardware running environment related to a terminal device according to an embodiment of the present invention.
It should be noted that fig. 1 may be a schematic structural diagram of a hardware operating environment of a terminal device. The terminal equipment of the embodiment of the invention can be equipment for executing the multi-platform live broadcast method provided by the invention aiming at the storage equipment taking the nand f l ash as a storage medium, and the terminal equipment can be a mobile terminal, a data storage control terminal, a PC or a portable computer and other terminals.
As shown in fig. 1, the terminal device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a storage device 1005, and a communication bus 1002. Wherein the communication bus 1002 is used to enable connected communication between these components. The user interface 1003 may include a display (Di sp l ay), an input unit such as a Keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface, a wireless interface. The network interface 1004 may optionally include a standard wired interface, a wireless interface (e.g., WI-F I interface). The storage device 1005 may be a non-volatile storage device (e.g., an F I ash storage device), a high speed RAM storage device, or a stable storage device (non-vo l at i l e memory), such as a disk storage device. The storage 1005 may also optionally be a storage separate from the aforementioned processor 1001.
It will be appreciated by those skilled in the art that the terminal device structure shown in fig. 1 is not limiting of the terminal device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.
As shown in fig. 1, an operating system, a network communication module, a user interface module, and a computer program may be included in the storage device 1005, which is a type of computer storage medium. The operating system is a program for managing and controlling the hardware and software resources of the sample terminal device, and supports the running of computer programs and other software or programs.
In the terminal device shown in fig. 1, the user interface 1003 is mainly used for data communication with each terminal; the network interface 1004 is mainly used for connecting a background server and carrying out data communication with the background server; and the processor 1001 may be configured to invoke a computer program stored in the storage device 1005 and perform the following operations:
determining the resolution and the coding format supported by a target live broadcast platform to be pushed;
creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing;
and sending the encoded push data to a target live broadcast platform server based on the push module.
Further, the processor 1001 may call a computer program stored in the storage device 1005, and further perform the following operations:
receiving live broadcast audio and video data packaged and sent by a network transmission sending end of a network camera based on a network transmission receiving end;
unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and transmitting the plug flow data to a specified live broadcast platform server based on a plug flow module.
Further, the processor 1001 may call a computer program stored in the storage device 1005, and further perform the following operations:
after the step of unpacking the live audio and video data, the method further comprises the following steps:
and performing video decoding and video rendering on the coded video obtained by unpacking the live audio and video data, and performing audio decoding and audio playing on the coded audio obtained by unpacking the live audio and video data.
Further, the processor 1001 may call a computer program stored in the storage device 1005, and further perform the following operations: before the step of encapsulating the unpacked live audio and video data and the custom data added by the anchor into push data, the method further comprises the following steps:
and acquiring the rendered video, the played audio and the custom data added by the anchor.
Further, the processor 1001 may call a computer program stored in the storage device 1005, and further perform the following operations:
the step of the push application sending the push data to a specified live platform server based on a push module comprises the following steps:
and based on the push live interface layer, determining whether the push data is transmitted to a specified live broadcast platform server through a preset transmission protocol push module or a preset third-party live broadcast SDK.
Referring to fig. 2, fig. 2 is a flow chart of an embodiment of a multi-platform live broadcast method according to the present invention.
In this embodiment, the multi-platform live broadcast method of the present invention includes:
step S10, determining the resolution and the coding format supported by a target live platform to be pushed;
step S20, creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing;
and step S30, the encoded push data is sent to the target live broadcast platform server based on the push module.
As shown in fig. 3, the plug flow module has the following composition, and specifically includes the following modules: (1) live platform resource management. The plug flow parameter requirements of different platforms are saved, and the plug flow parameter requirements mainly comprise the following parameters: video code stream size, resolution size, audio code stream size, platform server URI, wherein the URI comprises a transmission protocol. Typical transmission protocols are RTMP, SRT, etc. And (2) detecting parameters of the live broadcast platform. The module mainly groups different platforms, wherein the grouping is based on resolution and coding format, and the resolution and the coding format are divided into a group. (3) The encoder management, namely the encoder management, manages different encoders needed by different platforms. (4) transmission module management. This is the module interfacing with the live platform, configuring different transport modules according to the transport protocol required by the push platform, supporting RTMP, SRT, webRTC and so on.
As shown in fig. 4, the conventional method has a significant disadvantage that each push is an independent encoder when the push is live, which increases the consumption of the CPU and the memory. Therefore, as shown in fig. 5, a live platform is newly added, and App needs to plug to the new platform; detecting the resolution and the coding format supported by the platform, and if no encoder meeting the conditions exists, creating an encoder; if a satisfactory encoder exists, media data is acquired from this encoder; and detecting a transmission protocol used by the platform, and sending the media data to the appointed platform through the transmission module.
Optionally, before the step of creating the corresponding encoder according to the resolution and the encoding format to perform encoding processing, the method further includes:
receiving live broadcast audio and video data packaged and sent by a network transmission sending end of a network camera based on a network transmission receiving end;
unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and carrying out coding processing on the plug flow data according to the resolution and the coder corresponding to the coding format.
The traditional plug flow mode has high cost when the plug flow is carried out, cloud service is needed, and a cloud virtual mobile phone cannot be directly operated by a user and cannot be registered and logged in, so that multi-platform live broadcast cannot be realized. Or the physical device is encapsulated and virtualized into a plurality of devices through a HAL (Hardware Abstract ion Layer ) layer. The drawbacks are that it is achievable in android systems but not in iOS (iPhone Operat ion System, apple mobile device operating system) and that the system HAL layer code needs to be modified. In addition, the live App (application software) generally only recognizes the front camera and the rear camera, and does not traverse all cameras for the user to select, which means that a plurality of live apps can access a specific camera or two cameras at the same time, and resource robbery exists, so that the method is not practical for realizing live broadcasting of a plurality of platforms. More commonly, a host buys a plurality of mobile phones to run different live apps, thereby realizing multi-platform live broadcast. The method has the defects of high cost and troublesome operation, and the shooting angles of a plurality of mobile phones are often different, so that the situation that the anchor is out of the way can occur.
The push application in this embodiment is an EMEET Stream App, which receives live audio and video data recorded during live broadcasting by a network transmission transmitting end of a network camera I PCamera, which is sent in a packaged manner by a network transmission receiving end. The plug-flow application can be installed on the network camera or the multimedia equipment in communication connection with the network camera, and the corresponding installation equipment has the functions of unpacking, packaging, sending and the like. The push application unpacks the direct broadcast audio and video data after receiving the direct broadcast audio and video data, packages the unpacked data and the user-defined data actively added by the anchor, such as various material data for creation, including audio, video, characters, pictures, moving pictures and the like, into push data, and sends the push data to a specified live platform server, such as a Twitch server and a fighting fish server through a push module.
Optionally, after the step of unpacking the live audio and video data, the method further includes:
and performing video decoding and video rendering on the coded video obtained by unpacking the live audio and video data, and performing audio decoding and audio playing on the coded audio obtained by unpacking the live audio and video data.
After receiving and decompressing Bao Zhibo audio and video data, the plug-flow application performs video decoding and video rendering on the encoded video obtained after the unpacking, and performs audio decoding and audio playing on the encoded audio, so that a host can watch own live broadcast pictures and sound during live broadcast.
Optionally, before the step of encapsulating the unpacked live audio and video data and the custom data added by the anchor into push data, the method further includes:
and acquiring the rendered video, the played audio and the custom data added by the anchor.
Because the live broadcast equipment end provided with the push application, such as a network camera, is configured, and the available material data for creation is limited, the push application is used for a host to carry out custom operations, such as registration, login and the like before live broadcast push, and the like, and the recorded live broadcast content is referred to for display during live broadcast to add material data, such as superimposed words, pictures and the like, after the creation processing is completed, the material data is pushed to a spectator. The host can self-define creation of the live content through live broadcast pictures and sounds on the push application, and the live content is created and processed on the push application before being sent to each live broadcast platform server during live broadcast. Therefore, in this embodiment, the video rendered after unpacking, the audio played, and the custom data added by the anchor based on the video and/or audio may be packaged as push data.
Optionally, the step of sending the push data to the specified live platform server by the push application based on a push module includes:
and based on the push live interface layer, determining whether the push data is transmitted to a specified live broadcast platform server through a preset transmission protocol push module or a preset third-party live broadcast SDK.
In this embodiment, the simultaneous live broadcasting of multiple platforms is supported by integrating respective live broadcasting platform SDKs (Software Deve l opment Ki t software development packages) and various transmission protocols, where the integrated live broadcasting platform SDKs include, but are not limited to, SDKs of systems such as PCs (Persona l Computer, personal computers), android, or ios.
In the embodiment, the network transmission receiving end receives live broadcast audio and video data packaged and sent by the network transmission sending end of the network camera; unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and transmitting the plug flow data to a specified live broadcast platform server based on a plug flow module. The push data is sent to the appointed live broadcast platform server through the push module of the push application, so that simultaneous live broadcast of multiple platforms can be realized without the cloud, modifying the system bottom code and adding physical equipment, the working cost of a host is effectively reduced, the operation is simplified, the working efficiency is improved, the consistency of pictures and sounds of different live broadcast platforms can be maintained, and the control can be performed in different operating systems.
Referring to fig. 6, fig. 6 is a system schematic diagram of an embodiment of a multi-platform live broadcast system according to the present invention.
In this embodiment, the multi-platform live broadcast system includes: the network camera, push application and live broadcast platform;
the network camera comprises a network transmission receiving end;
the push application receives live broadcast audio and video data packaged and sent by the network transmission sending end of the network camera based on the network transmission receiving end;
and the push application unpacks the live broadcast audio and video data, packages unpacked live broadcast audio and video data and host-added custom data into push data, and sends the push data to a specified live broadcast platform server based on the push module.
Optionally, the network camera further includes:
the lens module is used for collecting original video data;
the video coding module is used for coding the original video data to obtain coded video data;
the microphone array is used for collecting original audio data;
the audio coding module is used for coding the original audio data to obtain coded audio data;
the network transmission transmitting end is used for packaging the coded video data and the coded audio data into live audio and video data and transmitting the live audio and video data to the network transmission receiving end of the plug-flow application.
Optionally, the network transmission receiving end of the push application is configured to receive live audio and video data that is sent by the network transmission sending end of the network camera in a packaged manner, and unpack the live audio and video data into encoded video data and encoded audio data;
the plug-flow application further comprises:
the video decoding module is used for decoding the coded video data into original video data and performing video rendering;
the audio decoding module is used for decoding the encoded audio data into original audio data and playing the audio;
and the multiplexer is used for packaging the unpacked live broadcast audio and video data and the custom data added by the anchor into push stream data.
Optionally, the push application includes a network transmission receiving end and a push module, and the push module includes a push live interface layer, a preset transmission protocol push module and a preset third party live SDK;
and the push application is used for determining whether to send push data to a specified live broadcast platform server through the preset transmission protocol push module or the third-party live broadcast SDK based on the push live broadcast interface layer.
In this embodiment, the multi-platform live broadcast system may be divided into four parts.
Wherein the first part, the webcam. Comprising the following steps: (1) The lens module is mainly used for collecting original video data, namely RAW data, namely original data of converting captured light source signals into digital signals by a CMOS (Comp l ementary Meta l Oxi de Semi conductor ) or CCD (Charge-coup l ed Devi ce, charge coupled device) image sensor. The lens module comprises four parts: lens (l ens), sensor (sensor), FPC (F l exi b l ePr i ntedCi rcu it, flexible circuit board), DSP (Di gita l Si gna l Process i ng ). And (3) Lens: the lens is a lens, belongs to an optical device and mainly aims to collect light. Sensor: the sensor is an electronic device, the sensor collects ambient light and converts the ambient light into an electric signal, and then the electric signal is converted into a digital image signal through an I SP (Image Si gna l Process I ng ) and is output to a Digital Signal Processor (DSP) for processing and is converted into standard GRB, YUV and other format image signals. These RGB, YUV are RAW data. FPC: the flexible board is a flexible circuit board and is mainly used for being connected with other circuit boards. DSP: and the digital signal processor processes the image signal from the ISP and converts the image signal into format data such as GRB, YUV and the like. (2) The video coding module mainly codes GRB, YUV and other data generated by the DSP into H264, H265 and other video formats. The original video data is relatively large, the network bandwidth is limited, and the original video data needs to be compressed into a format suitable for network transmission, and the process is coding. (3) Microphone array, mainly responsible for pick-up, produces PCM (Pu l se Code Modu l at i on ) data. (4) Audio coding module, again, to reduce bandwidth consumption, it is necessary to encode audio data, which mainly serves to convert PCM into AAC format. (5) The network transmission transmitting end is responsible for packaging and transmitting the audio and video data according to the format agreed by the transmission protocol, and the protocols supported by the module include RTSP (Rea l T ime Streami ng Protoco l, real-time streaming protocol), RTMP (Rea l T ime Messagi ng Protoco l, real-time message transmission protocol), ND I (Network Dev I ce I nterface, network equipment interface), SRT (Secure Re l I ab l e Transport, safe and reliable transmission protocol) and the like.
Wherein the second part, EMEET Stream App. Comprising the following steps: (1) The network transmission receiving end receives the audio and video data from the network camera I PCamera, also supports RTSP, RTMP, ND I, SRT and other protocols, and simultaneously unpacks the received data into video frames (H264/H265 and other formats) and audio frames (AAC format). (2) And the video decoding module converts the video formats such as H264/H265 and the like into the formats such as RGB/YUV and the like for playing and displaying. (3) And the audio decoding module converts AAC into PCM data and outputs the PCM data to a loudspeaker for playing. (4) muxer packs and combines audio frame, video frame and host added custom data again, encapsulates according to agreed protocol or format, such as MP4 format, RTMP protocol, SRT protocol, etc., stores to local file of network camera or sends to live platform. (5) And the plug flow module is mainly used for sending the audio and video data to the appointed live broadcast platform server.
And the third part is a live broadcast platform. Which is a service program belonging to a third party. Generally, in order to meet high concurrency demands, live services in the cloud are service clusters. The live broadcast platform is responsible for receiving the audio and video contents of different anchor and forwarding the same to the audience.
The fourth part, the plug flow module comprises: (1) And the plug live interface layer unifies interfaces of different plug flow modes, thereby facilitating upper layer operation. As shown in fig. 7, li ve I interface is an interface layer, and TwitchLi ve, kuai shoulve, and RTMPLi ve are implementation layers, respectively implementing Twitch, fast-handed, and RTMP push live. Wherein TwitchLi ve and Kuai shoulLi ve encapsulate SDKs of a Twitch platform and a fast-handed live platform respectively. RTMPLi ve is a push module encapsulated according to the RTMP protocol. Li veFactoy is a module that creates a specific instance of Li ve I interface, i.e., the use of Twitch, fast-hand or RTMP modules is determined by the upper layer according to the actual situation. (2) The third party live broadcast SDK is the SDK provided by the live broadcast platform, and the SDK is integrated by an EMEET Stream App. According to the actual situation, such as Twitch, fast hand, li nked hn, etc. (3) And the RTMP plug flow module is packaged according to an RTMP protocol, and transmits the audio and video data to the live broadcast platform through the RTMP protocol, wherein the difference between the RTMP plug flow module and the SDK using the live broadcast platform is that the SDK is a private protocol and the RTMP is a public protocol. (4) RTMP service resource management, which manages different server resources supporting RTMP live broadcast platforms. Since a live platform usually has a plurality of servers deployed in different country regions (this is to allow users to access quickly), these server resources need to be maintained and updated periodically, and this module is to update these server resources in time, for example, the server address is changed, the RTMP push path is changed, and all these server resources need to be updated in time. (5) SRT plug flow module. The module packaged according to the SRT protocol transmits the audio and video data to the live broadcast platform through the SRT protocol, and is different from RTMP in that the SRT is processed more strongly than the RTMP in reliability, low delay and high quality. Reliability: the method is suitable for any network environment, and efficiently processes the interference such as network packet loss, jitter, bandwidth fluctuation and the like; low delay: due to the adoption of the UDP transmission mode and the use of an ARQ packet loss recovery mechanism, the low-delay transmission based on the public network can be realized; high quality: the transmission and error correction mechanism of the SRT can maximize and utilize the available bandwidth and remove network errors and interference, can transmit video streams with higher code rate under the same network environment, and can ensure the high quality of video under bad network conditions.
In addition, referring to fig. 8, an embodiment of the present invention further provides a multi-platform live broadcast apparatus, where the apparatus includes:
the determining module M1 is used for determining the resolution and the coding format supported by the target live platform to be pushed;
and the encoding module M2 is used for creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing.
And the transmission module M3 is used for transmitting the encoded push data to the target live broadcast platform server based on the push module.
Optionally, the multi-platform live broadcast device further comprises a unpacking module, which is used for receiving live broadcast audio and video data packaged and sent by a network transmission sending end of the network camera based on the network transmission receiving end;
unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and carrying out coding processing on the plug flow data according to the resolution and the coder corresponding to the coding format.
Optionally, the unpacking module is further configured to perform video decoding and video rendering on the encoded video obtained by unpacking the live audio-video data, and perform audio decoding and audio playing on the encoded audio obtained by unpacking the live audio-video data.
Optionally, the unpacking module is further configured to obtain the rendered video, the played audio, and the host added custom data.
Optionally, the unpacking module is further used for
And based on the push live interface layer, determining whether the push data is transmitted to a specified live broadcast platform server through a preset transmission protocol push module or a preset third-party live broadcast SDK.
The multi-platform live broadcast device provided by the invention adopts the multi-platform live broadcast method in the embodiment, and solves the technical problems of high memory occupation, high power consumption and easy hardware loss of the multi-platform live broadcast method in the prior art. Compared with the prior art, the multi-platform live broadcast device provided by the embodiment of the invention has the same beneficial effects as the multi-platform live broadcast method provided by the embodiment, and other technical features in the multi-platform live broadcast device are the same as the features disclosed by the method of the embodiment, and are not repeated herein.
In addition, the embodiment of the invention also provides multi-platform live broadcast equipment, which comprises: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the multi-platform live method as claimed in any one of the preceding claims.
In addition, the embodiment of the invention also provides a computer storage medium, and a computer program is stored on the computer storage medium, and when the computer program is executed by a processor, the steps of the multi-platform live broadcast method are realized.
The steps implemented when the computer program running on the processor is executed may refer to various embodiments of the multi-platform live broadcast method of the present invention, which are not described herein.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing embodiment numbers of the present invention are merely for the purpose of description, and do not represent the advantages or disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment. Based on such understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art in the form of a software product stored in a computer storage medium (such as an F l ash storage device, a ROM/RAM, a magnetic disk, an optical disc), comprising instructions for causing a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.), a controller for controlling the storage medium to perform the method according to the embodiments of the present invention.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.
Claims (11)
1. The multi-platform live broadcast method is characterized by being applied to plug-flow application and comprises the following steps of:
determining the resolution and the coding format supported by a target live broadcast platform to be pushed;
creating a corresponding encoder according to the resolution and the encoding format to carry out encoding processing;
and sending the encoded push data to a target live broadcast platform server based on the push module.
2. The multi-platform live broadcast method of claim 1, further comprising, prior to the step of creating a corresponding encoder for encoding processing based on the resolution and encoding format:
receiving live broadcast audio and video data packaged and sent by a network transmission sending end of a network camera based on a network transmission receiving end;
unpacking the live audio and video data, packaging the unpacked live audio and video data and the custom data added by the anchor into plug flow data, and carrying out coding processing on the plug flow data according to the resolution and the coder corresponding to the coding format.
3. The multi-platform live broadcast method of claim 2, wherein after the step of unpacking the live audio-video data, further comprising:
and performing video decoding and video rendering on the coded video obtained by unpacking the live audio and video data, and performing audio decoding and audio playing on the coded audio obtained by unpacking the live audio and video data.
4. The multi-platform live broadcast method of claim 2, wherein before the step of encapsulating the unpacked live audio-video data and the host added custom data into plug-flow data, further comprising:
and acquiring the rendered video, the played audio and the custom data added by the anchor.
5. The multi-platform live broadcast method of claim 2, wherein the step of transmitting the push-based data to a designated live broadcast platform server based on a push module comprises:
and based on the push live interface layer, determining whether the push data is transmitted to a specified live broadcast platform server through a preset transmission protocol push module or a preset third-party live broadcast SDK.
6. A multi-platform live broadcast system, the multi-platform live broadcast system comprising: the network camera, push application and live broadcast platform;
the network camera comprises a network transmission receiving end;
the plug flow application comprises a network transmission receiving end and a plug flow module;
the push application receives live broadcast audio and video data packaged and sent by the network transmission sending end of the network camera based on the network transmission receiving end;
and the push application unpacks the live broadcast audio and video data, packages unpacked live broadcast audio and video data and host-added custom data into push data, and sends the push data to a specified live broadcast platform server based on the push module.
7. The multi-platform live broadcast system of claim 6, wherein the webcam further comprises:
the lens module is used for collecting original video data;
the video coding module is used for coding the original video data to obtain coded video data;
the microphone array is used for collecting original audio data;
the audio coding module is used for coding the original audio data to obtain coded audio data;
the network transmission transmitting end is used for packaging the coded video data and the coded audio data into live audio and video data and transmitting the live audio and video data to the network transmission receiving end of the plug-flow application.
8. The multi-platform live broadcast system of claim 6, wherein the network transmission receiving end of the push application is configured to receive live audio and video data packaged and sent by the network transmission sending end of the network camera, and unpack the live audio and video data into encoded video data and encoded audio data;
the plug-flow application further comprises:
the video decoding module is used for decoding the coded video data into original video data and performing video rendering;
the audio decoding module is used for decoding the encoded audio data into original audio data and playing the audio;
and the multiplexer is used for packaging the unpacked live broadcast audio and video data and the custom data added by the anchor into push stream data.
9. The multi-platform live broadcast system of claim 6, wherein the push module comprises a push live broadcast interface layer, a preset transport protocol push module, and a preset third party live broadcast SDK;
and the push application is used for determining whether to send push data to a specified live broadcast platform server through the preset transmission protocol push module or the third-party live broadcast SDK based on the push live broadcast interface layer.
10. A multi-platform live broadcast device, the multi-platform live broadcast device comprising: memory, a processor and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the multi-platform live method of any of claims 1 to 5.
11. A computer storage medium, characterized in that the computer storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the multi-platform live method according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211426714.XA CN116074544A (en) | 2022-11-15 | 2022-11-15 | Multi-platform live broadcast method, system, equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211426714.XA CN116074544A (en) | 2022-11-15 | 2022-11-15 | Multi-platform live broadcast method, system, equipment and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116074544A true CN116074544A (en) | 2023-05-05 |
Family
ID=86172331
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211426714.XA Pending CN116074544A (en) | 2022-11-15 | 2022-11-15 | Multi-platform live broadcast method, system, equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116074544A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116567356A (en) * | 2023-06-07 | 2023-08-08 | 佛山市炫新智能科技有限公司 | Live broadcast display system based on multi-platform push stream and push method thereof |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105491387A (en) * | 2015-12-04 | 2016-04-13 | 武汉斗鱼网络科技有限公司 | H264 hardware coding system based on three platforms of Intel, AMD and Nvidia |
| US20170187986A1 (en) * | 2015-12-23 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Live streaming media method, publish side live, server and terminal |
| CN107682712A (en) * | 2017-09-30 | 2018-02-09 | 安徽联智创新软件有限公司 | A kind of net cast network plug-flow management system |
| CN108777818A (en) * | 2018-06-15 | 2018-11-09 | 成都睿想腾飞科技有限公司 | A method of live streaming software support while multichannel plug-flow |
| CN110798698A (en) * | 2019-11-25 | 2020-02-14 | 盈动电子竞技(海南自贸区)有限公司 | Multi-server stream pushing method and equipment for live broadcast application program |
| CN111064972A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live video control method based on IPV9 |
| CN111064973A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live broadcast system based on IPV9 |
| CN112929681A (en) * | 2021-01-19 | 2021-06-08 | 广州虎牙科技有限公司 | Video stream image rendering method and device, computer equipment and storage medium |
-
2022
- 2022-11-15 CN CN202211426714.XA patent/CN116074544A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105491387A (en) * | 2015-12-04 | 2016-04-13 | 武汉斗鱼网络科技有限公司 | H264 hardware coding system based on three platforms of Intel, AMD and Nvidia |
| US20170187986A1 (en) * | 2015-12-23 | 2017-06-29 | Le Holdings (Beijing) Co., Ltd. | Live streaming media method, publish side live, server and terminal |
| CN107682712A (en) * | 2017-09-30 | 2018-02-09 | 安徽联智创新软件有限公司 | A kind of net cast network plug-flow management system |
| CN108777818A (en) * | 2018-06-15 | 2018-11-09 | 成都睿想腾飞科技有限公司 | A method of live streaming software support while multichannel plug-flow |
| CN110798698A (en) * | 2019-11-25 | 2020-02-14 | 盈动电子竞技(海南自贸区)有限公司 | Multi-server stream pushing method and equipment for live broadcast application program |
| CN111064972A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live video control method based on IPV9 |
| CN111064973A (en) * | 2019-11-28 | 2020-04-24 | 湖北工业大学 | Live broadcast system based on IPV9 |
| CN112929681A (en) * | 2021-01-19 | 2021-06-08 | 广州虎牙科技有限公司 | Video stream image rendering method and device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| "EMEETSTREAM版本0.1.6(build 2)", pages 1 - 3, Retrieved from the Internet <URL:https://www.pgyer.com/T8Av> * |
| XIEJIASHU: ""OBS（Open Broadcaster Software）桌面视频直播软件、直播推流工具使用方法指南", pages 1 - 13, Retrieved from the Internet <URL:https://blog.csdn.net/xiejiashu/article/details/68483758> * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116567356A (en) * | 2023-06-07 | 2023-08-08 | 佛山市炫新智能科技有限公司 | Live broadcast display system based on multi-platform push stream and push method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107846633B (en) | Live broadcast method and system | |
| CN109889543B (en) | Video transmission method, root node, child node, P2P server and system | |
| CN106992959B (en) | 3D panoramic audio and video live broadcast system and audio and video acquisition method | |
| CN110740363A (en) | Screen projection method and system and electronic equipment | |
| KR100810223B1 (en) | System and method for providing live streaming service between terminals | |
| TWI435568B (en) | Method and system for multimedia audio video transfer | |
| WO2014139423A1 (en) | Live system, method based on mobile terminal and mobile terminal | |
| CN111092898B (en) | Message transmission method and related equipment | |
| JP6377784B2 (en) | A method for one-to-many audio-video streaming with audio-video synchronization capture | |
| CN102447875A (en) | Method and system for centralized monitoring of video session terminals and relevant devices | |
| CN110996122B (en) | Video frame transmission method, device, computer equipment and storage medium | |
| CN110062268A (en) | A kind of audio-video sends and receives processing method and processing device with what screen played | |
| CN104185082A (en) | Audio/video live broadcast method and system, client and server | |
| CN114339382A (en) | Video playing method, device, equipment and computer storage medium | |
| CN116074544A (en) | Multi-platform live broadcast method, system, equipment and medium | |
| US20160352798A1 (en) | Systems and methods for capture and streaming of video | |
| CN114630051A (en) | Video processing method and system | |
| CN107276990B (en) | Streaming media live broadcasting method and device | |
| KR101701742B1 (en) | Apparatus and method for live streaming between mobile communication terminals | |
| CN111510720A (en) | Real-time streaming media data transmission method, electronic device and server | |
| MX2007002241A (en) | Method and apparatus for signaling transmission code set in data broadcasting. | |
| CN108809924A (en) | The method and apparatus for executing real-time Communication for Power Network | |
| US9838463B2 (en) | System and method for encoding control commands | |
| WO2023216798A1 (en) | Audio and video transcoding apparatus and method, and device, medium and product | |
| CN110602065A (en) | Live broadcast stream pushing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |