WO2021204141A1

WO2021204141A1 - Video live-streaming control, bridging, stream control and broadcast control methods, and client

Info

Publication number: WO2021204141A1
Application number: PCT/CN2021/085783
Authority: WO
Inventors: 王斐航; 林忠涛
Original assignee: 广州华多网络科技有限公司
Priority date: 2020-04-10
Filing date: 2021-04-07
Publication date: 2021-10-14
Also published as: CN111510780B; CN111510780A

Abstract

A video live-streaming control, bridging, stream control and broadcast control methods, and a client. Said control method comprises: when a live-streaming program is started by a simulation program which is running in a current first operating system and used for simulating a running environment of a second operating system, establishing a communication channel between the live-streaming program and a stream control program running in the first operating system; after the live-streaming program starts a preview mode, broadcasting a live-streaming-start preview message, and the simulation program notifying a stream control program of the live-streaming-start preview message, so as to trigger the stream control program to enter a preview state and to open a preview window to display a video stream; and after the live-streaming program starts a live-streaming-start mode, broadcasting a cloud address message which is acquired by the live-streaming program and used for receiving the video stream, and the simulation program notifying the stream control program of the cloud address message, so as to trigger the stream control program to enter a live-streaming-start state and to transmit the video stream to a cloud address. The present invention can prevent program personnel from developing different live-streaming programs for different operating systems, and can ensure the image quality of video streams collected by video live streaming.

Description

Video live broadcast control, bridging, flow control, broadcast control method and client

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 10, 2020, the application number is 202010281352.4, and the invention title is "Video Live Control, Bridge, Stream Control, Broadcast Control Method and Client", all of which The content is incorporated in this application by reference.

Technical field

The present invention relates to computer system communication technology, in particular to a video live broadcast control method and a corresponding client, and correspondingly relates to a video direct bridge method, a video live streaming control method, and a video live broadcast control method during the execution of the control method.

Background technique

Although the technology of running an application program developed for another operating system through an emulator in one operating system is very common, there are still many technical obstacles in some special application scenarios. In a very common scenario, when it involves the need to exchange large-volume audio and video data with the cloud, due to some efficiency issues involved in cross-system interaction, unsatisfactory situations often occur.

For example, in the most basic application situation, when the Internet video live broadcast platform releases the launching tool, in order to ensure that the anchors of each platform can be covered, three applications are usually released: PC launching application, Android launching application, IOS launching application. These three applications run on windows system computers, Android phones and Apple phones respectively.

Most of the business functions of these three applications are exactly the same, but because the development languages and development environments of these three applications are completely different, the same business scenario requires PC R&D, Android R&D, and IOS R&D to be developed separately. And in the later maintenance process, it also needs the research and development of three platforms to maintain the application separately, which will cause unnecessary waste of manpower.

In this context, in order to save the PC part of the construction period, the developer will choose to let the anchor install a customized emulator on the PC, first establish an environment suitable for Android App running on the PC machine, and then use the emulator The Android launch tool is installed inside, so that R&D only needs to develop the Android launch application to meet the needs of the PC and Android platforms.

Although this solution saves the manpower of PC research and development, it also has some problems. In actual operation, it is found that because the preview page of the start-up is not completely collected and displayed on the PC side, some models will have problems such as blurred screens and freezes. , The performance of the corresponding live broadcast room is also very poor, which seriously affects the user experience of the live broadcast.

In theory, it is not limited to the Windows operating system on the PC side and the Android operating system on the mobile phone, as long as it involves running an application developed for another operating system in one operating system, especially when it comes to application requirements such as live broadcasting. May encounter the same or similar situations.

Summary of the invention

The present invention provides a live video control method, a live video bridge bridging method, a live video streaming control method, and a live video broadcast control method for the first purpose of avoiding repeated development.

The second object of the present invention is to provide a client terminal, so as to realize hardware support for the first object and the method.

In order to meet the various objectives of the present invention, the present invention provides the following technical solutions:

The video live broadcast control method of the present invention includes the following steps:

When the simulation program running in the current first operating system for simulating the operating environment of the second operating system starts the live program, the simulation program establishes the live program running in the operating environment and the live program running in the first operating system. The communication channel of the flow control program;

After the live broadcast program starts the preview mode, the broadcast start preview message is broadcast, and the simulation program notifies the flow control program of the start preview message to trigger it to enter the preview state, so that it opens a preview window to display the video stream it collects;

After the live broadcast program starts the broadcast mode, it broadcasts the cloud address message it has acquired for receiving the video stream, and the simulation program notifies the cloud address message to the streaming control program to trigger it to enter the broadcast state and make it broadcast to the corresponding The cloud address to transmit the video stream.

In a further embodiment, the method further includes the following steps:

When the live broadcast program is closed, the broadcast preview stop message, the simulation program notifies the stream control program of the preview stop message to trigger it to enter the stop broadcast state, and stop pushing the video stream to the cloud address.

In a preferred embodiment, the communication channel includes a first direct channel between a live program and a simulation program and a second direct channel between the simulation program and the flow control program.

In some embodiments, the first direct channel is established as follows: the live program carries service components registered in the simulation running environment, and the simulation program carries broadcast receivers registered in the simulation running environment The simulation program transmits the messages generated by the flow control program to the live program through the service process constructed by the service component; the simulation program receives the messages broadcast by the live program through the broadcast receiver and transmits them Give the described flow control program.

In some embodiments, the second direct channel is established as follows: the second direct channel is established between the simulation program and the flow control program by reading and writing sockets or files with each other.

In a preferred embodiment, the simulation program and the flow control program are suitable for running on the current first operating system, the live broadcast program is suitable for running on a second operating system that is different from the current operating system, and the simulation program is suitable for Simulate the second operating system in the first operating system to create a corresponding simulated running environment for the live program.

In some embodiments, the first operating system is any one of Windows or operating systems based on Linux and Unix, and the second operating system is Android, IOS, Linux, Windows that are different from the first operating system. Any of the operating systems.

In a preferred embodiment, in the process of establishing the communication channel by the simulation program, the application package name representing the main process of the live program and the service name representing its service process are passed to the flow control program for all The flow control program identifies the main process and the service process of the live broadcast program to determine the identity of message delivery.

In some embodiments, when the flow control program enters the preview state, a tool container for controlling the video stream collected by the flow control program is opened. The tool container includes at least one control that can affect the special effects of the preview video in the preview window. The message generated from the control is transmitted to the live broadcast program through the communication channel.

In a preferred embodiment, the preview window is closed when the flow control program enters the stop state.

A method for bridging video live broadcast of the present invention includes the following steps:

Create a second operating system simulation operating environment that is different from the current first operating system for running live broadcast programs;

Establishing a communication channel between the live broadcast program and the flow control program running in the current first operating system;

Forward the start preview message broadcast by the live broadcast program to the flow control program via the communication channel, and trigger it to enter the preview state so as to preview the video stream it collects;

Through the communication channel, forward the cloud address message broadcast by the live broadcast program to the flow control program, triggering it to enter the on-air state, so as to push the video stream to the cloud address.

In a further embodiment, the method further includes the following steps:

Via the communication channel, forward the preview stop broadcast message broadcast by the live broadcast program to the stream control program, triggering it to enter the stop broadcast state, so as to end the push of the video stream.

In a preferred embodiment, the communication channel includes a first direct channel between the simulation program and the live program implemented by the method, and a second direct channel between the simulation program and the flow control program.

A video live streaming method of the present invention includes the following steps:

Receiving a broadcast preview message forwarded by a simulation program used to provide a simulated running environment of the second operating system, start a preview window to display the video stream collected by the current bottom layer driver of the first operating system, and enter the preview state;

Receiving the message containing the cloud address transmitted by the simulation program, entering the on-air state, and pushing the video stream to the cloud address;

Receive the preview stop broadcast message delivered by the simulation program, enter the stop broadcast state, and end the push of the video stream.

In some embodiments, when entering the preview state, a tool container for controlling the video stream is opened, and the tool container includes at least one control that can affect the special effect of the preview video in the preview window.

In some embodiments, when entering the stop state, the preview window is closed.

In a more preferred embodiment, the flow control program that implements this method and the simulation program realize communication by reading and writing sockets or files with each other, so as to realize the transmission of the message.

In some embodiments, the flow control program that implements the method sends a notification message to the simulation program, so that the notification message is displayed in the simulation running environment created by the simulation program.

A method for controlling live video broadcasting of the present invention includes the following steps:

In response to the start preview mode command acting on the application in the second operating system simulation running environment, triggering the display of the preview window in the first operating system that provides the simulation running environment to display the commands via the underlying driver of the first operating system The captured video stream;

In response to the start-up mode instruction acting on the application program, triggering the first operating system to call the remote interface to push the video stream to the cloud address obtained and provided by the application program;

In response to the preview stop instruction acting on the application program, the bottom layer driver of the first operating system is triggered to stop collecting the video stream.

In a further embodiment, the first operating system runs a simulation program for creating the operating environment of the second operating system module, and the first operating system runs a first-level control program for capturing video from the underlying driver The action of streaming and the execution of the action of pushing the video stream.

In some embodiments, the live broadcast program that implements the method sends a message in the broadcast mode of the broadcast mechanism of the second operating system to implement the trigger operation on the first operating system.

In some embodiments, the live program that implements the method carries a service component for receiving messages generated in the first operating system, so as to display the messages generated in the first operation to the user.

In a preferred embodiment, the first operating system is any one of Windows or operating systems based on Linux and Unix, and the second operating system is Android, IOS, Linux, and Linux, which are different from the first operating system. Any of the Windows operating systems.

A client of the present invention includes a central processing unit and a memory, and the central processing unit is used to call and run one or more computer programs stored in the memory to execute the various types of the first object of the present invention as described above. Provided method.

Compared with the prior art, the advantages of the present invention are as follows:

In the live video control method of the present invention, the flow control program running on the first operating system is responsible for collecting and pushing video streams to realize flow control, and the simulation program running on the first operating system is responsible for creating a simulation operating environment of the second operating system to load and run. The live program is also responsible for the message transfer between the live program and the flow control program to realize the bridge, and the live program assumes the role of the user control terminal to realize the broadcast control. As a result, the live program developed for the second operating system can run on the second operating system. In an operating system, the live program is responsible for controlling the playback status. With the aid of the bridge function of the simulation program to achieve message transmission, the flow control program can call the underlying driver interface of the first operating system to collect the video stream and directly push it to the live program. With the provided cloud address, the video stream does not need to be transferred through the simulation program and the live program, which can greatly improve the push efficiency of the video stream. When the video stream is received by the cloud server and pushed to the terminal device of the user in the live broadcast room, the user can obtain high fidelity Video stream to obtain a clearer video viewing effect. Accordingly, developers do not need to adapt to the mechanism and development language of the first operating system to re-develop and maintain another version of the live program, which can greatly save program development consumption and development cost.

The simulation program implemented by the video live streaming bridge method of the present invention can not only simulate the operating environment of the second operating system in the first operating system, so as to ensure that the live program is called and run normally, but also, through the establishment of the live program and The communication channel between flow control programs realizes the message bridging mechanism, which in fact creates the basic conditions for the separation of video streaming and live broadcast control. By extension, as long as the relevant application complies with the interface specification of the simulation program, the simulation program It can serve it and provide a docking channel between the flow control programs. Therefore, the bridging method is equivalent to providing applications of different operating systems running on the first operating system and enabling separate collection and push of video streams. The service mechanism is convenient for Internet developers to adapt and call, and it is convenient to be compatible with the application programs of the second operating system developed by different developers.

The flow control program implemented by the video live streaming method of the present invention runs on the first operating system and is specifically responsible for the collection, push and preview of the video stream, and can be used for the second operation through the simulation program through the corresponding communication channel. The live program developed by the system provides the service of pushing the video stream, which is convenient for the anchor users to watch and operate on the first operating system (usually a PC), which is more convenient to control and obtain better preview effects. In addition, since it complies with the mechanism of the communication channel with the simulation program, for example, it realizes message transfer with the simulation program through socket or file reading and writing, which is actually equivalent to providing a standard interface , It is convenient to serve different simulation programs, and it is convenient to access virtual machines developed by different developers for the second operating system, so that it is convenient to match different virtual machines to be compatible with more types of second operating systems on the first operating system. Live programs and provide push streaming services for them.

The live program implemented by the video live broadcast control method of the present invention can access the simulation program running on the first operating system of the present invention as long as the relevant response and message broadcasting is implemented according to the method, and run on the simulation program created by the simulation program. In the simulated operating environment of, it realizes the effect of transferring its own streaming function to the first operating system for previewing, collecting and pushing. Therefore, in theory, the live program can also be developed lightly, and it does not necessarily need to provide a complete preview. , Capture and push video stream related functions. Of course, in actual development, developers often provide full-featured live broadcast programs. In any case, the live broadcast programs developed for the second operating system can access the simulation programs as long as they comply with the specifications of this broadcast control method. , And it runs virtually in the first operating system, which realizes function expansion and facilitates the operation of anchor users.

The additional aspects and advantages of the present invention will be partly given in the following description, which will become obvious from the following description, or be understood through the practice of the present invention.

Description of the drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only It is an embodiment of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the provided drawings without creative work.

FIG. 1 is a schematic flowchart of a typical embodiment of the video live broadcast control method of the present invention;

FIG. 2 is a schematic flowchart of a typical embodiment of the video live streaming bridge method of the present invention;

FIG. 3 is a schematic flowchart of a typical embodiment of a video live streaming method of the present invention;

FIG. 4 is a schematic flowchart of a typical embodiment of a method for controlling live video broadcasting according to the present invention.

Detailed ways

The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are only used to explain the present invention, but cannot be construed as limiting the present invention.

Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present invention refers to the presence of the described features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and/or groups of them. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may also be present. In addition, “connected” or “coupled” used herein may include wireless connection or wireless coupling. The term "and/or" as used herein includes all or any unit and all combinations of one or more associated listed items.

Those skilled in the art can understand that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present invention belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with the meaning in the context of the prior art, and unless specifically defined as here, they will not be idealized or overly Explain the formal meaning.

Those skilled in the art can understand that the "client", "terminal", and "terminal device" used herein include both wireless signal receiver devices, which only have wireless signal receiver devices without transmitting capabilities, and include A device for receiving and transmitting hardware, which has a device capable of receiving and transmitting hardware for two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication) System), which can combine voice, data processing, fax and/or data communication capabilities; PDA (Personal Digital Assistant), which can include radio frequency receivers, pagers, Internet/Intranet access, web browsers, and notes Notebook, calendar, and/or GPS (Global Positioning System) receiver; conventional laptop and/or palmtop computer or other equipment, which has and/or includes a conventional laptop and/or radio frequency receiver Palm-size computer or other equipment. The "client", "terminal", and "terminal device" used here may be portable, transportable, installed in vehicles (air, sea and/or land), or suitable and/or configured to be locally Operate, and/or in a distributed form, operate in any other location on the earth and/or space. The "client", "terminal", and "terminal device" used here can also be communication terminals, internet terminals, music/video playback terminals, such as PDA, MID (Mobile Internet Device, mobile Internet device) and/or Mobile phones with music/video playback functions can also be devices such as smart TVs and set-top boxes.

The hardware referred to by the names "server", "client", "service node" and so on in the present invention is essentially a device with the equivalent capabilities of a personal computer, which is a central processing unit (including arithmetic unit and controller) , Memory, input equipment, output equipment and other necessary components disclosed by the von Neumann principle. The computer program is stored in its memory. The central processing unit transfers the program stored in the external memory into the memory to run and execute the program. The command interacts with input and output devices to complete specific functions. The concept of "server" referred to in the present invention can also be extended to the situation of server clusters in the same way.

The present invention is proposed for the application scenario of the host side of live video. In this scenario, the host uses a personal computer installed with a first operating system, such as a Windows system, as a client, and runs the video according to the present invention on the desktop. The simulation program implemented by the live bridge method and the flow control program implemented according to the video live streaming control method of the present invention, in the interface formed after the simulation program runs, open the video live broadcast control method according to the present invention, which is suitable for running in the first Second, the live program in the operating system can use the live program as a controller to control the flow control program to collect and preview the video stream, and further can operate the live program to drive the flow control program to push the video stream to the cloud server, which is pushed by the server For users in the live broadcast room to watch.

It should be understood that in such a scenario, the video stream referred to in the present invention, unless otherwise selectively controlled by the host user, generally can include two types of stream data, video and audio, and these video streams may not be collected by the underlying driver of the client. The original data means that the anchor user can edit and process the video and/or audio with the special effects tools provided by the flow control program, so that the flow control program will finally push the edited and processed video stream to The cloud server.

Both the first operating system and the second operating system involved in the present invention can be various currently known operating systems or unknown operating systems, including operating systems suitable for various mobile terminals and personal computers. Usually, the first operating system and the second operating system are different operating systems, so the bridging is just achieved through the simulation program. According to the current operating system ecology, the first operating system of the present invention recommends using any one of Windows or Linux and Unix-based operating systems, and the second operating system is Android, which is different from the first operating system. Any one of IOS, Linux, Windows operating system. For example, in various embodiments disclosed in the subsequent disclosure of the present invention, for the convenience of description, the first operating system can be set as the Windows operating system and the second operating system is the Android operating system. However, it should not be understood that the present invention The second operating system excludes other operating systems including IOS.

Although each program referred to in the present invention adopts a static expression, those skilled in the art should understand that when the program runs on a personal computer or related equipment, it appears as a dynamic process in the memory, and the function of the process is realized, It is the code of the program that can be executed in the computer, so this correspondence between the program and the process should not constitute an obstacle that affects the spirit of the invention and the scope of protection of the present invention.

It should be pointed out that the embodiments of the various methods to be disclosed in the present invention are all different parts or different forms obtained based on the same inventive principle. Therefore, unless expressly stated otherwise, those skilled in the art should not rely on the differences in the various embodiments. Separate explanations between the two and mechanically divide the relevance between the various embodiments.

Please refer to FIG. 1. In a typical embodiment of the video live broadcast control method of the present invention, the method includes the following steps:

Step S11, when the simulation program running in the current first operating system for simulating the operating environment of the second operating system starts the live program, the simulation program establishes the live program running in the operating environment and runs in the first operation The communication channel of the flow control program in the system:

As mentioned above, the first operating system of the present invention runs, for example, the Windows operating system, and the second operating system may be, for example, the Android operating system. In the mobile terminal. The flow control program and the simulation program are both developed by the Windows operating system. The said flow control program is actually equivalent to opening a service, which can collect video streams and realize video preview in the desktop window of the first operating system, and can push the collected video streams to the cloud with a designated cloud address. At the server, so that the cloud server can push the video stream to the relevant participating users in the live broadcast room controlled by the host user.

Usually, the anchor user will run the flow control program and the simulation program first, and when the developer develops, he can also integrate the simulation program and the flow control program into the same program, so that they can call each other associatively, simplifying the anchor user Start operation.

When the flow control program is initially started, it is in a stopped state by default, so there is no need to open the preview window (which can also be opened) and the corresponding tool container, and it can run in the background.

When the simulation program is started, it is responsible for creating the simulation running environment of the second operating system, which actually functions as the operating system virtual machine, and finally provides a second operating system operation interface, which is convenient for the anchor user to install and start it. , Close and uninstall related applications, including the live broadcast program.

The simulation program establishes the communication channel in accordance with the protocol mechanism preset by the developer during or after the start-up process. The communication channel includes a first direct channel between a live program and a simulation program and a second direct channel between the simulation program and the flow control program.

The first direct channel is mainly to maintain the message transmission between the simulation program and the live program, including two-direction message transmission, that is, the message channel from the simulation program to the live program and the message channel from the live program to the simulation program.

The message channel through which the simulation program transmits messages to the live broadcast program is realized by using the service components carried by the live broadcast program. During the development of the live broadcast program, the code related to the service component is embedded, and the service component has been declared in its Androidmanifest.xml configuration information file. Therefore, when the live broadcast program is running, it will be parsed and registered in the simulation running environment to construct A service process. The simulation program implements binding with the service process and maintains communication with it to realize inter-process communication, and can send messages to the live program through the service process, usually by forwarding and transmitting the messages generated by the flow control program.

The message channel through which the live broadcast program transmits messages to the simulation program implements message broadcast by using the broadcast mechanism of the second operating system itself, such as the broadcast mechanism in Android, by the live broadcast program. In the simulation program, a broadcast receiver is pre-declared in its Androidmanifest.xml file. This broadcast interface device can be used to filter and receive various messages broadcast by the live broadcast program. The live program uses this mechanism to include the messages that need to be broadcast into the intent to achieve broadcast. The simulation program filters and receives the messages broadcast by the live program through its own broadcast receiver, and then can deliver it to all The flow control program described.

The second channel is mainly to maintain the message transfer between the simulation program and the flow control program running on the client. Since both the simulation program and the flow control program run in the same operating system, the two can adopt a simpler way to achieve message transmission in the same way. For example, in the Windows operating system, between the simulation program and the flow control program, the socket mechanism or the file read and write mechanism of the Windows system can be used to read and write the socket or file, one write and one read , To achieve message delivery.

It can be seen that after the above process, an end-to-end communication channel has been established between the live program, the simulation program, and the flow control program.

The simulation program can theoretically run multiple applications developed for the second operating system. Therefore, in theory, it can be allowed to simulate multiple processes in the operating environment, in order to avoid confusion caused by multiple processes sending messages at the same time, or to clarify these processes As any one of them is the recipient of the message, you can consider letting the parties in the communication channel identify the identities of these processes. Therefore, in a modified embodiment, in the process of establishing the communication channel, the simulation program may further pass the application package name of the live program on behalf of its main process and the service name on behalf of its service process to The flow control program enables the flow control program to identify the sending identity of the message to be transmitted, and it can also clarify the receiving identity of the object whose message needs to be transmitted. Therefore, when the flow control program receives a live program message from the simulation program through a socket or file, its legitimacy can be determined based on the application package name and service name; and when the flow control program needs to broadcast to a specific live program When the program sends a message, it can specify the recipient in the message to be delivered, so that the simulation program can correctly deliver the message to the corresponding main process or service process. It can be seen that by allowing the flow control program to hold the application package name and service name of the live program, it is beneficial for the flow control program to identify the main process and service process of the live program to determine the message delivery identity.

Step S12: After the live broadcast program starts the preview mode, the broadcast preview message is broadcast, and the simulation program notifies the flow control program of the broadcast preview message to trigger it to enter the preview state, so that it opens the preview window to display the video stream it collects:

After the live broadcast program is started, the simulation program has also established the communication channel. Therefore, barrier-free communication between the live broadcast program and the flow control program can be realized through the simulation program, and the anchor user can perform regular routines. Live broadcast operation.

According to a general live broadcast control program, an entry or control or other control method related to the start of the preview mode will be provided on the live program interface, or directly when the user switches to a certain page, it is regarded as the start of the preview mode.

No matter how the preview instruction is sent to prompt the live broadcast program to enter the start preview mode, the live broadcast program will be triggered in this mode, taking the method in the Android system as an example, construct an intent, and run it in the simulation Broadcasting a message in the environment that indicates that the anchor user needs to start the broadcast preview is called the start broadcast preview message.

According to the operating system principle of Android, the broadcast receiver registered by the simulation program will receive the broadcast preview message, and after filtering the reception, it can write it to the relevant socket in accordance with the rules and format agreed with the flow control program Or in the file.

Correspondingly, the flow control program reads the on-air preview message from the socket or file according to the method pre-agreeed with the simulation program, and is triggered by it to switch itself into the preview state. In the preview state, open a preview window for displaying images in the current window or by creating a new window. At the same time, the flow control program will also control the camera and audio circuit to collect the video stream for it by calling the underlying driver of the first operating system. Unless the host user has a setting that prohibits immediate display for safety reasons, the video stream will be synchronized. Displayed in the preview window.

It can be understood that since the flow control program itself is a program developed for the first operating system, it does not need to be converted by different operating system mechanisms. Therefore, it will have a higher video stream acquisition efficiency, ensure the video stream acquisition quality, and minimize the time The impact of problems such as delay and dropped frames.

Generally, taking into account the needs of the host user, in a modified embodiment, after entering this preview state, it can be further displayed along with the call of the preview window, and the synchronous account opening is used to control the video stream collected by the flow control program. The tool container is usually represented as a toolbar, which can be an independent window or integrated with other windows such as the preview window into the same window. The tool container can generally provide multiple controls, and the functions implemented by these controls may be different. A typical application is to provide one or more special effect controls in the tool container. The host user can manipulate one or more such controls to perform special effects processing on the video effect, so that the final video image displayed in the preview window appears Corresponding special effects, and these special effects can be superimposed on the video stream data, and subsequently pushed to the live room users to watch. In addition, the tool container may of course also include controls that act on the preview video in the preview window to achieve other different functions, which should be understood in the art.

In order to facilitate the synchronization control of the live program, after the user operates the controls of the tool container, the flow control program can generate relevant messages in response to the manipulation of the controls, and send them back to the live program through the communication channel. , So that the live program can make further decisions and processing accordingly.

Step S13: After the live broadcast program starts the broadcast mode, it broadcasts the cloud address message it has acquired for receiving the video stream, and the simulation program notifies the cloud address message to the streaming control program to trigger it to enter the broadcast state, so that It transmits the video stream to the corresponding cloud address:

When the host user completes the relevant settings in the preview mode, the broadcast will often officially start. At this time, the live broadcast program can provide a control or any other method recognized by the live broadcast program to initiate the start of the broadcast mode to the live broadcast program. instruction.

The live broadcast program is triggered by the start instruction of the host user, and according to the design logic of the program developer, the cloud address used to upload the video stream can be obtained from the cloud server first. In a usual design, the live broadcast program will query the authentication service of the cloud server after passing the authentication, and then obtain the cloud address by querying the stream management service of the cloud server. These methods involving security and addressing are all known to those skilled in the art, and do not affect the creative spirit and protection scope of the present invention, and will not be repeated. In short, the live broadcast program will obtain the cloud address provided by the cloud server through certain code logic. The cloud address can theoretically be used to push the video stream collected by the live broadcast program itself, or it can be used to provide the flow control program for execution. Pushing the video stream collected by the flow control program, the present invention focuses on the latter.

In order for the streaming program to push streaming, the live streaming program broadcasts a cloud address message by constructing an intent according to the broadcast mechanism of the second operating system such as Android, that is, the cloud address is included in the intent and becomes the message body, Broadcast in a simulated operating environment.

In the same way, the broadcast receiver pre-registered by the simulation program will filter and receive the cloud address message, and then write through the socket or file channel, and the flow control program will The socket or file is read to obtain the cloud address message, and the simulation program also completes the task of notifying the cloud address message to the flow control program.

After the stream control program receives the cloud address message, it is triggered by this notification and enters the on-air state, so it can call the communication interface of the first operating system, transmit the video stream to the cloud address, and upload the video stream to Corresponding remote server, and the remote server will push this video stream directly or after processing to each user in the live broadcast room where the anchor user is located, and finally display it on the desktop of these users.

In the same way, if the communication with the remote server is interrupted or other failures occur during the period, the flow control program can reversely send the corresponding message back to the live program through the communication channel in the above manner, so as to allow the host The user makes further adjustments and controls.

In an embodiment further developed on the basis of this typical embodiment, the following steps may also be included:

When the live broadcast program is closed, the broadcast preview stop broadcast message, the simulation program notifies the stream control program of the preview stop broadcast message to trigger it to enter the stop broadcast state, and stop pushing the video stream to the cloud address:

When the host user wants to end the live broadcast, similarly, he can issue a preview stop broadcast instruction to the live broadcast program. The live broadcast program then constructs a message body according to the aforementioned broadcast mechanism to broadcast the preview stop broadcast message. After the simulation program filters through its broadcast receiver and receives the preview stop-broadcast message, it notifies it to the flow control program through a socket or a file. After the flow control program obtains the notification by reading the socket or the file, it triggers itself to switch to the stop state, so it can close the previously opened preview window and/or tool container, and synchronously stop pushing the cloud address to the cloud address. The entire live broadcast process is over.

It can be seen from the above video live broadcast control method of the present invention that this method makes the live broadcast process of the host user on the personal computer more smooth, even if the operating system of the live broadcast program used by the host is not the operating system of the personal computer, it can still be used. The simulation program and the flow control program of this method realize smooth and high-definition video live broadcast. Accordingly, developers only need to develop one live broadcast program, which can be applied to two operating systems.

Please refer to FIG. 2, a typical embodiment of the video live broadcast bridge method of the present invention is actually a specific application example obtained on the basis of each embodiment of the video live broadcast control method of the present invention, which is the basis for the realization of the simulation program Therefore, it involves the establishment principle and process of the communication channel, so I won’t repeat it. This method includes the following steps:

Step S21, creating a second operating system simulation operating environment different from the current first operating system for running the live program:

As mentioned earlier, when the simulation program acts as the virtual machine of the operating system and runs in the first operating system such as Windows, a simulation operating environment of the second operating system such as Android is created to run the video stream controller. Live program.

Step S22, establishing a communication channel between the live broadcast program and the flow control program running in the current first operating system:

During or after the simulation program establishes the simulation operating environment, the second direct channel can be established with the flow control program through sockets or file reading and writing, and the live program can be established with the live program. The first direct channel mainly includes the message channel from the broadcast message initiated by the live program using the broadcast mechanism to the broadcast receiver pre-registered by the simulation program, and the reverse from the simulation program to the live program The establishment of the cross-process message channel between the service processes constructed by the service component and the availability of two direct channels completes the establishment of the communication channel, so that the live program and the flow control program can communicate in both directions through the simulation program information.

Step S23, forwarding the on-air preview message broadcast by the live program to the stream control program via the communication channel, and trigger it to enter the preview state so as to preview the video stream it collects:

With reference to the foregoing, the simulation program uses the communication channel, with the aid of its broadcast receiver, to subsequently receive the start-up preview message broadcast by the live program, and then through the communication channel, use sockets or file read and write methods to transfer The broadcast preview message is passed to the flow control program, thereby triggering the flow control program to enter the preview state. Accordingly, as mentioned above, the flow control program can start to collect the video stream and output the video stream to the preview The window is for the anchor user to preview.

Step S24, forwarding the cloud address message broadcast by the live broadcast program to the stream control program via the communication channel, triggering it to enter the on-air state, so as to push the video stream to the cloud address:

In the same way, after the simulation program receives the cloud address message broadcast by the live broadcast program through the communication channel, with the help of its broadcast receiver, it can use the socket or file reading and writing method through the communication channel. The cloud address message is passed to the flow control program, thereby triggering the flow control program to enter the broadcast state. In this state, the flow control program starts to push the collected video stream to the cloud address. The stream may further include special effects loaded by the host user.

Via the communication channel, forward the preview stop broadcast message broadcast by the live broadcast program to the stream control program, and trigger it to enter the stop broadcast state to end the push of the video stream:

After the simulation program obtains the preview stop message broadcast by the live program from the communication channel by means of its broadcast receiver, the message is transferred to the flow control program by writing a socket or a file in the same way. The stream control program obtains the notification of the message through the corresponding reading socket or file, triggers itself to switch to the stop broadcast state, and closes the preview window and tool container that have been opened before, and stops pushing the video stream to the cloud address. In this case, Users in the live broadcast room will also not be able to obtain the real-time video stream of the anchor user.

It can be seen from the above video live streaming bridge method of the present invention that the simulation program actually plays the role of the operating system virtual machine and acts as a bridge between the flow control program of the first operating system and the live program of the second operating system. As long as the simulation program is implemented to run according to the logic of any second operating system, the simulation program can actually provide a running environment for the live program of any operating system, and it can be adapted to the flow control program. The role of standardized interfaces helps to avoid repeated development by developers for the live broadcast program itself.

Please refer to FIG. 3, a typical embodiment of the video live streaming control method of the present invention is actually a specific application example obtained on the basis of the various embodiments of the video live streaming control method of the present invention, and is an implementation of the streaming control program. The method that it relies on, so it involves the establishment principle and process of the communication channel, so I won’t repeat it. This method includes the following steps:

Step S31: Receive the start-up preview message forwarded by the simulation program used to provide the simulation running environment of the second operating system, start the preview window to display the video stream collected by the current bottom driver of the first operating system, and enter the preview state:

For the flow control program of the present invention, when it is running, it is in the off-broadcast state by default, but it uses the socket or file used in the communication channel mechanism to obtain the message passed by the simulation program. When it obtains the start preview message initiated by the live program, it triggers itself to enter the preview state, starts a preview window, and calls the underlying driver of the current first operating system, such as audio driver or camera driver, To capture the video stream, display the obtained video stream in the preview window for the anchor user to preview.

During the preview process of the host user, it may be necessary to set the display effect of the video stream. Therefore, it is usually possible to provide a tool container as described, and one or more controls are provided in the tool container. By manipulating these controls, you can add the video stream to the host. Video special effects and/or audio special effects, it should be understood that this special effect can eventually be presented to users in the live broadcast room.

Step S32: Receive the message containing the cloud address transmitted by the simulation program, enter the on-air state, and push the video stream to the cloud address:

During the preview process of the streaming program, it will further obtain the cloud address message broadcast by the live program and transmitted by the simulation program through a socket or file, obtain the cloud address from it, and then send it to the Pushing the video stream by the cloud address means that the stream control program has entered the start state, and the host user's video stream is pushed to the live room user via the remote server.

Step S33: Receive the preview stop broadcast message delivered by the simulation program, enter the stop broadcast state, and end the push of the video stream:

In the process of streaming, the streaming control program can obtain the stop-broadcast message broadcasted by the live program transmitted by the simulation program through the socket or file in the same way, thereby triggering itself to return to the stop-broadcasting state. In this state, The flow control program will end pushing the video stream collected by the machine to the cloud address. Generally, in combination with the foregoing, the flow control program can also close various open windows and containers, including the preview window and tool container.

During the operation of the flow control program, especially in the process of pushing video streams, if the remote server returns relevant information, or the communication process encounters a failure, or encounters any other requirements, the flow control program can pass through the socket The word or file sends a notification message to the simulation program. The notification message can be fed back to the live program by the simulation program through the service process, and the main process of the live program can display it on the interface in the simulation running environment for the anchor user to know.

From the above video live streaming control method of the present invention, it can be seen that the streaming control program functions as a standardized and independent service for pushing video streams. As long as the simulation program and the live streaming program comply with their communication methods, they can be called. Collect and push video streams to remote servers. When the streaming program is opened for service in this way, the development work of the developers of the live program will become simple, and only need to debug and adapt the simulation program and the existing live program belonging to the second operating system to make it compatible with The flow control program is fine.

Please refer to FIG. 4, the typical embodiment of the live video broadcast control method of the present invention is actually a specific application example obtained on the basis of the various embodiments of the live video control method of the present invention, and is a result of the implementation of the live broadcast program. Dependent method, so it involves the establishment principle and process of communication channel, so I won’t repeat it. This method includes the following steps:

Step S41, in response to the start preview mode command acting on the application program in the second operating system simulation running environment, trigger the display of the preview window in the first operating system providing the simulation running environment to display the information via the first operating system The video stream collected by the underlying driver:

As mentioned above, after the flow control program and the simulation program are running on the first operating system, the live program can be installed and run in the simulation running environment of the second operating system created by the simulation program. After the program is started, the broadcast mechanism of the second operating system is used in the live program to broadcast messages to the simulation program, and a service component can be carried to construct a service process that communicates with the simulation program, so that it can receive messages from the simulation program in reverse. The program and the flow control program communicate with each other through reading and writing sockets and files. Therefore, once the live broadcast program runs successfully, the communication channel between the programs is established.

In this case, the host user initiates the preview mode command through the user interface of the live broadcast application, and the command is transmitted to the first operating system through the simulation program in the form of broadcast start preview message, and the first operating system , The message can be read and responded to by a service such as a flow control program, so as to display a preview container on the desktop of the first operating system according to the logic of the flow control program, and at the same time, call the first The bottom layer driver of the operating system collects the video stream, and outputs the collected video stream for display in the preview window.

Step S42, in response to the start-up mode instruction acting on the application program, trigger the first operating system to call the remote interface to push the video stream to the cloud address obtained and provided by the application program:

When the host user confirms to start the live broadcast, he can initiate an instruction to start the broadcast mode through the entry or control provided by the application program, that is, the live broadcast program. As mentioned earlier, the live program will first query the authentication service and stream management service of the cloud server, and obtain the cloud address used by the cloud server to receive the video stream of the host user. However, the live program uses the broadcast mechanism of the second operating system. A message containing the cloud address is broadcast in the simulation operating environment, and the simulation program obtains the message and writes it into a socket or file for reading by the first operating system. The first operating system is triggered by this message, and can obtain the cloud address through a stream control program such as the aforementioned, and then call the remote interface to push the video stream to the remote server pointed to by the cloud address, and the remote server will The video stream is pushed to the users in the live broadcast room.

Step S43, in response to the preview stop instruction acting on the application program, trigger the bottom layer driver of the first operating system to stop collecting the video stream:

When the host user confirms the end of the live broadcast, similarly, the preview stop instruction is sent to the live broadcast program. After the instruction is transmitted to the first operating system through the simulation program in the form of broadcast, it triggers the underlying driver of the first operating system to stop collecting the said The video stream, correspondingly, closes the start state and enters the stop state.

According to this exemplary embodiment, in theory, as long as the first operating system supports the realization of the functional logic realized by the related methods of the flow control program and the simulation program, whether these programs are provided in the form of services or independent programs , Does not affect the implementation of this exemplary embodiment.

From the above video live broadcast control method of the present invention, it can be seen that the live broadcast program functions as a playback controller for the video stream in the live broadcast room. Although it runs in the memory space of the simulated operating environment based on the first operating system, It was developed for the second operating system. Therefore, developers do not need to develop and maintain the live program for the first operating system, as long as the live program includes a configuration that can be switched to run in the simulated operating environment. When the live program leaves the first operating system and runs in the real second operating system, it can still resume its normal function of collecting and pushing video streams by relying on its own software design logic.

In yet another exemplary embodiment of the present invention, a client is provided, including a central processing unit and a memory, and the central processing unit is configured to call and run the live program, streaming program, and bridge program stored in the memory. Any one or any more of them can execute each step in the above-mentioned related method embodiment of the present invention corresponding to each program, so as to facilitate the physical manipulation of the anchor user.

In summary, the present invention can prevent programmers from developing different live programs for different operating systems, and can provide host users with live programs developed for the second operating system on a personal computer using the first operating system, and can ensure The image quality of the video stream collected by the live video broadcast.

Those skilled in the art can understand that the package of the present invention relates to a device for performing one or more of the operations and methods described in the present invention. These devices may be specially designed and manufactured for the required purpose, or may also include known devices in general-purpose computers. These devices have computer programs stored in their memory, which are selectively activated or reconfigured. Such a computer program may be stored in a device (for example, a computer) readable medium or in any type of medium suitable for storing electronic instructions and respectively coupled to a bus. The computer readable medium includes, but is not limited to, any Types of disks (including floppy disks, hard disks, CD-ROMs, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory) , Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic card or optical card. That is, the readable medium includes any medium that stores or transmits information in a readable form by a device (for example, a computer).

Those skilled in the art can understand that computer program instructions can be used to implement each block in these structural diagrams and/or block diagrams and/or flow diagrams and combinations of blocks in these structural diagrams and/or block diagrams and/or flow diagrams. . Those skilled in the art can understand that these computer program instructions can be provided to processors of general-purpose computers, professional computers, or other programmable data processing methods for implementation, so that the computer or other programmable data processing method processors can execute this The structure diagram and/or block diagram and/or flow diagram of the disclosure of the invention are specified in the block or multiple blocks.

Those skilled in the art can understand that the various operations, methods, and steps, measures, and solutions in the process that have been discussed in the present invention can be alternated, changed, combined, or deleted. Further, various operations, methods, and other steps, measures, and solutions in the process that have been discussed in the present invention can also be alternated, changed, rearranged, decomposed, combined, or deleted. Further, the steps, measures, and schemes in the various operations, methods, and procedures disclosed in the present invention in the prior art can also be alternated, changed, rearranged, decomposed, combined or deleted.

The above are only part of the embodiments of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also It should be regarded as the protection scope of the present invention.

Claims

A method for controlling live video broadcast is characterized in that it comprises the following steps:

When the simulation program running in the current first operating system for simulating the operating environment of the second operating system starts the live program, the simulation program establishes the live program running in the operating environment and the live program running in the first operating system. The communication channel of the flow control program;

After the live broadcast program starts the preview mode, the broadcast start preview message is broadcast, and the simulation program notifies the flow control program of the start preview message to trigger it to enter the preview state, so that it opens a preview window to display the video stream it collects;

After the live broadcast program starts the broadcast mode, it broadcasts the cloud address message it has acquired for receiving the video stream, and the simulation program notifies the cloud address message to the streaming control program to trigger it to enter the broadcast state and make it broadcast to the corresponding The cloud address to transmit the video stream.
The method according to claim 1, characterized in that it further comprises the following steps:

When the live broadcast program is closed, the broadcast preview stop message, the simulation program notifies the stream control program of the preview stop message to trigger it to enter the stop broadcast state, and stop pushing the video stream to the cloud address.
The method according to claim 1, wherein the communication channel comprises a first direct channel between a live program and a simulation program and a second direct channel between the simulation program and the flow control program.
The method according to claim 3, wherein the first direct channel is established in the following manner: the live broadcast program carries service components registered in the simulation running environment, and the simulation program carries service components registered in the simulation operation environment. The broadcast receiver in the simulation operating environment, the simulation program transmits the message generated by the flow control program to the live program through the service process constructed by the service component; the simulation program receives the live broadcast through the broadcast receiver The message broadcast by the program is passed to the flow control program.
The method according to claim 3, wherein the second direct channel is established in the following manner: the simulation program and the flow control program establish a socket or file by reading and writing to each other. The second direct channel described.
The method according to claim 1, wherein the simulation program and the flow control program are adapted to run on a current first operating system, and the live broadcast program is adapted to run on a second operating system different from the current operating system, The simulation program is suitable for simulating the second operating system in a first operating system to create a corresponding simulation running environment for the live program.
The method according to claim 6, wherein the first operating system is any one of Windows or operating systems based on Linux and Unix, and the second operating system is different from the first operating system Any one of Android, IOS, Linux, Windows operating systems.
The method according to claim 1, wherein, in the process of establishing the communication channel by the simulation program, the application package name representing the main process of the live program and the service name representing the service process of the live program are passed to the stream Control program for the flow control program to identify the main process and the service process of the live program to determine the message delivery identity.
The method according to any one of claims 1 to 8, wherein when the flow control program enters the preview state, a tool container for controlling the video stream collected by the flow control program is opened, and the tool container includes at least one The control that affects the special effect of the preview video of the preview window, and the message generated by acting on the control is transmitted to the live program through the communication channel.
The method according to claim 2, wherein the preview window is closed when the flow control program enters the stop state.
A bridging method for live video streaming, which is characterized in that it comprises the following steps:

Create a second operating system simulation operating environment that is different from the current first operating system for running live broadcast programs;

Establishing a communication channel between the live broadcast program and the flow control program running in the current first operating system;

Forward the start preview message broadcast by the live broadcast program to the flow control program via the communication channel, and trigger it to enter the preview state so as to preview the video stream it collects;

Through the communication channel, forward the cloud address message broadcast by the live broadcast program to the flow control program, triggering it to enter the on-air state, so as to push the video stream to the cloud address.
The method according to claim 11, further comprising the following steps:

Via the communication channel, forward the preview stop broadcast message broadcast by the live broadcast program to the stream control program, triggering it to enter the stop broadcast state, so as to end the push of the video stream.
The method according to claim 11, wherein the communication channel comprises the first direct channel between the simulation program and the live program implemented by the method and the first direct channel between the simulation program and the flow control program. Two direct channels.
The method according to claim 13, wherein the first direct channel is established in the following manner: the live broadcast program carries service components registered in the simulation running environment, and the simulation program carries service components registered in the simulation operation environment. The broadcast receiver in the simulation operating environment, the simulation program transmits the message generated by the flow control program to the live program through the service process constructed by the service component; the simulation program receives the live broadcast through the broadcast receiver The message broadcast by the program is passed to the flow control program.
The method according to claim 13, wherein the second direct channel is established in the following manner: the simulation program and the flow control program establish a socket or file by reading and writing to each other. The second direct channel described.
The method according to claim 1, wherein, in the process of establishing the communication channel by the simulation program, the application package name representing the main process of the live program and the service name representing the service process of the live program are passed to the stream Control program for the flow control program to identify the main process and the service process of the live program to determine the message delivery identity.
A method for streaming video live broadcast, which is characterized in that it includes the following steps:

Receiving a broadcast preview message forwarded by a simulation program used to provide a simulated running environment of the second operating system, starting a preview window to display the video stream collected by the current bottom layer driver of the first operating system, and entering a preview state;

Receiving the message containing the cloud address transmitted by the simulation program, entering the on-air state, and pushing the video stream to the cloud address;

Receive the preview stop broadcast message delivered by the simulation program, enter the stop broadcast state, and end the push of the video stream.
The method according to claim 17, wherein when the preview state is entered, a tool container for controlling the video stream is opened, and the tool container includes at least one control capable of influencing a special effect of the preview video of the preview window .
18. The method according to claim 17, wherein the preview window is closed when the broadcast stop state is entered.
The method according to any one of claims 17 to 19, wherein the flow control program that implements the method and the simulation program communicate with each other by reading and writing sockets or files, and Realize the delivery of the message.
The method according to claim 20, wherein the flow control program that implements the method sends a notification message to the simulation program, so that the notification message is displayed in the simulation running environment created by the simulation program.
A method for controlling live video broadcasting, which is characterized in that it comprises the following steps:

In response to the start preview mode command acting on the application in the second operating system simulation running environment, triggering the display of the preview window in the first operating system that provides the simulation running environment to display the commands via the underlying driver of the first operating system The captured video stream;

In response to the start-up mode instruction acting on the application program, triggering the first operating system to call the remote interface to push the video stream to the cloud address obtained and provided by the application program;

In response to the preview stop instruction acting on the application program, the bottom layer driver of the first operating system is triggered to stop collecting the video stream.
22. The method of claim 22, wherein the first operating system runs a simulation program for creating the second operating system module operating environment, and the first operating system runs a first-level control program for implementing The execution of the action of collecting the video stream by the underlying driver and the action of pushing the video stream.
22. The method according to claim 22, wherein the live program that implements the method sends a message in the broadcasting mode of the broadcasting mechanism of the second operating system, so as to realize the trigger operation of the first operating system.
The method according to claim 22, wherein the live program that implements the method carries a service component for receiving messages generated in the first operating system, so as to display the messages generated in the first operation To the user.
The method according to claims 22 to 25, wherein the first operating system is any one of Windows or operating systems based on Linux and Unix, and the second operating system is similar to the first operating system. Any one of different Android, IOS, Linux, Windows operating systems.
A client, comprising a central processing unit and a memory, wherein the central processing unit is used to call and run one or more computer programs stored in the memory to execute any one of claims 1 to 26 The method described.