WO2019071680A1

WO2019071680A1 - Plug-in loading method and apparatus

Info

Publication number: WO2019071680A1
Application number: PCT/CN2017/109671
Authority: WO
Inventors: 杨亮; 陈少杰; 张文明
Original assignee: 武汉斗鱼网络科技有限公司
Priority date: 2017-10-09
Filing date: 2017-11-07
Publication date: 2019-04-18
Also published as: CN107678801A; CN107678801B

Abstract

A plug-in loading method and apparatus, at least used for solving the technical problem in the prior art of how to improve the running stability of a live broadcasting program at an anchor end. The method comprises: starting an anchor end live broadcast application program (S101); acquiring a master control process of the anchor end live broadcast application program (S102); the master control process creates a plug-in loading process; and the plug-in loading process loads a target plug-in to be loaded into the live broadcast application program during the plug-in loading process (S103).

Description

Method and device for loading plug-in

Technical field

The present invention relates to the field of computer technologies, and in particular, to a method and apparatus for loading a plug-in.

Background technique

Webcasting has gradually become a mainstream entertainment and communication method in modern life. Throughout the current broadcast of the main broadcast application of each platform, and even other types of applications, all run in a single-process mode, that is, all the functions of the program are integrated in one process.

Although the single-process mode has the advantages of simple architecture and convenient maintenance, it has low stability. For example, if a plug-in has a problem, it may cause the entire process to crash and must be solved by restarting the program.

Therefore, the prior art has a technical problem of how to improve the running stability of the live broadcast program on the anchor end.

Summary of the invention

The embodiment of the invention provides a method and a device for loading a plug-in, at least for solving the technical problem of how to improve the running stability of a live broadcast program on the anchor end.

In a first aspect, the present invention provides a method for loading a plug-in, including:

Start the live broadcast application on the anchor side;

Obtaining a master control process of the broadcast application of the anchor end;

The master process creates a plug-in loading process;

The plug-in loading process loads a target plug-in that needs to be loaded in the live application in the plug-in loading process.

Optionally, before the plug-in loading process loads the target plug-in that needs to be loaded in the live application in the plug-in loading process, the method further includes:

The master process creates a plugin enumerator process;

The plug-in enumerator process enumerates, to the master process, an alternative plug-in that exists in the current load directory and can be loaded;

The master process determines to determine the target plugin from the candidate plugin, and notifies the plugin to load the target plugin.

Optionally, the master process creates a plug-in enumerator process, including:

The master process calls a process creation function;

The master process sends the ID of the master process to the process creation function;

The master process runs the process creation function to create the plug-in enumerator process, after the plug-in enumerator process is initialized, initializes the process environment, and based on the master control of the process creation function The ID of the process is connected to the master process.

Optionally, the plug-in enumerator process initializes the process environment after the creation is completed, and connects to the master process according to the ID of the master process that the process creation function has, including:

Registering the enumeration plugin function and the enumeration exit function in the plugin enumerator process;

Obtaining an ID of the master process that the process creation function has is connected to the master process.

Optionally, the plug-in enumerator process enumerates, to the master process, candidate plug-ins that are present in the current loading directory and can be loaded, including:

The plug-in enumerator receives an enumeration instruction sent by the master control process, and invokes the enumeration plug-in function process to read all plug-ins in the loading directory;

Filter out legitimate plugins from all of the plugins;

Extracting the backup plug-in supported by the anchor live application from the legal plug-in.

Optionally, after the plug-in enumerator process enumerates, to the master process, an optional plug-in that exists in the current loading directory and can be loaded, the method further includes:

The master process sends an exit instruction to the plug-in enumerator process;

The plug-in enumerator invokes the enumeration exit function to end the process based on the exit instruction.

Optionally, the method further includes:

The master process monitors whether the plug-in loading process crashes;

If the master process monitors that the plugin loading process crashes, re-create the plugin loading process.

In a second aspect, the present invention provides a device for loading a plug-in, comprising:

a startup unit, configured to start a live broadcast application of the anchor;

a master control process acquiring unit, configured to acquire a master control process of the live broadcast application of the anchor end;

a process creation unit for creating a plugin loading process;

a plug-in loading unit, configured to load a target plug-in that needs to be loaded in the live application during the plug-in loading process.

Optionally, the process creation unit is further configured to create a plug-in enumerator process before loading the target plug-in that needs to be loaded in the live application in the plug-in loading process;

The device further includes a plug-in enumeration unit, configured to enumerate, to the master process, an optional plug-in existing in the current loading directory and loadable;

a determining unit, configured to determine that the target plugin is determined from the candidate plugin, and notify the plugin to load the target plugin.

Optionally, the process creation unit is configured to invoke a process creation function, and send the ID of the master process to the process creation function, and run the process creation function to create the plug-in enumerator process, the plug-in After the creation of the enumerator process, the process environment is initialized, and the master process is connected based on the ID of the master process that the process creation function has.

Optionally, the device further includes a plug-in loading process environment initial unit, configured to initialize a process environment after the plug-in enumerator process is created, and based on the master process of the process creation function The ID is connected to the master process, and is specifically configured to register an enumeration plug-in function and an enumeration exit function in the plug-in enumerator process, and obtain an ID connection of the master process that is included in the process creation function. Master process.

Optionally, the plug-in enumeration unit is configured to receive an enumeration instruction sent by the master control process, and invoke the enumeration plug-in function process to read all plug-ins in the loading directory; from all the plug-ins The legal plug-in is filtered out; and the backup plug-in supported by the anchor live application is filtered out from the legal plug-in.

Optionally, the device further includes an exit unit, configured to: after the plug-in enumerator process enumerates, to the master process, an optional plug-in that exists in the current loading directory and can be loaded, The applet process sends an exit instruction, and the enumeration exit function is called to end the process based on the exit instruction.

Optionally, the device further includes a plug-in loading process monitoring unit, configured to monitor whether the plug-in loading process crashes, and re-create the plug-in loading process if the plug-in loading process is detected to crash.

In a third aspect, the invention provides a computer readable storage medium having stored thereon a computer program, which when executed by a processor, implements the steps of the method of any of the first aspects.

In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, the processor implementing the program to implement any of the first aspects The steps of the method.

The above one or more technical solutions in the embodiments of the present application have at least one or more of the following technical effects:

In the technical solution of the embodiment of the present invention, the host broadcast application is first started, and the main control process of the live application of the anchor is obtained, and then the main control process further creates a plug-in loading process, and the live loading process needs to be loaded in the plug-in loading process. The target plugin loaded in the application. Since the embodiment of the present invention adopts multiple processes and uses the plug-in loading process to load the plug-in, the abnormality of the plug-in function does not affect the master process, so that the program crashes due to one or more abnormal functions, so that the host-side live application is more The system runs stably, thereby solving the technical problem of how to improve the running stability of the live broadcast program on the anchor end in the prior art.

DRAWINGS

1 is a flowchart of a method for performing live broadcast according to an embodiment of the present invention;

2 is a schematic structural diagram of a device for playing a live broadcast according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a computer device according to an embodiment of the present invention.

Detailed ways

In order to solve the above technical problem, the technical solution provided by the present invention is as follows:

The technical solutions of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments. It is understood that the specific features of the embodiments and the embodiments of the present application are the detailed description of the technical solutions of the present application, and are not limited to the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments of the present application and the embodiments may be combined with each other.

The term "and/or" in this context is merely an association describing the associated object, indicating that there may be three relationships, for example, A and/or B, which may indicate that A exists separately, and both A and B exist, respectively. B these three situations. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

The first aspect of the present invention provides a method for performing live broadcast. Referring to FIG. 1 , it is a flowchart of a method for performing live broadcast according to an embodiment of the present invention. The method includes:

S101: Start the broadcast application of the anchor end;

S102: Acquire a master control process of the live broadcast application of the anchor end;

S103: The master process creates an encoding process and/or a plug-in loading process, where the encoding process is used to encode a video stream based on the control of the master process, and feed back the video stream to the master process. The plug-in loading process is configured to load a plug-in in the plug-in loading process, and invoke the loaded plug-in according to the control of the master process;

S104: The master control process acquires the video stream.

S105: The master control process sends the video stream to a live broadcast server, so that the live broadcast service The device forwards the video stream to a viewer device accessing the live server.

Specifically, the anchor live application is started in S101. In the specific implementation process, the live broadcast application of the anchor terminal can be installed in a live broadcast application on a host device such as a mobile phone, a tablet computer or a desktop computer, and the present invention does not impose any specific limitation. In order to facilitate the introduction of the technical solution in the embodiment of the present invention, the following describes an example of a live broadcast application on a desktop computer installed in a Windows system.

In S102, with the activation of the live broadcast application of the anchor end, the main control process of the live broadcast application of the anchor end is created accordingly. Next, in S103, the master process will create an encoding process and/or a plugin loading process as needed.

Specifically, the encoding process in the embodiment of the present invention is used to encode a video stream based on the control of the master control process, and feed back the video stream to the master control process. More specifically, the encoding process receives the initialization command and the encoding command of the master process for initialization and encoding, and passes the encoded data to the master process.

The plugin loading process is used to load the plugin in the plugin loading process and call the loaded plugin according to the control of the master process. More specifically, the plug-in loading process acts as a loader for the anchor application plug-in of the anchor, and loads the original plug-in to be loaded into the host process into the plug-in loading process, so that the master process no longer needs to load the plug-in, and thus This avoids the situation where the master process crashes due to plugin issues. The plug-in loading process in the embodiment of the present invention simulates the environment of the main control process on which the plug-in is loaded and run, and responds to the command from the main control process to obtain an image, obtain audio, and the feedback generated by the plug-in to the main control process. Image and audio, so that the image from the plug-in and the sound can be played normally in the master process.

The master process is the main process of the live broadcast application of the anchor, and manages the plug-in loading process and the encoding process. Moreover, if the encoding process is abnormally exited, the master process will automatically re-create the encoding process, so that the encoding process can be restarted quickly.

Further, since the main control process, the encoding process, and the plug-in loading process are different processes, in order to enable multiple processes to cooperatively provide various services of the live broadcast application of the anchor, the processes interact by means of process communication. The manner of the process communication includes the RPC (Remote Procedure Call Protocol) and the named pipe, etc., and the present invention is not limited thereto, and those skilled in the art to which the present invention pertains may select according to the actual situation. In the following text, RPC is taken as an example to introduce.

For the anchor live application in the embodiment of the present invention, the main control process, the encoding process, and the plug-in loading process run in the background, and interact with the anchor through the interface program in the foreground. The interface program is a program for real human-computer interaction for the main broadcast. In the implementation, the interface program needs to submit the video screen preview window to the main control process for drawing. The call of the anchor on the interface will be forwarded to the master process through RPC. Moreover, the interface program needs to start and resume the master process in the event that the master process crashes abnormally.

The above is an introduction to each process in the multi-process mode of the present invention. In the specific implementation process, not in In the same situation, the anchor may have different requirements. For example, the anchor may use the single-process mode for maintenance, or use the multi-process mode for the program temperature, and the plug-in loading process will not be created if the plug-in is not needed. In order to meet different requirements, the anchor live application in the embodiment of the present invention may operate in at least the following four operating modes.

First mode:

The first mode LIVE_ENCODER_PROXY_WITH_SERVER is specifically a multi-process mode. In the first mode, after the host broadcast application starts, the master process RpcLiveEncoderServer will be created (the master process can also be named other), and then the host broadcast application will treat the master process RpcLiveEncoderServer as the server, as the client. Access the master process RpcLiveEncoderServer. Then, the call of the host application to the master process is sent to the master process RpcLiveEncoderServer through the RPC request.

In the first mode, the master process RpcLiveEncoderServer will further create the encoding process RpcLiveEncoderEncoder (the encoding process can also be named other) and the plug-in loading process RpcPluginLoader (the plug-in loading process can also be named other).

Second mode:

The second mode LIVE_ENCODER_PROXY_WITHOUT_SERVER is also in multi-process mode. Different from the first mode, in the second mode, the live broadcast application of the anchor does not need to create the master RpcLiveEncoderServer, but is connected to an existing preset master process. Similarly, the host live application will treat the master process RpcLiveEncoderServer as a server, and access the default master process as the client according to the address of the server.

In a specific implementation process, the master process in the second mode may be a process created by an upper layer application. Compared with the first mode, the second mode has the advantage of facilitating the extension of the main control process from the upper layer application; the advantage of the first mode compared to the second mode is that the main broadcast application directly manages the main control process, and the main control process is It is created by the anchor application itself, so the possibility of incompatibility is low.

In the second mode, the master process RpcLiveEncoderServer further creates the encoding process RpcLiveEncoderEncoder and the plugin loading process RpcPluginLoader.

The third mode:

The third mode LIVE_ENCODER_PROXY_STUB is also in multi-process mode. The master process RpcLiveEncoderServer in the third mode is similar to the master process in the first mode or the second mode. In the third mode, the master process RpcLiveEncoderServer further creates one of the encoding process RpcLiveEncoderServer or the plug-in loading process RpcPluginLoader. In the embodiment of the present invention, the host process RpcLiveEncoderServer is instructed to generate the encoding process RpcLiveEncoderServer or the plug-in loading process RpcPluginLoader by setting a flag bit in advance.

Specifically, the first flag bit LIVE_ENCODER_REMOTE_PLUGIN indicates that a plug-in needs to be created. The RpcPluginLoader is loaded, and the second flag bit LIVE_ENCODER_REMOTE_ENCODER indicates that the encoding process RpcLiveEncoderServer needs to be created.

Then, in the specific implementation process, if you need to add a plug-in and reduce the impact of the plug-in on the master process, you can select the third mode and set the first flag bit in advance; if you need separate process coding, avoid the coding function to master When the process affects and does not need to add a plug-in, the third mode can be selected and the second flag bit is set in advance.

The fourth mode:

The fourth mode LIVE_ENCODER_LOCAL_SERVICE is a single process mode. In the fourth mode, the anchor live application creates only one process, the master process. The encoding function of the live broadcast application of the anchor is implemented in the main control process, and the plugin is also loaded in the main control process.

In the fourth mode, the master process RpcLiveEncoderServer does not further create other processes.

Based on the above descriptions of the four modes of operation, it can be seen that the host-end live application obtains the master process in different modes in different modes. Therefore, S102 in the embodiment of the present invention is specifically implemented by the following process:

Determining a preset operation mode of the anchor live application;

If the operating mode is the first mode, the master process of the anchor live application is created, wherein in the first mode, the master process creates the encoding process and the plugin Loading process

If the operation mode is the second mode, the preset address of the master process is obtained, and the master process is accessed based on the address, where the master process is in the second mode. The encoding process and the plugin loading process will be created.

Specifically, it is first determined which of the above four modes is set in advance. If the running mode set by the anchor is the first mode, the host control process is created by the anchor live application; if the running mode set by the anchor is the second mode, the live broadcast application of the anchor acquires the address of the preset master process. And then access the preset master process based on the address. Moreover, in the first mode and the second mode, the obtained master process further creates an encoding process and a plug-in loading process.

In addition, if the running mode set by the anchor is the fourth mode, the anchor live application only creates a process of the master process and runs in the single process mode. If the running mode set by the anchor is the third mode, after the anchor live application acquires the master process, the master process further needs to create a process in the encoding process or the plug-in loading process. Specifically, before S103, the method further includes: determining a running mode of the preset broadcast application of the anchor end, and if the running mode is the third mode, further obtaining a flag bit indicating that the encoding process or the plug-in loading process needs to be created, if the marking bit is A flag bit, S103 is specifically: creating a plug-in loading process by the master process; if the flag bit is the second flag bit, S103 Specifically, the encoding process is created by the master process.

It can be seen from the above description that by setting different operating modes, the anchor broadcast application can be run in a single process or multiple processes to meet different performance requirements. And, by setting different mark bits, one of the plug-in loading function and the encoding function can be separated from the master process.

Further, the coding function is usually one of the core functions of the live broadcast application of the main broadcast. In the prior art, each function, interface and parameters for implementing the coding function are encapsulated in the coding module. When the main control process runs, the coding module is loaded into the main control process, and the main control process can arbitrarily call various functions, interfaces and parameters of the coding function. In the embodiment of the present invention, the live broadcast application of the anchor end can be run in a single process or in multiple processes. Therefore, in order to enable the live broadcast application of the anchor to be normally encoded, whether it is a single process or a multi-process operation, the embodiment of the present invention At least the following improvements have been made to the encoding module:

Encapsulating the encoding function of the encoding module in the prior art in a module, that is, the LiveEncoderLocalService encoding local service module; adding a module for receiving the instruction and forwarding the received instruction to the encoding local service module, that is, LiveEncoderStub encoding Stub module; add a module for simulating the original output interface of the encoding module, and transparently transmitting the result of the local service module processing, that is, the LiveEncoderProxy encoding proxy module. The LiveEncoderLocalService encoding local service module is connected to the LiveEncoderStub encoding stub module, and the LiveEncoderProxy encoding proxy module is connected to the LiveEncoderLocalService encoding local service module. Optionally, the LiveEncoderLocalService encoding local service module, the LiveEncoderStub encoding stub module, and the LiveEncoderProxy encoding proxy module are encapsulated in one module, that is, the LiveEncoder encoding module.

Based on the above description of the improvement of the coding module, how to apply the improved coding module coding in the embodiment of the present invention will be described below.

(1) If the master process creates the encoding process, that is, the anchor live application runs in the first mode, the second mode, and the third mode (under the second flag bit), the master process acquires the video stream in S104. :

The master process sends an encoding instruction to the encoding stub module to notify the encoding process to encode the video stream;

The encoding process calls an encoding local service module connected to the encoding stub module to encode the video stream;

The encoding process calls the encoding proxy interface module connected to the encoding local service module to feed back the encoded video stream to the main control process.

Specifically, if the main control process calls the encoding module to create the encoding process, the encoding function of the live broadcast application of the anchor is implemented independently from the main control process to the encoding process. Encoding process slave In the coding module. When encoding is required, the master process sends an encoding instruction to the encoding stub module by communicating with the RPC between the encoding processes to notify the encoding process to start encoding. The encoding process, in response to the encoding instruction, invokes a related method and function in the encoding local service module to execute the encoding instruction, thereby encoding the video stream. Then, the encoding process calls the encoding proxy module LiveEncoderProxy to feed back the encoded video stream to the main control process through the RPC channel.

Further, in S105, the main control process sends the video stream to the live broadcast server, and then the live broadcast server forwards the video stream to the viewer device of the viewer watching the anchor. Furthermore, the viewer device decodes the video stream and decodes the live video for viewing by the viewer.

(2) If the master process does not create the encoding process, that is, the anchor live application runs in the third mode and the third mode (the first flag bit), the master process obtains the video stream in S104, including:

The master process sends an encoding instruction to the encoding stub module;

The encoding stub module calls an encoding local service module connected to the encoding stub module to encode the video stream;

The encoding local service module calls an encoding proxy interface module connected to the encoding local service module to feed back the encoded video stream to the main control process.

Specifically, if the encoding process is not created by the master process, the encoding function of the live broadcast application of the anchor is still implemented by the master process. Specifically, when the master process needs to encode the video stream, an encoding instruction is generated. The master process then sends an encoding instruction to the encoding stub module. Next, the encoding stub module sends an encoding instruction to the encoding local service module to invoke the method and function in the encoding local service module to encode the video stream. Then, the encoding local service module sends the encoded video stream to the encoding proxy module, and then the encoding proxy module transparently transmits the video stream to the main control process.

The above is an introduction to how the main control process and the encoding process use the improved encoding module. The following describes how the main control process creates the encoding process and the encoding process specific encoding process.

First, the process of creating a coding process by the master process is introduced. In the embodiment of the present invention, the process of creating a coding process by the master process includes the following processes:

The master process calls a process creation function;

The master process runs the process creation function to create the encoding process, and after the creation process is completed, the process process initializes the process environment, and based on the ID connection of the master process that the process creation function has Master process.

Specifically, the master process calls the Windows system process to create the CreateProcess function to create an encoding process RpcLiveEncoderEncoder. When the main control process calls the CreateProcess function, the process ID of the master process is passed to the CreateProcess function as a command line parameter. After the encoding process RpcLiveEncoderEncoder created by the CreateProcess function is started, the process ID of the master process can be obtained from the command line. Then, the master process runs the CreateProcess function to create the encoding process RpcLiveEncoderEncoder. Next, the encoding process initializes the process environment and establishes a connection with the master process. Specifically, the encoding process initializes the process environment and connects to the master process by the following process:

The encoding process registers an initialization service function and a video stream encoding service function;

Creating a video shared memory that stores video material and an audio shared memory that stores audio material;

Obtaining an ID of the master control process that is included in the process creation function, and connecting the master control process;

And sending, according to the connection with the master process, an identifier of the video shared memory and an identifier of the audio shared memory to the master process.

Specifically, the encoding process RpcLiveEncoderEncoder creates an RPC communication client instance handle hRpc by calling the RpcCreatelnstance method, and correspondingly, the master process is the server. Then, the RpcRegisterProcedure function is called, the initialization service function init_encoder of the encoding process RpcLiveEncoderEncoder is registered to hRpc, and the video stream encoding service function encode_raw_buffer is registered to hRpc. In the embodiment of the present invention, the initialization service function init_encoder is used to respond to the initialization command of the main control process, and completes the encoding initialization operation according to the rated parameter in the initialization command, and the encoded stream server function encode_raw_buffer is used to respond to the coding instruction of the main control process, and is The control process specifies the video material and audio material in the memory to be encoded, and returns the encoded data to the master process.

Next, create a memory that stores the video material and audio material. In the embodiment of the present invention, considering that if a video stream with a higher resolution, such as a 1080p video stream, is required, the data amount of each frame of the image is relatively large; and, if the data between the processes is used by the RPC, In order to avoid unnecessary transmission of encoded data, the embodiment of the present invention adopts a shared memory manner to optimize performance. Specifically, call the Windows system's CreateFileMapping function to create a video shared memory called RpcLiveEncoderEncoder_video_X, and create an audio shared memory called RpcLiveEncoderEncoder_Audio_X. Of course, in the specific implementation process, the naming of the video shared memory and the audio shared memory may be set according to actual conditions, and the present invention does not specifically limit.

Next, the encoding process reads the process ID of the master process in the command line, and calls the RpcStartService function to establish an RPC communication channel with the master process. Then, the client_process_ready function of the master process is invoked based on the RPC channel to notify the master process that the encoding process initializes the process environment.

The main control process obtains the sessionld session ID of the encoding process in the RPC connection from the RPC channel according to the call of the encoding process to the client_process_ready function, and determines the initialization of the encoding process environment. Completion, and then in the subsequent process, notify the encoding process to encode the video stream as needed.

In addition, the master process acquires the video shared memory identifier and the audio shared memory identifier created by the encoding process based on the rpc_encoder_sharememory_name function of the encoding process of the sessionld of the encoding process.

Through the above process, the creation of the encoding process is completed. Next, the specific encoding process of the encoding process is introduced.

First, in order to enable the viewer device to successfully acquire and decode the video stream, it is necessary to first provide the decoding header parameter to the viewer device. Therefore, before the video stream is encoded by the encoding process and fed back to the master process, it also includes:

The master control process sends an initialization command to the encoding process, where the initialization command includes a nominal parameter for guiding the encoding process to determine a decoding header parameter, where the decoding header parameter includes a video decoding header parameter and an audio decoding header parameter;

The encoding process runs the initialization service function based on the initial command to obtain the video decoding header parameter and the audio decoding header parameter based on the nominal parameter;

Transmitting, by the encoding process, the video decoding header parameter and the audio decoding header parameter to the main control process, so that the main control process sends the video decoding header parameter and the audio decoding header parameter to the The live broadcast server forwards the video decoding header parameter and the audio decoding header parameter to the audience end device.

Specifically, the master process sends an initialization init_encoder command to the encoding process. The command includes a nominal parameter for directing the encoding process to determine the decoding header parameters, and the nominal parameter is determined by the master process based on the selection of the anchor. In the embodiment of the present invention, the rated parameters include but are not limited to resolution information (width width and height height), video bit rate videoBitrate, video frame rate fps, video encoder type videoCodec, audio encoder channel number audioChannel, audio The sampling rate is audioSampleRate, the audio bit rate audioBitrate, and the audio encoder type audioCodec.

The encoding process receives the initialization init_encoder from the RPC channel, and responds by initializing the service function init_encoder. Specifically, the CreateMediaFile function is called to create a multimedia file IMediaFile interface. Then, according to the video encoder type videoCodec in the init_encoder command, the CreateEncoder method of the IMediaFile interface is used to create a VideoEncoder interface videoEncoder of type videoCodec, and a video encoder interface audioEncoder of type audioCodec according to the audio encoder type audioCodec in the init_encoder command. The Open method that calls the video encoder interface videoEncoder will encode the width of the video. Width, height height, video bit rate videoBitrate, video frame rate fps as parameters to initialize the video encoder interface IMediaEncoder, and call the audio encoder audioEncoder Open method to encode the audio The audio channel audioEncoder is initialized by the audio channel of the device, the audio sample rate audioSampleRate, and the audio bit rate audioBitrate as parameters.

Then, the encoding process calls the readVideoHeader of the IMediaFile interface to obtain the video decoding header parameter videoHeader of the video encoder interface IMediaEncoder, and the readAudioHeader method of the IMediaFile interface to obtain the audio decoding header parameter audioHeader of the audio encoder audioEncoder.

Then, the encoding process sends the obtained video decoding header parameter videoHeader and the audio decoding header parameter audioHeader to the main control process through the RPC channel. The main control process further sends the video decoding header parameter videoHeader and the audio decoding header parameter audioHeader to the live broadcast server, and the live video server sends the video decoding header parameter videoHeader and the audio decoding header parameter audioHeader to each viewer device. Then, the viewer device can decode and play the video and audio according to the video decoding header parameter videoHeader and the audio decoding header parameter audioHeader.

After obtaining the decoding header parameters, the encoded video stream can be obtained. In the embodiment of the present invention, obtaining an encoded video stream by using an encoding process in S104 is specifically performed by the following process:

The encoding process receives an encoding command sent by the master control process;

Extracting, according to the encoding command, the video material pre-stored by the master control process from the video sharing memory, and extracting audio material pre-stored by the master control process from the audio sharing memory;

Running the video stream encoding service function to encode the video material, obtain video encoded data, and encoding the audio material to obtain audio encoded data, the video stream including the video encoded data and the audio encoding data;

And feeding back the video encoded data and the audio encoded data to the master process.

Specifically, the main control process pre-stores the video material to be encoded into the video shared memory according to the identifier of the video shared memory obtained in the foregoing, and pre-stores the audio material to be encoded according to the identifier of the audio shared memory obtained in the foregoing. Into the audio shared memory. In the specific implementation process, the content of the video material and the audio material are usually matched, for example, from the same file or collected by the anchor device. Then, when the video stream needs to be encoded, the master process sends an encoding command encode_raw_buffer to the encoding process through the RPC channel between the encoding process.

After the encoding process receives the encode_raw_buffer command, it responds by the encoding function, specifically: calling the Encode method of the videoEncoder interface, reading the video material in the video shared memory for encoding, returning the video encoded data to the main control process; and calling the Encode of the audioEncoder The method encodes the audio material in the audio shared memory, and returns the audio encoded data to the main control process. The video encoded data and the audio encoded data constitute a video stream, and the two correspond to each other according to a time axis.

The above is an introduction to encoding the video stream of the encoding process. Through the above process, the encoding process implements a separately encoded video stream.

After receiving the video stream returned by the encoding process, the master process stores the video stream in the sending queue and sends it to the live server. Then, after receiving the video stream, the live broadcast server forwards the video stream to each viewer device, and the viewer device sequentially decodes the received video stream, and then plays the live broadcast video of the anchor to the user, thereby completing the live broadcast.

In addition, in the specific implementation process, the encoding process may be crashed due to an abnormality. In order to enable the broadcast application to continue to be encoded and continue to be broadcasted, the live broadcast method in the embodiment of the present invention may further include:

The master process monitors whether the encoding process crashes;

If the master process monitors that the encoding process crashes, the master process re-creates the encoding process.

Specifically, the master process monitors the running status of the encoding process. If the encoding process crashes, the encoding process is re-created in the manner described above for creating the encoding process, and the description is not repeated here.

The above is an introduction to the encoding process. The following describes the plugin loading process and the method of plugin loading.

In the embodiment of the present invention, the plug-in of the live broadcast application of the anchor is loaded inside the plug-in loading process, and even if the plug-in is abnormal, the operation of the master process is not affected. Similar to the encoding process, the plug-in process is also connected to the master process through the RPC channel, and the plug-in process invokes the relevant interface of the master process through the RPC channel in order to simulate the environment in which the plug-in is loaded into the master process, so that the plug-in can be loaded normally. And running.

The prior art master process needs to know which plugins are available before loading the plugin, so the plugin needs to be enumerated before the plugin is loaded. The plug-in loading process in the embodiment of the present invention also needs to enumerate the plug-in to the main control process before loading the plug-in. Specifically, before the plugin loading process loads the target plugin that needs to be loaded in the live broadcast application in the plugin loading process, the plugin loading process further includes:

The master process creates a plugin enumerator process;

First, the process of creating a plug-in enumerator process by the master process is introduced. The master process creates a plug-in enumerator process by:

The master process calls a process creation function;

The master process loads the RpcPluginManager Remote Plugin Manager module, which is responsible for the creation of the plugin enumerator process RpcPluginFinder and commands the RpcPluginFinder to enumerate the functionality of the plugin. After loading the RpcPluginManager module into the main control process, call the CreatePluginManager function in the module to create a plugin manager interface IPluginManager.

Next, the master process calls the initialization init method of the plugin manager interface IPluginManager, so that the init method further calls the Windows system process creation function CreateProcess to create the plugin enumerator process RpcPluginFinder. When the CreateProcess function is called, the master process also passes the process ID of the master process as a command line argument to the CreateProcess function.

Next, the master process runs the create function CreateProcess to create the plugin enumerator process RpcPluginFinder.

Through the above process, the creation of the plugin enumerator process is completed. After the plug-in enumerator process is created, it also needs to initialize the process environment and establish a connection with the master process. Specifically, the plugin enumerator process initializes the environment through the following process:

Specifically, an RPC communication client instance handle hRpc is created by calling RpcCreatelnstance; the RpcRegisterProcedure function is called, and the function function of the enumeration plug-in, that is, the enumeration plug-in function get_plugins is registered to hRpc, and the enumeration plug-in function is used to respond to the master process. The enumeration plugin instruction enumerates the plugin; calls the RpcRegisterProcedure function, and registers the enumeration exit function to hRpc. The enumeration exit function is used to respond to the enumeration exit instruction sent by the master process after all the plugins are obtained, so that the plugin is The process exits; according to the process ID of the master process in the command line, the RpcStartService function is called to establish an RPC channel with the master process; the client_process_ready function of the master process is called to notify the master process that the plugin enumerator process is initialized. carry out.

The main control process obtains the sessionld session ID of the plug-in enumerator process in the RPC connection from the RPC channel according to the call of the plugin enumerator process to the client_process_ready function, and determines that the plug-in enumerator process environment initialization is completed, and then in the subsequent process. Invoke the plugin enumerator process to enumerate plugins as needed.

Through the above process, the creation of the plugin enumerator process is completed. Next, the specific enumeration process of the plug-in enumerator process is introduced. The plugin enumerator process enumerates to the master process the candidate plugins that are present in the current load directory and that can be loaded, including:

Filter out legitimate plugins from all of the plugins;

Specifically, the candidate plug-in in the embodiment of the present invention refers to a plug-in that is selected and loaded by the master process. When the master process needs to load the plugin, it will first send an enumeration instruction to the plugin enumerator process to get all the alternative plugins. The plug-in enumerator process receives the enumeration sent by the master process from the RPC channel. In the future, the enumeration function get_plugins is called to enumerate all the plug-ins in the current load directory.

Specifically, the get_plugins function calls the FindFirstFile function to get the first DLL (Dynamic Link Library) file in the load directory. Next, the get_plugins function calls the FindNextFile function to continue to get the next DLL file in the load directory. If the FindNextFile function returns true, it means that the next DLL file is obtained, then execute the FindNextFile function again and continue to get the next DLL file; if the FindNextFile function returns false, it means that all the DLL files in the loading directory have been obtained. As long as the return value of the FindNextFile function is true, the next DLL file is fetched until the FindNextFile function returns a false value.

In the specific implementation process, the anchor may add some illegal plug-ins to the loading directory, but the illegal plug-ins cannot be loaded. Therefore, after obtaining all the plug-ins, it is illegal to further determine whether the obtained plug-ins are legal. The plugin is filtered out.

Specifically, the LoadLibrary function of the calling system loads each DLL file. If the LoadLibrary function can be successfully called, the DLL file is legal. Otherwise, if the LoadLibrary function fails, the DLL file is invalid. After calling the LoadLibrary function for each DLL file, the illegal DLL file is unloaded from the load directory, and the legal DLL file is retained.

Further, for the anchor-side live application, although a plug-in is legal, the anchor-side live application may not support the plug-in. Therefore, in the embodiment of the present invention, in order to ensure an alternative provided for the final control process. The plug-ins are all plug-ins supported by the live broadcast application on the host, and further need to be filtered for the legal DLL file to obtain the plug-in supported by the broadcast application of the anchor.

In the specific implementation process, the plug-in enumerator process performs filtering according to different supporting conditions, and the present invention does not specifically limit. For example, the plug-in supported by the host-side live application is specifically a plug-in with an export function. Then, the specific method for determining whether the legal plug-in can be supported is: calling the GetProcAddress function of the system to obtain the export function in a legal DLL file. If the export function can be obtained, it is determined that the plug-in corresponding to the DLL file is a plug-in that the live broadcast application can support, and then the plug-in is reserved as an alternative plug-in; otherwise, if the export function is not obtained, the DLL file is determined. The corresponding plug-in is a plug-in that is not supported by the live broadcast application, and is then loaded from the directory. Uninstall the plugin.

Through the above process, one or more alternative plugins are obtained. Next, the plugin enumerator process provides the name of the determined alternative plugin to the master process through the RPC channel with the master process.

Further, in order to save system resources of the anchor device, after the plug-in enumerator process enumerates the candidate plug-ins existing in the current loading directory and can be loaded to the main control process, the method further includes:

The master process sends an exit instruction to the plug-in enumerator process;

Specifically, after receiving the plugin enumerated by the plugin enumerator process, the master process sends an exit instruction to the plugin enumeration process. After the plugin enumerator process receives the exit instruction, it calls the enumeration exit function to exit the plugin enumerator process.

Next, the master process automatically selects the target plugin from the candidate plugin, or determines the corresponding target plugin according to the selection operation of the anchor, and then notifies the plugin loading process by the name of the target plugin through the RPC channel of the plugin loading process. Further, the plug-in loading process acquires the target plug-in DLL file according to the name of the target process, thereby loading and running the target plug-in.

A specific example, assuming that the currently loaded plug-in needs to call the adding method of the master process to load, then the plug-in loading process calls the adding method of the master process by calling the RPC channel of the master process, so that the plug-in detects the adding method, thereby Can load successfully. Further, the main control process invokes the plug-in to run, and the plug-in loading process receives the instruction that is triggered by the main control process and is triggered by the main control process, and triggers the plug-in to run. The plug-in loading process needs to use the subtraction method. Then, the plug-in loading process calls the subtraction method of the master process by calling the RPC channel of the master process, so that the plug-in uses the subtraction method to process, thereby obtaining the processing result. The plug-in loading process obtains the result of running the plug-in, and feeds back the running result of the plug-in to the main control process through the RPC channel of the main control process.

In addition, if the loaded plug-in generates images and audio, that is, the plug-in needs to display images and sounds to the anchor, the plug-in loading process needs to transfer the images and audio generated by the plug-in to the master process through the RPC channel of the master process. Then, the master process displays the image according to the image drawing window generated by the plug-in, and plays the audio through the player, thereby displaying the image and sound generated by the plug-in to the anchor.

Of course, the master process also needs to monitor the plug-in loading process. Therefore, the embodiment of the present invention further includes the following solutions:

The master process monitors whether the plug-in loading process crashes;

Specifically, the master process monitors the running state of the plug-in loading process. If the plug-in loading process is detected to crash, the plug-in enumerator process and plug-in are recreated by creating a plug-in enumerator process and a plug-in loading process as described above. The loading process is not repeated here.

Based on the same inventive concept as the method of the live broadcast in the foregoing embodiment, the second aspect of the present invention further provides a live broadcast device, as shown in FIG. 2, including:

The startup unit 101 is configured to start a live broadcast application of the anchor end;

The master control process obtaining unit 102 is configured to acquire a master control process of the live broadcast application of the anchor end;

The process creation unit 103 is configured to create an encoding process and/or a plug-in loading process, where the encoding process is used to encode a video stream based on the control of the master process, and feed back the video stream to the master process. The plug-in loading process is configured to load a plug-in in the plug-in loading process, and invoke the loaded plug-in according to the control of the master process;

a video stream obtaining unit 104, configured to acquire the video stream;

The sending unit 105 is configured to send the video stream to a live broadcast server, so that the live broadcast server forwards the video stream to a viewer device that accesses the live broadcast server.

Specifically, the master control process obtaining unit 102 is configured to determine a preset operation mode of the anchor live application, and if the operation mode is the first mode, create the a master process, wherein, in the first mode, the master process creates the encoding process and the plug-in loading process; if the running mode is the second mode, acquiring the preset master An address of the process, and accessing the master process based on the address, wherein, in the second mode, the master process creates the encoding process and the plug-in loading process.

Further, the device further includes:

a mode determining unit, configured to determine, before the creating a coding process or a plug-in loading process by the main control process, a preset operation mode of the live broadcast application of the anchor end;

a mark obtaining unit, configured to: if the running mode is the third mode, acquire a mark bit indicating that the encoding process and/or the plug-in loading process needs to be created before creating an encoding process and/or a plug-in loading process;

If the flag bit is the first tag bit, the process creation unit 103 is specifically configured to create the plug-in loading process by using the master process;

If the flag bit is the second flag bit, the process creation unit 103 is specifically configured to create the encoding process by using the master process.

Further, if the master process creates the encoding process, the video stream obtaining unit 104 is configured to send an encoding instruction to the encoding stub module to notify the encoding process to encode a video stream, where the encoding process is invoked and An encoding local service module coupled to the encoding stub module encodes the video stream, and the encoding process invokes an encoding proxy interface module coupled to the encoding local service module to feed back the encoded video stream to the video stream obtaining unit.

If the master process does not create the encoding process, the video stream obtaining unit 104 is configured to send an encoding instruction to the encoding stub module, where the encoding stub module invokes the encoding stub module. The coupled coded local service module encodes the video stream, and the coded local service module invokes a coded proxy interface module coupled to the coded local service module to feed back the encoded video stream to the video stream acquisition unit.

(1) If the embodiment of the present invention needs to create an encoding process, the process creating unit 103 is configured to invoke a process creation function, pass the ID of the master process to the process creation function, and run the process creation function to create The encoding process, after the creation process is completed, initialize the process environment, and connect the master control process based on the ID of the master process that the process creation function has.

Further, the device further includes an encoding process environment initial unit, configured to initialize a process environment after the encoding process is completed, and connect the master control based on an ID of the master process that the process creation function has a process, specifically for registering an initialization service function and a video stream encoding service function in the encoding process, creating a video shared memory storing a video material, and storing an audio shared memory of the audio material, acquiring the method that the process creation function has The ID of the master process is connected to the master process, and the identifier of the video shared memory and the identifier of the audio shared memory are sent to the master process based on the connection between the code process and the master process.

Further, the apparatus further includes an encoding initialization unit, configured to send an initialization command to the encoding process before the encoding the video stream by the encoding process and feeding back to the main control process, where the initialization command includes a nominal parameter for directing the encoding process to determine a decoding header parameter, the decoding header parameter including a video decoding header parameter and an audio decoding header parameter, the initialization service function being executed based on the initial command to obtain based on the nominal parameter Transmitting the video decoding header parameter and the audio decoding header parameter to the main control process, so that the main control process sets the video decoding header parameter and the audio decoding header parameter The audio decoding header parameter is sent to the live broadcast server, and the live broadcast server forwards the video decoding header parameter and the audio decoding header parameter to the audience end device.

The video stream obtaining unit 104 is configured to receive an encoding command sent by the master control process, and extract, according to the encoding command, a video material pre-stored by the master control process from the video sharing memory, and from the Extracting audio material pre-stored by the master process in the audio sharing memory; running the video stream encoding service function to encode the video material, obtaining video encoded data, and encoding the audio material to obtain audio encoding Data, the video stream including the video encoded data and the audio encoded data; and the video encoded data and the audio encoded data are fed back to the master process.

Further, the apparatus further includes an encoding process monitoring unit configured to monitor whether the encoding process crashes; if the encoding process is detected to crash, re-creating the encoding process.

(2) If the master process creates the plug-in loading process, the device further includes:

Then, the process creation unit 103 is further configured to create a plug-in enumerator process before loading the target plug-in that needs to be loaded in the live application in the plug-in loading process;

Specifically, the process creation unit 103 is configured to invoke a process creation function, and send the ID of the master process to the process creation function, and run the process creation function to create the plug-in enumerator process. After the plug-in enumerator process is created, the process environment is initialized, and the master process is connected based on the ID of the master process that the process creation function has.

Further, the device further includes a plug-in loading process environment initial unit, configured to initialize a process environment after the plug-in enumerator process is created, and based on the ID connection of the master process that the process creation function has The master control process is specifically configured to register an enumeration plug-in function and an enumeration exit function in the plug-in enumerator process, and obtain an ID of the master control process that is included in the process creation function to connect the master control process.

Further, the plug-in enumeration unit is configured to receive an enumeration instruction sent by the master control process, invoke the enumeration plug-in function process to read all plug-ins in the loading directory, and filter from all the plug-ins. A legal plug-in is extracted; and the backup plug-in supported by the anchor live application is filtered out from the legal plug-in.

Further, the apparatus further includes an exit unit, configured to enumerate the plug-in to the plug-in after the plug-in enumerator process enumerates the candidate plug-in existing in the current load directory and can be loaded to the master process The process sends an exit instruction, and the enumeration exit function is called to end the process based on the exit instruction.

Further, the device further includes a plug-in loading process monitoring unit, configured to monitor whether the plug-in loading process crashes, and if the plug-in loading process is detected to crash, re-creating the plug-in loading process.

The various changes and specific examples of the method for performing the live broadcast in the foregoing embodiment of FIG. 1 are also applicable to the live broadcast device of the present embodiment. The foregoing description of the live broadcast method can be clearly known to those skilled in the art. The implementation method of the live broadcast device, so for the sake of brevity of the description, it will not be described in detail here.

Based on the same inventive concept as the method of live broadcast in the foregoing embodiment, the present invention further provides a computer readable storage medium having stored thereon a computer program, which is implemented by a processor to implement the foregoing The steps of any method of the method of live broadcast.

Based on the same inventive concept as the live broadcast method in the foregoing embodiment, the present invention further provides a computer device, as shown in FIG. 3, including a memory 204, a processor 202, and being stored on the memory 204 and operable on the processor 202. The computer program, the processor 202, when the program is executed, implements the steps of any of the methods of controlling the progress bar as described above.

Wherein, in FIG. 3, a bus architecture (represented by bus 200), bus 200 can include any number of interconnected buses and bridges, and bus 200 will include one or more processors and memory 204 represented by processor 202. The various circuits of the memory are linked together. The bus 200 can also link various other circuits, such as peripherals, voltage regulators, and power management circuits, as is known in the art, and therefore, will not be further described herein. Bus interface 206 provides an interface between bus 200 and receiver 201 and transmitter 203. Receiver 201 and transmitter 203 may be the same component, i.e., a transceiver, providing means for communicating with various other devices on a transmission medium.

The processor 202 is responsible for managing the bus 200 and the usual processing, and the memory 204 can be used to store data used by the processor 202 when performing operations.

Those skilled in the art will appreciate that embodiments of the present invention can be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware. Moreover, the invention can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) including computer usable program code.

The present invention has been described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (system), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flowchart illustrations and/or FIG. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing device to produce a machine for the execution of instructions for execution by a processor of a computer or other programmable data processing device. Implemented in flowchart one Means of a process or a plurality of processes and/or block diagrams of a function specified in a block or blocks.

The computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture comprising the instruction device. The apparatus implements the functions specified in one or more blocks of a flow or a flow and/or block diagram of the flowchart.

These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device. The instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.

It is apparent that those skilled in the art can make various modifications and variations to the invention without departing from the spirit and scope of the invention. Thus, it is intended that the present invention cover the modifications and modifications of the invention

Claims

A method for loading a plug-in, comprising:

Start the live broadcast application on the anchor side;

Obtaining a master control process of the broadcast application of the anchor end;

The master process creates a plug-in loading process;

The plug-in loading process loads a target plug-in that needs to be loaded in the live application in the plug-in loading process.
The method of claim 1, wherein before the plug-in loading process loads the target plug-in to be loaded in the live application, the plug-in loading process further includes:

The master process creates a plugin enumerator process;

The plug-in enumerator process enumerates, to the master process, an alternative plug-in that exists in the current load directory and can be loaded;

The master process determines to determine the target plugin from the candidate plugin, and notifies the plugin to load the target plugin.
The method of claim 2 wherein said master process creates a plugin enumerator process comprising:

The master process calls a process creation function;

The master process sends the ID of the master process to the process creation function;

The master process runs the process creation function to create the plug-in enumerator process, after the plug-in enumerator process is initialized, initializes the process environment, and based on the master control of the process creation function The ID of the process is connected to the master process.
The method according to claim 3, wherein the plug-in enumerator process initializes a process environment after the creation is completed, and connects the host based on an ID of the master process that the process creation function has Control process, including:

Registering the enumeration plugin function and the enumeration exit function in the plugin enumerator process;

Obtaining an ID of the master process that the process creation function has is connected to the master process.
The method of claim 4, wherein the plug-in enumerator process enumerates, to the master process, candidate plug-ins that are present in the current load directory and that can be loaded, including:

The plug-in enumerator receives an enumeration instruction sent by the master control process, and invokes the enumeration plug-in function process to read all plug-ins in the loading directory;

Filter out legitimate plugins from all of the plugins;

Extracting the backup plug-in supported by the anchor live application from the legal plug-in.
The method of claim 5, further comprising: after the plug-in enumerator process enumerates, to the master process, an optional plug-in that is present in the current load directory and can be loaded,

The master process sends an exit instruction to the plug-in enumerator process;

The plug-in enumerator invokes the enumeration exit function to end the process based on the exit instruction.
The method of claim 1 wherein the method further comprises:

The master process monitors whether the plug-in loading process crashes;

If the master process monitors that the plugin loading process crashes, re-create the plugin loading process.
A device for loading a plug-in, comprising:

a startup unit, configured to start a live broadcast application of the anchor;

a master control process acquiring unit, configured to acquire a master control process of the live broadcast application of the anchor end;

a process creation unit for creating a plugin loading process;

a plug-in loading unit, configured to load a target plug-in that needs to be loaded in the live application during the plug-in loading process.
A computer readable storage medium having stored thereon a computer program, characterized in that the program, when executed by a processor, implements the steps of the method of any of claims 1-7.
A computer apparatus comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, wherein the processor executes the program to implement any one of claims 1-7 The steps of the method.