CN113038155A - Live broadcast method and device - Google Patents
Live broadcast method and device Download PDFInfo
- Publication number
- CN113038155A CN113038155A CN202110236072.6A CN202110236072A CN113038155A CN 113038155 A CN113038155 A CN 113038155A CN 202110236072 A CN202110236072 A CN 202110236072A CN 113038155 A CN113038155 A CN 113038155A
- Authority
- CN
- China
- Prior art keywords
- window
- mode
- capture
- target process
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 229
- 230000008569 process Effects 0.000 claims abstract description 187
- 230000001960 triggered effect Effects 0.000 claims abstract description 22
- 238000012544 monitoring process Methods 0.000 claims description 36
- 230000006378 damage Effects 0.000 claims description 22
- 238000004590 computer program Methods 0.000 claims description 3
- 230000008859 change Effects 0.000 abstract description 9
- 230000006870 function Effects 0.000 description 32
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 2
- 230000033764 rhythmic process Effects 0.000 description 2
- 238000012216 screening Methods 0.000 description 2
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001151 other effect Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000008707 rearrangement Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/2187—Live feed
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/258—Client or end-user data management, e.g. managing client capabilities, user preferences or demographics, processing of multiple end-users preferences to derive collaborative data
- H04N21/25866—Management of end-user data
- H04N21/25891—Management of end-user data being end-user preferences
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/431—Generation of visual interfaces for content selection or interaction; Content or additional data rendering
- H04N21/4312—Generation of visual interfaces for content selection or interaction; Content or additional data rendering involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/45—Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
- H04N21/466—Learning process for intelligent management, e.g. learning user preferences for recommending movies
- H04N21/4668—Learning process for intelligent management, e.g. learning user preferences for recommending movies for recommending content, e.g. movies
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Graphics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a live broadcast method and a live broadcast device, which are applied to a live broadcast application program, wherein the method comprises the following steps: when detecting that the window capture function of the live application program is triggered, determining a target process needing window capture; carrying out window capture on the target process, and adding the captured window image into a live stream for pushing; when the target process is monitored to be switched from the first window mode to the second window mode, window capture is switched to capture the window image in the second window mode, and the captured image is switched in time according to the change of the window mode of the target process, so that the function of live broadcast in the office field is improved, and the use experience of a user is improved.
Description
Technical Field
The embodiment of the application relates to a data processing technology, in particular to a live broadcast method and a live broadcast device.
Background
With the development of network live broadcast, live broadcast starts to move to the office field, and in the office field, a slide show (also called PPT) is one of the most common live broadcast contents. The general slide can be divided into three modes, which correspond to three different window pictures. The first one is a common window mode which is mainly used for editing, previewing and other operations, and all the operations of modifying, beautifying and the like of the PPT are in the common window mode; the second mode is a projection mode, an entrance is in a common window mode, the PPT can be displayed in full screen after clicking, the next page can be automatically switched by clicking, auxiliary demonstration functions such as a painting brush and the like can be used in the process, and a window in the projection mode is a window which is mainly required to show demonstration when a main broadcast is live broadcast; the third is a speaker mode, which can be started under a plurality of screens and is used for displaying remark contents to the speaker, has the functions of prompter, rhythm control and the like, and is not displayed externally generally.
In the related art, live broadcast software only captures images in a common window mode during live broadcast, and does not automatically switch captured windows when a speaker shows the images, and at the moment, a screen enters a full-screen mode, so that the speaker usually cannot keep track that the live broadcast window is still in the common window mode instead of the required full-screen mode, and inconvenience in use is caused.
Disclosure of Invention
The application provides a live broadcasting method and a live broadcasting device, which are used for solving the problem that a captured picture cannot be automatically switched along with a picture of a target process when the target process is captured by a window for live broadcasting in the prior art.
In a first aspect, an embodiment of the present application provides a live broadcast method, where the method is applied to a live broadcast application program, and the method includes:
when detecting that the window capture function of the live application program is triggered, determining a target process needing window capture;
carrying out window capture on the target process, and adding the captured window image into a live stream for pushing;
and when the target process is monitored to be switched from a first window mode to a second window mode, switching window capture is carried out so as to capture a window image in the second window mode.
In a second aspect, an embodiment of the present application further provides a live broadcast apparatus, where the apparatus is applied in a live broadcast application, and the apparatus includes:
the target process determining module is used for determining a target process needing window capture when detecting that the window capture function of the live application program is triggered;
the window capturing module is used for carrying out window capturing on the target process and adding a captured window image into a live stream for pushing;
and the window capturing and switching module is used for switching window capturing to capture a window image in the second window mode when the target process is monitored to be switched from the first window mode to the second window mode.
In a third aspect, an embodiment of the present application further provides an electronic device, where the electronic device includes:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the methods described above.
In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, where the computer program is executed by a processor to implement the method described above.
The technical scheme that this application provided has following beneficial effect:
in this embodiment, when the live application detects that its window capture function is triggered, can confirm the target process that needs to carry out window capture, then carry out window capture to the target process, and add the window image captured into the live stream and carry out the propelling movement, simultaneously, the live application can also monitor the change of the window display mode of target process, when monitoring that the target process switches from first window mode to second window mode, carry out the switch of window capture, in order to capture the window image under the second window mode, thereby in time switch the image captured according to the change of the window mode of target process, thereby the function of live broadcast in the office field has been perfected more, user's use experience has been promoted.
Drawings
Fig. 1 is a flowchart of an embodiment of a live broadcast method provided in an embodiment of the present application;
fig. 2 is a schematic view of a live interface in an edit window mode according to an embodiment of the present application;
fig. 3 is a schematic view of a live interface in a projection window mode according to an embodiment of the present application;
fig. 4 is a flowchart of a live broadcasting method according to a second embodiment of the present application;
FIG. 5 is a pseudo code diagram of a target process according to a second embodiment of the present disclosure;
fig. 6 is a schematic diagram of a function signature of the winner tproc type provided in the second embodiment of the present application;
fig. 7 is a timing diagram of capture window switching according to a second embodiment of the present application;
fig. 8 is a block diagram illustrating a structure of a live broadcast apparatus according to a third embodiment of the present application;
fig. 9 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be further noted that, for the convenience of description, only some of the structures related to the present application are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of an embodiment of a live broadcasting method provided in an embodiment of the present application, where the embodiment may be executed by a live broadcasting application program, and specifically includes the following steps:
In this embodiment, a window capture function may be included in the live application. The window capture referred by the window capture function refers to that when video live broadcast is carried out on the terminal, a window of designated software opened on the terminal is selected for capture, image data of the captured window is obtained and synthesized into a video picture to be sent out.
In one implementation, a window capture button may be present on the interface of the live application, which when clicked by the user triggers a window capture function. When the live application program detects that the window capture function is triggered, the target process specified by the user and needing window capture is further determined.
In one example, a user may specify a target process for which window capture is desired as follows: when the live application program detects that the window capture function is triggered, the process information (such as a process name and the like) of a currently running process displayed on a desktop can be acquired, and then the acquired process information is organized into a process list and displayed to a user. The user may select at least one process in the process list as the target process.
And 120, carrying out window capture on the target process, and adding the captured window image into a live stream for pushing.
In this step, after the live application program determines a target process that needs window capture, the live application program starts window capture on the target process to obtain a window image of the target process, and then adds the window image into a live stream for pushing. At the same time, on the client side of the live application, the captured window image may be displayed in the live window of the client. For example, assuming that an application scene is a scene in which a slideshow is used for live lecture, if a target process is a slideshow (also referred to as PPT) process, after a currently opened slideshow window is captured to obtain a window image, the window image of the slideshow window may be displayed in a live interface of a live APP, and a window of a live broadcaster (such as a presenter) may also be displayed in the live interface, such as the live interface shown in fig. 2, which may include a slideshow window image 10 and a presenter window 20.
In this embodiment, the target process may include at least two display modes (or window modes) including a first window mode and a second window mode. Illustratively, the first window mode may be a non-full-screen editing window mode, and the second window mode may be a full-screen window mode. For example, if the target process is a slide process, the first window mode may be a normal non-full screen window mode in which editing, previewing, etc. of the slide contents may be performed, such as the window mode with a toolbar shown in window image 10 of FIG. 2. The second window mode may be a slide show mode, i.e. a mode in which the PPT is displayed full screen. In other embodiments, the second window mode may also be a slide speaker mode for presenting the lecture content, the words, the rhythm control, and other effects to the speaker, but since the speaker mode is not generally presented to the outside, the second window mode is generally a slide show mode, and when necessary, the speaker mode may also be the second window mode.
In this embodiment, the live application program may also monitor the target process and learn the change of the display mode performed by the target in time during the process of capturing the target process. When the live application monitors that the target process is switched from the first window mode to the second window mode, window capture can be switched in time to capture window images in the second window mode, the window images in the second window mode are added into the live stream to be pushed, and the window images in the second window mode are displayed in a live interface of the current client. For example, as shown in the live interface of fig. 2, when the slideshow progress is switched from the normal edit window mode to the full-screen show mode, the live APP may monitor the switching in time, then capture the window image of the slideshow in the show mode, and display the window image of the slideshow in the live interface, as shown in the window image 30 of fig. 3, thereby realizing that the window image changes due to the change of the display mode of the monitored progress. The window image 30 may be displayed in a full screen in the live interface, and the hierarchical relationship of the presenter window 20 may be adjusted, for example, the window image is displayed in an upper layer.
When the method is implemented, the method for the live application to listen to the target process may include, but is not limited to, the following ways: setting a message hook in the target process by using a message mechanism of an operating system to monitor the target process, and monitoring all behaviors of the target process such as creation, destruction, mouse response and the like, so as to screen out an event switched to a second window mode; or polling the system window, scanning the global visual window at regular intervals, and once a window switched to the second window mode is found, considering that the window is switched to the second window mode.
In this embodiment, when the live application detects that its window capture function is triggered, can confirm the target process that needs to carry out window capture, then carry out window capture to the target process, and add the window image captured into the live stream and carry out the propelling movement, simultaneously, the live application can also monitor the change of the window display mode of target process, when monitoring that the target process switches from first window mode to second window mode, carry out the switch of window capture, in order to capture the window image under the second window mode, thereby in time switch the image captured according to the change of the window mode of target process, thereby the function of live broadcast in the office field has been perfected more, user's use experience has been promoted.
Example two
Fig. 4 is a flowchart of an embodiment of a live broadcast method according to the second embodiment of the present application, in this embodiment, a live broadcast application may include a main thread and a monitoring thread, where the monitoring thread is mainly responsible for monitoring and determining a change in a display mode of a target process, and the main thread is mainly responsible for processing and displaying content related to live broadcast. The embodiment may specifically include the following steps:
In this step, when the main thread of the live application detects that its window capture button is clicked, it is determined that the window capture function is triggered, and then the live application may determine a target process that needs to be window captured, which is specified by the user.
In an embodiment, the step of determining a target process that needs to perform window capture further includes the following steps:
at step 410-1, the process name of the target process specified by the user is determined.
In this step, when the main thread of the live application detects that the window capture function is triggered, the process name of the currently running process and the process name displayed on the desktop can be acquired, and then the acquired process name is organized into a process list and displayed to the user. The user may select at least one process in the process list as the target process. The main thread may obtain the process name of the target process selected by the user.
And step 410-2, calling an interface provided by the system according to the process name to obtain a corresponding first window handle.
In this step, after the main thread obtains the process name of the target process, an interface API provided by the operating system may be called to obtain the first window handle corresponding to the process name. Having the first window handle, the main thread may query for information related to the target process based on the first window handle.
Step 410-3, query the class name corresponding to the first window handle.
In one implementation, after the main thread obtains a first window handle of the target process, a GetClassName may be called to obtain a class name corresponding to the first window handle.
And step 410-4, if the class name corresponding to the first window handle is the class name related to the target process, determining that the target process is detected.
In this step, after obtaining the corresponding class name according to the first window handle, the main thread may determine whether the class name is a class name related to the target process, and if so, the main thread may determine that the target process is detected.
For example, if the target process is microsoft slide show process, microsoft slides have developed for many years to form a plurality of popular versions, such as 03, 07, 10, 13, 16, 19, etc., in which the class name of the slide show mode or speaker mode is screen class; the generic non-full screen window mode (or referred to as generic window mode) has a class name PP FrameClass, where "" # denotes a version code corresponding to a version number, for example, for the 03 version, it is "PP 11 FrameClass", for the 07 version, it is "PP 12 FrameClass", and for the 13 and later versions, it is "PPTFrameClass". Therefore, the class name related to the microsoft slide process may include screen class or PP frame class, and after the corresponding class name is obtained according to the window handle, it may be determined whether the class name is screen class or PP frame class, and if not, it may be determined that the currently determined process is not the target process; otherwise, if the class name corresponding to the window handle is one of the two, the currently judged process is judged to be the target process. The corresponding pseudo code of the determination process may be as shown in fig. 5, where in fig. 5, the class name is the corresponding class name obtained from the window handle.
And step 420, hanging a designated message hook in the target process.
In one implementation, after detecting the target process, the main thread may send a monitoring notification to the monitoring thread, where the monitoring notification may include relevant information of the target process, such as a process name, a class name, and the like of the target process, so as to notify the monitoring thread to monitor the target process.
After receiving the monitoring notification, the monitoring thread can call a related system API through a message mechanism of the operating system to hang a specified message hook and specify a callback function in the target process, and the operating system records the callback function of the specified message hook.
Illustratively, the designated message hook may comprise a global hook (e.g., SetWindowshook for windows operating systems) or a lightweight message hook (e.g., SetWinEventhook for windows operating systems). The global hooks can monitor all the behaviors of creating, destroying, mouse responding and the like of a target process, but the global hooks can influence the running stability of the system, the system can be crashed if the global hooks are careless, and each piece of large security software has strict monitoring on the API of the system, is easy to be considered as virus software and deleted, and influences the appearance of a user. Therefore, the present embodiment may preferably inject a lightweight message hook into the target process, and the method of setting the global hook allows the listener to process the message, and the listener may interrupt the continuous delivery of the message, while the lightweight message hook may specify a specific thread and monitor a small amount of basic messages of the thread, without the need to insert the processing of the message, that is, without interrupting the continuous delivery of the message.
And 430, carrying out window capture on the target process, and adding the captured window image into a live stream for pushing.
When the target process is detected, the main thread can capture the window of the target process, so that the window image of the window displayed by the target process in real time is obtained, and then the window image captured in real time is added into the live stream for pushing. For example, if the target process is a slide process and the window mode of the current slide displayed on the desktop is a non-full-screen normal window mode, the main thread may capture a window image corresponding to the normal window mode.
In one implementation, the main thread may synthesize the captured window image and the image of the anchor collected by the camera to generate a synthesized image, and then add the synthesized image into the live stream for pushing, so that the spectator client may directly decode the live stream after receiving the pushed stream to obtain the synthesized image to be displayed in the spectator client.
In this step, when the user clicks a switch button in the target process, the target process may create a window of a new mode and generate a window creation message. For example, in the normal window mode of the slideshow process, when the user clicks the show button, the slideshow process may create a full screen window and generate a window creation message. The operating system captures the window creation message, and sends the window creation message to a monitoring thread of the live broadcast application program through a callback function, and the monitoring thread can further judge whether the window creation message is a target window creation message.
In one embodiment, step 440 may further include the steps of:
step 440-1, monitoring the object creation message of the target process through the designated message hook.
In practice, many operations of the target process are related to object creation during running, for example, object creation may be triggered when a window is created, object creation may also be triggered when a menu is created, object creation may also be triggered when a title is created, and the like. The designated message hook can monitor all object creating messages of the target process, and call back all the monitored object creating messages to the monitoring thread through the call-back function.
Step 440-2, determining whether the object creation message is a window creation message.
In this step, after receiving the object creation message called back by the system, the monitoring thread further determines whether the object creation message is a window creation message. For example, for a slide show process, the show mode is provided by a window, so the listening thread may first filter out non-window creation messages, leaving the window creation message EVENT _ OBJECT _ CREATE resulting from window creation.
Step 440-3, if the object creation message is a window creation message, determining whether the window creation message is created by a window object; and if the window creation message is created by the window object, determining that the target window creation message is monitored.
If the current object creation message is a window creation message, the listening thread needs to make further judgment to judge whether the window creation message is created by a window object. In particular, since the create message may be generated by the creation of the object itself, it may also be generated by the creation of a child object of the object. However, the embodiment only focuses on the creation message generated by creating the window object itself, because the message generated by creating the child object of the window object, such as creating a popup window, a toolbar, etc., is based on the action after the window object is created, and therefore, the message generated by creating the child object of the window object can be filtered out.
And if the monitoring thread judges that the current window creation message is created by the window object, judging that the target window creation message is monitored.
In this step, after the listening thread monitors the target window creation message, further finer window attribute screening may be performed to determine whether the window creation triggered by the current operation of the user is a window corresponding to the second window mode, and if the window creation triggered by the current operation of the user is a window corresponding to the second window mode, step 460 is performed.
In one embodiment, step 450 may further comprise the steps of:
step 450-1, obtain a second window handle of the target window creation message.
In this step, a second window handle may be included in the target window creation message, which may be taken from the target window creation message.
In one implementation, if the message hook is designated as SetWinEventHook, a function of the type wineventroc may be registered according to the signature of the SetWinEventHook function, and the complete signature of the function may be as shown in fig. 6. As can be seen from fig. 6, after the message hook captures the target window creation message, the second window handle hwnd of the message can be taken, and many related messages can be queried according to the second window handle.
And 450-2, inquiring the corresponding window class name and window style according to the second window handle.
After the listening thread obtains the second window handle, the GetClassName may be called to obtain the class name corresponding to the second window handle.
And step 450-3, judging whether the window class name is the appointed window class name.
Wherein the specified window class name is a class name associated with the second window mode. For example, if the first window mode is a normal window mode of a slide that is not a full screen; the second window mode is a full-screen window mode (or referred to as a presentation mode). As shown in step 410-4, the class name of the normal window mode is typically PP frame class, and the class name of the presentation window mode is typically screen class. After the window class name corresponding to the second window handle is obtained, if the class name is PP × FrameClass, it can be determined that the current window is created as a normal window; if the class name is screen class, it can be preliminarily decided that the current window is created as a window for the presentation mode.
Step 450-4, if the window class name is the designated window class name, determining that the window style is the maximized and minimized window style, and if the window style is not the maximized and minimized window style, determining that the window corresponding to the target window creation message is created as the window corresponding to the second window mode.
The reason why the preliminary determination in step 450-3 is that the current window is created as the window in the presentation mode is that the window class name of the speaker mode is also screen class, and therefore, the further determination needs to be made in combination with the window style.
For example, for a slide show process, the show window cannot be minimized and maximized, while the speaker mode window has a maximized and minimized attribute style, from which it can be determined whether the current window is a show mode window or a speaker window. If the window style obtained from the second window handle is not the window style with the maximization and the minimization, it can be determined that the window corresponding to the target window creation message is created as the window corresponding to the presentation mode.
In this step, when the listening thread determines that the window corresponding to the target window creation message is created and is a window corresponding to the second window mode, a switching notification may be sent to the main thread. After receiving the switching notification, the main thread can judge that the target process is switched from the first window mode to the second window mode, so that window capture can be switched in time to capture the window image in the second window mode, the picture of the target process is updated to the window image in the second window mode in the live broadcast interface of the client in time, and meanwhile, the window image in the second window mode is added into the live broadcast stream for stream pushing.
And 470, monitoring the destruction message of the target window of the target process through the designated message hook.
In this embodiment, the designated message hook may also listen for a target window destroy message for the target process. The process is similar to listening for target window creation messages. Specifically, the monitoring thread may monitor all object destruction messages of the target process by specifying the message hook, and then determine whether each object destruction message is a window destruction message, and if the object destruction message is a window destruction message and the window destruction message is generated by a window object instead of a window sub-object, it may be determined that the target window destruction message of the target process is monitored.
After the monitoring thread monitors the target window destruction message, further performing finer window attribute screening to determine whether the window destruction triggered by the current operation by the user is a window corresponding to the second window mode, and if the window destruction triggered by the current operation by the user is a window corresponding to the second window mode, performing step 490.
In one embodiment, the determination of step 480 may be as follows: and acquiring a third window handle of the target window destruction message, inquiring a corresponding window class name and a corresponding window style according to the third window handle, and if the window class name is a specified window class name and the window style is not a maximized or minimized window style, judging that the window corresponding to the target window destruction message is destroyed as a window corresponding to the second window mode.
In this step, when the listening thread determines that the window corresponding to the target window destruction message is created and is a window corresponding to the second window mode, a destruction switching notification may be sent to the main thread. After receiving the destroy switching notification, the main thread can judge that the target process is switched from the second window mode to the first window mode, so that window capture can be switched in time to capture the window image in the first window mode, the picture of the target process is updated to be the window image in the first window mode in the live broadcast interface of the client in time, and meanwhile, the window image in the first window mode is added into the live broadcast stream for stream pushing.
In order to enable those skilled in the art to better understand the embodiment, the embodiment is exemplarily described below by a specific example, where an application scenario of the embodiment is that when a window capturing PPT process is implemented on an operating system of a terminal to perform live broadcast, a capturing screen automatically follows the PPT to perform full-screen slide show switching. The implementation process of the application scenario can be divided into two steps from a large direction: the first step is to detect the "action" of the PPT and to issue a notification as soon as it is perceived to be projected in full screen; the second step is that the routine receiving the notification responds by switching the window capture area from the normal edit window to the full screen presentation window, thereby achieving the automatic slide program state following transition of the plug flow content.
It should be noted that, in this embodiment, a specific operating system is not limited, and may include a windows operating system, a MAC operating system, an android operating system, and the like, where this example takes windows as an example for illustration, and a detailed process is shown with reference to a timing diagram of fig. 7, and includes the following processes:
1. a user opens a PPT main thread;
2. a user starts a live application program, starts a window capture function and designates a target process needing window capture as a PPT main thread;
3. detecting a PPT process by a main thread of a live broadcast application program;
4. the PPT main thread creates an editing window (namely a common mode window) and displays the editing window;
5. after detecting the PPT process, the main thread captures a window image to obtain a window image of an editing window, displays the window image in a live interface of a client (as shown in FIG. 2), and adds the window image into a live stream to carry out stream pushing;
6. after detecting the PPT process, the main thread informs a monitoring thread of a live broadcast application program to monitor the PPT process;
7. a monitoring thread refers to a windows system mechanism and hangs a lightweight message hook to a PPT process;
8. recording the callback of the message hook by the windows system;
9. the user clicks a showing button in the PPT main thread, the PPT main thread creates a full-screen speech window (namely a showing mode window), generates a corresponding window creating message and sends the window creating message to an operating system;
10. the operating system monitors the window creation message through a message hook and sends the window creation message to a monitoring thread through a callback function;
11. the monitoring thread performs message filtering on the window creation message, and determines whether the window creation corresponding to the window creation message is a window corresponding to the projection mode (i.e., the processes of steps 440 and 450 described above), if so, the monitoring thread generates a switching notification and sends the switching notification to the main thread;
12. the main thread performs window capture on a full-screen window of the PPT main thread according to the switching notification to obtain a window image of the full-screen window, displays the window image of the full-screen window in a live interface of the client (as shown in FIG. 3), and adds the window image into a live stream to perform stream pushing.
In this embodiment, a window switching scheme is provided for window capture in a live application program to automatically follow a target process, after a target process requiring window capture is determined, a specified message hook is hung in the target process, a target window creation message of the target process is monitored through the specified message hook, if a window corresponding to the target window creation message is created as a window corresponding to a second window mode, the live application program may determine that it is detected that the target process is switched from the first window mode to the second window mode, and switch window capture in time to capture a window image in the second window mode, so that when the window capture target process is directly broadcast, a capture picture is automatically switched following a window mode change of the target process, and a window capture function of the live application program is completed.
EXAMPLE III
Fig. 8 is a block diagram of a live broadcast apparatus according to a third embodiment of the present application, where the apparatus may be applied to a live broadcast application, and includes the following modules:
a target process determining module 810, configured to determine a target process that needs to perform window capture when it is detected that a window capture function of the live application is triggered;
a window capture module 820, configured to perform window capture on the target process, and add the captured window image into a live stream for pushing;
the first window capture switching module 830 is configured to, when it is monitored that the target process is switched from the first window mode to the second window mode, perform switching of window capture to capture a window image in the second window mode.
In one embodiment, the first window mode is a non-full screen editing window mode; the second window mode is a full-screen window mode.
In one embodiment, the apparatus further comprises:
and the hooking module is used for hooking a specified message hook in the target process after the target process needing window capture is determined.
In one embodiment, the apparatus further comprises:
a target window creation message monitoring module, configured to monitor a target window creation message of the target process through the specified message hook;
a first window judging module, configured to judge whether the window creation corresponding to the target window creation message is a window corresponding to the second window mode; if yes, calling a window switching module; if not, the window capture module 820 is invoked;
and the window switching module is used for judging that the target process is switched from the first window mode to the second window mode.
In an embodiment, the target window creation message monitoring module is specifically configured to:
monitoring an object creation message of the target process through the designated message hook;
judging whether the object creating message is a window creating message or not;
if the object creation message is a window creation message, judging whether the window creation message is created by a window object;
and if the window creation message is created by the window object, determining that the target window creation message is monitored.
In an embodiment, the first window determining module is specifically configured to:
acquiring a second window handle of the target window creation message;
inquiring a corresponding window class name and a window style according to the second window handle;
judging whether the window class name is a specified window class name;
if the window class name is the appointed window class name, judging that the window style is the maximized and minimized window style;
and if the window style is not the maximized and minimized window style, judging that the window creation corresponding to the target window creation message is the window corresponding to the second window mode.
In one embodiment, the apparatus further comprises:
and the second window capture switching module is used for switching window capture to capture a window image in the first window mode when the target process is monitored to be switched from the second window mode to the first window mode.
In one embodiment, the apparatus further comprises:
the target window destroying message monitoring module is used for monitoring a target window destroying message of the target process through a specified message hook;
the second window judging module is used for judging whether the window destruction corresponding to the target window destruction message is the window corresponding to the second window mode; and if so, judging that the target process is switched back to the first window mode from the second window mode.
In an embodiment, the target process determining module 810 is specifically configured to:
determining a process name of a target process specified by a user;
calling an interface provided by a system according to the process name to obtain a corresponding first window handle;
querying a class name corresponding to the first window handle;
and if the class name corresponding to the first window handle is the class name related to the target process, determining that the target process is detected.
The live broadcast device provided by the embodiment of the application can execute the live broadcast method provided by the first embodiment or the second embodiment of the application, and has the corresponding functional modules and beneficial effects of the execution method.
Example four
Fig. 9 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present disclosure, where the electronic device may be a PC computing device, or a tablet computer or another mobile device, which is not limited in this embodiment. As shown in fig. 9, the electronic device includes a processor 910, a memory 920, an input device 930, and an output device 940; the number of the processors 910 in the electronic device may be one or more, and one processor 910 is taken as an example in fig. 9; the processor 910, the memory 920, the input device 930, and the output device 940 in the electronic apparatus may be connected by a bus or other means, and fig. 9 illustrates an example of connection by a bus.
The memory 920 is used as a computer readable storage medium for storing software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the above-mentioned embodiments in the embodiments of the present application. The processor 910 executes various functional applications and data processing of the electronic device by executing software programs, instructions and modules stored in the memory 920, namely, implements the methods mentioned in the above method embodiments.
The memory 920 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the memory 920 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, the memory 920 may further include memory located remotely from the processor 910, which may be connected to devices/terminals/servers via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 930 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function controls of the electronic apparatus. The output device 940 may include a display device such as a display screen.
EXAMPLE five
The fifth embodiment of the present application further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are used for executing the method in the above-mentioned method embodiment.
Of course, the storage medium provided in the embodiments of the present application contains computer-executable instructions, and the computer-executable instructions are not limited to the method operations described above, and may also perform related operations in the method provided in any embodiments of the present application.
From the above description of the embodiments, it is obvious for those skilled in the art that the present application can be implemented by software and necessary general hardware, and certainly can be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling an electronic device (which may be a personal computer, a server, or a network device) to execute the methods described in the embodiments of the present application.
It should be noted that, in the embodiment of the apparatus, the included units and modules are merely divided according to functional logic, but are not limited to the above division as long as the corresponding functions can be implemented; in addition, specific names of the functional units are only used for distinguishing one functional unit from another, and are not used for limiting the protection scope of the application.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present application and the technical principles employed. It will be understood by those skilled in the art that the present application is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the application. Therefore, although the present application has been described in more detail with reference to the above embodiments, the present application is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims (12)
1. A live broadcasting method is applied to a live broadcasting application program, and comprises the following steps:
when detecting that the window capture function of the live application program is triggered, determining a target process needing window capture;
carrying out window capture on the target process, and adding the captured window image into a live stream for pushing;
and when the target process is monitored to be switched from a first window mode to a second window mode, switching window capture is carried out so as to capture a window image in the second window mode.
2. The method of claim 1, wherein the first window mode is a non-full screen editing window mode; the second window mode is a full-screen window mode.
3. The method of claim 1 or 2, wherein after determining that a target process for window capture is required when detecting that a window capture function of the live application is triggered, the method further comprises:
and hanging a designated message hook in the target process.
4. The method according to claim 3, wherein before the switching of window capture to capture a window image in the second window mode when the target process is monitored to switch from the first window mode to the second window mode, the method further comprises:
monitoring a target window creation message of the target process through the designated message hook;
judging whether the window creation corresponding to the target window creation message is a window corresponding to the second window mode;
if so, judging that the target process is switched from a first window mode to a second window mode;
if not, continuing to execute the step of carrying out window capture on the target process and adding the captured window image into the live stream for pushing.
5. The method of claim 4, wherein listening to the target window creation message of the target process through the designated message hook comprises:
monitoring an object creation message of the target process through the designated message hook;
judging whether the object creating message is a window creating message or not;
if the object creation message is a window creation message, judging whether the window creation message is created by a window object;
and if the window creation message is created by the window object, determining that the target window creation message is monitored.
6. The method according to claim 5, wherein the determining whether the window creation corresponding to the target window creation message is a window corresponding to the second window mode comprises:
acquiring a second window handle of the target window creation message;
inquiring a corresponding window class name and a window style according to the second window handle;
judging whether the window class name is a specified window class name;
if the window class name is the appointed window class name, judging that the window style is the maximized and minimized window style;
and if the window style is not the maximized and minimized window style, judging that the window creation corresponding to the target window creation message is the window corresponding to the second window mode.
7. The method according to claim 1, wherein after the switching of window capture to capture a window image in a second window mode when the target process is monitored to switch from a first window mode to the second window mode, the method further comprises:
and when the target process is monitored to be switched back to the first window mode from the second window mode, switching window capture is carried out so as to capture a window image in the first window mode.
8. The method according to claim 7, wherein before the switching of window capture to capture the window image in the first window mode when the target process switches from the second window mode back to the first window mode is monitored, the method further comprises:
monitoring a target window destroying message of the target process through a designated message hook;
judging whether the window destruction corresponding to the target window destruction message is the window corresponding to the second window mode;
and if so, judging that the target process is switched back to the first window mode from the second window mode.
9. The method of claim 1, wherein determining the target process requiring window capture comprises:
determining a process name of a target process specified by a user;
calling an interface provided by a system according to the process name to obtain a corresponding first window handle;
querying a class name corresponding to the first window handle;
and if the class name corresponding to the first window handle is the class name related to the target process, determining that the target process is detected.
10. A live device, the device being applied in a live application, the device comprising:
the target process determining module is used for determining a target process needing window capture when detecting that the window capture function of the live application program is triggered;
the window capturing module is used for carrying out window capturing on the target process and adding a captured window image into a live stream for pushing;
and the window capturing and switching module is used for switching window capturing to capture a window image in the second window mode when the target process is monitored to be switched from the first window mode to the second window mode.
11. An electronic device, characterized in that the electronic device comprises:
one or more processors;
a storage device for storing one or more programs,
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.
12. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236072.6A CN113038155B (en) | 2021-03-03 | 2021-03-03 | Live broadcast method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110236072.6A CN113038155B (en) | 2021-03-03 | 2021-03-03 | Live broadcast method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113038155A true CN113038155A (en) | 2021-06-25 |
| CN113038155B CN113038155B (en) | 2024-03-01 |
Family
ID=76466005
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110236072.6A Active CN113038155B (en) | 2021-03-03 | 2021-03-03 | Live broadcast method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113038155B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060048058A1 (en) * | 2001-05-25 | 2006-03-02 | Learning Tree International | System and method for electronic presentations |
| CN106126244A (en) * | 2016-06-28 | 2016-11-16 | 北京金山云网络技术有限公司 | A kind of document processing method and device |
| CN107766024A (en) * | 2017-10-20 | 2018-03-06 | 威创集团股份有限公司 | A kind of PPT projection control methods and system based on combination |
| CN112269886A (en) * | 2020-11-11 | 2021-01-26 | 游密科技(深圳)有限公司 | Shared image acquisition method, window sharing method and shared image acquisition device |
-
2021
- 2021-03-03 CN CN202110236072.6A patent/CN113038155B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060048058A1 (en) * | 2001-05-25 | 2006-03-02 | Learning Tree International | System and method for electronic presentations |
| CN106126244A (en) * | 2016-06-28 | 2016-11-16 | 北京金山云网络技术有限公司 | A kind of document processing method and device |
| CN107766024A (en) * | 2017-10-20 | 2018-03-06 | 威创集团股份有限公司 | A kind of PPT projection control methods and system based on combination |
| CN112269886A (en) * | 2020-11-11 | 2021-01-26 | 游密科技(深圳)有限公司 | Shared image acquisition method, window sharing method and shared image acquisition device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113038155B (en) | 2024-03-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107659416B (en) | Conference record sharing method and device, conference terminal and storage medium | |
| US7598975B2 (en) | Automatic face extraction for use in recorded meetings timelines | |
| CN106454407A (en) | Video live broadcast method and device | |
| TWI604375B (en) | Methods for screen sharing and apparatuses using the same | |
| CN107483812B (en) | Multi-platform parallel live broadcast method and device | |
| WO2023000639A1 (en) | Video generation method and apparatus, electronic device, storage medium, and program | |
| CN110784735A (en) | Live broadcast method and device, mobile terminal, computer equipment and storage medium | |
| CN113032080B (en) | Page implementation method, application program, electronic device and storage medium | |
| CN114095776B (en) | Screen recording method and electronic equipment | |
| CN105872832A (en) | Video calling method, video calling system and display device | |
| CN113178015A (en) | House source interaction method and device, electronic equipment and storage medium | |
| JP2010206745A (en) | Communication terminal, display control method, and display control program | |
| WO2023273029A1 (en) | Video processing method and apparatus, device, storage medium, and program product | |
| CN111064930A (en) | Split screen display method, display terminal and storage device | |
| CN110716704A (en) | WEB-based spliced large screen display layout processing method | |
| CN112449159B (en) | Monitoring video display control method and device, electronic equipment and storage medium | |
| CN113038155B (en) | Live broadcast method and device | |
| CN114762320A (en) | Video highlight screen recording method and device and readable storage medium | |
| CN112055164A (en) | Information interaction method, device, terminal and storage medium | |
| CN113014852A (en) | Information prompting method, device and equipment | |
| CN109145273B (en) | Comment following display method, device, equipment and storage medium | |
| CN111107283A (en) | Information display method, electronic equipment and storage medium | |
| WO2018149170A1 (en) | Cross-application control method and device | |
| CN112118414B (en) | Video session method, electronic device, and computer storage medium | |
| CN113891135B (en) | Multimedia data playing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |