CN111190558A

CN111190558A - Screen projection control method and device, computer readable storage medium and computer equipment

Info

Publication number: CN111190558A
Application number: CN201811358530.8A
Authority: CN
Inventors: 陈汉聪; 刘嘉卿; 林经纬; 吴怡; 林叶辉
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-11-15
Filing date: 2018-11-15
Publication date: 2020-05-22
Anticipated expiration: 2038-11-15
Also published as: CN111190558B

Abstract

The application relates to a screen projection control method, a screen projection control device, a computer readable storage medium and computer equipment, wherein the method comprises the following steps: positioning a target window in a window set corresponding to a current screen; calling an operating system interface to acquire window positions corresponding to windows in the window set except the target window; superposing screen data corresponding to windows except the target window in the window set according to corresponding window positions to obtain screen projection data; and sending the screen projection data, wherein the sent screen projection data is used for screen projection display. The scheme that this application provided can improve and throw the screen effect.

Description

Screen projection control method and device, computer readable storage medium and computer equipment

Technical Field

The present application relates to the field of computer technologies, and in particular, to a screen projection control method and apparatus, a computer-readable storage medium, and a computer device.

Background

With the development of computer technology, Personal Computers (PCs) have become an indispensable part of people's daily life and work. At present, in many scenes, such as teaching or meeting, related content needs to be displayed through a display screen with a large size, so that all participants can conveniently view the content intended to be viewed.

In a conventional screen projection technology, a picture on the screen of the whole screen projection terminal is projected to a screen projection device in a whole screen projection mode. Therefore, in the screen projection process, the screen projection effect can be influenced by redundant pictures on the screen of the screen projection terminal, and the screen projection quality is reduced.

Disclosure of Invention

Therefore, it is necessary to provide a screen projection control method, a screen projection control device, a computer-readable storage medium, and a computer device for solving the technical problem of screen projection quality compromise at present.

A screen projection control method comprises the following steps:

positioning a target window in a window set corresponding to a current screen;

calling an operating system interface to acquire window positions corresponding to windows in the window set except the target window;

superposing screen data corresponding to windows except the target window in the window set according to corresponding window positions to obtain screen projection data;

and sending the screen projection data, wherein the sent screen projection data is used for screen projection display.

A screen projection control device, comprising:

the positioning module is used for positioning a target window in a window set corresponding to a current screen;

the calling module is used for calling an operating system interface to acquire the window position corresponding to each window except the target window in the window set;

the superposition module is used for superposing the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data;

and the sending module is used for sending the screen projection data, and the sent screen projection data is used for screen projection display.

A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the above-described screen projection control method.

A computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the above-described screen projection control method.

According to the screen projection control method, the screen projection control device, the computer readable storage medium and the computer equipment, after the target window is positioned in the window set corresponding to the current screen, the operating system interface is automatically called, and the window position corresponding to each window except the target window in the window set is obtained. In this way, the screen data corresponding to the windows in the window set except the target window can be superposed according to the corresponding window positions to obtain screen projection data, namely the screen images of the target window are eliminated; and the screen projection data is sent again to project the screen, so that the whole screen on the screen of the whole screen projection terminal is not projected onto the screen projection equipment, the screen projection effect is prevented from being influenced by redundant pictures, and the screen projection quality is improved.

Drawings

FIG. 1 is a diagram of an exemplary implementation of a screen projection control method;

FIG. 2 is a flow diagram illustrating a screen projection control method according to an embodiment;

FIG. 3 is a diagram of a screen interface of a projection terminal in one embodiment;

FIG. 4 is a screen interface diagram of a screen projection terminal displaying a screen projection control interface in one embodiment;

FIG. 5 is a schematic diagram of an interface displayed by the screen projection device after receiving screen projection data in one embodiment;

FIG. 6 is a schematic diagram of an interface displayed by the screen projection device after receiving screen projection data in another embodiment;

FIG. 7 is a flow diagram that illustrates not collecting screen data for a target window in one embodiment;

FIG. 8 is a schematic diagram of an interface displayed by the screen projection device after receiving screen projection data in another embodiment;

FIG. 9 is a schematic diagram of an interface displayed by the screen projection device after receiving screen projection data in another embodiment;

FIG. 10 is a flow diagram of selecting screen data for a window for collection in one embodiment;

FIG. 11 is a schematic diagram of an interface displayed by the screen projection device after receiving screen projection data in another embodiment;

FIG. 12 is a block diagram showing the construction of a screen projection control apparatus according to an embodiment;

FIG. 13 is a block diagram showing the construction of a screen projection control apparatus according to another embodiment;

FIG. 14 is a block diagram showing a configuration of a computer device according to an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Fig. 1 is an application environment diagram of a screen projection control method in one embodiment. Referring to fig. 1, the screen projection control method is applied to a screen projection control system. The screen projection control system includes a screen projection terminal 110, a server 120, a screen projection device 130, and a display 140. The screen projection terminal 110 and the screen projection device 130 are connected to the server 120 through a network, and the display 140 is connected to the screen projection device 130. The screen projection terminal 110 may specifically be a desktop terminal or a mobile terminal, and the mobile terminal may specifically be at least one of a mobile phone, a tablet computer, a notebook computer, and the like. The server 120 may be an independent server, or may be a server cluster formed by a plurality of independent servers. The screen projection device 130 may be an electronic device such as a screen projection box or a projector. It is understood that the hardware devices shown in fig. 1 are only some of the hardware devices related to the application environment of the present application, and do not constitute a limitation to the application environment of the present application, and a specific hardware device may include more or less hardware devices than those shown in the drawings, for example, when the projection device 130 is a projection box, the application environment may include the display 140, and when the projection device 130 is a projector, the application environment may not include the display 140.

Fig. 2 is a flowchart illustrating a screen projection control method according to an embodiment. The present embodiment is mainly illustrated by applying the screen projection control method to the screen projection terminal 110 in fig. 1. Referring to fig. 2, the screen projection control method specifically includes the following steps:

s202, positioning a target window in a window set corresponding to the current screen.

The window is a user interface which is displayed in a screen by the computer equipment in a window form. The user can interact with the computer device through the window. The computer device can change the size of the window, the display state of the window in the screen and the display position of the window in the screen according to preset operation triggered by a user. The preset operations include touch operations, mouse click or drag operations, shortcut key operations, and the like. The display state of the window in the screen is such as a maximized state or a minimized state. Of course, the window may also be a user interface with a fixed size and/or a fixed display state and/or a fixed display position, and the user cannot change the size of the window, the display state of the window in the screen, or the display position of the window in the screen by operation.

FIG. 3 illustrates a screen interface diagram of a screen projection terminal in one embodiment. Referring to FIG. 3, a number of windows are shown in the interface diagram, such as window 301, window 302, and window 303. The user can change the size of the window 301 or the window 302, the display state in the screen, and the display position in the screen by a mouse click or drag operation. The window 303 is a window with a fixed size and a fixed display position, and cannot be changed by a user. It is understood that the interface diagram shown in fig. 3 is only for illustration, the interface diagram specifically used in an actual scenario may include more or less contents than those shown in fig. 3, and the window in the diagram may be another characteristic, which is not limited in this embodiment.

The set of windows may include one or more windows. The window set corresponding to the current screen may be a set of all windows currently opened by the screen projection terminal. It will be appreciated that in other embodiments, typically a computer device presents a window in visual form after opening the window, i.e., in the current screen. The computer device can operate the window according to the user instruction. Such as moving, maximizing, or minimizing, etc. When the computer device minimizes the window according to the user instruction, the content in the window is no longer visually displayed in the current screen. The embodiments in the application are technical solutions applied to a screen projection scene, and the screen projection is to transmit content displayed on a screen (including a virtual screen) of a screen projection terminal to a screen projection device for display. In this embodiment, the window set corresponding to the current screen may also be a set of windows that are currently opened by the screen projection terminal and are not minimized.

The target window is the window selected as the target for some operation. In the embodiment of the present application, the target window may be a window that needs to be subjected to screen projection, or may be a window that does not need to be subjected to screen projection. The number of target windows may be one or more.

Specifically, after the screen projection terminal starts screen projection according to an instruction triggered by a user, an instruction transmitted by other computer equipment, or automatically, a window to be processed as a target may be determined, that is, a target window is positioned in a window set corresponding to a current screen.

In one embodiment, S202 includes: displaying window identifications of all windows in a window set corresponding to a current screen through a screen projection control interface; receiving a screen projection control instruction triggered by user operation; determining a window identifier selected by a screen projection control instruction; and positioning a target window in a window set corresponding to the current screen according to the determined window identification.

The screen projection control interface is an interface which is provided by the screen projection terminal and is used for a user to operate so as to interact with the screen projection terminal. The screen projection control interface may also be a window. The window identifier is used to uniquely identify a window. A window identification such as a window handle, etc. The screen projection control command is a computer program for triggering screen projection control.

Specifically, after the screen-casting terminal opens a screen-casting application or runs a screen-casting code, a screen-casting control interface can be displayed, and the window identifiers of the windows in the window set corresponding to the current screen are displayed in the screen-casting control interface. The screen projection terminal can then detect predefined user operation trigger, and trigger a corresponding screen projection control instruction when detecting the user trigger operation. The user trigger operation may be a trigger operation acting on the window identifier, such as a touch operation or a cursor click operation on the window identifier. The screen projection terminal can position a target window in the window set corresponding to the current screen according to the window identification selected by the screen projection control instruction.

Fig. 4 shows a screen interface diagram of a screen projection terminal displaying a screen projection control interface in one embodiment. Referring to fig. 4, the interface diagram includes a screen projection control interface 410, and a window identifier 411 is displayed in the screen projection control interface 410. The user can select the window identifier which is intended to be used as the target window through touch operation or cursor clicking operation. It will be appreciated that the interface diagram shown in FIG. 4 is for illustration only, and that an interface diagram specifically used in an actual scenario may include more or less content than that shown in FIG. 4

In further embodiments, an input box may also be included in the screen-projection control interface, in which the user may enter the window identification of the target window. Of course, the screen projection device may also detect a predefined trigger operation outside the screen projection control interface, such as clicking a predefined physical button, a voice instruction, or a shaking operation triggered when windows in a window set corresponding to the current screen are displayed in turn.

In the embodiment, a way for a user to select the target window in a user-defined manner is provided, so that in the actual screen projection process, the user can independently select the target window to project the screen or project the screen according to own will, and the flexibility of screen projection control is improved.

In one embodiment, S202 includes: reading screen projection control configuration information; and positioning a target window in the window set corresponding to the current screen according to the screen projection control configuration information.

The screen projection control configuration information is data which is configured in advance and used for controlling the screen projection process. The screen projection control configuration information may include a window identifier of the target window.

Specifically, when the developer sets the screen projection control configuration information, the developer may configure the window identifier of the target window in the screen projection control configuration information. For example, the window handle of the target window is configured in the screen projection control configuration information. Therefore, when the computer equipment is used for projecting the screen, the screen projecting control configuration information can be read, and the window handle of the target window is extracted from the screen projecting control configuration information so as to position the target window in the window set corresponding to the current screen.

In this embodiment, the target window is set in advance through the screen projection control configuration information, so that the target window can be quickly positioned in the window set corresponding to the current screen according to the screen projection control configuration information in the actual screen projection, and the screen projection efficiency is improved.

S204, calling an operating system interface, and acquiring the window position corresponding to each window except the target window in the window set.

The os Interface is an Application Programming Interface (API) provided by the os of the computer device to a user. Those skilled in the art will appreciate that the interfaces provided by different operating systems for obtaining the window position may be different, and the number of the interfaces may be one or more. The window position corresponding to the window is the display area position of the window in the current screen area. The window position may be an absolute position or a relative position.

The operating system may be, for example, a Windows (Windows) operating system. And the screen projection terminal based on the Windows operating system can call an operating system interface to traverse each window in the current screen to obtain the window handle of each window. For example, a screen-casting application is installed on the screen-casting terminal, and when the screen-casting application is run by the screen-casting terminal to prepare for screen casting, the screen-casting application may call an enum windows function, which enumerates all windows on the current screen, and transmits a window handle to a callback function of the screen-casting application. After traversing each window, the screen projection terminal can continue to call the operating system interface to obtain the window position corresponding to each window. For example, the screen projection application may call the createcomapatiblebitmap function to obtain the window position corresponding to each window.

For another example, the operating system may be a Mac OS (apple operating System) operating system. And the screen projection terminal based on the Mac OS operating system can call an operating system interface to traverse each window in the current screen, and acquire the window handle of each window and the window position corresponding to each window. For example, under the MAC OS operating system, by calling the system API: cfarrayrewsref ═ CGWindowListCreate (kccgwindowlistoptionscreenavowindow window, ((CGWindowID) kccgnulwindowi D)); and traversing the windows to obtain window handles of the windows and obtain the corresponding window positions of the windows.

S206, overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

The windows in the window set except the target window are windows needing screen projection operation. The number of windows in the window set other than the target window may be one or more.

The screen data corresponding to a window is the data in the area of the window in the screen. The screen projection, as the name implies, is to display data displayed on the screen of one device on another device or devices so that the data displayed on the screens of these devices are consistent. The screen projection data is data transmitted between devices in the screen projection process, that is, data for performing screen projection display.

In an embodiment, the number of the windows except the target window in the window set is one, and then the screen projection terminal can directly acquire the screen data of the only window except the target window in the window set at this time to obtain the screen projection data. Specifically, the screen projection terminal can draw the screen data corresponding to the only window except the target window in the window set onto the canvas according to the corresponding window position. Therefore, the screen projection equipment can directly obtain canvas data as screen projection data; or coding the canvas data to obtain screen projection data adaptive to the current network environment.

Referring to fig. 3 again, assuming that the window set corresponding to the current screen includes a window 301, a window 302, and a window 303, the screen projection terminal selects the window 301 and the window 303 as target windows according to a user trigger operation. At this time, the number of windows other than the target window in the window set is one, and the only window is the window 302.

Fig. 5 shows a schematic interface diagram displayed after the screen projection device receives the screen projection data in this embodiment. Comparing fig. 3 and 5, it can be seen that the screen in fig. 5 excludes the contents of window 301 and window 303 compared to the screen in fig. 3.

In one embodiment, S206 includes: when the number of the windows except the target window in the window set exceeds one, determining the stacking sequence of the windows except the target window in the window set in the current screen; and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

Wherein the stacking order of the windows is an arrangement order of the stacked plurality of windows. The screen area of the computer device is limited, and after the computer device opens a new window, the new window may partially cover the window that has already been opened. It can be understood that, in addition to the position information corresponding to the two dimensions of the screen, there is also position information in the dimension perpendicular to the screen in the window positions corresponding to the windows, and the position relationship between the windows in the dimension perpendicular to the screen is reflected by the position information. For example, the positional relationship between windows in the stacked plurality of windows is indicated by the Z-order of the windows. The plurality of windows are positioned along an imaginary Z-axis extending outward from the screen. The top window in Z order covers all other windows and the bottom window in Z order is covered by all other windows. When a window is newly opened by the computer device, the window may be positioned in the Z-order by placing it behind a given window, or on the top or bottom of a stack of windows, etc.

Specifically, the screen projection terminal may determine, according to the position information of the dimension perpendicular to the screen in the window position corresponding to each window, a stacking order of windows in the window set, other than the target window, in the current screen. After the stacking sequence is determined, the screen projection terminal can sequentially superimpose the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions according to the stacking sequence to obtain the screen projection data.

Referring to fig. 3 again, assuming that the window set corresponding to the current screen includes a window 301, a window 302, and a window 303, the screen-casting terminal selects the window 303 as a target window according to a user trigger operation. At this time, the number of windows other than the target window in the window set is plural, i.e., window 301 and window 302. Fig. 6 is a schematic diagram showing an interface displayed after the screen projection device receives screen projection data in this embodiment. Comparing fig. 3 and fig. 6, it can be seen that the screen in fig. 6 excludes the content of the window 303 compared to the screen in fig. 3, and the stacking order of the

windows

301 and 302 is consistent with the stacking order in the screen projection terminal, that is, consistent with the screen interface of the screen projection terminal except for the content of the window 303.

Referring to fig. 3, 5 and 6, it is apparent that, in the screen projection in the embodiment of the present application, the entire screen on the screen of the entire screen projection terminal is not projected onto the screen projection device, but the content of some windows is selectively excluded and is not projected, so that the screen projection content is controlled by a user.

It can be understood that the sequential overlapping of the screen data corresponding to the windows in the window set except the target window according to the corresponding window positions is to restore the position relationship of the content of each window in the screen picture of the screen projection terminal except the content in the target window.

In another embodiment, when there is no mutually overlapping region between windows except the target window in the window set, the screen projection terminal may directly superimpose the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions without determining the stacking order of the windows except the target window in the window set in the current screen, so as to obtain the screen projection data.

And S208, sending screen projection data, wherein the sent screen projection data are used for screen projection display.

Specifically, after the screen projection terminal acquires the screen projection data, the screen projection data can be directly uploaded to the server, and the server forwards the screen projection data to the screen projection equipment which establishes screen projection connection with the screen projection terminal so as to project the screen. Of course, after the screen projection terminal acquires the screen projection data, the screen projection data can also be directly sent to the screen projection equipment which establishes the screen projection connection.

After the screen projection equipment receives the screen projection data, the image processor renders the screen projection data to obtain displayable pictures, and the displayable pictures are transmitted to the display for displaying. The screen projection equipment can also be used for rendering a displayable picture according to the screen projection data and then forming optical projection in a physical optical mode.

According to the screen projection control method, after the target window is positioned in the window set corresponding to the current screen, the operating system interface is automatically called, and the window position corresponding to each window except the target window in the window set is obtained. In this way, the screen data corresponding to the windows in the window set except the target window can be superposed according to the corresponding window positions to obtain screen projection data, namely the screen images of the target window are eliminated; and the screen projection data is sent again to project the screen, so that the whole screen on the screen of the whole screen projection terminal is not projected onto the screen projection equipment, the screen projection effect is prevented from being influenced by redundant pictures, and the screen projection quality is improved.

In one embodiment, S206 includes: determining the stacking sequence of each window in the window set in the current screen; and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

Specifically, the screen projection terminal may determine the stacking order of each window in the window set in the current screen according to the position information of the dimension perpendicular to the screen in the window position corresponding to each window in the window set. At this time, the stacking order refers to all windows included in the window set.

In one embodiment, sequentially superimposing, according to the stacking order, screen data corresponding to windows in the window set except the target window according to corresponding window positions to obtain screen projection data, including: according to the stacking sequence, sequentially superposing and drawing the screen data corresponding to the windows except the target window in the window set on the canvas according to the corresponding window positions to obtain intermediate image data; and coding the intermediate image data to obtain screen projection data adaptive to the current network environment.

Among these, a canvas (canvas) may be understood as a graphics container. The screen projection terminal or an application program running on the screen projection terminal can draw on the canvas by calling a drawing interface, such as drawing texts, lines, rectangles, circles and other graphics. In embodiments of the present application, a screen projection application running on a screen projection terminal may draw screen data corresponding to a window on a canvas by calling a drawing interface. And the screen projection code running on the screen projection terminal can draw the screen data corresponding to the window on the canvas by calling the drawing interface.

Image coding, also called image compression, refers to representing an image or information contained in an image with a small number of bits under the condition that a certain quality (a requirement of a signal-to-noise ratio or a subjective evaluation score) is satisfied.

In one embodiment, the sequentially overlaying and drawing the screen data corresponding to the windows except the target window in the window set on the canvas according to the corresponding window positions according to the stacking sequence to obtain the intermediate image data includes: initializing a canvas; traversing each window in the window set according to the stacking sequence; when the traversed window is not the target window, overlaying and drawing the screen data corresponding to the traversed window on the canvas according to the corresponding window position, and continuing to traverse; and when the traversed window is the target window, skipping the target window and continuing the traversal.

Specifically, the screen projection terminal may initialize the canvas, traverse each window in the window set according to the stacking order, and compare the window handle corresponding to the currently traversed window with the window handle of the target window.

When the window handle corresponding to the currently traversed window is inconsistent with the window handle of the target window, judging that the currently traversed window is not the window needing to be eliminated, overlaying and drawing the screen data corresponding to the currently traversed window to the canvas according to the window position corresponding to the currently traversed window, and continuing traversing until the last window.

When the window handle corresponding to the currently traversed window is consistent with the window handle of the target window, the currently traversed window is judged to be the window needing to be eliminated, and the window is skipped, namely, the screen data corresponding to the window does not need to be drawn on the canvas, and then the traversal is continued until the last window.

Thus, the image content on the canvas is the screen content of the current screen content excluding the screen data of the target window, i.e., the screen content intended for screen projection.

When the traversed window is in a minimized state, the screen data corresponding to the window is empty. When the screen projection terminal draws the canvas, the screen projection terminal can skip the drawing as a target window and can also draw a layer with complete transparency once.

It can be understood that the background image in the initialized canvas is the screen background of the current screen of the screen projection terminal.

In this embodiment, the screen data of the windows other than the target window are sequentially drawn on the canvas according to the stacking sequence of the windows in the current screen in the window set to obtain the screen content excluding the screen data of the target window, so that when the screen is projected again, all the pictures on the screen of the whole screen projection terminal are not projected on the screen projection equipment, the screen projection effect is prevented from being influenced by redundant pictures, and the screen projection quality is improved.

It will be appreciated that, in general, the files of screen images drawn on a canvas are typically large, i.e. the number of bytes of the image files is large, typically exceeding 10M. When the image file is large, the occupied bandwidth in the transmission process is wide, and the requirement on the network quality is high. And the image precision is not required to be so high in the screen projection process. Therefore, the screen projection device usually encodes the intermediate image data obtained through the canvas, and under the condition that a certain quality (a signal-to-noise ratio requirement or a subjective evaluation score) is met and the image or information contained in the image is matched with the current network environment, the screen projection data is obtained by representing the image or the information contained in the image with a small number of bits.

FIG. 7 illustrates a flow diagram for not collecting screen data for a target window in one embodiment. Referring to fig. 7, after determining the target window, the screen projection terminal may first obtain screen data corresponding to each window, and then traverse the windows according to the stacking order of the windows, that is, the stacking order of each window in the window set in the current screen, to determine whether the traversed window is the target window. And when the traversed window is not the target window, overlaying and drawing the screen data corresponding to the traversed window on the canvas according to the window position corresponding to the traversed window. And the screen projection terminal can judge whether traversal is finished or not after the drawing is finished. If yes, obtaining an output canvas image to obtain intermediate image data, and coding the intermediate image data into screen projection data; if not, the windows are continuously traversed according to the stacking sequence of the windows. And when the traversed window is the target window, continuing to traverse the windows according to the stacking sequence of the windows. Of course, the screen projection terminal may also acquire the screen data corresponding to the traversed window when the window is traversed and when it is determined that the traversed window is not the target window.

In the above embodiment, after the intermediate image data obtained through the canvas is encoded to obtain the screen projection data, the screen projection data is transmitted to perform screen projection, so that the transmitted data volume is reduced, the screen projection efficiency is improved, and the influence of the network environment is small.

In another embodiment, according to the stacking order, the screen data corresponding to the windows in the window set except the target window are sequentially superimposed according to the corresponding window positions, and the step of obtaining the screen projection data may be completed by a screen projection code or a screen projection application running in the screen projection terminal, or completed by the screen projection code or the screen projection application running in the screen projection terminal calling an interface provided by an operating system. Of course, the screen data corresponding to the windows in the window set except the target window may also be sequentially overlaid and drawn on the canvas according to the corresponding window positions only according to the stacking order, and the step of obtaining the intermediate image data is completed by a screen projection code running in the screen projection terminal or a screen projection application calling an interface provided by the operating system.

For example, in the screen projection terminal based on the Mac OS, the operating system interface may be invoked to sequentially overlay and draw the screen data corresponding to the windows in the window set, except the target window, onto the canvas according to the corresponding window positions, so as to obtain the intermediate image data.

For example, under the MAC OS operating system, by calling the system API: CGImageRef (cgwindowlistcreateimagefromararray (rect, mutWindowsIds, kcgdowimageno miniresolution) to perform layer merging, that is, drawing the screen data of each window in an overlapping manner.

In one embodiment, the screen projection control method further comprises: determining a window set corresponding to the current screen according to the frequency greater than or equal to the preset frequency; and after determining the window set corresponding to the current screen each time, re-executing the step of calling the operating system interface and acquiring the window positions corresponding to the windows in the window set except the target window.

It can be understood that, in the present application, the frequency of obtaining the screen projection data is controlled by controlling and determining the frequency of the window set corresponding to the current screen, so as to control the frame rate of the screen projection data transmission performed by the screen projection terminal. Due to the special physiological structure of human eyes, if the frame rate of the viewed pictures is higher than a certain frame rate, the pictures are considered to be coherent, and the phenomenon is called visual retention. According to the method and the device, the frame rate of screen projection data transmission of the screen projection terminal is controlled, so that the content displayed on the screen projection equipment is smooth in human vision.

Specifically, a developer can set a preset frequency, so that the screen projection device can determine a window set corresponding to the current screen according to the frequency greater than or equal to the preset frequency, and after determining the window set corresponding to the current screen each time, call an operating system interface to obtain window positions corresponding to windows in the window set except for a target window; determining the stacking sequence of each window in the window set in the current screen; according to the stacking sequence, sequentially superposing screen data corresponding to windows except the target window in the window set according to corresponding window positions to obtain screen projection data; and sending screen projection data, wherein the sent screen projection data are used for screen projection display. Therefore, the frame rate of the screen projection data transmission of the screen projection terminal can be controlled to be greater than or equal to the level of the preset frequency.

For example, the preset frequency may be specifically 15 times per second, that is, the screen projection terminal may execute the steps from S202 to S208 once every 66ms, so as to achieve a screen frame capture of 15fps excluding a specific window.

In this embodiment, the frame rate of the screen projection data transmission performed by the screen projection terminal is controlled by controlling the frequency of the screen projection data, so that the content displayed on the screen projection device is visually smooth for human eyes, and the screen projection quality is improved.

In one embodiment, the screen projection control method further comprises: when the target window is a window needing screen projection, screen projection data are directly obtained according to the screen data of the target window; and when the target window is a window which does not need to be projected, continuing to execute the step of calling the interface of the operating system and acquiring the window position corresponding to each window except the target window in the window set.

Specifically, as described in the foregoing embodiments, the target window in the present application may be a window that needs to be projected, or may be a window that does not need to be projected. Then, when determining the target window, the screen projection terminal may determine whether the target window is a window that needs to be projected or a window that does not need to be projected.

And when the target window is a window which does not need to be subjected to screen projection, the screen projection terminal continues to execute the steps from S204 to S208 in the embodiment. And when the target window is a window needing screen projection, the screen projection terminal directly obtains screen projection data according to the screen data of the target window.

In one embodiment, when the number of the target windows is one, the screen projection device can directly draw the screen data corresponding to the target windows onto the canvas to obtain intermediate image data; and encoding the intermediate image data to obtain screen projection data adaptive to the current network environment, and directly or indirectly transmitting the screen projection data to the screen projection equipment.

Referring to fig. 3 again, assuming that the window set corresponding to the current screen includes a window 301, a window 302, and a window 303, the screen-casting terminal selects the window 302 as a target window according to a user trigger operation. At this time, the number of target windows is one, i.e., window 302. Fig. 8 is a schematic diagram showing an interface displayed after the screen projection device receives screen projection data in this embodiment. Comparing fig. 3 and 8, it can be seen that the screen in fig. 8 selectively screens the contents of window 302 as compared to the screen in fig. 3.

Of course, when the number of the target windows is only one, the screen projection terminal may project the screen data of only the target windows as the screen projection data, as shown in fig. 9. Comparing fig. 3 and fig. 9, at this time, the display screen of the projection device only includes the screen data of the target window, but does not include the background data of the projection terminal.

In one embodiment, when the number of the target windows is multiple, the screen projection device may sequentially overlay and draw screen data corresponding to the multiple target windows onto a canvas according to corresponding window positions according to a stacking order of the windows in the window set in the current screen, so as to obtain intermediate image data; and coding the intermediate image data to obtain screen projection data adaptive to the current network environment.

FIG. 10 illustrates a flow diagram for selecting screen data for a window for collection in one embodiment. Referring to fig. 10, after determining the target window, and the target window is a window that needs to be subjected to screen projection, the screen projection terminal may traverse the window, and determine whether the traversed window is the target window. When the traversed window is not the target window, continuing to traverse the window; and when the traversed window is the target window, drawing the screen data corresponding to the traversed window on the canvas. And the screen projection terminal can judge whether traversal is finished or not after the drawing is finished. If yes, obtaining an output canvas image to obtain intermediate image data, and coding the intermediate image data into screen projection data; if not, the window is continuously traversed. When the number of the target windows is one, the traversal sequence of the traversal windows is not limited; when a plurality of target windows are provided, traversal can be performed according to the stacking sequence of the windows in the current screen in the window set, and when the screen data corresponding to the target windows are drawn on the canvas, the screen data can also be drawn in an overlapping manner according to the corresponding window positions.

Referring to fig. 3 again, assuming that the window set corresponding to the current screen includes a window 301, a window 302, and a window 303, the screen projection terminal selects the window 301 and the window 302 as target windows according to a user trigger operation. At this time, the number of target windows is plural, i.e., window 301 and window 302. Fig. 11 is a schematic diagram showing an interface displayed after the screen projection device receives screen projection data in this embodiment. Comparing fig. 3 and 11, it can be seen that the screen in fig. 11 selectively screens the contents of window 301 and window 302 compared to the screen in fig. 3.

Referring to fig. 3, 9 and 11, it is apparent that, in the screen projection in the embodiment of the present application, the screen is not projected on the screen projection device completely from the whole screen of the screen projection terminal, but the content of some windows is selectively projected, so that the screen projection content is controlled in a user-defined manner.

In the above embodiment, after the target window is positioned in the window set corresponding to the current screen and the target window is a window to be subjected to screen projection, screen projection data is directly obtained according to the screen data corresponding to the target window, that is, the screen image of one or more target windows is selectively selected; and the screen projection data is sent again to project the screen, so that the whole screen on the screen of the whole screen projection terminal is not projected onto the screen projection equipment, the screen projection effect is prevented from being influenced by redundant pictures, and the screen projection quality is improved.

It is understood that in the embodiments of the present application, the screen projection data may or may not include the screen background data of the computer device. The screen background data of the computer device comprises a background picture, a desktop icon, a task bar and the like. When the screen projection data comprise screen background data of the computer equipment, the background data of the canvas is the screen background data of the computer equipment; the background data of the canvas may be empty when the screen projection data does not include screen background data of the computer device.

In one embodiment, the screen projection control method further comprises: acquiring a screen projection code and a local terminal identifier; sending the screen projection code and the terminal identification to a server; and the transmitted screen projection code and the terminal identification are used for indicating the server to establish wireless screen projection connection between the terminal corresponding to the terminal identification and the screen projection equipment corresponding to the screen projection equipment identification after finding the screen projection equipment identification associated with the screen projection code. Sending screen projection data, comprising: uploading screen projection data to a server; and the uploaded screen projection data are used for instructing the server to forward the screen projection data to the screen projection equipment according to the wireless screen projection connection.

The screen-casting code is a certificate for screen casting, and may be a character string including at least one character of a number, a letter, or a symbol. Screen-casting codes such as "123456" or "abg 12 d", etc. Of course, the screen-shot code may also include characters of other languages, such as chinese characters. The terminal identifier is used to uniquely identify a terminal, and may be a character string including at least one character of a number, a letter, or a symbol. The terminal identifier may specifically be a terminal serial number or a currently logged-in user identifier. The currently logged-in user identifier may be a currently logged-in terminal system account or a currently logged-in application account.

Specifically, the screen projection terminal can receive a screen projection code input by a user on a screen projection terminal interface, or receive a screen projection code sent by other terminals or screen projection equipment, so as to obtain the screen projection code. In this embodiment, a screen projection code may be integrated into a terminal code of the screen projection terminal, so that the terminal has a screen projection function. The screen projection code is a computer readable instruction for executing the screen projection operation after the screen projection terminal runs. The screen projecting terminal or the screen projecting equipment can project the screen code through sound wave propagation.

In one embodiment, the screen projection terminal can also be provided with a screen projection application or a non-screen projection application integrated with screen projection codes. The screen projection application is an application program special for screen projection, and the non-screen projection application integrated with the screen projection code is not an application program special for screen projection but with a screen projection function, such as a WeChat application and the like.

Specifically, the screen projection terminal can start the screen projection application or the non-screen projection application integrated with the screen projection code according to a user instruction, and continuously acquire the screen projection code input by the user in the screen projection application or the non-screen projection application integrated with the screen projection code. And then, the screen projection terminal can continue to acquire the local terminal identification after receiving the screen projection instruction.

Further, after the screen projection terminal acquires the screen projection code and the local terminal identification, the screen projection code and the terminal identification are correspondingly uploaded to the server. After receiving the screen projection code and the terminal identification, the server searches the screen projection equipment identification associated with the screen projection code, judges the intention of the screen projection terminal and the screen projection equipment corresponding to the searched screen projection equipment identification to perform screen projection, and establishes screen projection connection between the screen projection terminal and the screen projection equipment corresponding to the screen projection equipment identification.

In one embodiment, after the screen projection connection between the screen projection terminal and the screen projection device corresponding to the screen projection device identifier is established, the server can feed back a notification of the success of the screen projection connection to the screen projection terminal, so that the screen projection terminal can execute subsequent operations. When the server does not find the screen projection equipment identification associated with the screen projection code, the server can also feed back a notice that the screen projection equipment identification associated with the screen projection code is not inquired to the screen projection terminal, and the screen projection terminal displays prompt information of screen projection code error according to the notice so as to remind a user of re-inputting the screen projection code.

In one embodiment, the screen projection terminal can start timing after sending the screen projection code and the terminal identifier to the server, and when the timing duration reaches the waiting duration and does not receive the notification fed back by the server, the screen projection terminal can judge that the screen projection terminal is disconnected from the server and display the prompt message of the network abnormity so as to remind a user of replacing the network or trying again later.

After the screen projection terminal and the screen projection equipment successfully establish screen projection connection, the screen projection terminal can directly upload screen projection data to a server after acquiring the screen projection data, and the server forwards the screen projection data to the screen projection equipment which establishes screen projection connection with the screen projection terminal so as to project a screen.

Naturally, the screen projection terminal can also acquire and upload screen projection data when uploading the screen projection code and the terminal identifier correspondingly to the server, and the server transmits the screen projection data to the screen projection equipment after determining the screen projection equipment which the screen projection terminal intends to project the screen.

In the above embodiment, after the screen projecting code and the local terminal identifier are obtained, the screen projecting code and the terminal identifier are sent to the server, and the server can establish screen projecting connection between the local screen projecting device and the screen projecting device corresponding to the screen projecting device identifier after finding the screen projecting device identifier associated with the screen projecting code. Therefore, local screen data can be acquired and sent to the server after the screen projection connection is successful, and then the screen data is forwarded to the screen projection equipment by the server for display, so that the screen projection from the local to the screen projection equipment is completed. Therefore, the screen can be projected without being connected with the screen projecting equipment in a wired mode or being connected to the same wireless local area network with the screen projecting equipment, and the screen projecting application range is greatly widened.

It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, the steps are not necessarily executed in sequence as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least a part of the steps in the above embodiments may include multiple sub-steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of performing the sub-steps or the stages is not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least a part of the sub-steps or the stages of other steps.

As shown in fig. 12, in one embodiment, a screen projection control apparatus 1200 is provided. Referring to fig. 12, the screen projection control apparatus 1200 includes: a positioning module 1201, a calling module 1202, an overlay module 1203 and a sending module 1204.

A positioning module 1201, configured to position a target window in a window set corresponding to a current screen.

The invoking module 1202 is configured to invoke an operating system interface, and obtain window positions corresponding to windows in the window set, except for the target window.

And a 12 superimposing module 1204, configured to superimpose, according to the corresponding window position, screen data corresponding to windows other than the target window in the window set, so as to obtain screen projection data.

And a sending module 1205 for sending the screen projection data, where the sent screen projection data is used for screen projection display.

After the target window is located in the window set corresponding to the current screen, the screen projection control device 1200 automatically calls the operating system interface to obtain the window position corresponding to each window in the window set except the target window. In this way, the screen data corresponding to the windows in the window set except the target window can be superposed according to the corresponding window positions to obtain screen projection data, namely the screen images of the target window are eliminated; and the screen projection data is sent again to project the screen, so that the whole screen on the screen of the whole screen projection terminal is not projected onto the screen projection equipment, the screen projection effect is prevented from being influenced by redundant pictures, and the screen projection quality is improved.

In one embodiment, the positioning module 1201 is further configured to display, through the screen-casting control interface, window identifiers of windows in a window set corresponding to the current screen; receiving a screen projection control instruction triggered by user operation; determining a window identifier selected by a screen projection control instruction; and positioning a target window in a window set corresponding to the current screen according to the determined window identification.

In one embodiment, the overlay module 1204 is further configured to determine a stacking order of the windows in the window set in the current screen; and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

In an embodiment, the superimposing module 1204 is further configured to sequentially superimpose and draw the screen data corresponding to the windows in the window set, other than the target window, onto the canvas according to the corresponding window positions according to the stacking order, so as to obtain intermediate image data; and coding the intermediate image data to obtain screen projection data adaptive to the current network environment.

In one embodiment, the overlay module 1204 is further configured to initialize a canvas; traversing each window in the window set according to the stacking sequence; when the traversed window is not the target window, overlaying and drawing the screen data corresponding to the traversed window on the canvas according to the corresponding window position, and continuing to traverse; and when the traversed window is the target window, skipping the target window and continuing the traversal.

In one embodiment, the stacking module 1204 is further configured to determine a stacking order of the windows in the window set other than the target window in the current screen when the number of the windows in the window set other than the target window exceeds one; and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

In one embodiment, the positioning module 1201 is further configured to determine a window set corresponding to the current screen according to a frequency greater than or equal to a preset frequency; and after determining the window set corresponding to the current screen each time, re-executing the step of calling the operating system interface and acquiring the window positions corresponding to the windows in the window set except the target window.

In an embodiment, when the target window is a window that needs to be subjected to screen projection, the overlay module 1204 is further configured to obtain screen projection data directly according to the screen data of the target window.

As shown in fig. 13, in one embodiment, the screen projection control apparatus 1200 further includes: a connection module 1205.

A connection module 1205 used for acquiring the screen projection code and the local terminal identifier; sending the screen projection code and the terminal identification to a server; and the transmitted screen projection code and the terminal identification are used for indicating the server to establish wireless screen projection connection between the terminal corresponding to the terminal identification and the screen projection equipment corresponding to the screen projection equipment identification after finding the screen projection equipment identification associated with the screen projection code.

The sending module 1204 is further configured to upload the screen projection data to a server; and the uploaded screen projection data are used for instructing the server to forward the screen projection data to the screen projection equipment according to the wireless screen projection connection.

FIG. 14 is a diagram illustrating an internal structure of a computer device in one embodiment. The computer device may specifically be the screen projection terminal 110 in fig. 1. As shown in fig. 14, the computer apparatus includes a processor, a memory, a network interface, an input device, and a display screen connected through a system bus. Wherein the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program that, when executed by the processor, causes the processor to implement a screen projection control method. The internal memory may also have a computer program stored therein, which when executed by the processor, causes the processor to perform the screen projection control method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

Those skilled in the art will appreciate that the architecture shown in fig. 14 is merely a block diagram of some of the structures associated with the disclosed aspects and is not intended to limit the computing devices to which the disclosed aspects apply, as particular computing devices may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In one embodiment, the screen projection control apparatus provided in the present application may be implemented in the form of a computer program that is executable on a computer device such as the one shown in fig. 14. The memory of the computer device may store various program modules constituting the screen projection control apparatus, such as a positioning module 1201, a calling module 1202, a superimposing module 1203, and a sending module 1204 shown in fig. 12. The computer program constituted by the respective program modules causes the processor to execute the steps in the screen projection control method of the respective embodiments of the present application described in the present specification.

For example, the computer device shown in fig. 14 may locate the target window in the window set corresponding to the current screen through the locating module 1201 in the screen-projection control apparatus 1200 shown in fig. 12. The calling module 1202 calls an operating system interface to obtain window positions corresponding to windows in the window set except for the target window. And superposing the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions by the superposition module 1203 to obtain the screen projection data. The screen projection data is sent by the sending module 1204, and the sent screen projection data is used for screen projection display.

In one embodiment, a computer device is provided, comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the above-described screen projection control method. Here, the steps of the screen projection control method may be steps in the screen projection control methods of the above-described respective embodiments.

In one embodiment, a computer-readable storage medium is provided, in which a computer program is stored, which, when executed by a processor, causes the processor to perform the steps of the above-described screen projection control method. Here, the steps of the screen projection control method may be steps in the screen projection control methods of the above-described respective embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a non-volatile computer-readable storage medium, and can include the processes of the embodiments of the methods described above when the program is executed. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present application. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A screen projection control method comprises the following steps:

positioning a target window in a window set corresponding to a current screen;

2. The method of claim 1, wherein positioning the target window in the window set corresponding to the current screen comprises:

displaying window identifications of all windows in a window set corresponding to a current screen through a screen projection control interface;

receiving a screen projection control instruction triggered by user operation;

determining a window identifier selected by the screen projection control instruction;

and positioning a target window in a window set corresponding to the current screen according to the determined window identification.

3. The method according to claim 1, wherein the superimposing the screen data corresponding to the windows in the window set except the target window according to the corresponding window positions to obtain the screen projection data comprises:

determining the stacking sequence of each window in the window set in the current screen;

and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

4. The method according to claim 3, wherein the sequentially superimposing, according to the stacking order, the screen data corresponding to the windows in the window set except the target window according to the corresponding window positions to obtain the screen projection data comprises:

according to the stacking sequence, sequentially overlaying and drawing screen data corresponding to windows except the target window in the window set on a canvas according to corresponding window positions to obtain intermediate image data;

and coding the intermediate image data to obtain screen projection data adaptive to the current network environment.

5. The method according to claim 4, wherein the sequentially overlaying and drawing the screen data corresponding to the windows in the window set except the target window on the canvas according to the corresponding window positions according to the stacking order to obtain the intermediate image data comprises:

initializing a canvas;

traversing each window in the window set according to the stacking sequence;

when the traversed window is not the target window, overlaying and drawing the screen data corresponding to the traversed window on the canvas according to the corresponding window position, and continuing to traverse;

and when the traversed window is the target window, skipping the target window and continuing the traversal.

6. The method according to claim 1, wherein the superimposing the screen data corresponding to the windows in the window set except the target window according to the corresponding window positions to obtain the screen projection data comprises:

when the number of windows except the target window in the window set exceeds one, then

Determining the stacking sequence of windows in the window set except the target window in the current screen;

7. The method of claim 1, further comprising:

determining a window set corresponding to the current screen according to the frequency greater than or equal to the preset frequency;

and after determining the window set corresponding to the current screen each time, re-executing the calling operating system interface, and acquiring the window position corresponding to each window except the target window in the window set.

8. The method of claim 1, further comprising:

when the target window is a window needing to be projected, then

Screen projection data are directly obtained according to the screen data of the target window;

when the target window is a window which does not need to be projected, then

And continuing to execute the calling operating system interface, and acquiring the window positions corresponding to the windows in the window set except the target window.

9. The method according to any one of claims 1-8, further comprising:

acquiring a screen projection code and a local terminal identifier;

sending the screen projection code and the terminal identification to a server; the screen projection code and the terminal identification are sent and used for indicating the server to establish wireless screen projection connection between the terminal corresponding to the terminal identification and the screen projection equipment corresponding to the screen projection equipment identification after the screen projection equipment identification associated with the screen projection code is found out;

the sending the screen projection data comprises:

uploading the screen projection data to the server; and the uploaded screen projection data are used for indicating the server to forward the screen projection data to the screen projection equipment according to the wireless screen projection connection.

10. A screen projection control device, comprising:

11. The apparatus according to claim 10, wherein the positioning module is further configured to display, through the screen-casting control interface, a window identifier of each window in the window set corresponding to the current screen; receiving a screen projection control instruction triggered by user operation; determining a window identifier selected by the screen projection control instruction; and positioning a target window in a window set corresponding to the current screen according to the determined window identification.

12. The apparatus of claim 10, wherein the overlay module is further configured to determine a stacking order of the windows in the window set in the current screen; and according to the stacking sequence, sequentially overlapping the screen data corresponding to the windows except the target window in the window set according to the corresponding window positions to obtain screen projection data.

13. The device of claim 10, wherein the positioning module is further configured to determine a window set corresponding to the current screen according to a frequency greater than or equal to a preset frequency; and after determining the window set corresponding to the current screen each time, re-executing the calling operating system interface, and acquiring the window position corresponding to each window except the target window in the window set.

14. A computer-readable storage medium, storing a computer program which, when executed by a processor, causes the processor to carry out the steps of the method according to any one of claims 1 to 9.

15. A computer device comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the method according to any one of claims 1 to 9.