CN117950610A - Screen transmission control method, electronic equipment and computer readable storage medium - Google Patents
Screen transmission control method, electronic equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN117950610A CN117950610A CN202211292727.2A CN202211292727A CN117950610A CN 117950610 A CN117950610 A CN 117950610A CN 202211292727 A CN202211292727 A CN 202211292727A CN 117950610 A CN117950610 A CN 117950610A
- Authority
- CN
- China
- Prior art keywords
- screen
- window
- control
- receiving end
- display data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000005540 biological transmission Effects 0.000 title claims abstract description 122
- 238000000034 method Methods 0.000 title claims abstract description 77
- 230000004044 response Effects 0.000 claims description 15
- 238000001914 filtration Methods 0.000 claims description 8
- 238000010586 diagram Methods 0.000 description 10
- 230000003993 interaction Effects 0.000 description 9
- 238000009877 rendering Methods 0.000 description 6
- 238000004088 simulation Methods 0.000 description 6
- 241000209140 Triticum Species 0.000 description 5
- 235000021307 Triticum Nutrition 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000004891 communication Methods 0.000 description 3
- 230000003111 delayed effect Effects 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 230000002860 competitive effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application provides a screen transfer control method, electronic equipment and a computer readable storage medium, wherein the screen transfer control method comprises the following steps: transmitting first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end. By the mode, opposite end control can be performed in the screen transmission process.
Description
Technical Field
The application relates to the technical field of display, in particular to a screen transmission control method, electronic equipment and a computer readable storage medium.
Background
With the advent and diversification of scenes such as business meetings, home entertainment, competitive games, etc., screen transfer (screen throwing) as a display technology for displaying contents on opposite ends has begun to be widely used in the above scenes. The method is characterized in that screen content of a screen transmitting end is collected and transmitted to a screen transmitting receiving end to synchronously display the content by utilizing a wireless or wired transmission technology.
The screen transmission mode has the limitation that only the synchronization of display pictures can be carried out, and more functional interaction between two devices is lacked.
Disclosure of Invention
The embodiment of the application provides a screen transmission control method, electronic equipment and a computer readable storage medium, which are used for controlling opposite-end equipment by using one piece of equipment in a screen transmission process, so that the opposite-end control function in the screen transmission process is increased.
The application provides a screen transmission control method which is applied to a screen transmission sending end and comprises the following steps: transmitting first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
The application also provides a screen transmission control method which is applied to the screen transmission receiving end and comprises the following steps: acquiring first display data sent by a screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end; displaying on a screen-throwing window based on the first display data; transmitting second display data corresponding to the operation window to the screen transmitting end so that the screen transmitting end displays the second display data on a control window of the screen transmitting end; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
The application also provides an electronic device comprising a processor and a memory, the memory being for storing program data, the processor being for executing the program data to implement a method as described above.
The present application also provides a computer readable storage medium having stored therein program data for carrying out the method as described above when executed by a processor.
The screen transmission control method provided by the embodiment of the application comprises the following steps: the screen transmitting end transmits first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays the first display data on the screen throwing window; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end. Through the mode, the content of the operation window of the screen transmitting and receiving end can be displayed on the display interface of the screen transmitting and receiving end, so that the screen transmitting and receiving end and the screen transmitting and receiving end can both control the screen transmitting and receiving end, the screen transmitting and receiving end can be conveniently controlled in the screen transmitting process, the screen transmitting and receiving end is facilitated to be reflected and played, and interaction of users at two ends is also facilitated.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a first embodiment of a screen transfer control method provided by the application;
FIG. 2 is a flow chart of step 11 according to an embodiment of the present application;
FIG. 3 is a schematic flow chart of a second embodiment of a screen transfer control method provided by the present application;
FIG. 4 is a flow chart of step 23 in an embodiment of the application;
FIG. 5 is a first interactive schematic diagram of a screen-passing control method provided by the application;
FIG. 6 is a schematic flow chart of a third embodiment of a screen transfer control method provided by the present application;
fig. 7 is a schematic flow chart of a fourth embodiment of a screen transmission control method provided by the application;
FIG. 8 is a second interactive schematic diagram of the screen-transfer control method provided by the present application;
fig. 9 is a schematic flow chart of a fifth embodiment of a screen-passing control method provided by the application;
fig. 10 is a schematic flow chart of a sixth embodiment of a screen transmission control method provided by the present application;
FIG. 11 is a third interactive schematic diagram of a screen-transfer control method provided by the present application;
FIG. 12 is a schematic diagram of an embodiment of an electronic device according to the present application;
Fig. 13 is a schematic structural diagram of an embodiment of a computer readable storage medium according to the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
In the existing wireless screen transmission scene, if the content on the conference flat plate needs to be controlled, for example, video conference software is opened, when the content shared on the conference flat plate by a computer is shared through the video conference software, a user needs to walk from the computer end to the conference flat plate end, and the user is required to operate on the conference flat plate, so that the experience of the user is poor.
Based on the above, the embodiment of the application provides a screen transfer control method, which is mainly applied to a screen transfer transmitting end to a screen transfer receiving end, wherein the screen transfer transmitting end can be a mobile phone, a computer and the like, and the screen transfer receiving end can be a conference flat panel or a television lamp. The following specifically describes an example in which the screen transmitting end is a computer and the screen receiving end is a conference panel.
Referring to fig. 1, fig. 1 is a flowchart of a first embodiment of a screen transmission control method provided by the present application, where an execution body of the method is a screen transmission transmitting end, such as a computer, and the method includes:
Step 11: and sending the first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data.
The display interface of the computer can comprise a display window and a control window.
The display window is used for displaying contents requiring screen transfer, such as video frames, picture frames, PPT (PowerPoint) document frames, live frames, etc. In a specific embodiment, the display window may be a window of an application program, such as a video playing software, conference software, live broadcasting software, or document playing software (word or PDF, etc.).
The control window is used for displaying a control interface to acquire an input event of a user by using the control interface, so as to control a transmission receiving end such as a conference flat plate.
Step 12: receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end.
The display interface of the conference flat panel can comprise a screen transmission window and an operation window.
The screen transmission window is used for displaying first display data sent by the computer, namely, the content in the display window of the computer can be synchronously transmitted to the screen transmission window of the conference flat plate for display, and understandably, the display content of the screen transmission window is possibly delayed relative to the display content of the display window based on delay of data transmission.
The operation window is used for displaying an operation interface so as to acquire an input instruction of a user by utilizing the operation interface, thereby controlling the conference flat plate.
Step 13: displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
At the screen transmitting end, the user can control the screen transmitting receiving end through the control window, and at the screen transmitting receiving end, the user can control the screen transmitting receiving end through the operation window, namely, the screen transmitting end transmits the display picture to the screen transmitting receiving end, and the screen transmitting end and the users at the two ends of the screen transmitting receiving end can both perform operation control on the screen transmitting receiving end, so that the screen transmitting process has better functional experience.
The following is an example of application scenarios of a computer and a conference tablet: in the scene of a remote video conference, a computer used by a host transmits course content to a conference panel, the conference panel comprises a screen throwing window and an operation window, the screen throwing window displays the course content, the operation window displays control buttons, such as a volume adjusting button, a brightness adjusting button, an open/close button and the like, simultaneously, the display content of the operation window is transmitted to the computer as second display data, and is displayed in the control window of the computer, the host can control the electronic equipment B through the control window, for example, the host finds that a microphone of the conference panel is not opened, and can actively adjust to start the microphone of the conference panel, so that the control operation on the conference panel is not needed before the host walks to the conference panel under the condition that the microphone is not opened and voice communication is not possible.
It can be understood that the control window of the computer may include not only the display content corresponding to the second display data, but also a control button for the computer. For example, the control window may include two parts, a first control window for receiving an input instruction to control the computer, and a second control window for acquiring an input event to control the conference board.
Optionally, in an embodiment, as shown in fig. 2, fig. 2 is a schematic flow chart of step 11 in an embodiment of the present application, and step 11 may specifically include:
step 111: and acquiring full-screen display data of the screen transmission transmitting end.
The screen data can be acquired by utilizing an acquisition tool of a computer system, and an installed hardware tool or software tool can also be adopted.
Taking Windows10 as an example, WGC (Windows Graphics Capture) technology can be adopted to collect the screen picture, and the collected data is generally YUV data or RGB data. Of course, in other embodiments, screen capture techniques such as GDI, DXGI, magnification may be utilized, which are not examples.
Step 112: and filtering the second display data from the full-screen display data of the screen transmission end to obtain the first display data corresponding to the display window.
The second display data is the data displayed in the control window, so that the display data of the control window is filtered out by using the full-screen display data in the step, so that the display data of the display window, namely the first display data, is left.
Taking Windows10 as an example, window filtering can be performed by utilizing Magnification and WGC for screen data acquisition, and SetWindowDisplayAffinity can be used for window filtering if WGC is used for screen data acquisition.
Step 113: and sending the first display data to the screen transmitting receiving end.
If the wired mode is adopted, the RGB or YUV data is generally transmitted to the conference flat plate through HDMI (High Definition Multimedia Interface, high-definition multimedia) or DP (Decentralized Periphery) protocol (a Profibus communication protocol running on 485 serial port); if the wireless mode is adopted, an h.264 encoder (such as libx and 264) can be adopted to compress and encode the RGB or YUV data, and then the H.264 data is transmitted to the conference flat plate through the IEEE 802.11 protocol matched with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After the conference flat receives the data, it determines whether the terminal data is encoded and compressed, if so, it needs to decode the display data by a corresponding decoder (e.g. the decoder libx of h.264). And then rendering the RGB or YUV data, if the conference board is a Windows operating system, the OpenGL (Open Graphics Library ) may be used to render the display data, and if the conference board is a macOS operating system, the Metal (rendering interface) may be used to render the display data.
Referring to fig. 3 for a screen transmitting receiving end, fig. 3 is a flow chart of a second embodiment of a screen transmitting control method provided by the present application, where an execution body of the method is the screen transmitting receiving end, and a conference flat board is taken as an example, and the method includes:
Step 31: and acquiring first display data sent by the screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end.
The display interface of the conference flat panel may include a display window and a control window.
The display window is used for displaying contents requiring screen transfer, such as video frames, picture frames, PPT (PowerPoint) document frames, live frames, etc. In a specific embodiment, the display window may be a window of an application program, such as a video playing software, conference software, live broadcasting software, or document playing software (word or PDF, etc.).
The control window is used for displaying a control interface so as to acquire an input event of a user by utilizing the control interface, thereby controlling the conference flat plate.
Step 32: and displaying on the screen throwing window based on the first display data.
The display interface of the conference flat panel can comprise a screen transmission window and an operation window.
The screen transmitting window is used for displaying first display data transmitted by a screen transmitting end, such as a computer, namely, the content in the display window of the computer can be synchronously transmitted to the screen transmitting window of the conference flat plate for display, and understandably, the display content of the screen transmitting window is possibly delayed relative to the display content of the display window based on delay of data transmission.
The operation window is used for displaying an operation interface so as to acquire an input instruction of a user by utilizing the operation interface, thereby controlling the transmission screen receiving end.
Step 33: transmitting second display data corresponding to the operation window to the screen transmitting end so that the screen transmitting end displays the second display data on a control window of the screen transmitting end; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
At the computer end, a user can control the conference flat plate through the control window, at the conference flat plate end, the user can control the conference flat plate through the operation window, namely, the computer transmits a display screen to the conference flat plate, and the users at the computer end and the conference flat plate end can both perform operation control on the conference flat plate, so that the screen transmission process has better functional experience.
It can be understood that the control window of the computer may include not only the display content corresponding to the second display data, but also a control button for the computer. For example, the control window may include two parts, a first control window for receiving an input instruction to control the computer, and a second control window for acquiring an input event to control the conference board.
Optionally, in an embodiment, as shown in fig. 4, fig. 4 is a schematic flow chart of step 33 in an embodiment of the present application, and the step 33 may specifically include:
Step 331: and acquiring full-screen display data of the screen transmitting receiving end.
The screen data can be acquired by utilizing an acquisition tool of a system of the conference flat board, and an installed hardware tool or software tool can also be adopted.
Taking Windows10 as an example, WGC (Windows Graphics Capture) technology can be adopted to collect the screen picture, and the collected data is generally YUV data or RGB data. Of course, in other embodiments, screen capture techniques such as GDI, DXGI, magnification may be utilized, which are not examples.
Step 332: and filtering the first display data of the full screen display data of the screen transmitting receiving end to obtain second display data corresponding to the operation window.
The first display data is data displayed on the operation window, so that the display data of the operation window is actually filtered by using the full-screen display data, so that the display data of the screen transmission window, namely the first display data, is left.
Taking Windows10 as an example, window filtering can be performed by utilizing Magnification and WGC for screen data acquisition, and SetWindowDisplayAffinity can be used for window filtering if WGC is used for screen data acquisition.
Step 333: and sending the second display data to the screen transmission end.
If the wired mode is adopted, the RGB or YUV data is generally transmitted to a computer through HDMI (High Definition Multimedia Interface, high-definition multimedia) or DP (Decentralized Periphery) protocol (a Profibus communication protocol running on 485 serial port); if the wireless mode is adopted, an h.264 encoder (such as libx and 264) can be adopted to compress and encode the RGB or YUV data, and then the H.264 data is transmitted to a computer through the IEEE 802.11 protocol matched with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After the computer receives the data, whether the terminal data is subjected to encoding compression is judged, and if so, the display data is required to be decoded by a corresponding decoder (for example, a decoder libx264 of h.264). And then rendering the RGB or YUV data, if the computer is a Windows operating system, the OpenGL (Open Graphics Library ) can be used for rendering the display data, and if the computer is a macOS operating system, the Metal (rendering interface) can be used for rendering the display data.
Referring to fig. 5 in combination with fig. 1 and fig. 3, fig. 5 is a first interaction schematic diagram of the screen transmission control method provided by the present application, and a screen transmission interaction process of a screen transmission end (for example, a computer) and a screen transmission receiving end (for example, a conference flat board) is described in the present application:
① The screen transmitting end collects full screen display data;
② The screen transmitting end filters the acquired full-screen display data to display data of the control window to obtain first display data;
③ The screen transmitting end transmits first display data to the screen receiving end;
④ The screen transmitting receiving end displays display content corresponding to the first display data in a screen transmitting window;
⑤ The screen transmitting receiving end collects full screen display data;
⑥ The screen transmitting receiving end filters the acquired full-screen display data to display data of the operation window to obtain second display data;
⑦ The screen transmitting receiving end transmits second display data to the screen transmitting end;
⑧ And the screen transmitting end displays display content corresponding to the second display data in the control window.
It should be noted that, the above-mentioned steps ①-④ and ⑤-⑧ can be regarded as two independent processes, and the two processes can be performed simultaneously, for example, the display data of the control window filtered in the step ② is the content corresponding to the second display data displayed in the control window by the computer in the step ⑧.
Unlike the prior art, the screen transfer control method provided by the embodiment includes: the screen transmitting end transmits first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays the first display data on the screen throwing window; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end. Through the mode, the content of the operation window of the screen transmitting and receiving end can be displayed on the display interface of the screen transmitting and receiving end, so that the screen transmitting and receiving end and the screen transmitting and receiving end can both control the screen transmitting and receiving end, the screen transmitting and receiving end can be conveniently controlled in the screen transmitting process, the screen transmitting and receiving end is facilitated to be reflected and played, and interaction of users at two ends is also facilitated.
Referring to fig. 6, fig. 6 is a schematic flow chart of a third embodiment of a screen transmission control method provided by the present application, where the method is applied to a screen transmission end, and the method includes:
Step 61: and sending the first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data.
Step 62: receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end.
Step 63: and displaying the second display data on a control window of the screen transmission end.
Steps 61-63 are similar to the embodiments described above and will not be described again.
Step 64: based on the input event of the control window, corresponding control data is generated.
Step 65: and sending control data to the screen transmitting receiving end so that the screen transmitting receiving end generates a corresponding control instruction based on the control data simulation input event to control an operation window of the screen transmitting receiving end.
The input event may include a conventional input manner such as a mouse, a keyboard, a touch screen, etc., and may also include an interactive input such as a voice input, a gesture input, etc. In one embodiment, the input event may be captured using an interface of an operating system, for example, a Windows operating system in which an API interface for capturing the input event is configured.
It will be appreciated that the data captured by the mouse, keyboard, etc. are numeric or ascii characters, and can be simply converted into binary data. The binary data is then transmitted to the conference board. After receiving the corresponding data, the conference panel can be simulated into a control instruction of a corresponding input event, so that the conference panel is controlled by a computer.
When the event simulation of the conference flat is realized, the detail to be noted is provided, and because the conference flat picture rendered on the computer is actually the screen throwing content of the computer is filtered, the factor is also needed to be considered when the event simulation is carried out on the conference flat, namely the transparent transmission of the input event is realized.
Specifically, after the computer sends the control data to the conference flat board, the conference flat board generates a corresponding control instruction based on the control data simulation input event. When an input event is simulated, the screen throwing window should be filtered, that is, all control data is transmitted to the operation window.
In the control process, taking a mouse as an example, when the system captures the input operation of the mouse, the position information of the mouse cursor is obtained, corresponding control data is determined according to the position information, and the control data is sent to the conference flat plate. After the conference panel receives the control data, an input event of the mouse is simulated in the operation window according to the position information, so that corresponding clicking operation is completed, and the control of the conference panel is realized.
Referring to fig. 7, fig. 7 is a flowchart of a fourth embodiment of a screen transmission control method provided by the present application, where the method is applied to a screen transmission receiving end, and the method includes:
Step 71: and acquiring first display data sent by the screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end.
Step 72: and displaying on the screen throwing window based on the first display data.
Step 73: and sending second display data corresponding to the operation window to the screen transmission end so that the screen transmission end can display the second display data on a control window of the screen transmission end.
Step 74: control data sent by the screen transmission end is obtained, and the control data is generated by the screen transmission end based on an input event of a control window.
Step 75: and generating a corresponding control instruction based on the control data simulation input event so as to control an operation window of the transmission screen receiving end.
This embodiment is similar to the above-described embodiment and will not be described in detail here.
In an alternative embodiment, step 75 may specifically include: simulating an input event of the operating window based on the control data; generating a corresponding control instruction according to the input event; and controlling an operation window of the transmission screen receiving end according to the control instruction.
It can be understood that, because the display screen of the screen transmitting receiving end includes a screen throwing window and an operation window, the screen throwing window does not need to be operated, when the input event is simulated based on the control data, the screen throwing window should be filtered out, so that the transparent transmission from all the input events to the operation window is realized, that is, all the control instructions should be corresponding to the operation window, thereby realizing the operation of the operation window and further controlling the screen transmitting receiving end.
Referring to fig. 8 in combination with fig. 6 and fig. 7, fig. 8 is a second interaction schematic diagram of the screen transmission control method provided by the present application, and a control interaction process between a screen transmission end and a screen transmission receiving end is described:
① The screen transmitting end collects input events;
② The screen transmitting end filters the input events, and the remaining input events are filtered to be real input events corresponding to the control window;
③ The method comprises the steps that a first real input event based on a control window determines corresponding position information and forms control data;
④ The screen transmitting end transmits control data to the screen receiving end;
⑤ The screen transmitting receiving end generates a corresponding control instruction based on the control data simulation input event, and the control instruction is utilized to control the screen transmitting receiving end.
For example, in a scenario where a teacher performs a remote video teaching on a student, an electronic device a used by the teacher transfers course content to an electronic device B of the student, where the electronic device B includes a screen-throwing window and an operation window, the screen-throwing window displays the course content, the operation window displays a control button, for example, an open/close button, and at the same time, the display content of the operation window is transmitted as second display data to the electronic device a and displayed in a control window of the electronic device a, and the teacher can control the electronic device B through the control window. For example, a teacher finds that the student does not open wheat, the teacher clicks an open wheat/close wheat button in a control window through a mouse, the system captures clicking operation of the mouse and determines clicking position of the mouse to form control data and sends the control data to the electronic equipment B, and after the electronic equipment B receives the control data, the clicking operation of the corresponding open wheat/close wheat button is simulated in the operation window according to the clicking operation and clicking position of the mouse, so that the control of a transmission receiving end by a transmission transmitting end is completed.
Referring to fig. 9, fig. 9 is a flowchart of a fifth embodiment of a screen transmission control method provided by the present application, where the method is applied to a screen transmission transmitting end, and the method includes:
step 91: and sending the first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data.
Step 92: receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end.
Step 93: displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
Step 94: and responding to triggering file copying operation, and acquiring copied file data.
Step 95: responding to the acquired file pulling instruction sent by the screen transmitting receiving end, and sending the acquired file data to the screen transmitting receiving end; the file pulling instruction is generated by the screen transmitting receiving end in response to the triggering file pasting operation.
The conference flat plate responds to the file pasting operation and pulls the target file data with the amplitude to the computer so as to realize the cross-equipment copying and pasting of the target file data.
Alternatively, in an embodiment, step 94 may be specifically: responding to the triggering file copying operation, reading file data in a shear plate of a screen transmitting end, and transmitting identification information to a screen transmitting receiving end; the identification information is used for enabling the screen transmitting receiving end to write temporary data in the shear plate of the screen transmitting receiving end, and sending a file pulling instruction to the screen transmitting end in response to the condition that the shear plate data of the screen transmitting receiving end is empty.
The method comprises the steps that a computer responds to triggering file copying operation, reads file data in a shear plate of a conference flat plate, and sends identification information to the conference flat plate; after receiving the first identification information, the conference flat plate writes temporary data in a shear plate of the conference flat plate, and when the conference flat plate responds to the condition that the shear plate data of the conference flat plate is empty, the conference flat plate is considered to trigger pasting operation, and a file pulling instruction is sent to a computer; after the computer obtains the file pulling instruction, the copied file data is sent to the conference flat plate, and cross-equipment copying and pasting (file transmission) of the file data is completed.
Referring to fig. 10 for a screen transmitting receiving end, fig. 10 is a flowchart of a sixth embodiment of a screen transmitting control method provided by the present application, where the method is applied to the screen transmitting receiving end, and the method includes:
Step 101: and acquiring first display data sent by the screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end.
Step 102: and displaying on the screen throwing window based on the first display data.
Step 103: transmitting second display data corresponding to the operation window to the screen transmitting end so that the screen transmitting end displays the second display data on a control window of the screen transmitting end; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
Step 104: and responding to the triggering file pasting operation, and sending a file pulling instruction to a screen transmitting end.
Step 105: acquiring file data sent by a screen transmitting end in response to a pulling instruction; the file data is acquired by the screen transmitting end in response to triggering file copying operation.
Optionally, in an embodiment, step 104 may specifically include: in response to receiving the identification information sent by the screen transmitting end, writing temporary data in a shear plate of the screen transmitting receiving end; the identification information is generated by the screen transmitting end in response to the triggering file copying operation; and transmitting a file pulling instruction to the screen transmitting end in response to the condition that the data of the shear plate of the screen transmitting receiving end is empty.
Referring to fig. 11 in combination with fig. 9 and fig. 10, fig. 11 is a third interaction schematic diagram of the on-screen control method provided by the present application, and a file sharing interaction process between an on-screen sending end and an on-screen receiving end is described:
① The screen transmitting end triggers file copying operation;
② The transmitting end reads file data in a shear plate of the transmitting end;
③ The screen transmitting end transmits identification information to the screen transmitting receiving end;
④ The screen transmitting receiving end responds to the identification information, and temporary data are written in a shear plate of the screen transmitting receiving end;
⑤ The screen transmitting receiving end responds to the condition that the shear plate data of the screen transmitting receiving end are empty, and sends a file pulling instruction to the screen transmitting end;
⑥ The screen transmitting end responds to the file pulling instruction and transmits file data to the screen transmitting receiving end.
Referring to fig. 12, fig. 12 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device 120 includes a processor 121 and a memory 122, and the memory 122 is used for storing program data.
When the electronic device 120 is a screen transmitting end, the processor 121 is configured to execute the above program data to implement the following method: transmitting first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
When the electronic device 120 is a screen transmitting receiving end, the processor 121 is configured to execute the above program data to implement the following method: acquiring first display data sent by a screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end; displaying on a screen-throwing window based on the first display data; transmitting second display data corresponding to the operation window to the screen transmitting end so that the screen transmitting end displays the second display data on a control window of the screen transmitting end; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
In this embodiment, the electronic device 120 may be a PC (personal computer) computer, a notebook, a television, a mobile phone, a tablet computer, an intelligent wearable device, etc. For example, the screen projection scene can be a combination of a mobile phone and a television, the conference scene can be a combination of a user side (computer) and a conference side (computer), the scene can be a combination of a computer side and a projection device, and the like.
Referring to fig. 13, fig. 13 is a schematic structural diagram of an embodiment of a computer readable storage medium provided in the present application, the computer readable storage medium 130 stores therein program data 131, and the program data 131 is executed by a processor to perform:
Transmitting first display data corresponding to the display window to the screen transmitting receiving end so that the screen transmitting receiving end displays on the screen throwing window based on the first display data; receiving second display data sent by a screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end; displaying on a control window of the screen transmission transmitting end based on the second display data; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end. Or acquiring first display data sent by the screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end; displaying on a screen-throwing window based on the first display data; transmitting second display data corresponding to the operation window to the screen transmitting end so that the screen transmitting end displays the second display data on a control window of the screen transmitting end; wherein the control window is configured to respond to an input event based on the control window to control operation of the transmission receiving end.
The display screen provided by the embodiment of the present application has been described in detail, and specific examples are applied to illustrate the principles and embodiments of the present application, and the description of the above embodiments is only used to help understand the method and core idea of the present application; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.
Claims (13)
1. The screen transmission control method is applied to a screen transmission sending end and is characterized by comprising the following steps of:
transmitting first display data corresponding to a display window to a screen transmitting receiving end so that the screen transmitting receiving end displays on a screen throwing window based on the first display data;
Receiving second display data sent by the screen transmitting receiving end; the second display data corresponds to an operation window of the screen transmitting receiving end;
displaying on a control window of the screen transmission end based on the second display data; the control window is configured to respond to an input event based on the control window so as to control the screen transmitting receiving end.
2. The screen transfer control method according to claim 1, wherein,
The step of sending the first display data corresponding to the display window to the screen transmitting receiving end comprises the following steps:
Acquiring full screen display data of the screen transmission end;
Filtering the second display data by the full-screen display data of the screen transmission end to obtain first display data corresponding to a display window;
And sending the first display data to the screen transmitting receiving end.
3. The screen transfer control method according to claim 1, wherein,
The method further comprises the steps of:
generating corresponding control data based on the input event of the control window;
And sending the control data to the screen transmitting receiving end so that the screen transmitting receiving end simulates the input event based on the control data to generate a corresponding control instruction so as to control the operation window of the screen transmitting receiving end.
4. The screen transfer control method according to claim 1, wherein,
The method further comprises the steps of:
Responding to triggering file copying operation to obtain copied file data;
responding to a file pulling instruction sent by a screen transmitting receiving end, and sending the obtained file data to the screen transmitting receiving end;
The file pulling instruction is generated by the screen transmitting receiving end in response to triggering file pasting operation.
5. The screen transfer control method according to claim 4, wherein,
The response to triggering the file copying operation, obtaining copied file data, and sending an identification message to the screen transmitting receiving end, including:
Responding to triggering file copying operation, reading file data in a shear plate of the screen transmitting end, and transmitting identification information to the screen transmitting receiving end;
The identification information is used for enabling the screen transmitting receiving end to write temporary data in the shear plate of the screen transmitting receiving end, and sending the file pulling instruction to the screen transmitting end in response to the condition that the shear plate data of the screen transmitting receiving end is empty.
6. The screen transmission control method is applied to a screen transmission receiving end and is characterized by comprising the following steps of:
Acquiring first display data sent by a screen transmission end, wherein the first display data corresponds to a display window of the screen transmission end;
Displaying on a screen throwing window based on the first display data;
Transmitting second display data corresponding to an operation window to the screen transmission end so that the screen transmission end displays on a control window of the screen transmission end based on the second display data; the control window is configured to respond to an input event based on the control window so as to control the screen transmitting receiving end.
7. The screen transfer control method according to claim 6, wherein,
The step of sending the second display data corresponding to the operation window to the screen transmission end comprises the following steps:
Acquiring full screen display data of the screen transmitting receiving end;
Filtering the first display data by the full screen display data of the screen transmitting receiving end to obtain second display data corresponding to an operation window;
and sending the second display data to the screen transmission end.
8. The screen transfer control method according to claim 6, wherein,
The method further comprises the steps of:
The control data sent by the screen transmission end is obtained, and the control data is generated by the screen transmission end based on the input event of the control window;
And simulating the input event based on the control data to generate a corresponding control instruction so as to control the operation window of the screen transmission receiving end.
9. The screen transfer control method according to claim 8, wherein,
The step of simulating the input event based on the control data to generate a corresponding control instruction so as to control the operation window of the screen transmission receiving end comprises the following steps:
simulating an input event for the operating window based on the control data;
generating a corresponding control instruction according to the input event;
And controlling the operation window of the screen transmission receiving end according to the control instruction.
10. The screen transfer control method according to claim 6, wherein,
The method further comprises the steps of:
Responding to triggering file pasting operation, and sending a file pulling instruction to the screen transmission sending end;
acquiring file data sent by the screen transmitting end in response to the pulling instruction; the file data is acquired by the screen transmitting end in response to triggering file copying operation.
11. The screen transfer control method according to claim 10, wherein,
The response to triggering the file pasting operation, sending a file pulling instruction to the screen transmission sending end, includes:
writing temporary data in a shear plate of the screen transmission receiving end in response to receiving the identification information sent by the screen transmission sending end; the identification information is generated by the screen transmission transmitting end in response to the triggering file copying operation;
And responding to the data of the shear plate of the screen transmission receiving end is empty, and sending the file pulling instruction to the screen transmission sending end.
12. An electronic device comprising a processor and a memory, the memory for storing program data, the processor for executing the program data to implement the method of any of claims 1-11.
13. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein program data for implementing the method according to any of claims 1-11 when being executed by a processor.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211292727.2A CN117950610A (en) | 2022-10-21 | 2022-10-21 | Screen transmission control method, electronic equipment and computer readable storage medium |
| PCT/CN2023/114032 WO2024082803A1 (en) | 2022-10-21 | 2023-08-21 | Screen-mirroring control method and apparatus, electronic device, and computer-readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211292727.2A CN117950610A (en) | 2022-10-21 | 2022-10-21 | Screen transmission control method, electronic equipment and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117950610A true CN117950610A (en) | 2024-04-30 |
Family
ID=90736850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211292727.2A Pending CN117950610A (en) | 2022-10-21 | 2022-10-21 | Screen transmission control method, electronic equipment and computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN117950610A (en) |
| WO (1) | WO2024082803A1 (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101195520B1 (en) * | 2012-02-22 | 2012-10-29 | 알서포트 주식회사 | Remote control method and remote control apparatus for controlling a mobile terminal |
| CN111625371A (en) * | 2019-12-03 | 2020-09-04 | 蘑菇车联信息科技有限公司 | Text transmission method, device, PC (personal computer) terminal, mobile terminal and system |
| CN112162716A (en) * | 2020-09-28 | 2021-01-01 | 北京小米移动软件有限公司 | File transmission method, device, terminal and storage medium |
| CN114979730A (en) * | 2021-02-25 | 2022-08-30 | 广州视源电子科技股份有限公司 | Screen transmission interaction method and device, computer equipment and storage medium |
| CN113515244B (en) * | 2021-03-24 | 2024-03-22 | 深圳乐播科技有限公司 | Terminal remote control method, device, equipment and storage medium based on screen throwing |
| CN114040242B (en) * | 2021-09-30 | 2023-07-07 | 荣耀终端有限公司 | Screen projection method, electronic equipment and storage medium |
-
2022
- 2022-10-21 CN CN202211292727.2A patent/CN117950610A/en active Pending
-
2023
- 2023-08-21 WO PCT/CN2023/114032 patent/WO2024082803A1/en unknown
Also Published As
| Publication number | Publication date |
|---|---|
| WO2024082803A1 (en) | 2024-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109510990A (en) | Image processing method and device, computer readable storage medium, electronic equipment | |
| EP4376423A1 (en) | Virtual object interaction method and device, and storage medium and computer program product | |
| CN101673185B (en) | Image display touch control method and system thereof | |
| CN112799891B (en) | iOS device testing method, device, system, storage medium and computer device | |
| WO2013134936A1 (en) | Screen recording method, and screen recording control method and device | |
| CN112073770B (en) | Display device and video communication data processing method | |
| CN203552237U (en) | Interactive electronic demonstration system | |
| CN112911320A (en) | Live broadcast method and device, computer equipment and storage medium | |
| EP3103081B1 (en) | Collaborative group video production system | |
| TWI740208B (en) | Image transmission device, image display system with remote screen capture function, and remote screen capture method | |
| CN117950610A (en) | Screen transmission control method, electronic equipment and computer readable storage medium | |
| CN113316011B (en) | Control method, system, equipment and storage medium of electronic whiteboard system | |
| CN103294193A (en) | Multi-terminal interaction method, device and system | |
| JP4321751B2 (en) | Drawing processing apparatus, drawing processing method, drawing processing program, and electronic conference system including the same | |
| CN111356009B (en) | Audio data processing method and device, storage medium and terminal | |
| CN115334063A (en) | Question-answering type virtual human control method, system, equipment and storage medium | |
| CN114025147A (en) | Data transmission method and system for VR teaching, electronic equipment and storage medium | |
| CN106339193B (en) | Wireless handheld interaction method and handheld interactive multimedia system | |
| CN112214149A (en) | Intelligent interactive display device, screen capture method and system thereof, and electronic equipment | |
| JP6499582B2 (en) | SENDING COMPUTER, RECEIVING COMPUTER, METHOD EXECUTED BY THE SAME, AND COMPUTER PROGRAM | |
| JP6606251B2 (en) | SENDING COMPUTER, RECEIVING COMPUTER, METHOD EXECUTED BY THE SAME, AND COMPUTER PROGRAM | |
| CN113628312B (en) | Cloud rendering method and device based on Unity3D built-in rendering pipeline | |
| CN115103054B (en) | Information processing method, device, electronic equipment and medium | |
| WO2024120072A1 (en) | Device sharing method, electronic device, and computer-readable storage medium | |
| CN111694442B (en) | Method for reducing mouse delay of distributed KVM system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |