WO2024082803A1

WO2024082803A1 - Screen-mirroring control method and apparatus, electronic device, and computer-readable storage medium

Info

Publication number: WO2024082803A1
Application number: PCT/CN2023/114032
Authority: WO
Inventors: 冯君骅
Original assignee: 广州视臻信息科技有限公司
Priority date: 2022-10-21
Filing date: 2023-08-21
Publication date: 2024-04-25
Also published as: CN117950610A

Abstract

A screen-mirroring control method and apparatus, an electronic device (120), and a computer-readable storage medium (130). The screen-mirroring control method comprises: sending, to a screen-mirroring receiving end, first display data corresponding to a display window, such that the screen-mirroring receiving end performs display in a screen-mirroring window on the basis of the first display data (11); receiving second display data sent by the screen-mirroring receiving end, wherein the second display data corresponds to an operation window of the screen-mirroring receiving end (12); and performing display in a control window of a screen-mirroring sending end on the basis of the second display data, wherein the control window is configured to perform a control operation on the screen-mirroring receiving end in response to a control-window-based input event (13). Peer control can be performed during screen mirroring.

Description

Screen transmission control method, device, electronic device and computer readable storage medium

This application claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on October 21, 2022, with application number 202211292727.2 and invention name “A screen transmission control method, electronic device and computer-readable storage medium”, the entire contents of which are incorporated by reference in this disclosure.

Technical Field

The present disclosure relates to the field of display technology, and in particular to a screen transmission control method, device, electronic device, and computer-readable storage medium.

Background technique

With the emergence and diversification of business meetings, home entertainment, competitive games and other scenarios, screen transmission (screen projection) as a display technology for displaying the display content of the other end has begun to be widely used in the above scenarios. Its characteristics are to collect the screen content of the screen transmission end, and use wireless or wired transmission technology to transmit it to the screen transmission receiving end for synchronous display of the content.

The above screen transmission method has limitations. It can only synchronize the display images, and lacks more functional interactions between the two devices.

Summary of the invention

The embodiments of the present disclosure provide a screen transmission control method, an electronic device, and a computer-readable storage medium, which are used to control a peer device using one device during a screen transmission process, thereby increasing the peer control function during the screen transmission process.

The present disclosure provides a screen transmission control method, which is applied to a screen transmission sending end, and the screen transmission control method includes: sending first display data corresponding to a display window to a screen transmission receiving end, so that the screen transmission receiving end displays in a projection window based on the first display data; receiving second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end; displaying in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control operations on the screen transmission receiving end.

The present disclosure also provides a screen transmission control method, which is applied to a screen transmission receiving end, and the screen transmission control method includes: acquiring first display data sent by a screen transmission sending end, the first display data corresponding to a display window of the screen transmission sending end; displaying in a projection window based on the first display data; sending second display data corresponding to an operation window to the screen transmission sending end, so that the screen transmission sending end displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

The present disclosure also provides a screen transmission control device, which is applied to a screen transmission sending end, and the device includes: a first sending module, which is configured to send first display data corresponding to a display window to a screen transmission receiving end, so that the screen transmission receiving end displays in a projection window based on the first display data; a receiving module, which is configured to receive second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end; a first display module, which displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control operations on the screen transmission receiving end.

The present disclosure also provides a screen transmission control device, which is applied to a screen transmission receiving end, and the device includes: an acquisition module, which is configured to acquire first display data sent by a screen transmission sending end, and the first display data corresponds to a display window of the screen transmission sending end; a second display module, which is configured to display in a projection window based on the first display data; a second sending module, which is configured to send second display data corresponding to an operation window to the screen transmission sending end, so that the screen transmission sending end displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

The present disclosure also provides an electronic device, which includes a processor and a memory, wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the above method.

The present disclosure also provides a computer-readable storage medium, in which program data is stored. When the program data is executed by a processor, it is used to implement the above method.

The screen transmission control method provided by the embodiment of the present disclosure includes: the screen transmission sending end sends the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receives the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; based on the second display data, it is displayed in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end. Through the above method, the content of the operation window of the screen transmission receiving end can be displayed on the display interface of the screen transmission sending end, so that both the screen transmission sending end and the screen transmission receiving end can control the screen transmission receiving end, so that during the screen transmission process, the screen transmission receiving end can be conveniently controlled, which is helpful for the embodiment and playback of the screen transmission content, and also convenient for the interaction of users at both ends.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only for reference only. For some embodiments of the present invention, those skilled in the art can derive other drawings based on these drawings without paying any creative work.

FIG1 is a flow chart of a first embodiment of a screen transmission control method provided by the present disclosure;

FIG2 is a flow chart of step 11 in an embodiment of the present disclosure;

FIG3 is a flow chart of a second embodiment of a screen transmission control method provided by the present disclosure;

FIG4 is a flow chart of step 23 in an embodiment of the present disclosure;

FIG5 is a first interactive schematic diagram of the screen transmission control method provided by the present disclosure;

FIG6 is a flow chart of a third embodiment of a screen transmission control method provided by the present disclosure;

FIG7 is a schematic flow chart of a fourth embodiment of a screen transmission control method provided by the present disclosure;

FIG8 is a second interactive schematic diagram of the screen transmission control method provided by the present disclosure;

FIG9 is a flowchart of a fifth embodiment of a screen transmission control method provided by the present disclosure;

FIG10 is a flow chart of a sixth embodiment of a screen transmission control method provided by the present disclosure;

FIG11 is a third interactive schematic diagram of the screen transmission control method provided by the present disclosure;

FIG12 is a schematic structural diagram of an embodiment of a screen transmission control device provided by the present disclosure and applied to a screen transmission sending end;

FIG13 is a structural diagram of an embodiment of a screen transmission control device applied to a screen transmission receiving end provided by the present disclosure;

FIG14 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure;

FIG. 15 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present disclosure.

Detailed ways

The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

In the existing wireless screen transmission scenario, if you need to control the content on the conference tablet, such as opening the video conferencing software and sharing the content on the conference tablet through the video conferencing software, you need to Requiring users to move from their computers to the conference tablet and perform operations on the conference tablet does not provide a good user experience.

Based on this, the embodiment of the present disclosure provides a screen transmission control method, which is mainly used for transmitting the screen from a screen transmission sending end to a screen transmission receiving end, wherein the screen transmission sending end can be a mobile phone, a computer, etc., and the screen transmission receiving end can be a conference tablet, a TV light. The following is a specific description using the screen transmission sending end as a computer and the screen transmission receiving end as a conference tablet as an example.

Referring to FIG. 1 , FIG. 1 is a flow chart of a first embodiment of a screen transmission control method provided by the present disclosure. The execution subject of the method is a screen transmission sending end, such as a computer. The method includes:

Step 11: Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.

Among them, the display interface of the computer may include a display window and a control window.

The display window is used to display the content that needs to be transmitted, such as video screen, picture screen, PPT (PowerPoint, slide) document screen, live broadcast screen, etc. In a specific embodiment, the display window can be a window of an application, such as a video player software, conference software, live broadcast software, or document player software (word or PDF, etc.).

The control window is used to display a control interface, so as to use the control interface to obtain user input events, thereby controlling the screen transmission receiving end, such as a conference tablet.

Step 12: Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.

Among them, the display interface of the conference tablet can include a screen transmission window and an operation window.

The screen transmission window is used to display the first display data sent by the computer, that is, the content in the display window of the computer will be synchronously transmitted to the screen transmission window of the conference tablet for display. It can be understood that the display content of the screen transmission window may be delayed relative to the display content of the display window due to the delay in data transmission.

The operation window is used to display the operation interface, so as to obtain the user's input instructions through the operation interface, so as to control the conference tablet.

Step 13: Display in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Among them, at the screen transmission end, the user can control the screen transmission receiving end through the control window. At the screen transmission receiving end, the user can control the screen transmission receiving end through the operation window, that is, the screen transmission sending end transmits the display screen to the screen transmission receiving end. Users at both the screen transmission sending end and the screen transmission receiving end can operate and control the screen transmission receiving end, which can provide a better functional experience in the screen transmission process.

The following application scenarios of computers and conference tablets are used as examples: in the scenario of a remote video conference, the computer used by the host transmits the course content to the conference tablet. The conference tablet includes a projection window and an operation window. The projection window displays the course content, and the operation window displays control buttons, such as volume adjustment buttons, brightness adjustment buttons, microphone on/off buttons, etc. At the same time, the display content of the operation window is transmitted to the computer as the second display data and displayed in the control window of the computer. The host can control electronic device B through the control window. For example, if the host finds that the microphone of the conference tablet is not turned on, the host can actively adjust to turn on the microphone of the conference tablet, so there is no need to walk to the conference tablet to control the conference tablet when the microphone is not turned on and voice communication is impossible.

It is understandable that the control window of the computer may include not only the display content corresponding to the second display data, but also the control buttons for the computer. For example, the control window may include two parts, a first control window and a second control window, wherein the first control window is used to receive input instructions to control the computer, and the second control window is used to obtain input events to control the conference tablet.

Optionally, in one embodiment, as shown in FIG. 2 , FIG. 2 is a flow chart of step 11 in one embodiment of the present disclosure, and step 11 may specifically include:

Step 111: Obtain the full-screen display data of the screen transmission sending end.

The screen data can be collected by using the collection tools provided by the computer system, or by using installed hardware tools or software tools.

Taking Windows 10 as an example, WGC (Windows Graphics Capture) technology can be used to collect screen images, and the collected data is generally YUV data or RGB data. Of course, in other embodiments, screen collection technologies such as GDI, DXGI, Magnification, etc. can also be used, and examples are not given here one by one.

Step 112: Filter the second display data with the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window.

The second display data is the data displayed in the control window. Therefore, this step actually uses the full-screen display data to filter out the display data of the control window, so as to leave the display data of the display window, that is, the first display data. 1. Display data.

Taking Windows 10 as an example, window filtering can be performed by using Magnification and WGC for screen data collection. If WGC is used for screen data collection, Set Window Display Affinity can be used for window filtering.

Step 113: Send the first display data to the screen transmission receiving end.

The transmission of display data can be done by wired or wireless means. If the wired method is used, the RGB or YUV data is generally transmitted to the conference flat panel through the HDMI (High Definition Multimedia Interface) or DP (Decentralized Periphery) protocol (a profibus communication protocol running on a 485 serial port); if the wireless method is used, the h.264 encoder (such as libx264) can be used to compress and encode the RGB or YUV data, and then the h.264 data is transmitted to the conference flat panel through the IEEE 802.11 protocol in conjunction with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After receiving the data, the conference flat panel determines whether the data has been encoded and compressed. If so, the display data must first be decoded by the corresponding decoder (such as the h.264 decoder libx264). Then render the RGB or YUV data. If the conference tablet is running the Windows operating system, you can use OpenGL (Open Graphics Library) to render the display data. If the conference tablet is running the macOS operating system, you can use Metal (rendering interface) to render the display data.

As for the screen transmission receiving end, please refer to FIG. 3, which is a flow chart of the second embodiment of the screen transmission control method provided by the present disclosure. The execution subject of the method is the screen transmission receiving end. Taking the conference tablet as an example, the method includes:

Step 31: Acquire the first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.

Among them, the display interface of the conference tablet may include a display window and a control window.

The control window is used to display the control interface, so as to obtain the user's input events through the control interface. And control the conference tablet.

Step 32: Display in the projection window based on the first display data.

The screen transmission window is used to display the first display data sent by the screen transmission sender, such as a computer. That is, the content in the display window of the computer will be synchronously transmitted to the screen transmission window of the conference tablet for display. It can be understood that the display content of the screen transmission window may be delayed relative to the display content of the display window based on the delay in data transmission.

The operation window is used to display the operation interface, so as to obtain the user's input instructions through the operation interface, so as to control the screen transmission receiving end.

Step 33: Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Among them, on the computer side, users can control the conference tablet through the control window, and on the conference tablet side, users can control the conference tablet through the operation window, that is, the computer will transmit the display screen to the conference tablet, and users on both the computer and the conference tablet can operate and control the conference tablet, which can make the screen transmission process have a better functional experience.

Optionally, in one embodiment, as shown in FIG. 4 , FIG. 4 is a flow chart of step 33 in one embodiment of the present disclosure, and step 33 may specifically include:

Step 331: Obtain full-screen display data from the screen transmission receiving end.

The screen data can be collected by using the collection tools provided by the conference tablet system, or by using installed hardware tools or software tools.

Taking Windows 10 as an example, WGC (Windows Graphics Capture) technology can be used to capture screen images. The captured data is generally YUV data or RGB data. In other embodiments, screen acquisition technologies such as GDI, DXGI, and Magnification may also be used, and examples are not given here one by one.

Step 332: Filter the first display data with the full-screen display data of the screen transmission receiving end to obtain the second display data corresponding to the operation window.

Among them, the first display data is the data displayed in the operation window. Therefore, this step actually uses the full-screen display data to filter out the display data of the operation window to leave the display data of the screen transmission window, that is, the first display data.

Taking Windows 10 as an example, both Magnification and WGC can be used for screen data collection to perform window filtering. If WGC is used for screen data collection, SetWindowDisplayAffinity can be used for window filtering.

Step 333: Send the second display data to the screen transmission end.

Among them, the transmission of display data can be done by wired or wireless means. If the wired method is used, the RGB or YUV data is generally transmitted to the computer through the HDMI (High Definition Multimedia Interface) or DP (Decentralized Periphery) protocol (a profibus communication protocol running on the 485 serial port); if the wireless method is used, the h.264 encoder (such as libx264) can be used to compress and encode the RGB or YUV data, and then the h.264 data is transmitted to the computer through the IEEE 802.11 protocol in conjunction with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After the computer receives the data, it determines whether the data has been encoded and compressed. If so, the display data must first be decoded by the corresponding decoder (such as the h.264 decoder libx264). Then render the RGB or YUV data. If the computer is running Windows operating system, you can use OpenGL (Open Graphics Library) to render the display data. If the computer is running macOS operating system, you can use Metal (rendering interface) to render the display data.

In combination with FIG. 1 and FIG. 3 , refer to FIG. 5 , which is a first interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the screen transmission interaction process of the screen transmission sending end (taking a computer as an example) and the screen transmission receiving end (taking a conference tablet as an example):

①The screen transmission sending end collects full-screen display data;

② The screen transmission sending end filters the collected full-screen display data to control the display data of the window and obtains the first Display Data;

③ The screen transmission sending end sends the first display data to the screen transmission receiving end;

④ The screen transmission receiving end displays the display content corresponding to the first display data in the screen transmission window;

⑤The screen transmission receiving end collects full-screen display data;

⑥ The screen transmission receiving end filters the display data of the operation window with the collected full-screen display data to obtain the second display data;

⑦ The screen transmission receiving end sends the second display data to the screen transmission sending end;

⑧ The screen transmission sending end displays the display content corresponding to the second display data in the control window.

It is worth noting that the above steps ①-④ and steps ⑤-⑧ can be regarded as two independent processes, and the two processes can be carried out simultaneously. For example, the display data of the control window filtered in step ② is the content corresponding to the second display data displayed by the computer in the control window in step ⑧.

Different from the prior art, the screen transmission control method provided in this embodiment includes: the screen transmission sending end sends the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receives the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; based on the second display data, it is displayed in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end. Through the above method, the content of the operation window of the screen transmission receiving end can be displayed on the display interface of the screen transmission sending end, so that both the screen transmission sending end and the screen transmission receiving end can control the screen transmission receiving end, so that during the screen transmission process, the screen transmission receiving end can be conveniently controlled, which is helpful for the embodiment and playback of the screen transmission content, and also convenient for the interaction of users at both ends.

Referring to FIG. 6 , FIG. 6 is a flow chart of a third embodiment of a screen transmission control method provided by the present disclosure, the method is applied to a screen transmission sending end, and the method includes:

Step 61: Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.

Step 62: Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.

Step 63: Display in the control window of the screen transmission sending end based on the second display data.

The above steps 61-63 are similar to the above embodiment and will not be repeated here.

Step 64: Generate corresponding control data based on the input event of the control window.

Step 65: Send control data to the screen transmission receiving end, so that the screen transmission receiving end generates corresponding control instructions based on the control data simulated input event to control the operation window of the screen transmission receiving end.

The input event may include conventional input methods such as mouse, keyboard, touch screen, etc., and may also include interactive input such as voice input, gesture input, etc. In one embodiment, the input event may be captured using an interface of the operating system. Taking the Windows operating system as an example, the Windows operating system is configured with an API interface for capturing input events.

It can be understood that since the data captured by the mouse, keyboard, etc. are all numerical values or ascii characters, they can be easily converted into binary data. Then the binary data is transmitted to the conference tablet. After the conference tablet receives the corresponding data, it can simulate the control instructions of the corresponding input event, thus realizing the computer controlling the conference tablet.

When implementing event simulation on the conference tablet, there is a detail that needs attention. Since the conference tablet screen rendered on the computer actually filters the computer's screen content, we also need to take this factor into consideration when simulating events on the conference tablet, that is, to achieve transparent transmission of input events.

Specifically, after the computer sends the control data to the conference tablet, the conference tablet generates corresponding control instructions based on the control data simulation input event. When simulating the input event, the projection window should be filtered out, that is, all control data is transparently transmitted to the operation window.

In the control process, taking the mouse as an example, when the system captures the input operation of the mouse, obtains the position information of the mouse cursor, determines the corresponding control data based on the position information, and sends the control data to the conference tablet. After receiving the control data, the conference tablet simulates the input event of the mouse in the operation window according to the position information to complete the corresponding click operation, thereby realizing the control of the conference tablet.

As for the screen transmission receiving end, please refer to FIG. 7 , which is a flow chart of a fourth embodiment of the screen transmission control method provided by the present disclosure. The method is applied to the screen transmission receiving end, and the method includes:

Step 71: Acquire the first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.

Step 72: Display in the projection window based on the first display data.

Step 73: Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end.

Step 74: Obtain control data sent by the screen transmission sending end, where the control data is generated by the screen transmission sending end based on input events of the control window.

Step 75: Generate corresponding control instructions based on the control data simulation input event to control the operation window of the screen transmission receiving end.

This embodiment is similar to the above embodiment and will not be described in detail here.

In an optional embodiment, step 75 may specifically include: simulating input events of the operation window based on the control data; generating corresponding control instructions according to the input events; and controlling the operation window of the screen transmission receiving end according to the control instructions.

It can be understood that because the display screen of the screen transmission receiving end includes a projection window and an operation window, and the projection window does not need to be operated, when simulating input events based on control data, the projection window should be filtered out to achieve transparent transmission of all input events to the operation window, that is, all control instructions should correspond to the operation window, so as to operate the operation window and then control the screen transmission receiving end.

In combination with FIG. 6 and FIG. 7 , refer to FIG. 8 , which is a second interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the control interaction process between the screen transmission sending end and the screen transmission receiving end:

①The screen transmission sending end collects input events;

② The screen transmission sending end filters the input events, and the remaining input events are the real input events of the corresponding control window;

③ First, based on the real input event of the control window, determine the corresponding position information and form control data;

④The screen transmission sending end sends control data to the screen transmission receiving end;

⑤ The screen transmission receiving end generates corresponding control instructions based on the control data simulation input event, and uses the control instructions to control the screen transmission receiving end.

For example, in a scenario where a teacher is giving a remote video lecture to a student, the teacher's electronic device A transmits the course content to the student's electronic device B. Electronic device B includes a projection window and an operation window. The projection window displays the course content, and the operation window displays control buttons, such as microphone on/off. button, and at the same time, the display content of the operation window is transmitted to electronic device A as the second display data and displayed in the control window of electronic device A. The teacher can control electronic device B through the control window. For example, if the teacher finds that the student has not turned on the microphone, the teacher clicks the microphone on/off button in the control window with the mouse. The system captures the mouse click operation and determines the mouse click position, forms control data and sends it to electronic device B. After receiving the control data, electronic device B simulates the corresponding microphone on/off button click operation in the operation window according to the mouse click operation and click position, thus completing the control of the screen transmission sending end over the screen transmission receiving end.

Referring to FIG. 9 , FIG. 9 is a flow chart of a fifth embodiment of a screen transmission control method provided by the present disclosure, the method is applied to a screen transmission sending end, and the method includes:

Step 91: Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.

Step 92: Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.

Step 93: Display in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Step 94: In response to triggering the file copy operation, obtain the copied file data.

Step 95: In response to obtaining the file pull instruction sent by the screen transmission receiving end, send the obtained file data to the screen transmission receiving end; wherein, the file pull instruction is generated by the screen transmission receiving end in response to triggering a file pasting operation.

The computer responds to the file copy operation to obtain the target file data, and the conference tablet responds to the file paste operation to pull the target file data of the amplitude to the computer to achieve cross-device copying and pasting of the target file data.

Optionally, in one embodiment, step 94 may be specifically as follows: in response to triggering a file copy operation, reading the file data in the clipboard of the screen transmission sending end, and sending an identification information to the screen transmission receiving end; wherein the identification information is used to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, sending a file pull instruction to the screen transmission sending end.

The computer responds to the trigger file copy operation and reads the file data in the clipboard of the conference tablet. The conference tablet receives the first identification information and sends an identification information to the conference tablet; after receiving the first identification information, the conference tablet writes temporary data in the clipboard of the conference tablet, and when the conference tablet responds that the clipboard data of the conference tablet is empty, it is considered that the conference tablet has triggered a paste operation and sends a file pull instruction to the computer; after the computer obtains the file pull instruction, it sends the copied file data to the conference tablet to complete the cross-device copy and paste (file transfer) of the file data.

As for the screen transmission receiving end, please refer to FIG. 10 , which is a flow chart of a sixth embodiment of the screen transmission control method provided by the present disclosure. The method is applied to the screen transmission receiving end, and the method includes:

Step 101: Acquire first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.

Step 102: Display in the projection window based on the first display data.

Step 103: Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Step 104: In response to triggering the file pasting operation, sending a file pull instruction to the screen transmission sending end.

Step 105: Obtain the file data sent by the screen transmission sending end in response to the pull instruction; wherein the file data is obtained by the screen transmission sending end in response to triggering a file copy operation.

Optionally, in one embodiment, step 104 may specifically include: in response to receiving identification information sent by the screen transmission sending end, writing temporary data in the clipboard of the screen transmission receiving end; wherein the identification information is generated by the screen transmission sending end in response to triggering a file copy operation; in response to the data in the clipboard of the screen transmission receiving end being empty, sending a file pull instruction to the screen transmission sending end.

In combination with FIG. 9 and FIG. 10 , refer to FIG. 11 , which is a third interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the file sharing interactive process between the screen transmission sending end and the screen transmission receiving end:

①The screen transmission sender triggers the file copy operation;

② The screen transmission sending end reads the file data in the clipboard of the screen transmission sending end;

③ The screen transmission sending end sends an identification information to the screen transmission receiving end;

④ The screen transmission receiving end writes temporary data into the clipboard of the screen transmission receiving end in response to the identification information;

⑤ When the clipboard data of the screen transmission receiving end is empty, the screen transmission receiving end sends a file pull instruction to the screen transmission sending end;

⑥ The screen transmission sending end responds to the file pull instruction and sends the file data to the screen transmission receiving end.

Referring to FIG. 12 , FIG. 12 is a schematic diagram of the structure of a screen transmission control device applied to a screen transmission sending end provided by the present disclosure, and the screen transmission control device applied to the screen transmission sending end includes:

A first sending module is configured to send first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays the first display data in the projection window;

A receiving module, configured to receive second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end;

The first display module displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Optionally, in one embodiment, the first sending module includes:

A first acquisition unit, configured to acquire full-screen display data from a screen transmission sending end;

A first filtering unit is configured to filter the second display data from the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window;

The first sending unit is configured to send the first display data to the screen transmission receiving end.

Optionally, in one embodiment, the device comprises:

The first control module is configured to generate corresponding control data based on input events of the control window; the control data is sent to the screen transmission receiving end so that the screen transmission receiving end generates corresponding control instructions based on the control data simulated input events to control the operation window of the screen transmission receiving end.

Optionally, in one embodiment, the device comprises:

The first response module is configured to obtain the copied file data in response to triggering a file copy operation; in response to obtaining a file pull instruction sent by the screen transmission receiving end, send the obtained file data to the screen transmission receiving end; wherein the file pull instruction is generated by the screen transmission receiving end in response to triggering a file paste operation.

Optionally, in one embodiment, the first response module is arranged at:

In response to triggering a file copy operation, the file data in the clipboard of the screen transmission sending end is read, and an identification information is sent to the screen transmission receiving end;

Among them, the identification information is set to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, send a file pull instruction to the screen transmission sending end.

Referring to FIG. 13 , FIG. 13 is a screen transmission control device provided by the present disclosure and applied to a screen transmission receiving end. The screen transmission control device applied to a screen transmission receiving end includes:

An acquisition module, configured to acquire first display data sent by the screen transmission sending end, the first display data corresponding to a display window of the screen transmission sending end;

A second display module, configured to display in the projection window based on the first display data;

The second sending module is configured to send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

Optionally, in one embodiment, the second sending module includes:

A second acquisition unit is configured to acquire full-screen display data of a screen transmission receiving end;

A second filtering unit is configured to filter the first display data by the full-screen display data of the screen transmission receiving end to obtain second display data corresponding to the operation window;

The second sending unit is configured to send the second display data to the screen transmission end.

Optionally, in one embodiment, the device comprises:

The second control module is configured to obtain control data sent by the screen transmission sending end, and the control data is generated by the screen transmission sending end based on the input events of the control window; based on the control data, the corresponding control instructions are generated to control the operation window of the screen transmission receiving end by simulating input events.

Optionally, in one embodiment, the second control module is arranged at:

Simulate input events of the operation window based on the control data;

Generate corresponding control instructions according to input events;

The operation window of the screen transmission receiving end is controlled according to the control instructions.

Optionally, in one embodiment, the device further comprises:

The second response module is configured to send a file pull instruction to the screen transmission sending end in response to a triggered file paste operation; obtain the file data sent by the screen transmission sending end in response to the pull instruction; wherein the file data is obtained by the screen transmission sending end in response to a triggered file copy operation.

Optionally, in one embodiment, a second response module is configured to write temporary data in the clipboard of the screen transmission receiving end in response to receiving identification information sent by the screen transmission sending end; wherein the identification information is generated by the screen transmission sending end in response to triggering a file copy operation; in response to the data in the clipboard of the screen transmission receiving end being empty, send a file pull instruction to the screen transmission sending end.

Please refer to FIG. 14 , which is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. The electronic device 120 includes a processor 121 and a memory 122 , and the memory 122 is used to store program data.

Wherein, when the electronic device 120 is a screen transmission sending end, the processor 121 is used to execute the above-mentioned program data to implement the following method: sending the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receiving the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; displaying in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.

Wherein, when the electronic device 120 is a screen transmission receiving end, the processor 121 is used to execute the above-mentioned program data to implement the following method: obtain the first display data sent by the screen transmission sending end, the first display data corresponds to the display window of the screen transmission sending end; display in the projection window based on the first display data; send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.

In this embodiment, the electronic device 120 can be a PC (personal computer), a notebook, a television, or a mobile phone, a tablet computer, a smart wearable device, etc. For example, it can be a screen projection scene of a mobile phone and a television, a conference scene of a user terminal (computer) and a conference terminal (computer), a scene of a computer terminal and a projection device, etc.

Referring to FIG. 15 , FIG. 15 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present disclosure. The computer-readable storage medium 130 stores program data 131, and the program data 131 is executed by a processor to perform:

Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; display in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end. Or obtain the first display data sent by the screen transmission sending end, the first display data corresponds to the display window of the screen transmission sending end; display in the projection window based on the first display data; send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.

The display screen provided by the embodiment of the present disclosure is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present disclosure. The description of the above embodiments is only used to help understand the method and core idea of the present disclosure. At the same time, for technical personnel in this field, according to the idea of the present disclosure, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present disclosure.

Claims

A screen transmission control method is applied to a screen transmission sending end, wherein the screen transmission control method comprises:

Sending first display data corresponding to the display window to a screen transmission receiving end, so that the screen transmission receiving end displays the first display data in the projection window;

Receiving second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end;

The control window of the screen transmission sending end is displayed based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
The screen transmission control method according to claim 1, wherein:

The sending of the first display data corresponding to the display window to the screen transmission receiving end includes:

Obtaining full-screen display data of the screen transmission sending end;

Filter the second display data by the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window;

The first display data is sent to the screen transmission receiving end.
The screen transmission control method according to claim 1, wherein:

The method further comprises:

Based on the input event of the control window, generating corresponding control data;

The control data is sent to the screen transmission receiving end so that the screen transmission receiving end simulates the input event based on the control data to generate corresponding control instructions to control the operation window of the screen transmission receiving end.
The screen transmission control method according to claim 1, wherein:

The method further comprises:

In response to triggering a file copy operation, obtaining copied file data;

In response to obtaining a file pull instruction sent by the screen transmission receiving end, sending the obtained file data to the screen transmission receiving end;

The file pull instruction is generated by the screen transmission receiving end in response to triggering a file pasting operation.
The screen transmission control method according to claim 4, wherein:

The step of obtaining the copied file data in response to triggering the file copy operation and sending identification information to the screen transmission receiving end includes:

In response to triggering a file copy operation, reading file data in the clipboard of the screen transmission sending end, and sending identification information to the screen transmission receiving end;

Among them, the identification information is set to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, send the file pulling instruction to the screen transmission sending end.
A screen transmission control method is applied to a screen transmission receiving end, wherein the screen transmission control method comprises:

Acquire first display data sent by the screen transmission sending end, where the first display data corresponds to a display window of the screen transmission sending end;

Displaying in the projection window based on the first display data;

Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
The screen transmission control method according to claim 6, wherein:

The sending of the second display data corresponding to the operation window to the screen transmission sending end includes:

Obtaining full-screen display data of the screen transmission receiving end;

Filtering the first display data by the full-screen display data of the screen transmission receiving end to obtain second display data corresponding to the operation window;

Send the second display data to the screen transmission sending end.
The screen transmission control method according to claim 6, wherein:

The method further comprises:

Acquire control data sent by the screen transmission sending end, where the control data is generated by the screen transmission sending end based on an input event of the control window;

The input event is simulated based on the control data to generate corresponding control instructions to control the operation window of the screen transmission receiving end.
The screen transmission control method according to claim 8, wherein:

The simulating the input event based on the control data to generate a corresponding control instruction to control the operation window of the screen transmission receiving end includes:

simulating an input event of the operation window based on the control data;

Generate a corresponding control instruction according to the input event;

The operation window of the screen transmission receiving end is controlled according to the control instruction.
The screen transmission control method according to claim 6, wherein:

The method further comprises:

In response to triggering a file pasting operation, sending a file pulling instruction to the screen transmission sending end;

Obtain the file data sent by the screen transmission sending end in response to the pull instruction; wherein the file data is obtained by the screen transmission sending end in response to triggering a file copy operation.
The screen transmission control method according to claim 10, wherein:

The step of sending a file pull instruction to the screen transmission sending end in response to triggering the file pasting operation includes:

In response to receiving the identification information sent by the screen transmission sending end, writing temporary data in the clipboard of the screen transmission receiving end; wherein the identification information is generated by the screen transmission sending end in response to triggering a file copy operation;

In response to the data in the clipboard of the screen transmission receiving end being empty, the file pulling instruction is sent to the screen transmission sending end.
A screen transmission control device is applied to a screen transmission sending end, wherein the device comprises:

A first sending module is configured to send first display data corresponding to the display window to a screen transmission receiving end, so that the screen transmission receiving end displays the first display data in the projection window;

A receiving module, configured to receive second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end;

The first display module displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
The device according to claim 12, wherein the first sending module comprises:

A first acquisition unit, configured to acquire full-screen display data of the screen transmission sending end;

A first filtering unit is configured to filter the second display data by the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window;

The first sending unit is configured to send the first display data to the screen transmission receiving end.
The device according to claim 12, wherein the device comprises:

The first control module is configured to generate corresponding control data based on the input event of the control window; and send the control data to the screen transmission receiving end so that the screen transmission receiving end simulates the input event based on the control data to generate corresponding control instructions to control the operation window of the screen transmission receiving end.
The device according to claim 12, wherein the device comprises:

The first response module is configured to obtain the copied file data in response to triggering a file copy operation; in response to obtaining a file pull instruction sent by a screen transmission receiving end, send the obtained file data to the screen transmission receiving end; wherein the file pull instruction is generated by the screen transmission receiving end in response to triggering a file paste operation.
The device according to claim 15, wherein the first response module is arranged:

In response to triggering a file copy operation, reading file data in the clipboard of the screen transmission sending end, and sending an identification information to the screen transmission receiving end;

Among them, the identification information is set to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, send the file pulling instruction to the screen transmission sending end.
A screen transmission control device is applied to a screen transmission receiving end, wherein the device comprises:

An acquisition module, configured to acquire first display data sent by a screen transmission sending end, wherein the first display data corresponds to a display window of the screen transmission sending end;

A second display module, configured to display the first display data in a projection window;

The second sending module is configured to send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
The device according to claim 17, wherein the second sending module comprises:

A second acquisition unit is configured to acquire full-screen display data of the screen transmission receiving end;

The second filtering unit is configured to filter the full-screen display data of the screen transmission receiving end to the first display display data to obtain second display data corresponding to the operation window;

The second sending unit is configured to send the second display data to the screen transmission end.
An electronic device, wherein the electronic device comprises a processor and a memory, the memory is configured to store program data, and the processor is configured to execute the program data to implement the method according to any one of claims 1-11.
A computer-readable storage medium, wherein program data is stored in the computer-readable storage medium, and when the program data is executed by a processor, it is configured to implement the method according to any one of claims 1 to 11.