WO2024082803A1 - Procédé et appareil de commande de duplication d'écran, dispositif électronique, et support de stockage lisible par ordinateur - Google Patents

Procédé et appareil de commande de duplication d'écran, dispositif électronique, et support de stockage lisible par ordinateur Download PDF

Info

Publication number
WO2024082803A1
WO2024082803A1 PCT/CN2023/114032 CN2023114032W WO2024082803A1 WO 2024082803 A1 WO2024082803 A1 WO 2024082803A1 CN 2023114032 W CN2023114032 W CN 2023114032W WO 2024082803 A1 WO2024082803 A1 WO 2024082803A1
Authority
WO
WIPO (PCT)
Prior art keywords
screen transmission
control
receiving end
window
display data
Prior art date
Application number
PCT/CN2023/114032
Other languages
English (en)
Chinese (zh)
Inventor
冯君骅
Original Assignee
广州视臻信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州视臻信息科技有限公司 filed Critical 广州视臻信息科技有限公司
Publication of WO2024082803A1 publication Critical patent/WO2024082803A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units

Definitions

  • the present disclosure relates to the field of display technology, and in particular to a screen transmission control method, device, electronic device, and computer-readable storage medium.
  • screen transmission as a display technology for displaying the display content of the other end has begun to be widely used in the above scenarios. Its characteristics are to collect the screen content of the screen transmission end, and use wireless or wired transmission technology to transmit it to the screen transmission receiving end for synchronous display of the content.
  • the above screen transmission method has limitations. It can only synchronize the display images, and lacks more functional interactions between the two devices.
  • the embodiments of the present disclosure provide a screen transmission control method, an electronic device, and a computer-readable storage medium, which are used to control a peer device using one device during a screen transmission process, thereby increasing the peer control function during the screen transmission process.
  • the present disclosure provides a screen transmission control method, which is applied to a screen transmission sending end, and the screen transmission control method includes: sending first display data corresponding to a display window to a screen transmission receiving end, so that the screen transmission receiving end displays in a projection window based on the first display data; receiving second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end; displaying in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control operations on the screen transmission receiving end.
  • the present disclosure also provides a screen transmission control method, which is applied to a screen transmission receiving end, and the screen transmission control method includes: acquiring first display data sent by a screen transmission sending end, the first display data corresponding to a display window of the screen transmission sending end; displaying in a projection window based on the first display data; sending second display data corresponding to an operation window to the screen transmission sending end, so that the screen transmission sending end displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the present disclosure also provides a screen transmission control device, which is applied to a screen transmission sending end, and the device includes: a first sending module, which is configured to send first display data corresponding to a display window to a screen transmission receiving end, so that the screen transmission receiving end displays in a projection window based on the first display data; a receiving module, which is configured to receive second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end; a first display module, which displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control operations on the screen transmission receiving end.
  • the present disclosure also provides a screen transmission control device, which is applied to a screen transmission receiving end, and the device includes: an acquisition module, which is configured to acquire first display data sent by a screen transmission sending end, and the first display data corresponds to a display window of the screen transmission sending end; a second display module, which is configured to display in a projection window based on the first display data; a second sending module, which is configured to send second display data corresponding to an operation window to the screen transmission sending end, so that the screen transmission sending end displays in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the present disclosure also provides an electronic device, which includes a processor and a memory, wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the above method.
  • the present disclosure also provides a computer-readable storage medium, in which program data is stored.
  • program data When the program data is executed by a processor, it is used to implement the above method.
  • the screen transmission control method provided by the embodiment of the present disclosure includes: the screen transmission sending end sends the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receives the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; based on the second display data, it is displayed in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.
  • the content of the operation window of the screen transmission receiving end can be displayed on the display interface of the screen transmission sending end, so that both the screen transmission sending end and the screen transmission receiving end can control the screen transmission receiving end, so that during the screen transmission process, the screen transmission receiving end can be conveniently controlled, which is helpful for the embodiment and playback of the screen transmission content, and also convenient for the interaction of users at both ends.
  • FIG1 is a flow chart of a first embodiment of a screen transmission control method provided by the present disclosure
  • FIG2 is a flow chart of step 11 in an embodiment of the present disclosure.
  • FIG3 is a flow chart of a second embodiment of a screen transmission control method provided by the present disclosure.
  • FIG4 is a flow chart of step 23 in an embodiment of the present disclosure.
  • FIG5 is a first interactive schematic diagram of the screen transmission control method provided by the present disclosure.
  • FIG6 is a flow chart of a third embodiment of a screen transmission control method provided by the present disclosure.
  • FIG7 is a schematic flow chart of a fourth embodiment of a screen transmission control method provided by the present disclosure.
  • FIG8 is a second interactive schematic diagram of the screen transmission control method provided by the present disclosure.
  • FIG9 is a flowchart of a fifth embodiment of a screen transmission control method provided by the present disclosure.
  • FIG10 is a flow chart of a sixth embodiment of a screen transmission control method provided by the present disclosure.
  • FIG11 is a third interactive schematic diagram of the screen transmission control method provided by the present disclosure.
  • FIG12 is a schematic structural diagram of an embodiment of a screen transmission control device provided by the present disclosure and applied to a screen transmission sending end;
  • FIG13 is a structural diagram of an embodiment of a screen transmission control device applied to a screen transmission receiving end provided by the present disclosure
  • FIG14 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
  • FIG. 15 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present disclosure.
  • the embodiment of the present disclosure provides a screen transmission control method, which is mainly used for transmitting the screen from a screen transmission sending end to a screen transmission receiving end, wherein the screen transmission sending end can be a mobile phone, a computer, etc., and the screen transmission receiving end can be a conference tablet, a TV light.
  • the screen transmission sending end can be a mobile phone, a computer, etc.
  • the screen transmission receiving end can be a conference tablet, a TV light.
  • the following is a specific description using the screen transmission sending end as a computer and the screen transmission receiving end as a conference tablet as an example.
  • FIG. 1 is a flow chart of a first embodiment of a screen transmission control method provided by the present disclosure.
  • the execution subject of the method is a screen transmission sending end, such as a computer.
  • the method includes:
  • Step 11 Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.
  • the display interface of the computer may include a display window and a control window.
  • the display window is used to display the content that needs to be transmitted, such as video screen, picture screen, PPT (PowerPoint, slide) document screen, live broadcast screen, etc.
  • the display window can be a window of an application, such as a video player software, conference software, live broadcast software, or document player software (word or PDF, etc.).
  • the control window is used to display a control interface, so as to use the control interface to obtain user input events, thereby controlling the screen transmission receiving end, such as a conference tablet.
  • Step 12 Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.
  • the display interface of the conference tablet can include a screen transmission window and an operation window.
  • the screen transmission window is used to display the first display data sent by the computer, that is, the content in the display window of the computer will be synchronously transmitted to the screen transmission window of the conference tablet for display. It can be understood that the display content of the screen transmission window may be delayed relative to the display content of the display window due to the delay in data transmission.
  • the operation window is used to display the operation interface, so as to obtain the user's input instructions through the operation interface, so as to control the conference tablet.
  • Step 13 Display in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the user can control the screen transmission receiving end through the control window.
  • the user can control the screen transmission receiving end through the operation window, that is, the screen transmission sending end transmits the display screen to the screen transmission receiving end.
  • Users at both the screen transmission sending end and the screen transmission receiving end can operate and control the screen transmission receiving end, which can provide a better functional experience in the screen transmission process.
  • the following application scenarios of computers and conference tablets are used as examples: in the scenario of a remote video conference, the computer used by the host transmits the course content to the conference tablet.
  • the conference tablet includes a projection window and an operation window.
  • the projection window displays the course content
  • the operation window displays control buttons, such as volume adjustment buttons, brightness adjustment buttons, microphone on/off buttons, etc.
  • the display content of the operation window is transmitted to the computer as the second display data and displayed in the control window of the computer.
  • the host can control electronic device B through the control window.
  • the host can actively adjust to turn on the microphone of the conference tablet, so there is no need to walk to the conference tablet to control the conference tablet when the microphone is not turned on and voice communication is impossible.
  • control window of the computer may include not only the display content corresponding to the second display data, but also the control buttons for the computer.
  • the control window may include two parts, a first control window and a second control window, wherein the first control window is used to receive input instructions to control the computer, and the second control window is used to obtain input events to control the conference tablet.
  • FIG. 2 is a flow chart of step 11 in one embodiment of the present disclosure, and step 11 may specifically include:
  • Step 111 Obtain the full-screen display data of the screen transmission sending end.
  • the screen data can be collected by using the collection tools provided by the computer system, or by using installed hardware tools or software tools.
  • WGC Windows Graphics Capture
  • GDI GDI
  • DXGI DXGI
  • Magnification etc.
  • Step 112 Filter the second display data with the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window.
  • the second display data is the data displayed in the control window. Therefore, this step actually uses the full-screen display data to filter out the display data of the control window, so as to leave the display data of the display window, that is, the first display data. 1. Display data.
  • window filtering can be performed by using Magnification and WGC for screen data collection. If WGC is used for screen data collection, Set Window Display Affinity can be used for window filtering.
  • Step 113 Send the first display data to the screen transmission receiving end.
  • the transmission of display data can be done by wired or wireless means. If the wired method is used, the RGB or YUV data is generally transmitted to the conference flat panel through the HDMI (High Definition Multimedia Interface) or DP (Decentralized Periphery) protocol (a profibus communication protocol running on a 485 serial port); if the wireless method is used, the h.264 encoder (such as libx264) can be used to compress and encode the RGB or YUV data, and then the h.264 data is transmitted to the conference flat panel through the IEEE 802.11 protocol in conjunction with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After receiving the data, the conference flat panel determines whether the data has been encoded and compressed.
  • HDMI High Definition Multimedia Interface
  • DP Decentralized Periphery protocol
  • the h.264 encoder such as libx264
  • TCP Transmission Control Protocol
  • UDP User Datagram Protocol
  • the display data must first be decoded by the corresponding decoder (such as the h.264 decoder libx264). Then render the RGB or YUV data. If the conference tablet is running the Windows operating system, you can use OpenGL (Open Graphics Library) to render the display data. If the conference tablet is running the macOS operating system, you can use Metal (rendering interface) to render the display data.
  • the corresponding decoder such as the h.264 decoder libx264.
  • FIG. 3 is a flow chart of the second embodiment of the screen transmission control method provided by the present disclosure.
  • the execution subject of the method is the screen transmission receiving end. Taking the conference tablet as an example, the method includes:
  • Step 31 Acquire the first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.
  • the display interface of the conference tablet may include a display window and a control window.
  • the display window is used to display the content that needs to be transmitted, such as video screen, picture screen, PPT (PowerPoint, slide) document screen, live broadcast screen, etc.
  • the display window can be a window of an application, such as a video player software, conference software, live broadcast software, or document player software (word or PDF, etc.).
  • the control window is used to display the control interface, so as to obtain the user's input events through the control interface. And control the conference tablet.
  • Step 32 Display in the projection window based on the first display data.
  • the display interface of the conference tablet can include a screen transmission window and an operation window.
  • the screen transmission window is used to display the first display data sent by the screen transmission sender, such as a computer. That is, the content in the display window of the computer will be synchronously transmitted to the screen transmission window of the conference tablet for display. It can be understood that the display content of the screen transmission window may be delayed relative to the display content of the display window based on the delay in data transmission.
  • the operation window is used to display the operation interface, so as to obtain the user's input instructions through the operation interface, so as to control the screen transmission receiving end.
  • Step 33 Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • users can control the conference tablet through the control window
  • users can control the conference tablet through the operation window, that is, the computer will transmit the display screen to the conference tablet
  • users on both the computer and the conference tablet can operate and control the conference tablet, which can make the screen transmission process have a better functional experience.
  • control window of the computer may include not only the display content corresponding to the second display data, but also the control buttons for the computer.
  • the control window may include two parts, a first control window and a second control window, wherein the first control window is used to receive input instructions to control the computer, and the second control window is used to obtain input events to control the conference tablet.
  • FIG. 4 is a flow chart of step 33 in one embodiment of the present disclosure, and step 33 may specifically include:
  • Step 331 Obtain full-screen display data from the screen transmission receiving end.
  • the screen data can be collected by using the collection tools provided by the conference tablet system, or by using installed hardware tools or software tools.
  • WGC Windows Graphics Capture
  • the captured data is generally YUV data or RGB data.
  • screen acquisition technologies such as GDI, DXGI, and Magnification may also be used, and examples are not given here one by one.
  • Step 332 Filter the first display data with the full-screen display data of the screen transmission receiving end to obtain the second display data corresponding to the operation window.
  • the first display data is the data displayed in the operation window. Therefore, this step actually uses the full-screen display data to filter out the display data of the operation window to leave the display data of the screen transmission window, that is, the first display data.
  • both Magnification and WGC can be used for screen data collection to perform window filtering. If WGC is used for screen data collection, SetWindowDisplayAffinity can be used for window filtering.
  • Step 333 Send the second display data to the screen transmission end.
  • the transmission of display data can be done by wired or wireless means. If the wired method is used, the RGB or YUV data is generally transmitted to the computer through the HDMI (High Definition Multimedia Interface) or DP (Decentralized Periphery) protocol (a profibus communication protocol running on the 485 serial port); if the wireless method is used, the h.264 encoder (such as libx264) can be used to compress and encode the RGB or YUV data, and then the h.264 data is transmitted to the computer through the IEEE 802.11 protocol in conjunction with TCP (Transmission Control Protocol), UDP (User Datagram Protocol) and other transmission protocols. After the computer receives the data, it determines whether the data has been encoded and compressed.
  • HDMI High Definition Multimedia Interface
  • DP Decentralized Periphery protocol
  • TCP Transmission Control Protocol
  • UDP User Datagram Protocol
  • the display data must first be decoded by the corresponding decoder (such as the h.264 decoder libx264). Then render the RGB or YUV data. If the computer is running Windows operating system, you can use OpenGL (Open Graphics Library) to render the display data. If the computer is running macOS operating system, you can use Metal (rendering interface) to render the display data.
  • the corresponding decoder such as the h.264 decoder libx264.
  • FIG. 5 is a first interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the screen transmission interaction process of the screen transmission sending end (taking a computer as an example) and the screen transmission receiving end (taking a conference tablet as an example):
  • the screen transmission sending end collects full-screen display data
  • the screen transmission sending end filters the collected full-screen display data to control the display data of the window and obtains the first Display Data;
  • the screen transmission sending end sends the first display data to the screen transmission receiving end
  • the screen transmission receiving end displays the display content corresponding to the first display data in the screen transmission window
  • the screen transmission receiving end collects full-screen display data
  • the screen transmission receiving end filters the display data of the operation window with the collected full-screen display data to obtain the second display data
  • the screen transmission receiving end sends the second display data to the screen transmission sending end
  • the screen transmission sending end displays the display content corresponding to the second display data in the control window.
  • the display data of the control window filtered in step 2 is the content corresponding to the second display data displayed by the computer in the control window in step 8.
  • the screen transmission control method provided in this embodiment includes: the screen transmission sending end sends the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receives the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; based on the second display data, it is displayed in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.
  • the content of the operation window of the screen transmission receiving end can be displayed on the display interface of the screen transmission sending end, so that both the screen transmission sending end and the screen transmission receiving end can control the screen transmission receiving end, so that during the screen transmission process, the screen transmission receiving end can be conveniently controlled, which is helpful for the embodiment and playback of the screen transmission content, and also convenient for the interaction of users at both ends.
  • FIG. 6 is a flow chart of a third embodiment of a screen transmission control method provided by the present disclosure, the method is applied to a screen transmission sending end, and the method includes:
  • Step 61 Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.
  • Step 62 Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.
  • Step 63 Display in the control window of the screen transmission sending end based on the second display data.
  • Step 64 Generate corresponding control data based on the input event of the control window.
  • Step 65 Send control data to the screen transmission receiving end, so that the screen transmission receiving end generates corresponding control instructions based on the control data simulated input event to control the operation window of the screen transmission receiving end.
  • the input event may include conventional input methods such as mouse, keyboard, touch screen, etc., and may also include interactive input such as voice input, gesture input, etc.
  • the input event may be captured using an interface of the operating system. Taking the Windows operating system as an example, the Windows operating system is configured with an API interface for capturing input events.
  • the data captured by the mouse, keyboard, etc. are all numerical values or ascii characters, they can be easily converted into binary data. Then the binary data is transmitted to the conference tablet. After the conference tablet receives the corresponding data, it can simulate the control instructions of the corresponding input event, thus realizing the computer controlling the conference tablet.
  • the conference tablet After the computer sends the control data to the conference tablet, the conference tablet generates corresponding control instructions based on the control data simulation input event.
  • the projection window should be filtered out, that is, all control data is transparently transmitted to the operation window.
  • the conference tablet simulates the input event of the mouse in the operation window according to the position information to complete the corresponding click operation, thereby realizing the control of the conference tablet.
  • FIG. 7 is a flow chart of a fourth embodiment of the screen transmission control method provided by the present disclosure. The method is applied to the screen transmission receiving end, and the method includes:
  • Step 71 Acquire the first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.
  • Step 72 Display in the projection window based on the first display data.
  • Step 73 Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end.
  • Step 74 Obtain control data sent by the screen transmission sending end, where the control data is generated by the screen transmission sending end based on input events of the control window.
  • Step 75 Generate corresponding control instructions based on the control data simulation input event to control the operation window of the screen transmission receiving end.
  • step 75 may specifically include: simulating input events of the operation window based on the control data; generating corresponding control instructions according to the input events; and controlling the operation window of the screen transmission receiving end according to the control instructions.
  • the projection window should be filtered out to achieve transparent transmission of all input events to the operation window, that is, all control instructions should correspond to the operation window, so as to operate the operation window and then control the screen transmission receiving end.
  • FIG. 8 is a second interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the control interaction process between the screen transmission sending end and the screen transmission receiving end:
  • the screen transmission sending end filters the input events, and the remaining input events are the real input events of the corresponding control window;
  • the screen transmission sending end sends control data to the screen transmission receiving end
  • the screen transmission receiving end generates corresponding control instructions based on the control data simulation input event, and uses the control instructions to control the screen transmission receiving end.
  • the teacher's electronic device A transmits the course content to the student's electronic device B.
  • Electronic device B includes a projection window and an operation window.
  • the projection window displays the course content
  • the operation window displays control buttons, such as microphone on/off. button, and at the same time, the display content of the operation window is transmitted to electronic device A as the second display data and displayed in the control window of electronic device A.
  • the teacher can control electronic device B through the control window. For example, if the teacher finds that the student has not turned on the microphone, the teacher clicks the microphone on/off button in the control window with the mouse.
  • the system captures the mouse click operation and determines the mouse click position, forms control data and sends it to electronic device B. After receiving the control data, electronic device B simulates the corresponding microphone on/off button click operation in the operation window according to the mouse click operation and click position, thus completing the control of the screen transmission sending end over the screen transmission receiving end.
  • FIG. 9 is a flow chart of a fifth embodiment of a screen transmission control method provided by the present disclosure, the method is applied to a screen transmission sending end, and the method includes:
  • Step 91 Send the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays it in the projection window based on the first display data.
  • Step 92 Receive the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end.
  • Step 93 Display in a control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • Step 94 In response to triggering the file copy operation, obtain the copied file data.
  • Step 95 In response to obtaining the file pull instruction sent by the screen transmission receiving end, send the obtained file data to the screen transmission receiving end; wherein, the file pull instruction is generated by the screen transmission receiving end in response to triggering a file pasting operation.
  • the computer responds to the file copy operation to obtain the target file data
  • the conference tablet responds to the file paste operation to pull the target file data of the amplitude to the computer to achieve cross-device copying and pasting of the target file data.
  • step 94 may be specifically as follows: in response to triggering a file copy operation, reading the file data in the clipboard of the screen transmission sending end, and sending an identification information to the screen transmission receiving end; wherein the identification information is used to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, sending a file pull instruction to the screen transmission sending end.
  • the computer responds to the trigger file copy operation and reads the file data in the clipboard of the conference tablet.
  • the conference tablet receives the first identification information and sends an identification information to the conference tablet; after receiving the first identification information, the conference tablet writes temporary data in the clipboard of the conference tablet, and when the conference tablet responds that the clipboard data of the conference tablet is empty, it is considered that the conference tablet has triggered a paste operation and sends a file pull instruction to the computer; after the computer obtains the file pull instruction, it sends the copied file data to the conference tablet to complete the cross-device copy and paste (file transfer) of the file data.
  • FIG. 10 is a flow chart of a sixth embodiment of the screen transmission control method provided by the present disclosure. The method is applied to the screen transmission receiving end, and the method includes:
  • Step 101 Acquire first display data sent by the screen transmission sending end, where the first display data corresponds to the display window of the screen transmission sending end.
  • Step 102 Display in the projection window based on the first display data.
  • Step 103 Send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • Step 104 In response to triggering the file pasting operation, sending a file pull instruction to the screen transmission sending end.
  • Step 105 Obtain the file data sent by the screen transmission sending end in response to the pull instruction; wherein the file data is obtained by the screen transmission sending end in response to triggering a file copy operation.
  • step 104 may specifically include: in response to receiving identification information sent by the screen transmission sending end, writing temporary data in the clipboard of the screen transmission receiving end; wherein the identification information is generated by the screen transmission sending end in response to triggering a file copy operation; in response to the data in the clipboard of the screen transmission receiving end being empty, sending a file pull instruction to the screen transmission sending end.
  • FIG. 11 is a third interactive schematic diagram of the screen transmission control method provided by the present disclosure, and introduces the file sharing interactive process between the screen transmission sending end and the screen transmission receiving end:
  • the screen transmission sending end reads the file data in the clipboard of the screen transmission sending end
  • the screen transmission sending end sends an identification information to the screen transmission receiving end
  • the screen transmission receiving end writes temporary data into the clipboard of the screen transmission receiving end in response to the identification information
  • the screen transmission sending end responds to the file pull instruction and sends the file data to the screen transmission receiving end.
  • FIG. 12 is a schematic diagram of the structure of a screen transmission control device applied to a screen transmission sending end provided by the present disclosure, and the screen transmission control device applied to the screen transmission sending end includes:
  • a first sending module is configured to send first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays the first display data in the projection window;
  • a receiving module configured to receive second display data sent by the screen transmission receiving end; wherein the second display data corresponds to an operation window of the screen transmission receiving end;
  • the first display module displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the first sending module includes:
  • a first acquisition unit configured to acquire full-screen display data from a screen transmission sending end
  • a first filtering unit is configured to filter the second display data from the full-screen display data of the screen transmission sending end to obtain the first display data corresponding to the display window;
  • the first sending unit is configured to send the first display data to the screen transmission receiving end.
  • the device comprises:
  • the first control module is configured to generate corresponding control data based on input events of the control window; the control data is sent to the screen transmission receiving end so that the screen transmission receiving end generates corresponding control instructions based on the control data simulated input events to control the operation window of the screen transmission receiving end.
  • the device comprises:
  • the first response module is configured to obtain the copied file data in response to triggering a file copy operation; in response to obtaining a file pull instruction sent by the screen transmission receiving end, send the obtained file data to the screen transmission receiving end; wherein the file pull instruction is generated by the screen transmission receiving end in response to triggering a file paste operation.
  • the first response module is arranged at:
  • the file data in the clipboard of the screen transmission sending end is read, and an identification information is sent to the screen transmission receiving end;
  • the identification information is set to enable the screen transmission receiving end to write temporary data in the clipboard of the screen transmission receiving end, and in response to the clipboard data of the screen transmission receiving end being empty, send a file pull instruction to the screen transmission sending end.
  • FIG. 13 is a screen transmission control device provided by the present disclosure and applied to a screen transmission receiving end.
  • the screen transmission control device applied to a screen transmission receiving end includes:
  • An acquisition module configured to acquire first display data sent by the screen transmission sending end, the first display data corresponding to a display window of the screen transmission sending end;
  • a second display module configured to display in the projection window based on the first display data
  • the second sending module is configured to send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays the second display data in the control window of the screen transmission sending end; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the second sending module includes:
  • a second acquisition unit is configured to acquire full-screen display data of a screen transmission receiving end
  • a second filtering unit is configured to filter the first display data by the full-screen display data of the screen transmission receiving end to obtain second display data corresponding to the operation window;
  • the second sending unit is configured to send the second display data to the screen transmission end.
  • the device comprises:
  • the second control module is configured to obtain control data sent by the screen transmission sending end, and the control data is generated by the screen transmission sending end based on the input events of the control window; based on the control data, the corresponding control instructions are generated to control the operation window of the screen transmission receiving end by simulating input events.
  • the second control module is arranged at:
  • the operation window of the screen transmission receiving end is controlled according to the control instructions.
  • the device further comprises:
  • the second response module is configured to send a file pull instruction to the screen transmission sending end in response to a triggered file paste operation; obtain the file data sent by the screen transmission sending end in response to the pull instruction; wherein the file data is obtained by the screen transmission sending end in response to a triggered file copy operation.
  • a second response module is configured to write temporary data in the clipboard of the screen transmission receiving end in response to receiving identification information sent by the screen transmission sending end; wherein the identification information is generated by the screen transmission sending end in response to triggering a file copy operation; in response to the data in the clipboard of the screen transmission receiving end being empty, send a file pull instruction to the screen transmission sending end.
  • FIG. 14 is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure.
  • the electronic device 120 includes a processor 121 and a memory 122 , and the memory 122 is used to store program data.
  • the processor 121 when the electronic device 120 is a screen transmission sending end, the processor 121 is used to execute the above-mentioned program data to implement the following method: sending the first display data corresponding to the display window to the screen transmission receiving end, so that the screen transmission receiving end displays in the projection window based on the first display data; receiving the second display data sent by the screen transmission receiving end; wherein the second display data corresponds to the operation window of the screen transmission receiving end; displaying in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.
  • the processor 121 when the electronic device 120 is a screen transmission receiving end, the processor 121 is used to execute the above-mentioned program data to implement the following method: obtain the first display data sent by the screen transmission sending end, the first display data corresponds to the display window of the screen transmission sending end; display in the projection window based on the first display data; send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to perform control operations on the screen transmission receiving end.
  • the electronic device 120 can be a PC (personal computer), a notebook, a television, or a mobile phone, a tablet computer, a smart wearable device, etc.
  • PC personal computer
  • it can be a screen projection scene of a mobile phone and a television, a conference scene of a user terminal (computer) and a conference terminal (computer), a scene of a computer terminal and a projection device, etc.
  • FIG. 15 is a schematic diagram of the structure of an embodiment of a computer-readable storage medium provided by the present disclosure.
  • the computer-readable storage medium 130 stores program data 131, and the program data 131 is executed by a processor to perform:
  • the first display data sent by the screen transmission sending end corresponds to the display window of the screen transmission sending end; display in the projection window based on the first display data; send the second display data corresponding to the operation window to the screen transmission sending end, so that the screen transmission sending end displays in the control window of the screen transmission sending end based on the second display data; wherein the control window is configured to respond to input events based on the control window to control the screen transmission receiving end.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

L'invention concerne un procédé et un appareil de commande de duplication d'écran, un dispositif électronique (120) et un support de stockage lisible par ordinateur (130). Le procédé de commande de duplication d'écran comprend les étapes suivantes: l'envoi, à une extrémité de réception de duplication d'écran, des premières données d'affichage correspondant à une fenêtre d'affichage, de sorte que l'extrémité de réception de duplication d'écran effectue un affichage dans une fenêtre de duplication d'écran sur la base des premières données d'affichage (11); la réception de secondes données d'affichage envoyées par l'extrémité de réception de duplication d'écran, les secondes données d'affichage correspondant à une fenêtre de fonctionnement de l'extrémité de réception de duplication d'écran (12); et l'exécution d'un affichage dans une fenêtre de commande d'une extrémité d'envoi de duplication d'écran sur la base des secondes données d'affichage, la fenêtre de commande étant configurée pour effectuer une opération de commande sur l'extrémité de réception de duplication d'écran en réponse à un événement d'entrée basé sur une fenêtre de commande (13). Une commande d'homologue peut être effectuée pendant la mise en miroir d'écran.
PCT/CN2023/114032 2022-10-21 2023-08-21 Procédé et appareil de commande de duplication d'écran, dispositif électronique, et support de stockage lisible par ordinateur WO2024082803A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211292727.2 2022-10-21
CN202211292727.2A CN117950610A (zh) 2022-10-21 2022-10-21 一种传屏控制方法、电子设备以及计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2024082803A1 true WO2024082803A1 (fr) 2024-04-25

Family

ID=90736850

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/114032 WO2024082803A1 (fr) 2022-10-21 2023-08-21 Procédé et appareil de commande de duplication d'écran, dispositif électronique, et support de stockage lisible par ordinateur

Country Status (2)

Country Link
CN (1) CN117950610A (fr)
WO (1) WO2024082803A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130219324A1 (en) * 2012-02-22 2013-08-22 Rsupport Co., Ltd. Remote control method and apparatus for a mobile terminal
CN111625371A (zh) * 2019-12-03 2020-09-04 蘑菇车联信息科技有限公司 一种文本传输方法、装置、pc端、移动终端及系统
CN112162716A (zh) * 2020-09-28 2021-01-01 北京小米移动软件有限公司 一种文件传输方法、装置、终端及存储介质
CN113515244A (zh) * 2021-03-24 2021-10-19 深圳乐播科技有限公司 基于投屏的终端遥控方法、装置、设备及存储介质
CN114040242A (zh) * 2021-09-30 2022-02-11 荣耀终端有限公司 投屏方法和电子设备
CN114979730A (zh) * 2021-02-25 2022-08-30 广州视源电子科技股份有限公司 传屏交互方法、装置、计算机设备和存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130219324A1 (en) * 2012-02-22 2013-08-22 Rsupport Co., Ltd. Remote control method and apparatus for a mobile terminal
CN111625371A (zh) * 2019-12-03 2020-09-04 蘑菇车联信息科技有限公司 一种文本传输方法、装置、pc端、移动终端及系统
CN112162716A (zh) * 2020-09-28 2021-01-01 北京小米移动软件有限公司 一种文件传输方法、装置、终端及存储介质
CN114979730A (zh) * 2021-02-25 2022-08-30 广州视源电子科技股份有限公司 传屏交互方法、装置、计算机设备和存储介质
CN113515244A (zh) * 2021-03-24 2021-10-19 深圳乐播科技有限公司 基于投屏的终端遥控方法、装置、设备及存储介质
CN114040242A (zh) * 2021-09-30 2022-02-11 荣耀终端有限公司 投屏方法和电子设备

Also Published As

Publication number Publication date
CN117950610A (zh) 2024-04-30

Similar Documents

Publication Publication Date Title
US10419618B2 (en) Information processing apparatus having whiteboard and video conferencing functions
WO2017101440A1 (fr) Procédé et système de synchronisation d'annotation à distance
US7908325B1 (en) System and method for event-based collaboration
CN110515580B (zh) 一种显示控制方法、装置及终端
WO2023045872A1 (fr) Procédé, appareil et dispositif de saut de page, et support de stockage et produit-programme
CN101673185B (zh) 一种图像显示触摸控制方法及其系统
EP4376423A1 (fr) Procédé et dispositif d'interaction d'objet virtuel, et support de stockage et produit-programme informatique
WO2019010827A1 (fr) Procédé et appareil d'acquisition vidéo, dispositif vidéo et support d'informations
CN108011951A (zh) 远程协助方法、装置、系统及电子设备
TW201334535A (zh) 在視訊會議中協同操作分享內容的方法、視訊會議系統以及電腦程式產品
KR20220041791A (ko) 생방송 정보 처리 방법, 장치, 기기, 저장 매체 및 컴퓨터 프로그램
CN112911320A (zh) 一种直播方法、装置、计算机设备和存储介质
WO2022117028A1 (fr) Procédés de partage et d'affichage d'opérations d'entrée de présentateur basés sur une diffusion en continu en direct d'un jeu et dispositif
JP2018525744A (ja) アプリケーション及びデータをタッチスクリーンコンピュータ間で相互共有する方法並びにこの方法を実施するコンピュータプログラム
Roussel Exploring new uses of video with videoSpace
WO2024082803A1 (fr) Procédé et appareil de commande de duplication d'écran, dispositif électronique, et support de stockage lisible par ordinateur
TWI740208B (zh) 影像傳輸裝置、具有遠端畫面擷取功能的影像顯示系統及遠端畫面影像擷取方法
CN112578977B (zh) 显示内容批注方法、装置及系统
JP4321751B2 (ja) 描画処理装置、描画処理方法および描画処理プログラム、並びにそれらを備えた電子会議システム
TWI328768B (en) System and method for recording and executing the operation process of a cursor device
CN106210665B (zh) 一种基于视频采集的远程主机控制方法及系统
CN113316011B (zh) 电子白板系统的控制方法、系统、设备及存储介质
CN111356009B (zh) 音频数据的处理方法、装置、存储介质以及终端
CN112214149A (zh) 智能交互显示装置及其截屏方法、系统及电子设备
JP6499582B2 (ja) 送信側コンピュータ、受信側コンピュータ、及びそれらにより実行される方法、並びにコンピュータプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23878798

Country of ref document: EP

Kind code of ref document: A1