CN114510205B - Display interaction method, display device, electronic device and storage medium - Google Patents

Display interaction method, display device, electronic device and storage medium Download PDF

Info

Publication number
CN114510205B
CN114510205B CN202111654737.1A CN202111654737A CN114510205B CN 114510205 B CN114510205 B CN 114510205B CN 202111654737 A CN202111654737 A CN 202111654737A CN 114510205 B CN114510205 B CN 114510205B
Authority
CN
China
Prior art keywords
display
display device
application window
interaction method
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202111654737.1A
Other languages
Chinese (zh)
Other versions
CN114510205A (en
Inventor
郭琛
聂超
赵丽菲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to CN202111654737.1A priority Critical patent/CN114510205B/en
Publication of CN114510205A publication Critical patent/CN114510205A/en
Application granted granted Critical
Publication of CN114510205B publication Critical patent/CN114510205B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0486—Drag-and-drop
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

The application discloses a display interaction method, display interaction equipment, electronic equipment and a storage medium. The method comprises the following steps: under the condition that the first display device enters a preset mode, at least one first application window is displayed in a first display area of the first display device, and pictures of at least one second display device are displayed in a second display area, wherein the first display device is connected with the second display device through a local area network, a target display device is determined in the second display device in response to first operation of a user on the first application window, and an application file and an operation log of the first application window are sent to the target display device so as to display the corresponding first application window on the target display device. According to the method and the device, the two display areas are arranged on the first display device, the second display device can be controlled according to the application window displayed in one of the display areas, and therefore data transmission and content synchronization on different display devices are achieved.

Description

Display interaction method, display device, electronic device and storage medium
Technical Field
The present disclosure relates to the field of data communications, and in particular, to a display interaction method, a display device, an electronic device, and a storage medium.
Background
In the related art, different devices can be connected under the same local area network, one device is a receiving end, the other devices are transmitting ends, and a communication mechanism is established by clicking a screen-throwing key in the different devices, so that the problem of interconnection among the different devices is solved. Therefore, the screen of the current device can be transmitted to the receiving end in real time after the screen is captured, so that the mirror image screen projection effect is realized, or the device screen of the touch transmitting end on the receiving end device is realized, and the reverse touch effect is realized when the coordinate point data is transmitted back to the transmitting end.
However, the above-mentioned devices are interconnected to achieve basic picture synchronization only by a method of screen capturing and picture transmission, or achieve the effect of touching a large screen with a small screen by transmitting coordinate point information of touch, so that it is difficult to truly achieve data transmission and content synchronization between the devices.
Disclosure of Invention
The embodiment of the application provides a display interaction method for a first display device, which comprises the following steps:
displaying at least one first application window in a first display area of the first display device and displaying a picture of at least one second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
Determining a target display device in the second display device in response to a first operation of the first application window by a user;
and sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
In some embodiments, the display interaction method includes:
and the first picture data is sent to the second display device through a screen throwing protocol so as to display a picture of the first display device on the second display device, wherein the first picture data comprises a screen shot or a screen recording of the first display device.
In some embodiments, the display interaction method further comprises:
searching for the second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation of a user in case that the first display device enters a predetermined mode;
and responding to the third operation of the searched second display device by the user, and displaying the corresponding second display device in the second display area.
In some embodiments, the displaying the corresponding second display device in the second display area includes:
Receiving second picture data sent by the second display equipment through a screen projection protocol, wherein the second picture data comprises screen capturing or screen recording of the second display equipment;
and rendering the second picture data to correspondingly display the picture of the second display device in the second display area.
In some embodiments, the first application window includes a first control, and the display interaction method includes:
and responding to a first dragging operation of the user on the first control, and controlling the first application window to move according to the track of the first dragging operation.
In some embodiments, the determining, in response to a first operation of the first application window by a user, a target display device in the second display device includes:
and responding to a first dragging operation of the user on the first control, and determining the corresponding second display device as the target display device under the condition that the first application window moves to the second display area and the first application window coincides with the second display device.
In some embodiments, the first application window includes a second control, and the display interaction method includes:
And responding to a second dragging operation of the user on the second control, and controlling the first application window to zoom according to the direction and the distance of the second dragging operation.
In some embodiments, the display interaction method further comprises:
displaying a first predicted display range of the scaled first application window according to the direction and distance of the second dragging operation in the process of the second dragging operation by the user;
and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
In some embodiments, the display interaction method includes:
displaying the first application window in a full screen mode on the first display device under the condition that the first application window is enlarged to a first display proportion; or (b)
And closing the first application window on the first display device under the condition that the first application window is reduced to a second display proportion.
In some embodiments, the display interaction method further comprises:
and after the application file and the operation log of the first application window are sent to the target display device, closing the first application window on the first display device.
In some embodiments, the screen of the second display device includes a second application window, and the display interaction method includes:
responding to a fourth operation of a user on the second application window, and sending a request instruction for the second application window to the corresponding second display equipment;
receiving an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction;
and displaying the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, the sending, in response to a fourth operation of the second application window by the user, a request instruction for the second application window to the corresponding second display device includes:
and responding to a fourth dragging operation of the user to move the second application window to the second display area, and sending a request instruction for the second application window to the corresponding second display equipment.
In some embodiments, the displaying the second application window in the second display area includes:
determining a second predicted display range of the second application window in the second display area according to the fourth dragging operation;
Rendering is carried out according to the second prediction display range so as to display the second application window in the prediction display range.
The embodiment of the application also provides a display device, which comprises:
the display module is used for displaying at least one first application window in a first display area of the first display device and displaying at least one picture of the second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
a determining module, configured to determine a target display device in the second display device in response to a first operation of the first application window by a user;
and the transmission module is used for sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
The application also provides electronic equipment, which comprises a memory and a processor, wherein the memory stores a computer program, and the display interaction method is realized when the computer program is executed by the processor.
The present application also provides a non-transitory computer readable storage medium containing a computer program which, when executed by a processor, causes the processor to perform the display interaction method.
According to the display interaction method, the display device, the electronic device and the computer readable storage medium, through the fact that the first display device is connected with the second display device in a preset mode, the first display device displays the first display area and the second display area, the first display area displays a first application window, the second display area displays a picture of the second display device, the second display device can be selected as a target display device according to first operation of a user on the first application window, and an application file and an operation log of the first application window are sent to the target display device, so that the corresponding first application window is displayed on the target display device, and data content synchronization among different devices is achieved.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is a flow chart of a display interaction method according to some embodiments of the present application.
Fig. 2 is a block diagram of a display device according to some embodiments of the present application.
Fig. 3 is a schematic view of a scenario illustrating an interaction method according to some embodiments of the present application.
Fig. 4-5 are flow diagrams illustrating a method of displaying interactions according to some embodiments of the present application.
Fig. 6 is a schematic diagram of another module of a display device according to some embodiments of the present application.
Fig. 7 is a schematic view of a scenario illustrating an interaction method according to some embodiments of the present application.
Figures 8-9 some embodiments of the present application show a flow diagram of an interaction method.
FIG. 10 is a schematic view of a scenario illustrating an interaction method in certain embodiments of the present application.
Fig. 11-12 are flow diagrams illustrating a method of display interaction according to some embodiments of the present application.
Fig. 13 is a schematic view of a scenario illustrating an interaction method according to some embodiments of the present application.
Fig. 14-16 are flow diagrams illustrating a method of display interaction according to some embodiments of the present application.
FIG. 17 is a schematic view of a scenario illustrating an interaction method according to some embodiments of the present application.
Detailed Description
Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary only for the purpose of explaining the present application and are not to be construed as limiting the present application.
Referring to fig. 1, an embodiment of the present application provides a display interaction method, which is used for a first display device, and the display interaction method includes the steps of:
01, under the condition that a first display device enters a preset mode, displaying at least one first application window in a first display area of the first display device and displaying a picture of at least one second display device in a second display area, wherein the first display device is connected with the second display device through a local area network;
02, determining a target display device in the second display device in response to a first operation of the first application window by the user;
and 03, sending the application file and the operation log of the first application window to the target display device so as to display the corresponding first application window on the target display device.
Referring further to fig. 2, a display device 100 is provided in an embodiment of the present application. The display device 100 includes a display module 110, a determination module 120, and a transmission module 130.
Step 01 may be implemented by the display module 110, step 02 may be implemented by the determination module 120, and step 03 may be implemented by the transmission module 130. Alternatively, the display module 110 may be configured to display at least one first application window in a first display area of the first display device and display a picture of at least one second display device in a second display area when the first display device enters the predetermined mode, where the first display device is connected to the second display device through a local area network. The determination module 120 may be configured to determine a target display device among the second display devices in response to a first operation of the first application window by the user. The transmission module 130 may be configured to send the application file and the operation log of the first application window to the target display device, so as to display the corresponding first application window on the target display device.
The application further provides an electronic device, which comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the display interaction method is realized, that is, the processor can be used for displaying at least one first application window in a first display area of the first display device and at least one picture of the second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network. The processor may also be configured to determine a target display device in the second display device in response to a first operation of the first application window by the user, and send an application file and an operation log of the first application window to the target display device to display the corresponding first application window on the target display device.
In the display interaction method, the display device and the electronic device in the embodiments, by connecting the first display device and the second display device in a predetermined mode, the first display device displays two areas of the first display area and the second display area, and displays the first application window in the first display area, the second display area displays the picture of the second display device, and the second display device can be selected as the target display device according to the first operation of the user in the first application window, and the application file and the operation log of the first application window are sent to the target display device, so that the corresponding first application window is displayed in the target display device, and the data content synchronization between different devices is realized.
In some embodiments, the first display device may be a root device/device, i.e., a hardware device, including a display screen, such as a cell phone, a tablet, and a conference kiosk. For example, in the present application, the first display device may be described by taking a conference integrated machine as an example, that is, the display interaction method and the display device 100 are applied to, but not limited to, the conference integrated machine. The display device 100 may be hardware or software preinstalled in the conference kiosk and may perform the display interaction method when running is started on the conference kiosk. For example, the display device 100 may be an underlying software code segment of a conference kiosk or part of an operating system.
The device that may provide a plurality of functional services in the first display device of the present embodiment may include a control service, an event service, a presentation service, and the like. Also, the first display device may include a display screen, and the first display area and the second display area may be displayed in the display screen. In addition, in order to be able to facilitate user operations, the display screen may include a touch screen, that is, the user may directly touch the display screen by hand to input corresponding operations.
The second display device may be a cell phone, tablet, personal computer, etc. The second display device includes a plurality of second display devices, and the plurality of second display devices may be connected to the first display device through a local area network such as an ethernet network, an FDDI network, or a wireless local area network.
It should be noted that, the predetermined mode may be a window communication mode, that is, when the first display device is connected to the second display device through the local area network and the first display device enters the window communication mode, the first display device displays at least one first application window in the first display area and displays a screen of at least one second display device in the second display area.
In step 02, the first operation of the first application window by the user may be, but is not limited to, a click, drag, zoom-in or zoom-out operation, etc., for example, the first operation may be a drag operation.
For example, referring to fig. 3, when the user drags the first application window in the first display area to the position corresponding to the second display device 1 in the first display area, the second display device 1 may be used as the target display device, and the first application window may be displayed in the second display device 1.
In step 03, it may be understood that when the first application window is opened in the first display device, the first display device may open or create an application file of the first application window and an operation log, where the application file may include, but is not limited to, an application cache, a document, and the like, and the operation log includes an operation record of the user in the first application window. Such as editing, modifying operations, etc.
The first display device may implement data transmission of an application file, an operation log, and the like through Socket technology, specifically, after the target display device is selected, the second display area may display a picture displayed by the target device, and may further send the application file and the operation log to the target display device through Socket technology in the first application window, so that the target display device may display the first application window. In this way, synchronization of the data content of the first display device and the target display device is achieved.
It should be noted that Socket refers to an abstraction of endpoints that perform bidirectional communication between application processes on different hosts in a network. One socket is the end of the network where processes communicate, providing a mechanism for application layer processes to exchange data using network protocols.
Referring to fig. 4, in some embodiments, the display interaction method further includes:
04, sending first picture data to the second display device through a screen projection protocol so as to display a picture of the first display device on the second display device, wherein the first picture data comprises a screen shot or a screen recording of the first display device.
Referring further to fig. 2, in some embodiments, step 04 may be implemented by the transmission module 130. Alternatively, the transmission module 130 may be configured to send the first screen data to the second display device through a screen-casting protocol, so as to display a screen of the first display device on the second display device, where the first screen data includes a screen shot or a screen recording of the first display device.
In some embodiments, the processor may be configured to send the first screen data to the second display device via a screen-casting protocol to display a screen of the first display device on the second display device, the first screen data including a screen shot or a screen recording of the first display device.
In this embodiment, the first display device and the second display device each display a screen in a mirror image mode.
Specifically, the screen projection protocol may include DLNA, miracast, an airlay protocol, and the like, and in this embodiment, the screen projection protocol may be described as a DLNA protocol. Under the operation of a first user, the first display device can perform lossless compression on first picture data of the first display device through a DLNA protocol and transmit the first picture data to the target display device, and the target display device decompresses the compressed first picture data according to the DLNA protocol, so that the compressed first picture data are displayed in corresponding coordinates to display the first picture data of the first display device. Thus, the screen projection sharing between the first display device and the target display device is realized.
Those skilled in the art will appreciate that the DLNA protocol employs a universal plug and play (Universal Plug and Play, UPnP) device architecture, which is a set of network protocols that aim to enable various devices in a home network (data sharing, communication, and entertainment) and a corporate network to be seamlessly connected to each other and to simplify implementation of the associated network. The Upnp DA divides roles in the home network into three types: control points, devices, and services. Communication between them is achieved mainly by HTTP technology.
Referring to fig. 5, in some embodiments, before step 01, the interactive display method further includes:
001, searching for a second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation by a user in case the first display device enters a predetermined mode;
002, in response to the third operation of the searched second display device by the user, displaying the corresponding second display device in the second display area.
Referring further to fig. 6, in some embodiments, the display device 100 further includes a connection module 140, the step 001 may be implemented by the connection module 140, and the step 002 may be implemented by the display module 110. Alternatively, the connection module 140 may be configured to search for a second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation of the user in case that the first display device enters a predetermined mode. The display module 110 may be configured to display the corresponding second display device in the second display area in response to a third operation of the searched second display device by the user.
In some embodiments, the processor is configured to search for a second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation by the user in the event that the first display device enters the predetermined mode. The processor may be further configured to display the corresponding second display device in the second display area in response to a third operation of the searched second display device by the user.
Specifically, both the second operation and the third operation may be clicking operations, that is, if the first display device enters the predetermined mode and receives the clicking operation of the user, the first display device may search, through a screen-throwing protocol, for a second display device that is under the same local area network and that is connectable to the first display device, where the search results may be ordered according to the strength of network information, and when the user performs the clicking operation on the second display device in the search results, the second display device selected by the user is connected, and the first display device may display, in the second display area, a screen of the second display device selected by the user by clicking.
For example, referring to fig. 7, in some embodiments, the first display area includes a device list, the user clicks a "search" pop-up device selection pop-up window on the upper right corner of the left device list, searches other terminal devices (second display devices) under the same local area network as the currently operated device (first display device) through DLNA protocol, orders the search results according to the strength of network signals, and refreshes the search results once every 1 minute, at this time, connection operation can be performed on the searched terminal devices, the terminal device that establishes connection enters a "connected device" directory, and further, clicks "determine" to perform real-time display of pictures of the connected devices.
Referring to fig. 8, in some embodiments, step 002 includes:
0021, receiving second picture data sent by a second display device through a screen projection protocol, wherein the second picture data comprises a screen shot or a screen recording of the second display device;
0022, rendering the second picture data to correspondingly display a picture of the second display device in the second display area.
In some embodiments, steps 0021 and 0022 may be implemented by the display module 110, or the display module 110 may be configured to receive second screen data sent by a second display device via a projection protocol, the second screen data comprising a second display device screen shot or a screen recording. The display module 110 may be further configured to render the second screen data to correspondingly display a screen of the second display device in the second display area.
In some embodiments, the processor is configured to receive second screen data sent by the second display device via a projection protocol, the second screen data including a second display device screen shot or a screen recording. The processor may be further configured to render the second picture data to correspondingly display a picture of the second display device in the second display area.
In sub-step 0021, the second display device may transmit the screen capture protocol to the first display device at a screen capture or recording speed of 60 frames or more.
Therefore, the first display device can display the picture of the second display device, and the screen projection sharing among the interconnected devices is realized.
Referring to fig. 9, in some embodiments, the first application window includes a first control, and the device interaction method further includes:
and 05, responding to the first dragging operation of the user on the first control, and controlling the first application window to move along the track of the first dragging operation.
Referring to fig. 6, in some embodiments, the display device 100 further includes a control module 150, where step 05 may be implemented by the control module 150, or the control module 150 may be configured to control the first application window to move along the track of the first drag operation in response to the first drag operation of the first control by the user.
In some implementations, the processor may be configured to control movement of the first application window in a trajectory of the first drag operation in response to a first drag operation of the first control by the user.
It should be noted that the controls generally include, but are not limited to, the following information: element identification, element type, action type of the element, phonetic description of the element, etc. Wherein the element identity is unique for each element, by which the element can be found. Element types may include groups, text, images, and the like. The action types of the elements may include clicking, dragging, etc. In the application, the first control is used for moving the first application window according to the track of the first drag operation under the first drag operation, so that the first application window moves to a zone where the track is terminated. Thus, the first application window can be dragged to any position.
Further, if the first drag operation of the user on the first control is responded, the first application window is controlled to move to the second display area according to the track of the first drag operation, and the corresponding second display device is determined to be the target display device under the condition that the first application window is overlapped with the picture of the second display device, and if the first application window is not overlapped with the picture of the second display device, the first application window is still located in the corresponding area.
Referring to fig. 10, for example, in some examples, a first control is disposed on top of a first application window. If the user clicks the screen directly by hand or clicks the first control on the top of one of the first application windows by using the mouse, the first application window can be dragged, and the first application window can hover at the loose hand position after dragging.
Referring to fig. 11, in some embodiments, the display interaction method further includes:
06, responding to the second dragging operation of the second control by the user, and controlling the first application window to zoom according to the direction and the distance of the second dragging operation.
In some embodiments, step 06 may be implemented by the control module 150, or the control module 150 may be further configured to control the first application window to zoom in and out in accordance with the direction and distance of the second drag operation in response to the second drag operation of the second control by the user.
In some implementations, the processor is further operable to control the first application window to zoom in a direction and distance of the second drag operation in response to a second drag operation of the second control by the user.
Specifically, the second control is used for realizing the enlargement or the reduction of the first window according to the dragging action of the user. And dragging after the user selects the second control, and scaling the first application window according to the dragging direction and distance of the user. The dragging direction is used for determining the magnification or the reduction of the first application window, when the first application window is dragged towards the first application window, the first application window is reduced, when the first application window is dragged towards the direction away from the first application window, the first application window is magnified, the dragging distance is used for determining the magnification or the reduction multiple of the first application window, and the greater the dragging distance is, the greater the magnification or the reduction degree of the first application window is.
For example, in some examples, the second control is located in the lower right corner of the first application window, and when the user single points to the lower right corner second control and drags, the function of zooming in and out of the first application window is implemented, and the window of the first application is zoomed in or out to a position where the drag stops.
Referring to fig. 12, in some embodiments, step 06 includes:
061, displaying the first predicted display range of the scaled first application window according to the direction and distance of the second dragging operation in the process of the second dragging operation of the user;
062, after the second dragging operation is completed, rendering according to the first prediction display range, so as to display the scaled first application window in the first prediction display range.
In some embodiments, substeps 061 and 062 may be implemented by the control module 150, or the control module 150 may be further configured to display, during the second drag operation performed by the user, the first predicted display range of the scaled first application window according to the direction and distance of the second drag operation; and rendering according to the first predicted display range after the second dragging operation is completed, so as to display the scaled first application window in the first predicted display range.
In some embodiments, the processor may be configured to display the first predicted display range of the scaled first application window according to the direction and distance of the second drag operation during the second drag operation by the user; and rendering according to the first predicted display range after the second dragging operation is completed, so as to display the scaled first application window in the first predicted display range.
Referring to fig. 13, the first predicted display range may be displayed in a wire frame manner, that is, when the user performs the second drag operation to zoom the first application window, the zoomed position of the first application window is displayed in a wire frame manner, so as to prompt the user of the size of the drag, and the first application window remains in the original state at this time during the drag. And rendering in the line width range after the second dragging operation is completed, so that the scaled first application window is displayed in the line width range.
Referring further to fig. 12, in some embodiments, step 06 further comprises:
063, displaying the first application window on the first display device in a full screen manner under the condition that the first application window is enlarged to a first display proportion; or (b)
064, closing the first application window on the first display device under the condition that the first application window is reduced to the second display proportion.
In some embodiments, step 063 or step 064 may be implemented by the control module 150, or the control module 150 may be further configured to display the first application window on the first display device in full screen when the first application window is enlarged to the first display scale, and close the first application window on the first display device when the first application window is reduced to the second display scale.
In some embodiments, the processor is configured to display the first application window full screen on the first display device if the first application window is scaled up to the first display scale, and close the first application window on the first display device if the first application window is scaled down to the second display scale.
Specifically, the specific numerical values of the first display proportion and the second display proportion are not limited, and the first display proportion may be larger than the second display proportion, for example, the first display proportion may be eighty five percent of the full screen display size of the current first display device, and the second display proportion may be twenty five percent of the full screen display size of the current first display device, that is, in the case that the first application window is enlarged to eighty five percent of the full screen display size of the current first display device, the first application window is displayed in the full screen of the first display device, and in the case that the first application window is reduced to twenty five percent of the full screen display size of the current first display device, the first application window is closed in the first display device.
Referring to fig. 14, in some embodiments, the display interaction method further includes:
and 07, after the application file and the operation log of the first application window are sent to the target display device, closing the first application window at the first display device.
In some embodiments, step 07 may be implemented by the control module 150, or the control module 150 may be configured to close the first application window on the first display device after sending the application file and the operation log of the first application window to the target display device.
In some embodiments, the processor may be configured to close the first application window at the first display device after sending the application file and the operation log of the first application window to the target display device.
Thus, the first application window is prevented from being simultaneously controlled by the first display device and the second display device.
Referring to fig. 15, in some embodiments, the screen of the second display device includes a second application window, and the display interaction method further includes:
08, responding to the fourth operation of the user on the second application window, and sending a request instruction for the second application window to the corresponding second display device;
09, receiving an application file and an operation log of a second application window, which are correspondingly sent by the second display device according to the request instruction;
and 10, displaying the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, step 08-10 may be implemented by the transmission module 130, step 16 may be implemented by the display module 110, or in other words, the transmission module 130 may be configured to, in response to a fourth operation of the second application window by the user, send a request instruction for the second application window to the corresponding second display device, and receive an application file and an operation log of the second application window that are correspondingly sent by the second display device according to the request instruction; the display module 110 may be further configured to display the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, the processor may be configured to send a request instruction for the second application window to the corresponding second display device in response to a fourth operation of the second application window by the user, and receive an application file and an operation log of the second application window that are correspondingly sent by the second display device according to the request instruction, and may be further configured to display the second application window in the second display area according to the application file and the operation log of the second application window.
Therefore, the second application window of the second display device can be transferred to the first display device for display, and the diversity of interaction modes is increased.
In addition, in some embodiments, the second application window of the second display device may also be transferred to other second display devices in the second display area according to the user's operation.
Referring to fig. 16, in some embodiments, in response to a fourth operation of the second application window by the user, step 08 includes:
081, in response to a fourth drag operation of the user moving the second application window to the second display area, sending a request instruction for the second application window to the corresponding second display device.
The step 10 comprises the following steps:
101, determining a second predicted display range of a second application window in a second display area according to a fourth dragging operation;
102, rendering according to the second prediction display range to display the second application window in the prediction display range.
In some embodiments, sub-step 081 may be implemented by the transmission module 130, and sub-steps 101 and 102 may be implemented by the display module 110. Alternatively, the transmission module 130 may be further configured to send a request instruction for the second application window to the corresponding second display device in response to a fourth drag operation of the user moving the second application window to the second display area. The display module 110 may further determine a second predicted display range of the second application window in the second display area according to the fourth drag operation, and render according to the second predicted display range to display the second application window in the predicted display range.
In some embodiments, the processor may be configured to, in response to a fourth drag operation by the user to move the second application window to the second display area, send a request instruction for the second application window to the corresponding second display device, determine, according to the fourth drag operation, a second predicted display range of the second application window in the second display area, and render according to the second predicted display range to display the second application window in the predicted display range.
The second predicted display range may be displayed in a wire frame manner. For example, please refer to fig. 17, that is, when the user drags the second application window from the second display device 2 in the second display area to the blank area of the first display area through the fourth drag operation, the position of the second application window in the second display area is first indicated in a line frame manner, and during the drag process, the first application window remains in the original state. And further, after the second dragging operation is completed, rendering is performed within the line width range, so that a second application window displayed within the line width range is displayed.
The present application provides a non-transitory computer readable storage medium containing a computer program which, when executed by a processor, causes the processor to perform the above-described display interaction method.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any other combination. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present application, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), or the like.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
In addition, each functional module in each embodiment of the present application may be integrated into one merging module, or each module may exist alone physically, or two or more modules may be integrated into one module.
The foregoing is merely specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily think about changes or substitutions within the technical scope of the present application, and the changes and substitutions are intended to be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (14)

1. A display interaction method for a first display device, the display interaction method comprising:
displaying at least one first application window in a first display area of the first display device and displaying a picture of at least one second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
determining a target display device in the second display device in response to a first operation of the first application window by a user;
Transmitting the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment;
the display interaction method comprises the following steps:
searching for the second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation of a user in case that the first display device enters a predetermined mode;
responding to third operation of the searched second display device by a user, and displaying the corresponding second display device in the second display area;
the displaying the corresponding second display device in the second display area includes:
receiving second picture data sent by the second display equipment through a screen projection protocol, wherein the second picture data comprises screen capturing or screen recording of the second display equipment;
and rendering the second picture data to correspondingly display the picture of the second display device in the second display area.
2. The display interaction method according to claim 1, further comprising:
and sending first picture data to the second display device through a screen throwing protocol so as to display a picture of the first display device on the second display device, wherein the first picture data comprises screen capturing or screen recording of the first display device.
3. The display interaction method of claim 1, wherein the first application window comprises a first control, the display interaction method further comprising:
and responding to a first dragging operation of the user on the first control, and controlling the first application window to move according to the track of the first dragging operation.
4. A display interaction method according to claim 3, wherein said determining a target display device in said second display device in response to a first operation of said first application window by a user comprises:
and responding to a first dragging operation of the user on the first control, and determining the corresponding second display device as the target display device under the condition that the first application window moves to the second display area and the first application window coincides with the second display device.
5. The display interaction method of claim 1, wherein the first application window comprises a second control, the display interaction method comprising:
and responding to a second dragging operation of the user on the second control, and controlling the first application window to zoom according to the direction and the distance of the second dragging operation.
6. The display interaction method according to claim 5, further comprising:
displaying a first predicted display range of the scaled first application window according to the direction and distance of the second dragging operation in the process of the second dragging operation by the user;
and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
7. The display interaction method according to claim 5, further comprising:
displaying the first application window in a full screen mode on the first display device under the condition that the first application window is enlarged to a first display proportion; or (b)
And closing the first application window on the first display device under the condition that the first application window is reduced to a second display proportion.
8. The display interaction method according to claim 1, further comprising:
and after the application file and the operation log of the first application window are sent to the target display device, closing the first application window on the first display device.
9. The display interaction method according to claim 1, wherein the screen of the second display device includes a second application window, the display interaction method further comprising:
responding to a fourth operation of a user on the second application window, and sending a request instruction for the second application window to the corresponding second display equipment;
receiving an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction;
and displaying the second application window in the second display area according to the application file and the operation log of the second application window.
10. The display interaction method according to claim 9, wherein the sending, in response to a fourth operation of the second application window by the user, a request instruction for the second application window to the corresponding second display device includes:
and responding to a fourth dragging operation of the user to move the second application window to the second display area, and sending a request instruction for the second application window to the corresponding second display equipment.
11. The method of claim 10, wherein displaying the second application window in the second display area comprises:
Determining a second predicted display range of the second application window in the second display area according to the fourth dragging operation;
rendering is carried out according to the second prediction display range so as to display the second application window in the prediction display range.
12. A display device for implementing the display interaction method of any one of claims 1-11, comprising:
the display module is used for displaying at least one first application window in a first display area of the first display device and displaying at least one picture of the second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
a determining module, configured to determine a target display device in the second display device in response to a first operation of the first application window by a user;
and the transmission module is used for sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
13. An electronic device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to perform the display interaction method of any of claims 1-11.
14. A non-transitory computer readable storage medium containing a computer program, characterized in that the computer program, when executed by a processor, causes the processor to perform the display interaction method of any of claims 1-11.
CN202111654737.1A 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium Active CN114510205B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111654737.1A CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111654737.1A CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Publications (2)

Publication Number Publication Date
CN114510205A CN114510205A (en) 2022-05-17
CN114510205B true CN114510205B (en) 2024-03-15

Family

ID=81547582

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111654737.1A Active CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN114510205B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115884444A (en) * 2022-10-27 2023-03-31 张春帅 Method for establishing instant wireless communication interaction network by multiple intelligent devices

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140034100A (en) * 2012-09-10 2014-03-19 삼성전자주식회사 Operating method associated with connected electronic device with external display device and electronic device supporting the same
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信（杭州）有限公司 Shooting method and electronic equipment
WO2020206655A1 (en) * 2019-04-11 2020-10-15 深圳市柔宇科技有限公司 Interaction method, flexible electronic device, and readable storage medium
CN111813302A (en) * 2020-06-08 2020-10-23 广州视源电子科技股份有限公司 Screen projection display method and device, terminal equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3842941A1 (en) * 2019-12-25 2021-06-30 Shanghai Transsion Co., Ltd. A sidebar interaction method, device, and computer-readable storage medium

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140034100A (en) * 2012-09-10 2014-03-19 삼성전자주식회사 Operating method associated with connected electronic device with external display device and electronic device supporting the same
WO2020206655A1 (en) * 2019-04-11 2020-10-15 深圳市柔宇科技有限公司 Interaction method, flexible electronic device, and readable storage medium
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信（杭州）有限公司 Shooting method and electronic equipment
CN111813302A (en) * 2020-06-08 2020-10-23 广州视源电子科技股份有限公司 Screen projection display method and device, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN114510205A (en) 2022-05-17

Similar Documents

Publication Publication Date Title
EP2962478B1 (en) System and method for multi-user control and media streaming to a shared display
JP3787404B2 (en) Camera control system and control method thereof
US9569159B2 (en) Apparatus, systems and methods for presenting displayed image information of a mobile media device on a large display and control of the mobile media device therefrom
WO2021052255A1 (en) Method and apparatus for controlling hotspot recommendation pop-up window, and medium and electronic device
WO2023274354A1 (en) Video sharing method and apparatus, device, and medium
US11729355B2 (en) Video conferencing during real time collaboration on a virtual canvas
CN111049868B (en) Data interaction system and data interaction method
US20120042265A1 (en) Information Processing Device, Information Processing Method, Computer Program, and Content Display System
US11070894B2 (en) Methods, systems, and media for presenting interactive elements within video content
US20240062443A1 (en) Video sharing method and apparatus, device, and medium
CN110618780A (en) Interaction device and interaction method for interacting multiple signal sources
WO2018184443A1 (en) Terminal screen display processing method and apparatus and storage medium
CN103037243A (en) Method and system for switching network videos among multiple intelligent terminals and intelligent terminals
WO2018184442A1 (en) Terminal control method and device
US10795563B2 (en) Visualization of a network map using carousels
WO2024022306A1 (en) Interaction method and apparatus, and electronic device and computer-readable medium
CN114510205B (en) Display interaction method, display device, electronic device and storage medium
CN105874807A (en) Methods, systems, and media for remote rendering of web content on a television device
CN114327327A (en) Display equipment and control method for multi-equipment screen projection on same screen display
CN110647286A (en) Screen element control method, device, equipment and storage medium
US6483878B1 (en) Video communication system, video communication processing method and storage medium thereof
CN100410849C (en) Information processing device for setting background image, display method and program thereof
WO2024046386A1 (en) Live-streaming room visiting method and apparatus, and device and medium
CN113766305A (en) Display device and mirror image screen projection audio output control method
CN103561239A (en) On-spot work video monitoring method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant