CN114510205A - Display interaction method, display device, electronic device and storage medium - Google Patents

Display interaction method, display device, electronic device and storage medium Download PDF

Info

Publication number
CN114510205A
CN114510205A CN202111654737.1A CN202111654737A CN114510205A CN 114510205 A CN114510205 A CN 114510205A CN 202111654737 A CN202111654737 A CN 202111654737A CN 114510205 A CN114510205 A CN 114510205A
Authority
CN
China
Prior art keywords
display
application window
display device
interaction method
user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202111654737.1A
Other languages
Chinese (zh)
Other versions
CN114510205B (en
Inventor
郭琛
聂超
赵丽菲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BOE Technology Group Co Ltd
Original Assignee
BOE Technology Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BOE Technology Group Co Ltd filed Critical BOE Technology Group Co Ltd
Priority to CN202111654737.1A priority Critical patent/CN114510205B/en
Publication of CN114510205A publication Critical patent/CN114510205A/en
Application granted granted Critical
Publication of CN114510205B publication Critical patent/CN114510205B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/0486—Drag-and-drop
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

The application discloses a display interaction method, a display interaction device, an electronic device and a storage medium. The method comprises the following steps: under the condition that a first display device enters a preset mode, at least one first application window is displayed in a first display area of the first display device, and at least one picture of a second display device is displayed in a second display area, wherein the first display device is connected with the second display device through a local area network, a target display device is determined in the second display device in response to first operation of a user on the first application window, and an application file and an operation log of the first application window are sent to the target display device, so that the corresponding first application window is displayed on the target display device. According to the method and the device, the two display areas are arranged on the first display device, and the second display device can be controlled according to the application window displayed by one of the display areas, so that data transmission and content synchronization on different display devices are realized.

Description

Display interaction method, display device, electronic device and storage medium
Technical Field
The present application relates to the field of data communications, and in particular, to a display interaction method, a display device, an electronic device, and a storage medium.
Background
In the related technology, different devices can be connected to the same local area network, one device is a receiving end, the other devices are transmitting ends, and a communication mechanism is established by clicking a screen-throwing key in different devices, so that the problem of interconnection among different devices is solved. Therefore, the image of the current equipment can be transmitted to the receiving end in real time after being captured, the mirror image screen projection effect is achieved, or when the equipment image of the touch transmitting end on the receiving end equipment transmits coordinate point data back to the transmitting end, the reverse touch effect is achieved.
However, the above-mentioned device interconnection merely realizes the basic picture synchronization by a method of capturing a screen and transmitting a picture, or achieves the effect of touching a large screen with a small screen by transmitting coordinate point information of touch, and it is difficult to truly realize data transmission and content synchronization between devices.
Disclosure of Invention
The embodiment of the application provides a display interaction method, which is used for a first display device, and the display interaction method comprises the following steps:
under the condition that the first display equipment enters a preset mode, displaying at least one first application window in a first display area of the first display equipment and displaying at least one picture of second display equipment in a second display area, wherein the first display equipment is connected with the second display equipment through a local area network;
determining a target display device in the second display device in response to a first operation of the first application window by a user;
and sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
In some embodiments, the display interaction method comprises:
first picture data sent by a screen projection protocol are sent to the second display device so as to display a picture of the first display device on the second display device, wherein the first picture data comprise a screen capture or a screen recording of the first display device.
In some embodiments, the display interaction method further comprises:
searching for the second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation of a user in a case where the first display device enters a predetermined mode;
responding to a third operation of the user on the searched second display equipment, and displaying the corresponding second display equipment in the second display area.
In some embodiments, said displaying a corresponding said second display device in said second display area comprises:
receiving second picture data sent by the second display device through a screen projection protocol, wherein the second picture data comprises a screen shot or a screen recording of the second display device;
rendering the second picture data to correspondingly display the picture of the second display device in the second display area.
In some embodiments, the first application window includes a first control, and the display interaction method includes:
and responding to a first dragging operation of a user on the first control, and controlling the first application window to move according to the track of the first dragging operation.
In some embodiments, the determining a target display device in the second display device in response to the first operation of the first application window by the user comprises:
and responding to a first dragging operation of a user on the first control, and determining that the corresponding second display equipment is the target display equipment under the condition that the first application window is moved to the second display area and the first application window is overlapped with the second display equipment.
In some embodiments, the first application window includes a second control, and the display interaction method includes:
and responding to a second dragging operation of the user on the second control, and controlling the first application window to be zoomed according to the direction and the distance of the second dragging operation.
In some embodiments, the display interaction method further comprises:
displaying a first prediction display range of the zoomed first application window according to the direction and the distance of the second dragging operation in the process of the second dragging operation by the user;
and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
In some embodiments, the display interaction method comprises:
under the condition that the first application window is enlarged to a first display scale, displaying the first application window on the first display device in a full screen mode; or
And closing the first application window on the first display equipment under the condition that the first application window is reduced to a second display scale.
In some embodiments, the display interaction method further comprises:
and after the application file and the operation log of the first application window are sent to the target display equipment, closing the first application window on the first display equipment.
In some embodiments, the screen of the second display device includes a second application window, and the display interaction method includes:
responding to a fourth operation of the user on the second application window, and sending a request instruction for the second application window to the corresponding second display device;
receiving an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction;
and displaying the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, the sending, to the corresponding second display device, a request instruction for the second application window in response to a fourth operation on the second application window by the user includes:
and responding to a fourth dragging operation of the user for moving the second application window to the second display area, and sending a request instruction for the second application window to the corresponding second display equipment.
In some embodiments, said displaying said second application window in said second display area comprises:
determining a second predicted display range of the second application window in the second display area according to the fourth dragging operation;
rendering according to the second prediction display range to display the second application window in the prediction display range.
An embodiment of the present application further provides a display device, including:
the display module is used for displaying at least one first application window in a first display area of the first display device and displaying at least one picture of second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
the determining module is used for responding to a first operation of a user on the first application window to determine a target display device in the second display device;
and the transmission module is used for sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
The application also provides an electronic device, which comprises a memory and a processor, wherein the memory stores a computer program, and the computer program is executed by the processor to realize the display interaction method.
The present application also provides a non-transitory computer-readable storage medium containing a computer program which, when executed by a processor, causes the processor to perform the display interaction method.
In the display interaction method, the display device, the electronic device and the computer-readable storage medium according to the embodiments of the present application, in the predetermined mode, the first display device is connected to the second display device, the first display device displays two areas, namely, the first display area and the second display area, and displays the first application window in the first display area, and the second display area displays the screen of the second display device, and the second display device can be selected as the target display device according to the first operation of the user in the first application window, and the application file and the operation log of the first application window are sent to the target display device, so that the corresponding first application window is displayed on the target display device, and thus, data content synchronization between different devices is achieved.
Drawings
The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
FIG. 1 is a flow diagram illustrating a method of displaying interactions, in accordance with certain implementations of the present application.
FIG. 2 is a block diagram of a display device according to some embodiments of the present application.
FIG. 3 is a schematic diagram of a scenario illustrating an interaction method according to some embodiments of the present application.
Fig. 4-5 are flow diagrams illustrating a method of displaying interactions, in accordance with certain implementations of the present application.
FIG. 6 is a schematic diagram of another module of a display device according to some embodiments of the present application.
FIG. 7 is a schematic diagram of a scenario illustrating an interaction method according to some embodiments of the present application.
FIGS. 8-9 show flow diagrams of interaction methods according to some embodiments of the present application.
FIG. 10 is a schematic diagram of a scenario illustrating an interaction method according to some embodiments of the present application.
FIGS. 11-12 are flow diagrams illustrating a method of displaying interactions, in accordance with certain implementations of the present application.
FIG. 13 is a schematic diagram of a scenario illustrating an interaction method according to some embodiments of the present application.
Fig. 14-16 are flow diagrams illustrating a method of displaying interactions, in accordance with certain implementations of the present application.
FIG. 17 is a schematic diagram of a scenario illustrating an interaction method according to some embodiments of the present application.
Detailed Description
Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative and are only for the purpose of explaining the present application and are not to be construed as limiting the present application.
Referring to fig. 1, an embodiment of the present application provides a display interaction method for a first display device, where the display interaction method includes:
01, under the condition that a first display device enters a preset mode, displaying at least one first application window in a first display area of the first display device and displaying at least one picture of a second display device in a second display area, wherein the first display device is connected with the second display device through a local area network;
02, determining a target display device in the second display device in response to a first operation of the first application window by a user;
and 03, sending the application file and the operation log of the first application window to the target display device so as to display the corresponding first application window on the target display device.
Referring further to fig. 2, the present embodiment provides a display device 100. The display apparatus 100 includes a display module 110, a determination module 120, and a transmission module 130.
Step 01 may be implemented by the display module 110, step 02 may be implemented by the determination module 120, and step 03 may be implemented by the transmission module 130. Or, the display module 110 may be configured to display at least one first application window in a first display area of the first display device and display at least one screen of the second display device in a second display area of the first display device when the first display device enters the predetermined mode, where the first display device is connected to the second display device through the local area network. The determination module 120 may be configured to determine a target display device in the second display device in response to a first operation of the first application window by the user. The transmission module 130 may be configured to transmit the application file and the operation log of the first application window to the target display device, so as to display the corresponding first application window on the target display device.
The application further provides an electronic device, which includes a processor and a memory, where the memory stores a computer program, and when the computer program is executed by the processor, the display interaction method is implemented, that is, the processor may be configured to display at least one first application window in a first display area of a first display device and at least one second application window in a second display area of the first display device, where the first display device is connected to the second display device through a local area network. The processor may be further configured to determine a target display device in the second display device in response to a first operation of the first application window by the user, and send the application file and the operation log of the first application window to the target display device to display the corresponding first application window on the target display device.
In the display interaction method, the display device and the electronic device according to the embodiments, in the predetermined mode, the first display device is connected to the second display device, the first display device displays two areas of the first display area and the second display area, the first application window is displayed in the first display area, the screen of the second display device is displayed in the second display area, and the second display device can be selected as the target display device according to the first operation of the user in the first application window, and the application file and the operation log of the first application window are sent to the target display device, so that the corresponding first application window is displayed on the target display device, and thus, the data content synchronization between different devices is realized.
In some embodiments, the first display device may be a root device/device, i.e., a hardware device, including a display screen, such as a mobile phone, a tablet, and a conference kiosk. For example, in the present application, the first display device may be described by taking a conference kiosk as an example, that is, the display interaction method and the display device 100 are applied to, but not limited to, a conference kiosk. The display device 100 may be hardware or software preinstalled in the conference kiosk and may perform the display interaction method when it is started up and run on the conference kiosk. For example, the display device 100 may be an underlying software code segment of a conference kiosk or part of an operating system.
The first display device of the present embodiment may be a device that provides a plurality of functional services, and may include a control service, an event service, a presentation service, and the like. Also, the first display device may include a display screen, and the first display region and the second display region may be displayed in the display screen. In addition, in order to facilitate the user operation, the display screen may include a touch screen, that is, the user may directly touch the display screen with a hand to input the corresponding operation.
The second display device may be a mobile phone, a tablet, a personal computer, etc. The second display devices may be connected to the first display device through a local area network such as an ethernet network, an FDDI network, or a wireless local area network.
It should be noted that the predetermined mode may be a window communication mode, that is, when the first display device is connected to the second display device through the local area network and the first display device enters the window communication mode, the first display device displays at least one first application window in the first display area and displays at least one screen of the second display device in the second display area.
In step 02, the first operation of the first application window by the user may be, but is not limited to, a click operation, a drag operation, a zoom-in or zoom-out operation, and the like.
For example, please refer to fig. 3, when the user drags the first application window to the position corresponding to the second display device 1 in the first display area by dragging the first application window in the first display area, the second display device 1 may be used as the target display device, and the first application window is displayed in the second display device 1.
In step 03, it is understood that when the first application window is opened in the first display device, the first display device may open or create an application file of the first application window and an operation log, where the application file may include, but is not limited to, an application cache, a document, and the like, and the operation log includes an operation record of the user in the first application window. Such as editing, modifying operations, etc.
The first display device can realize data transmission of application files, operation logs and the like through a Socket technology, specifically, after the target display device is selected, the second display area can display a picture displayed by the target device, and the application files and the operation logs can be sent to the target display device through the Socket technology in the first application window, so that the target display device can display the first application window. In this way, data content synchronization of the first display device and the target display device is achieved.
It should be noted that Socket refers to an abstraction of an endpoint that performs bidirectional communication between application processes on different hosts in a network. A socket is the end of a process's communication over a network and provides a mechanism for application layer processes to exchange data using a network protocol.
Referring to fig. 4, in some embodiments, the method for displaying interaction further includes:
and 04, sending the first picture data to the second display device through the screen projection protocol so as to display the picture of the first display device on the second display device, wherein the first picture data comprises a screen shot or a screen recording of the first display device.
Referring further to fig. 2, in some embodiments, step 04 may be implemented by the transmission module 130. Or, the transmission module 130 may be configured to send the first picture data to the second display device through the screen-casting protocol, so as to display the picture of the first display device on the second display device, where the first picture data includes a screen shot or a screen recording of the first display device.
In some embodiments, the processor may be configured to send the first screen data to the second display device via a screen-casting protocol to display the screen of the first display device on the second display device, where the first screen data includes a screen shot or a screen recording of the first display device.
In this embodiment, the first display device and the second display device both display screens in the mirror mode.
Specifically, the screen projection protocol may include DLNA, Miracast, Airplay protocol, and the like, and in this embodiment, the screen projection protocol may be described as a DLNA protocol. Under the operation of a first user, the first display device can perform lossless compression on first picture data of the first display device through a DLNA protocol and transmit the first picture data to the target display device, and the target display device decompresses the compressed first picture data according to the DLNA protocol so as to display the first picture data of the first display device in a corresponding coordinate. Thus, screen projection sharing between the first display device and the target display device is achieved.
As will be appreciated by those skilled in the art, the DLNA protocol employs a Universal Plug and Play (UPnP) device architecture, which is a set of network protocols aimed at enabling seamless connectivity between various devices in home networks (data sharing, communication and entertainment) and corporate networks, and simplifying the implementation of the associated networks. Upnp DA separates roles in the home network into three types: control points, devices and services. Communication between them is mainly achieved by HTTP technology.
Referring to fig. 5, in some embodiments, before step 01, the interactive display method further includes:
001, in response to a second operation by the user, searching for a second display device connected to the same local area network and connectable to the first display device through the screen-casting protocol, in case the first display device enters a predetermined mode;
002, in response to a third operation of the searched second display device by the user, displaying the corresponding second display device in the second display area.
Referring further to fig. 6, in some embodiments, the display device 100 further includes a connection module 140, and step 001 may be implemented by the connection module 140 and step 002 may be implemented by the display module 110. Alternatively, the connection module 140 may be configured to search for a second display device connected to the same local area network and connectable to the first display device through the screen-casting protocol in response to a second operation of the user when the first display device enters the predetermined mode. The display module 110 may be configured to display the corresponding second display device in the second display area in response to a third operation of the searched second display device by the user.
In some embodiments, the processor is configured to search for a second display device connected to the same local area network and connectable to the first display device through the screen-casting protocol in response to a second operation by the user if the first display device enters the predetermined mode. The processor may be further configured to display a corresponding second display device in the second display area in response to a third operation of the searched second display device by the user.
Specifically, the second operation and the third operation may be click operations, that is, in a case where the first display device enters the predetermined mode, if the click operation of the user is received, the first display device may search, through a screen-casting protocol, for a second display device that is in the same local area network and connectable to the first display device, where the search result may be sorted according to the strength of the network information, and after the user performs the click operation on the second display device in the search result, the user is connected to click the selected second display device, and the first display device may display, in the second display area, a picture of the second display device that the user clicks the selected second display device.
For example, referring to fig. 7, in some embodiments, the first display area includes a device list, the user clicks a "search" popup device at the top right corner of the left device list to select a popup window, searches for other terminal devices (second display devices) in the same local area network as a currently operated device (first display device) through a DLNA protocol, sorts the search results according to the strength of network signals, and refreshes the search results once every 1 minute, at this time, the searched terminal devices can be connected, the terminal device that has established connection enters a "connected device" directory, and further, clicks "determine" to perform real-time display of a screen of the connected device.
Referring to fig. 8, in some embodiments, step 002 includes:
0021 receiving second picture data sent by a second display device through a screen projection protocol, wherein the second picture data comprises a screen shot or a screen recording of the second display device;
0022 rendering the second picture data to correspondingly display a picture of the second display device in the second display area.
In some embodiments, steps 0021 and 0022 may be implemented by the display module 110, or the display module 110 may be configured to receive second picture data sent by the second display device through the screen projection protocol, where the second picture data includes a screen shot or a screen recording of the second display device. The display module 110 may be further configured to render the second screen data to correspondingly display a screen of the second display device in the second display area.
In some embodiments, the processor is configured to receive second picture data sent by the second display device via the screen projection protocol, where the second picture data includes a screen shot or a screen recording of the second display device. The processor may be further configured to render the second screen data to correspondingly display a screen of the second display device in the second display area.
In sub-step 0021, the second display device may transmit a screen-capture protocol to the first display device at a screen capture or recording speed of 60 frames or more.
Therefore, the first display device can display the picture of the second display device, and screen projection sharing among the interconnected devices is achieved.
Referring to fig. 9, in some embodiments, the first application window includes a first control, and the device interaction method further includes:
and 05, responding to a first dragging operation of the user on the first control, and controlling the first application window to move according to the track of the first dragging operation.
Referring to fig. 6, in some embodiments, the display device 100 further includes a control module 150, and step 05 may be implemented by the control module 150, or the control module 150 may be configured to control the first application window to move along a track of a first drag operation in response to the first drag operation of the first control by the user.
In some embodiments, the processor may be operative to control the first application window to move in a trajectory of a first drag operation in response to the first drag operation of the first control by the user.
It should be noted that the control generally includes, but is not limited to, the following information: an element identification, an element type, an action type of the element, a phonetic utterance of the element, and the like. Wherein the element identification is unique for each element by which the element can be found. The element types may include groups, text, images, and the like. The action type of an element may include clicking, dragging, and the like. In the application, the first control is used for moving the first application window to a region where the track is terminated by pressing the track of the first drag operation under the first drag operation. In this manner, the first application window may be dragged to an arbitrary location.
Further, if the first application window is controlled to move to the second display area according to the track of the first dragging operation in response to the first dragging operation of the first control by the user, and the corresponding second display device is determined to be the target display device under the condition that the pictures of the first application window and the second display device are overlapped, and the first application window and the second display device are still located in the corresponding area under the condition that the pictures of the first application window and the second display device are not overlapped.
Referring to fig. 10, for example, in some examples, a first control is disposed at the top of a first application window. If the user directly clicks the screen by hand or clicks the first control at the top of one of the first application windows by a mouse, the first application window can be dragged, and the first application window can be hovered at the position of the loose hand after dragging.
Referring to fig. 11, in some embodiments, the display interaction method further includes:
and 06, responding to a second dragging operation of the user on the second control, and controlling the first application window to zoom according to the direction and the distance of the second dragging operation.
In some embodiments, step 06 may be implemented by the control module 150, or the control module 150 may be further configured to control the first application window to zoom in and out according to a direction and a distance of a second dragging operation of the second control.
In some embodiments, the processor may be further configured to control the first application window to zoom in and out of a direction and distance of a second drag operation in response to the second drag operation of the second control by the user.
Specifically, the second control is used for realizing the enlargement or reduction of the first window according to the dragging action of the user. And when the user selects the second control and drags the second control, the first application window is zoomed according to the dragging direction and distance of the user. The dragging direction is used for determining that the first application window is enlarged or reduced, when the first application window is dragged towards the first application window, the first application window is reduced, when the first application window is dragged towards the direction deviating from the first application window, the first application window is enlarged, the dragging distance is used for determining the times of enlargement or reduction of the first application window, and the larger the dragging distance is, the larger the degree of enlargement or reduction of the first application window is.
For example, in some examples, the second control is located in the lower right corner of the first application window, and when the user holds the second control in the lower right corner with a single pointer and drags, the function of zooming in and out the first application window is implemented, and the window of the first application is zoomed in or out to a position where dragging stops.
Referring to fig. 12, in some embodiments, step 06 includes:
061, in the process of the second dragging operation of the user, displaying a first prediction display range of the zoomed first application window according to the direction and the distance of the second dragging operation;
062, after the second drag operation is completed, rendering is performed according to the first predicted display range to display the scaled first application window in the first predicted display range.
In some embodiments, the sub-steps 061 and 062 may be implemented by the control module 150, or the control module 150 may be further configured to display the first predicted display range of the scaled first application window according to a direction and a distance of the second drag operation during the second drag operation performed by the user; and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
In some embodiments, the processor may be configured to display a first predicted display range of the scaled first application window according to a direction and a distance of the second drag operation during the second drag operation by the user; and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
Referring to fig. 13, the first prediction display range may be displayed in a wire frame manner, that is, when the user performs the second dragging operation to zoom the first application window, the zoomed position of the first application window is displayed in the wire frame manner, so as to prompt the user about the size of the dragging operation, and in the dragging process, the first application window is maintained in the original state at this time. And then, after the second dragging operation is completed, rendering is performed within the line width range, so that the scaled first application window is displayed within the line width range.
Referring further to fig. 12, in some embodiments, step 06 further includes:
063, displaying the first application window in full screen on the first display device under the condition that the first application window is enlarged to the first display scale; or
064, closing the first application window at the first display device in the case that the first application window is reduced to the second display scale.
In some embodiments, step 063 or step 064 may be implemented by the control module 150, or the control module 150 may be further configured to display the first application window in a full screen on the first display device if the first application window is enlarged to the first display scale, and to close the first application window on the first display device if the first application window is reduced to the second display scale.
In some embodiments, the processor is configured to display the first application window in full screen on the first display device if the first application window is enlarged to the first display scale, and to close the first application window on the first display device if the first application window is reduced to the second display scale.
Specifically, specific values of the first display ratio and the second display ratio are not limited, and the first display ratio may be greater than the second display ratio, for example, the first display ratio may be eighty-five percent of the full-screen display size of the current first display device, and the second display ratio may be twenty-five percent of the full-screen display size of the current first display device, that is, when the first application window is enlarged to eighty-five percent of the full-screen display size of the current first display device, the first application window is displayed on the full screen of the first display device, and when the first application window is reduced to twenty-five percent of the full-screen display size of the current first display device, the first application window is closed on the first display device.
Referring to fig. 14, in some embodiments, the method for displaying interaction further includes:
07, after sending the application file and the operation log of the first application window to the target display device, closing the first application window on the first display device.
In some embodiments, step 07 may be implemented by the control module 150, or the control module 150 may be configured to close the first application window on the first display device after sending the application file and the operation log of the first application window to the target display device.
In some embodiments, the processor may be configured to close the first application window at the first display device after sending the application file and the oplog for the first application window to the target display device.
Therefore, the first application window is prevented from being simultaneously controlled by the first display device and the second display device.
Referring to fig. 15, in some embodiments, the screen of the second display device includes a second application window, and the method for displaying interaction further includes:
08, responding to a fourth operation of the user on the second application window, and sending a request instruction for the second application window to the corresponding second display device;
09, receiving the application file and the operation log of the second application window correspondingly sent by the second display device according to the request instruction;
and 10, displaying the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, the steps 08 to 10 may be implemented by the transmission module 130, and the step 16 may be implemented by the display module 110, or in other words, the transmission module 130 may be configured to, in response to a fourth operation on the second application window by the user, send a request instruction for the second application window to the corresponding second display device, and receive an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction; the display module 110 may be further configured to display the second application window in the second display area according to the application file and the operation log of the second application window.
In some embodiments, the processor may be configured to, in response to a fourth operation on the second application window by the user, send a request instruction for the second application window to the corresponding second display device, and receive an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction, and the processor may be further configured to display the second application window in the second display area according to the application file and the operation log of the second application window.
Therefore, the second application window of the second display device can be transferred to the first display device to be displayed, and the diversity of interaction modes is increased.
In addition, in some embodiments, the second application window of the second display device may also be transferred to other second display devices in the second display area according to a user operation.
Referring to FIG. 16, in some embodiments, in response to a fourth operation by the user on the second application window, step 08 includes:
081, in response to a fourth drag operation in which the user moves the second application window to the second display area, a request instruction for the second application window is transmitted to the corresponding second display device.
The step 10 comprises:
101, determining a second predicted display range of the second application window in the second display area according to the fourth dragging operation;
and 102, rendering according to the second prediction display range to display a second application window in the prediction display range.
In some embodiments, sub-step 081 may be implemented by the transmission module 130 and sub-steps 101 and 102 may be implemented by the display module 110. In other words, the transmission module 130 may be further configured to send a request instruction for the second application window to the corresponding second display device in response to a fourth drag operation of the user moving the second application window to the second display area. The display module 110 may further determine a second predicted display range of the second application window in the second display area according to the fourth dragging operation, and render according to the second predicted display range to display the second application window in the predicted display range.
In some embodiments, the processor may be configured to send a request instruction for the second application window to the corresponding second display device in response to a fourth drag operation of the user moving the second application window to the second display area, and the processor may further determine a second predicted display range of the second application window in the second display area according to the fourth drag operation and perform rendering according to the second predicted display range to display the second application window in the predicted display range.
The second prediction display range may be displayed in a wire frame manner. For example, please refer to fig. 17, that is, when the user drags the second application window from the second display device 2 in the second display area to the blank area of the first display area through the fourth dragging operation, the position of the second application window in the second display area is first indicated in a line frame manner, and the first application window remains in the original state during the dragging process. And then, after the second drag operation is completed, rendering is performed within the line width range, thereby displaying the second application window within the line width range.
A non-transitory computer-readable storage medium containing a computer program which, when executed by a processor, causes the processor to perform the above-described display interaction method is provided.
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware or any other combination. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a Digital Video Disk (DVD)), or a semiconductor medium (e.g., a Solid State Disk (SSD)), among others.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
In addition, functional modules in the embodiments of the present application may be integrated into one combined module, or each module may exist alone physically, or two or more modules are integrated into one module.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (16)

1. A display interaction method for a first display device, the display interaction method comprising:
under the condition that the first display equipment enters a preset mode, displaying at least one first application window in a first display area of the first display equipment and displaying at least one picture of second display equipment in a second display area, wherein the first display equipment is connected with the second display equipment through a local area network;
determining a target display device in the second display device in response to a first operation of the first application window by a user;
and sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
2. The display interaction method of claim 1, further comprising:
and sending first picture data to the second display equipment through a screen projection protocol so as to display the picture of the first display equipment on the second display equipment, wherein the first picture data comprises a screen shot or a screen recording of the first display equipment.
3. The display interaction method of claim 1, wherein the display interaction method comprises:
searching for the second display device connected to the same local area network and connectable to the first display device through a screen-casting protocol in response to a second operation of a user in a case where the first display device enters a predetermined mode;
responding to a third operation of the user on the searched second display equipment, and displaying the corresponding second display equipment in the second display area.
4. The display interaction method of claim 3, wherein the displaying the corresponding second display device in the second display area comprises:
receiving second picture data sent by the second display device through a screen projection protocol, wherein the second picture data comprises screen capture or screen recording of the second display device;
rendering the second picture data to correspondingly display the picture of the second display device in the second display area.
5. The display interaction method of claim 1, wherein the first application window comprises a first control, the display interaction method further comprising:
and responding to a first dragging operation of a user on the first control, and controlling the first application window to move according to the track of the first dragging operation.
6. The display interaction method of claim 5, wherein the determining a target display device in the second display device in response to a first operation of the first application window by a user comprises:
and responding to a first dragging operation of a user on the first control, and determining that the corresponding second display equipment is the target display equipment under the condition that the first application window is moved to the second display area and the first application window is overlapped with the second display equipment.
7. The display interaction method of claim 1, wherein the first application window comprises a second control, the display interaction method comprising:
and responding to a second dragging operation of the user on the second control, and controlling the first application window to be zoomed according to the direction and the distance of the second dragging operation.
8. The display interaction method of claim 7, further comprising:
displaying a first prediction display range of the zoomed first application window according to the direction and the distance of the second dragging operation in the process of the second dragging operation by the user;
and after the second dragging operation is finished, rendering according to the first prediction display range so as to display the scaled first application window in the first prediction display range.
9. The display interaction method of claim 7, further comprising:
under the condition that the first application window is enlarged to a first display scale, displaying the first application window on the first display device in a full screen mode; or
And closing the first application window on the first display equipment under the condition that the first application window is reduced to a second display scale.
10. The display interaction method of claim 1, further comprising:
and after the application file and the operation log of the first application window are sent to the target display equipment, closing the first application window on the first display equipment.
11. The display interaction method of claim 1, wherein the screen of the second display device comprises a second application window, the display interaction method further comprising:
responding to a fourth operation of the user on the second application window, and sending a request instruction for the second application window to the corresponding second display device;
receiving an application file and an operation log of the second application window, which are correspondingly sent by the second display device according to the request instruction;
and displaying the second application window in the second display area according to the application file and the operation log of the second application window.
12. The display interaction method according to claim 11, wherein the sending, to the corresponding second display device, the request instruction for the second application window in response to the fourth operation of the second application window by the user comprises:
and responding to a fourth dragging operation of the user for moving the second application window to the second display area, and sending a request instruction for the second application window to the corresponding second display equipment.
13. The display interaction method of claim 12, wherein the displaying the second application window in the second display area comprises:
determining a second predicted display range of the second application window in the second display area according to the fourth dragging operation;
rendering according to the second prediction display range to display the second application window in the prediction display range.
14. A display device, comprising:
the display module is used for displaying at least one first application window in a first display area of the first display device and displaying at least one picture of second display device in a second display area under the condition that the first display device enters a preset mode, wherein the first display device is connected with the second display device through a local area network;
the determining module is used for responding to a first operation of a user on the first application window to determine a target display device in the second display device;
and the transmission module is used for sending the application file and the operation log of the first application window to the target display equipment so as to display the corresponding first application window on the target display equipment.
15. An electronic device comprising a memory and a processor, the memory having stored therein a computer program that, when executed by the processor, causes the processor to perform the display interaction method of any one of claims 1-13.
16. A non-transitory computer-readable storage medium containing a computer program, wherein the computer program, when executed by a processor, causes the processor to perform the display interaction method of any one of claims 1-13.
CN202111654737.1A 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium Active CN114510205B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111654737.1A CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111654737.1A CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Publications (2)

Publication Number Publication Date
CN114510205A true CN114510205A (en) 2022-05-17
CN114510205B CN114510205B (en) 2024-03-15

Family

ID=81547582

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111654737.1A Active CN114510205B (en) 2021-12-30 2021-12-30 Display interaction method, display device, electronic device and storage medium

Country Status (1)

Country Link
CN (1) CN114510205B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115834754A (en) * 2022-09-29 2023-03-21 歌尔科技有限公司 Interaction control method and device, head-mounted display equipment and medium
CN115884444A (en) * 2022-10-27 2023-03-31 张春帅 Method for establishing instant wireless communication interaction network by multiple intelligent devices

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140034100A (en) * 2012-09-10 2014-03-19 삼성전자주식회사 Operating method associated with connected electronic device with external display device and electronic device supporting the same
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信（杭州）有限公司 Shooting method and electronic equipment
WO2020206655A1 (en) * 2019-04-11 2020-10-15 深圳市柔宇科技有限公司 Interaction method, flexible electronic device, and readable storage medium
CN111813302A (en) * 2020-06-08 2020-10-23 广州视源电子科技股份有限公司 Screen projection display method and device, terminal equipment and storage medium
US20210200431A1 (en) * 2019-12-25 2021-07-01 Shanghai Transsion Co., Ltd. Sidebar Interaction Method, Device, and Computer-readable Storage Medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20140034100A (en) * 2012-09-10 2014-03-19 삼성전자주식회사 Operating method associated with connected electronic device with external display device and electronic device supporting the same
WO2020206655A1 (en) * 2019-04-11 2020-10-15 深圳市柔宇科技有限公司 Interaction method, flexible electronic device, and readable storage medium
US20210200431A1 (en) * 2019-12-25 2021-07-01 Shanghai Transsion Co., Ltd. Sidebar Interaction Method, Device, and Computer-readable Storage Medium
CN111083379A (en) * 2019-12-31 2020-04-28 维沃移动通信（杭州）有限公司 Shooting method and electronic equipment
CN111813302A (en) * 2020-06-08 2020-10-23 广州视源电子科技股份有限公司 Screen projection display method and device, terminal equipment and storage medium

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115834754A (en) * 2022-09-29 2023-03-21 歌尔科技有限公司 Interaction control method and device, head-mounted display equipment and medium
CN115884444A (en) * 2022-10-27 2023-03-31 张春帅 Method for establishing instant wireless communication interaction network by multiple intelligent devices

Also Published As

Publication number Publication date
CN114510205B (en) 2024-03-15

Similar Documents

Publication Publication Date Title
US9569159B2 (en) Apparatus, systems and methods for presenting displayed image information of a mobile media device on a large display and control of the mobile media device therefrom
US9723123B2 (en) Multi-screen control method and device supporting multiple window applications
CN114510205B (en) Display interaction method, display device, electronic device and storage medium
WO2023274354A1 (en) Video sharing method and apparatus, device, and medium
CN111049868B (en) Data interaction system and data interaction method
US20120042265A1 (en) Information Processing Device, Information Processing Method, Computer Program, and Content Display System
US20090006977A1 (en) Method and System of Computer Remote Control that Optimized for Low Bandwidth Network and Low Level Personal Communication Terminal Device
CN103037265A (en) Method and system for first terminal to control second terminal and intelligent terminal
CN103037243B (en) The method of handover network video, system and intelligent terminal between multiple intelligent terminal
US20240062443A1 (en) Video sharing method and apparatus, device, and medium
EP3090565B1 (en) Methods, systems, and media for remote rendering of web content on a television device
US20030009524A1 (en) System and method for point to point integration of personal computers with videoconferencing systems
CN103037244A (en) Method and system for switching network videos among multiple intelligent terminals and intelligent terminals
WO2024022306A1 (en) Interaction method and apparatus, and electronic device and computer-readable medium
WO2021218556A1 (en) Information display method and apparatus, and electronic device
US20180136809A1 (en) Visualization of a network map using carousels
WO2022111375A1 (en) Video sharing method and apparatus, electronic device, and storage medium
CN103037266A (en) Terminal video switching method, system and intelligent terminal based on proxy server
WO2024046386A1 (en) Live-streaming room visiting method and apparatus, and device and medium
JP5281324B2 (en) Screen output converter, display device, and screen display method
CN110928506B (en) Data interaction system and data interaction method
CN103561239A (en) On-spot work video monitoring method
US20080036695A1 (en) Image display device, image display method and computer readable medium
KR102067642B1 (en) Apparataus and method for providing videotelephony in a portable terminal
TWI649655B (en) Remote management method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant