CN117032874B - Remote control method, device, computing equipment and storage medium - Google Patents
Remote control method, device, computing equipment and storage medium Download PDFInfo
- Publication number
- CN117032874B CN117032874B CN202311293973.4A CN202311293973A CN117032874B CN 117032874 B CN117032874 B CN 117032874B CN 202311293973 A CN202311293973 A CN 202311293973A CN 117032874 B CN117032874 B CN 117032874B
- Authority
- CN
- China
- Prior art keywords
- communication protocol
- computing device
- function
- request
- control request
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 80
- 238000003860 storage Methods 0.000 title claims abstract description 30
- 238000004891 communication Methods 0.000 claims abstract description 87
- 238000004088 simulation Methods 0.000 claims abstract description 23
- 230000004044 response Effects 0.000 claims abstract description 20
- 238000005316 response function Methods 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims description 98
- 230000015654 memory Effects 0.000 claims description 18
- 238000005096 rolling process Methods 0.000 claims description 13
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000013519 translation Methods 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 8
- 238000012545 processing Methods 0.000 description 7
- 230000003068 static effect Effects 0.000 description 7
- 239000011800 void material Substances 0.000 description 7
- 230000008901 benefit Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000013461 design Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012905 input function Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003032 molecular docking Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000007723 transport mechanism Effects 0.000 description 2
- 230000009471 action Effects 0.000 description 1
- 238000004026 adhesive bonding Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 239000003292 glue Substances 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000007639 printing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a remote control method, a device, a computing device and a storage medium, wherein the remote control method is executed in a controlled computing device, and the controlled computing device displays information through a first communication protocol, and the method comprises the following steps: receiving a control request from a master computing device, the control request being generated based on a second communication protocol; acquiring a function pointer of a response function pointed by a control request; converting the pointing direction of the function pointer into an objective function, wherein the objective function and the response function have the same function and are generated based on a first communication protocol; generating a simulation request of the control request by using the function pointer after the address is converted; in response to the simulation request, a response result of the control request is simulated in the controlled computing device. The method replaces a control request based on a second communication protocol with a simulation request based on a first communication protocol, and enables a computing device using the first communication protocol to process a remote control request based on the second communication protocol.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a remote control method, a remote control device, a computing device, and a storage medium.
Background
The display server in Linux is a program, and the main task of the display server is to coordinate the graphical display of clients and the input and output of the client. The display server communicates with its clients through a display service protocol. Currently, the display technology stack is mainly divided into X11 and Wayland.
Currently, applications written in dependence on X11 occupy the vast majority of the market, and before these applications are migrated to the waiand platform, a "glue layer" is required to allow them to run normally in waiand for transitional purposes, for which an organization writes xwaiand modules, which can be understood to be an X11 server running on top of the waiand server, which allows the X client to run in waiand session.
Through the gluing function of XWayland, the previously X11-based written programs can run in Wayland, but when remote control is involved, since the input of the server is being taken over by Wayland Compositor, the remote control is not specifically implemented in XWayland, resulting in these X11-based written applications losing remote control capability.
Disclosure of Invention
The present invention has been made in view of the above problems, and it is an object of the present invention to provide a remote control method, apparatus, computing device and storage medium that overcomes or at least partially solves the above problems.
According to one aspect of the present invention, there is provided a remote control method performed in a controlled computing device, the controlled computing device displaying information through a first communication protocol, the method comprising: receiving a control request from a master computing device, the control request being generated based on a second communication protocol; acquiring a function pointer of a response function pointed by a control request; converting the pointing direction of the function pointer into an objective function, wherein the objective function and the response function have the same function and are generated based on a first communication protocol; generating a simulation request of the control request by using the function pointer after the address is converted; in response to the simulation request, a response result of the control request is simulated in the controlled computing device.
Optionally, in the remote control method according to the present invention, wherein a compatibility layer resides in the controlled computing device, the compatibility layer being adapted to run an application based on the second communication protocol on the computing device based on the first communication protocol; obtaining a function pointer of a response function to which a control request is directed, comprising: sending a control request to a compatible layer; the control request is parsed by the compatibility layer to obtain the function pointer.
Optionally, in the remote control method according to the present invention, wherein parsing the control request with the compatible layer to obtain the function pointer includes: determining a request type of a control request, wherein the request type indicates operation events, and the operation events comprise a mouse moving event, a mouse typing event and a keyboard typing event; based on the operational event, a function pointer is determined.
Optionally, in the remote control method according to the present invention, the parsing the control request with the compatible layer to obtain the function pointer further includes: and analyzing the control request to obtain the operation data corresponding to the operation event.
Optionally, in the remote control method according to the present invention, wherein generating the analog request of the control request using the function pointer after converting the pointing address includes: the function pointer indicating the operational event and the corresponding operational data are encapsulated using a first communication protocol to generate a simulation request.
Optionally, in the remote control method according to the present invention, in response to the simulation request, a response result of the control request is simulated in the controlled computing device, including: determining the objective function based on a function pointer contained in the simulation request; and operating the objective function by taking the operation data as a function parameter to obtain a response result.
Optionally, in the remote control method according to the present invention, the obtaining operation data corresponding to the operation event includes: if the mouse typing event is mouse wheel input, the rolling direction and the rolling circle number of the mouse wheel are obtained; based on the scroll direction and the number of scroll turns, the scroll increment of the mouse wheel is calculated as the operation data of the mouse typing event.
Optionally, in the remote control method according to the present invention, wherein: the compatibility layer is a layer that allows application modules written using the second communication protocol to be run in the context of the first communication protocol.
Optionally, in the remote control method according to the present invention, the first communication protocol is a waiand protocol, and the second communication protocol is an X11 protocol.
According to yet another aspect of the present invention, there is provided a remote control apparatus residing in a controlled computing device, the controlled computing device displaying information via a first communication protocol, the apparatus comprising: the receiving module is suitable for receiving a control request from the main control computing equipment, and the control request is generated based on a second communication protocol;
the acquisition module is suitable for acquiring a function pointer of a response function pointed by the control request; the conversion module is suitable for converting the pointing direction of the function pointer into an objective function, and the objective function and the response function have the same function and are generated based on a first communication protocol; the generation module is suitable for generating a simulation request of the control request by utilizing the function pointer after the conversion of the pointing address; and the response module is suitable for responding to the simulation request and simulating a response result of the control request in the controlled computing equipment.
According to yet another aspect of the present invention, there is provided a computing device comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the above-described method.
According to yet another aspect of the present invention, there is provided a readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the above-described method.
According to the scheme of the invention, a computing device using the first communication protocol can be enabled to process a remote control request based on the second communication protocol by generating an analog request based on the first communication protocol (Wayland) to replace a control request based on the second communication protocol (X11).
According to the scheme of the invention, the seamless compatibility of the remote control function energy under the first communication protocol environment based on the software which is developed under the second communication protocol and contains the remote function is realized by generating the association relation between the function pointer of the second communication protocol and the objective function of the first communication protocol, and the method has the advantages of small code modification quantity, strong universality, wide application range and the like, and greatly reduces the workload and the cost of compatibility adaptation of software manufacturers containing the remote control function under the first communication protocol.
The foregoing description is only an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more clearly understood and to make the same and other objects, features and advantages of the present invention more readily apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flow chart of a prior art method of responding to X11 remote control commands in a Watand environment;
FIG. 2 shows a schematic diagram of a computing device according to one embodiment of the invention;
FIG. 3 shows a flow chart of a remote control method according to one embodiment of the invention;
fig. 4 shows a schematic structural view of a remote control device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
X11 is a window system protocol originally developed in 1987 and widely used on Linux and other UNIX systems, which uses a traditional client-server model in which an application (client) sends requests to an X11 server, which processes the requests and displays the graphics of the application on a screen.
However, X11 has some problems such as difficult debugging, design insecurity, performance problems, and the like. For this reason, wayland, a modern window system protocol was proposed, first published in 2008, to replace X11.
X11 is a graphics system of a tablet, although the evolution is ongoing, due to the fact that the design is complex, a common X11 graphics environment at least comprises three roles of X Server, window Manager and X Client, frequent interaction is needed between the three roles, on the one hand, the graphics performance is affected, and on the other hand, the complexity of the structure greatly limits the capability of the graphics system to expand functions, so that the graphics system is difficult to iterate quickly. Meanwhile, for desktop environments, "XServer" is an uncontrollable role, and different desktop environments are difficult to realize the differentiation function, so that the development of the desktop environments is stopped. The Wayland is a new graphic environment which is created to solve the problems, is simpler and easier to expand, can be completely controlled by a desktop environment developer, and greatly improves the upper limit of the desktop environment.
Wayland uses a completely new design compared to X11, and uses simpler, more modern protocols and techniques to provide better performance and security, and better support for modern graphics hardware.
However, since many applications are still written based on the X11 protocol, in order to enable them to run in the waiand environment, xwaiand has been developed as a compatible layer, when an application written using the X11 protocol needs to be run, the waiand server starts the xwaiand process, and then displays the windows of these applications in the waiand window manager, so that the user can write the application using the X11 protocol in the waiand environment. However, when remote control is involved, since the input of the server is taken over by Wayland Compositor, the remote control is not specifically implemented in xwaiand, as a result, these X11 writing-based applications lose the remote control capability.
In the Wayland display protocol, the X application program wants to realize desktop control, and needs to interface with a remote control function for the Wayland display protocol, so that the remote control request of X is actually realized at the bottom layer in the Wayland server, as shown in FIG. 1, FIG. 1 shows a flow chart of a method for responding to the remote control instruction of X11 in the Wayland environment in the prior art.
Xapp (application implemented based on the second communication protocol) sends a remote control request based on the XTEST protocol (extended protocol based on the second communication) to XWayland (compatible layer) through XHOok (hook function), and XWayland sends the remote control request to Wayland Server (Server based on the first communication protocol) through Wayland fake-input protocol (extended protocol based on the first communication protocol), and the remote control request is uploaded to Libinput (input system) by Wayland Server, thereby implementing remote control.
The method can be seamlessly compatible with a complete X11 program remote control function, but because the XTEST protocol is uniform, the implementation modes of the Wayland-end remote control protocol are different, when the remote control is realized, the XWayland code is required to be modified according to the Wayland protocol contents of different platforms, and the universality is poor.
In order to solve the technical problems, the invention provides a remote control method. The method enables a computing device using a first communication protocol to process remote control requests based on a second communication protocol by generating analog requests based on the first communication protocol (waiand) in place of control requests based on the second communication protocol (X11). The method may be implemented as an application program installed in computing device 200.
The computing device 200 described above may be implemented as a server, such as an application server, web server, or the like; but not limited to, desktop computers, notebook computers, processor chips, tablet computers, and the like. FIG. 2 shows a schematic diagram of a computing device 200 according to one embodiment of the invention. In a basic configuration, computing device 200 includes at least one processing unit 202 and system memory 204. According to one aspect, depending on the configuration and type of computing device, system memory 204 includes, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories.
According to one aspect, the system memory 204 includes an operating system 205. The system memory 204 also includes program modules 250. According to one aspect, the operating system 205 is, for example, adapted to control the operation of the computing device 200. Further, examples are practiced in connection with a graphics library, other operating systems, or any other application program and are not limited to any particular application or system. This basic configuration is illustrated in fig. 2 by those components within dashed line 208. According to one aspect, computing device 200 has additional features or functionality. For example, according to one aspect, computing device 200 includes additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in fig. 2 by removable storage device 209 and non-removable storage device 210.
As set forth hereinabove, according to one aspect, program modules are stored in the system memory 204. According to one aspect, program modules may include one or more applications, the invention is not limited in the type of application, for example, the application may include: email and contacts applications, word processing applications, spreadsheet applications, database applications, slide show applications, drawing or computer-aided application, web browser applications, etc. In an embodiment according to the present invention, the application program in the program module may be a remote control apparatus 400, the remote control apparatus 400 being configured to perform the remote control method 300 of the present invention.
According to one aspect, the examples may be practiced in a circuit comprising discrete electronic components, a packaged or integrated electronic chip containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic components or a microprocessor. For example, examples may be practiced via a system on a chip (SOC) in which each or many of the components shown in fig. 2 may be integrated on a single integrated circuit. According to one aspect, such SOC devices may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all of which are integrated (or "burned") onto a chip substrate as a single integrated circuit. When operating via an SOC, the functionality described herein may be operated via dedicated logic integrated with other components of computing device 200 on a single integrated circuit (chip). Embodiments of the invention may also be practiced using other techniques capable of performing logical operations (e.g., AND, OR, AND NOT), including but NOT limited to mechanical, optical, fluidic, AND quantum techniques. In addition, embodiments of the invention may be practiced within a general purpose computer or in any other circuit or system.
According to one aspect, the computing device 200 may also have one or more input devices 212, such as a keyboard, mouse, pen, voice input device, touch input device, and the like. Output device(s) 214 such as a display, speakers, printer, etc. may also be included. The foregoing devices are examples and other devices may also be used. Computing device 200 may include one or more communication connections 216 that allow communication with other computing devices 218, which other computing devices 218 may be printing devices, such as printers. Examples of suitable communication connections 216 include, but are not limited to: RF transmitter, receiver and/or transceiver circuitry; universal Serial Bus (USB), parallel and/or serial ports.
The term computer readable media as used herein includes computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information (e.g., computer readable instructions, data structures, or program modules). System memory 204, removable storage 209, and non-removable storage 210 are all examples of computer storage media (i.e., memory storage). Computer storage media may include Random Access Memory (RAM), read Only Memory (ROM), electrically erasable read only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and that can be accessed by computer device 200. According to one aspect, any such computer storage media may be part of computing device 200. Computer storage media does not include a carrier wave or other propagated data signal.
According to one aspect, communication media is embodied by computer readable instructions, data structures, program modules, or other data in a modulated data signal (e.g., carrier wave or other transport mechanism) and includes any information delivery media. According to one aspect, the term "modulated data signal" describes a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio Frequency (RF), infrared, and other wireless media.
In an embodiment according to the invention, the computing device 200 is configured to execute an application remote control method 300 according to the invention. The computing device 200 includes one or more processors and one or more readable storage media storing program instructions that, when configured to be executed by the one or more processors, cause the computing device to perform the application remote control method 300 in embodiments of the invention.
According to one embodiment of the invention, the remote control 400 of the computing device 200 is configured to execute the application remote control method 300 according to the invention. Wherein the remote control apparatus 400 contains a plurality of program instructions for executing the application remote control method 300 of the present invention, which may instruct the processor to execute the application remote control method 300 according to the present invention, and the computing device 200 implements the processing of the remote control request based on the second communication protocol by the computing device using the first communication protocol by executing the application remote control method 300 of the present invention.
Fig. 3 shows a flow chart of a remote control method 300 according to an embodiment of the invention. The remote control method 300 is adapted to be executed in a controlled computing device (e.g., the aforementioned computing device 200) that displays information via a first communication protocol, which in this embodiment may be the Wayland protocol.
As shown in fig. 3, the purpose of the remote control method 300 is to provide a method that enables a computing device using a first communication protocol to process remote control requests based on a second communication protocol by generating analog requests based on the first communication protocol (waiand) instead of control requests based on the second communication protocol (X11).
The remote control method 300 begins at step 302, where a control request is received from a host computing device, the control request being generated based on a second communication protocol. A remote control application (hereinafter referred to as XApp) resides in the master computing device, through which a user may send remote control requests to the controlled computing device, the control requests typically being generated based on a second communication protocol (X11 protocol).
In some embodiments, the control request includes at least an operation event and operation data corresponding to the operation event. Specifically, the operation event is a mouse movement event, a mouse typing event, a keyboard typing event, etc., the operation data is specific data of the operation events, for example, for the mouse movement event, absolute displacement or relative displacement generated by the movement of the mouse from the point a to the point B of the display interface is the operation data of the mouse movement event. For a mouse typing event, left key typing, right key typing, scroll wheel scrolling direction, number of scrolling turns, etc., operating data for a mouse typing event, etc.
In one particular example, generation of pseudo code for a portion of a control request is shown.
#include<X11/extensions/XTest.h>
The// X application sends simulated keyboard press and release events via the X11 protocol (without having to operate the keyboard by hand, with code control keyboard)
XTestFakeKeyEvent(Display *dpy, unsigned int keycode,
Bool is_press, unsigned long delay)
{
XExtDisplayInfo *info = find_display (dpy);
register xXTestFakeInputReq *req;
XTestCheckExtension (dpy, info, 0);
LockDisplay(dpy);
GetReq(XTestFakeInput, req);
req->reqType = info->codes->major_opcode;
req->xtReqType = X_XTestFakeInput;
req->type = is_press ? KeyPress : KeyRelease;
req->detail = keycode;
req->time = delay;
UnlockDisplay(dpy);
SyncHandle();
return 1;
}
The// X application sends mouse button press and release event (left, right, middle) pieces (without clicking the mouse button by hand, with code control of the mouse button) via the X11 protocol
int
XTestFakeButtonEvent(Display *dpy, unsigned int button,
Bool is_press, unsigned long delay)
{
XExtDisplayInfo *info = find_display (dpy);
register xXTestFakeInputReq *req;
XTestCheckExtension (dpy, info, 0);
LockDisplay(dpy);
GetReq(XTestFakeInput, req);
req->reqType = info->codes->major_opcode;
req->xtReqType = X_XTestFakeInput;
req->type = is_press ? ButtonPress : ButtonRelease;
req->detail = button;
req->time = delay;
UnlockDisplay(dpy);
SyncHandle();
return 1;
}
The// X application sends mouse movement events via the X11 protocol (mouse movement need not be handled by hand, mouse movement is simulated with code, the effect is the same as for hand operation)
int
XTestFakeMotionEvent(Display *dpy, int screen, int x, int y, unsigned long delay)
{
XExtDisplayInfo *info = find_display (dpy);
register xXTestFakeInputReq *req;
XTestCheckExtension (dpy, info, 0);
LockDisplay(dpy);
GetReq(XTestFakeInput, req);
req->reqType = info->codes->major_opcode;
req->xtReqType = X_XTestFakeInput;
req->type = MotionNotify;
req->detail = False;
if (screen == -1)
req->root = None;
else
req->root = RootWindow(dpy, screen);
req->rootX = x;
req->rootY = y;
req->time = delay;
UnlockDisplay(dpy);
SyncHandle();
return 1;
}
{
XExtDisplayInfo *info = find_display (dpy);
register xXTestFakeInputReq *req;
XTestCheckExtension (dpy, info, 0);
LockDisplay(dpy);
GetReq(XTestFakeInput, req);
req->reqType = info->codes->major_opcode;
req->xtReqType = X_XTestFakeInput;
req->type = is_press ? ButtonPress : ButtonRelease;
req->detail = button;
req->time = delay;
UnlockDisplay(dpy);
SyncHandle();
return 1;
}
The// X application sends mouse movement events (without the need to manipulate mouse movements by hand, with code simulating mouse movements, with the same effect as hand operations) of specified offsets (X, y) relative to the current pointer position via the X11 protocol
int
XTestFakeRelativeMotionEvent(Display *dpy, int dx, int dy, unsigned long delay)
{
XExtDisplayInfo *info = find_display (dpy);
register xXTestFakeInputReq *req;
XTestCheckExtension (dpy, info, 0);
LockDisplay(dpy);
GetReq(XTestFakeInput, req);
req->reqType = info->codes->major_opcode;
req->xtReqType = X_XTestFakeInput;
req->type = MotionNotify;
req->detail = True;
req->root = None;
req->rootX = dx;
req->rootY = dy;
req->time = delay;
UnlockDisplay(dpy);
SyncHandle();
return 1;
}
After receiving the control request, the controlled computing device proceeds to step 304, where a function pointer of a response function to which the control request points is obtained. Each operation event corresponds to at least one response function, for example, a left mouse button typing event corresponds to a left mouse button response function; a mouse wheel entry event, a corresponding mouse wheel response function, etc.
It is easy to understand that, since the control request is generated based on the X11 protocol, the controlled computing device is in the waiand environment, and cannot process the control request, so the control request may be forwarded to the compatible layer (xwaiand) for processing, where the compatible layer may implement the operation of the application based on the second communication protocol on the computing device based on the first communication protocol, and no detailed description of the compatible layer is provided herein.
Specifically, first, a control request is sent to the compatible layer. The control request is then parsed by the compatibility layer to obtain the function pointer. The compatibility layer determines the request type of the control request, wherein the request type indicates operation events, and the operation events comprise a mouse moving event, a mouse typing event and a keyboard typing event. Based on the operational event, a function pointer is determined.
In other words, the XTEST Fakeinput request based on the first protocol is processed by the compatibility layer, and a series of function pointers such as FakeAxisInputPtr, fakeKeyBoardInputPtr, fakeButtonInputPtr, fakeMotionOffsetInputPtr, fakeMotionAbsoluteInputPtr of the scrieneptr called by the request are resolved.
It should be noted that, if the mouse typing event is input by the mouse wheel, the rolling direction and the number of rolling turns of the mouse wheel are obtained first, and based on the rolling direction and the number of rolling turns, the rolling increment of the mouse wheel is calculated and is used as the operation data of the mouse typing event.
In one particular example, a compatible layer is shown parsing a control request, resulting in partial pseudocode of a function pointer for a response function.
Processing requests from X applications (simulating keyboard input, mouse clicks, mouse movements, etc.) in/XWayland
ProcXTestFakeInput(ClientPtr client)
{
REQUEST(xXTestFakeInputReq);
int nev, n, type, rc;
xEvent *ev;
DeviceIntPtr dev = NULL;
........
switch (type) {
case KeyPress:
return BadDevice;
if (ev->u.u.detail<dev->key->xkbInfo->desc->min_key_code ||
ev->u.u.detail>dev->key->xkbInfo->desc->max_key_code) {
client->errorValue = ev->u.u.detail;
return BadValue;
}
need_ptr_update = 0;
break;
case MotionNotify:
if (!dev->valuator)
return BadDevice;
if (!(extension || ev->u.keyButtonPointer.root == None)) {
rc = dixLookupWindow(&root, ev->u.keyButtonPointer.root,
client, DixGetAttrAccess);
if (rc != Success)
return rc;
if (root->parent) {
client->errorValue = ev->u.keyButtonPointer.root;
return BadValue;
}
/* Add the root window's offset to the valuators */
if ((flags&POINTER_ABSOLUTE)&&firstValuator<= 1&&numValuators>0) {
if (firstValuator == 0)
valuators[0]+= root->drawable.pScreen->x;
if (firstValuator<2&&firstValuator + numValuators>1)
valuators[1 - firstValuator]+= root->drawable.pScreen->y;
}
}
if (ev->u.u.detail != xTrue&&ev->u.u.detail != xFalse) {
client->errorValue = ev->u.u.detail;
return BadValue;
}
/* FIXME: Xinerama! */
break;
case ButtonPress:
case ButtonRelease:
if (!dev->button)
return BadDevice;
if (!ev->u.u.detail || ev->u.u.detail>dev->button->numButtons) {
client->errorValue = ev->u.u.detail;
return BadValue;
}
break;
}
if (screenIsSaved == SCREEN_SAVER_ON)
dixSaveScreens(serverClient, SCREEN_SAVER_OFF, ScreenSaverReset);
int posx, posy;
ScreenPtr pScreen = screenInfo.screens[0];
miPointerGetPosition(inputInfo.pointer,&posx,&posy);
Type of/determine this request
switch (type) {
Mouse movement request
case MotionNotify:
valuator_mask_set_range(&mask, firstValuator, numValuators, valuators);
nevents = GetPointerEvents(xtest_evlist, dev, type, 0, flags,&mask);
If the current context (Session) is Wayland then call the FakeMotionAbsolute input function pointer ready to dock the mouse to move Wayland protocol
if (IsWaylandSession()) {
(*pScreen->FakeMotionAbsoluteInput)(pScreen, posx, posy);
}
break;
case ButtonPress:
If the current context (Session) is Wayland then call the FakeMotionAbsolute input function pointer, ready to dock the mouse button Wayland protocol
if (BottonMapProc(ev)>0 &&IsWaylandSession()) {
(*pScreen->FakeMotionAbsoluteInput)(pScreen, posx, posy);
(*pScreen->FakeButtonInput) (pScreen, 1, BottonMapProc(ev));
}
break;
case ButtonRelease:
valuator_mask_set_range(&mask, firstValuator, numValuators, valuators);
nevents = GetPointerEvents(xtest_evlist, dev, type, ev->u.u.detail,
flags,&mask);
if (BottonMapProc(ev)>0 &&IsWaylandSession()) {
(*pScreen->FakeMotionAbsoluteInput)(pScreen, posx, posy);
(*pScreen->FakeButtonInput) (pScreen, 0, BottonMapProc(ev));
}
Determining the direction of the current mouse wheel, calculating the increment of scrolling according to the direction
if ((ev->u.u.detail == 4 || ev->u.u.detail == 5)&&IsWaylandSession()) {
int test = ev->u.u.detail == 5 ? 4 : -4;
(*pScreen->FakeAxisInput)(pScreen, 0, test);
}
break;
case KeyPress:
if (IsWaylandSession()) {
(*pScreen->FakeKeyBoardInput)(pScreen, 1, KeyboardMapProc(ev));
}
break;
case KeyRelease:
nevents =
GetKeyboardEvents(xtest_evlist, dev, type, ev->u.u.detail);
if (IsWaylandSession()) {
(*pScreen->FakeKeyBoardInput)(pScreen, 0, KeyboardMapProc(ev));
}
break;
}
if (!IsWaylandSession()) {
for (i = 0; i<nevents; i++)
mieqProcessDeviceEvent(dev,&xtest_evlist[i],miPointerGetScreen(inputInfo.pointer));
}
if (need_ptr_update)
miPointerUpdateSprite(dev);
return Success;
}
After the corresponding function pointer is obtained, step 306 is entered, the pointing direction of the function pointer is converted into an objective function, and the objective function and the response function have the same function and are generated based on the first communication protocol.
It will be readily appreciated that the function pointer in the control request should point to a response function based on the second communication protocol, i.e. the X11 protocol, and when the controlled computing device is a wailand environment, it includes all functions based on wailand, and therefore, it is necessary to point the function pointer to a function based on the wairandd protocol, i.e. the above-mentioned objective function.
In one particular example, a portion of pseudo code is shown that points a function pointer to an object function.
The input protocol is simulated by the aid of a/docking Wayland mouse wheel, and pSgreen- > FakeAxisInput is converted into a request for mouse scrolling to the Wayland server
static void fake_axis_input(ScreenPtr pScreen, uint32_t axis, double value)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
org_kde_kwin_fake_input_axis(xwl_screen->fake_input, axis, wl_fixed_from_double(value));
}
The input protocol of the analog input of the Watand keyboard is converted into pSgreen- > FakeKeyBoard input to request keyboard actions to the Watand server
static void fake_keyboard_input(ScreenPtr pScreen, uint32_t keyState, uint32_t button)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
// 1 WL_KEYBOARD_KEY_STATE_PRESSED 0 WL_KEYBOARD_KEY_STATE_RELEASED
org_kde_kwin_fake_input_keyboard_key(xwl_screen->fake_input, button, keyState);
}
static void fake_button_input(ScreenPtr pScreen, uint32_t buttonState, uint32_tbutton)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
// 1 WL_POINTER_BUTTON_STATE_PRESSED 0 WL_POINTER_BUTTON_STATE_RELEASED
org_kde_kwin_fake_input_button(xwl_screen->fake_input, button, buttonState);
}
The relative movement protocol of the/docking Watand mouse converts pScreen- > FakeMotionOffsetInput into a request for the relative movement of the mouse to the Watand server
static void fake_motion_offset_input(ScreenPtr pScreen, int32_t x, int32_t y)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
org_kde_kwin_fake_input_pointer_motion(xwl_screen->fake_input, wl_fixed_from_double(x), wl_fixed_from_double(y));
}
The// interfacing Wayland mouse absolute movement protocol converts pSgreen- > FakeMotionAbsolute input into a request for absolute movement of the mouse to Wayland server
static void fake_motion_absolute_input(ScreenPtr pScreen, int32_t x, int32_t y)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
org_kde_kwin_fake_input_pointer_motion_absolute(xwl_screen->fake_input, wl_fixed_from_double(x), wl_fixed_from_double(y));
}
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
org_kde_kwin_fake_input_axis(xwl_screen->fake_input, axis, wl_fixed_from_double(value));
}
static void fake_keyboard_input(ScreenPtr pScreen, uint32_t keyState, uint32_t button)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
// 1 WL_KEYBOARD_KEY_STATE_PRESSED 0 WL_KEYBOARD_KEY_STATE_RELEASED
org_kde_kwin_fake_input_keyboard_key(xwl_screen->fake_input, button, keyState);
}
static void fake_button_input(ScreenPtr pScreen, uint32_t buttonState, uint32_t button)
{
struct xwl_screen *xwl_screen = xwl_screen_get(pScreen);
........
Function pointer to actual processing function
pScreen->FakeAxisInput = fake_axis_input;
pScreen->FakeKeyBoardInput = fake_keyboard_input;
pScreen->FakeButtonInput = fake_button_input;
pScreen->FakeMotionOffsetInput = fake_motion_offset_input;
pScreen->FakeMotionAbsoluteInput = fake_motion_absolute_input。
Subsequently, in step 308, a simulated request for the control request is generated using the function pointer after the translation of the point address. Specifically, a function pointer indicating an operational event and corresponding operational data are encapsulated using a first communication protocol to generate a simulation request. That is, the function pointer after address conversion and the operation data included in the original control request are utilized to form the simulation request, and the function pointer of the simulation request points to the target function generated based on the first communication protocol Wayland, so that the target function can directly run in the Wayland environment.
Finally, in step 310, in response to the simulation request, a response result of the control request is simulated in the controlled computing device. Specifically, the objective function is determined based on the function pointer contained in the simulation request. And operating the objective function by taking the operation data as a function parameter to obtain a response result.
The method provided by the invention can enable a computing device using a first communication protocol to process a remote control request based on a second communication protocol by generating an analog request based on the first communication protocol (Wayland) to replace a control request based on the second communication protocol (X11).
According to the method provided by the invention, the association relation between the function pointer of the second communication protocol and the objective function of the first communication protocol is generated, so that the seamless compatibility of the remote control function under the first communication protocol environment of the software which is developed under the second communication protocol and contains the remote function is realized, the code modification amount is small, the universality is strong, the application range is wide, and the like, and the workload and the cost of compatibility adaptation of software manufacturers with the remote control function under the first communication protocol are greatly reduced.
Fig. 4 shows a schematic structural view of a remote control device 400 according to an embodiment of the present invention. The apparatus 400 resides in a controlled computing device that displays information via a first communication protocol. The apparatus 400 includes a receiving module 402, an obtaining module 404, a converting module 406, a generating module 408, and a responding module 410 coupled to each other. Wherein the receiving module 402 is adapted to receive a control request from the master computing device, the control request being generated based on the second communication protocol. The acquisition module 404 is adapted to acquire a function pointer of a response function to which the control request is directed. The conversion module 406 is adapted to convert the pointing direction of the function pointer into an objective function, which has the same function as the response function and is generated based on the first communication protocol. The generation module 408 is adapted to generate a simulated request for the control request using the function pointer after the translation of the pointing address. The response module 410 is adapted to respond to the simulation request by simulating the response result of the control request in the controlled computing device.
It should be noted that, the working principle and flow of the remote control device 400 provided in this embodiment are similar to those of the remote control method 300 described above, and the description of the remote control method 300 is referred to for related points and is not repeated here.
The various techniques described herein may be implemented in connection with hardware or software or, alternatively, with a combination of both. Thus, the methods and apparatus of the present invention, or certain aspects or portions of the methods and apparatus of the present invention, may take the form of program code (i.e., instructions) embodied in tangible media, such as removable hard drives, U-drives, floppy diskettes, CD-ROMs, or any other machine-readable storage medium, wherein, when the program is loaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention.
In the case of program code execution on programmable computers, the computing device will generally include a processor, a storage medium readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and at least one output device. Wherein the memory is configured to store program code; the processor is configured to perform the method of the invention in accordance with instructions in said program code stored in the memory.
By way of example, and not limitation, readable media comprise readable storage media and communication media. The readable storage medium stores information such as computer readable instructions, data structures, program modules, or other data. Communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. Combinations of any of the above are also included within the scope of readable media.
In the description provided herein, algorithms and displays are not inherently related to any particular computer, virtual system, or other apparatus. Various general-purpose systems may also be used with examples of the invention. The required structure for a construction of such a system is apparent from the description above. In addition, the present invention is not directed to any particular programming language. It should be appreciated that the teachings of the present invention as described herein may be implemented in a variety of programming languages and that the foregoing description of specific languages is provided for disclosure of preferred embodiments of the present invention.
In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.
Similarly, it should be appreciated that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects.
Those skilled in the art will appreciate that the modules or units or components of the devices in the examples disclosed herein may be arranged in a device as described in this embodiment, or alternatively may be located in one or more devices different from the devices in this example. The modules in the foregoing examples may be combined into one module or may be further divided into a plurality of sub-modules.
Those skilled in the art will appreciate that the modules in the apparatus of the embodiments may be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components of the embodiments may be combined into one module or unit or component and, furthermore, they may be divided into a plurality of sub-modules or sub-units or sub-components.
Furthermore, those skilled in the art will appreciate that while some embodiments described herein include some features but not others included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the invention and form different embodiments.
Furthermore, some of the embodiments are described herein as methods or combinations of method elements that may be implemented by a processor of a computer system or by other means of performing the functions. Thus, a processor with the necessary instructions for implementing the described method or method element forms a means for implementing the method or method element. Furthermore, the elements of the apparatus embodiments described herein are examples of the following apparatus: the apparatus is for carrying out the functions performed by the elements for carrying out the objects of the invention.
As used herein, unless otherwise specified the use of the ordinal terms "first," "second," "third," etc., to describe a general object merely denote different instances of like objects, and are not intended to imply that the objects so described must have a given order, either temporally, spatially, in ranking, or in any other manner.
While the invention has been described with respect to a limited number of embodiments, those skilled in the art, having benefit of the above description, will appreciate that other embodiments are contemplated within the scope of the invention as described herein. Furthermore, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter.
Claims (11)
1. A remote control method performed in a controlled computing device, the controlled computing device displaying information via a first communication protocol, the controlled computing device having a compatibility layer residing therein, the compatibility layer being adapted to run an application based on a second communication protocol on a computing device based on the first communication protocol, the method comprising:
receiving a control request from a master computing device, the control request generated based on a second communication protocol;
the method specifically comprises the steps of sending the control request to the compatible layer, and analyzing the control request by the compatible layer to obtain the function pointer;
converting the pointing direction of the function pointer into an objective function, wherein the objective function has the same function as the response function and is generated based on a first communication protocol;
generating a simulation request of the control request by using a function pointer after the address is converted;
and responding to the simulation request, and simulating a response result of the control request in the controlled computing device.
2. The method of claim 1, wherein parsing the control request with the compatibility layer to obtain the function pointer comprises:
determining a request type of the control request, wherein the request type indicates operation events, and the operation events comprise a mouse moving event, a mouse typing event and a keyboard typing event;
the function pointer is determined based on the operational event.
3. The method of claim 2, wherein parsing the control request with the compatibility layer to obtain the function pointer further comprises:
and analyzing the control request to obtain the operation data corresponding to the operation event.
4. A method as claimed in claim 3, wherein generating a simulated request for the control request using a function pointer after translation to an address comprises:
and encapsulating a function pointer indicating an operation event and corresponding operation data by using the first communication protocol to generate the simulation request.
5. The method of claim 3, wherein simulating, in the controlled computing device, a response result of the control request in response to the simulation request comprises:
determining the objective function based on a function pointer contained in the simulation request;
and taking the operation data as a function parameter, and running the objective function to obtain the response result.
6. The method of claim 3, wherein obtaining the operational data corresponding to the operational event comprises:
if the mouse typing event is mouse wheel input, the rolling direction and the rolling circle number of the mouse wheel are obtained;
and calculating the rolling increment of the mouse wheel based on the rolling direction and the rolling circle number, and taking the rolling increment as operation data of the mouse typing event.
7. The method of claim 1, wherein: the compatibility layer is a layer that allows application modules written using a second communication protocol to be run in the context of a first communication protocol.
8. The method of any of claims 1-7, wherein the first communication protocol is a waiand protocol and the second communication protocol is an X11 protocol.
9. A remote control apparatus residing in a controlled computing device, the controlled computing device displaying information via a first communication protocol, the controlled computing device residing in a compatibility layer, the compatibility layer adapted to run an application based on a second communication protocol on a computing device based on the first communication protocol, the apparatus comprising:
a receiving module adapted to receive a control request from a master computing device, the control request generated based on a second communication protocol;
the obtaining module is suitable for obtaining a function pointer of a response function pointed by the control request, and specifically comprises the steps of sending the control request to the compatible layer, and analyzing the control request by utilizing the compatible layer so as to obtain the function pointer;
the conversion module is suitable for converting the pointing direction of the function pointer into an objective function, and the objective function has the same function as the response function and is generated based on a first communication protocol;
the generation module is suitable for generating a simulation request of the control request by using the function pointer after the conversion of the pointing address;
and the response module is suitable for responding to the simulation request and simulating a response result of the control request in the controlled computing equipment.
10. A computing device, comprising:
at least one processor; and
a memory storing program instructions, wherein the program instructions are configured to be adapted to be executed by the at least one processor, the program instructions comprising instructions for performing the method of any of claims 1-8.
11. A readable storage medium storing program instructions which, when read and executed by a computing device, cause the computing device to perform the method of any of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311293973.4A CN117032874B (en) | 2023-10-08 | 2023-10-08 | Remote control method, device, computing equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311293973.4A CN117032874B (en) | 2023-10-08 | 2023-10-08 | Remote control method, device, computing equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117032874A CN117032874A (en) | 2023-11-10 |
| CN117032874B true CN117032874B (en) | 2024-02-23 |
Family
ID=88645251
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311293973.4A Active CN117032874B (en) | 2023-10-08 | 2023-10-08 | Remote control method, device, computing equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117032874B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117873414B (en) * | 2024-03-12 | 2024-06-07 | 麒麟软件有限公司 | X11-based linux program appointed display method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105006892A (en) * | 2015-07-13 | 2015-10-28 | 上海电力学院 | Power distribution network intelligent feeder terminal based on embedded advanced RISC machines (ARM) and multiple communication protocols |
| CN106203091A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | A kind of virtual machine escape detection method and device |
| CN109256858A (en) * | 2018-09-30 | 2019-01-22 | 北京金风科创风电设备有限公司 | Virtual synchronous generator system, communication method thereof and new energy control system |
| CN113064846A (en) * | 2021-04-14 | 2021-07-02 | 中南大学 | Zero-copy data transmission method based on Rsockets protocol |
| CN113242534A (en) * | 2021-06-11 | 2021-08-10 | 合肥工业大学 | General type multi-protocol digital instrument thing allies oneself with module based on WI-FI |
| CN115599268A (en) * | 2022-11-25 | 2023-01-13 | 统信软件技术有限公司（Cn） | Screen capture method, computing device and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11099768B2 (en) * | 2020-01-21 | 2021-08-24 | EMC IP Holding Company LLC | Transitioning from an original device to a new device within a data storage array |
-
2023
- 2023-10-08 CN CN202311293973.4A patent/CN117032874B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105006892A (en) * | 2015-07-13 | 2015-10-28 | 上海电力学院 | Power distribution network intelligent feeder terminal based on embedded advanced RISC machines (ARM) and multiple communication protocols |
| CN106203091A (en) * | 2016-06-30 | 2016-12-07 | 北京奇虎科技有限公司 | A kind of virtual machine escape detection method and device |
| CN109256858A (en) * | 2018-09-30 | 2019-01-22 | 北京金风科创风电设备有限公司 | Virtual synchronous generator system, communication method thereof and new energy control system |
| CN113064846A (en) * | 2021-04-14 | 2021-07-02 | 中南大学 | Zero-copy data transmission method based on Rsockets protocol |
| CN113242534A (en) * | 2021-06-11 | 2021-08-10 | 合肥工业大学 | General type multi-protocol digital instrument thing allies oneself with module based on WI-FI |
| CN115599268A (en) * | 2022-11-25 | 2023-01-13 | 统信软件技术有限公司（Cn） | Screen capture method, computing device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117032874A (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11494010B2 (en) | Touch support for remoted applications | |
| WO2019144681A1 (en) | Automated testing method and apparatus, storage medium and electronic device | |
| US6466973B2 (en) | Method and system for managing storage devices over a network | |
| CN102221993B (en) | The declarative definition of complex user interface Status Change | |
| US5621434A (en) | Cursor manipulation system and method | |
| CN117032874B (en) | Remote control method, device, computing equipment and storage medium | |
| Gettys et al. | The X window system, version 11 | |
| US20100082733A1 (en) | Extensible remote programmatic access to user interface | |
| US7275240B2 (en) | Method and system for recording macros in a language independent syntax | |
| JP2014519079A (en) | Method for adapting user interface and input control | |
| CN102520899A (en) | Universal cockpit display management system and method for developing corresponding display and control systems | |
| CN110968305A (en) | Applet visualization generation method, device, equipment and storage medium | |
| JP2014135013A (en) | Image transfer method, server apparatus, and program | |
| CN108733602B (en) | Data processing | |
| CN111414085B (en) | Input method operation method, system and medium of kylin mobile operation system | |
| US20200274856A1 (en) | Isolated data processing modules | |
| CN113778622A (en) | Cloud desktop keyboard event processing method, device, equipment and storage medium | |
| CN111949265A (en) | Data processing method and device and electronic equipment | |
| CN113835816A (en) | Virtual machine desktop display method, device, equipment and readable storage medium | |
| Giatsintov et al. | Architecture of the graphics system for embedded real-time operating systems | |
| CN110930485B (en) | Character rotation method and device based on animation, server and storage medium | |
| US20240053996A1 (en) | Bridging ui elements across multiple operating systems | |
| Wang et al. | A framebuffer oriented graphical human-machine interaction mechanism for intelligent in-vehicle systems | |
| EP1437647B1 (en) | Method and system for recording macros in a language independent syntax | |
| CN117519845A (en) | Method and device for realizing virtual mouse |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |