CN114816585A - Remote input redirection method across operating environments - Google Patents

Remote input redirection method across operating environments Download PDF

Info

Publication number
CN114816585A
CN114816585A CN202210698900.2A CN202210698900A CN114816585A CN 114816585 A CN114816585 A CN 114816585A CN 202210698900 A CN202210698900 A CN 202210698900A CN 114816585 A CN114816585 A CN 114816585A
Authority
CN
China
Prior art keywords
event
desktop
input
input event
current
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202210698900.2A
Other languages
Chinese (zh)
Other versions
CN114816585B (en
Inventor
温研
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Linzhuo Information Technology Co Ltd
Original Assignee
Beijing Linzhuo Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Linzhuo Information Technology Co Ltd filed Critical Beijing Linzhuo Information Technology Co Ltd
Priority to CN202210698900.2A priority Critical patent/CN114816585B/en
Publication of CN114816585A publication Critical patent/CN114816585A/en
Application granted granted Critical
Publication of CN114816585B publication Critical patent/CN114816585B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/445—Program loading or initiating
    • G06F9/44536—Selecting among different versions
    • G06F9/44542—Retargetable
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/452—Remote windowing, e.g. X-Window System, desktop virtualisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/542—Event management; Broadcasting; Multicasting; Notifications
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/54—Interprogram communication
    • G06F9/544—Buffers; Shared memory; Pipes

Landscapes

  • Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • User Interface Of Digital Computer (AREA)
  • Debugging And Monitoring (AREA)

Abstract

The invention discloses a remote input redirection method across operating environments, which comprises the steps of acquiring an input event aiming at an android system and sent by a user through a remote assistance function of an android application, extracting information in the input event, and constructing an event aiming at the desktop system according to a corresponding event data structure in the desktop system, so that the purpose that the user remotely controls the desktop system through the android application operating in the android compatible environment is realized.

Description

Remote input redirection method across operating environments
Technical Field
The invention belongs to the technical field of android application data sharing, and particularly relates to a cross-operating-environment remote input redirection method.
Background
An android compatible environment (hereinafter referred to as an android compatible environment) on a desktop operating system such as Linux or Windows is a technology for running an android application on the Linux or Windows operating system, and has an important significance for enriching the application ecology of the operating system. Under the general condition, a typical android compatible environment mainly includes an android application display unit and an android application compatible service, the android application display unit is used for realizing a user interface of an android application on a desktop operating system, the android application compatible service is used for providing the android compatible environment running on the desktop operating system, and the android application is effectively run on the desktop operating system through the mutual cooperation of the android application display unit and the android application compatible service, for example, an implementation scheme of the existing android compatible environment has xDroid, which includes xDroid ui and xDroid server, the xDroid ui is the android application display unit, and the xDroid server is the android application compatible service.
Some android applications developed at present provide a function of remote assistance for a user, for example, a video conference android application provides a function of remotely controlling a currently displayed desktop for the user. Therefore, when a user uses such an android application in an android-compatible environment, a requirement for remotely controlling a desktop of a desktop system may also be generated in the process of sharing a screen of the desktop system, and specifically, when the user a uses the android-compatible environment to run a certain android application in a Linux or Windows desktop system to share a desktop of the Linux or Windows system, the user b located remotely may need to directly control the desktop of the Linux or Windows system by using a remote assistance function provided by the android application. However, the remote assistance function provided by the existing android application only supports the desktop of the android system remotely operated by the user, so for the android application used in the android compatible environment, the existing function only supports partial operation of the user in the android compatible environment remotely executed, and cannot meet the requirement that the user wants to operate the desktop of the Linux or Windows system.
Disclosure of Invention
In view of the above, the invention provides a cross-operating-environment remote input redirection method, which can enable a user to remotely control a desktop of a desktop system in which an android compatible environment is located by using an android application.
The invention provides a remote input redirection method across running environments, which comprises the following steps:
step 1, acquiring an application package name and Activity of an android application when the virtual display of the android application is created, taking the application package name and Activity as a shared package creation name and a shared Activity creation, and initializing the values of the current event time and the previous event time to be zero;
step 2, when an input event is generated in the android compatible environment, acquiring a package name of the current android application corresponding to the current Activity as the current package name, and if the current package name is the same as the created sharing package name, executing the step 3; otherwise, quitting the flow;
step 3, when the input event is a touch event, acquiring the screen size (x) of the desktop system S ，y S ) If the previous event time is not zero, executing the step 4, and if the previous event time is zero, updating the previous event time by using the current system time and then executing the step 4; when the input event isWhen the keyboard event happens, executing step 8;
step 4, acquiring the window size (x) of the top Activity of the current android application A ，y A ) The formula x' = x is adopted according to the mouse position (x, y) in the input event S ×（x/x A ），y′= y S ×（y/y A ) Calculating to obtain coordinate values (x ', y') relative to the desktop system screen; if the event type constant is ACTION _ MOVE, executing step 5; otherwise, executing step 6;
step 5, taking the occurrence time of the input event as the current event time, and if the previous event time is zero, updating the previous event time by the current event time and then executing step 6; if the previous event time is not zero, obtaining the difference value between the previous event time and the current event time, if the difference value is larger than a set threshold value, updating the previous event time by the current event time, and then executing the step 6, and if the difference value is not larger than the set threshold value, not updating the previous event time, and then executing the step 7;
step 6, sending desktop input event information consisting of coordinate values (x ', y') of a desktop system screen, an input event type and an event type constant to an android application display unit in an asynchronous RPC mode, and executing step 9;
step 7, a standard event processing flow of the android system is not executed, and step 9 is executed;
step 8, sending the desktop input event information formed by the information contained in the input event to an android application display unit in an asynchronous RPC mode, and executing step 9;
and 9, the android application display unit adopts the received desktop input event information, constructs a standard desktop input event according to a data structure of the event message in the desktop system, and sends the desktop input event to the desktop system.
Further, when an input event is generated in the android compatible environment in step 2, a manner of acquiring the package name of the current android application corresponding to the current Activity as the current package name is as follows: and acquiring a thread corresponding to the current Activity, and acquiring the package name of the current android application as the current package name according to the thread.
Further, the process that the android application display unit in the step 9 adopts the received desktop input event information, constructs a standard desktop input event according to a data structure of an event message in the desktop system, and sends the desktop input event to the desktop system includes the following steps:
9.1, if the android application display unit runs in a Linux system, executing the step 9.2; if the operation is in the Windows system, executing the step 9.5;
step 9.2, acquiring the protocol type of the display server of the Linux system, and executing step 9.3 if the protocol type is the X11 protocol; if the protocol type is the Wayland protocol, step 9.4 is executed;
step 9.3, acquiring a handle of the current main display of the Linux system, if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ DOWN or ACTION _ UP, constructing a mouse pressing or lifting event by adopting the received desktop input event information, and then sending the mouse pressing or lifting event to the Linux system; if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ MOVE, constructing a mouse moving event by adopting the received desktop input event information, and then sending the mouse moving event to a Linux system; if the event type in the desktop input event information is a keyboard event, constructing the keyboard event by adopting the received desktop input event information, and then sending the keyboard event to a Linux system;
step 9.4, adopting the received desktop Input Event information to construct an Input _ Event type desktop Input Event which accords with a Linux end Input Event data structure, and writing the desktop Input Event into a file handle of a mouse or keyboard device of the Linux system;
step 9.5, if the event type in the desktop input event information is a touch event, adopting the received desktop input event information to construct a mouse event, and then sending the mouse event to the Windows system; and if the event type in the desktop input event information is a keyboard event, constructing the keyboard event by adopting the received desktop input event information, and then sending the keyboard event to the Windows system.
Further, if the protocol type in the step 9.2 is X11 protocol, step 9.4 is executed, specifically: and adopting the received desktop Input Event information to construct a desktop Input Event which is in an Input _ Event type and accords with a Linux end Input Event data structure, writing the desktop Input Event into a file handle corresponding to a mouse or keyboard device of the Linux system, and finishing sending the Input Event to the Linux system.
Further, the manner of acquiring the protocol type of the display server of the Linux system in step 9.2 is as follows: and obtaining the protocol TYPE of the display server of the Linux system by obtaining the value of the variable XDG _ SESSION _ TYPE.
Further, the threshold value set in step 5 is a time difference between two mouse movement events in the desktop system.
Has the advantages that:
according to the method and the device, the information in the input event is extracted by obtaining the input event aiming at the android system sent by the user through the remote assistance function of the android application, and the event aiming at the desktop system is constructed according to the corresponding event data structure in the desktop system, so that the purpose that the user remotely controls the desktop system through the android application running in the android compatible environment is realized.
Drawings
FIG. 1 is a flowchart of a method for remote input redirection across runtime environments according to the present invention.
Detailed Description
The invention is described in detail below by way of example with reference to the accompanying drawings.
To facilitate understanding of the technical solutions provided by this patent, the technical terms of the android system and the related terms of the desktop system related to this patent are listed as follows:
display (Display), which is a class for fetching data information from a frame buffer and presenting the data information on a screen, wherein virtual Display (VirtualDisplay) is a sub-class of Display. The status of a Display generally includes two types, a read status (mCurrentState) and a to-be-drawn status (mdrowingstate), where a Display in the to-be-drawn status is generally a subset of a Display in the read status.
The display device (DisplayDevice) is a class for describing a display screen and is used for displaying rendering data in DisplaySurface on the display screen.
Events (Event), events in the android system mainly include keyboard events (KeyEvent) and touch events (MotionEvent), where the keyboard events mainly refer to physical key events on the device, such as: events such as pressing of a back key, and popping up of a menu key; the touch event mainly refers to an event generated by actions performed on a control on a program interface, such as: double-click, slide, etc., and touch events define a series of event-type constants to further classify the events, wherein the event-type constants include ACTION _ DOWN, ACTION _ UP, and ACTION _ MOVE, etc.
The display server is a program running in the desktop operating system and is responsible for coordinating input and output between the client and other parts of the operating system and between hardware and the operating system. The display server communicates with the client based on a display server protocol to display information of the client in the display device, the display server protocol in the Windows system is X Window, the display server protocol in the Linux system comprises X11, Wayland and Mir, and X11 is the most common display server protocol in the Linux distribution.
The android application provides remote assistance function which only supports the desktop of the android system remotely operated by the user, so that for using the android application in the android compatible environment, the existing function only supports partial operation of the user in the android compatible environment remotely executed by the user, and the requirement that the user wants to operate the desktop of the Linux or Windows system cannot be met.
The invention provides a remote input redirection method across running environments, the flow of which is shown in figure 1, and the method comprises the following steps:
step 1, acquiring an application package name and Activity of the android application when the virtual display is created by the android application, taking the application package name and Activity as the creation of a shared package name and the creation of shared Activity, and initializing the current event time and the previous event time to be zero.
Step 2, when an input event is generated in the android compatible environment, acquiring a thread corresponding to the current Activity, acquiring a package name applied to the current android as the current package name according to the thread, and executing the step 3 if the current package name is the same as the created shared package name; otherwise, the process is exited.
And 3, acquiring the type of the input event, and when the input event is a touch event (MotionEvent), acquiring the screen size of the desktop system as (x) S ，y S ) If the previous event time is not zero, executing the step 4, and if the previous event time is zero, acquiring the current system time as the previous event time and then executing the step 4; when the input event is a keyboard event (KeyEvent), step 8 is performed.
Step 4, acquiring the window size of the top Activity of the current android application and recording as (x) A ，y A ) Extracting mouse position coordinates in an input event to be (x, y), wherein the mouse position coordinates are coordinate values relative to the current Activity because the current input event is the input event of the android system, converting the mouse position coordinates into coordinate values relative to a desktop system screen is needed in order that the desktop system can respond to the input event, and obtaining the coordinate values (x) relative to the desktop system screen by adopting the following formula ′ ，y ′ ）：x ′ = x S ×(x/x A )，y ′ = y S ×(y/y A ) (ii) a If the event type constant is ACTION _ MOVE, executing step 5; otherwise, step 6 is executed.
Step 5, acquiring the occurrence time of the input event as the current event time, and if the previous event time is zero, updating the previous event time by the current event time and then executing step 6; if the previous event time is not zero, the difference value between the previous event time and the current event time is obtained, if the difference value is larger than the set threshold value, the step 6 is executed after the current event time is updated, and if the difference value is not larger than the set threshold value, the step 7 is executed without updating the previous event time.
Step 6, coordinate value (x) of the desktop system screen ′ ，y ′ ) The information of the input event type, the event type constant, the target Activity and the like forms desktop input event information, and the desktop is inputAnd (4) sending the incoming event information to an android application display unit in an asynchronous RPC mode, and executing the step 9.
And 7, executing a standard event processing flow of the android system, and executing a step 9.
And 8, forming desktop input event information by information contained in the input event, sending the desktop input event information to an android application display unit in an asynchronous Remote Procedure Call (RPC) mode, and executing the step 9.
And 9, the android application display unit adopts the received desktop input event information, constructs a standard desktop input event according to the data structure of the event message in the desktop system, and sends the desktop input event to the desktop system.
Further, in step 9, the process that the android application display unit adopts the received desktop input event information, constructs a standard desktop input event according to a data structure of the event message in the desktop system, and sends the desktop input event to the desktop system specifically includes the following steps:
9.1, if the android application display unit runs in a Linux system, executing the step 9.2; and if the android application display unit runs in the Windows system, executing the step 9.5.
Step 9.2, acquiring the protocol TYPE of the display server of the Linux system, for example, acquiring the value of the variable XDG _ SESSION _ TYPE to obtain the protocol TYPE of the display server of the Linux system, and if the protocol TYPE of the display server is the X11 protocol, executing step 9.3; if the protocol type of the display server is the Wayland protocol, step 9.4 is performed.
Step 9.3, acquiring a handle of a current main display of the Linux system, if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ DOWN or ACTION _ UP, adopting the received desktop input event information to construct a mouse-DOWN or mouse-UP event XTestFakeButtonEvent, and then sending the XTestFakeButtonEvent to the Linux system; if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ MOVE, constructing a mouse movement event XTestFakeMotionEvent by adopting the received desktop input event information, and then sending the XTestFakeMotionEvent to a Linux system; and if the event type in the desktop input event information is the keyboard event, constructing the keyboard event XTestFakeKeyEvent by adopting the received desktop input event information, and then sending the XTestFakeKeyEvent to the Linux system.
And 9.4, adopting the received desktop Input Event information to construct a desktop Input Event which is in an Input _ Event type and accords with a Linux end Input Event data structure, writing the desktop Input Event into a file handle corresponding to a mouse or keyboard device of the Linux system, and finishing sending the Input Event to the Linux system.
Step 9.5, if the Event type in the desktop input Event information is a touch Event, adopting the received desktop input Event information to construct a Mouse Event Mouse _ Event, and then sending the Mouse Event Mouse _ Event to the Windows system; and if the Event type in the desktop input Event information is the keyboard Event, constructing a keyboard Event Keybd _ Event by adopting the received desktop input Event information, and then sending the Keybd _ Event to the Windows system.
Further, for the Linux system with the display server having the protocol type of X11 protocol, the android application display unit running in the Linux system can also be implemented by adopting step 9.4, that is, by adopting the received desktop Input Event information, constructing a desktop Input Event which is of the type of Input _ Event and conforms to the Linux end Input Event data structure, writing the desktop Input Event into a file handle corresponding to a mouse or a keyboard device of the Linux system, and completing sending the Input Event to the Linux system.
Example 1:
in this implementation, the method for remotely redirecting input across operating environments provided by the present invention is implemented at a system level by remotely controlling a desktop system through an android application running in an android compatible environment, and specifically includes the following steps:
and S1, acquiring the application package name and Activity of the android application when the virtual display is created by the android application, taking the application package name and Activity as the creation sharing package name and the creation sharing Activity, and initializing the current event time and the previous event time to be zero.
Specifically, by modifying an android system function related to creating a virtual display in an android compatible environment, an application package name and Activity of an android application which performs a virtual display creation operation are acquired inside the android system function when the virtual display is created by the android system, the application package name is marked as packageringScreen, an association of the virtual display, the packageName and the Activity is established, and the packageringScreen and the Activity are stored in a public storage area. The android system function includes a constructor of mediaproject, createVirtualDisplay, displaymanager global.
S2, when an input event is generated in the android compatible environment, acquiring a thread corresponding to the current Activity, acquiring the package name of the current android application as the current package name according to the thread, and redirecting the input event to the desktop system.
In this implementation, the redirection of the InputEvent is completed by modifying the implementation of the inputmanager. InjectInputEvent is a public boroean InjectInputEvent (InputEvent, int mode), the function is essentially an Event of type InputEvent, and the events referred to in this application are MotionEvent and KeyEvent.
The method specifically comprises the following steps:
s2.1, acquiring a thread corresponding to the current Activity as currentThread, acquiring a package name of the current process as the current package name by a currentPackageName or currentOpPackageName method of currentThread, and executing S2.2 if the current package name is equal to the package SharingScreen and the event type is MotionEvent; if the current packet name is equal to packageSharingScreen and the event type is KeyEvent, S2.6 is performed.
S2.2, acquiring the window size of the top Activity of the current android application through currentThread, and recording the window size as activitySize (activitySize.x, activitySize.y), wherein the mouse coordinates (x, y) in the MotionEvent are relative to the current Activity, and the shared screen is a window of a desktop system, so that the actual window size of the window which needs to be operated by the remote user is the window size of the desktop system, and is recorded as screen Size (screen Size.x, screen Size.y), and therefore the window needs to be operated by the remote userThe mouse coordinates (x, y) in the MotionEvent obtained by the getX and getY methods are converted into mouse coordinates (x, y) with respect to the screenSize ′ ，y ′ ) According to the formula (x) ′ ，y ′ ）：x ′ = x S ×(x/x A )，y ′ = y S ×(y/y A ) Then there is x ′ =ScreenSize.x×(x/activitySize.x)，y ′ = screen size. y × (y/activity size. y); if the event type constant is ACTION _ MOVE, executing S2.3; otherwise, S2.4 is performed.
S2.3, recording the time difference between two mouse moving Move events in the desktop system as moveEventInternal; acquiring the Event time of the current input Event through an Event interface, recording the Event time as currentMoveEventTime, and executing S2.4 after updating previousMoveEventTime by currentMoveEventTime if the previous Event time previousMoveEventTime is zero; if previous event time previousvoventtime is not zero, obtaining a difference value between previousveventtime and currentmoveventtime, if the difference value is greater than moveventinternal, updating previousveventtime by currentmoveventtime, and then executing S2.4, and if the difference value is not greater than moveventinternal, not updating previousvoventtime, and executing S2.5.
The reason for performing this step is: the shared screen is actually a desktop of a desktop system, the remote control end is usually a mobile phone or Pad, the frequency of Move events generated when the sliding distance between a mouse and a finger is the same between the two systems is obviously different, generally, more Move events generated by the mobile phone are generated, the more Move events generated by the mobile phone are invalid events for the desktop system, if all the Move events generated by the mobile phone are forwarded to the desktop system, only unnecessary system overhead is increased, and a specific processing mechanism is designed for reducing the forwarding of unnecessary Move events.
S2.4, coordinate value (x) of desktop system screen ′ ，y ′ ) The information such as the input event type, the event type constant, the target Activity and the like forms desktop input event information, the desktop input event information is sent to the xDSL in an asynchronous RPC mode, and the desktop input event information is returned after the transmission of the RPC call is completedAnd then S3 is executed without waiting for the xdroidus process to be completed.
And S2.5, not executing the standard injectInputEvent event processing flow of the android system, and returning the function to execute S3.
And S2.6, forming information contained in the input event into desktop input event information, sending the desktop input event information to the xDroidUI in an asynchronous RPC mode, returning after finishing RPC calling, and executing S3 without waiting for finishing the xDroidUI processing.
S3, the xDroidUI adopts the received desktop input event information, constructs a standard desktop input event according to the data structure of the event message in the desktop system, and sends the desktop input event to the desktop system. Here, the processing of mouse and keyboard events by xdroid ui must be sent to the entire desktop operating system, rather than in the process, and specifically includes the following steps:
s3.1, if the desktop system is a Linux system, executing S3.2; if the desktop system is a Windows system, S3.6 is executed.
S3.2, acquiring a protocol TYPE of a display server through a value of XDG _ SESSION _ TYPE, acquiring a current main display through a function XOpenDisplay for an X11 protocol, executing S3.3 if the event TYPE is a MotionEvent, and executing S3.4 if the event TYPE is a Keyevent; s3.5 is performed for the Wayland protocol.
And S3.3, if the event type constant is ACTION _ DOWN or ACTION _ UP, using XTestFakeButtonEvent to send a mouse Down/Up event, and if the event type constant is ACTION _ MOVE, using XTestFakeMotionEvent to send a mouse moving event.
S3.4, sending the keyboard event by using XTestFakeKeyEvent.
And S3.5, acquiring file handles (/ dev/Input/Event X) of the keyboard and mouse events, acquiring the file handles of the events, converting the mouse/keyboard events into standard events of Linux-end data structures (comprising type, code and value) of type Input _ Event, and writing the standard events into the file handles (display managers) of the keyboard and the mouse devices through a write function.
And S3.6, sending a Mouse Event by using the Mouse _ Event and sending a keyboard Event by using the Keybd _ Event.
S4, monitor the destruction of VirtualDisplay, and end the monitoring of InputEvent, i.e., the InputEvent is not redirected any more.
In summary, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (6)

1. A method for remote input redirection across runtime environments, comprising the steps of:
step 1, acquiring an application package name and Activity of an android application when the virtual display of the android application is created, taking the application package name and Activity as a shared package creation name and a shared Activity creation, and initializing the values of the current event time and the previous event time to be zero;
step 2, when an input event is generated in the android compatible environment, acquiring a package name of the current android application corresponding to the current Activity as the current package name, and if the current package name is the same as the created sharing package name, executing the step 3; otherwise, quitting the flow;
step 3, when the input event is a touch event, acquiring the screen size (x) of the desktop system S ，y S ) If the previous event time is not zero, executing the step 4, and if the previous event time is zero, updating the previous event time by using the current system time and then executing the step 4; when the input event is a keyboard event, executing step 8;
step 4, acquiring the window size (x) of the top Activity of the current android application A ，y A ) The formula x' = x is adopted according to the mouse position (x, y) in the input event S × (x/ x A )，y′= y S × (y/ y A ) Calculating to obtain coordinate values (x ', y') relative to the desktop system screen; if the event type constant is ACTION _ MOVE, then go to step 5; otherwise, executing step 6;
step 5, taking the occurrence time of the input event as the current event time, and if the previous event time is zero, updating the previous event time by using the current event time and then executing step 6; if the previous event time is not zero, obtaining the difference value between the previous event time and the current event time, if the difference value is larger than a set threshold value, updating the previous event time by the current event time, and then executing the step 6, and if the difference value is not larger than the set threshold value, not updating the previous event time, and then executing the step 7;
step 6, sending desktop input event information consisting of coordinate values (x ', y') of a desktop system screen, an input event type and an event type constant to an android application display unit in an asynchronous RPC mode, and executing step 9;
step 7, a standard event processing flow of the android system is not executed, and step 9 is executed;
step 8, sending the desktop input event information formed by the information contained in the input event to an android application display unit in an asynchronous RPC mode, and executing step 9;
and 9, the android application display unit adopts the received desktop input event information, constructs a standard desktop input event according to the data structure of the event message in the desktop system, and sends the desktop input event to the desktop system.
2. The remote input redirection method according to claim 1, wherein in step 2, when the input event is generated in the android compatible environment, a manner of obtaining a package name of a current android application corresponding to the current Activity as the current package name is as follows: and acquiring a thread corresponding to the current Activity, and acquiring the package name of the current android application as the current package name according to the thread.
3. The remote input redirection method according to claim 1, wherein the android application display unit in step 9 adopts the received desktop input event information, constructs a standard desktop input event according to a data structure of an event message in a desktop system, and sends the desktop input event to the desktop system, and the method includes the following steps:
9.1, if the android application display unit runs in a Linux system, executing the step 9.2; if the operation is in the Windows system, executing the step 9.5;
step 9.2, acquiring the protocol type of the display server of the Linux system, and executing step 9.3 if the protocol type is the X11 protocol; if the protocol type is the Wayland protocol, step 9.4 is executed;
step 9.3, acquiring a handle of the current main display of the Linux system, if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ DOWN or ACTION _ UP, constructing a mouse pressing or lifting event by adopting the received desktop input event information, and then sending the mouse pressing or lifting event to the Linux system; if the event type in the desktop input event information is a touch event and the event type constant is ACTION _ MOVE, constructing a mouse moving event by adopting the received desktop input event information, and then sending the mouse moving event to a Linux system; if the event type in the desktop input event information is a keyboard event, constructing the keyboard event by adopting the received desktop input event information, and then sending the keyboard event to a Linux system;
step 9.4, adopting the received desktop Input Event information to construct an Input _ Event type desktop Input Event which accords with a Linux end Input Event data structure, and writing the desktop Input Event into a file handle of a mouse or keyboard device of the Linux system;
step 9.5, if the event type in the desktop input event information is a touch event, adopting the received desktop input event information to construct a mouse event, and then sending the mouse event to the Windows system; and if the event type in the desktop input event information is a keyboard event, constructing the keyboard event by adopting the received desktop input event information, and then sending the keyboard event to the Windows system.
4. The remote input redirection method according to claim 3, wherein if the protocol type in step 9.2 is X11, the step 9.4 is executed, specifically: and adopting the received desktop Input Event information to construct a desktop Input Event which is in an Input _ Event type and accords with a Linux end Input Event data structure, writing the desktop Input Event into a file handle corresponding to a mouse or keyboard device of the Linux system, and finishing sending the Input Event to the Linux system.
5. The remote input redirection method according to claim 3, wherein said step 9.2 is to obtain the protocol type of the display server of the Linux system by: and obtaining the protocol TYPE of the display server of the Linux system by obtaining the value of the variable XDG _ SESSION _ TYPE.
6. The remote input redirection method according to claim 1, wherein the threshold value set in step 5 is a time difference between two mouse movement events in a desktop system.
CN202210698900.2A 2022-06-20 2022-06-20 Remote input redirection method across operating environments Active CN114816585B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210698900.2A CN114816585B (en) 2022-06-20 2022-06-20 Remote input redirection method across operating environments

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210698900.2A CN114816585B (en) 2022-06-20 2022-06-20 Remote input redirection method across operating environments

Publications (2)

Publication Number Publication Date
CN114816585A true CN114816585A (en) 2022-07-29
CN114816585B CN114816585B (en) 2022-09-02

Family

ID=82521846

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210698900.2A Active CN114816585B (en) 2022-06-20 2022-06-20 Remote input redirection method across operating environments

Country Status (1)

Country Link
CN (1) CN114816585B (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114995716A (en) * 2022-08-02 2022-09-02 北京麟卓信息科技有限公司 Operation method of android application related file on desktop system
CN115309465A (en) * 2022-10-11 2022-11-08 北京麟卓信息科技有限公司 Low-delay mouse drive optimization method across operating environments
CN115640154A (en) * 2022-12-23 2023-01-24 北京麟卓信息科技有限公司 Cross-operation environment interaction delay optimization method based on intelligent trajectory prediction
CN116954809A (en) * 2023-09-19 2023-10-27 北京大学 Remote control method, device and system based on android device and electronic device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112511585A (en) * 2020-10-16 2021-03-16 麒麟软件有限公司 Input redirection method and system based on android system remote desktop
CN113220474A (en) * 2021-04-21 2021-08-06 麒麟软件有限公司 Android application sharing Linux system screen data method based on Android-EMUGL
CN113535652A (en) * 2021-09-16 2021-10-22 北京麟卓信息科技有限公司 Method and device for accessing multimedia file in android operating environment
US20210334119A1 (en) * 2018-10-22 2021-10-28 Citrix Systems, Inc. Providing a virtual desktop within a computing environment
CN113934464A (en) * 2021-12-14 2022-01-14 北京鲸鲮信息系统技术有限公司 Method and device for starting android application in Linux system and electronic equipment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210334119A1 (en) * 2018-10-22 2021-10-28 Citrix Systems, Inc. Providing a virtual desktop within a computing environment
CN112511585A (en) * 2020-10-16 2021-03-16 麒麟软件有限公司 Input redirection method and system based on android system remote desktop
CN113220474A (en) * 2021-04-21 2021-08-06 麒麟软件有限公司 Android application sharing Linux system screen data method based on Android-EMUGL
CN113535652A (en) * 2021-09-16 2021-10-22 北京麟卓信息科技有限公司 Method and device for accessing multimedia file in android operating environment
CN113934464A (en) * 2021-12-14 2022-01-14 北京鲸鲮信息系统技术有限公司 Method and device for starting android application in Linux system and electronic equipment

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114995716A (en) * 2022-08-02 2022-09-02 北京麟卓信息科技有限公司 Operation method of android application related file on desktop system
CN114995716B (en) * 2022-08-02 2022-09-30 北京麟卓信息科技有限公司 Operation method of android application related file on desktop system
CN115309465A (en) * 2022-10-11 2022-11-08 北京麟卓信息科技有限公司 Low-delay mouse drive optimization method across operating environments
CN115309465B (en) * 2022-10-11 2022-12-06 北京麟卓信息科技有限公司 Cross-operating-environment low-delay mouse drive optimization method
CN115640154A (en) * 2022-12-23 2023-01-24 北京麟卓信息科技有限公司 Cross-operation environment interaction delay optimization method based on intelligent trajectory prediction
CN116954809A (en) * 2023-09-19 2023-10-27 北京大学 Remote control method, device and system based on android device and electronic device
CN116954809B (en) * 2023-09-19 2023-12-12 北京大学 Remote control method, device and system based on android device and electronic device

Also Published As

Publication number Publication date
CN114816585B (en) 2022-09-02

Similar Documents

Publication Publication Date Title
CN114816585B (en) Remote input redirection method across operating environments
JP6092381B2 (en) Method, system, and computer-readable storage medium for providing access to a remote application via a web client
US9152436B2 (en) Gesture support for shared sessions
USRE46386E1 (en) Updating a user session in a mach-derived computer system environment
CN110083324A (en) Method, apparatus, electronic equipment and the computer storage medium of Image Rendering
US20090006977A1 (en) Method and System of Computer Remote Control that Optimized for Low Bandwidth Network and Low Level Personal Communication Terminal Device
US20210216203A1 (en) Method and terminal for displaying input method virtual keyboard
JP2001014250A (en) Information processing method, collaboration server, collaboration system and storage medium for storing information processing program
CN111263175A (en) Interaction control method and device for live broadcast platform, storage medium and electronic equipment
CN114615220B (en) Message sending control method and device
WO2023040242A1 (en) Information display device
KR100611255B1 (en) Remote conference method of sharing work space
CN110933227A (en) Assistance method, device, equipment and medium for intelligent terminal
Andujar et al. A cost-effective approach for developing application-control GUIs for virtual environments
CN114510308B (en) Method, device, equipment and medium for storing application page by mobile terminal
CN116401462A (en) Interactive data analysis method and system applied to digital sharing
KR100608737B1 (en) Method for interfacing with user when remotely logining in mobile communication terminal
CN112286419B (en) Method, device, terminal and storage medium for loading list data
KR20020034347A (en) Server-thin client system
JP2001197461A (en) Sharing operation method for multimedia information operation window
CN113778596A (en) Remote assistance method and device and electronic equipment
US9141443B2 (en) Method and system for integrating visual controls with legacy applications
US20170270108A1 (en) System for supporting remote accesses to a host computer from a mobile computing device
KR20020048229A (en) Mixed contents server-client system
US20230080554A1 (en) Information display device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant