CN117707450A

CN117707450A - Concurrent method, equipment and storage medium for screen collaboration and keyboard and mouse sharing

Info

Publication number: CN117707450A
Application number: CN202310848788.0A
Authority: CN
Inventors: 房家鹏
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2023-07-11
Filing date: 2023-07-11
Publication date: 2024-03-15

Abstract

The application provides a concurrent method, equipment and storage medium for screen collaboration and keyboard and mouse sharing. The method comprises the steps of determining whether the second electronic equipment searched based on the first connection technology can be searched through the second connection technology, and when the second electronic equipment can be searched through the second connection technology, establishing screen cooperative connection between the first electronic equipment and the second electronic equipment by adopting the second connection technology which can establish the screen cooperative connection between the first electronic equipment and the second electronic equipment and establish the mouse sharing connection between the first electronic equipment and the second electronic equipment, so that the screen cooperative function and the mouse sharing function can be used concurrently without mutual influence.

Description

Concurrent method, equipment and storage medium for screen collaboration and keyboard and mouse sharing

Technical Field

The present disclosure relates to the field of electronic devices, and in particular, to a concurrent method, device, and storage medium for screen collaboration and keyboard and mouse sharing.

Background

The screen collaboration is specifically realized by cooperatively connecting devices of different systems through a distributed technology, and further using a Sink terminal (receiving terminal) as an extended screen of a Source terminal (transmitting terminal), thereby realizing multi-screen office experience. The mouse sharing means that a plurality of electronic devices that establish a mouse sharing connection can share and use a set of Input/Output (I/O) devices such as a keyboard and a mouse.

However, at present, the screen collaboration and the mouse sharing cannot be used simultaneously (simultaneously), i.e. the mouse sharing connection is not supported under the screen collaboration scene. Accordingly, under a mouse sharing scene, the screen cooperative connection is not supported.

Disclosure of Invention

In order to solve the technical problems, the application provides a concurrent method, equipment and storage medium for screen collaboration and mouse sharing, which aim to enable the screen collaboration and mouse sharing to be used concurrently without mutual influence.

In a first aspect, the present application provides a concurrent method of screen collaboration and keyboard and mouse sharing. The method is applied to a system comprising a first electronic device and a second electronic device, and comprises the following steps: the first electronic device responds to the operation of establishing the screen cooperative connection, and searches for the second electronic device based on the first connection technology; when the second electronic device is searched based on the first connection technology, the first electronic device determines whether the second electronic device can be searched through the second connection technology; when the second electronic equipment is searched based on the second connection technology, the first electronic equipment establishes screen cooperative connection with the second electronic equipment based on the second connection technology; the first electronic equipment can also establish a mouse sharing connection with the second electronic equipment based on the second connection technology; when the second electronic equipment is not searched based on the second connection technology, the first electronic equipment establishes screen cooperative connection with the second electronic equipment based on the first connection technology; the first connection technology and the second connection technology are mutually exclusive, and at the same time, the first electronic device can only establish communication connection with the second electronic device based on the first connection technology or the second connection technology, wherein the communication connection comprises screen cooperative connection and mouse sharing connection.

The operating systems of the first electronic device and the second electronic device may be the same or different.

By way of example, in some possible cases, the first electronic device may be a Windows system, such as a PC of the Windows system.

For example, in some possible cases, the second electronic device may be an Android system, such as a pad of the Android system, a folding screen mobile phone, or the like.

In particular, in practical applications, the first electronic device may be regarded as a Source terminal in the following embodiments, and the second electronic device may be regarded as a Sink terminal in the following embodiments.

The first connection technology is a transmission technology implemented by means of NFC, and is a connection transmission service as described in the following embodiments.

Accordingly, the screen collaboration connection between the first electronic device and the second electronic device established based on the first connection technology is, as will be described below, oneHop connection. Regarding the functional modules that need to be involved in the screen cooperative connection between the first electronic device and the second electronic device established based on the first connection technology, and the interaction logic between the functional modules can be referred to fig. 4, and the embodiment shown in fig. 4 will not be described herein.

The second connection technology is a connection technology based on an identity authentication system, which is realized to perform trusted interconnection between systems and devices, such as a data transmission unit realized in an interconnection module in the following embodiments.

Accordingly, the screen collaboration between the first electronic device and the second electronic device established based on the second connection technology is connected, as will be described below with respect to the MagicLink connection. Regarding the functional modules that need to be involved in the screen cooperative connection between the first electronic device and the second electronic device established based on the second connection technology, and the interaction logic between the functional modules can be referred to fig. 4, and the embodiment shown in fig. 4 will not be described herein.

In addition, regarding the above-mentioned operation of triggering the establishment of the screen cooperative connection, reference may be made to fig. 4, 5A and 5B in the following embodiments, and for the embodiment portions of these drawings, details are not repeated here.

In addition, for specific implementation details of the first electronic device in determining whether the second electronic device can be searched by the second connection technology, reference may be made to fig. 4 in the following embodiment, and a processing portion of the online state of the query interconnection device shown in the embodiment shown in fig. 4, which is not described herein again.

Therefore, whether the second electronic equipment searched based on the first connection technology can be searched through the second connection technology or not is determined, and when the second electronic equipment can be searched through the second connection technology, the screen cooperative connection between the first electronic equipment and the second electronic equipment is established through the second connection technology which can establish the screen cooperative connection between the first electronic equipment and the second electronic equipment and the mouse sharing connection between the first electronic equipment and the second electronic equipment, so that the screen cooperative function and the mouse sharing function can be used concurrently without mutual influence.

According to a first aspect, when the second electronic device is searched based on the first connection technology, the method further comprises: the first electronic device determines whether the second electronic device supports the establishment of the screen cooperative connection by adopting a second connection technology; when the second electronic equipment is determined to support the establishment of the screen cooperative connection by adopting the second connection technology, the first electronic equipment executes the step of establishing the screen cooperative connection with the second electronic equipment based on the second connection technology; when it is determined that the second electronic device does not support the establishment of the screen cooperative connection using the second connection technology, the first electronic device performs the step of establishing the screen cooperative connection with the second electronic device based on the first connection technology.

For details of the specific implementation of the first electronic device in determining whether the second electronic device supports the establishment of the screen collaboration connection using the second connection technology, reference may be made to fig. 4 in the following embodiment, and a processing portion of whether the query opposite end versions shown in the embodiment shown in fig. 4 are matched, which will not be described herein.

Therefore, when the second electronic equipment can be searched through the second connection technology, whether the second electronic equipment supports the establishment of the screen cooperative connection with the first electronic equipment by adopting the second connection technology or not is further considered, so that the screen cooperative connection between the first electronic equipment and the second electronic equipment can be established by selecting a proper connection technology according to the judgment result and the actual situation.

According to the first aspect, or any implementation manner of the first aspect, the determining, by the first electronic device, whether the second electronic device supports the establishment of the screen collaboration connection using the second connection technology includes: the first electronic equipment acquires a version number supported by the second electronic equipment; the first electronic device determines whether the version number supported by the second electronic device is the same as the version number supported by the first electronic device; the version numbers supported by the first electronic equipment and the version numbers supported by the second electronic equipment are version numbers of functional modules realized based on the second connection technology; when the version number supported by the second electronic equipment is the same as the version number supported by the first electronic equipment, the second electronic equipment is determined to support the establishment of screen cooperative connection by adopting a second connection technology; and when the version number supported by the second electronic equipment is different from the version number supported by the first electronic equipment, determining that the second electronic equipment does not support to establish screen cooperative connection by adopting a second connection technology.

The above-mentioned functional modules are, for example, data transmission units implemented in the interconnection modules described in the following embodiments.

Due to the data transmission units of the same version, the two can be guaranteed to establish the screen cooperative connection based on the second connection technology. Therefore, whether the first electronic device and the second electronic device adopt the same version of data transmission unit or not can be determined whether to adopt the first connection technology to establish the screen cooperative connection or the second connection technology to establish the screen cooperative connection, so that the method is better suitable for practical application scenes.

According to the first aspect, or any implementation manner of the first aspect, the determining, by the first electronic device, whether the second electronic device supports the establishment of the screen collaboration connection using the second connection technology includes: the first electronic equipment acquires a version number supported by the second electronic equipment; the first electronic device determines whether the version number supported by the second electronic device is the same as a preset version number; the version number supported by the second electronic device and the preset version number are version numbers of functional modules realized based on the second connection technology; when the version number supported by the second electronic equipment is the same as the preset version number, the second electronic equipment is determined to support the establishment of the screen cooperative connection by adopting a second connection technology; and when the version number supported by the second electronic equipment is different from the preset version number, determining that the second electronic equipment does not support to establish screen cooperative connection by adopting a second connection technology.

Because the newly released version notification can add new functions on the basis of the original version, whether the version number of the data transmission unit of the second electronic device is a preset data transmission unit capable of realizing the screen cooperative connection by adopting the second connection technology or is larger than the set version number can be determined, and whether the screen cooperative connection is established by adopting the first connection technology or the screen cooperative connection is established by adopting the second connection technology can be determined, so that the method is better suitable for practical application scenes.

According to the first aspect, or any implementation manner of the first aspect, when the second electronic device is started, the version number supported by the second electronic device is written into a functional module integrated with the second connection technology by a screen collaboration service of the second electronic device.

The above functional module is, for example, a device management unit in an interconnection module in the Android device in the following embodiment.

For the process of writing the version number currently supported by the data transmission unit through the screen collaborative service item device management unit when the Android device or the Sink terminal is started, refer to fig. 4 in the embodiment and the embodiment shown in fig. 4, which are not described herein.

According to the first aspect, or any implementation manner of the first aspect, when the second electronic device is searched based on the second connection technology, after the first electronic device establishes a screen collaboration connection with the second electronic device based on the second connection technology, the method further includes: based on the second connection technology, the first electronic device establishes a mouse-shared connection with the second electronic device.

Therefore, under the condition that the first electronic equipment and the second electronic equipment establish the screen cooperative connection based on the second connection technology, the establishment of the mouse sharing between the first electronic equipment and the second electronic equipment is established based on the second connection technology, and the concurrent use of the screen cooperative function and the mouse sharing function is realized.

According to the first aspect, or any implementation manner of the first aspect, in a scenario where the screen collaboration and the keyboard-mouse sharing are concurrent, and a floating window that is an extension screen of the first electronic device is displayed on a screen of the second electronic device, the method further includes: when the first electronic device detects that the mouse moves in the main screen of the first electronic device, starting the edge detection of the main screen; the mouse is connected with the first electronic equipment; when the mouse moves out of the main screen through the edge detection of the main screen, the first electronic device sends a first mouse crossing request to the second electronic device, wherein the first mouse crossing request carries a first mouse offset; after receiving a first mouse crossing request, the second electronic device calculates a first mouse position according to a first mouse offset; when the first mouse position is located outside the expansion screen area, the second electronic device makes a first confirmation message aiming at the first mouse crossing request and sends the first confirmation message to the first electronic device; wherein the first confirmation message indicates that the mouse traverses from the home screen to the screen of the second electronic device; the first electronic device hides the mouse cursor of the first style displayed on the main screen, and displays the mouse cursor of the second style on the screen of the second electronic device based on the key mouse sharing function.

The mouse in the aspect is an external mouse connected with the first electronic equipment or a self-contained touch pad.

For specific details of the electronic device implementing the home screen edge detection, reference may be made to fig. 6, fig. 7A to fig. 7C in the following embodiments, and descriptions of the embodiments corresponding to these drawings will not be repeated herein.

After the first mouse traversing request, for example, after the collaborative mode setting is completed in the embodiment shown in fig. 6, the mouse sharing service of the Windows device detects the mouse movement for the first time, and confirms the mouse traversing request sent to the Android device when the mouse exceeds the edge of the main screen.

The first confirmation message is, for example, a message that the Android device first sends to the Windows device to confirm that the mouse passes through to the pad screen in the embodiment shown in fig. 6.

For the first mouse traversing request, the functional modules involved in the first acknowledgement message transmission process, and specific interactions between the functional modules, refer to fig. 6, and portions of the embodiment shown in fig. 6, which are not described herein again.

And displaying a second-style mouse cursor on a screen of the second electronic device, specifically, closing and displaying the second-style mouse at the calculated first mouse position.

Wherein a first style of mouse cursor, such as arrow style cursor 10a as described in the embodiments below, is used. A second type of mouse cursor, such as a dot-style cursor 10b as described in the embodiments below.

Regarding the switching of the mouse in this aspect, for example, the switching of the mouse in the scene shown in (1) in fig. 7B to the scene shown in (2) in fig. 7B.

According to the first aspect, or any implementation manner of the first aspect, the method further includes: when the first mouse position is located in the expansion screen area, the second electronic device makes a second confirmation message aiming at the first mouse crossing request and sends the second confirmation message to the first electronic device; the second confirmation message indicates that the mouse passes through the main screen into the expansion screen; the first electronic device conceals a first-style mouse cursor displayed on the main screen, and displays the first-style mouse cursor on the expansion screen based on the screen cooperation function.

Understandably, the scene is a scene in which a user quickly drags the mouse, and moves the mouse into the expansion screen directly after the mouse passes through. For example, from the scene of (1) in fig. 7B, the scene of (2) in fig. 7C is directly switched.

According to the first aspect, or any implementation manner of the first aspect, the method further includes: in the process of displaying a second-style mouse cursor in a screen of the second electronic device, when the mouse continues to be moved, the first electronic device acquires a second mouse offset of the mouse and sends the second mouse offset to the second electronic device; the second electronic device calculates a second mouse position according to the second mouse offset; when the second mouse position is located in the expansion screen area, the second electronic device makes a third confirmation message; the third confirmation message indicates that the mouse passes through the screen of the second electronic device into the expansion screen; the first electronic device changes the mouse cursor of the second style into the mouse cursor of the first style, and sets the mouse cursor of the first style into the expansion screen based on the screen cooperation function.

For implementation details of this aspect, reference may be made to fig. 6 and fig. 7B, and portions of the corresponding embodiments of these drawings, which are not described herein.

According to the first aspect, or any implementation manner of the first aspect, the method further includes: in the process of displaying a mouse cursor of a first style in the expansion screen, when the mouse is continuously moved, the first electronic equipment starts the edge detection of the expansion screen; when the mouse moves out of the expansion screen through the edge detection of the expansion screen, the first electronic device sends a second mouse crossing request to the second electronic device, and the second mouse crossing request carries a third mouse offset; after receiving a second mouse crossing request, the second electronic device calculates a third mouse position according to a third mouse offset; when the third mouse position is located outside the expansion screen area, the second electronic device makes a fourth confirmation message aiming at the second mouse traversing request and sends the fourth confirmation message to the first electronic device; the fourth confirmation message indicates that the mouse passes through the expansion screen to the screen of the second electronic device; the first electronic device hides the mouse cursor of the first type displayed on the expansion screen, and displays the mouse cursor of the second type on the screen of the second electronic device based on the key mouse sharing function.

Regarding the switching of the mouse in this aspect, for example, the switching of the mouse in the scene shown in (1) in fig. 7C to the scene shown in (2) in fig. 7C.

For implementation details of this aspect, reference may be made to fig. 6 and 7C, and portions of the corresponding embodiments of these drawings, which are not described herein.

In a second aspect, the present application provides an electronic device. The electronic device includes: a memory and a processor, the memory and the processor coupled; the memory stores program instructions that, when executed by the processor, cause the electronic device to perform the instructions of the first aspect or of the method in any possible implementation of the first aspect.

The electronic device may be the first electronic device described in the first aspect, or may be the second electronic device.

Any implementation manner of the second aspect and the second aspect corresponds to any implementation manner of the first aspect and the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation manner of the second aspect may be referred to the technical effects corresponding to the first aspect and any implementation manner of the first aspect, which are not described herein.

In a third aspect, the present application provides a computer readable medium for storing a computer program comprising instructions for performing the method of the first aspect or any possible implementation of the first aspect.

Any implementation manner of the third aspect and any implementation manner of the third aspect corresponds to any implementation manner of the first aspect and any implementation manner of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation manner of the third aspect may be referred to the technical effects corresponding to the first aspect and any implementation manner of the first aspect, which are not described herein.

In a fourth aspect, the present application provides a computer program comprising instructions for performing the method of the first aspect or any possible implementation of the first aspect.

Any implementation manner of the fourth aspect and any implementation manner of the fourth aspect corresponds to any implementation manner of the first aspect and any implementation manner of the first aspect, respectively. Technical effects corresponding to any implementation manner of the fourth aspect may be referred to the technical effects corresponding to any implementation manner of the first aspect, and are not described herein.

In a fifth aspect, the present application provides a chip comprising processing circuitry, a transceiver pin. Wherein the transceiver pin and the processing circuit communicate with each other via an internal connection path, the processing circuit performing the method of the first aspect or any one of the possible implementation manners of the first aspect to control the receiving pin to receive signals and to control the transmitting pin to transmit signals.

Any implementation manner of the fifth aspect and any implementation manner of the fifth aspect corresponds to any implementation manner of the first aspect and any implementation manner of the first aspect, respectively. Technical effects corresponding to any implementation manner of the fifth aspect may be referred to the technical effects corresponding to any implementation manner of the first aspect, and are not described herein.

In a sixth aspect, the present application provides a concurrent system for screen collaboration and keyboard-mouse sharing, which may include the first electronic device and the second electronic device referred to in the first aspect or any of the possible implementations of the first aspect.

Any implementation manner of the sixth aspect corresponds to any implementation manner of the first aspect and the first aspect, respectively. Technical effects corresponding to any implementation manner of the sixth aspect may be referred to the technical effects corresponding to any implementation manner of the first aspect, and are not described herein.

Drawings

Fig. 1A is a schematic diagram of a hardware structure of an Android device shown in an exemplary manner;

fig. 1B is a schematic diagram of a software architecture of an Android device shown in an exemplary manner;

FIG. 2A is a schematic diagram of a hardware architecture of an exemplary Windows device;

FIG. 2B is a schematic diagram of a software architecture of an exemplary Windows device;

FIG. 3 is a schematic diagram of an interconnection module involved in an exemplary Windows device and Android device;

FIG. 4 is an interaction schematic diagram of functional modules involved in an implementation process of the exemplary screen collaboration and mouse sharing concurrent method;

FIGS. 5A and 5B are user interface diagrams of exemplary illustrated trigger device discovery;

fig. 6 is an interaction schematic diagram of functional modules involved in mouse switching in a scenario where a screen coordination function and a mouse sharing function are concurrent, which is exemplarily shown;

fig. 7A to 7C are schematic diagrams illustrating changes of a cursor on a Windows device screen, an extension screen, and an Android device screen in an exemplary mouse switching process.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art based on the embodiments herein without making any inventive effort, are intended to be within the scope of the present application.

The term "and/or" is herein merely an association relationship describing an associated object, meaning that there may be three relationships, e.g., a and/or B, may represent: a exists alone, A and B exist together, and B exists alone.

The terms first and second and the like in the description and in the claims of embodiments of the present application are used for distinguishing between different objects and not necessarily for describing a particular sequential order of objects. For example, the first target object and the second target object, etc., are used to distinguish between different target objects, and are not used to describe a particular order of target objects.

In the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as examples, illustrations, or descriptions. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In the description of the embodiments of the present application, unless otherwise indicated, the meaning of "a plurality" means two or more. For example, the plurality of processing units refers to two or more processing units; the plurality of systems means two or more systems.

With the popularity of electronic devices, an increasing number of users possess multiple (at least 2) electronic devices. In order to facilitate users to use a plurality of electronic devices for office work and entertainment, a screen collaboration function and a keyboard and mouse sharing function are favored by more and more users.

Based on the screen collaboration function, the user may implement a multi-screen (at least 2 screens) experience. For example, a plurality of files (documents, pictures and the like) at the Source end, such as a file A and a file B, are opened at the Source end, and a file B is opened at the Sink end, so that tiling display of the file A and the file B is realized, and convenience is brought to a user to check at the same time.

Based on the keyboard-mouse sharing module, electronic devices such as mobile phones, tablets and the like which do not have or are not connected with Input/Output (I/O) devices such as a keyboard, a mouse and the like can share and use a keyboard, a mouse or a touch pad of a computer.

Although the screen coordination function and the keyboard-mouse sharing function greatly facilitate the user to use a plurality of electronic devices for office and entertainment, the screen coordination function is based on the connection transmission service (such as a bump on hop) to realize the call of a Bluetooth (BT) chip or a wireless fidelity (wireless fidelity, wiFi) chip, so that the screen coordination connection between the Source end and the Sink end is established. The keyboard and mouse sharing function is based on a data transmission unit (such as MagicLink) in an interconnection module (such as MagicRing trust), and the call to a BT chip (hereinafter referred to as BT) or a WiFi chip (hereinafter referred to as WiFi). Therefore, when BT, wiFi is preempted by the connection transfer service, the data transfer unit cannot use BT, wiFi. Accordingly, when BT, wiFi is preempted by the data transfer unit, the connection transfer service cannot use BT, wiFi.

That is, the screen collaboration function and the mouse sharing function cannot be used concurrently at present, that is, the mouse sharing connection is not supported in the screen collaboration scene. Accordingly, under a mouse sharing scene, the screen cooperative connection is not supported. In this way, under the screen collaboration scene, the user needs to access different key mice for the Source terminal and the Sink terminal respectively, and further, the processing of the display content in the Source terminal and the Sink terminal is realized by controlling the different key mice. In the case of the mouse sharing scenario, if the file stored on the Source end (such as file a) is to be viewed on the Sink end, the file a stored on the Source end needs to be copied and transferred to the Sink end for storage (regarded as file a ') through the shared mouse, and then the file a' stored on the Sink end is opened through the shared mouse. If the user edits the content in file A' at the Sink end, file A at the Source end cannot synchronize the modification.

In view of this, the present application provides a concurrent method for screen collaboration and mouse sharing, which aims to enable the screen collaboration and mouse sharing to be used concurrently without affecting each other. In particular, in the application, in order to enable the screen collaboration function and the keyboard-mouse sharing function to be used concurrently, the capability of supporting MagicLink channel connection is added in the screen collaboration service corresponding to the screen collaboration service, namely, the capability of calling the function module/service/interface in the interconnection module is added in the screen collaboration service corresponding to the screen collaboration service, so that the screen collaboration connection can also be realized through the data transmission unit in the interconnection module.

Therefore, in the BT and WiFi shared scenes, which are occupied by the data transmission units, the establishment of the screen cooperative connection can be realized based on the same data transmission unit, and the establishment of the screen cooperative connection can be realized without the need of user intervention to disconnect the shared connection of the keyboard and the mouse.

With respect to the above-mentioned screen collaboration service, it may also be directly expressed as a collaboration service in some possible cases, and embodiments of the present application will be described by taking the screen collaboration service as an example.

In order to better understand the technical solution provided by the embodiments of the present application, before describing the technical solution of the embodiments of the present application, a hardware structure and a software architecture of an electronic device applicable to the embodiments of the present application are first described with reference to the accompanying drawings.

The electronic device to which the embodiments of the present application can be applied includes, but is not limited to, a mobile phone (such as a folding screen mobile phone), a tablet, a PC, and the like, which are not specifically mentioned herein.

In addition, it should be noted that, in the technical solution provided in the embodiments of the present application, any of the above electronic devices may be used as a Source terminal or a Sink terminal.

In addition, the operating system of the electronic device includes, but is not limited to, windows system, android system and iOS system. That is, in the technical solution provided in the embodiments of the present application, the operating systems of the Source end and the Sink end may be the same or different.

For convenience of explanation, in the embodiment of the present application, the Source end is a Windows device, for example, the operating system is a PC of a Windows system; the Sink terminal is Android equipment, for example, an operating system is a tablet of an Android system. It should be understood that this limitation is merely an example set forth to better illustrate the technical solution of the present embodiment and is not intended by the only limitation of the present embodiment.

Referring to fig. 1A, a hardware structure of an Android device is schematically shown.

Referring to fig. 1A, an exemplary Android device may include: processor 110, external memory interface 120, internal memory 121, universal serial bus (universal serial bus, USB) interface 130, charge management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, keys 190, motor 191, indicator 192, camera 193, display 194, and subscriber identity module (subscriber identification module, SIM) card interface 195, among others.

The processor is used for executing program instructions, and further, the technical scheme that the screen coordination function and the keyboard and mouse sharing function are used concurrently is achieved.

The audio module 170 may include, among other things, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and the like.

The sensor module 180 may include a pressure sensor, a gyroscope sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc., which are not further listed herein, but are not limited thereto.

The USB interface can also comprise a keyboard and a mouse.

The BT and WLAN (WiFi in the scheme) in the wireless communication module are specifically used for realizing the sharing connection of the key mouse and the cooperative connection of the screen.

Regarding the functions and the purposes of the above hardware included in the Android device, reference may be made to the hardware structure description document of the existing Android device, which is not described herein.

Furthermore, it should be understood that the Android device shown in fig. 1A is only one example, and in a specific implementation, the Android device may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in fig. 1A may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

Referring to fig. 1B, a software architecture of an Android device is schematically shown.

Before describing the software structure of the Android device, first, a description is given of an architecture that may be adopted by the Android device and other operating systems, such as a software system of a Windows system.

Specifically, in practical applications, both Windows devices and Android devices, a software system (operating system) of the device may be a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. And, for Windows device and Android device, both can be divided into user mode and kernel mode.

As shown in fig. 1B, the layered architecture of the Android device divides the software into several layers, each with a clear role and division. The layers communicate with each other through a software interface. In some implementations, the Android system is divided into five layers, from top to bottom, an Application layer (APP layer), a framework layer (Application Framework, FWK layer), an Zhuoyun rows (Android run) and system libraries, a hardware abstraction layer (Hardware Abstract Layer, HAL layer), and a Kernel layer (Linux Kernel layer), respectively.

The APP layer may include a series of application packages. As shown in FIG. 1B, the application package may include camera, game, video, live, etc. applications, which are not explicitly recited herein, and are not limiting in this application.

Wherein the FWK layer provides an application programming interface (application programming interface, API) and programming framework for the APP layer applications. In some implementations, these programming interfaces and programming frameworks can be described as functions. As shown in fig. 1B, the FWK layer may include functions of a connection transmission service, a mouse sharing service, a screen cooperative service, an interconnection module, an interconnection device authentication service, a cross-device cooperative authentication service, and the like, which are not further listed herein, but are not limited thereto.

The connection transmission service is a connection technology used by the native screen collaboration service to establish collaborative connection.

The mouse sharing service is a service process for realizing the mouse sharing function. In particular, in each embodiment of the application, the mouse sharing service also interacts with the screen collaboration service and the data transmission unit in the interconnection module, so that the mouse sharing function and the screen collaboration function are used concurrently. The specific operations that can be performed by the mouse sharing service are described in detail in the following description of the embodiments of fig. 4 and 6, which are not repeated here.

The screen collaboration service is a service process for realizing the screen collaboration function. In particular, in the embodiments of the present application, the screen collaboration service not only has the characteristics of the screen collaboration service in the existing solution, but also increases the transformation characteristics of the MagicLink channel switching and compatibility processing. The specific operation of the screen collaboration service can be described in detail in the following description of the embodiments of fig. 4 and 6, which is not repeated here.

The interconnection module may also be represented as a trust ring or a MagicRing trust ring. In particular to a connection technology between different devices. The connection technology is based on an identity authentication system, realizes cross-system multi-device trusted interconnection, and enables services to be intelligently circulated and shared among devices. In particular, in the embodiments of the present application, in order to implement the screen collaboration function and the mouse sharing function, the interconnection module needs to include a device management unit (service/module/function), an access networking unit, and a data transmission unit shown in fig. 3.

And the device management unit is used for storing and synchronizing the service states and the attributes of the services accessed in the interconnection module.

Regarding the access networking unit, the access networking unit is configured to invoke BT or WiFi to discover the trust ring device.

It should be noted that, in this embodiment, the trust ring device is an electronic device that also integrates the above-mentioned mouse sharing service, screen collaboration service, and interconnection module.

The data transmission unit is used for establishing MagicLink connection so as to realize data negotiation, transmission and other operations in a key mouse sharing scene and a screen collaboration scene.

The function of the collaboration service and the data transmission module in the Android device is the same as that of the collaboration service and the data transmission module in the Windows device, and will not be described here again.

Regarding the functional modules/units disclosed in the interconnection module shown in fig. 3, in a scheme for implementing the concurrent use of the screen collaboration function and the mouse sharing function provided in the present application, specific interaction and processing logic are described in detail in the following description of the embodiments of fig. 4 and 6, which are not repeated herein.

In addition, it should be noted that, in the implementation of this case, only the functional modules/units required for the interconnection module are shown in fig. 3, and in practical application, according to the service requirement, the interconnection module may further include a unified interconnection framework, an adaptive virtualization service, an audio virtualization service, and the like, which are not listed here, but are not limited in this application.

With continued reference to fig. 1B, an exemplary interconnection device authentication service and a cross-device cooperative authentication service are used to authenticate Sink devices, thereby ensuring security in a screen cooperative and mouse sharing scenario.

The FWK layer is described herein, and other layering of the Android device is described further below in conjunction with fig. 1B.

With continued reference to FIG. 1B, for the exemplary Android Runtime, it includes a core library and virtual machines. Android run is responsible for scheduling and management of the Android system.

The core library consists of two parts: one part is a function which needs to be called by java language, and the other part is a core library of android.

The APP layer and the FWK layer run in virtual machines. The virtual machine executes java files of the application program layer and the application program framework layer as binary files. The virtual machine is used for executing the functions of object life cycle management, stack management, thread management, security and exception management, garbage collection and the like.

With continued reference to FIG. 1B, for example, a plurality of functional modules may be included for the system library. For example: surface manager (surface manager), media Libraries (Media Libraries), three-dimensional (3D) graphics processing Libraries (e.g., openGL ES), two-dimensional (2D) graphics engines (e.g., SGL), etc.

The surface manager is used for managing the display subsystem and providing fusion of 2D and 3D layers for a plurality of application programs. Media libraries support a variety of commonly used audio, video formats for playback and recording, still image files, and the like. The media library may support a variety of audio video encoding formats, such as: MPEG4, h.264, MP3, AAC, AMR, JPG, PNG, etc. The three-dimensional graphic processing library is used for realizing three-dimensional graphic drawing, image rendering, synthesis, layer processing and the like.

It will be appreciated that the 2D graphics engine described above is a drawing engine for 2D drawing.

With continued reference to FIG. 1B, an exemplary HAL layer is an interface layer between the operating system kernel (kernel layer) and the hardware circuitry, which aims to isolate the FWK layer from the kernel so that Android does not over-rely on the kernel, thereby enabling development of FWK without regard to drivers.

With continued reference to FIG. 1B, by way of example, the HAL layer may include various interfaces therein, such as an audio-visual interface, a USB interface, a Bluetooth interface, a WiFi interface, etc., which are not to be construed as limiting the present application.

With continued reference to FIG. 1B, the kernel layer in the Android system is illustratively the layer between hardware and software. The kernel layer may include various processes/threads, power management modules, and various drivers, such as WiFi drivers, bluetooth drivers, and kernel drivers (Uinput drivers) for emulating user input events, etc.

As for the software architecture of the Android device, it will be understood that the layers and the functional modules included in the layers in the software Android device shown in fig. 1B and fig. 3 do not constitute a specific limitation on the Android device. In some possible implementations, the Android device may include more or fewer layers than illustrated, and more or fewer components may be included in each layer, without limitation of the present application.

Referring to fig. 2A, a hardware structure of a Windows device is exemplarily shown.

Referring to fig. 2A, an exemplary Windows device may include: processor 210, internal memory interface 220, external memory 230, wireless communication module 240, mouse interface 250, keyboard interface 260, charge management module 270, power management module 280, battery 290, and the like.

The mouse interface and the keyboard interface may be the same USB interface, i.e. the USB interface may be used as a mouse interface or a keyboard interface.

Regarding the functions and uses of the above hardware included in the Windows device, reference may be made to the hardware structure description document of the existing Windows device, which is not repeated here.

Furthermore, it should be understood that the Windows device shown in FIG. 2A is only one example, and in particular implementations, the Windows device may have more or fewer components than shown in the figures, may combine two or more components, or may have different component configurations. The various components shown in fig. 2A may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

Referring to fig. 2B, a software architecture of a Windows device is exemplarily shown.

From the above description, the Windows system can be classified into a user mode and a kernel mode. As shown in fig. 2B, from the user state perspective, the NTDLL, the service process, the system process, the environment subsystem, the interconnection module, the user process, and the like may be included in order from bottom to top. In this embodiment, the concurrent use of the screen collaboration function and the mouse sharing function mainly depends on the interconnection module, and the mouse sharing service, the screen collaboration service, the connection transmission service, the interconnection device authentication service, and the cross-device collaboration authentication service in the service process.

The roles of the interconnection module, the mouse sharing service, the screen cooperative service, the connection transmission service and the interconnection device authentication service in the Windows device and the cross-device cooperative authentication service are the same as those of the interconnection module, the mouse sharing service, the screen cooperative service, the connection transmission service and the interconnection device authentication service in the Android device, and the cross-device cooperative authentication service are not repeated here.

For the functions of other processes, systems, NTDLLs, etc. in the user mode, reference may be made to the description of the existing Windows system architecture, which is not repeated here.

In addition, it should be noted that, in some possible implementations, the user process may include a process corresponding to an application similar to the above-mentioned computer manager providing the function of turning on the screen collaboration and the function of sharing the mouse. The management of various services and the scheduling of various virtualized services can be realized through the corresponding process (such as a service management process) of a computer manager. For example, the scheduling of a mouse sharing service and a screen collaboration service in a service process.

With continued reference to FIG. 2B, from the perspective of the kernel state, the system may include, in order from bottom to top, a hardware abstraction layer, a kernel and device driver layer, a window management and image rendering module, and a Windows execution body. In particular, in the technical solution provided in this embodiment, the screen collaboration function and the keyboard-mouse sharing function need to rely on WiFi or BT to implement corresponding connection, so functional modules related to WiFi and BT, such as WiFi driver, bluetooth driver, and the like, need to be involved.

In addition, for Windows devices without I/O devices, it is also necessary to implement external keyboards, mice, etc. by means of USB interfaces, or bluetooth, or WiFi. Thus, in addition to the WiFi, bluetooth related functional modules, USB related functional modules, such as USB drivers, are also required.

For descriptions of the Windows executor in kernel mode, other functional modules related in the kernel and device driver layers, window management and drawing modules, hardware abstraction layers, etc., reference may be made to the description of the existing Windows system architecture, and no further description is given here.

As to the software architecture of the Windows device, it will be understood that the layers and the functional modules included in the layers in the software Windows device shown in fig. 2B and 3 do not constitute a specific limitation of the Windows device. In some possible implementations, the Windows device may include more or fewer layers than shown, and more or fewer components may be included in each layer, which is not limiting in this application.

Based on the above-described Windows device and Android device with the hardware structure and software architecture shown in the examples, the following describes a procedure for accessing the screen collaboration service to the interconnection module in conjunction with fig. 4, so that the screen collaboration function and the keyboard-mouse sharing function can be used concurrently.

In order to facilitate distinguishing functional modules in Windows devices (in this embodiment, the Source end) and Android devices (in this embodiment, the Sink end), in the following embodiments, a screen collaboration service, a connection transmission service, an interconnection module in the Source end, and a device management unit, an access networking unit, and a data transmission unit in the interconnection module are respectively represented as a Source screen collaboration service, a Source connection transmission service, a Source interconnection module, a Source device management unit, a Source access networking unit, and a Source data transmission unit; the screen collaboration service, the connection transmission service and the interconnection module in the Sink terminal, and the equipment management unit, the access networking unit and the data transmission unit in the interconnection module are respectively expressed as the Sink screen collaboration service, the Sink connection transmission service, the Sink interconnection module, the Sink equipment management unit, the Sink access networking unit and the Sink data transmission unit.

Referring to fig. 4, for example, when a user triggers a screen collaboration service at a Source terminal, the Source screen collaboration service will trigger a Source connection transmission service to discover a Sink terminal that can currently establish a screen collaboration connection with the Source terminal in response to the operation behavior.

The connection transmission service in this embodiment is a transmission technology implemented by means of NFC technology. Based on the characteristics of NFC, the Source terminal can automatically discover Sink terminals that are nearby and also support the connection transport service.

With continued reference to fig. 4, when the Source connection transmission service discovers that there is a Sink end near the Sink end with which a screen collaboration connection can be established, the Sink end currently discovered (searched) will be reported to the Source screen collaboration service.

Regarding the active discovery operation triggered by the user perceived by the Source screen collaboration service in this embodiment, in some possible implementations, for example, the user selects a screen collaboration option under the intelligent interconnection directory in a window (as shown in fig. 5A) corresponding to the window after the "computer housekeeping" application is started, selects an "extended mode" in a collaboration mode displayed on the right side of the window, and then clicks a control of "connect immediately" to trigger.

It should be understood that the above description is only an example for better understanding of the technical solution of the present embodiment, and is not the only limitation of the present embodiment. In practical application, the user can also trigger the touch by touching the edge of the Windows device with the edge of the Android device through a set shortcut. The present application is not limited in this regard.

In addition, after the Source connection transmission service is reported to the Sink end currently found by the Source screen collaboration service, in order to better meet the user requirements, the Sink end currently found and capable of establishing the screen collaboration connection can be displayed on the screen of the Source end. With respect to the window of available Sink ends displayed at Source end, for example, as shown in fig. 5B.

As shown in fig. 5B, in some possible implementations, to ensure that the electronic device desired by the user can be found, 3 pieces of prompt information, such as (1) to (3) shown in fig. 5B, may be displayed in the window.

With continued reference to fig. 5B, in an exemplary, in some possible implementations, the currently discovered Sink ends displayed in the list of available devices may be displayed in descending order according to conditions such as last use, or number of uses/frequency, etc., so that the user may quickly discover the Sink end devices that are frequently used.

It should be understood that the above description is only an example for better understanding of the technical solution of the present embodiment, and is not the only limitation of the present embodiment.

With continued reference to fig. 4, an exemplary embodiment of the present disclosure, after a Source end discovers an available device through a Source connection transmission service, if a user clicks a "connection" option corresponding to a certain available device, such as a "connection" option corresponding to a "GDI" device in fig. 5B, based on a concurrent scheme of a screen collaboration function and a keyboard-mouse sharing function provided in an embodiment of the present disclosure, in a case that a Source screen collaboration service increases a transformation characteristic of switching and compatibility processing with a MagicLink channel, the Source end queries an online state of the interconnection device in a Source device management unit according to a unique identifier of a device corresponding to the available device (subsequently denoted as an interconnection device) reported by the Source connection transmission service.

Specifically, the online status of the interconnection device in this embodiment refers to whether the interconnection device can be found (searched) by the Source access networking unit in the Source end. If the service is found by the Source access networking unit, the screen system service of the interconnection device can be confirmed, and the transformation characteristics of switching and compatibility processing with the magicLink channel are also added, namely, the Source end can establish screen cooperative connection with the interconnection device serving as the Sink end based on the data transmission unit, and the magicLink connection used by the key mouse sharing connection is adopted to establish the screen cooperative connection, so that the concurrency of the screen cooperative function and the key mouse sharing function is realized.

With continued reference to fig. 4, for example, if the Source device management unit does not query the interconnection device on the managed path (local, cloud) after receiving the indication of querying the online status of the interconnection device sent by the Source screen collaboration service, that is, if the interconnection device is not online, the Source device management unit may trigger the Source access networking unit to discover the interconnection device. That is, the Source access networking unit searches the Sink access networking unit in the accessory area, and matches the device with the same device unique identifier as the interconnection device from the searched Sink end.

Illustratively, if the Source access networking unit discovers the interconnection device, the online status of the interconnection device is online. For this case, the subsequent processing will continue according to the step of fig. 4.

For example, if the Source access networking unit does not find the interconnection device, in a possible case, the Source access networking unit may continue to search for Sink access networking units in the attachment area until the interconnection device is found, and feedback to the Source screen collaboration service that the interconnection device is online, and then perform subsequent processing according to the steps of fig. 4. In another possible case, when the Source access networking unit does not search the interconnection device within a set time, the Source access networking unit directly depends on the Source connection transmission service and the Sink connection transmission service of the interconnection device to establish a screen cooperative connection between the Source connection transmission service and the Sink connection transmission service of the interconnection device. The logic of establishing the screen cooperative connection between the Source connection transmission service and the Sink connection transmission service can be referred to the processing flow of fig. 4, in which the connection type is connection type 2, and will not be described herein.

In this embodiment, the Source access networking unit searches the Sink access networking unit of the interconnection device.

Furthermore, it should be noted that, since the version numbers of the data transmission units (MagicLink) adopted by different electronic devices may be different, the data transmission units with different version numbers may support the establishment of different connections, for example, for some versions, such as the data transmission units with versions below 7.2, may only support the establishment of a mouse sharing connection, and not support the establishment of a screen collaboration connection. Therefore, in order to ensure that the Source terminal can establish the MagicLink connection with the Sink data transmission unit of the Sink terminal through the Source data transmission unit, further, the screen collaboration connection between the Source terminal and the Sink terminal is realized based on the MagicLink connection, and when the Source screen collaboration service knows that the interconnection equipment is online, the Source equipment management unit is informed to acquire the version number currently supported by the interconnection equipment.

With continued reference to fig. 4, exemplary version numbers of Sink data transmission units in the interconnect device may be obtained by a Sink screen collaboration service after the device is powered on and written to a Sink device management unit, for example.

With continued reference to fig. 4, the exemplary Source device management unit obtains the version number currently supported by the interconnection device from the Sink device management unit, and in a possible case, determines whether the version number currently supported by the interconnection device matches the version number of the Source end, for example, determines whether the version number of the Sink end is the same as the version number of the Source end. If so, a match is determined.

For example, in another possible case, it may also be determined whether the version number currently supported by the interconnect device is greater than the set version number, or belongs to any version number in the set version number set/list. If the version number is greater than the set version number or belongs to any version number in the set version number set/list, then the two are determined to match.

With continued reference to fig. 4, for example, when the version number supported by the interconnection device matches the version number supported by the Source end, the connection type determined by the Source device management unit is connection type 1, that is, the screen cooperative connection may also be implemented based on the MagicLink connection established by the data transmission unit. Otherwise, determining that the connection type is connection type 2, that is, the screen cooperative connection does not support the implementation of the MagicLink connection established by using the data transmission unit, and the implementation needs to be based on the oneHop connection corresponding to the connection transmission service.

With continued reference to fig. 4, in the case of connection type 1, for example, the Source screen collaboration service triggers, through the Source access networking unit, the Source data transmission unit and the Sink data transmission unit of the interconnection device, and establishes a MagicLink connection between the Source end and the Sink end based on a MagicLink transmission technology.

With continued reference to fig. 4, illustratively, after the Source terminal establishes a MagicLink connection with the Sink terminal, the Sink data transmission unit will pull up the Sink screen collaboration service.

With continued reference to fig. 4, after the Sink screen collaboration service is pulled, the MagicLink connection established between the Sink data transmission unit and the Source data transmission unit performs data negotiation with the Source terminal (data negotiation related to establishing the screen collaboration connection). Regarding the data information to be negotiated in the data negotiation process performed by the Source terminal and Sink terminal, reference may be made to the existing document about screen collaboration, which is not described herein. In this embodiment, the negotiated data is specifically performed through a MagicLink connection, and is not an oneHop connection.

With continued reference to fig. 4, the Sink terminal completes data negotiation with the Source terminal through the MagicLink connection, and the screen collaboration connection between the Sink terminal and the Source terminal is formally established successfully.

Since, in the case of connection type 1, the screen cooperative connection is implemented using the same connection as the mouse shared connection, i.e., the MagicLink connection. Therefore, for the case that the connection type is connection type 1, the Source end and the Sink end can realize concurrent use of the screen coordination function and the keyboard-mouse sharing function, so that the Sink end is used as an external extension screen of the Source end, and a user can also directly operate the Sink end by using the keyboard-mouse of the Source end. Because the whole process does not need to replace the mouse equipment, the mouse device can be greatly convenient for users to use, and the user experience is improved.

Processing logic for connection type 1 is described herein.

With continued reference to fig. 4, in the case of connection type 2, illustratively, the Source screen collaboration service is established with the Sink connection transmission service of the interconnection device through the Source connection transmission service, and based on the oneHop transmission technology, an oneHop connection between the Source terminal and the Sink terminal is established.

With continued reference to fig. 4, illustratively, after the Source end establishes oneHop connection with the Sink end, the Sink connection transport service will pull up the Sink screen collaboration service.

With continued reference to fig. 4, exemplary, after the Sink screen collaboration service is pulled, the oneHop connection established by the Sink connection transport service and the Source connection transport service performs data negotiation with the Source end. Regarding the data information to be negotiated in the data negotiation process performed by the Source terminal and Sink terminal, reference may be made to the existing document about screen collaboration, which is not described herein.

With continued reference to fig. 4, exemplary, after the Sink end completes data negotiation with the Source end through oneHop connection, the screen collaboration connection between the Sink end and the Source end will be formally established successfully.

Since, in the case of connection type 2, the screen cooperative connection adopts a connection different from the mouse shared connection, i.e., one is realized through oneHop connection and one is realized through MagicLink connection. Therefore, for the case that the connection type is connection type 2, the Source terminal and Sink terminal cannot realize the concurrent use of the screen collaboration function and the keyboard-mouse sharing function.

Processing logic for connection type 2 is described herein.

Through the description of fig. 4, it is understood that in the screen collaboration and mouse sharing method provided in the application, the match collaboration connection and the mouse sharing connection can be realized through MagicLink connection. In a scenario where the screen coordination function and the mouse sharing function coexist based on this mode, a switching flow for the same set of mice is shown in fig. 6.

It can be understood that the keyboard and the mouse are matched crossing, and in the case of crossing the mouse, the input of the keyboard is also switched to the screen to which the mouse currently crosses. Therefore, the following directly describes the switching flow of the mouse.

Referring to fig. 6, a scenario in which a mouse sharing and a screen co-existence exist is illustrated, that is, a screen co-connection and a mouse sharing connection are both implemented through a MagicLink connection established by a data transmission unit in a Source end and a data transmission unit in a Sink end.

In addition, as can be seen from fig. 5A, the cooperative mode may include a mirror mode and an extended mode. In this embodiment, taking the collaborative mode selected by the user as the expansion mode, and taking a window serving as an expansion screen of the Windows device on the Android device (e.g. pad) as a floating window as an example.

With continued reference to fig. 6, for exemplary purposes, in order to determine the switching of the mouse, after the Source screen collaboration service informs the Sink screen collaboration service that the currently selected collaboration mode is the expansion mode through the Source data transmission unit and the Sink data transmission unit, the Sink screen collaboration service informs the Sink key mouse that the current collaboration mode of the sharing service is the expansion mode.

The content displayed in the extended screen in the form of a floating window displayed on the Android device is also content in the Windows device. Therefore, when the cursor corresponding to the mouse is located in the screen (hereinafter referred to as the main screen) and the extension screen of the Windows device, the same cursor style, such as the cursor 10a of the arrow style in fig. 7B and 7C, can be adopted. The cursor pattern on the Android device screen (outside the extended screen area) may be the cursor 10B in the dot pattern of fig. 7B and 7A.

And when the corresponding cursors of the mouse are displayed on different screens, the corresponding cursor patterns are different. And the Android screen is provided with an extension screen serving as Windows equipment and a screen area of the Android equipment. Hiding, when the Sink mouse sharing service knows that the current collaborative mode is an expansion mode and the corresponding expansion screen in the expansion mode is in a floating window form, the Sink mouse sharing service can read the size of the expansion screen in the floating window form currently displayed on the screen of the Android device. For example, the coordinates of the top left corner vertex and the bottom right corner vertex of the expansion screen.

With continued reference to fig. 6, the Source screen collaboration service, when setting the collaboration mode to the extended mode, notifies the Source mouse sharing service in addition to the Sink screen collaboration service.

With continued reference to fig. 6, the Source mouse sharing service, after knowing that the current collaborative mode is the extended mode, sets the home screen coordinates and the extended screen coordinates in order to be able to monitor the mouse switching.

As for the coordinates of the home screen, coordinates of the top left corner vertex and the bottom right corner vertex of the home screen may be set. As shown in (1) in fig. 7A, for a case where the main screen size is 1920×1080, the vertex coordinates of the upper left corner of the main screen may be set to (0, 0), and the vertex coordinates of the lower right corner may be set to (1920, 1080).

And for the coordinates of the extension screen, it can be determined by the size of the extension screen read from the Sink-mouse sharing service. For example, in fig. 7A, when the vertex position table of the upper left corner of the extension screen corresponding to the screen of the Android device shown in (1) is (x 1, y 1), and the vertex coordinate of the lower right corner is (x 2, y 2), the vertex coordinate (exStart, eyStart) of the upper left corner of the extension screen set by the Source mouse sharing service may be taken as (x 1, y 1), and the vertex coordinate (exEnd, yend) of the lower right corner may be taken as (x 2, y 2).

With continued reference to FIG. 6, exemplary, based on the above settings, when the mouse moves within the home screen of the Windows device, the Source mouse sharing service perceives movement of the mouse, which will initiate the home screen edge detection procedure. For example, coordinate information of the cursor 10a on the main screen is acquired, and further it is determined whether the cursor 10a has moved to the edge of the main screen, for example, an area with X-axis coordinates 1920 shown in (1) of fig. 7B, based on the coordinate information of the cursor 10 a.

With continued reference to FIG. 6, by way of example, when the X-axis coordinate corresponding to the cursor 10a exceeds the X-axis coordinate of the side edge of the home screen, such as 1920 shown in FIG. 7B (1), it is determined that the cursor 10a is beyond the edge of the home screen, at which point the mouse will traverse, i.e., traverse from the home screen of the Windows device onto the pad screen.

With continued reference to fig. 6, by way of example, upon determining that the cursor 10a is beyond the edge of the home screen, the Source mouse sharing service will initiate a mouse traversal request to the Sink mouse sharing service. Referring to fig. 6, the mouse traversing request is sent to the Source data transmission unit through the Source mouse sharing service, then sent to the Sink data transmission unit by the Source data transmission unit based on the MagicLink transmission technology, and finally sent to the Sink mouse sharing service by the Sink data transmission unit.

With continued reference to fig. 6, an exemplary mouse traversing request carries coordinate information of the cursor 10a, and the set coordinates (exStart, eyStart) and (exEnd, eyEnd) of the extension screen have corresponding relationships with (x 1, y 1) and (x 2, y 2) of the extension screen displayed on the pad screen, respectively. Therefore, according to the coordinate information, the Sink mouse sharing service can calculate the position of the cursor corresponding to the mouse to be displayed on the pad screen (hereinafter referred to as the mouse position). Since the size of the expansion screen displayed in the form of a floating window on the pad is already determined, it can be determined whether the mouse has moved into or out of the expansion screen area according to the currently determined mouse position.

For convenience of explanation, in this embodiment, the mouse is located outside the extended screen area when the mouse crossing occurs for the first time. With continued reference to fig. 6, by way of example, upon determining that the mouse is located outside the extended screen area displayed on the pad, the Sink mouse sharing service will inform the Source mouse sharing service through the Sink data transmission unit, the Source data transmission unit that the current scene mouse has traversed to the pad screen, i.e., the confirmation mouse shown in fig. 6 has traversed to the pad screen, as changed from (1) in fig. 7B to (2) in fig. 7B.

As can be seen from the above description of the cursor patterns on the main screen, the extension screen and the pad screen, when the mouse passes through to the pad screen, the cursor corresponding to the mouse is switched from the cursor 10a displayed on the Windows device screen (1) in fig. 7B to the cursor 10B displayed on the pad screen (2) in fig. 7B.

As can be seen from the switching of the mice shown in fig. 7B (1) and fig. 7B (2), the Source mouse sharing service also needs to hide the cursor 10a displayed on the main screen and display the cursor 10B on the pad screen, specifically, at the determined mouse position of the Sink mouse sharing service, when the mouse passes from the main screen to the pad screen.

With continued reference to fig. 6, for example, after the mouse passes through the gray area of the Android device screen (pad screen) shown in fig. 7B (2), if the user continues to press the mouse coordinates to move the mouse, during the movement of the mouse, the Source mouse sharing service will acquire the current mouse offset, and send the mouse offset to the Sink mouse sharing service through the Source data transmission unit and the Sink data transmission unit.

With continued reference to FIG. 6, the Sink mouse sharing service will recalculate the shifted mouse position based on the received mouse offset and the previously determined mouse coordinates, and determine whether the mouse is moving into the extended screen area, still outside the extended screen area, or moving to the edge of the pad (traversing back to the edge position of the Windows device home screen) based on the recalculated mouse position.

For convenience of explanation, this embodiment takes the case where the mouse moves into the extended screen area as an example. With continued reference to fig. 6, by way of example, when it is determined that the mouse is located in the area of the expansion screen displayed on the pad, the Sink mouse sharing service will inform the Source mouse sharing service through the Sink data transmission unit and the Source data transmission unit that the current scene mouse has traversed into the expansion screen, i.e. the confirmation mouse shown in fig. 6 traverses into the expansion screen, as changed from (1) in fig. 7C to (2) in fig. 7C.

As can be seen from the above description of the cursor patterns on the main screen, the extension screen and the pad screen, when the mouse passes through to the main screen or the extension screen, the cursor pattern corresponding to the mouse is the cursor 10a. Therefore, when the mouse passes through the expansion screen, the cursor corresponding to the mouse is switched from the cursor 10b displayed on the Android screen (gray area) in fig. 7C to the cursor 10a displayed on the expansion screen in fig. 7C (2).

As can be seen from the switching of the mice shown in fig. 7C (1) and fig. 7C (2), when the mouse passes from the pad screen to the expansion screen, the Source mouse sharing service also needs to hide the cursor 10b displayed on the gray area of the pad screen, and display the cursor 10a on the expansion screen, specifically, at the determined mouse position of the Sink mouse sharing service.

In one possible implementation, for a scene where the mouse passes from the pad screen gray area to the expansion screen, the cursor may be moved along with the movement of the mouse, so the cursor 10a on the pad screen that is moved to the expansion screen may be replaced by the cursor 10a directly.

The above description is merely an example for better understanding of the technical solution of the present embodiment, and is not intended to be the only limitation of the present embodiment.

In addition, it should be noted that, since the mouse traverses to the expansion screen, the operation performed in the expansion screen is realized through the screen collaboration service. Therefore, the operation of setting the cursor 10a into the extended screen area needs to be completed by the screen collaboration service when the mouse passes through to the extended screen.

With continued reference to fig. 6, for example, if the user still presses the left mouse button to move after the mouse passes through the extension screen, the Source mouse sharing service senses the movement of the mouse when the mouse moves within the extension screen, and starts the extension screen edge detection process. For example, coordinate information of the cursor 10a on the expansion screen is acquired, and whether the cursor 10a has moved to the edge of the expansion screen, such as an area with X1 or an area with X2 or an area with Y1 or an area with Y2, which are shown in (1) of fig. 7C, is determined according to the coordinate information of the cursor 10 a. That is, it is determined whether the cursor 10a moves to the upper edge, or the lower edge, or the left edge, or the right edge, or the like of the extension screen, as in the four black line areas of the extension screen shown in (2) of fig. 7C.

With continued reference to FIG. 6, by way of example, when the coordinate information of the mouse pair is outside the extended screen area, such as back to the gray area of the pad screen, it is determined that the mouse is beyond the edge of the extended screen, at which point the mouse will traverse, i.e., traverse from the extended screen to the gray area of the pad screen.

With continued reference to FIG. 6, by way of example, upon determining that the mouse is beyond the edge of the extension screen, the Source mouse sharing service will initiate a mouse pass-through request to the Sink mouse sharing service. Referring to fig. 6, the mouse traversing request is sent to the Source data transmission unit through the Source mouse sharing service, then sent to the Sink data transmission unit by the Source data transmission unit based on the MagicLink transmission technology, and finally sent to the Sink mouse sharing service by the Sink data transmission unit.

With continued reference to fig. 6, exemplary mouse traversing requests arrive at the Sink-mouse sharing service, which will re-determine the current mouse position of the mouse and determine whether the mouse is moving outside the extended screen area, still within the extended screen area, or traversing directly back to the home screen based on the re-determined mouse position.

For convenience of explanation, the mouse is taken as an example of the mouse being located outside the extended screen area. With continued reference to fig. 6, by way of example, upon determining that the mouse is located in the gray area of the pad screen, the Sink mouse sharing service will inform the Source mouse sharing service through the Sink data transmission unit, the Source data transmission unit that the current scene mouse has traversed to the pad screen, i.e., the confirmation mouse illustrated in fig. 6 has traversed to the pad screen, as changed from (2) in fig. 7C to (1) in fig. 7C.

As can be seen from the above description of the cursor patterns on the main screen, the extension screen and the pad screen, when the mouse passes through to the pad screen, the cursor corresponding to the mouse is switched from the cursor 10a displayed on the extension screen in fig. 7C (2) to the cursor 10b displayed on the pad screen in fig. 7C (1).

As can be seen from the switching of the mice shown in fig. 7C (2) and fig. 7C (1), the Source mouse sharing service also needs to hide the cursor 10a displayed on the extension screen and display the cursor 10b on the pad screen, specifically, at the determined mouse position of the Sink mouse sharing service, when the mouse passes from the extension screen to the pad screen.

In addition, it should be noted that, in practical applications, the fixed task bars may be displayed or set around the expansion screen, such as the upper edge, the lower edge, the left edge, and the right edge. For these fixed taskbar operations, the function is shared based on the keyboard and mouse, rather than the screen collaboration function. Thus, in some possible implementations, the mouse may be configured to move from within the extended screen area to the top, bottom, left, and right edges of the extended screen, confirming that the mouse has traversed, i.e., from within the extended screen to the pad screen.

Accordingly, when traversing from the pad screen to the extension screen, the mouse can be confirmed to traverse from the pad screen to the extension screen only when the mouse moves into the extension screen from the upper edge, the lower edge, the left edge and the right edge of the extension screen.

Therefore, based on the method for screen collaboration and sharing concurrency of the key mouse, the problem that the existing screen collaboration connection and the key mouse sharing connection have conflicts and cannot be concurrent is solved. Meanwhile, based on the switching flow of the mouse, a user can control the electronic equipment which establishes the screen cooperative connection based on the same set of key mice.

Furthermore, it will be appreciated that the electronic device, in order to achieve the above-described functions, comprises corresponding hardware and/or software modules that perform the respective functions. The steps of an algorithm for each example described in connection with the embodiments disclosed herein may be embodied in hardware or a combination of hardware and computer software. Whether a function is implemented as hardware or computer software driven hardware depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application in conjunction with the embodiments, but such implementation is not to be considered as outside the scope of this application.

In addition, it should be noted that, in an actual application scenario, the concurrent method of screen collaboration and mouse sharing provided in the foregoing embodiments implemented by the electronic device may also be implemented by a chip system included in the electronic device, where the chip system may include a processor. The chip system may be coupled to a memory such that the chip system, when running, invokes a computer program stored in the memory, implementing the steps performed by the electronic device described above. The processor in the chip system can be an application processor or a non-application processor.

In addition, the embodiment of the application further provides a computer readable storage medium, and the computer storage medium stores computer instructions, which when executed on the electronic device, cause the electronic device to execute the related method steps to implement the concurrent method of screen collaboration and keyboard and mouse sharing in the embodiment.

In addition, the embodiment of the application also provides a computer program product, when the computer program product runs on the electronic device, the electronic device is caused to execute the related steps, so as to realize the concurrent method of screen collaboration and mouse sharing in the embodiment.

In addition, embodiments of the present application also provide a chip (which may also be a component or module) that may include one or more processing circuits and one or more transceiver pins; the processing circuit executes the relevant method steps to realize the concurrent method of screen collaboration and keyboard and mouse sharing in the embodiment so as to control the receiving pin to receive signals and control the sending pin to send signals.

In addition, as can be seen from the foregoing description, the electronic device, the computer-readable storage medium, the computer program product, or the chip provided in the embodiments of the present application are used to perform the corresponding methods provided above, and therefore, the advantages achieved by the method can refer to the advantages in the corresponding methods provided above, which are not repeated herein.

The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A concurrent method of screen collaboration and keyboard-mouse sharing, applied to a system comprising a first electronic device and a second electronic device, the method comprising:

the first electronic device responds to the operation of establishing screen cooperative connection, and searches the second electronic device based on a first connection technology;

when the second electronic device is searched based on the first connection technology, the first electronic device determines whether the second electronic device can be searched through a second connection technology;

when the second electronic equipment is searched based on the second connection technology, the first electronic equipment establishes screen cooperative connection with the second electronic equipment based on the second connection technology; the first electronic equipment can also establish a mouse sharing connection with the second electronic equipment based on the second connection technology;

when the second electronic equipment is not searched based on the second connection technology, the first electronic equipment establishes screen cooperative connection with the second electronic equipment based on the first connection technology;

the first connection technology and the second connection technology are mutually exclusive, and at the same time, the first electronic device can only establish communication connection with the second electronic device based on the first connection technology or the second connection technology, and the communication connection comprises the screen cooperative connection and the mouse sharing connection.

2. The method of claim 1, wherein upon searching for the second electronic device based on the first connection technique, the method further comprises:

the first electronic device determines whether the second electronic device supports the establishment of the screen cooperative connection by adopting the second connection technology;

when the second electronic equipment supports the establishment of the screen cooperative connection by adopting the second connection technology, the first electronic equipment executes the step of establishing the screen cooperative connection with the second electronic equipment based on the second connection technology;

and when the second electronic equipment is determined not to support the establishment of the screen cooperative connection by adopting the second connection technology, the first electronic equipment executes the step of establishing the screen cooperative connection with the second electronic equipment based on the first connection technology.

3. The method of claim 2, wherein the first electronic device determining whether the second electronic device supports establishing the screen collaboration connection using the second connection technique comprises:

the first electronic equipment acquires a version number supported by the second electronic equipment;

The first electronic device determines whether the version number supported by the second electronic device is the same as the version number supported by the first electronic device; the version numbers supported by the first electronic device and the version numbers supported by the second electronic device are version numbers of functional modules realized based on the second connection technology;

when the version number supported by the second electronic equipment is determined to be the same as the version number supported by the first electronic equipment, determining that the second electronic equipment supports to establish the screen cooperative connection by adopting the second connection technology;

and when the version number supported by the second electronic equipment is different from the version number supported by the first electronic equipment, determining that the second electronic equipment does not support the establishment of the screen cooperative connection by adopting the second connection technology.

4. The method of claim 2, wherein the first electronic device determining whether the second electronic device supports establishing the screen collaboration connection using the second connection technique comprises:

the first electronic device determines whether the version number supported by the second electronic device is the same as a preset version number; the version number supported by the second electronic device and the preset version number are version numbers of functional modules realized based on the second connection technology;

When the version number supported by the second electronic equipment is identical to the preset version number, determining that the second electronic equipment supports to establish the screen cooperative connection by adopting the second connection technology;

and when the version number supported by the second electronic equipment is determined to be different from the preset version number, determining that the second electronic equipment does not support the establishment of the screen cooperative connection by adopting the second connection technology.

5. The method according to any one of claims 2 to 4, wherein the version number supported by the second electronic device is written by a screen collaboration service of the second electronic device into a functional module integrating the second connection technology when the second electronic device is started up.

6. The method according to any one of claims 1 to 5, wherein upon searching for the second electronic device based on the second connection technology, the first electronic device establishes a screen collaboration connection with the second electronic device based on the second connection technology, the method further comprising:

based on the second connection technology, the first electronic device establishes a mouse sharing connection with the second electronic device.

7. The method of claim 6, wherein in a scenario where screen collaboration and keyboard sharing are concurrent, and where a floating window is displayed on a screen of the second electronic device as an extended screen of the first electronic device, the method further comprises:

when the first electronic device detects that a mouse moves in a main screen of the first electronic device, starting the edge detection of the main screen; the mouse is connected with the first electronic equipment;

when the mouse moves out of the main screen through the edge detection of the main screen, the first electronic equipment sends a first mouse crossing request to the second electronic equipment, wherein the first mouse crossing request carries a first mouse offset;

after receiving the first mouse traversing request, the second electronic device calculates a first mouse position according to the first mouse offset;

when the first mouse position is located outside the expansion screen area, the second electronic device makes a first confirmation message aiming at the first mouse crossing request and sends the first confirmation message to the first electronic device; wherein the first confirmation message indicates that the mouse traverses from the home screen to a screen of the second electronic device;

And hiding the mouse cursor of the first style displayed on the main screen by the first electronic equipment, and displaying the mouse cursor of the second style on the screen of the second electronic equipment based on the key mouse sharing function.

8. The method of claim 7, wherein the method further comprises:

when the first mouse position is located in the expansion screen area, the second electronic device makes a second confirmation message aiming at the first mouse crossing request and sends the second confirmation message to the first electronic device; wherein the second confirmation message indicates that the mouse passes from the main screen into the expansion screen;

and hiding the mouse cursor of the first style displayed on the main screen by the first electronic equipment, and displaying the mouse cursor of the first style on the expansion screen based on the screen cooperation function.

9. The method of claim 7, wherein the method further comprises:

in the process of displaying the mouse cursor of the second pattern in the screen of the second electronic device, when the mouse is continuously moved, the first electronic device acquires a second mouse offset of the mouse and sends the second mouse offset to the second electronic device;

The second electronic device calculates a second mouse position according to the second mouse offset;

when the second mouse position is positioned in the expansion screen area, the second electronic equipment makes a third confirmation message; wherein the third confirmation message indicates that the mouse passes through from the screen of the second electronic device into the expansion screen;

the first electronic device changes the second-style mouse cursor into the first-style mouse cursor, and sets the first-style mouse cursor into the expansion screen based on the screen cooperation function.

10. The method according to claim 8 or 9, characterized in that the method further comprises:

in the process of displaying the mouse cursor of the first style in the expansion screen, when the mouse is continuously moved, the first electronic equipment starts the edge detection of the expansion screen;

when the mouse moves out of the expansion screen through the edge detection of the expansion screen, the first electronic device sends a second mouse crossing request to the second electronic device, and the second mouse crossing request carries a third mouse offset;

after receiving the second mouse traversing request, the second electronic device calculates a third mouse position according to the third mouse offset;

When the third mouse position is located outside the expansion screen area, the second electronic device makes a fourth confirmation message aiming at the second mouse traversing request and sends the fourth confirmation message to the first electronic device; wherein the fourth confirmation message indicates that the mouse traverses from the expansion screen to the screen of the second electronic device;

and hiding the mouse cursor of the first type displayed on the expansion screen by the first electronic device, and displaying the mouse cursor of the second type on the screen of the second electronic device based on the key mouse sharing function.

11. An electronic device, the electronic device comprising: a memory and a processor, the memory and the processor coupled; the memory stores program instructions that, when executed by the processor, cause the electronic device to perform the concurrent method of screen collaboration and keyboard and mouse sharing of any one of claims 1 to 10.

12. A computer readable storage medium comprising a computer program which, when run on an electronic device, causes the electronic device to perform the concurrent method of screen collaboration and keyboard and mouse sharing of any one of claims 1 to 10.