WO2022100239A1

WO2022100239A1 - Device cooperation method, apparatus and system, electronic device and storage medium

Info

Publication number: WO2022100239A1
Application number: PCT/CN2021/116501
Authority: WO
Inventors: 杨俊拯; 缪敬; 谭柯; 钟卫东
Original assignee: Oppo广东移动通信有限公司
Priority date: 2020-11-16
Filing date: 2021-09-03
Publication date: 2022-05-19
Also published as: CN112286618A

Abstract

A device cooperation method, apparatus and system, an electronic device and a storage medium. The method comprises: a first device displaying a first interface, the first interface being used to display a function list associated with a target application, the target application running on the first device, and the function list comprising a separation control option of the target application (S410); in response to a first selection operation for the separation control option, displaying a second interface, the second interface being used to display a first associated device list associated with the first device (S420); and in response to a second selection operation for at least one associated device in the first associated device list, enable the target application of the first device to be controlled by the at least one associated device (S430). The method can select at least one associated device from multiple devices to implement a selection function performed by a user on a first device, so that the first device can integrate functions provided by multiple associated devices, and simultaneously call functions of multiple devices of different types, to achieve cooperative work of the multiple devices.

Description

Device cooperation method, apparatus, system, electronic device and storage medium

technical field

The present application relates to the field of electronic technologies, and in particular, to a device cooperation method, apparatus, system, electronic device, and storage medium.

Background technique

With the growth of the number of information devices, in some application scenarios, users need to combine the services of different devices to jointly complete and execute a task, so as to improve the efficiency of task execution. For example, in the process of using a mobile phone to listen to music, in order to improve the effect of audio playback, the user can play the sound through a smart speaker.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a device cooperation method, apparatus, system, electronic device, and storage medium.

In a first aspect, an embodiment of the present application provides a device coordination method, and the method includes:

The first device displays a first interface, the first interface is used to display a function list associated with a target application, the target application runs on the first device, and the function list includes the separation of the target application control options;

In response to the first selection operation for the separation manipulation option, the first device displays a second interface for displaying a first associated device list associated with the first device;

In response to a second selection operation for at least one associated device in the first associated device list, causing the target application of the first device to be manipulated by the at least one associated device.

In a second aspect, an embodiment of the present application provides an apparatus for coordinating equipment, and the apparatus includes:

A display unit, configured to display a first interface, where the first interface is configured to display a function list associated with a target application, the target application running on the first device, and the function list includes the target application separate control options;

The display unit is further configured to display a second interface in response to a first selection operation for the separation manipulation option, where the second interface is configured to display a list of first associated devices associated with the first device;

A processing unit, configured to cause the target application of the first device to be controlled by the at least one associated device in response to a second selection operation for at least one associated device in the first associated device list.

In a third aspect, embodiments of the present application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be processed by the above-mentioned processing The above program includes instructions for executing steps in any method of the first aspect of the embodiments of the present application.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the computer program as described in the first embodiment of the present application. In one aspect some or all of the steps described in any method.

In a fifth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute as implemented in the present application. Examples include some or all of the steps described in any method of the first aspect. The computer program product may be a software installation package.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

1 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

2 is a schematic diagram of a software structure of an electronic device provided by an embodiment of the present application;

3a is a schematic structural diagram of a device coordination system provided by an embodiment of the present application;

3b is a schematic structural diagram of another device coordination system provided by an embodiment of the present application;

4 is a schematic flowchart of a device coordination method provided by an embodiment of the present application;

5a is a schematic diagram of a first interface display provided by an embodiment of the present application;

5b is a schematic diagram of a first interface display provided by an embodiment of the present application;

5c is a schematic diagram of a second interface display provided by an embodiment of the present application;

5d is a schematic diagram of another second interface display provided by an embodiment of the present application;

5e is a schematic diagram of another second interface display provided by an embodiment of the present application;

5f is a schematic diagram of another second interface display provided by an embodiment of the present application;

6a is a schematic structural diagram of a function provided by an electronic device provided by an embodiment of the present application;

6b is a schematic structural diagram of a multi-device collaboration provided by an embodiment of the present application;

7a is a schematic structural diagram of another device coordination system provided by an embodiment of the present application;

7b is a schematic structural diagram of another device coordination system provided by an embodiment of the present application;

8 is a schematic flowchart of invoking a target function on an associated device provided by an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a device coordination apparatus provided by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, unless otherwise stated, “/” means or means, for example, A/B can mean A or B; “and/or” in this document is only a description of the associated object The association relationship of , indicates that there can be three kinds of relationships, for example, A and/or B, can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "plurality" refers to two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first", "second" may expressly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise specified, "plurality" means two or more.

The device collaboration method provided in the embodiments of the present application may be applied to handheld devices, vehicle-mounted devices, wearable devices, augmented reality (AR) devices, virtual reality (VR) devices, projection devices, projectors, or connected to Other devices of the wireless modem can also be various specific forms of user equipment (UE), terminal device (terminal device), mobile phone (smart phone), smart screen, smart TV, smart watch, notebook computer, smart audio , camera, gamepad, mouse, microphone, station (station, STA), access point (access point, AP), mobile station (mobile Station, MS), personal digital assistant (personal digital assistant, PDA), personal computer ( Personal computer, PC) or a relay device and other terminal devices or servers, the embodiments of the present application do not impose any restrictions on the specific types of terminal devices and servers.

For example, the terminal device may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (Session Initiation Protocol, SIP) phone, a Wireless Local Loop (WLL) station , Personal Digital Assistant (PDA) devices, handheld devices with wireless communication capabilities, computing devices or other processing devices connected to wireless modems, in-vehicle devices, connected car terminals, computers, laptop computers, handheld communications equipment, handheld computing equipment, satellite wireless equipment, wireless modem cards, television set top boxes (STBs), customer premise equipment (CPEs) and/or other equipment for communicating on wireless devices and Next-generation communication devices, for example, mobile terminals in a 5G network or mobile terminals in a future evolved Public Land Mobile Network (Public Land Mobile Network, PLMN) network, etc.

As an example and not a limitation, when the terminal device is a wearable device, the wearable device may also be a general term for the intelligent design of daily wear and the development of wearable devices using wearable technology, such as glasses, gloves, Watches, clothing and shoes, etc. A wearable device is a portable device that is worn directly on the body or integrated into the user's clothing or accessories. Wearable device is not only a hardware device, but also realizes powerful functions through software support, data interaction, and cloud interaction. In a broad sense, wearable smart devices include full-featured, large-scale, complete or partial functions without relying on smart phones, such as smart watches or smart glasses, and only focus on a certain type of application function, which needs to be used in conjunction with other devices such as smart phones. , such as various types of smart bracelets and smart jewelry that monitor physical signs.

Illustratively, two electronic devices, a notebook computer and a mobile phone, are taken as examples. When the laptop and mobile phone are connected through wireless communication technology (such as Bluetooth, Wi-Fi, Zigbee, near field communication, etc.) or data cable (such as USB data cable), when the user finishes the game through the mobile phone, he can call it through device collaboration. The laptop computer is used to display the game screen of the mobile phone; or, when the user finishes the game on the laptop computer, the mobile phone can be called to control the game on the laptop computer by means of device cooperation.

In the embodiment of the present application, the first device performing device coordination and the associated device may be directly connected, for example, through Bluetooth, WiFi, etc., to realize the direct connection between the two electronic devices; Connect with other electronic devices such as cloud servers to achieve indirect connection. In the process of device coordination, the connection between the two electronic devices may be switched between direct connection and indirect connection, which is not limited in this embodiment of the present application.

Exemplarily, FIG. 1 shows a schematic structural diagram of an electronic device 100 . Taking the electronic device as a mobile phone as an example, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a USB interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile phone Communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, and SIM card Interface 195, etc. The sensor module 180 may include a gyro sensor 180A, an acceleration sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an ambient light sensor 180E, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, and a touch sensor 180K (of course, the electronic device 100 Other sensors may also be included, such as temperature sensors, pressure sensors, distance sensors, bone conduction sensors, etc. (not shown).

It can be understood that, the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or Neural-network Processing Unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The controller may be the nerve center and command center of the electronic device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

The processor 110 may execute the screen projection method provided by the embodiments of the present application, so as to enrich the screen projection function, improve the flexibility of the screen projection, and improve the user experience. The processor 110 may include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU may cooperate to execute the screen projection method provided by the embodiments of the present application. For example, some algorithms in the screen projection method are executed by the CPU, and another part of the algorithms are executed by the GPU. for faster processing efficiency.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one. The display screen 194 may be used to display information entered by or provided to the user as well as various graphical user interfaces (GUIs). For example, the display 194 may display photos, videos, web pages, or documents, or the like. As another example, display 194 may display a graphical user interface. The graphical user interface includes a status bar, a hideable navigation bar, a time and weather widget (widget), and an application icon, such as a browser icon. The status bar includes operator name (eg China Mobile), mobile network (eg 4G), time and remaining battery. The navigation bar includes a back button icon, a home button icon, and a forward button icon. In addition, it can be understood that, in some embodiments, the status bar may further include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like. It can also be understood that, in other embodiments, the graphical user interface may further include a Dock bar, and the Dock bar may include commonly used application icons and the like. When the processor detects a touch event of the user's finger (or stylus, etc.) on an application icon, in response to the touch event, the processor opens the user interface of the application corresponding to the application icon, and displays the user interface on the display 194. The user interface of the application.

In the embodiment of the present application, the display screen 194 may be an integrated flexible display screen, or a spliced display screen composed of two rigid screens and a flexible screen located between the two rigid screens. After the processor 110 executes the screen projection method provided by the embodiment of the present application, the processor 110 may control the external audio output device to switch the output audio signal.

Camera 193 (front camera or rear camera, or one camera can be both front camera and rear camera) is used to capture still images or video. Generally, the camera 193 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex or concave) for collecting light signals reflected by the object to be photographed, and transmitting the collected light signals to the image sensor . The image sensor generates an original image of the object to be photographed according to the light signal.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area may store operating system, code of application programs (such as camera application, WeChat application, etc.), and the like. The storage data area may store data created during the use of the electronic device 100 (such as images, videos, etc. captured by a camera application) and the like.

The internal memory 121 may also store one or more computer programs corresponding to the screen projection method provided by the embodiments of the present application. The one or more computer programs are stored in the aforementioned memory 211 and configured to be executed by the one or more processors 110, the one or more computer programs include instructions, the computer programs may include an account verification module, priority Compare modules. The account verification module is used to authenticate the system authentication accounts of other terminal devices in the local area network; the priority comparison module can be used to compare the priority of the audio output request service and the priority of the current output service of the audio output device. The state synchronization module can be used to synchronize the device state of the audio output device currently connected by the terminal device to other terminal devices, or synchronize the device state of the audio output device currently connected by other devices to the local. When the code of the screen projection method stored in the internal memory 121 is executed by the processor 110, the processor 110 may control the sending end to process the screen projection data.

In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

Of course, the code of the screen projection method provided by the embodiment of the present application may also be stored in an external memory. In this case, the processor 110 may run the code of the screen projection method stored in the external memory through the external memory interface 120, and the processor 110 may control the sending end to perform screen projection data processing.

The gyroscope sensor 180A can be used to determine the motion attitude of the electronic device 100 . In some embodiments, the angular velocity of electronic device 100 about three axes (ie, x, y, and z axes) may be determined by gyro sensor 180A. That is, the gyro sensor 180A can be used to detect the current motion state of the electronic device 100, such as shaking or stillness.

When the display screen in the embodiment of the present application is a foldable screen, the gyro sensor 180A can be used to detect a folding or unfolding operation acting on the display screen 194 . The gyroscope sensor 180A may report the detected folding operation or unfolding operation to the processor 110 as an event to determine the folding state or unfolding state of the display screen 194 .

The acceleration sensor 180B can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). That is, the gyro sensor 180A can be used to detect the current motion state of the electronic device 100, such as shaking or stillness. When the display screen in the embodiment of the present application is a foldable screen, the acceleration sensor 180B can be used to detect a folding or unfolding operation acting on the display screen 194 . The acceleration sensor 180B may report the detected folding operation or unfolding operation to the processor 110 as an event to determine the folding state or unfolding state of the display screen 194 .

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The mobile phone emits infrared light outward through light-emitting diodes. Phones use photodiodes to detect reflected infrared light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the phone. When insufficient reflected light is detected, the phone can determine that there are no objects near the phone. When the display screen in the embodiment of the present application is a stackable screen, the proximity light sensor 180G can be arranged on the first screen of the foldable display screen 194, and the proximity light sensor 180G can detect the first screen according to the optical path difference of the infrared signal. The size of the folding or unfolding angle between the screen and the second screen.

The gyro sensor 180A (or the acceleration sensor 180B) may send the detected motion state information (such as angular velocity) to the processor 110. The processor 110 determines whether the current state is the hand-held state or the tripod state based on the motion state information (for example, when the angular velocity is not 0, it means that the electronic device 100 is in the hand-held state).

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100 , which is different from the location where the display screen 194 is located.

Exemplarily, the display screen 194 of the electronic device 100 displays a main interface, and the main interface includes icons of multiple applications (such as a camera application, a WeChat application, etc.). The user clicks the icon of the camera application in the main interface through the touch sensor 180K, which triggers the processor 110 to start the camera application and turn on the camera 193 . Display screen 194 displays an interface of a camera application, such as a viewfinder interface.

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 . In this embodiment of the present application, the mobile communication module 150 may also be used for information interaction with other terminal devices, that is, sending screen projection related data to other terminal devices, or the mobile communication module 150 may be used to receive a screen projection request, and send the received screen projection request. The screencasting request is encapsulated into a message in the specified format.

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 . In the embodiment of the present application, the wireless communication module 160 is configured to establish a connection with the receiving end, and display the screen projection content through the receiving end. Alternatively, the wireless communication module 160 may be used to access an access point device, send a message corresponding to a screen projection request to other terminal devices, or receive a message corresponding to an audio output request sent from other terminal devices.

In addition, the electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc. The electronic device 100 may receive the key 190 input and generate the key signal input related to user settings and function control of the electronic device 100 . The electronic device 100 may use the motor 191 to generate vibration alerts (eg, vibration alerts for incoming calls). The indicator 192 in the electronic device 100 may be an indicator light, which may be used to indicate a charging state, a change in power, and may also be used to indicate a message, a missed call, a notification, and the like. The SIM card interface 195 in the electronic device 100 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 .

It should be understood that, in practical applications, the electronic device 100 may include more or less components than those shown in FIG. 1 , which are not limited in the embodiments of the present application. The illustrated electronic device 100 is only an example, and the electronic device 100 may have more or fewer components than those shown, two or more components may be combined, or a different configuration of components may be present. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

Exemplarily, FIG. 2 shows a software structural block diagram of the electronic device 100 . The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer. The application layer can include a series of application packages.

As shown in Figure 2, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 2, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: surface manager (surface manager), media library (media library), 3D graphics processing library (eg: OpenGL ES), 2D graphics engine (eg: SGL), etc.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

At present, through Miracast and DLNA screen projection technology, multimedia data (picture output and audio output) can be put into a certain multimedia peripheral, such as put into a TV, use the TV screen to play the picture, and the TV speaker to play the sound; or put it into a smart speaker, Play sound with your smart speaker. However, Miracast and DLNA screen projection technology can only realize the launch of a single category of equipment, and cannot use the services of multiple devices to work together.

In order to solve the above problems, the present application proposes a device coordination solution, which implements the function selected by the user on the first device by selecting at least one associated device from multiple devices, so that the first device can integrate the functions provided by multiple associated devices. function, and call the functions of multiple different types of devices at the same time to achieve the collaborative work of multiple devices.

Exemplarily, the technical solutions of the embodiments of the present application may be applied to the device cooperation system 30 as shown in FIG. 3a. The device cooperation system 30 may include a first device 310 and a plurality of associated devices 320 . The plurality of associated devices 320 may include electronic device 320A, electronic device 320B, and electronic device 320C. Meanwhile, the first device 310 and the plurality of associated devices 320 may be connected to each other through wireless network or wired data communication.

Specifically, the first device 310 and the multiple associated devices 320 may be devices under the same user account. For example, when a user logs in to a mobile phone, desktop computer, smart screen, laptop computer, relay device and smart watch using the same user account, the first device 310 and/or the associated device 320 may be a mobile phone, desktop computer, smart screen, Notebook computer, relay device and smart watch, and the mobile phone, desktop computer, smart screen, notebook computer, relay device and smart watch can communicate with each other through wireless network.

Specifically, the first device 310 and multiple associated devices 320 may be connected to the same WLAN network through a relay device (such as a router). For example, when the user connects the mobile phone, desktop computer, smart screen, notebook computer and smart watch to the Wi-Fi network provided by the relay device, the first device 310 and the associated device 320 may include the mobile phone, desktop computer, smart screen , laptop computer, relay device and smart watch, and the mobile phone, desktop computer, smart screen, laptop computer, relay device and smart watch form a WLAN network, so that each device in the WLAN network can be implemented through the relay device. communicate with each other.

Further, the first device can call at least one associated device to display a screen of its currently running application, and the first device can also call at least one associated device to operate its current application. For example, the first device 310 invokes the electronic device 320A to display a screen, while the first device 310 invokes the electronic device 320B to operate the current application and invokes the electronic device 320C to play a sound.

It should be understood that the device coordination system 30 may also include other numbers of electronic devices, which are not specifically limited herein.

Exemplarily, the technical solutions of the embodiments of the present application may be applied to the device coordination system 30 as shown in FIG. 3b. The device coordination system 30 may include a first device 310 , multiple associated devices 320 and a cloud device 330 . The first device 310 and the multiple associated devices 320 may be devices under the same user account, and the cloud device 330 is used to manage the associated devices under the same user account, and store the function list of the first device 310 and the multiple associated devices 320 and authorization information. When the first device 310 requests device collaboration, it can acquire multiple associated devices 320 that meet the functional requirements of the first device from the cloud device.

Please refer to FIG. 4 , which is a schematic flowchart of a device coordination method provided by an embodiment of the present application. As shown in FIG. 4 , the device coordination method includes the following operations.

S410. The first device displays a first interface, where the first interface is used to display a function list associated with a target application, the target application runs on the first device, and the function list includes the target application 's separate manipulation options.

In this embodiment of the present application, when the user opens an application on the first device, the user can initiate a related collaborative process through any preset operation. The preset arbitrary operation may be set by the user, or may be set by the system of the first device, which is not limited in this embodiment of the present application. For example, on a mobile phone, it can be set to evoke a collaborative process by long-pressing the screen of a currently running application; on a computer, it can be set to evoke a collaborative process by clicking the right mouse button on the running application. After the collaborative process is aroused, the system will display the first window on the current interface, and the content of the first window is related to the user, the current application, and the first device.

Optionally, the function list further includes a screen separation option and/or a sound separation option.

It is understandable that the way each device obtains the service list will be implemented differently on different devices, and some will be somewhat coupled with the mechanism of the operating system. The timing of obtaining the service list and reporting the service list may vary on different devices. different.

Among them, for different devices and target applications, the functions included in the function list in the target application are different. After starting the collaboration process, the first device used by the user may determine a function list that needs to be coordinated under the target application, and the function list may include separate control options, separate screen options, and separate sound options. For example, if the current target application is a game application, the function list may include separate control options, separate screen options, and separate sound options. When the current target application is Photo Album, the function list may include a split control option and a split screen option. When the current target application is a voice call, the function list may include a separation control option and a separation sound option.

Further, with respect to the realization functions of the target application, the function list may further include separate positioning options, separate payment options, separate search options, and the like.

In actual operation, the application can achieve a variety of tasks, and the list of functions required to achieve each task may be different. For example, when the target application is social software, when the user uses the social software to make a video call, at this time The required function list of the target application includes a separation control option, a separation screen option, and a separation sound option; when the user uses the social software to make a voice call, the required function list of the target application includes a separation control option and a separation sound option. Therefore, the method of determining the function list of the current target application may include, but is not limited to: starting a certain APP on the first device used by the user, executing a certain task in a certain APP on the first device used by the user, During the process of a device performing a certain task, the user inputs a trigger instruction or the like through a set method.

Exemplarily, the manner in which the user evokes the collaborative process may also include other triggering manners, such as Near Field Communication (NFC), Ultra Wide Band (UWB), etc. Taking NFC as an example, when the user is in the When playing games on the mobile phone, the user can touch the TV, and the TV can be determined as an associated device that meets the needs of mobile phone collaboration; the user touches the smart speaker again, and the smart speaker can be determined as an associated device that meets the needs of mobile phone collaboration. ; The user touches the computer again, and the computer or the device carried on the computer can be determined as an associated device that meets the needs of mobile phone collaboration.

Exemplarily, the manner in which the user evokes the collaborative process may also include a non-contact specific gesture to trigger. The first device can also detect a non-contact gesture operation near it, and trigger the display of the first interface when the non-contact gesture operation meets the preset first device detection information generation conditions, and then determine the first interface according to the user's selection. A device requires assisted functions and at least one associated device that satisfies the assisted functions. The user only needs to perform a non-contact gesture operation near the first device to start the assisting operation process, and does not need to touch the first device and the associated device, and the operation is convenient and user-friendly.

It is worth noting that the first device and the associated device of the present invention are only functionally classified. In practical applications, each device in the device cooperation system may have the functions of a first device and an associated device, and when performing a cooperation response, it can be selected as the first device or as the associated device according to specific conditions.

It should be noted that while the first device controls multiple associated devices to perform cooperative processing, it can also participate in the cooperative processing. Specifically, the at least one associated device and the first device perform cooperative processing according to respective cooperation response information.

Exemplarily, the first interface may be suspended at a fixed position of the current interface. As shown in FIG. 5a, the starting coordinate position 1 of the upper left corner and the coordinate position 2 of the lower right corner of the first interface can be set by the user, or can be set by the system of the first electronic device, which is not limited in this application.

Exemplarily, the display position of the first interface may also be determined according to the display direction of the screen of the first device. For example, when the screen of the first device is displayed in a landscape orientation, the display position of the first interface may be as shown in Figure 5a; when the screen of the first device is displayed in a portrait orientation, the display position of the first interface may be as shown in Figure 5a 5b.

Exemplarily, the display position of the first interface can be moved. When the user drags or slides the first interface, the first interface may move according to the user's dragging direction or delineation direction.

Exemplarily, the display position of the first interface may be random. The display position of the first interface may be the position displayed by the user last time, or may be the display position randomly determined by the system, or may be the position where the user touches the screen, which is not limited in this embodiment of the present application.

Optionally, a device operating system and/or an application program in the first device provides the function list.

In this embodiment of the present application, the first device may include an application program provided by an operating system of the device, for example, voice call, sound playback, location positioning, and the like. The first device may also include an application for the user to apply his own business service, for example, an application downloaded by the user for social networking, an application for watching movies, an application for listening to music, and the like. Each application has its own list of features provided. Since the functions in the function list can be divided into services provided by the device operating system on the first device and applications that apply their own business services, when the user starts a collaborative process on different applications, the function list will follow The list of devices under the username varies with different applications. Only valid functions will be displayed on the first screen.

Wherein, the method further includes: when the first device is started, the first device scans the device operating system and/or all the application programs to obtain a service list of the first device, the service The list includes a list of functions provided by the device operating system and/or all the applications; the first device reports the service list to the cloud device.

Specifically, when the first device starts up, it scans the application programs developed in the given manner on the first device, and records the functions that these application programs can provide in the form of a list. At the same time, when the application program is installed or uninstalled, the first device can monitor it and record the monitored content. When the device operating system on the first device and/or the functions provided in all the application programs change, it is reported to the cloud device to synchronize these changes.

Further, a device under the cloud device management can also upload its own service list to the cloud device, and the cloud device can store these service lists uploaded by each device according to the granularity of devices, applications, and functions. When the first device needs other devices to cooperate to complete a certain function, the first device can directly search for the device that can realize the function that the first device needs to coordinate through the cloud device without the adaptation of the third-party application, and use the system application method to adapt The ability to configure devices enables third-party applications to achieve collaboration between devices without any adaptation during the development phase.

S420. In response to the first selection operation for the separation manipulation option, the first device displays a second interface, where the second interface is configured to display a first associated device list associated with the first device.

Wherein, the first selection operation is mainly used to select the function of the target application that needs to perform cooperative operation from the function list. The first selection operation may be any preset operation, may be set by the user, or may be set by the system of the first electronic device, which is not limited in this application. According to different first devices and different display modes of the first device, the first selection operation may include various implementation forms. For example, for a first device that can support a touch screen, such as a smart phone, the first selection operation can be a touch screen operation; for a first device controlled by a mouse, such as a desktop computer, the first selection operation can be a click operation; For a remotely controlled first device, such as a smart TV, the first selection operation may be a selection operation.

Wherein, the method further includes: in response to a third selection operation for the split screen option, displaying, by the first device, the second interface, the second interface being configured to display a screen associated with the first device a second list of associated devices; and/or, in response to a fifth selection operation for the separate sound option, the first device displays the second interface, the second interface being used to display a connection with the first device A list of associated third associated devices.

In this application, the first associated device list includes a list of associated devices that can implement operation functions; the second associated device list includes a list of associated devices that can display images; the third associated device list includes a list of associated devices that can play sound. It should be noted that, when the function list includes the split screen option and/or the split sound option, the first device may also display the second interface in response to the first selection operation for the split screen option/or the split sound option.

Specifically, after the user starts the collaboration process of the target application on the first device, a first interface is displayed on the current interface of the first device, and the first interface displays the functions that the target application can perform collaboration, including the separation control option , at least one of split screen option, split sound option. Then, the user selects the function that needs to be coordinated from the function list according to his needs, specifically, the user selects the function that needs to be coordinated through the first selection operation.

In a possible implementation manner, the target associated device list is obtained from a cloud device, and the target associated device list includes the first associated device list, the second associated device list, and the third associated device list At least one item in the device list.

The first device and the devices under the management of the cloud device can upload their own service lists to the cloud device, and the cloud device can store these service lists uploaded by each device according to the granularity of devices, applications, and functions. After the first device selects the function that needs to be coordinated by the target application, the first device may obtain a device capable of implementing the function from the cloud device.

Optionally, the method further includes: the first device reporting a target account to the cloud device, where the target account is a user account logged in by the first device.

When managing multiple devices through the cloud device, the cloud device can be managed through the user account logged in on the device. When the user uses the user account to log in to the first device, the first device can upload the logged in user account to the cloud device, and the cloud device can store the service list with the granularity of the user account. The user account logged in by the first device may be a currently logged in terminal system account or a currently logged in application account or the like.

Exemplarily, after the user logs in with the user account of a device under the management scope of the cloud device, the user account for logging in the device can be uploaded to the cloud device.

Wherein, the user account logged in by each associated device in the target associated device list is the target account.

Specifically, after receiving the service list uploaded by each device, the cloud device may store the service list of each device according to the user account, specifically, the service lists uploaded by the devices of the same user account are stored together. Therefore, when the first device requests the cloud device to realize the selected function that needs to be coordinated, the cloud device can search for a device that is the same as the user account of the first device and can realize the function that needs to be coordinated, that is, the associated device, and finds The devices are sent to the first device in the form of a list. The first device displays the found device in the second interface, so that the user can execute the function on the first device through his own other device. Through the management of user accounts, other people can be prevented from using their own devices at will.

In this embodiment of the present application, after the user performs the first selection operation on the first interface, the first device displays the first interface on the current interface, and the first interface displays one or more associated devices obtained from the cloud device . For example, as shown in FIG. 5c, the user selects the separation operation option through the first selection operation on the first interface, the first device responds to the first selection operation, displays the second interface on the current interface, and displays the ability to The target application of the first device realizes the operation of the gamepad, mouse and keyboard, and touch of the mobile phone. As shown in FIG. 5d, the user selects the split screen option through the first selection operation on the first interface, the first device responds to the first selection operation, displays the second interface on the current interface, and the second interface displays The device's target application implements the screen display of the living room TV and my tablet. As shown in FIG. 5e, the user selects the option to separate the sound through the first selection operation on the first interface, the first device responds to the first selection operation, and displays the second interface on the current interface, and the second interface shows that the first device can respond to the first selection operation The device's target application implements sound playback for Bluetooth headsets, Bluetooth speakers, and living room TVs.

Exemplarily, the display position of the second interface may be determined according to the display position of the first interface. For example, when the user's first selection operation is the separation control option, the display position of the second interface may be located next to the separation control option, as shown in Figure 5c; when the user's first selection operation is the separation screen option, the second interface The display position of the interface can be located next to the split screen option, as shown in Figure 5d.

Exemplarily, the display position of the second interface may also be determined according to the display direction of the screen of the first device. For example, when the screen of the first device is displayed in a landscape orientation, the display position of the second interface can be located next to the separation control option, as shown in Figures 5c-5e; when the screen of the first device is displayed in a portrait orientation , the display position of the second interface can be suspended and displayed on the first interface, as shown in FIG. 5f .

Exemplarily, the display position of the second interface can be moved. When the user drags or slides the second interface, the second interface may move according to the user's dragging direction or delineation direction.

Exemplarily, the display position of the second interface may be random. The display position of the second interface may be the position displayed by the user last time, or may be the display position randomly determined by the system, or may be the position where the user touches the screen, which is not limited in this embodiment of the present application.

In a possible implementation manner, the associated device may be a device that the first device can detect and connect to. Depending on the scene where the user is located, there may be 0, 1 or more associated devices finally obtained that meet the requirements . For example, suppose that in the current scenario, the function list of the target application includes separate operation options, separate operation options, and separate sound options, and the device included in the scene is Device A (which can implement the functions of separate operation options and separate screen options), Device B (capable of separating operation options and separating sound options) and device C (capable of separating operation options, separating operation options, and separating sound options), the associated device combination that meets the requirements of the current scenario includes device A + device B , Device A + Device C, Device B + Device C, Device C, Device A + Device B + Device C, a total of 5 combinations.

Optionally, the first device may acquire attribute information of each terminal device within a preset range, where the attribute information includes functions that the terminal device can provide. The first device selects a terminal device capable of realizing the function requiring coordination from the respective terminal devices according to the attribute information of each terminal device, and stores the terminal device capable of realizing the function requiring coordination in the target associated device list.

In the scene where the user is located, the first device can initiate a search through the Bluetooth protocol, Hilink protocol, etc., detect and connect to the surrounding terminal devices, and then read the attribute information of each terminal device respectively, so as to determine each terminal device. The functions that can be implemented can be mentioned, for example, the profile information of each surrounding terminal device can be read, so as to obtain the functions of the device (the Bluetooth protocol specifies many profiles, and each profile describes a function; the hilink protocol is similar). In addition, each terminal device in the scene can also actively broadcast the functions that can be implemented by itself. After determining which terminal devices exist in the surrounding area and what functions each terminal device has respectively, one or more terminal devices that can realize the function selected by the user and need to be coordinated can be selected from these terminal devices, that is, the determined association equipment.

Optionally, the first device may also acquire a pre-stored device function comparison table, and the device function comparison table records the functions that each terminal device can provide. The first device determines, according to the device function comparison table, a terminal device that implements a function that needs to be coordinated within a preset range, and stores the terminal device that can implement the function that needs to be coordinated in a target associated device list.

In another way, the first device may also pre-build and store a device function comparison table, and the device function comparison table records the functions that each known type of terminal device can provide. For example, a smart speaker has an audio playback function, and a smart TV has an image playback function. Display function and audio playback function, etc. After detecting and connecting the surrounding terminal devices by wire or wireless (such as Bluetooth), the functions that each terminal device can provide can be determined by looking up a table, and finally the functions can be selected from these terminal devices.

Optionally, the method further includes: the first device sends a function request to the cloud device, where the function request is used to request a function query, and the function query includes at least one of the following: searching under the target account. All functions of , find all the associated devices including the function list, find all functions on the associated device.

Among them, after logging in to the first device, the user can request a function query from the cloud device, which mainly includes the following query methods:

1. By searching the function list, get all the functions available on all devices under the current user account.

2. Find all devices with this function by function, such as display function, positioning function, some functions may only be available on some devices, so that some devices can be filtered out.

3. Through the device search function, the first device can search for all the functions that can be provided on the selected device.

According to the query function, users can meet the needs of users in different application scenarios. For example, the user has device A, and the user needs the device to cooperate with device A to realize the separation operation function. If the user finds that the accessory has device B, the user can find all the functions that device B can provide to determine whether device B can Collaborate to achieve operational functions.

S430. In response to a second selection operation on at least one associated device in the first associated device list, the target application of the first device is controlled by the at least one associated device.

In this embodiment of the present application, after the second interface is displayed, the first device may determine at least one associated device to be coordinated according to the second selection operation of the user, so that the at least one associated device can control the first device.

Optionally, the method further includes: in response to a fourth selection operation on at least one associated device in the second associated device list, causing the screen of the target application of the first device to be displayed by the at least one associated device One associated device display; and/or, in response to a sixth selection operation for at least one associated device in the third associated device list, so that the sound of the target application of the first device is transmitted by the at least one Play with the associated device.

It should be noted that, the user may select the functions in the function list of the target application program to be executed by the associated device respectively through multiple selection operations. For example, the control function corresponding to the separate control option in FIG. 5a can be realized by a mouse and keyboard, the function of the display screen corresponding to the separate screen option in FIG. 5a can be realized by my tablet, and the separate sound option in FIG. 5a corresponds to The function of playing sound can be realized by a Bluetooth speaker. Wherein, the second selection operation is mainly used to select the function that the target application needs to perform a coordinated operation from the function list. The second selection operation may be any preset operation, may be set by the user, or may be set by the system of the first electronic device, which is not limited in this application. According to different first devices and different display modes of the first device, the second selection operation may include various implementation forms. For example, for a first device that can support a touch screen, such as a smart phone, the second selection operation can be a touch screen operation; for a first device controlled by a mouse, such as a desktop computer, the second selection operation can be a click operation; For a remotely controlled first device, such as a smart TV, the second selection operation may be a selection operation.

Exemplarily, after the first device determines at least one associated device according to the user's operation, the first device may invoke a remote service to perform interaction between the first device and the at least one associated device.

For example, as shown in FIG. 6a, the device A uses the device operating system to provide display functions, input functions and positioning functions to implement the function list in the running application program. As shown in Figure 6b, after device A starts the collaboration process, device B provides a positioning function to coordinate the separation positioning option of the function list in device A, device C provides an input function to coordinate the separation control option of the function list in device A, and device D A display function is provided to coordinate the split screen option in the function list in device A.

Optionally, the method further includes: the first device sends an acquisition request to the cloud device, where the acquisition request is used to request to acquire authorization information of a target function on the at least one associated device, the target function In order to realize the function of the separation operation function of the target application; the first device receives the authorization information sent by the cloud device in response to the acquisition request.

After the first device establishes a connection with the at least one associated device, the first device can verify through the cloud device whether the target function on the at least one associated device is authorized by the user. Specifically, when the user needs to coordinate the separate manipulation options on the first device, the first device can verify whether the target function on the associated device capable of implementing the manipulation function is authorized. For example, when the user needs the mouse on the desktop computer to operate the game control on the mobile phone, the mobile phone needs to verify whether the mouse on the desktop computer is authorized by the user; when the user needs the smart speaker to display the music being played on the mobile phone, the mobile phone needs to Verify whether the smart speaker is authorized by its user; when the user needs the smart TV to display the video being played on the mobile phone, the mobile phone needs to verify whether the smart TV is authorized by its user.

Optionally, the method further includes: if the authorization information includes an authorization result of the target function on the first associated device, the first device invokes the target function on the first associated device, the at least one The associated device includes the first associated device; if the authorization information does not include an authorization result of the target function on the at least one associated device, the first device terminates calling the target function of the at least one associated device, and A third interface is displayed, where the third interface is used to prompt a failure to invoke the target function of the at least one associated device.

Specifically, if the associated device providing the function is authorized, proceed to the next step; if not, the call of the entire function is terminated, and a third interface is displayed on the interface of the first device to indicate to the user that the collaborative call failed this time. The display position of the third interface is the same as the display position of the second interface.

Optionally, the method further includes: the first device detects whether the target function is available, and if the target function is unavailable, terminates calling the target function, and displays the third interface.

Wherein, when the associated device is authorized, the associated device detects whether the target function is available. If the target function is not activated, the target function is activated; if the target function fails to be activated, the invocation is terminated, and a third interface is displayed on the first device. Indicates the user that this collaborative call failed.

In summary, the device coordination method provided by the present application is introduced with reference to the accompanying drawings. The method can select at least one associated device from multiple devices to implement the function selected by the user on the first device, so that the first device can integrate multiple devices. The functions provided by an associated device can simultaneously call the functions of multiple different types of devices to achieve the collaborative work of multiple devices. At the same time, through the user's operation, at least one associated device is selected from multiple devices to implement the user's control function of the target application on the first device, thereby realizing the function that the associated device can control the first device.

In the specific implementation process, as shown in Figure 7a, the device coordination method provided by the present application can be implemented based on the coordination of three modules, which specifically includes: a coordination module, a service governance module and a connection module.

Wherein, both the first device and the associated device include the coordination module, the service management module and the connection module. The collaboration module is used to start the collaboration process; the service management module user scans the device operating system and/or all the applications to generate and report the service list; the connection module is used to communicate with cloud devices and other devices. establish a communication connection between.

The application program on the device may include functions carried by the device itself, such as display function, positioning function, input function, sound function, etc., and may also include a function list provided by the downloaded application program. The functions on the device can be exposed in a certain form, so that the service management module can scan and identify them. The developed functions need to conform to certain development specifications so that they can be recognized by the service management module on the device. In particular, some default functions developed on the device are the exposure of the capabilities on the device, such as display, sound, input, positioning, etc. These capabilities are consistent with the capabilities exposed by the existing device operating system, and many applications will If it is used, it is only exposed again in the form of a prescribed function. Make these functions accessible to remote devices as well.

Exemplarily, the development specification of the function is extensible, and it can also be various forms of remote invocation methods, such as http, gRPC and other mature methods. Special methods can also be used to meet special requirements, such as using a proprietary protocol to achieve the transmission efficiency of streaming data. A variety of remote invocation methods can be integrated by the framework in the technical solution, and appropriate methods can be adopted in appropriate scenarios.

Exemplarily, the method for device coordination provided by the present application also requires the coordination and cooperation of cloud devices to be implemented. As shown in Figure 7b, the cloud device may include a service management module for the user to store the service list of each device, a device management module for establishing a connection with each device, and a user management module for managing each device.

Exemplarily, as shown in FIG. 8 , the first device determining the associated device may further include the following steps:

S81. The first device establishes a connection between the first device and the associated device through the connection module on the first device and the device management module on the cloud device.

S82. The first device searches for the service management module on the cloud device, and verifies whether the first device has been authorized by the user to use the target function on the associated device.

The service management module on the cloud device stores the service list of each device and/or the authorization information of each device. After the first device establishes a connection with the associated device, it can send a request to the cloud device to obtain the target on the associated device. Authorization information for the function to invoke the associated device.

S83. When the service management module on the cloud device does not store the authorization information of the target function on the associated device, the cloud requests to obtain the one-time use authorization of the associated device.

S84. When the service management module on the cloud device stores the authorization information of the target function on the associated device or the cloud device obtains a one-time use authorization from the associated device, the gateway detection module on the associated device detects whether the target function is available.

S85. When the gateway detection module on the associated device detects that the target function is available, the first device invokes the target function on the associated device.

S86. When the gateway detection module on the associated device detects that the target function is unavailable, the associated device attempts to start the target function.

S87. When the associated device has not activated the target function or the cloud device has not obtained the one-time use authorization of the associated device, display on the first device that the target function on the associated device failed to be invoked.

Wherein, the first device may call the target function on the associated device in an agreed manner.

It can be understood that, in order to realize the above-mentioned functions, the electronic device includes corresponding hardware and/or software modules for executing each function. The present application can be implemented in hardware or in the form of a combination of hardware and computer software in conjunction with the algorithm steps of each example described in conjunction with the embodiments disclosed herein. Whether a function is performed by hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functionality for each particular application in conjunction with the embodiments, but such implementations should not be considered beyond the scope of this application.

In this embodiment, the electronic device can be divided into functional modules according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware. It should be noted that, the division of modules in this embodiment is schematic, and is only a logical function division, and there may be other division manners in actual implementation.

In the case where each functional module is divided corresponding to each function, FIG. 9 shows a schematic diagram of a device coordination apparatus. As shown in FIG. 9 , the device coordination apparatus 900 is applied to an electronic device, and the device coordination apparatus 900 may include: a display unit 910 and processing unit 920.

Among them, the display unit 910 may be used to support the electronic device to perform the above-mentioned S410, S420, etc., and/or other processes for the techniques described herein.

The processing unit 920 may be used to support the electronic device to perform the above-described S430, etc., and/or other processes for the techniques described herein.

It should be noted that, all relevant contents of the steps involved in the above method embodiments can be cited in the functional description of the corresponding functional module, which will not be repeated here.

The electronic device provided in this embodiment is used to execute the above-mentioned device coordination method, so it can achieve the same effect as the above-mentioned implementation method.

Where an integrated unit is employed, the electronic device may include a processing module, a memory module and a communication module. The processing module may be used to control and manage the actions of the electronic device, for example, may be used to support the electronic device to perform the steps performed by the display unit 910 and the processing unit 920 above. The storage module may be used to support the electronic device to execute stored program codes and data, and the like. The communication module can be used to support the communication between the electronic device and other devices.

The processing module may be a processor or a controller. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, and the like. The storage module may be a memory. The communication module may specifically be a device that interacts with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, and a Wi-Fi chip.

In one embodiment, when the processing module is a processor and the storage module is a memory, the electronic device involved in this embodiment may be a device having the structure shown in FIG. 1 .

This embodiment also provides a computer storage medium, where computer instructions are stored in the computer storage medium, and when the computer instructions are executed on the electronic device, the electronic device executes the above-mentioned relevant method steps to implement the device coordination method in the above-mentioned embodiment.

This embodiment also provides a computer program product, which, when the computer program product runs on a computer, causes the computer to execute the above-mentioned relevant steps, so as to realize the device coordination method in the above-mentioned embodiment.

In addition, the embodiments of the present application further provide a first device, which may specifically be a chip, a component or a module, and the first device may include a connected processor and a memory; wherein the memory is used to store computer execution instructions , when the first device is running, the processor can execute the computer-executed instructions stored in the memory, so that the chip executes the device coordination method in the above method embodiments.

Wherein, the electronic device, computer storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding method provided above. Therefore, for the beneficial effects that can be achieved, reference can be made to the corresponding provided above. The beneficial effects in the method will not be repeated here.

From the description of the above embodiments, those skilled in the art can understand that for the convenience and brevity of the description, only the division of the above functional modules is used as an example for illustration. In practical applications, the above functions can be allocated by different The function module is completed, that is, the internal structure of the first device is divided into different function modules, so as to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed first device and method may be implemented in other manners. For example, the first device embodiment described above is only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be The combination may either be integrated into another first device, or some features may be omitted, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, and the indirect coupling or communication connection of the first device or unit may be in electrical, mechanical or other forms.

Units described as separate components may or may not be physically separated, and components shown as units may be one physical unit or multiple physical units, that is, may be located in one place, or may be distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, which are stored in a storage medium , including several instructions to make a device (which may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (random access memory, RAM), magnetic disk or optical disk and other media that can store program codes.

The above content is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Covered within the scope of protection of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A device cooperation method, the method comprising:

The first device displays a first interface, the first interface is used to display a function list associated with a target application, the target application runs on the first device, and the function list includes the separation of the target application control options;

In response to the first selection operation for the separation manipulation option, the first device displays a second interface for displaying a first associated device list associated with the first device;

In response to a second selection operation for at least one associated device in the first associated device list, causing the target application of the first device to be manipulated by the at least one associated device.
The method of claim 1, wherein the function list further includes a split screen option;

The method also includes:

In response to a third selection operation for the split screen option, the first device displays the second interface for displaying a second associated device list associated with the first device;

In response to a fourth selection operation for at least one associated device in the second associated device list, the screen of the target application of the first device is displayed by the at least one associated device.
The method of claim 1 or 2, wherein the function list further includes a split sound option;

The method also includes:

In response to a fifth selection operation for the separate sound option, the first device displays the second interface for displaying a third list of associated devices associated with the first device;

In response to a sixth selection operation for at least one associated device in the third associated device list, causing the sound of the target application of the first device to be played by the at least one associated device.
The method according to any one of claims 1-3, wherein the target associated device list is obtained from a cloud device, and the target associated device list includes the first associated device list, the second associated device list and the At least one item in the third associated device list.
The method of claim 4, wherein the method further comprises:

The first device reports a target account to the cloud device, where the target account is a user account logged in by the first device.
The method according to claim 5, wherein a user account logged in by each associated device in the target associated device list is the target account.
The method according to any one of claims 1-6, wherein the function list is provided by a device operating system and/or an application program in the first device.
The method of claim 7, wherein the method further comprises:

When the first device is started, the first device scans the device operating system and/or all the applications to obtain a service list of the first device, where the service list includes the device operating system and/or the application. / or a list of all the functions provided by the said application;

The first device reports the service list to the cloud device.
The method of claim 8, wherein the method further comprises:

The first device sends a function request to the cloud device, where the function request is used to request a function query, and the function query includes at least one of the following: search for all functions under the target account, search for all functions including the function list , find all the functions on the associated device.
The method of claim 9, wherein the method further comprises:

The first device sends an acquisition request to the cloud device, where the acquisition request is used to request to acquire authorization information of a target function on the at least one associated device, where the target function is to implement a separation operation of the target application the function of the function;

The first device receives the authorization information sent by the cloud device in response to the acquisition request.
The method of claim 10, wherein the method further comprises:

If the authorization information includes the authorization result of the target function on the first associated device, the first device invokes the target function on the first associated device, and the at least one associated device includes the first associated device;

If the authorization information does not include the authorization result of the target function on the at least one associated device, the first device terminates calling the target function of the at least one associated device, and displays a third interface, the third interface It is used to prompt the failure of invoking the target function of the at least one associated device.
The method of claim 11, wherein the method further comprises:

The first device detects whether the target function is available, and if the target function is unavailable, terminates calling the target function, and displays the third interface.
A device cooperation device, the device includes:

A display unit, configured to display a first interface, where the first interface is configured to display a function list associated with a target application, the target application running on the first device, and the function list includes the target application separate control options;

The display unit is further configured to display a second interface in response to a first selection operation for the separation manipulation option, where the second interface is configured to display a list of first associated devices associated with the first device;

A processing unit, configured to cause the target application of the first device to be controlled by the at least one associated device in response to a second selection operation for at least one associated device in the first associated device list.
The apparatus of claim 13, wherein the function list further includes a split screen option;

The display unit is further configured to: in response to a second selection operation for the split screen option, display the second interface, where the second interface is configured to display a second associated device list associated with the first device ;

The processing unit is further configured to: in response to a fourth selection operation on at least one associated device in the second associated device list, cause the screen of the target application of the first device to be associated with the at least one device device display.
The apparatus of claim 13 or 14, wherein the function list further includes a split sound option;

The display unit is further configured to: display the second interface in response to a third selection operation for the separated sound option, where the second interface is configured to display a third associated device list associated with the first device ;

The processing unit is further configured to: in response to a sixth selection operation for at least one associated device in the third associated device list, so that the sound of the target application of the first device is associated with the at least one device device to play.
The apparatus according to any one of claims 13-15, wherein the target associated device list is obtained from a cloud device, and the target associated device list includes the first associated device list, the second associated device list and At least one item in the third associated device list.
The apparatus of claim 16, wherein the apparatus further comprises a transceiver unit,

The transceiver unit is configured to report a target account to the cloud device, where the target account is a user account logged in by the first device.
The apparatus according to claim 17, wherein a user account logged in by each associated device in the target associated device list is the target account.
The apparatus according to any one of claims 13-18, wherein the function list is provided by a device operating system and/or an application program in the first device.
The apparatus according to claim 19, wherein the processing unit is further configured to scan the device operating system and/or all the application programs when the first device is started, to obtain the information of the first device. a service list, the service list includes a list of functions provided by the device operating system and/or all the applications;

The transceiver unit is further configured to report the service list to the cloud device.
An electronic device comprising a processor, a memory, a communication interface, and one or more programs stored in the memory and configured to be executed by the processor, The program includes instructions for performing the steps in the method of any of claims 1-12.
A computer-readable storage medium comprising storing a computer program for electronic data exchange, wherein the computer program causes a computer to perform the method of any one of claims 1-12.