WO2022100309A1 - 桌面元数据的显示方法、访问方法及相关装置 - Google Patents

桌面元数据的显示方法、访问方法及相关装置 Download PDF

Info

Publication number
WO2022100309A1
WO2022100309A1 PCT/CN2021/121291 CN2021121291W WO2022100309A1 WO 2022100309 A1 WO2022100309 A1 WO 2022100309A1 CN 2021121291 W CN2021121291 W CN 2021121291W WO 2022100309 A1 WO2022100309 A1 WO 2022100309A1
Authority
WO
WIPO (PCT)
Prior art keywords
application
target
metadata
type
data
Prior art date
Application number
PCT/CN2021/121291
Other languages
English (en)
French (fr)
Inventor
杨俊拯
何�轩
毕浩
邓朝明
钟卫东
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2022100309A1 publication Critical patent/WO2022100309A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/04817—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance using icons
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces

Definitions

  • the present application relates to the technical field of information display, and in particular, to a display method, access method and related device of desktop metadata.
  • the operating status information of multiple devices can be displayed on the desktop of the current device, such as air conditioners and other devices.
  • the current device such as air conditioners and other devices.
  • Operating temperature, speaker playback control controls, these display information enable users to understand the status information of different objects at the device level, but it is difficult to meet the user's more refined cross-end use requirements for different devices.
  • Embodiments of the present application provide a desktop metadata display method, an access method, and a related device, so as to improve the comprehensiveness and refinement of the unified user interaction portal, that is, the unified desktop display information, and better meet the usage requirements.
  • an embodiment of the present application provides a method for displaying desktop metadata, including:
  • the unified desktop includes metadata
  • the metadata includes application type metadata and file type metadata
  • the application type metadata is associated with applications of one or more devices in the target device set, so
  • the metadata of the file type is associated with data of the same file type of at least one device in the target device set.
  • the unified desktop of the electronic device is used as a unified user interaction portal for multiple devices in the target device set, which can not only display the metadata of the applications associated with one or more devices in the target device set, but also enable users to realize the application-level interface.
  • Cross-end control can also display metadata of data of the same file type associated with at least one device in the target device set, realize the interaction mechanism of data dimensions of multiple devices, and enable users to perform cross-end control of data levels, which is conducive to improving unification
  • the comprehensiveness and refinement of desktop information display can better meet the needs of use.
  • an embodiment of the present application provides a method for accessing data, including:
  • the first electronic device displays a unified desktop, and the unified desktop includes metadata that associates data on the first electronic device and a second electronic device communicatively connected to the first electronic device;
  • the metadata on the first electronic device and the second electronic device are accessed according to the type of the target metadata.
  • an embodiment of the present application provides a display device for desktop metadata, including a display unit,
  • the display unit is configured to display a unified desktop, where the unified desktop includes metadata, the metadata includes application type metadata and file type metadata, and the application type metadata is associated with one or more of the target device sets.
  • the metadata of the file type is associated with data of the same file type of at least one device in the target device set.
  • an electronic device including:
  • a display unit configured to display a unified desktop, where the unified desktop includes metadata, and the metadata is associated with metadata of the electronic device and a second device connected to the electronic device;
  • the control unit is configured to control the electronic device and the metadata in the second device according to the type of the target metadata in response to the selection operation on the target metadata on the unified desktop.
  • embodiments of the present application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured by Executed by the processor, the program includes instructions for executing steps in any of the methods in the first aspect or the second aspect of the embodiments of the present application.
  • an embodiment of the present application provides a chip, including: a processor for calling and running a computer program from a memory, so that a device installed with the chip executes the first aspect or the second embodiment of the present application.
  • an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes a computer to execute the implementation of the present application Examples include some or all of the steps described in any of the methods of the first aspect or the second aspect.
  • an embodiment of the present application provides a computer program, wherein the computer program is operable to cause a computer to execute part or all of the steps described in any of the methods in the first aspect or the second aspect of the embodiments of the present application .
  • the computer program may be a software installation package.
  • Fig. 1 is the schematic diagram of the multi-device control interface of the current Apple mobile phone
  • FIG. 2A is a schematic flowchart of a method for displaying desktop metadata provided by an embodiment of the present application
  • Fig. 2a is a schematic diagram of displaying "music" metadata provided by an embodiment of the present application.
  • Fig. 2b is a schematic diagram of displaying another "music" metadata provided by an embodiment of the present application.
  • Fig. 2c is a schematic diagram of displaying "video" metadata provided by an embodiment of the present application.
  • Fig. 2d is a schematic diagram of displaying metadata of a "document" provided by an embodiment of the present application.
  • Fig. 2e is a schematic diagram of displaying metadata of a "photo" provided by an embodiment of the present application.
  • 2B is an exemplary diagram of a unified desktop provided by an embodiment of the present application.
  • 2C is an example interface change diagram after selecting "APP" metadata provided by an embodiment of the present application.
  • 2D is an example interface change diagram after selecting “music” metadata provided by an embodiment of the present application.
  • 2E is a schematic diagram of the architecture of a multi-device unified management system provided by an embodiment of the present application.
  • 2F is a schematic diagram of a search function metadata provided by an embodiment of the present application.
  • 2G is a schematic diagram of a unified desktop pop-up window displaying a search function interface provided by an embodiment of the present application
  • 2H is a schematic diagram of keyword information such as a guide pop-up window provided by an embodiment of the present application.
  • 2I is a schematic diagram of a search result display interface provided by an embodiment of the present application.
  • 2J is a schematic diagram of selecting APP metadata so that the device displays the application in the mobile phone 2 according to an embodiment of the present application;
  • 2K is a schematic diagram of a WeChat application that displays the mobile phone 2 on the current unified desktop provided by an embodiment of the present application;
  • FIG. 2L is a schematic diagram of displaying metadata of a WeChat application associated with the mobile phone 2 provided by an embodiment of the present application;
  • 2M is a schematic diagram of a control editing interface of a cross-end application of a WeChat application provided by an embodiment of the present application;
  • 2N is a schematic diagram of a decision-making process for enabling metadata on a unified desktop by a device according to an embodiment of the present application
  • 20 is an exemplary diagram of a distributed application on a mobile phone terminal provided by an embodiment of the present application.
  • 2P is an exemplary diagram of another distributed application on a mobile phone terminal provided by an embodiment of the present application.
  • FIG. 2Q is an exemplary diagram of another distributed application on the mobile phone terminal provided by an embodiment of the present application.
  • 2R is a schematic diagram of a game application scenario for separating screen sound and manipulation provided by an embodiment of the present application
  • 2S is an exemplary diagram of a PC-side distributed application provided by an embodiment of the present application.
  • 2T is an exemplary diagram of another PC-side distributed application provided by an embodiment of the present application.
  • 2U is a schematic flowchart of a redirection provided by an embodiment of the present application.
  • FIG. 2V is a schematic diagram of a functional interface of a redefining flow application provided by an embodiment of the present application.
  • FIG. 2W is a schematic diagram of redrawing a functional interface of a streaming application provided by an embodiment of the present application
  • 2X is a schematic diagram of a gesture change from palm opening to fisting provided by an embodiment of the present application
  • 2Y is a schematic diagram of a gesture change from making a fist to opening a palm provided by an embodiment of the present application
  • FIG. 3 is a schematic flowchart of a method for accessing desktop metadata provided by an embodiment of the present application
  • FIG. 4 is a block diagram of functional units of a display device for desktop metadata provided by an embodiment of the present application.
  • FIG. 5 is a block diagram of functional units of another display device for desktop metadata provided by an embodiment of the present application.
  • FIG. 6 is a block diagram of functional units of an electronic device provided by an embodiment of the present application.
  • FIG. 7 is a block diagram of functional units of another electronic device provided by an embodiment of the present application.
  • the Apple mobile phone supports the control of multiple devices.
  • the current information display interface includes the playback control buttons of the current device and the scene identifiers associated with devices in different locations, so that the user can select the target scene identifier to achieve For cross-end usage of devices in this target scenario.
  • FIG. 2A is a schematic flowchart of a method for displaying desktop metadata provided by an embodiment of the present application, which is applied to any electronic device in the target device set that supports a touch display function. As shown in the figure, the Methods include:
  • Step 201 Display a unified desktop, where the unified desktop includes metadata, the metadata includes application type metadata and file type metadata, and the application type metadata is associated with one or more devices in the target device set.
  • the metadata of the file type is associated with data of the same file type of at least one device in the target device set.
  • one metadata consists of the following:
  • the unique path has a unique device identification ID, and a unique path to the metadata under this device.
  • Metadata category file type (for example: music, video, document, photo, etc. metadata), application type (for example: device-level metadata, aggregate application-level metadata, independent application-level metadata).
  • Each kind of data has its own properties, such as the size of the picture, the duration of the video, etc.
  • These attributes of metadata can make the unified desktop display these data well, and can find a suitable opening method when the user triggers the use.
  • the target device set corresponds to the device associated with the user account of the unified desktop
  • the metadata refers to a function icon composed of at least one element
  • the element refers to the graphics displayed on the unified desktop and/or text.
  • the unified desktop may be opened by an application program displayed on the desktop, or it may be a desktop in the native desktop of the mobile phone, or it may be a function selected from system functions such as settings, etc.
  • the form is not uniquely limited.
  • the display mode of the metadata of the application type and the metadata of the file type can be set differently, and the differential setting includes the differential setting of various attributes such as display position and display color.
  • a device can display metadata for file types in the upper half of the unified desktop and application-type metadata in the lower half of the unified desktop.
  • the device may differentiate the brightness of the metadata of the file type and the metadata of the application type, or the shape of the graphics, and the like.
  • a normalized app icon can be used to expose metadata for apps that are only associated with a single device.
  • a conventional application icon can be used to display the metadata of the application associated with all devices, and the application icon at least contains text information such as "application” and "APP", which is used to prompt the user that the metadata is in the application
  • the hierarchy is associated with all devices, and the graphic part of the application icon can express the image of the integration of multiple devices, thereby more intuitively prompting the user of the role of the metadata.
  • a specially designed application icon can be used to display the metadata of the application associated with some devices.
  • the graphic part of the application icon corresponding to the data can display the image added by two mobile phones, and cooperate with similar text information such as "mobile phone 1+2" to intuitively remind the user of the role of the metadata.
  • the specific display manner can be various, which is not uniquely limited here.
  • a normalized application icon can be used to display metadata of data of the same file type only associated with a single device.
  • the application icon can be in the form of graphics and text, and which part to be selected or the text part can be identified by numbers such as numbers.
  • numbers such as numbers.
  • “2” can be added to the graphic or text part of the corresponding application icon to represent the mobile phone 2 .
  • a normalized application icon can be used to display metadata of data of the same file type associated with all devices, and the application icon at least contains text of specific file types such as "music", “video”, “document” and the like information, which is used to prompt the user that the metadata is associated with all devices at the data level, and the graphic part of the application icon can express an image of multiple devices integrated together, thereby more intuitively prompting the user of the role of the metadata.
  • a specially designed application icon can be used to display the metadata of the data of the same file type associated with some devices. For example, it is necessary to associate the mobile phone 1 and mobile phone 2 of the same mobile phone device type in the target device set to the same metadata.
  • the graphic part of the application icon corresponding to the metadata can display the image added by two mobile phones, and cooperate with similar text information such as "mobile phone 1+2" to intuitively remind the user of the role of the metadata.
  • the metadata can be expressed through image information and/or text information related to musical symbols.
  • the graphic information can be, for example, a note tree as shown in Figure 2a, with each tree branch associated with a corresponding device , and of course there are other ways, which are not limited here.
  • the image information can also follow the dynamic changes.
  • mobile phone 1 non-current device
  • mobile phone 1 includes music files. If it is a device in the target set, then as shown in Figure 2b, the image information part of the "music" metadata no longer contains the image information of the mobile phone 1.
  • the metadata can be expressed by image information and/or text information related to the video.
  • the graphic information + text information can be, for example, an application icon as shown in Figure 2c.
  • the application icon is represented by a play button symbol.
  • the metadata can be expressed by image information and/or text information related to the document, and the graphic information + text information can be, for example, an application icon as shown in Figure 2d, which is expressed by a document symbol
  • the document function expresses the association of multiple devices through the text information displayed in the embedded display.
  • the metadata can be expressed by image information and/or text information related to the photo
  • the graphic information + text information can be, for example, an application icon as shown in Figure 2e, and the application icon is expressed by an album symbol
  • the document function expresses the association of multiple devices through the text information displayed in the embedded display.
  • the unified desktop includes elements of application types such as "APP”, “My Tablet”, “My Computer”, “My Phone 1”, “My Phone 2”, etc. data, and metadata for file types such as "music", "video”, and "document”.
  • the device can display the applications of some or all of the devices in the target device set.
  • the part of the devices can be a specific type of device, a user-specified device, or other methods.
  • the display modes of the applications of some or all of the devices can be various, such as drop-down lists, pop-up windows, etc., which are not uniquely limited here.
  • the device will display the applications of mobile phone 1 and mobile phone 2 in a pop-up window on the unified desktop (the legend shows the application of mobile phone 2).
  • the application of the mobile phone 2 is displayed.
  • the example interface change diagram after selecting the "Music” metadata the device will switch the unified desktop to the display interface of the music files of multiple devices, and the device with the most music files will be displayed preferentially by default.
  • Music files it can also be other strategies to select the device that needs to display its music files, or, it can also be displayed in all, and it can be distinguished from different devices according to different colors. There are various display methods, which are not limited here).
  • the unified desktop can be specifically realized by means of a server and a terminal.
  • the server (the cloud system shown in the figure) is mainly responsible for the management of devices under the user name and the management of applications and data, which may specifically include a device management module, an application management module, a user management module, a device management module, an application management module, and a user management module. Any two modules in the module are connected in communication with each other.
  • the terminal includes a unified desktop system and/or a background support system
  • the unified desktop system includes a user interface UI module, a redirection sink module, an application decision module, and its own application.
  • the modules, the background support system includes at least one of a connection module, a distributed data service system, a distributed resource system, and a redirected source source system.
  • the unified desktop system is the main interactive entrance, and the user realizes unified management and use of multiple devices through this system.
  • the connection module is used to ensure the connection of multiple devices, and can specifically support various conventional communication technologies, such as mobile communication technology (the fifth generation mobile communication technology 5G, etc.), local area network short-distance wireless communication technology (Bluetooth, ultra-bandwidth UWB, etc. ).
  • mobile communication technology the fifth generation mobile communication technology 5G, etc.
  • local area network short-distance wireless communication technology Bluetooth, ultra-bandwidth UWB, etc.
  • the distributed data service system is used to realize data intercommunication among multiple devices.
  • the redirection source system and the distributed resource system are used to realize cross-end use of applications.
  • different devices can be provided with a unified desktop system and a background support system at the same time, or only one of them, depending on the capabilities of the devices.
  • mobile phones, PCs and other capable devices can have a unified desktop system and a back-end support system at the same time
  • customer front-end equipment CPE, routers, etc. can only have the back-end support system
  • the web terminal can only have a unified desktop system.
  • the unified desktop of the electronic device is used as a unified user interaction portal for multiple devices in the target device set, which can not only display the metadata of applications associated with one or more devices in the target device set, but also enable the user to It realizes the cross-end control of the application level, and can also display the metadata of the data of the same file type associated with at least one device in the target device set, and realizes the interaction mechanism of the data dimension of multiple devices, so that the user can carry out the cross-end control of the data level. It is beneficial to improve the comprehensiveness and refinement of the unified desktop information display, and better meet the usage requirements.
  • the unified desktop further includes search function metadata, where the search function metadata is used to perform data search in a target data set, and the target data set corresponds to the target Data for the devices in the device collection.
  • the display position and specific form of the search function metadata are not uniquely limited.
  • the display position may be at the corner of the unified desktop, or directly above the unified desktop, and the specific form may be graphics and/or text.
  • the unified desktop not only includes the metadata of the application type and the metadata of the file type, but also includes the metadata of the search function used for data search in the target data set, so as to more comprehensively expand the information displayed on the unified desktop.
  • the functions and applications enable users to conveniently use the search function metadata to query data among multiple devices.
  • the method further includes: displaying multiple file groups from multiple devices searched through the metadata of the search function, the attributes of the files in different file groups are different from each other, and each file group has different attributes.
  • the attributes include at least one of the following: color, size, font, shape.
  • the search function metadata supports at least one of the following search methods: searching by folders progressively; searching by data type; and searching by keywords.
  • the device displays a search function interface in a pop-up window on the unified desktop, and the search function interface may include a folder search guide identifier, a data type search guide identifier, and a keyword. Search for at least one of the lead identifiers.
  • the device displays the keyword information as shown in Figure 2H, such as a guide pop-up window, and the user enters the In order to search for the keyword "XXX", the device obtains the search result, and displays the search result display interface as shown in Figure 2I.
  • the search result display interface includes keywords such as area, search result display area, and the search result display area includes the equipment display area. , data display area and file type display area, different devices are distinguished by different colors. Correspondingly, files containing the keyword information from the same device are distinguished by the same color.
  • the file type display area provides display of search results by file type.
  • the device supports to provide a cross-device data search function through the metadata of the search function of the unified desktop, which is beneficial to more comprehensively expand the functions and applications of the information display of the unified desktop, so that the user can conveniently use the metadata of the search function.
  • Data query between multiple devices is beneficial to more comprehensively expand the functions and applications of the information display of the unified desktop, so that the user can conveniently use the metadata of the search function.
  • the method further includes: detecting a selection operation for the first metadata, displaying an application of at least one target device in the target device set; detecting a first operation for the at least one target device
  • the pre-selection operation of the first target application of the target device displays the prompt information added to the desktop; the confirmation addition operation for the first target application is detected, and the first target application of the first target device is displayed on the unified desktop.
  • a target application detecting a selection operation for the first metadata, displaying an application of at least one target device in the target device set; detecting a first operation for the at least one target device
  • the pre-selection operation of the first target application of the target device displays the prompt information added to the desktop; the confirmation addition operation for the first target application is detected, and the first target application of the first target device is displayed on the unified desktop.
  • the first metadata may be application type metadata, which is different from the second metadata of file type.
  • the prompt information added to the desktop may be an optional function identifier.
  • the device displays the mobile phone 2 on the current unified desktop.
  • the identification information of the mobile phone 2 can be added to the lower right corner of the metadata, and an association relationship between the displayed WeChat application metadata and the WeChat application on the mobile phone 2 can be established based on the aforementioned multi-device management system.
  • the device locates the cross-device application selected by the user to be added to the unified desktop through the metadata on the unified desktop, so as to realize the flexible cross-device metadata addition function and improve the convenience of use and metadata display. comprehensiveness.
  • the method further includes: detecting a pre-selection operation for the first target application displayed on the unified desktop, and displaying removal prompt information; detecting a pre-selection operation for the first target application After confirming the removal operation, initial metadata other than the first target application is displayed on the unified desktop, where the initial metadata is the metadata displayed on the unified desktop before the removal prompt information is displayed.
  • the device displays optional function identifiers, specifically including removing, starting Wait, after the user chooses to remove the function identifier, the device no longer displays the metadata of the WeChat application associated with the mobile phone 2 on the unified desktop.
  • the unified desktop supports the deletion of metadata, and the operation mode is convenient and efficient, which improves the flexibility of using the metadata of the unified desktop.
  • the method further includes: detecting an application editing request operation for a target application that needs to be run across devices, where the cross-device operation means that the target application is running in Running the target application on a target device other than the native device; displaying a control editing interface, where the control editing interface includes a control editing area and a control display area; wherein the control display area is used to display the first part of the target application Application interface, the first application interface includes at least one native control, and the native control refers to the control on the first application interface displayed by the target application on the native device; wherein, the control editing area is used for displaying The adaptation control corresponding to the selected native control in the at least one native control, the adaptation control refers to the control on the second application interface displayed by the target device when the target application is run, the The first application interface and the second application interface correspond to the same functional interface of the target application.
  • the target application that needs to run across devices may be an application running on the current device from other devices, or an application that needs to be run on other devices from the current device or another device (different from other devices). Make the only decision.
  • the control editing area specifically includes a toolbar, a layout bar and an adaptive space display bar.
  • the toolbar includes control filtering function buttons divided by control types, such as button Button, text box TextBox, picture box PictureBox, progress bar ProgressBar, etc.
  • the bar includes buttons for editing functions such as zoom, move, angle adjustment, and flip.
  • the control display area supports determining the adaptation control selected by the user based on the position calibration and other methods.
  • the unified desktop of the device provides the user with an entry for the application editing request operation, so that the user can customize the editing of the functional interface of the target application that needs to be run across other devices, without the need for a third party to develop and adapt the application.
  • the tedious operation improves the convenience and flexibility of application cross-terminal operation function interface settings, and improves the efficiency of multi-device management.
  • the method further includes: outputting the target content associated in the running state of the target application through target metadata.
  • the specific form of output target content may be displaying image information (image, video, etc.) through a display screen, playing corresponding audio information through a speaker, and outputting corresponding vibration information through a vibration sensor or the like.
  • the target application may be any type of application, such as a game application, a music application, an office application, a chat application, etc., which is not uniquely limited here.
  • the target content associated in the running state of the target application includes but is not limited to: audio, video, picture, text, action information (for example, vibration information) and the like.
  • the relationship between the target metadata and the target application may specifically include the following situations:
  • the target application is the local application of the current device
  • the target metadata is associated with the application of the current device (for example, the game application installed on the current device, WeChat, etc.),
  • the device displays multiple applications, including the target application, and the user further selects to start the target application, so that the target application is started and runs, And display its corresponding function interface. For example, if the target metadata is associated with a locally installed WeChat application and a game application, after clicking the target metadata, a pop-up window will display the WeChat application metadata and game application metadata. The user selects the WeChat application metadata, and the current device detects the WeChat application metadata. If the type of application metadata is the application type and is associated with the WeChat application of the current device, the current device runs the WeChat application and displays its functional interface.
  • the target metadata is associated with a single application of the current device, after the user chooses to activate the target metadata, the current device will run the single application and display its function interface.
  • the target metadata is directly associated with a locally installed WeChat application. After the user clicks on the target metadata, the current device runs the WeChat application and displays its functional interface.
  • the target application is the local application of the current device
  • the target metadata is associated with the data of the same file type of multiple devices in the target device set (for example: data such as documents, videos, pictures, music and other data of device 1 and device 2, These data can be read by a locally compatible application), the user selects the target metadata displayed on the unified desktop, and the device starts the target application.
  • the current device pop-up window displays the documents of multiple devices associated with the "document” metadata, the user further selects the target document, and the current device calls a locally compatible application such as WPS to read the target document and display the target document.
  • a locally compatible application such as WPS
  • the third type the target application is the application of the remote device, and the target metadata is the metadata of all applications associated with the remote device.
  • the user selects the target metadata displayed on the unified desktop, and the device further displays the number of installations on the remote device.
  • metadata of each application including the metadata of the target application, the user further selects the metadata of the target application, so that the target application is started and run on the remote device, and the current device can display the metadata of the target application across the terminals.
  • the functional interface can also play audio content, output action information, etc., and support reverse control, etc., and the functional interface displayed on the current device can be consistent with the size of the remote device, or it can be adaptive or predefined. , in a word, for the user, the target application of using the remote device across the terminals can be implemented on the current device.
  • the PC display screen can display the unified desktop, and at the same time display the same user interface as the mobile phone running the Meituan app. .
  • the functional interface of the target application displayed by the current device may be implemented by the current device based on any one of a distributed application mechanism, a locally compatible application mechanism, and a redirected application mechanism, which is not uniquely limited here.
  • a distributed application mechanism e.g., a locally compatible application mechanism
  • a redirected application mechanism e.g., a redirected application mechanism
  • the method further includes: detecting a selection operation for target metadata of the unified desktop; performing a selection operation of the target metadata according to the type of the target metadata manual.
  • the process of performing the use of the target metadata according to the type of the target metadata includes:
  • Step 11 judge the type of the target metadata
  • Step 12 if the type of the target metadata is a file type, execute a local application process, and the local application process refers to reading the data associated with the target metadata through a local adaptation application, and the local adaptation application reads the data associated with the target metadata.
  • the local application process refers to reading the data associated with the target metadata through a local adaptation application
  • the local adaptation application reads the data associated with the target metadata. Refers to the application installed on the current device that can use the data associated with the target metadata;
  • the data associated with the metadata of the file type is some general data, such as videos, documents, pictures, etc., and there are professional software on each system to view and edit these contents.
  • the device will use the local application to run these data, which can ensure the delay of the screen and operation, because it is run locally, so the experience of these two aspects is better.
  • the local application process includes the following steps:
  • the redirection process is performed.
  • Step 13 if the type of the target metadata is an application type, obtain application-related information of the target metadata;
  • Step 14 according to the application-related information, determine whether the current device has a suitable distributed application
  • Step 15 If there is a suitable distributed application, execute a distributed application process, and the distributed application process refers to: the background service of the application associated with the target metadata runs on the remote device, and the user of the application runs on the remote device.
  • the interface UI runs on the current device, and the UI and the background service are connected through the distributed capability system to realize the cross-end operation of the distributed application;
  • the distributed application process includes the following steps: establishing a connection between the current device and the target device that provides the service of the target metadata; interacting with the server to verify whether the current device uses the service on the target device or not through the user's Authorization; if authorized by the user, the distributed application on the current device invokes the service on the target device in an agreed manner.
  • the multi-device collaborative management system architecture for distributed applications includes a server and a terminal, the server includes a user management module, a service management module and a device management module, and the terminal includes a collaboration module, a service management module and a connection module.
  • the device management module is responsible for the management of multiple devices under the user name.
  • the service management module is responsible for serving the application and providing the capability of remote invocation.
  • Collaboration module interaction is responsible for providing users with an interactive way to achieve the goal of multi-device collaboration.
  • the distributed application will pass the application information, source device information, and target device information to the distributed application framework for processing, and let it run on the target device. A detailed description will be given below.
  • the device is managed through the user account, and the device is registered in the user management module of the cloud. There will be a connection module on the device, which will regularly report its own information to the device management module in the cloud to ensure the accessibility of the device.
  • the application can query all devices under the user name through the device management module.
  • the service management module can manage the display service, sound service, input service, location service, payment service and other services on the terminal in a unified manner.
  • the main process is as follows:
  • Step 151 The developer develops the service through a certain interface specification
  • the developed service needs to conform to certain development specifications so that it can be recognized by the service management module on the device.
  • Some default services will be developed on the device. These services are the exposure of capabilities on the device, such as display, sound, input, positioning, etc. These capabilities are consistent with the capabilities exposed by the existing device operating system. Many applications will If it is used, it is only exposed again in the form of a prescribed service. Make these capabilities accessible to remote devices as well.
  • Step 152 Register the service
  • the service management module on the device When the service management module on the device starts, it will scan the services developed according to the given method on the device and record these services; at the same time, when the service is installed and uninstalled, the service management module on the device can monitor these services. changes and record them.
  • the service management module in the cloud will store these services according to the granularity of devices, applications and services.
  • Step 153 Service query
  • the user After logging in, the user can request the service query from the service management module in the cloud.
  • the service management module There are mainly the following query methods:
  • step 153 we find the service that needs to be used, and the device that provides this service.
  • Step 1541 Establish a connection between the current device and the service providing device through the connection module and the cloud device management module.
  • Step 1542 look up the cloud service management module, and verify whether the current device has passed the authorization of the user for the service on the device that provides the service.
  • Step 1544 is skipped if authorized; step 1543 is not performed.
  • Step 1543 A prompt is popped up on the remote device, and the device that needs to provide the service is authorized to call the service.
  • Step 1544 the service gateway module on the device providing the service detects whether the current service is available, if the current service is not started, the current service is started; if the startup fails, the call is terminated, and a prompt pops up on the current device that the access to the remote service fails.
  • Step 1545 the application on the current device invokes the service on the device that provides the service in an agreed manner.
  • the device providing services here can be a remote device or a current device. If it is the current device, step 1541 is omitted.
  • Multi-device collaborative interaction If developers use the interface specifications provided by manufacturers to develop applications, they can choose the capabilities of different devices in the application; in reality, third-party applications generally do not use special methods to deal with them. Therefore, we can use system calls to achieve our goal.
  • a user opens an application on the device regardless of whether the application has been developed using the specified interface specification, relevant collaborative processes can be evoked through some special interactions.
  • the game application running on the mobile phone supports the display of other devices, specifically, displaying the distributed application function selection pop-up window on the game interface, and providing separate images, optional support
  • the devices for separating images include the living room TV shown in the figure, my tablet, etc., and the optional distributed application functions also include separating sound, separating control, and so on.
  • the game application running on the mobile phone supports the display of other devices.
  • the distributed application function selection pop-up window is displayed on the game interface, and a separate sound is provided.
  • Optional Devices that support sound separation include Bluetooth headsets, Bluetooth speakers, and living room TVs, etc.
  • the optional distributed application functions also include separation of images and separation of controls.
  • the game application running on the mobile phone supports the display of other devices.
  • the distributed application function selection pop-up window is displayed on the game interface, and separate control is provided.
  • Optional Devices that support separate control include gamepads, mouse keyboards, and mobile phone touch controls.
  • Optional distributed application functions also include separate images and sounds.
  • the user can control the game application through the mouse and keyboard, and display the game interface through the PC.
  • the game application running on the PC supports the use of other devices to separate images, separate sounds, and separate controls, thereby effectively improving game experience.
  • the collaboration module will pull up the system pop-up window.
  • the collaboration module accesses the service governance module in the cloud to obtain the service list.
  • the service list represents the list of capabilities that can be coordinated, and will be displayed as capabilities that the system can collaborate with.
  • the collaboration module accesses the service management module in the cloud to obtain the device list.
  • a device list When the user selects a capability, a device list will pop up.
  • the device list here does not display the device name, but will be combined with specific capabilities as expressions, such as mouse and keyboard, mobile phone touch, etc.
  • Step 16 if there is no suitable distributed application, then find out whether the current device has a local compatible application corresponding to the target metadata;
  • Step 17 If the current device has a locally compatible application corresponding to the target metadata, start a local simulation environment and run the locally compatible application in the simulation environment.
  • the locally compatible application refers to an application that is consistent with the application type indicated by the application-related information, such as an audio playback application. If the application-related information indicates the audio playback application 1 of the remote device, and the local There is an audio playback application 2 of the same type as the audio playback application 1, then the audio playback application 2 can be started. Further, the audio data stream in the audio playback application 1 can be transferred to the current device through the aforementioned multi-device management system, and the The audio playback application 2 implements playback.
  • the locally compatible application runs on the current device, reads data from the remote device, or temporarily synchronizes the data on the remote device to the local device.
  • PC and mobile phone are the two most commonly used devices by users, but PC cannot directly run Android applications (such as games) in principle. There is a part of the application module, so that the application on the mobile phone can be run on the PC.
  • Step 18 if the current device does not have a local compatible application corresponding to the target metadata, start a redirection process, and the redirection process refers to: the target application associated with the target metadata runs on the remote device. above, redirecting the picture and/or sound of the target application from the remote device to the current device, and redirecting the peripheral input information of the current device to the target application of the remote device.
  • the redirection process does not require the support of third-party developers
  • the program runs on the remote device, redirects the screen and sound of the program from the remote device to the current device, and redirects the peripheral input information of the current device. into the specified program of the remote device.
  • the application redirection solution provided by the embodiments of the present application can support modes such as one-to-many, many-to-one, and many-to-many. In this way, although the target device does not have an application installed, it can implement the corresponding functions when the application runs on the source device, and supports reverse control interaction.
  • Step 21 according to the type of the current device and the type of the device where the application or data associated with the metadata resides, determine whether the content flow is from a large screen to a small screen. If yes, go to step 22, otherwise go to step 25.
  • Step 22 Interact with the server to check whether the target metadata has an application definition based on control binding redirection.
  • the described control-based redirection specifically obtains a new application form by assembling some components in the original application, will use the system application to capture the content information of the current application, and redraw at the remote end.
  • the redirection system on the device uses the pre-made application definitions to complete the cross-end use of the application.
  • the application editor will run on a development device such as a PC, and possibly on a device participating in cross-end.
  • the device redefines the functional interface in the running state of the application, and the application on the source device completes the transition from the source device to the target device.
  • Figure 2V is a schematic diagram of redefining the functional interface of the streaming application.
  • the original application interface of the QQ Music application on the mobile phone includes multiple native controls. Through the implementation mechanism of the aforementioned control editing interface, the QQ Music displayed on the watch across devices can be defined.
  • the adaptation controls in the adaptation function interface of the in which the external background part can be preset or customized, and the adaptation controls selected for editing are specifically the playback controls, the previous control, the next control, and the name control.
  • the controls displayed on the functional interface of the target device in this way may be different from the controls on the original application interface, and attributes such as color, layout, and shape can be changed.
  • device B is the user's current device and device A is the remote device.
  • the user wants to use the app on device A on device B.
  • an application across terminals it mainly includes two processes:
  • the screen display process specifically includes the following steps (1.) to (7.).
  • Step (1.) The information collection module of device A obtains the application definition for device A of the application, if it does not exist locally, obtains it from the cloud, and then saves it to the local cache.
  • Step (2.) In the application definition, there will be multiple control information. For each control, there will be a unique path pointing to the native control in the running application. The information collection module obtains the native control through this unique path, and Get the information of the native control, such as text, image address, progress value, etc.
  • Step (3.) Repeat step 2 to obtain all the control information in the application definition, integrate all the information accordingly and transmit it to the sending module of the device A.
  • the integrated information includes the following:
  • each control information is as follows:
  • control ID control value
  • Step (4.) The sending module of device A compresses the information and sends it to the receiving module of device B.
  • the key frame has all the data
  • the difference frame has the data of the difference with the key frame.
  • Step (5.) The receiving module of the device B restores the information of the running application through the historical data and the latest data received, and sends it to the application display module of the device B
  • Step (6.) The application display module of the device B obtains the application definition of the current application through the cloud, and saves it locally.
  • Step (7.) The application display module parses the application definition and draws the display content.
  • Step A Traverse the bound controls in the application definition.
  • Step B Generate an adaptation control according to the definition information of each control.
  • the value on the adapted control is determined. If the property binding information on the control is assumed to be the copy value strategy, The value is directly assigned to the current adaptation control.
  • Step C Traverse the external resource controls, and set a specified value, such as a given picture, for each external resource control.
  • Step D According to the order of the controls in the application definition, draw all the two controls at one time to complete the display.
  • the application of device B is a redrawn application
  • there are two ways to do reverse control one is to adapt the feedback of the control, and the other is to do the mapping through the positional relationship.
  • the former is simpler, but may not be allowed for security reasons; the latter is more complex, but more general, and theoretically not restricted.
  • the main process includes the following steps.
  • Step (1.) The application display module of the device B collects the input operation of the local device.
  • Step (2.) In the input processing module of the device B, an application definition is acquired, an adaptation event is generated, and relevant operation information is generated.
  • the application on device B is an application form rather than a picture form
  • a corresponding control event such as a click event
  • the application definition for device B of the application is obtained from the cloud and cached locally on device B.
  • Device B generates information with a control ID, such as (control ID, click), according to the application definition and in combination with the adaptation control that generates the event.
  • a control ID such as (control ID, click)
  • Step (3.) The sending module of the device B compresses the operation information and transmits it to the receiving module on the device A.
  • Step (4.) After receiving the compressed operation information, the receiving module on the device A restores the original operation information, and transmits it to the input injection module of the device A.
  • Step (5.) After receiving the operation information, the input injection module of the device A injects it into the running application.
  • the input injection module here will choose two ways to inject input according to the situation of the device and application. The specific steps are as follows:
  • Step A The input injection module of device A obtains the application definition and caches it locally.
  • Step B Extract the input bindings of each control from the application definition.
  • Step C For an input operation, find the control that generates the input, find the input binding information, and check whether the event has been registered.
  • Step D If it has been registered, convert the input operation into the event of the native control according to the registration information, and inject it into the application.
  • the click event of the adapted control is converted into the click event of the native control.
  • Step E If not registered, find the location information of the control from the application definition. Convert the input operation into an operation for this location information and inject it into the application.
  • the click event of the adapted control is converted into a click operation at the position of the control in the screen of the running application.
  • Step (6.) The application running on the device A responds to the operation and updates the value of the native control, and these changes are updated on the device B through the screen display process.
  • Step 23 Interact with the server to check whether the target metadata has an application definition based on partial screen projection redirection.
  • the retargeting based on partial screen projection will change the original display and layout of the application to adapt to different screen sizes.
  • a plurality of partial pictures in the original application are captured and rearranged at the remote end to adapt to different screen sizes.
  • the redirection system on the device uses the pre-made application layout to complete the cross-end use of the application.
  • the application editor will run on a development device such as a PC, and possibly on a device participating in cross-end.
  • device B is the user's current device and device A is the remote device.
  • the user wants to use the app on device A on device B.
  • an application across terminals it mainly includes two processes.
  • the screen display process specifically includes the following steps (1.) to (7.).
  • Step (1.) Collect the picture of the device A running the App, which is called picture A, and this picture is consistent with that displayed by the App.
  • Step (2.) Device A obtains the layout of the application for device B, and if it does not exist locally, obtains it from the cloud.
  • Step (5.) According to the definition of the layout, the device A generates picture information according to the position, size, rotation and other information depicted in the definition of the selected content in the screen A, and adds it to the list.
  • Step (6.) Device A repeats step 5 until the contents of the selected part in the layout are added to the picture list.
  • Step (7.) Device A adds all the pictures defined by external resources in the layout to the picture list.
  • Step (8.) The device A sequentially writes the picture information into the same picture according to the hierarchical relationship defined by the layout.
  • Step (9.) If the selected content has similar parts in the new layout, the device A performs interpolation processing on the picture to smooth the transition of different pictures.
  • Step (10.) Device A transmits the generated picture to the encoding module.
  • Step (11.) The encoding module of device A edits the video into H264 or H265 format, and sends it to the sending module of device A.
  • Step (12.) The sending module of the device A sends the encoded data stream to the device B, using a proprietary streaming media protocol.
  • Step (14.) The display module of the device B displays the video stream decoded by the decoding module.
  • Step (2.) Device B calculates the currently triggered selected part according to the layout information of the application.
  • the layout information is cached locally, if not, it is obtained from the cloud application management module. Taking click as an example, when the current application is clicked, the position of a point will be obtained. Through the layout information, we can calculate which selected part of the original application the current point belongs to, and the clicked position is in the selected part. position offset.
  • Step (3.) Device B calculates the position of this point in the original application through the selected part and the relative offset position of the click.
  • the selected part may be scaled and rotated when it is made into a new application, this information is recorded in the application layout. Through this information and the relative position offset of the selected part obtained in the previous step, and then through The offset of the selected part in the original screen can calculate the position of the click in the original application.
  • the click here can be extended to other actions.
  • Step (4.) Device B uses device-level operation adaptation to adapt the operations of different input devices.
  • the operation of the touch screen is converted into the operation of the mouse button.
  • Step (5.) Device B sends the converted input operation to device A.
  • device A After device A receives the input operation, it will respond accordingly, such as clicking the play button to start playing music, and the button becomes pause at the same time.
  • the application on the source device completes the transition from the source device to the target device by redrawing the functional interface in the running state of the application.
  • a schematic diagram of the functional interface of the re-drawing (that is, the redirection mechanism based on partial screen projection) is used to transfer the application.
  • the original application interface of the QQ music application on the mobile phone includes multiple native controls.
  • the redirection mechanism takes a screenshot of the core area containing the main controls on the original application interface of the QQ Music app on the mobile phone and redraws it on the interface of the watch, so as to realize part of the screen projection. In this way, the controls displayed on the functional interface on the target device are transferred. Consistent with the controls of the original application interface, properties such as color, layout, shape, etc. cannot be changed.
  • Step 24 the current device detects whether the redirection from a large screen to a small screen is supported.
  • the main process of redirecting from a large screen to a small screen includes the following steps.
  • Step (1.) The acquisition module of the device A acquires the picture of the running application and transmits it to the encoding module.
  • Step (2.) The encoding module of device A encodes and compresses the picture, and transmits it to the decoding module of device B.
  • Step (3.) The decoding module of the device B transmits the decoded picture to the display module.
  • Step (4.) The display module of device B displays a picture. In this process, we perform special processing on the projected picture.
  • the display module of device B it is allowed to zoom in, zoom out, and move the displayed content in a certain interactive manner.
  • Step (5.) The input operation of the input device is transmitted to the input processing module of the device B for input processing.
  • the input processing module of device B will decide what kind of response the input operation will produce. It mainly distinguishes two kinds of operations. The first is to zoom and move the projected screen; the second is to transmit the operation to the remote end and trigger the operation response of the remote application.
  • the main flow includes the following steps A to F.
  • Step A The user triggers a mobile operation on device B through the input device.
  • Step B Device B determines whether the picture is an original picture or an enlarged picture.
  • Step C If it is an original picture, device B directly transmits the adapted operation to device A.
  • Step D If it is an enlarged picture, the device B judges again whether the current picture reaches the edge.
  • the picture reaching the edge refers to a part of the picture displayed on the smaller screen that is close to the edge of the enlarged picture.
  • the small screen has reached the upper edge of the screen. If the moving direction is downward at this time, the judgment is established. If the moving direction is not down, the judgment is also invalid at this time.
  • Step E If the edge has not been reached, the device B moves the screen according to the input moving direction.
  • the screen will move according to the movement direction formed by the input.
  • Step F If the edge is reached, the device B adapts the movement input and transmits it to the device A.
  • the operations after reaching the edge are transferred to device A for processing. Under these operations, the screen on device A will change in some cases, such as list scrolling, and these changes will be synchronized to the small screen of device B.
  • the interactive position will be scaled accordingly and then transmitted to the remote device.
  • Different devices may have inconsistent systems, and the information of different input devices will be converted, such as mouse key operation converted to touch screen operation.
  • Step (7.) The input injection module of the device A injects the input into the original application, gets a response, and refreshes the application screen.
  • the device uses different interactive forms on different devices to zoom in and out of the screen on the projection screen, and distinguish the differences between zooming, moving operations, and operations on the remote application itself, so that small-screen devices can both display larger and larger images. It can display and operate the application of the remote device without affecting the operation function of the application itself, thereby improving the use experience of large-screen to small-screen projection.
  • Step 25 use a general screen projection method to perform redirection.
  • the common application can stream the multimedia information of the application running on the machine to the target device through the screen projection method through the screen projection service.
  • the flow strategy module needs to push the application information, source device information, and target device information to the screencasting service, and the screencasting service completes the transfer.
  • the above processes are the default processes of the unified desktop. These processes are not completely fixed, allowing users to reset the priority of these methods as they like.
  • this solution includes a variety of cross-device application usage solutions. Mainly divided into three categories:
  • the flow of applications among multiple devices can also be implemented based on a distributed application framework.
  • the distributed application framework includes a source and destination determination module and a flow strategy module.
  • the first step is to enter various recognition modules according to the user's input behavior or input voice, etc.
  • the recognition module is responsible for identifying the source and destination devices that need to be transferred, because there are clear source and destination devices (such as gesture recognition, which can There may be clear source and destination device ID information), or there may be vague source and destination information (for example, the voice "from this machine to the computer", there may be multiple devices under the same account, so the identified source and destination are vague) , the identification module transmits the identified source and destination to the source and destination determination module for final determination.
  • the source and destination ID information is unclear, for example, the destination is a computer
  • the source and destination modules push the final source and destination IDs to the circulation strategy module to determine the strategy of how the application is circulated.
  • the circulation strategy module accesses the application information module to obtain the name and type of the application currently running in the foreground.
  • the device can adopt different application circulation strategies according to the application type to realize the cross-end operation of the application.
  • the application types can be divided into four types. , each of which is handled in different ways, which are described in detail below.
  • Distributed application is an application with separated front and back ends. Different UI systems can be run on different types of devices, and back-end operations can be run on devices different from UI.
  • the distributed application will pass the application information, source and destination information to the distributed application framework for processing, and let it run on the target device.
  • a distributed application flows between two devices.
  • device A runs a distributed application X, which is divided into two parts.
  • the UI of application X and the server of application X are both running.
  • the UWB points to device B
  • the UI of application X of device A is suspended after the application flow framework, and the UI of application X of device B starts to run, and the application X server of device A is used.
  • the UWB identification module sends clear source and destination device IDs to the source and destination determination modules. Therefore, the source and destination modules pass the ID information of source device A and destination device B to the circulation strategy module.
  • the circulation strategy module queries The distributed application service is currently in the foreground, so an application transfer request is initiated to the distributed application framework.
  • the distributed framework completes the entire distributed application transfer process, and is responsible for suspending the UI of application X on device A and pulling up the device. B's application X's UI and notifies the UI runtime to use device A's application X's server.
  • Screencasting application the current device running in the foreground is a screencasting service, and what is displayed to the user is a screencast delivered to the machine by other devices.
  • the transfer strategy module will push the source device information and target device information to the screencasting service, and the screencasting service will complete the transfer between different devices.
  • the screencasting flow under three devices when device B casts the screen to device A, device A touches device C through NFC, and after the application flow framework, it is changed to device B to cast the screen to device C.
  • the NFC identification module After the NFC touch, the NFC identification module sends a clear source and destination device ID to the source and destination determination modules. Therefore, the source and destination modules pass the ID information of source device A and destination device B to the circulation strategy module.
  • the circulation strategy module The screencasting service currently in the foreground is queried, so a screencasting transfer request is initiated to the screencasting service.
  • the screencasting transfer service completes the screencasting from device B to device A, and starts the screencasting from device B to device C, complete. The whole process of screen casting.
  • Relay application is different from distributed application. It pushes the data stream to the target device through the local file service, so that the target device uses the local adaptation application when running, such as a video player, When streaming, we convert the file into a data stream of the hypertext transfer protocol http for playback on the target device.
  • the initial situation is that the player is running on the mobile phone to play the local video.
  • the target is recognized as the computer, and the same video is played on the computer after applying the streaming framework.
  • the mobile terminal pauses playback.
  • the speech recognition system recognizes the vague destination type of the computer, and sends this information to the source and destination determination modules.
  • the source and destination determination modules access the connection service, query all currently connected computer devices, and find that there is only one computer type
  • the device confirms the destination ID, and pushes the information to the circulation strategy module.
  • the circulation strategy module finds that the foreground application is a relay application at this time, and pushes this information to the relay framework, which completes the circulation of the application, and finally
  • the player on the computer uses the file service on the mobile phone to complete data streaming, and runs the player application on the computer to play.
  • Ordinary applications can stream the software running on the machine to the target device through the screen projection service.
  • the flow strategy module needs to push the application information, source device information, and target device information to the screencasting service, and the screencasting service completes the flow.
  • the initial state is that a common application X is running in device A, and the user touches device A with device B using NFC.
  • the screencasting service is enabled on device B and the application in device A is displayed. screen of X.
  • the NFC identification module After the NFC touch, the NFC identification module sends a clear source and destination device ID to the source and destination determination modules. Therefore, the source and destination modules pass the ID information of source device A and destination device B to the circulation strategy module.
  • the circulation strategy module The application information is queried, and it is obtained that the application currently in the foreground is a common application, so a screen casting request is initiated to the screencasting service, and the screencasting service of device A initiates screencasting to device B and displays the screen of application X of device A to the device. on B.
  • the unified desktop supports playing the target content of the target application in the running state through metadata, so as to realize the application use in the multi-device interaction scenario and improve the practicability and flexibility of multi-device interaction.
  • the method further includes: the current device as the source device sends the target content to the target device to achieve the target Content flows across devices;
  • the source device is a content sending device determined by the identification device according to the first gesture detection associated data generated by a first number of devices, and the first number of devices includes the identification device, the source device, and the The target device, the identification device is any one of the first number of devices; the target device is the content determined by the identification device according to the second gesture detection associated data generated by the second number of devices A receiving device, the second number of devices includes the identification device and the target device.
  • the gesture detection associated data may be a gesture image of the user collected by the device through a camera, or may be a confidence level of the gesture calculated by the device according to the collected gesture image. Confidence determines the source device that needs to stream content across devices.
  • the gesture corresponding to the first gesture detection associated data may be a grasping gesture action of a human hand (for example, as shown in the example diagram in FIG. specific state)
  • the gesture corresponding to the second gesture detection associated data may be a release gesture action of a human hand (for example, the gesture example shown in FIG.
  • the first quantity may be 2, 3, 4, etc.
  • the second quantity is one less than the first quantity.
  • the device initiates the flow of streaming media information by recognizing the user's gesture, and by recognizing the user's hand grabbing and releasing actions, sends a flow instruction to the user's device, completing the application's streaming media information in multiple transfers between devices.
  • the method further includes: outputting information for prompting content flow.
  • the method further includes: entering the login information of the target user account through the unified desktop account login portal of the current device, and sending the login information of the target user account to the server,
  • the login information of the target user account is used by the server to perform the following operations: after verifying that the login information is legal, obtain the metadata set of the unified desktop of the target user account, and send the information to the peer device requesting login. receiving the metadata set of the unified desktop of the target user account from the server, where the metadata set is used for the current device to display the unified desktop.
  • the current device can obtain the metadata list from the server based on the account login mechanism, thereby reducing the data processing pressure of the local device, reducing the delay, and improving the user experience on the terminal side. .
  • an embodiment of the present application further provides a method for accessing metadata, including the following steps:
  • Step 301 the first electronic device displays a unified desktop, and the unified desktop includes metadata, and the metadata is associated with data on the first electronic device and a second electronic device that is communicatively connected to the first electronic device;
  • Step 302 in response to the selection operation on the target metadata on the unified desktop, access the metadata on the first electronic device and the second electronic device according to the type of the target metadata.
  • the application or data of different devices can be flexibly called for use through the unified desktop metadata, which improves the flexibility and convenience of data use in multi-device management scenarios, and improves user experience.
  • the metadata includes application type metadata and file type metadata; the application type metadata is associated with applications of the first electronic device and the second electronic device, the The metadata of the file type is associated with data of the same file type in the first electronic device and the second electronic device.
  • the accessing metadata on the first electronic device and the second electronic device according to the type of the target metadata includes:
  • the local application process refers to reading the data associated with the target metadata through a local adaptation application
  • the local adaptation application refers to the current An application installed on the device that can use the data associated with the target metadata
  • the type of the target metadata is an application type, obtain application-related information of the target metadata;
  • the distributed application process means that the background service of the application associated with the target metadata runs on the remote device, and the user interface UI of the application runs on the remote device. On the current device, the UI and the background service are connected through a distributed capability system to realize cross-terminal operation of the distributed application;
  • the current device has a locally compatible application corresponding to the target metadata, start a local simulation environment, and run the locally compatible application in the simulation environment;
  • a redirection process is started, and the redirection process refers to: the target application associated with the target metadata runs on the remote device, and the The picture and/or sound of the target application is redirected from the remote device to the current device, and the peripheral input information of the current device is redirected to the target application of the remote device.
  • the local application process includes the following steps:
  • the redirection process is executed.
  • the distributed application includes the following steps:
  • the distributed application on the current device invokes the service on the target device in an agreed manner.
  • the redirection process includes the following steps:
  • the type of the current device and the type of the device where the application or data associated with the target metadata is located determine whether it is a content flow from a large screen to a small screen;
  • the current device detects whether it supports large-screen to small-screen redirection.
  • An embodiment of the present application provides a display device for desktop metadata
  • the display device for desktop metadata may be a terminal.
  • the device for displaying desktop metadata is configured to execute the steps performed by the terminal in the above method for displaying desktop metadata.
  • the apparatus for displaying desktop metadata provided in this embodiment of the present application may include modules corresponding to corresponding steps.
  • the display device of desktop metadata may be divided into functional modules according to the foregoing method examples.
  • each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module.
  • the above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.
  • the division of modules in the embodiments of the present application is schematic, and is only a logical function division, and there may be other division manners in actual implementation.
  • FIG. 4 shows a possible schematic structural diagram of the display device for desktop metadata involved in the above embodiment.
  • the display device 4 of desktop metadata includes:
  • the display unit 40 is configured to display a unified desktop, where the unified desktop includes metadata, the metadata includes application type metadata and file type metadata, and the application type metadata is associated with one or more of the target device sets.
  • the metadata of the file type is associated with data of the same file type of at least one device in the target device set.
  • the unified desktop further includes search function metadata, where the search function metadata is used to perform data search in a target data set, and the target data set corresponds to the data of the devices in the target device set. data.
  • the display unit 40 is further configured to: display multiple file groups from multiple devices searched through the metadata of the search function.
  • the attributes of the files in different file groups are different from each other.
  • Each file group includes single or multiple files of the same device; the attributes include at least one of the following: color, size, font, and shape.
  • the search function metadata supports at least one of the following search methods: searching by folder progression; searching by data type; searching by keyword.
  • the display unit 40 is further configured to: detect the selection operation for the first metadata, and display the application of at least one target device in the target device set; detect the application for the at least one target device The pre-selection operation of the first target application of the first target device, the prompt information added to the desktop is displayed; the confirmation adding operation for the first target application is detected, and the unified desktop is displayed. the first target application.
  • the display unit 40 is further configured to: detect a pre-selection operation for the first target application displayed on the unified desktop, and display removal prompt information; The confirmation removal operation of the target application, the initial metadata other than the first target application is displayed on the unified desktop, and the initial metadata is the metadata displayed on the unified desktop before the removal prompt information is displayed data.
  • the display unit 40 displays the unified desktop, it is further configured to detect an application editing request operation for a target application that needs to be run across devices, where the cross-device operation means that the target application is running in Running the target application on a target device other than the native device; displaying a control editing interface, the control editing interface including a control editing area and a control display area;
  • control display area is used to display the first application interface of the target application
  • the first application interface includes at least one native control
  • the native control refers to the first application displayed by the target application on the native device.
  • control editing area is used to display the adaptation control corresponding to the selected native control in the at least one native control
  • adaptation control refers to the display of the target device when running the target application.
  • a control on a second application interface, the first application interface and the second application interface correspond to the same functional interface of the target application.
  • the apparatus further includes: an output unit 41, configured to output the target content associated with the running state of the target application through target metadata after the display unit 40 displays the unified desktop.
  • the target content includes at least one of the following: audio, video, picture, text, and action information.
  • the apparatus further includes: a using unit 42 for detecting a selection operation of target metadata for the unified desktop after the display unit 40 displays the unified desktop; according to the target metadata
  • the type of data executes the usage flow of the target metadata.
  • the using unit 42 is specifically configured to: determine the type of the target metadata;
  • the local application process refers to reading the data associated with the target metadata through a local adaptation application
  • the local adaptation application refers to the current An application installed on the device that can use the data associated with the target metadata
  • the type of the target metadata is an application type, obtain application-related information of the target metadata;
  • the distributed application process means that the background service of the application associated with the target metadata runs on the remote device, and the user interface UI of the application runs on the remote device. On the current device, the UI and the background service are connected through a distributed capability system to realize cross-terminal operation of the distributed application;
  • the current device has a locally compatible application corresponding to the target metadata, start a local simulation environment, and run the locally compatible application in the simulation environment;
  • a redirection process is started, and the redirection process refers to: the target application associated with the target metadata runs on the remote device, and the The picture and/or sound of the target application is redirected from the remote device to the current device, and the peripheral input information of the current device is redirected to the target application of the remote device.
  • the apparatus further includes a sending unit 43 for the current device as a source device to send the target content to the target device to realize the cross-device flow of the target content;
  • the source device is an identification device A content sending device determined according to the first gesture detection associated data generated by a first number of devices, where the first number of devices includes the identification device, the source device, and the target device, and the identification device is Any one of the first number of devices;
  • the target device is a content receiving device determined by the recognition device according to the second gesture detection associated data generated by the second number of devices, the second number of A device includes the identification device and the target device.
  • the output unit 41 is further configured to: output information for prompting content flow.
  • the display device 5 of desktop metadata includes: a processing module 50 and a communication module 51 .
  • the processing module 50 is used to control and manage the actions of the display device of the desktop metadata, eg, the steps performed by the display unit 40, and/or other processes used to perform the techniques described herein.
  • the communication module 51 is used to support the interaction between the display device of the desktop metadata and other devices.
  • the display device for desktop metadata may further include a storage module 52, and the storage module 52 is used to store program codes and data of the display device for desktop metadata.
  • the processing module 50 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), ASIC, FPGA or other programmable Logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure.
  • the processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
  • the communication module 51 may be a transceiver, an RF circuit, a communication interface, or the like.
  • the storage module may be a memory.
  • the above-mentioned desktop metadata display apparatuses can all execute the steps performed by the terminal in the above-mentioned desktop metadata display method shown in FIG. 2A .
  • FIG. 6 shows a possible schematic structural diagram of the electronic device involved in the foregoing embodiment.
  • the electronic device 6 includes:
  • a display unit 60 configured to display a unified desktop, where the unified desktop includes metadata, and the metadata is associated with data on the first electronic device and a second electronic device communicatively connected to the first electronic device;
  • the execution unit 61 is configured to, in response to the selection operation on the target metadata on the unified desktop, access the metadata on the first electronic device and the second electronic device according to the type of the target metadata.
  • the metadata includes application type metadata and file type metadata; the application type metadata is associated with applications of the first electronic device and the second electronic device, the The metadata of the file type is associated with data of the same file type in the first electronic device and the second electronic device.
  • the execution unit 61 is specifically configured to: determine the type of target metadata;
  • the local application process refers to reading the data associated with the target metadata through a local adaptation application
  • the local adaptation application refers to the current An application installed on the device that can use the data associated with the target metadata
  • the type of the target metadata is an application type, obtain application-related information of the target metadata;
  • the distributed application process means that the background service of the application associated with the target metadata runs on the remote device, and the user interface UI of the application runs on the remote device. On the current device, the UI and the background service are connected through a distributed capability system to realize cross-terminal operation of the distributed application;
  • a redirection process is started, and the redirection process refers to: the target application associated with the target metadata runs on the remote device, and the The picture and/or sound of the target application is redirected from the remote device to the current device, and the peripheral input information of the current device is redirected to the target application of the remote device.
  • the local application process includes the following steps:
  • the redirection process is performed.
  • the distributed application includes the following steps:
  • the distributed application on the current device invokes the service on the target device in an agreed manner.
  • the redirection process includes the following steps: according to the type of the current device and the type of the device where the application or data associated with the target metadata is located, determine whether it is a content flow from a large screen to a small screen; If the content from the big screen to the small screen flows, interact with the server to check whether the target metadata has an application definition based on control binding redirection; if so, execute the process based on control binding redirection; if not, Then interact with the server to check whether the target metadata has an application definition based on partial screen projection redirection;
  • the electronic device 7 includes: a processing module 70 and a communication module 71 .
  • the processing module 70 is used to control and manage the actions of the electronic device, eg, the steps performed by the display unit 60, and/or other processes used to perform the techniques described herein.
  • the communication module 71 is used to support the interaction between the electronic device and other devices.
  • the electronic device may further include a storage module 72 for storing program codes and data of the electronic device.
  • the processing module 70 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), ASIC, FPGA or other programmable Logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logical blocks, modules and circuits described in connection with this disclosure.
  • the processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
  • the communication module 71 may be a transceiver, an RF circuit, a communication interface, or the like.
  • the storage module may be a memory.
  • An embodiment of the present application further provides a chip, wherein the chip includes a processor, configured to call and run a computer program from a memory, so that a device installed with the chip executes the part described in the terminal in the above method embodiment or all steps.
  • Embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program causes the computer to execute the terminal as described in the foregoing method embodiments some or all of the steps described.
  • Embodiments of the present application further provide a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute the network as in the foregoing method embodiments Some or all of the steps described in the side device.
  • Embodiments of the present application further provide a computer program product, wherein the computer program product includes a computer program, and the computer program is operable to cause the computer to execute some or all of the steps described by the terminal in the above method embodiments.
  • the computer program product may be a software installation package.
  • the steps of the method or algorithm described in the embodiments of the present application may be implemented in a hardware manner, or may be implemented in a manner in which a processor executes software instructions.
  • Software instructions can be composed of corresponding software modules, and software modules can be stored in random access memory (Random Access Memory, RAM), flash memory, read only memory (Read Only Memory, ROM), erasable programmable read only memory ( Erasable Programmable ROM, EPROM), Electrically Erasable Programmable Read-Only Memory (Electrically EPROM, EEPROM), registers, hard disk, removable hard disk, CD-ROM, or any other form of storage medium known in the art.
  • An exemplary storage medium is coupled to the processor, such that the processor can read information from, and write information to, the storage medium.
  • the storage medium can also be an integral part of the processor.
  • the processor and storage medium may reside in an ASIC.
  • the ASIC may reside in access network equipment, target network equipment or core network equipment.
  • the processor and the storage medium may also exist in the access network device, the target network device or the core network device as discrete components.
  • the functions described in the embodiments of the present application may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server or data center via wired (eg coaxial cable, optical fiber, Digital Subscriber Line, DSL) or wireless (eg infrared, wireless, microwave, etc.) means.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the available media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, Digital Video Disc (DVD)), or semiconductor media (eg, Solid State Disk (SSD)) )Wait.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种桌面元数据的显示方法、访问方法及相关装置，包括：显示统一桌面（201），所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。有利于提高统一用户交互入口即统一桌面显示信息的全面性和精细化程度，更好的满足的使用需求。

Description

桌面元数据的显示方法、访问方法及相关装置 技术领域
本申请涉及信息显示技术领域，具体涉及一种桌面元数据的显示方法、访问方法及相关装置。
背景技术
目前，手机等电子设备作为多设备(例如：智能家居场景中音箱、空调、冰箱等)的统一用户交互入口时，当前设备的桌面上可以显示多个设备的运行状态信息，如空调等设备的运行温度、音箱的播放控制控件，这些显示信息使得用户能够了解不同对象在设备层级的状态信息，但难以满足用户针对不同设备的更加精细化的跨端使用需求。
发明内容
本申请实施例提供一种桌面元数据的显示方法、访问方法及相关装置，以期提高统一用户交互入口即统一桌面显示信息的全面性和精细化程度，更好的满足的使用需求。
第一方面，本申请实施例提供一种桌面元数据的显示方法，包括：
显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
可见，本示例中，电子设备的统一桌面作为目标设备集合中多设备的统一用户交互入口，不仅能够显示关联目标设备集合中一个或多个设备的应用的元数据，使得用户能够实现应用层级的跨端控制，还能够显示关联目标设备集合中至少一个设备的相同文件类型的数据的元数据，实现多设备的数据维度的交互机制，使得用户能够进行数据层级的跨端控制，有利于提高统一桌面信息显示的全面性和精细化程度，更好的满足的使用需求。
第二方面，本申请实施例提供一种数据的访问方法，包括：
第一电子设备显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述第一电子设备和与所述第一电子设备通信连接的第二电子设备上的数据；
响应于对所述统一桌面上目标元数据的选择操作，根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据。
可见，本示例中，通过统一桌面的元数据能够灵活调用不同设备的应用或数据进行使用，提高多设备管理场景中数据使用的灵活性和便捷性，提高用户体验。
第三方面，本申请实施例提供一种桌面元数据的显示装置，包括显示单元，
所述显示单元，用于显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
第四方面，本申请实施例提供一种电子装置，包括：
显示单元，用于显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述电子装置以及与所述电子装置连接的第二设备的元数据；
控制单元，用于响应于对所述统一桌面上目标元数据的选择操作，以根据所述目标元数据的类型控制所述电子装置以及所述第二设备中的元数据。
第五方面，本申请实施例提供一种电子设备，包括处理器、存储器、通信接口以及一个或多个程序，其中，所述一个或多个程序被存储在所述存储器中，并且被配置由所述处理器执行，所述程序包括用于执行本申请实施例第一方面或第二方面任一方法中的步骤的指令。
第六方面，本申请实施例提供了一种芯片，包括：处理器，用于从存储器中调用并运行计算机程序， 使得安装有所述芯片的设备执行如本申请实施例第一方面或第二方面任一方法中所描述的部分或全部步骤。
第七方面，本申请实施例提供了一种计算机可读存储介质，其中，所述计算机可读存储介质存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如本申请实施例第一方面或第二方面任一方法中所描述的部分或全部步骤。
第八方面，本申请实施例提供了一种计算机程序，其中，所述计算机程序可操作来使计算机执行如本申请实施例第一方面或第二方面任一方法中所描述的部分或全部步骤。该计算机程序可以为一个软件安装包。
附图说明
下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍。
图1是目前苹果手机的多设备控制界面的示意图；
图2A是本申请实施例提供的一种桌面元数据的显示方法的流程示意图；
图2a是本申请实施例提供的一种“音乐”元数据的显示示意图；
图2b是本申请实施例提供的另一种“音乐”元数据的显示示意图；
图2c是本申请实施例提供的一种“视频”元数据的显示示意图；
图2d是本申请实施例提供的一种“文档”元数据的显示示意图；
图2e是本申请实施例提供的一种“照片”元数据的显示示意图；
图2B是本申请实施例提供的一种统一桌面的示例图；
图2C是本申请实施例提供的一种选取“APP”元数据后的示例界面变化图；
图2D是本申请实施例提供的一种选取“音乐”元数据后的示例界面变化图；
图2E是本申请实施例提供的一种多设备统一管理系统的架构示意图；
图2F是本申请实施例提供的一种搜索功能元数据的示意图；
图2G是本申请实施例提供的一种统一桌面上弹窗显示搜索功能界面的示意图；
图2H是本申请实施例提供的一种关键字信息例如引导弹窗的示意图；
图2I是本申请实施例提供的一种搜索结果展示界面的示意图；
图2J是本申请实施例提供的一种选取APP元数据使得设备显示手机2中的应用的示意图；
图2K是本申请实施例提供的一种在当前的统一桌面上显示手机2的微信应用的示意图；
图2L是本申请实施例提供的一种显示有关联手机2的微信应用的元数据的示意图；
图2M是本申请实施例提供的一种微信应用的跨端应用的控件编辑界面的示意图；
图2N是本申请实施例提供的一种设备启用统一桌面上的元数据的决策流程示意图；
图2O是本申请实施例提供的一种手机端分布式应用的示例图；
图2P是本申请实施例提供的另一种手机端分布式应用的示例图；
图2Q是本申请实施例提供的另一种手机端分布式应用的示例图；
图2R是本申请实施例提供的一种分离画面声音和操控的游戏应用场景示意图；
图2S是本申请实施例提供的一种PC端分布式应用的示例图；
图2T是本申请实施例提供的另一种PC端分布式应用的示例图；
图2U是本申请实施例提供的一种重定向的流程示意图；
图2V是本申请实施例提供的一种重新定义流转应用的功能界面的示意图；
图2W是本申请实施例提供的一种重新绘制流转应用的功能界面的示意图
图2X是本申请实施例提供的一种由手掌张开到握拳的手势变化示意图；
图2Y是本申请实施例提供的一种由握拳到张开手掌的手势变化示意图；
图3是本申请实施例提供的一种桌面元数据的访问方法的流程示意图；
图4是本申请实施例提供的一种桌面元数据的显示装置的功能单元组成框图；
图5是本申请实施例提供的另一种桌面元数据的显示装置的功能单元组成框图；
图6是本申请实施例提供的一种电子装置的功能单元组成框图；
图7是本申请实施例提供的另一种电子装置的功能单元组成框图。
具体实施方式
为了使本技术领域的人员更好地理解本申请方案，下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别不同对象，而不是用于描述特定顺序。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其他步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
如图1所示，苹果手机支持针对多设备的控制，在当前的信息显示界面中，包括当前设备的播放控制按钮和关联不同地点的设备的场景标识，从而用户能够通过选取目标场景标识从而实现针对该目标场景中的设备的跨端使用。
可见，目前的统一用户交互入口所显示的信息不够全面和精细化，难以满足用户的使用需求。
基于上述问题，本申请实施例提供一种桌面元数据的显示方法，下面结合附图进行详细说明。
请参阅图2A，图2A是本申请实施例提供的一种桌面元数据的显示方法的流程示意图，应用于目标设备集合中的任意一个支持触控显示功能的电子设备，如图所示，该方法包括：
步骤201，显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
其中，一个元数据由以下内容组成：
1.显示图片，显示在统一桌面上的图标。
2.名称，显示在统一桌面的名称。
3.唯一路径，能访问这个元数据的唯一路径。
唯一路径中有唯一设备身份标识ID，以及在这个设备下元数据的唯一路径。
4.元数据类别：文件类型(例如：音乐、视频、文档、照片等元数据)、应用类型(例如：设备层级的元数据、聚合应用层级的元数据、独立应用层级的元数据)。
5.每种数据自己的一些属性，如图片的大小、视频的时长等。
元数据的这些属性能够让统一桌面很好的展现这些数据，并能在用户触发使用时，找到合适的打开方式。
本示例中，所述目标设备集合对应所述统一桌面的用户账号下关联的设备，所述元数据是指由至少一个元素组成的功能图标，所述元素是指所述统一桌面上显示的图形和/或文字。
具体实现中，所述统一桌面可以是通过显示在桌面上的应用程序来开启，或者，可以是手机原生桌面中的一个桌面，或者，可以是从设置等系统功能中选取的功能等，具体表现形式不做唯一限定。
具体实现中，应用类型的元数据和文件类型的元数据的显示方式可以做差异化设置，该差异化设置包括显示位置、显示颜色等各类属性的差异化设置。
例如，设备可以在统一桌面的上半部分显示文件类型的元数据，在统一桌面的下半部分显示应用类型的元数据。
又例如，设备可以将文件类型的元数据和应用类型的元数据的亮度差异化，或者图形的形状差异化等。
此外，需要注意的是，对于应用类型的元数据，结合“关联目标设备集合中一个或多个设备的应用”这一特性，其具体显示方式可以是多种多样的，此处不做唯一限定。
例如，可以利用常规化的应用图标来展示仅关联单个设备的应用的元数据。
又例如，可以利用常规化的应用图标来展示关联全部设备的应用的元数据，且该应用图标至少包含“应用”、“APP”之类的文字信息，用于提示用户该元数据是在应用层级关联所有设备的，并且，该应用图标的图形部分可以表达出多设备集成在一起的图像，从而更加直观的提示用户该元数据的作用。
又例如，可以利用特别设计的应用图标来展示关联部分设备的应用的元数据，例如需要将目标设备集合中的同为手机设备类型的手机1和手机2关联到同一个元数据，则该元数据对应的应用图标的图形部分可以展示两个手机相加的图像，且配合“手机1+2”等类似的文字信息来直观提示用户该元数据的作用。
对于文件类型的元数据，结合“关联所述目标设备集合中至少一个设备的相同文件类型的数据”这一特性，其具体显示方式可以是多种多样的，此处不做唯一限定。
例如，可以利用常规化的应用图标来展示仅关联单个设备的相同文件类型的数据的元数据，该应用图标可以是图形加文字的形式，且退选哪个或者文字部分可以通过数字编号等方式来直观的展示与对应设备之间的关联关系，如关联手机2中的音乐类型的数据的元数据，可以在对应应用图标的图形或者文字部分增加“2”来表示手机2。
又例如，可以利用常规化的应用图标来展示关联全部设备的相同文件类型的数据的元数据，且该应用图标至少包含具体文件类型如“音乐”、“视频”、“文档”之类的文字信息，用于提示用户该元数据是在数据层级关联所有设备，并且，该应用图标的图形部分可以表达出多设备集成在一起的图像，从而更加直观的提示用户该元数据的作用。
又例如，可以利用特别设计的应用图标来展示关联部分设备的相同文件类型的数据的元数据，例如需要将目标设备集合中的同为手机设备类型的手机1和手机2关联到同一个元数据，则该元数据对应的应用图标的图形部分可以展示两个手机相加的图像，且配合“手机1+2”等类似的文字信息来直观提示用户该元数据的作用。
下面针对统一桌面需要显示的“音乐”、“视频”、“文档”、“照片”的文件类型的元数据的具体表现形式进行详细说明。
对于“音乐”元数据，可以通过与音乐符号相关的图像信息和/或文字信息来表达该元数据，图形信息例如可以是如图2a所示的音符树，每个树杈关联一个对应的设备，当然还可以其他方式，此处不做唯一限定。
此外，当前元数据所关联的设备发生变化时，图像信息也可以跟随动态变化，如手机1(非当前设备)原来为目标设备集合中的设备，且手机1包括音乐文件，当手机1不再是目标集合中的设备，则如图2b所示，“音乐”元数据的图像信息部分不再包含手机1的图像信息。
对于“视频”元数据，可以通过与视频相关的图像信息和/或文字信息来表达该元数据，图形信息+文字信息例如可以是如图2c所示的应用图标，该应用图标通过播放按钮符号表达视频功能，通过内嵌显示的文字信息表达关联多设备。
对于“文档”元数据，可以通过与文档相关的图像信息和/或文字信息来表达该元数据，图形信息+文字信息例如可以是如图2d所示的应用图标，该应用图标通过文档符号表达文档功能，通过内嵌显示的文字信息表达关联多设备。
对于“照片”元数据，可以通过与照片相关的图像信息和/或文字信息来表达该元数据，图形信息+文字信息例如可以是如图2e所示的应用图标，该应用图标通过相册符号表达文档功能，通过内嵌显示的 文字信息表达关联多设备。
如图2B所示的统一桌面的示例图，该统一桌面包括“APP”、“我的平板”、“我的电脑”、“我的手机1”、“我的手机2”等应用类型的元数据，还包括“音乐”、“视频”、“文档”等文件类型的元数据。
其中，“APP”元数据被选取后，设备可以显示目标设备集合中部分或全部设备的应用，在显示部分设备的应用时，该部分设备可以是特定类型的设备、用户指定的设备或者其他方式约束的设备，该部分或全部设备的应用的显示方式可以是多种多样的，如下拉列表、弹窗等，此处不做唯一限定。
举例来说，如图2C所示的选取“APP”元数据后的示例界面变化图，设备会在统一桌面上弹窗显示手机1和手机2的应用(图例显示为手机2的应用)，若用户选择手机2，则显示手机2的应用。
又举例来说，如图2D所示的选取“音乐”元数据后的示例界面变化图，设备会将统一桌面切换为多个设备的音乐文件的展示界面，默认优先展示音乐文件最多的设备的音乐文件(也可以是其他策略来进行需要展示其音乐文件的设备的选择，或者，也可以全部展示出来，按照不同颜色区分来自不同设备，展示方式多种多样，此处不做唯一限定)。
具体实现中，如图2E所示的多设备统一管理系统的架构示意图，统一桌面具体可以通过服务器和终端的方式实现。
其中，服务器(图示为云端系统)主要负责用户名下设备的管理以及应用和数据的管理，具体可以包括设备管理模块、应用管理模块、用户管理模块，设备管理模块、应用管理模块、用户管理模块中任意两个模块相互通信连接。
其中，终端(图示为设备A、设备B、设备C)包括统一桌面系统和/或后台支撑系统，统一桌面系统包括用户接口UI模块、重定向下沉Sink模块、应用决策模块、自有应用模块中的至少一个，后台支撑系统包括连接模块、分布式数据服务系统、分布式资源系统、重定向源头source系统中的至少一个。
所述统一桌面系统是主要的交互入口，用户通过这个系统实现对多设备的统一管理和使用。
所述连接模块用于保证多设备的连接，具体可以支持各种常规的通信技术，如移动通信技术(第五代移动通信技术5G等)、局域网短距离无线通信技术(蓝牙、超带宽UWB等)。
所述分布式数据服务系统用于实现数据在多个设备间的互通。
所述重定向source系统和分布式资源系统用于实现应用的跨端使用。
其中，不同的设备可以同时设置有统一桌面系统和后台支撑系统，或者只有其中一个部分，取决于设备的能力。
例如，手机、PC等富能力设备可以同时有统一桌面系统和后台支撑系统，客户前置设备CPE、路由器等可以只有后台支撑系统，web端可以只有统一桌面系统。
可以看出，本申请实施例中，电子设备的统一桌面作为目标设备集合中多设备的统一用户交互入口，不仅能够显示关联目标设备集合中一个或多个设备的应用的元数据，使得用户能够实现应用层级的跨端控制，还能够显示关联目标设备集合中至少一个设备的相同文件类型的数据的元数据，实现多设备的数据维度的交互机制，使得用户能够进行数据层级的跨端控制，有利于提高统一桌面信息显示的全面性和精细化程度，更好的满足的使用需求。
在一个可能的示例中，如图2F所示，所述统一桌面还包括搜索功能元数据，所述搜索功能元数据用于在目标数据集合中进行数据搜索，所述目标数据集合对应所述目标设备集合中的设备的数据。
其中，所述搜索功能元数据的显示位置和具体形态均不作唯一限定，显示位置可以是处于统一桌面的角落，或者处于统一桌面的正上方等，具体形态可以是图形和/或文字。
可见，本示例中，统一桌面不仅包括应用类型的元数据和文件类型的元数据，还包括用于在目标数据集合中进行数据搜索的搜索功能元数据，从而更加全面的拓展统一桌面信息显示的功能和应用，使得用户可以方便的利用该搜索功能元数据在多设备间进行数据查询。
在本可能的示例中，所述方法还包括：显示通过所述搜索功能元数据搜索到的来自于多个设备的多个文件组，不同文件组的文件的属性互不相同，每个文件组包括同一设备的单个或多个文件；所述属性包括以下至少一种：颜色、大小、字体、形状。
在本可能的示例中，所述搜索功能元数据支持以下搜索方式的至少一种：按照文件夹递进的方式进行搜索；按照数据类型进行搜索；以及，按照关键字进行搜索。
举例来说，如图2G所示，用户选取搜索功能元数据后，设备在统一桌面上弹窗显示搜索功能界面，该搜索功能界面可以包括文件夹搜索引导标识、数据类型搜索引导标识和关键字搜索引导标识中的至少一种。
假设搜索功能界面包括文件夹搜索引导标识、数据类型搜索引导标识和关键字搜索引导标识，且用户选择关键字搜索引导标识，设备显示如图2H所示的关键字信息例如引导弹窗，用户录入了搜索关键词“XXX”，设备获取到搜索结果，显示如图2I所示的搜索结果展示界面，该搜索结果展示界面包括关键字例如区域、搜索结果展示区域，搜索结果展示区域包括设备展示区域、数据展示区域以及文件类型展示区域，不同设备通过不同颜色进行区分，相应的，来自同一个设备的包含该关键字信息的文件均使用相同颜色进行区分。文件类型展示区域提供按照文件类型对搜索结果进行展示。
可见，本示例中，设备支持通过统一桌面的搜索功能元数据提供跨设备的数据搜索功能，有利于更加全面的拓展统一桌面信息显示的功能和应用，使得用户可以方便的利用该搜索功能元数据在多设备间进行数据查询。
在一个可能的示例中，所述方法还包括：检测到针对第一元数据的选取操作，显示所述目标设备集合中至少一个目标设备的应用；检测到针对所述至少一个目标设备的第一目标设备的第一目标应用的预选取操作，显示添加到桌面的提示信息；检测到针对所述第一目标应用的确认添加操作，在所述统一桌面显示所述第一目标设备的所述第一目标应用。
其中，第一元数据可以是应用类型的元数据，与文件类型的第二元数据相互区别。所述添加到桌面的提示信息可以是可选功能标识。
举例来说，如图2J所示，假设用户通过选取APP元数据使得设备显示手机2中的应用，手机2中的应用包括微信应用，然后用户长按该微信应用，设备弹框显示针对该微信应用的至少一个可选功能标识，至少一个可选功能标识具体包括“添加到桌面”这一功能标识，用户选取该功能标识后，如图2K所示，设备在当前的统一桌面上显示手机2的微信应用，该元数据的右下角可以增加手机2的标识信息，并基于前述的多设备管理系统建立显示的微信应用元数据与手机2上的微信应用的关联关系。
可见，本示例中，设备通过统一桌面上的元数据定位用户选取的准备添加到统一桌面的跨端设备的应用，实现灵活的跨设备的元数据的添加功能，提高使用便捷性和元数据显示的全面性。
在一个可能的示例中，所述方法还包括：检测到针对所述统一桌面上显示的所述第一目标应用的预选取操作，显示移除提示信息；检测到针对所述第一目标应用的确认移除操作，在所述统一桌面上显示除所述第一目标应用之外的初始元数据，所述初始元数据为所述统一桌面在显示移除提示信息之前所显示的元数据。
举例来说，如图2L所示，假设统一桌面上显示有关联手机2的微信应用的元数据，且用户预选取操作针对该元数据，则设备显示可选功能标识，具体包括移除、启动等，用户选择移除功能标识后，设备在所述统一桌面上不再显示关联手机2的微信应用的元数据。
可见，本示例中，统一桌面支持元数据的删除，操作方式便捷高效，提高统一桌面元数据使用的灵活性。
在一个可能的示例中，所述显示统一桌面之后，所述方法还包括：检测到针对需要跨设备运行的目标应用的应用编辑请求操作，所述跨设备运行是指在除所述目标应用在原生设备之外的目标设备上运行所述目标应用；显示控件编辑界面，所述控件编辑界面包括控件编辑区域和控件显示区域；其中，所述控件显示区域用于显示所述目标应用的第一应用界面，所述第一应用界面包括至少一个原生控件，所述原生控件是指所述目标应用在原生设备显示的所述第一应用界面上的控件；其中，所述控件编辑区域用于显示所述至少一个原生控件中被选择的原生控件所对应的适配控件，所述适配控件是指所述目标设备在运行所述目标应用时所显示的第二应用界面上的控件，所述第一应用界面与所述第二应用界面对应所述目标应用的同一个功能界面。
其中，所述需要跨设备运行的目标应用可以是当前设备运行的来自其他设备的应用，也可以是需要在其他设备上运行的来自当前设备或者另外设备(不同于其他设备)的应用，具体不做唯一下定。
举例来说，如长按微信应用弹出功能列表，选中应用跨端使用功能，弹出可跨端使用的设备列表，选中目标设备，当前设备显示如图2M所示的微信应用的跨端应用的控件编辑界面。其中，控件编辑区域具体包括工具栏、布局栏和适配空间显示栏，工具栏包括按照控件类型划分的控件筛选功能按钮，如按钮Button、文本框TextBox、图片框PictureBox、进度条ProgressBar等，布局栏包括缩放、移动、角度调整、翻转等编辑功能按钮。其中，控件显示区域支持基于位置标定等方式确定用户选择的适配控件。
可见，本示例中，设备的统一桌面为用户提供应用编辑请求操作的入口，使得用户能够针对需要在其他设备跨端运行的目标应用的功能界面进行自定义编辑，无需第三方开发适配应用的冗繁操作，提高应用跨端运行功能界面设置的便捷性和灵活性，提高多设备管理效率。
在一个可能的示例中，所述显示统一桌面之后，所述方法还包括：通过目标元数据输出目标应用运行状态下所关联的目标内容。
具体实现中，输出目标内容的具体表现形式可以是通过显示屏展示影像信息(图像、视频等)，通过扬声器播放对应的音频信息，通过振动传感器等输出对应的振动信息。
其中，所述目标应用可以是任意类型的应用，如游戏应用、音乐应用、办公应用、聊天应用等，此处不做唯一限定。
其中，所述目标应用运行状态下所关联的目标内容包括但不限于：音频、视频、图片、文字、动作信息(例如：振动信息)等。
所述目标元数据与所述目标应用的关系具体可以包括以下几种情况：
第一种，目标应用为当前设备的本地应用，目标元数据关联当前设备的应用(例如当前设备安装的游戏应用、微信等)，
若目标元数据关联当前设备的多个应用，则用户选取该目标元数据后，设备显示多个应用，其中包括该目标应用，用户进一步选择启动该目标应用，从而该目标应用被启动并运行，并显示其对应的功能界面。例如，目标元数据关联本地安装的微信应用和游戏应用，则点击该目标元数据后，弹窗显示该微信应用元数据和游戏应用元数据，用户选择微信应用元数据，当前设备检测到该微信应用元数据的类型为应用类型的，且关联的是当前设备的微信应用，则当前设备运行微信应用并显示其功能界面。
若目标元数据关联当前设备的单个应用，则用户选择启动该目标元数据后，当前设备会运行该单个应用并显示其功能界面。例如，目标元数据直接关联本地安装的微信应用，用户点击该目标元数据后，当前设备运行微信应用并显示其功能界面。
第二种，目标应用为当前设备的本地应用，目标元数据关联目标设备集合中的多个设备的相同文件类型的数据(例如：设备1和设备2的文档、视频、图片、音乐等数据，可以使用本地兼容应用读取这些数据)，用户选取统一桌面上显示的该目标元数据，设备启动该目标应用。
例如，针对“文档”元数据，用户选取该元数据后，当前设备弹窗显示该“文档”元数据所关联的多个设备的文档，用户进一步选取目标文档，当前设备调用本地兼容应用例如WPS来读取目标文档并展示该目标文档。
第三种，目标应用为远端设备的应用，目标元数据为关联该远端设备的所有应用的元数据，用户选取统一桌面上显示的该目标元数据，设备进一步显示远端设备安装的多个应用的元数据，其中包括该目标应用的元数据，用户进一步选取该目标应用的元数据，从而该目标应用在远端设备上被启动并运行，同时当前设备能够跨端显示该目标应用的功能界面，还可以播放音频内容、输出动作信息等，以及支持实现反向控制等，且当前设备上显示的功能界面可以与远端设备的大小保持一致，也可以是经过自适应或者预定义的，总之，对于用户来说，能够在当前设备上实现跨端使用远端设备的目标应用。
例如，在PC上跨端使用手机上的美团应用进行外卖下单、支付等，即PC的显示屏上可以在展示统一桌面的同时，展示与手机运行美团应用的应用界面一致的用户界面。
其中，当前设备所显示的目标应用的功能界面，可以是当前设备基于分布式应用机制、本地兼容应 用机制、重定向应用机制中的任意一种所实现的，此处不做唯一限定，该三种实现机制的具体原理性说明在后续实施例中会进行详细介绍。
在一个可能的示例中，所述显示统一桌面之后，所述方法还包括：检测到针对所述统一桌面的目标元数据的选取操作；根据所述目标元数据的类型执行所述目标元数据的使用流程。
在本可能的示例中，如图2N所示，所述根据所述目标元数据的类型执行所述目标元数据的使用流程，包括：
步骤11，判断所述目标元数据的类型；
步骤12，若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
其中，文件类型的元数据所关联的数据是一些通用数据，如视频、文档、图片等，在各个系统上都有专业的软件来查看、编辑这些内容。为了更好的跨端体验，设备将会用本地的应用程序来运行这些数据，这样能保证画面和操作的延迟性，因为都是在本地运行的，因此这两方面的体验较好。所述本地应用流程包括以下步骤：
确定所述目标元数据所关联的数据的子类型；
判断当前设备是否存在适配所述子类型的数据的目标应用；
若存在适配所述子类型的数据的目标应用，则判断所述目标应用是否支持读取超文本传输协议Http流；
若支持读取Http流，则打开适配的目标应用并读取http流；
若不支持读取Http流，则接收根据所述目标元数据的路径确定的所述目标元数据的关联的目标数据，打开所述目标应用并读取所述目标数据；
若不存在适配所述子类型的数据的目标应用，则执行所述重定向的流程。
步骤13，若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
步骤14，根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
步骤15，若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
其中，所述分布式应用流程包括以下步骤：建立当前设备和提供所述目标元数据的服务的目标设备的连接；与服务器交互，验证当前设备使用所述目标设备上的服务是否通过了用户的授权；若已通过用户的授权，则当前设备上的分布式应用通过约定好的方式调用所述目标设备上的服务。
具体实现中，针对分布式应用的多设备协同管理系统架构，包括服务器和终端，服务器包括用户管理模块、服务管理模块和设备管理模块，终端包括协同模块、服务管理模块和连接模块。其中，设备管理模块负责用户名下多个设备的管理。服务管理模块负责将应用服务化，并提供远程调用的能力。协同模块交互负责为用户提供交互方式来达到多设备协同的目标。分布式应用会将应用信息、源设备信息、目标设备信息传递给分布式应用框架进行处理，让其运行到目标设备上去。下面进行详细说明。
设备的管理：设备通过用户账户来管理，将设备登记到云端的用户管理模块中。设备上会有连接模块，会定时和云端的设备管理模块上报自己的信息，来保证设备的可访问性。应用可以通过设备管理模块查询用户名下所有的设备。
服务的管理：服务管理模块能统一管理终端上的显示服务、声音服务、输入服务、定位服务、支付服务以及其他服务。
主要流程如下：
步骤151.开发者通过一定的接口规范开发服务
开发的服务需要符合一定的开发规范，这样才能被设备上的服务管理模块识别。
特别的，会在设备上开发一些默认的服务，这些服务是设备上能力的暴露，如显示、声音、输入、 定位等，这些能力和现有设备操作系统暴露的能力是一致的，很多应用都会用到，只是以规定好的服务的形式再做一次暴露。使得远程设备也能访问到这些能力。
步骤152.注册服务
设备上的服务管理模块在启动的时候会扫描一遍设备上按照给定方式开发的服务，并把这些服务记录下来；同时，在服务安装和卸载的时候，设备上的服务管理模块能够监听到这些变化，并把这些内容记录下来。
在设备上的服务管理模块中的内容发生变化时，将向云端的服务管理模块同步这些变化。
云端的服务管理模块会按照设备、应用、服务的粒度存储这些服务。
步骤153.服务的查询
用户登录后，可以向云端的服务管理模块请求服务的查询，主要有以下查询方式：
A.通过查找服务列表，获取当前用户名下设备上所有的服务
B.通过服务查找所有设备
查找所有的拥有这个服务的设备，如显示服务、定位服务，有些服务可能只在一些设备上有，可以筛选出一部分的设备。
C.通过设备查找服务
可以查找选择的设备上所有的服务
步骤154.服务的使用
通过步骤153，我们找到了需要使用的服务，以及提供此服务的设备。
通过以下步骤来使用服务。
步骤1541，通过连接模块以及云端设备管理模块，建立起当前设备和提供服务设备的连接。
步骤1542，查找云端服务治理模块，验证当前设备使提供服务设备上的服务是否通过了用户的授权。
如果已经授权了则跳过步骤1544；如果没有进行步骤1543。
步骤1543，在远端设备上弹出提示，需要提供服务的设备授权给此次服务调用。
如果提供服务的设备进行了授权，则进行下一步；如果没有则整个服务调用终止，并在当前设备弹出提示，访问远端服务失败。
步骤1544，提供服务的设备上的服务网关模块检测当前服务是否可用，如果当前服务没有启动，则启动当前服务；如启动失败则终止调用，并在当前设备弹出提示，访问远端服务失败。
步骤1545，当前设备上的应用通过约定好的方式调用提供服务的设备上的服务。
在这里提供服务的设备可以是远端设备也可以是当前设备。如果是当前设备则省略步骤1541。
多设备协同交互：如果开发者使用了厂家提供的接口规范开发应用，那么在应用中就可以选择不同的设备的能力；现实中，第三方应用一般不会采用特殊的方式来处理。因此可以采用系统调用的方式来达到我们的目标。在用户在设备上打开一个应用时，不管这个应用是否有使用规定的接口规范开发，通过一些特殊的交互就能唤起相关的协同流程。
例如，在手机上，我们通过长按当前运行的应用的屏幕唤起协同流程，系统会弹出一个悬浮窗，这个悬浮窗的内容和用户、应用、和当前设备相关。
一些典型的应用场景如下面图2O至图2T所示。
如图2O所示的一种分布式应用的示例，手机上运行的游戏应用支持使用其他设备的显示，具体是在游戏界面显示分布式应用功能选择弹窗，并提供分离图像，可选的支持分离图像的设备包括图示的客厅电视、我的平板等，可选的分布式应用功能还包括分离声音、分离操控等。
如图2P所示的另一种分布式应用的示例，手机上运行的游戏应用支持使用其他设备的显示，具体是在游戏界面显示分布式应用功能选择弹窗，并提供分离声音，可选的支持分离声音的设备包括图示的蓝牙耳机、蓝牙音箱、客厅电视等，可选的分布式应用功能还包括分离图像、分离操控等。
如图2Q所示的另一种分布式应用的示例，手机上运行的游戏应用支持使用其他设备的显示，具体是在游戏界面显示分布式应用功能选择弹窗，并提供分离操控，可选的支持分离操控的设备包括游戏手 柄、鼠标键盘、手机触控等，可选的分布式应用功能还包括分离图像、分离声音等。
如图2R所示，用户将手机端的游戏应用分离显示和分离操控后，可以通过鼠标和键盘进行操控，通过PC进行游戏界面的显示。
如图2S、图2T所示的另一种分布式应用的示例，PC上运行的游戏应用支持使用其他设备分离图像、分离声音和分离操控，从而有效提高游戏体验。
由于服务可以分为设备上的服务，以及应用自身的业务服务，因此当用户在不同的应用上拉起系统弹窗时，服务列表将随着用户名下的设备列表以及不同的应用的变化而变化。只有有效的服务才会显示在弹窗上。
多设备协作的流程如下：
a.通过当前设备上的交互方式，协同模块将拉起系统弹窗。
b.协同模块访问云端的服务治理模块获取服务列表。
服务列表代表了可以协同的能力列表，将展现为系统可以协同的能力。
c.根据选中的能力，协同模块访问云端的服务治理模块来获得设备列表。
当用户选择一种能力的时候，会弹出设备列表，这里的设备列表显示的不是设备名称，而是会和具体的能力相结合作为表述，如鼠标键盘，手机触控等。
d.当用户选择能力和能力下具体的设备的时候，触发服务使用流程。
步骤16，若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
步骤17，若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环境中的所述本地兼容应用。
其中，所述本地兼容应用是指与所述应用相关信息所指示的应用类型一致的应用，如音频播放类应用，若所述应用相关信息指示远端设备的音频播放应用1，而当前设备本地存在与该音频播放应用1类型相同的音频播放应用2，则可以启动音频播放应用2，进一步地，可以通过前述的多设备管理系统将音频播放应用1中的音频数据流转到当前设备，并通过音频播放应用2实现播放。
具体实现中，本地兼容应用运行在当前设备，从远端设备上读取数据，或把远端设备上的数据临时同步到本地。
以PC和手机为例，PC和手机是用户最常用的两个设备，但是PC原则上是不能直接运行安卓Android应用(如游戏)的，通过把android运行环境嵌入PC的统一桌面应用，作为自有应用模块的一部分，这样手机上的应用就能在PC上运行起来。
步骤18，若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
其中，重定向的流程不需要第三方开发商的支持，程序运行在远端设备上，把程序的画面、声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的指定程序中。相比于Miracast等方案的点对点模式，本申请实施例所提供的应用重定向方案能够支持一对多、多对一、多对多等模式。此种方式目标设备虽然没有安装应用，但却可以实现该应用运行在源设备上情况下的相应功能，并支持反向控制交互。
具体实现中，所述重定向的流程如图2U所示，具体包括以下步骤：
步骤21，根据当前设备的类型和元数据所关联的应用或数据所在设备的类型判断是否为大屏到小屏的内容流转。若是则执行步骤22，若否则执行步骤25。
步骤22，与服务器交互，查看所述目标元数据是否有基于控件绑定重定向的应用定义。
如果有，则执行基于控件绑定重定向的流程；
如果没有则执行步骤23。
其中，所述基于控件绑定重定向具体通过组装原有应用中的一些部件来获得新的应用形态，将使用 系统应用捕获当前应用的内容信息，并在远端重新绘制。使用应用编辑器开发应用定义保存到云端或本地，设备上的重定向系统使用预制的应用定义来完成应用的跨端使用。应用编辑器会运行在开发设备上如PC，也可能会在参与跨端的设备上。
设备通过应用运行状态下功能界面的重新定义，源设备上的应用从源设备完成到目标设备的转换。
如图2V所示的重新定义流转应用的功能界面的示意图，手机端QQ音乐应用的原始应用界面上包括多个原生控件，通过前述控件编辑界面的实现机制能够定义手表上跨设备显示的QQ音乐的适配功能界面中的适配控件，其中，外部背景部分可以预设或者自定义，被选中编辑的适配控件具体是播放控件、上一首控件、下一首控件、名称控件。，此种方式流转目标设备上的功能界面所显示的控件与原始应用界面的控件可以不同，支持改变颜色布局形状等属性。
假设设备B为用户当前设备，设备A为远端设备。用户想在设备B上使用设备A上的应用。当用户开始跨端使用应用时，主要包括两个流程：
1.画面显示流程，设备A上应用显示在设备B上
2.反向控制流程，通过设备B操作设备A上的应用
其中，画面显示流程具体包括如下步骤(1.)到(7.)。
步骤(1.)设备A的信息采集模块获取该应用的针对设备A的应用定义，如果本地不存在，到云端获取，然后保存到本地缓存。
步骤(2.)在应用定义中，会有多个控件信息，针对每个控件，会带有指向运行的应用中的原生控件的唯一路径，信息采集模块通过这个唯一路径获取这个原生控件，并得到原生控件的信息，如文本，图片地址，进度值等。
步骤(3.)重复步骤2，获得应用定义中所有的控件信息，把所有信息进行相应整合传给设备A的发送模块。
整合的信息包含了如下内容：
应用ID，控件总量,以及多个控件信息，每个控件信息如下：
控件ID，控件值
步骤(4.)设备A的发送模块对信息进行压缩发送给设备B的接收模块。
区别于画面的压缩，由于信息是文本数据，采用如下压缩的算法；
采用关键帧和差值帧的做法，关键帧带有全部的数据，差值帧带有和关键帧做好差值的数据。
步骤(5.)设备B的接收模块通过历史数据和接收到的最新的数据还原出运行的应用的信息，发送给设备B的应用显示模块
通过关键帧和差值帧还原出当前帧的数据
步骤(6.)设备B的应用显示模块通过云端获取当前应用的应用定义，保存到本地。
步骤(7.)应用显示模块解析应用定义，绘制显示内容。
应用定义中有绑定控件的信息，以及外部资源的信息，同时我们也获得了远端设备上原生应用的整合信息，通过这些信息我们重新绘制应用。
步骤A.遍历应用定义中的绑定控件。
步骤B.根据每个控件的定义信息，生成适配控件。
通过适配控件的类型，生词适配控件。
通过控件的属性信息，设置适配控件的基础属性，如字体、颜色等。
通过控件的属性绑定信息，以及远端设备上此控件的值，确定在适配控件上的值如假设此控件上的属性绑定信息为复制值策略，则把远端设备上的原生控件的值，直接赋予当前适配控件。
通过控件的事件绑定信息，为控件注入相关事件。
步骤C.遍历外部资源控件，为每个外部资源控件设置指定的值，如给定的图片等。
步骤D.根据应用定义中的控件顺序，一次绘制出所有的两种控件，完成显示。
具体实现中，由于设备B的应用是重新绘制的应用，因此有两种方式来做反向控制，一种是适配控 件的反馈，另外一种是通过位置关系来做映射。前一种是更加简单的方式，但是有可能处于安全机制的原因不允许操作；后一种会更加复杂，但是更加通用，理论上来说不会被限制。主要流程包括如下步骤。
步骤(1.)设备B的应用显示模块采集本地设备的输入操作。
步骤(2.)在设备B的输入处理模块中，获取应用定义，产生适配事件，并生成相关的操作信息。
由于设备B上的应用是一种应用形态而不是图片形态，因此当用户操作当前应用时，能产生相应的控件事件，如点击事件。从云端获取该应用的针对设备B的应用定义，并缓存在设备B本地。
设备B根据应用定义，结合产生事件的适配控件，生成带控件ID的信息，如(控件ID，点击)。
步骤(3.)设备B的发送模块压缩操作信息传输到设备A上的接收模块。
步骤(4.)设备A上的接收模块接收到压缩过的操作信息后还原出原始操作信息，传输到设备A的输入注入模块。
步骤(5.)设备A的输入注入模块接收到操作信息后，注入到运行的应用中。
这里的输入注入模块将根据设备和应用的情况，选择两种方式注入输入，具体步骤如下：
步骤A.设备A的输入注入模块获取应用定义，并缓存到本地。
步骤B.从应用定义中提取每个控件的输入绑定情况。
步骤C.针对一次输入操作，找到产生输入的控件，查找输入绑定信息，查看这个事件是否已经注册。
步骤D.如果已经注册，则将此输入操作按照注册信息转化为原生控件的事件，注入到应用中。
如将适配控件的点击事件转为为原生控件的点击事件。
步骤E.如果没有注册，则从应用定义中查找控件的位置信息。将输入操作转化为针对这个位置信息的操作，注入到应用中。
如将适配控件的点击事件转为在运行应用的画面中控件所在位置的一次点击操作。
步骤(6.)设备A运行的应用响应操作，对原生控件的值进行更新，这些变动走画面显示流程更新到设备B上。
步骤23，与所述服务器交互，查看所述目标元数据是否有基于部分投屏重定向的应用定义。
如果有，则执行基于部分投屏重定向的流程；如果没有则执行步骤24。
其中，所述基于部分投屏重定将改变应用原有的显示以及布局以适应不同的屏幕尺寸。
本申请实施例通过截取原有应用中的多个部分画面，并在远端重新布局以适应不用的屏幕尺寸。使用应用编辑器开发应用布局保存到云端或本地，设备上的重定向系统使用预制的应用布局来完成应用的跨端使用。应用编辑器会运行在开发设备上如PC，也可能会在参与跨端的设备上。
假设设备B为用户当前设备，设备A为远端设备。用户想在设备B上使用设备A上的应用。当用户开始跨端使用应用时，主要包括两个流程。
1.画面显示流程，设备A上应用显示在设备B上。
2.反向控制流程，通过设备B操作设备A上的应用。
其中，画面显示流程具体包括如下步骤(1.)到(7.)。
步骤(1.)采集设备A运行App的画面，称作画面A，这个画面和App显示的一致。
步骤(2.)设备A获取该应用的针对设备B的布局，如果本地不存在，到云端获取。
步骤(3.)如果云端也不存在，则设备A直接把第一步的内容传输到编码模块，流程结束。
步骤(4.)如果存在布局，则设备A第一次下载布局，并把布局中的外部数据同时下载下来。
步骤(5.)设备A按照布局的定义，把画面A中的选中的内容，依据定义中所描绘的位置、大小、旋转等信息生成图片信息，加入列表。
步骤(6.)设备A重复步骤5，直到布局中的选中部分的内容都加入到图片列表中。
步骤(7.)设备A把布局中外部的资源所定义的图片都加入到图片列表中。
步骤(8.)设备A按照布局所定义的层次关系，依次把这些图片信息都写入到同一个图片中。
步骤(9.)设备A如果选中的内容有在新的布局中相近的部分，对画面进行插值处理，来平滑不同 画面的过渡。
步骤(10.)设备A把生成的图片传输给编码模块。
步骤(11.)设备A编码模块把视频编辑为H264或者H265的格式，输送到设备A的发送模块。
步骤(12.)设备A的发送模块把编好码的数据流发送到设备B上，使用私有的流媒体协议。
步骤(13.)设备B上的接收模块接收到数据后，转发到解码模块。
步骤(14.)设备B的显示模块把解码模块解出的视频流展示出来。
反向控制流程中，由于画面和App的画面不一致，只是其中的一些部分组合而成，位置和原始的位置对比已经发生了很大的变化，对方向控制也将进行特殊的处理。主要流程如下步骤。
步骤(1.)设备B采集本地的输入操作。
步骤(2.)设备B通过应用的布局信息计算当前触发的被选中部分。
由于点击的是图片，因此不能产生类似控件事件的行为。需要通过位置计算来得到被点击的部分。
布局信息在本地有缓存，如果没有则到云端应用管理模块中获取。以点击为例，在当前应用点击时，会得到一个点的位置，通过布局信息，我们能计算出当前点属于原始应用中哪个被选中的部分，以及点击的位置针对在这个被选中的部分中的位置偏移。
步骤(3.)设备B通过被选中的部分以及点击的相对偏移位置计算这个点在原始应用中的位置。
由于被选中的部分在制作成新的应用时可能有缩放和旋转的行为，这些信息记录在应用布局中，通过这些信息以及上一步骤得到的针对这个被选中部分的相对位置偏移，再通过被选中的部分在原始画面中的偏移量，能够计算出点击的位置在原始应用的位置。
这里的点击可以扩展为其他的操作行为。
步骤(4.)设备B使用设备级的操作适配把不同的输入设备的操作进行适配。
如把触摸屏的操作转换成鼠键操作。
步骤(5.)设备B将转换好的输入操作发送给设备A。
设备A接收输入操作后将会作出相应的响应，如点击播放按钮开始播放音乐，同时按钮变成了暂停。
设备通过应用运行状态下功能界面的重新绘制，源设备上的应用从源设备完成到目标设备的转换。
如图2W所示的重新绘制(即采用基于部分投屏的重定向机制)流转应用的功能界面的示意图，手机端QQ音乐应用的原始应用界面上包括多个原生控件，通过基于部分投屏的重定向机制将手机端QQ音乐应用的原始应用界面上包含主要控件的核心区域截图并重新绘制到手表的界面上，从而实现部分投屏，此种方式流转目标设备上的功能界面所显示的控件与原始应用界面的控件保持一致，无法改变颜色布局形状等属性。
步骤24，当前设备检测是否支持大屏到小屏重定向。
若是，则执行大屏到小屏重定向的流程；
如果没有则执行步骤25。
其中，假设设备B为用户当前设备，设备A为远端设备。用户想在设备B上使用设备A上的应用。所述大屏到小屏重定向主要流程包括如下步骤。
步骤(1.)设备A的采集模块采集运行的应用的画面，传输到编码模块。
步骤(2.)设备A的编码模块把画面进行编码压缩，传输到设备B的解码模块。
步骤(3.)设备B的解码模块把解码好的画面传输给显示模块。
步骤(4.)设备B的显示模块显示画面，这个过程中，我们对投屏的画面进行了特殊处理。
在设备B的显示模块中，允许显示内容在一定的交互方式下进行放大、缩小、移动行为。
步骤(5.)输入设备的输入操作传输到设备B的输入处理模块，进行输入处理。
设备B的输入处理模块将会决定输入的操作将会产生什么样的响应。主要是区分两种操作，第一种是缩放、移动投屏画面；第二种是将操作传输到远端，触发远端应用的操作响应。
为了区别放大移动画面的操作和应用程序本身的操作响应，使用以下流程来做区分。主要流程包括如下步骤A至步骤F。
步骤A.用户通过输入设备在设备B上触发移动操作。
步骤B.设备B判断画面是否是原始画面还是放大过的画面。
步骤C.如果是原始画面，则设备B直接把适配好的操作传输到设备A上。
步骤D.如果是放大过的画面，则设备B再一次判断当前画面是否到达边缘。
画面到达边缘指较小的屏幕显示的画面的是放大后的画面的靠近边缘的一部分。小屏幕已经达到了画面的上部边缘，如果此时移动方向是向下的，则判断成立。如果移动方向不为下，此时判断也不成立。
步骤E.如果还没有到达边缘，则设备B按照输入的移动方向移动画面。
此时，画面将按照输入所形成的移动方向移动。
步骤F.如果到达了边缘，则设备B将此次移动输入适配好后传输到设备A上。
到达边缘后的操作转交给设备A处理，在这些操作下，一些情况下设备A上的画面将发生变化，如列表滚动，这些变化将同步到设备B的小屏上。
输入适配：
除了画面的缩放和移动，有些操作代表了对应用自身的操作，需要做适配后传输到远端设备上。
由于画面经过了放大和缩小，因此交互的位置会做相应的缩放以后，再传输到远端设备上。
不同的设备可能系统不一致，将会把不同的输入设备的信息做转换，如鼠键操作转换为触摸屏操作等。
步骤(6.)设备B的输入处理模块处理完操作后，一部分操作变成了设备A的操作输入，传输给设备A的输入注入模块。
步骤(7.)设备A的输入注入模块把输入注入给原始应用，得到响应，刷新应用画面。
设备通过不同设备上采用不同的交互形式，放大缩小投屏显示端的画面，并区分出对画面的缩放、移动操作和对远端应用本身的操作的差别，使得小屏设备上既能以更大的画面来展示、操作远端设备的应用，又不影响对应用本身的操作功能，从而提升大屏到小屏投屏的使用体验。
步骤25，使用通用的投屏方式来做重定向。
其中，普通应用可以通过投屏服务，将本机运行的应用的多媒体信息通过投屏的方式流转到目标设备。此时流转策略模块，需要将应用信息、源设备信息、目标设备信息推送给投屏服务，由投屏服务完成流转。
以上流程为统一桌面默认流程，这些流程并不是完全固定的，允许用户可以有喜好的重新设定这些方式的优先级。
此外，当统一桌面上的元数据被点击后，将触发应用使用流程，在当前设备上保留了元数据所关联的应用所属的原始设备的操作习惯，如在PC上是鼠标双击图标，在手机上是点击图标。为了不同设备之间，不同的系统之间能互相使用对方的应用，本方案包含了多种跨设备应用使用方案。主要分为三大类别：
与上述元数据的使用机制不同的，多设备之间的应用的流转还可以基于分布式应用框架实现，具体实现中，所述分布式应用框架包括源、目的判定模块和流转策略模块。
第一步，根据用户输入的行为、或者输入的语音等进入到各类的识别模块，识别模块负责识别出需要流转的源、目的设备，因为存在明确的源、目的设备(例如手势识别，可以有明确的源、目的设备ID信息)，也可能存在模糊的源目的信息(例如语音“从本机到电脑”，可能同一账号下有多台设备，因此识别出来的为模糊的源、目的)，识别模块将识别出来的源、目的输送到源、目的判定模块进行最终判定。
第二步，如果是不明确的源、目的ID信息，例如目的为电脑，则需要和连接管理模块进行交互，获取当前同一账号下所连接的所有设备信息，并获取所有电脑设备，如果只有一台电脑设备，则选定此设备为目的。如果有多台，则弹出电脑列表供用户进行选择后获取最终的源、目的ID，再进行下一步。
第三步，源、目的模块将最终的源、目的ID推送给流转策略模块进行应用如何流转的策略判定。
第四步，流转策略模块访问应用信息模块，获取当前在前台运行的应用名称、应用类型，设备可以根据应用类型采用不同的应用流转策略来实现应用的跨端运行，应用类型可以分为四种，每种有不同的 方式处理，下面进行详细描述。
(1)分布式应用，分布式应用为前后端分离的应用，在不同类型的设备上可以运行不同的UI系统，而后端运算可以在与UI不同的设备上运行。分布式应用会将应用信息、源目的信息传递给分布式应用框架进行处理，让其运行到目标设备上去。
举例来说，两台设备之间的分布式应用流转，初始状态中设备A运行了一个分布式应用X，其分为两部分应用X的UI和应用X服务端均在运行，当设备A通过UWB指向设备B时，经过应用流转框架后设备A的应用X的UI暂停运行，设备B的应用X的UI开始运行，并使用了设备A的应用X服务端。
UWB识别后，UWB识别模块向源、目的判定模块发送了明确的源、目的设备ID，因此源、目的模块将源设备A、目的设备B的ID信息传递给了流转策略模块，流转策略模块查询了当前在前台为分布式应用服务，因此向分布式应用框架发起了一次应用流转请求，分布式框架完成了整个分布式应用流转的过程，负责暂停设备A上应用X的UI，并拉起设备B的应用X的UI并通知UI运行时使用设备A上应用X的服务端。
(2)投屏应用，当前设备前台运行的为投屏服务，展示给用户的是其他设备投递到本机的一个投屏。流转策略模块会将源设备信息、目标设备信息推送给投屏服务，投屏服务将完成不同设备间的流转。
举例来说，三台设备下的投屏流转，在设备B向设备A投屏时，设备A通过NFC碰触了设备C，经过应用流转框架后改为设备B向设备C进行投屏。
NFC触碰后，NFC识别模块向源、目的判定模块发送了明确的源、目的设备ID，因此源、目的模块将源设备A、目的设备B的ID信息传递给了流转策略模块，流转策略模块查询了当前在前台为投屏服务，因此向投屏服务发起了投屏流转请求，投屏流转服务完成设备B到设备A投屏的结束，并开启了设备B到设备C的投屏，完成了整个投屏流转的过程。
(3)可接力应用，可接力应用与分布式应用不同，其通过本地的文件服务将数据流推送给目标设备，让目标设备运行时使用的是本机的适配应用，例如视频播放器，我们在流转时候将文件转换为超文本传输协议http的数据流在目标设备上进行播放。
举例来说，初始情况为手机中运行播放器播放本地视频，当用户在手机中输入语音为“在电脑上进行播放”，识别出目标为电脑，经过应用流转框架后在电脑中进行播放相同视频，而手机端暂停播放。
语音识别系统识别出了模糊的目的类型为电脑的设备，将此信息发送给源、目的判定模块，源、目的判定模块访问了连接服务，查询当前连接的所有电脑设备，发现只有一台电脑类型设备则确认了目的ID，并将此信息推送推了流转策略模块，流转策略模块发现此时前台应用为可接力应用，将此信息推送给了接力框架，由接力框架完成了应用的流转，最终在电脑的播放器使用了手机上的文件服务完成数据流传输、并在电脑上运行播放器应用进行播放。
(4)普通应用，普通应用可以通过投屏服务，将本机运行的软件通过投屏的方式流转到目标设备。此时流转策略模块，需要将应用信息、源设备信息、目标设备信息推送给投屏服务，由投屏服务完成流转。
举例来说，初始状态为设备A中运行了普通应用X，用户使用NFC将设备A碰触了设备B，经过应用流转框架的处理后，设备B上开启了投屏服务并显示设备A中应用X的画面。
NFC触碰后，NFC识别模块向源、目的判定模块发送了明确的源、目的设备ID，因此源、目的模块将源设备A、目的设备B的ID信息传递给了流转策略模块，流转策略模块查询了应用信息，获取当前在前台的应用为普通应用，因此向投屏服务发起投屏请求，设备A的投屏服务向设备B中发起投屏并将设备A的应用X的画面显示到设备B上。
可见，本示例中，统一桌面支持通过元数据来播放目标应用在运行状态下的目标内容，从而实现多设备交互场景下的应用使用，提高多设备交互的实用性和灵活性。
在一个可能的示例中，所述通过目标元数据输出目标应用运行状态下所关联的目标内容之后，所述方法还包括：当前设备作为源设备向目标设备发送所述目标内容以实现所述目标内容跨设备流转；
其中，所述源设备为识别设备根据第一数量个设备所生成的第一手势检测关联数据而确定的内容发送设备，所述第一数量个设备包括所述识别设备、所述源设备和所述目标设备，所述识别设备为所述第一数量个设备中的任意一个；所述目标设备为所述识别设备根据第二数量个设备所生成的第二手势检测关联数据而确定的内容接收设备，所述第二数量个设备包括所述识别设备和所述目标设备。
其中，手势检测关联数据可以是设备通过摄像头采集到的用户的手势图像，也可以是设备根据采集到的手势图像计算出的手势的置信度，识别设备能够根据多个设备的手势图像或者手势的置信度确定出需要进行内容跨设备流转的源设备。
其中，所述第一手势检测关联数据对应的手势可以是人手的抓取手势动作(例如：如图2X所示的示例图，由手掌张开到握拳的变化状态，或者，前述变化状态过程中的特定状态)，所述第二手势检测关联数据对应的手势可以是人手的释放手势动作(例如：如图2Y所示的手势示例图，由握拳到张开手掌的手势变化状态)。
其中，所述第一数量可以是2个、3个、4个等，第二数量比第一数量少一个。
可见，本示例中，设备通过对用户手势识别来发起流媒体信息的流转，通过识别用户的手部抓取和释放动作，向用户的设备发送流转的指令，完成应用的流媒体信息在多个设备间的流转。
在本可能的示例中，所述方法还包括：输出用于提示内容流转的信息。
可见，本示例中，通过主动显示提示信息来增强交互连贯性和流畅度，提高用户体验。
在一个可能的示例中，所述显示统一桌面之前，所述方法还包括：通过当前设备的统一桌面账号登陆入口录入目标用户账号的登陆信息，并向服务器发送所述目标用户账号的登陆信息，所述目标用户账号的登陆信息用于所述服务器执行以下操作：校验所述登陆信息合法后，获取所述目标用户账号的统一桌面的元数据集合，并向请求登陆的对端设备发送所述元数据集合；接收来自所述服务器的所述目标用户账号的所述统一桌面的元数据集合，所述元数据集合用于当前设备显示所述统一桌面。
可见，本示例中，当前设备在多设备管理系统架构下，能够基于账号登陆机制从服务器端获取元数据列表，从而降低本端设备的数据处理压力，降低时延，提高用户终端侧的使用体验。
与上述实施例一致的，如图3所示，本申请实施例还提供一种元数据的访问方法，包括以下步骤：
步骤301，第一电子设备显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述第一电子设备和与所述第一电子设备通信连接的第二电子设备上的数据；
步骤302，响应于对所述统一桌面上目标元数据的选择操作，根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据。
可以看出，本示例中，通过统一桌面的元数据能够灵活调用不同设备的应用或数据进行使用，提高多设备管理场景中数据使用的灵活性和便捷性，提高用户体验。
在一个可能的示例中，所述元数据包括应用类型的元数据和文件类型的元数据；所述应用类型的元数据关联所述第一电子设备和所述第二电子设备的应用，所述文件类型的元数据关联所述第一电子设备和所述第二电子设备中相同文件类型的数据。
在一个可能的示例中，所述根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据，包括：
判断所述目标元数据的类型；
若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环境中的所述本地兼容应用；
若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
在一个可能的示例中，所述本地应用流程包括以下步骤：
确定所述目标元数据所关联的数据的子类型；
判断当前设备是否存在适配所述子类型的数据的目标应用；
若存在适配所述子类型的数据的目标应用，则判断所述目标应用是否支持读取超文本传输协议Http流；
若支持读取Http流，则打开适配的目标应用并读取http流；
若不支持读取Http流，则接收根据所述目标元数据的路径确定的所述目标元数据的关联的目标数据，打开所述目标应用并读取所述目标数据；
若不存在适配所述子类型的数据的目标应用，则执行所述重定向的流程。
在一个可能的示例中，所述分布式应用包括以下步骤：
建立当前设备和提供所述目标元数据的服务的目标设备的连接；
与服务器交互，验证当前设备使用所述目标设备上的服务是否通过了用户的授权；
若已通过用户的授权，则当前设备上的分布式应用通过约定好的方式调用所述目标设备上的服务。
在一个可能的示例中，所述重定向的流程包括以下步骤：
根据当前设备的类型和所述目标元数据关联的应用或数据所在设备的类型，判断是否为大屏到小屏的内容流转；
若是大屏到小屏的内容流转，则与服务器交互，查看所述目标元数据是否有基于控件绑定重定向的应用定义；
如果有，则执行基于控件绑定重定向的流程；
如果没有，则与所述服务器交互，查看所述目标元数据是否有基于部分投屏重定向的应用定义；
若有，则执行基于部分投屏重定向的流程；
若没有，则当前设备检测是否支持大屏到小屏重定向。
若支持，则开始大屏到小屏重定向的流程；
若不支持，则使用通用的投屏方式来实现重定向。
本申请实施例提供一种桌面元数据的显示装置，该桌面元数据的显示装置可以为终端。具体的，桌面元数据的显示装置用于执行以上桌面元数据的显示方法中终端所执行的步骤。本申请实施例提供的桌面元数据的显示装置可以包括相应步骤所对应的模块。
本申请实施例可以根据上述方法示例对桌面元数据的显示装置进行功能模块的划分，例如，可以对应各个功能划分各个功能模块，也可以将两个或两个以上的功能集成在一个处理模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式。
在采用对应各个功能划分各个功能模块的情况下，图4示出上述实施例中所涉及的桌面元数据的显示装置的一种可能的结构示意图。如图4所示，桌面元数据的显示装置4包括：
显示单元40，用于显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
在本可能的示例中，所述统一桌面还包括搜索功能元数据，所述搜索功能元数据用于在目标数据集 合中进行数据搜索，所述目标数据集合对应所述目标设备集合中的设备的数据。
在本可能的示例中，所述显示单元40还用于：显示通过所述搜索功能元数据搜索到的来自于多个设备的多个文件组，不同文件组的文件的属性互不相同，每个文件组包括同一设备的单个或多个文件；所述属性包括以下至少一种：颜色、大小、字体、形状。
在本可能的示例中，所述搜索功能元数据支持以下搜索方式的至少一种：按照文件夹递进的方式进行搜索；按照数据类型进行搜索；按照关键字进行搜索。
在本可能的示例中，所述显示单元40还用于：检测到针对第一元数据的选取操作，显示所述目标设备集合中至少一个目标设备的应用；检测到针对所述至少一个目标设备的第一目标设备的第一目标应用的预选取操作，显示添加到桌面的提示信息；检测到针对所述第一目标应用的确认添加操作，在所述统一桌面显示所述第一目标设备的所述第一目标应用。
在本可能的示例中，所述显示单元40还用于：检测到针对所述统一桌面上显示的所述第一目标应用的预选取操作，显示移除提示信息；检测到针对所述第一目标应用的确认移除操作，在所述统一桌面上显示除所述第一目标应用之外的初始元数据，所述初始元数据为所述统一桌面在显示移除提示信息之前所显示的元数据。
在本可能的示例中，所述显示单元40显示统一桌面之后，还用于检测到针对需要跨设备运行的目标应用的应用编辑请求操作，所述跨设备运行是指在除所述目标应用在原生设备之外的目标设备上运行所述目标应用；显示控件编辑界面，所述控件编辑界面包括控件编辑区域和控件显示区域；
其中，所述控件显示区域用于显示所述目标应用的第一应用界面，所述第一应用界面包括至少一个原生控件，所述原生控件是指所述目标应用在原生设备显示的所述第一应用界面上的控件；
其中，所述控件编辑区域用于显示所述至少一个原生控件中被选择的原生控件所对应的适配控件，所述适配控件是指所述目标设备在运行所述目标应用时所显示的第二应用界面上的控件，所述第一应用界面与所述第二应用界面对应所述目标应用的同一个功能界面。
在本可能的示例中，所述装置还包括：输出单元41，用于在所述显示单元40显示统一桌面之后，通过目标元数据输出目标应用运行状态下所关联的目标内容。
在本可能的示例中，所述目标内容包括以下至少一种：音频、视频、图片、文字、动作信息。
在一个可能的示例中，所述装置还包括：使用单元42，用于在所述显示单元40显示统一桌面之后，检测到针对所述统一桌面的目标元数据的选取操作；根据所述目标元数据的类型执行所述目标元数据的使用流程。
在本可能的示例中，在所述根据所述目标元数据的类型执行所述目标元数据的使用流程方面，所述使用单元42具体用于：判断所述目标元数据的类型；
若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环境中的所述本地兼容应用；
若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
在本可能的示例中，所述装置还包括发送单元43，用于当前设备作为源设备向目标设备发送所述目标内容以实现所述目标内容跨设备流转；其中，所述源设备为识别设备根据第一数量个设备所生成的第一手势检测关联数据而确定的内容发送设备，所述第一数量个设备包括所述识别设备、所述源设备和所述目标设备，所述识别设备为所述第一数量个设备中的任意一个；所述目标设备为所述识别设备根据第二数量个设备所生成的第二手势检测关联数据而确定的内容接收设备，所述第二数量个设备包括所述识别设备和所述目标设备。
在本可能的示例中，所述输出单元41还用于：输出用于提示内容流转的信息。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
在采用集成的单元的情况下，本申请实施例提供的桌面元数据的显示装置的结构示意图如图5所示。在图5中，桌面元数据的显示装置5包括：处理模块50和通信模块51。处理模块50用于对桌面元数据的显示装置的动作进行控制管理，例如，显示单元40所执行的步骤，和/或用于执行本文所描述的技术的其它过程。通信模块51用于支持桌面元数据的显示装置与其他设备之间的交互。如图5所示，桌面元数据的显示装置还可以包括存储模块52，存储模块52用于存储桌面元数据的显示装置的程序代码和数据。
其中，处理模块50可以是处理器或控制器，例如可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，ASIC，FPGA或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块51可以是收发器、RF电路或通信接口等。存储模块可以是存储器。
其中，上述方法实施例涉及的各场景的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。上述桌面元数据的显示装置均可执行上述图2A所示的桌面元数据的显示方法中终端所执行的步骤。
在采用对应各个功能划分各个功能模块的情况下，图6示出上述实施例中所涉及的电子装置的一种可能的结构示意图。如图6所示，电子装置6包括：
显示单元60，用于显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述第一电子设备和与所述第一电子设备通信连接的第二电子设备上的数据；
执行单元61，用于响应于对所述统一桌面上目标元数据的选择操作，根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据。
在一个可能的示例中，所述元数据包括应用类型的元数据和文件类型的元数据；所述应用类型的元数据关联所述第一电子设备和所述第二电子设备的应用，所述文件类型的元数据关联所述第一电子设备和所述第二电子设备中相同文件类型的数据。
在一个可能的示例中，在所述根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据方面，所述执行单元61具体用于：判断所述目标元数据的类型；
若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环 境中的所述本地兼容应用；
若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
在一个可能的示例中，所述本地应用流程包括以下步骤：
确定所述目标元数据所关联的数据的子类型；
判断当前设备是否存在适配所述子类型的数据的目标应用；
若存在适配所述子类型的数据的目标应用，则判断所述目标应用是否支持读取超文本传输协议Http流；
若支持读取Http流，则打开适配的目标应用并读取http流；
若不支持读取Http流，则接收根据所述目标元数据的路径确定的所述目标元数据的关联的目标数据，打开所述目标应用并读取所述目标数据；
若不存在适配所述子类型的数据的目标应用，则执行所述重定向的流程。
在一个可能的示例中，所述分布式应用包括以下步骤：
建立当前设备和提供所述目标元数据的服务的目标设备的连接；
与服务器交互，验证当前设备使用所述目标设备上的服务是否通过了用户的授权；
若已通过用户的授权，则当前设备上的分布式应用通过约定好的方式调用所述目标设备上的服务。
在一个可能的示例中，所述重定向的流程包括以下步骤：根据当前设备的类型和所述目标元数据关联的应用或数据所在设备的类型，判断是否为大屏到小屏的内容流转；若是大屏到小屏的内容流转，则与服务器交互，查看所述目标元数据是否有基于控件绑定重定向的应用定义；如果有，则执行基于控件绑定重定向的流程；如果没有，则与所述服务器交互，查看所述目标元数据是否有基于部分投屏重定向的应用定义；
若有，则执行基于部分投屏重定向的流程；若没有，则当前设备检测是否支持大屏到小屏重定向。
若支持，则开始大屏到小屏重定向的流程；若不支持，则使用通用的投屏方式来实现重定向。
其中，上述方法实施例涉及的各步骤的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。
在采用集成的单元的情况下，本申请实施例提供的电子装置的结构示意图如图7所示。在图7中，电子装置7包括：处理模块70和通信模块71。处理模块70用于对电子装置的动作进行控制管理，例如，显示单元60所执行的步骤，和/或用于执行本文所描述的技术的其它过程。通信模块71用于支持电子装置与其他设备之间的交互。如图6所示，电子装置还可以包括存储模块72，存储模块72用于存储电子装置的程序代码和数据。
其中，处理模块70可以是处理器或控制器，例如可以是中央处理器(Central Processing Unit，CPU)，通用处理器，数字信号处理器(Digital Signal Processor，DSP)，ASIC，FPGA或者其他可编程逻辑器件、晶体管逻辑器件、硬件部件或者其任意组合。其可以实现或执行结合本申请公开内容所描述的各种示例性的逻辑方框，模块和电路。所述处理器也可以是实现计算功能的组合，例如包含一个或多个微处理器组合，DSP和微处理器的组合等等。通信模块71可以是收发器、RF电路或通信接口等。存储模块可以是存储器。
其中，上述方法实施例涉及的各场景的所有相关内容均可以援引到对应功能模块的功能描述，在此不再赘述。上述电子装置均可执行上述图3所示的桌面元数据的访问方法中终端所执行的步骤。
本申请实施例还提供了一种芯片，其中，该芯片包括处理器，用于从存储器中调用并运行计算机程序，使得安装有所述芯片的设备执行如上述方法实施例中终端所描述的部分或全部步骤。
本申请实施例还提供了一种计算机可读存储介质，其中，所述计算机可读存储介质存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如上述方法实施例中终端所描述的部分或全部步骤。
本申请实施例还提供了一种计算机可读存储介质，其中，所述计算机可读存储介质存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如上述方法实施例中网络侧设备所描述的部分或全部步骤。
本申请实施例还提供了一种计算机程序产品，其中，所述计算机程序产品包括计算机程序，所述计算机程序可操作来使计算机执行如上述方法实施例中终端所描述的部分或全部步骤。该计算机程序产品可以为一个软件安装包。
本申请实施例所描述的方法或者算法的步骤可以以硬件的方式来实现，也可以是由处理器执行软件指令的方式来实现。软件指令可以由相应的软件模块组成，软件模块可以被存放于随机存取存储器(Random Access Memory，RAM)、闪存、只读存储器(Read Only Memory，ROM)、可擦除可编程只读存储器(Erasable Programmable ROM，EPROM)、电可擦可编程只读存储器(Electrically EPROM，EEPROM)、寄存器、硬盘、移动硬盘、只读光盘(CD-ROM)或者本领域熟知的任何其它形式的存储介质中。一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于ASIC中。另外，该ASIC可以位于接入网设备、目标网络设备或核心网设备中。当然，处理器和存储介质也可以作为分立组件存在于接入网设备、目标网络设备或核心网设备中。
本领域技术人员应该可以意识到，在上述一个或多个示例中，本申请实施例所描述的功能可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(Digital Subscriber Line，DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如，软盘、硬盘、磁带)、光介质(例如，数字视频光盘(Digital Video Disc，DVD))、或者半导体介质(例如，固态硬盘(Solid State Disk，SSD))等。
以上所述的具体实施方式，对本申请实施例的目的、技术方案和有益效果进行了进一步详细说明，所应理解的是，以上所述仅为本申请实施例的具体实施方式而已，并不用于限定本申请实施例的保护范围，凡在本申请实施例的技术方案的基础之上，所做的任何修改、等同替换、改进等，均应包括在本申请实施例的保护范围之内。

Claims (24)

  1. 一种桌面元数据的显示方法，其特征在于，包括：
    显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
  2. 根据权利要求1所述的方法，其特征在于，所述统一桌面还包括搜索功能元数据，所述搜索功能元数据用于在目标数据集合中进行数据搜索，所述目标数据集合对应所述目标设备集合中的设备的数据。
  3. 根据权利要求2所述的方法，其特征在于，所述方法还包括：
    显示通过所述搜索功能元数据搜索到的来自于多个设备的多个文件组，不同文件组的文件的属性互不相同，每个文件组包括同一设备的单个或多个文件；
    所述属性包括以下至少一种：颜色、大小、字体、形状。
  4. 根据权利要求2所述的方法，其特征在于，所述搜索功能元数据支持以下搜索方式的至少一种：
    按照文件夹递进的方式进行搜索；
    按照数据类型进行搜索；
    按照关键字进行搜索。
  5. 根据权利要求1-4任一项所述的方法，其特征在于，所述方法还包括：
    检测到针对第一元数据的选取操作，显示所述目标设备集合中至少一个目标设备的应用；
    检测到针对所述至少一个目标设备的第一目标设备的第一目标应用的预选取操作，显示添加到桌面的提示信息；
    检测到针对所述第一目标应用的确认添加操作，在所述统一桌面显示所述第一目标设备的所述第一目标应用。
  6. 根据权利要求5所述的方法，其特征在于，所述方法还包括：
    检测到针对所述统一桌面上显示的所述第一目标应用的预选取操作，显示移除提示信息；
    检测到针对所述第一目标应用的确认移除操作，在所述统一桌面上显示除所述第一目标应用之外的初始元数据，所述初始元数据为所述统一桌面在显示移除提示信息之前所显示的元数据。
  7. 根据权利要求1所述的方法，其特征在于，所述显示统一桌面之后，所述方法还包括：
    检测到针对需要跨设备运行的目标应用的应用编辑请求操作，所述跨设备运行是指在除所述目标应用在原生设备之外的目标设备上运行所述目标应用；
    显示控件编辑界面，所述控件编辑界面包括控件编辑区域和控件显示区域；
    其中，所述控件显示区域用于显示所述目标应用的第一应用界面，所述第一应用界面包括至少一个原生控件，所述原生控件是指所述目标应用在原生设备显示的所述第一应用界面上的控件；
    其中，所述控件编辑区域用于显示所述至少一个原生控件中被选择的原生控件所对应的适配控件，所述适配控件是指所述目标设备在运行所述目标应用时所显示的第二应用界面上的控件，所述第一应用界面与所述第二应用界面对应所述目标应用的同一个功能界面。
  8. 根据权利要求1所述的方法，其特征在于，所述显示统一桌面之后，所述方法还包括：
    通过目标元数据输出目标应用运行状态下所关联的目标内容。
  9. 根据权利要求8所述的方法，其特征在于，所述目标内容包括以下至少一种：音频、视频、图片、文字、动作信息。
  10. 根据权利要求1所述的方法，其特征在于，所述显示统一桌面之后，所述方法还包括：
    检测到针对所述统一桌面的目标元数据的选取操作；
    根据所述目标元数据的类型执行所述目标元数据的使用流程。
  11. 根据权利要求10所述的方法，其特征在于，所述根据所述目标元数据的类型执行所述目标元数据的使用流程，包括：
    判断所述目标元数据的类型；
    若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
    若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
    根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
    若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
    若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
    若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环境中的所述本地兼容应用；
    若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
  12. 根据权利要求8-11任一项所述的方法，其特征在于，所述通过目标元数据输出目标应用运行状态下所关联的目标内容之后，所述方法还包括：
    当前设备作为源设备向目标设备发送所述目标内容以实现所述目标内容跨设备流转；
    其中，所述源设备为识别设备根据第一数量个设备所生成的第一手势检测关联数据而确定的内容发送设备，所述第一数量个设备包括所述识别设备、所述源设备和所述目标设备，所述识别设备为所述第一数量个设备中的任意一个；
    所述目标设备为所述识别设备根据第二数量个设备所生成的第二手势检测关联数据而确定的内容接收设备，所述第二数量个设备包括所述识别设备和所述目标设备。
  13. 根据权利要求12所述的方法，其特征在于，所述方法还包括：
    输出用于提示内容流转的信息。
  14. 根据权利要求1-12任一项所述的方法，其特征在于，所述显示统一桌面之前，所述方法还包括：
    通过当前设备的统一桌面账号登陆入口录入目标用户账号的登陆信息，并向服务器发送所述目标用户账号的登陆信息，所述目标用户账号的登陆信息用于所述服务器执行以下操作：校验所述登陆信息合法后，获取所述目标用户账号的统一桌面的元数据集合，并向请求登陆的对端设备发送所述元数据集合；
    接收来自所述服务器的所述目标用户账号的所述统一桌面的元数据集合，所述元数据集合用于当前设备显示所述统一桌面。
  15. 一种元数据的访问方法，其特征在于，包括：
    第一电子设备显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述第一电子设备和与所述第一电子设备通信连接的第二电子设备上的数据；
    响应于对所述统一桌面上目标元数据的选择操作，根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据。
  16. 根据权利要求15所述的方法，其特征在于，所述元数据包括应用类型的元数据和文件类型的元数据；所述应用类型的元数据关联所述第一电子设备和所述第二电子设备的应用，所述文件类型的元数据关联所述第一电子设备和所述第二电子设备中相同文件类型的数据。
  17. 根据权利要求16所述的方法，其特征在于，所述根据所述目标元数据的类型访问所述第一电子设备和所述第二电子设备上的元数据，包括：
    判断所述目标元数据的类型；
    若所述目标元数据的类型是文件类型，则执行本地应用流程，所述本地应用流程是指通过本地适配 应用读取所述目标元数据关联的数据，所述本地适配应用是指当前设备安装的能够使用所述目标元数据关联的数据的应用；
    若所述目标元数据的类型是应用类型，则获取所述目标元数据的应用相关信息；
    根据所述应用相关信息判断当前设备是否存在相适配的分布式应用；
    若存在相适配的分布式应用，则执行分布式应用流程，所述分布式应用流程是指：所述目标元数据关联的应用的后台服务运行在远端设备上，应用的用户接口UI运行在当前设备上，通过分布式能力系统连接所述UI和所述后台服务，实现所述分布式应用的跨端运行；
    若不存在相适配的分布式应用，则查找当前设备是否有与所述目标元数据对应的本地兼容应用；
    若当前设备存在与所述目标元数据对应的本地兼容应用，则启动本地模拟环境，并运行所述模拟环境中的所述本地兼容应用；
    若当前设备不存在与所述目标元数据对应的本地兼容应用，则启动重定向的流程，所述重定向的流程是指：所述目标元数据关联的目标应用运行在远端设备上，把所述目标应用的画面和/或声音从远端设备上重定向到当前设备，把当前设备的外设输入信息重定向到远端设备的所述目标应用中。
  18. 根据权利要求17所述的方法，其特征在于，所述本地应用流程包括以下步骤：
    确定所述目标元数据所关联的数据的子类型；
    判断当前设备是否存在适配所述子类型的数据的目标应用；
    若存在适配所述子类型的数据的目标应用，则判断所述目标应用是否支持读取超文本传输协议Http流；
    若支持读取Http流，则打开适配的目标应用并读取http流；
    若不支持读取Http流，则接收根据所述目标元数据的路径确定的所述目标元数据的关联的目标数据，打开所述目标应用并读取所述目标数据；
    若不存在适配所述子类型的数据的目标应用，则执行所述重定向的流程。
  19. 根据权利要求17所述的方法，其特征在于，所述分布式应用包括以下步骤：
    建立当前设备和提供所述目标元数据的服务的目标设备的连接；
    与服务器交互，验证当前设备使用所述目标设备上的服务是否通过了用户的授权；
    若已通过用户的授权，则当前设备上的分布式应用通过约定好的方式调用所述目标设备上的服务。
  20. 根据权利要求17所述的方法，其特征在于，所述重定向的流程包括以下步骤：
    根据当前设备的类型和所述目标元数据关联的应用或数据所在设备的类型，判断是否为大屏到小屏的内容流转；
    若是大屏到小屏的内容流转，则与服务器交互，查看所述目标元数据是否有基于控件绑定重定向的应用定义；
    如果有，则执行基于控件绑定重定向的流程；
    如果没有，则与所述服务器交互，查看所述目标元数据是否有基于部分投屏重定向的应用定义；
    若有，则执行基于部分投屏重定向的流程；
    若没有，则当前设备检测是否支持大屏到小屏重定向；
    若支持，则开始大屏到小屏重定向的流程；
    若不支持，则使用通用的投屏方式来实现重定向。
  21. 一种桌面元数据的显示装置，其特征在于，包括：
    显示单元，用于显示统一桌面，所述统一桌面包括元数据，所述元数据包括应用类型的元数据和文件类型的元数据，所述应用类型的元数据关联目标设备集合中一个或多个设备的应用，所述文件类型的元数据关联所述目标设备集合中至少一个设备的相同文件类型的数据。
  22. 一种电子装置，其特征在于，包括：
    显示单元，用于显示统一桌面，所述统一桌面包括元数据，所述元数据关联所述电子装置以及与所述电子装置连接的第二设备的元数据；
    控制单元，用于响应于对所述统一桌面上目标元数据的选择操作，以根据所述目标元数据的类型控制所述电子装置以及所述第二设备中的元数据。
  23. 一种电子设备，其特征在于，包括处理器、存储器，以及一个或多个程序，所述一个或多个程序被存储在所述存储器中，并且被配置由所述处理器执行，所述程序包括用于执行如权利要求1-20任一项所述的方法中的步骤的指令。
  24. 一种计算机可读存储介质，其特征在于，存储用于电子数据交换的计算机程序，其中，所述计算机程序使得计算机执行如权利要求1-20任一项所述的方法。
PCT/CN2021/121291 2020-11-16 2021-09-28 桌面元数据的显示方法、访问方法及相关装置 WO2022100309A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011281254.7 2020-11-16
CN202011281254.7A CN112269516B (zh) 2020-11-16 2020-11-16 桌面元数据的显示方法、访问方法及相关装置

Publications (1)

Publication Number Publication Date
WO2022100309A1 true WO2022100309A1 (zh) 2022-05-19

Family

ID=74339145

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/121291 WO2022100309A1 (zh) 2020-11-16 2021-09-28 桌面元数据的显示方法、访问方法及相关装置

Country Status (2)

Country Link
CN (2) CN115145439B (zh)
WO (1) WO2022100309A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115941911A (zh) * 2022-12-02 2023-04-07 浪潮卓数大数据产业发展有限公司 可视化大屏投屏效果自动校正偏色的方法及设备、介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115145439B (zh) * 2020-11-16 2024-06-11 Oppo广东移动通信有限公司 桌面元数据的显示方法、访问方法及相关装置
CN115150502B (zh) * 2021-03-31 2024-06-11 华为技术有限公司 一种应用小部件的显示方法及装置、存储介质
CN113190196B (zh) * 2021-04-27 2023-09-05 北京京东振世信息技术有限公司 多设备联动实现方法、装置、介质及电子设备
CN118113305A (zh) * 2022-11-29 2024-05-31 华为技术有限公司 应用跨设备流转方法、相关装置及通信系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049457A (zh) * 2011-10-17 2013-04-17 联想(北京)有限公司 关联方法、电子设备和混合系统
CN108733771A (zh) * 2017-04-21 2018-11-02 溥美公司 共享应用，包括允许检索、展示和遍历信息资源的共享应用
CN109388620A (zh) * 2018-09-30 2019-02-26 联想(北京)有限公司 一种跨设备访问数据的方法及第一电子设备
US20190391779A1 (en) * 2018-06-21 2019-12-26 Amadeus S.A.S. Cross device display synchronization
CN111666119A (zh) * 2019-03-06 2020-09-15 华为终端有限公司 Ui组件显示的方法及电子设备
CN112269516A (zh) * 2020-11-16 2021-01-26 Oppo广东移动通信有限公司 桌面元数据的显示方法、访问方法及相关装置

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1963967B1 (en) * 2005-10-07 2016-12-07 Citrix Systems, Inc. Methods for selecting between a predetermined number of execution methods for an application program
JP5837597B2 (ja) * 2010-08-30 2015-12-24 ヴイエムウェア インコーポレイテッドＶＭｗａｒｅ，Ｉｎｃ． シンアプリケーション、リモートアプリケーション、およびＳａａＳアプリケーションのための統合ワークスペース
US8866701B2 (en) * 2011-03-03 2014-10-21 Citrix Systems, Inc. Transparent user interface integration between local and remote computing environments
WO2014119889A1 (en) * 2013-01-31 2014-08-07 Samsung Electronics Co., Ltd. Method of displaying user interface on device, and device
US9936018B2 (en) * 2013-09-27 2018-04-03 Mcafee, Llc Task-context architecture for efficient data sharing
US9612723B1 (en) * 2015-12-30 2017-04-04 Palantir Technologies Inc. Composite graphical interface with shareable data-objects
CN109032746B (zh) * 2018-08-10 2021-08-24 广东小天才科技有限公司 一种可穿戴设备的显示界面自定义方法、系统及电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049457A (zh) * 2011-10-17 2013-04-17 联想(北京)有限公司 关联方法、电子设备和混合系统
CN108733771A (zh) * 2017-04-21 2018-11-02 溥美公司 共享应用，包括允许检索、展示和遍历信息资源的共享应用
US20190391779A1 (en) * 2018-06-21 2019-12-26 Amadeus S.A.S. Cross device display synchronization
CN109388620A (zh) * 2018-09-30 2019-02-26 联想(北京)有限公司 一种跨设备访问数据的方法及第一电子设备
CN111666119A (zh) * 2019-03-06 2020-09-15 华为终端有限公司 Ui组件显示的方法及电子设备
CN112269516A (zh) * 2020-11-16 2021-01-26 Oppo广东移动通信有限公司 桌面元数据的显示方法、访问方法及相关装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115941911A (zh) * 2022-12-02 2023-04-07 浪潮卓数大数据产业发展有限公司 可视化大屏投屏效果自动校正偏色的方法及设备、介质
CN115941911B (zh) * 2022-12-02 2024-06-04 浪潮卓数大数据产业发展有限公司 可视化大屏投屏效果自动校正偏色的方法及设备、介质

Also Published As

Publication number Publication date
CN115145439B (zh) 2024-06-11
CN112269516B (zh) 2022-06-17
CN115145439A (zh) 2022-10-04
CN112269516A (zh) 2021-01-26

Similar Documents

Publication Publication Date Title
WO2022100309A1 (zh) 桌面元数据的显示方法、访问方法及相关装置
KR102350329B1 (ko) 전화 통화 동안의 실시간 공유 기법
JP5939525B2 (ja) 共有サービス
KR102064952B1 (ko) 수신 데이터를 이용하여 어플리케이션을 운영하는 전자 장치
RU2631137C2 (ru) Связывание устройств
US20240053879A1 (en) Object Drag Method and Device
CN101594279B (zh) 信息处理设备、信息处理方法以及信息处理系统
WO2020204406A1 (en) Method and system for learning and enabling commands via user demonstration
US11204681B2 (en) Program orchestration method and electronic device
CN114286165B (zh) 一种显示设备、移动终端、投屏数据传输方法及系统
KR20130090947A (ko) 서비스 연동 수행 방법과 이를 위한 시스템
JP6258475B2 (ja) クライアントデバイスにメディアアセットを提供するための方法
US20130293486A1 (en) Touch-based remote control
US20240086231A1 (en) Task migration system and method
WO2022057393A1 (zh) 事件处理方法、装置、存储介质、移动终端及电脑
CN113784200B (zh) 通信终端、显示设备及投屏连接方法
WO2024045985A1 (zh) 屏幕控制方法、屏幕控制装置、电子设备、程序及介质
TW201541993A (zh) 互連視訊的播放
WO2015131813A1 (zh) 一种进行设备操作的方法和系统
WO2023284498A1 (zh) 视频播放方法、装置及存储介质
KR20140113826A (ko) 컨텐츠 공유를 위한 장치 및 방법, 이를 위한 통신 서비스 시스템
KR102245291B1 (ko) 전자 디바이스 및 전자 디바이스에서 캡쳐 이미지 이용 방법
CN113784186B (zh) 终端设备、服务器和通信控制方法
JP2014511515A (ja) 自動マッチングによる実オブジェクトのセットへのサービス展開
WO2015030467A1 (ko) 전자 장치 및 전자 장치에서 컨텐츠 제공 방법

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21890837

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21890837

Country of ref document: EP

Kind code of ref document: A1