WO2022218197A1 - 一种跨设备交互方法及设备 - Google Patents

一种跨设备交互方法及设备 Download PDF

Info

Publication number
WO2022218197A1
WO2022218197A1 PCT/CN2022/085461 CN2022085461W WO2022218197A1 WO 2022218197 A1 WO2022218197 A1 WO 2022218197A1 CN 2022085461 W CN2022085461 W CN 2022085461W WO 2022218197 A1 WO2022218197 A1 WO 2022218197A1
Authority
WO
WIPO (PCT)
Prior art keywords
information
user operation
electronic device
main control
control device
Prior art date
Application number
PCT/CN2022/085461
Other languages
English (en)
French (fr)
Inventor
何溯
孙冰心
孙斐
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022218197A1 publication Critical patent/WO2022218197A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials

Definitions

  • the embodiments of the present application relate to the field of terminal technologies, and in particular, to a cross-device interaction method and device.
  • IoT Internet of Things
  • APP application
  • the user can open the smart home APP installed on the terminal device, and the smart home APP obtains various data of the associated smart home device from the cloud platform and displays it.
  • the home APP issues control instructions to the smart home equipment, and the smart home APP sends the control instructions to the smart home equipment through the cloud platform to realize the control of the smart home equipment.
  • Embodiments of the present application provide a cross-device interaction method and device, which are used to realize the separation of business logic and UI across devices, reduce development difficulty for manufacturers, reduce usage costs for users, and improve data security.
  • an embodiment of the present application provides a cross-device interaction method.
  • the method includes: an electronic device, in response to user interface UI information from a master control device, displays a corresponding UI, where the UI at least includes a first function option, The first function option is associated with at least one user operation type; the electronic device sends user operation information to the main control device in response to the first user operation based on the first function option in the UI, and the user
  • the operation information includes at least the identification information of the first function option and the type of the first user operation, and the user operation information is used to trigger the main control device to perform a control operation or management operation corresponding to the user operation information .
  • the user operation information further includes user input information obtained based on the first user operation.
  • the main control device may be an IoT device or the like.
  • the business code and the UI code are all run in the main control device, and the manufacturer does not need to develop multiple APPs for different APP platforms, which can reduce development difficulty and save costs.
  • the main control device does not need to expose the business API to the outside world. All interactions are through the UI, which improves the security of data.
  • the user does not need to download the APP of different manufacturers on the electronic device.
  • the control of the main control device is beneficial to reduce the use cost of the user.
  • the UI information and the user operation information are information in a domain-specific language DSL format.
  • DSL (such as JSON format code, etc.) can be used to transmit UI and events (such as user operation information) between devices, which is beneficial to the parsing and identification of UI and events between devices.
  • the method further includes: before the electronic device responds to the UI information from the master control device, further includes: the electronic device responds to selecting the master control device from a list of devices to be connected in response to the second user operation to establish a communication connection with the master control device.
  • the main control device can send its own UI information to the electronic device after detecting that the connection is established with the electronic device, which helps to realize the separation of business logic and UI across the main control device and the electronic device.
  • the method further includes: the electronic device receives UI update information from the main control device, where the UI update information at least includes the main control device Change information of the UI after the control operation or management operation corresponding to the user operation information is performed; the electronic device updates the UI in response to the UI update information.
  • the electronic device updates the displayed UI according to the UI update information sent by the main control device to perform the corresponding control operation or management for the main control device.
  • the UI after the operation is conducive to maintaining the consistency between the business logic of the main control device and the UI displayed by the electronic device.
  • an embodiment of the present application provides a cross-device interaction method.
  • the method includes: a master control device sends user interface UI information to an electronic device, where the UI corresponding to the UI information at least includes a first function option, and the first function option is included in the UI.
  • a function option is associated with at least one user operation type;
  • the main control device receives user operation information from the electronic device, the user operation information is the first function of the electronic device based on the first function option in the UI Obtained by a user operation, the user operation information includes at least the identification information of the first function option and the type of the first user operation; the main control device performs the control operation or management corresponding to the operation information. operate.
  • the user operation information further includes user input information obtained based on the first user operation.
  • the UI information and the user operation information are information in a domain-specific language DSL format.
  • the main control device sending the UI information to the electronic device includes: after the main control device establishes a communication connection with the electronic device, sending the UI information to the electronic device.
  • the method further includes: the main control device sends UI update information to the electronic device, and the UI
  • the update information at least includes change information of the UI after the main control device performs a control operation or a management operation corresponding to the user operation information.
  • an embodiment of the present application further provides an electronic device, the electronic device includes: a memory, a processor, and a display screen; wherein the processor is connected to the memory and the display screen; One or more computer programs, the one or more computer programs comprising instructions that, when executed by the electronic device, cause the electronic device to perform the first aspect or any of the possibilities of the first aspect method described in the design.
  • an embodiment of the present application further provides a main control device, where the main control device includes: a memory and a processor; wherein the processor is connected to the memory; the memory stores one or more A computer program, the one or more computer programs include instructions, when the instructions are executed by the main control device, the main control device is made to perform the second aspect or any one of the possible designs of the second aspect. the method described.
  • an embodiment of the present application further provides an electronic device, the electronic device includes functional modules capable of executing the above-mentioned first aspect or any possible method in the design of the first aspect; these functional modules can be implemented by hardware , and can also be implemented by hardware executing corresponding software.
  • an embodiment of the present application further provides a main control device, the main control device includes functional modules capable of executing the second aspect or any possible method in the design of the second aspect; these functional modules can be configured by It can be realized by hardware, and can also be realized by executing corresponding software by hardware.
  • an embodiment of the present application further provides a computer storage medium, including computer instructions, when the computer instructions are run on a computer, the computer is made to execute the first aspect or any of the possible first aspects. Design the method described in, or perform the method described in the second aspect above or any possible design of the second aspect.
  • an embodiment of the present application further provides a program product that, when the program product runs on a computer, enables the computer to execute the first aspect or any possible design of the first aspect. method, or perform the method described in the above second aspect or any possible design of the second aspect.
  • a ninth aspect a chip provided by an embodiment of the present application is used to call and run a computer program stored in the memory from a memory, and execute the above-mentioned first aspect or any possible design method of the first aspect, Or a method for implementing the second aspect or any possible design of the second aspect.
  • Figure 1 is a schematic diagram of a smart home equipment control scheme
  • FIG. 2 is a schematic structural diagram of a cross-device interaction system provided by an embodiment of the present application
  • FIG. 3 is one of the schematic structural diagrams of an electronic device provided by an embodiment of the present application.
  • FIG. 4 is one of schematic diagrams of electronic device interfaces provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a display application interface provided by an embodiment of the present application.
  • FIG. 6 is one of the schematic view diagrams provided by the embodiment of the present application.
  • Fig. 7 is one of UI schematic diagrams provided by the embodiment of the present application.
  • FIG. 8 is the second UI schematic diagram provided by the embodiment of the present application.
  • FIG. 9 is a schematic structural diagram of an electronic device and an intelligent switch provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a cross-device interaction method provided by an embodiment of the present application.
  • FIG. 11 is the second schematic view of the view provided by the embodiment of the present application.
  • FIG. 12 is the second schematic structural diagram of the electronic device provided by the embodiment of the application.
  • FIG. 13 is a schematic structural diagram of a main control device provided by an embodiment of the present application.
  • IoT devices such as smart lights, smart sockets, and smart water meters
  • IoT devices such as smart lights, smart sockets, and smart water meters
  • APPs are cumbersome to use, and manufacturers need to maintain multiple APP operation and maintenance teams for different APP platforms.
  • the operation and maintenance costs are high, and the data transmission of IoT devices on the public network can easily lead to data leakage and illegal collection.
  • the embodiments of the present application provide a cross-device interaction solution.
  • business logic and user interface user interface, UI
  • applications can be displayed and interacted across devices, and users can use watches, tablet
  • UI user interface
  • any electronic device with a screen controls IoT devices
  • IoT devices there is no need for manufacturers to maintain multiple APP operation and maintenance teams, and users do not need to download multiple APPs from different manufacturers, which reduces the difficulty of manufacturer development and improves user experience.
  • all interactions are implemented through the UI, and IoT devices do not need to expose business application programming interfaces (APIs), which can improve the security of IoT device data.
  • APIs business application programming interfaces
  • FIG. 2 is a schematic diagram of the architecture of a possible cross-device interaction system provided by an embodiment of the present application.
  • the system includes multiple master control devices (such as smart switches, smart lights, smart refrigerators, etc.) and electronic devices.
  • the electronic devices It can be equipped with different operating systems and installed with the same display application, and the electronic device and the main control device can communicate through WiFi, Bluetooth, cellular network, etc.
  • the UI of the main control device is displayed on the electronic device, and the UI and events (such as the operation of the function options in the UI) are transmitted between the main control device and the electronic device. , can realize the interaction across the main control device and the electronic device, and realize the control of the main control device.
  • the main control device, the main control device in the embodiment of the present application can be an IoT device, for example, can be smart home devices such as smart lights, smart sockets, smart switches, smart air conditioners, smart refrigerators, etc., or can be smart traffic lights, smart Water meters, road side units (RSUs) and other IoT devices.
  • the main control device in the embodiment of the present application may also be a device such as a television. It should be understood that, the main control device in the embodiment of the present application may be a device without a screen, or a device with a screen.
  • JSON Javascript object notation
  • the event-driven in the computer field simply means that the computer performs the corresponding response operation based on the generated event.
  • the basic structure of an event-driven program consists of an event collector, an event sender and an event handler.
  • the event collector is specially responsible for collecting all events, including events from user operations (such as mouse, keyboard events, etc.), events from hardware (such as clock events, etc.) and events from software (such as operating system, application logic processing generated events, etc.).
  • the event sender is responsible for distributing the events collected by the collector to the target object. Event handlers do specific event response work.
  • the electronic devices in the embodiments of the present application may be portable terminal devices, such as mobile phones, tablet computers, portable computers, wearable electronic devices (such as smart watches, smart helmets, smart glasses, smart bracelets, etc.) , VR devices, augmented reality (AR) devices, etc.
  • the portable terminal includes but is not limited to carrying Hongmeng or other operating systems.
  • the electronic device in the embodiment of the present application may not be a portable terminal device, for example, a display device with a screen such as a TV and a personal computer (personal computer, PC).
  • the electronic device includes a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, key 190, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) ) card interface 195, etc.
  • a processor 110 an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, sensor module 180, key 190, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) ) card interface 195, etc.
  • SIM subscriber identification module
  • the sensor module 180 may include a pressure sensor, a gyroscope sensor, an acceleration sensor, a fingerprint sensor, a touch sensor, and the like
  • the audio module 170 may include a speaker, a microphone, an earphone interface, and the like.
  • Processor 110 may include one or more processing units.
  • the processor 110 may include an application processor (AP), a modem (modem), a graphics processor (graphics processing unit, GPU), an image signal processor (ISP), a controller, a video encoder Decoder, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (neural-network processing unit, NPU), etc.
  • AP application processor
  • modem modem
  • graphics processor graphics processor
  • ISP image signal processor
  • controller a video encoder Decoder
  • DSP digital signal processor
  • baseband processor baseband processor
  • neural-network processing unit neural-network processing unit
  • a memory may also be provided in the processor 110 for storing computer programs and/or data.
  • the memory in processor 110 is cache memory.
  • the memory may hold computer programs and/or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the computer program and/or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.
  • the processor 110 may include one or more interfaces.
  • the processor 110 includes a universal serial bus (USB) interface 130 and a subscriber identity module (SIM) interface 195 .
  • the processor 110 may further include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous Transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), and/or general-purpose input/output (general-purpose input/output, GPIO) interface, etc.
  • I2C integrated circuit
  • I2S integrated circuit built-in audio
  • PCM pulse code modulation
  • PCM pulse code modulation
  • UART universal asynchronous Transmitter
  • MIPI mobile industry processor interface
  • GPIO general-purpose input/output
  • the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device.
  • the electronic device may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
  • the USB interface 130 is an interface that conforms to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like.
  • the USB interface 130 can be used to connect a charger to charge the electronic device, and can also be used to transmit data between the electronic device and peripheral devices. It can also be used to connect headphones to play audio through the headphones.
  • the interface can also be used to connect other electronic devices, such as augmented reality (AR) devices.
  • AR augmented reality
  • the SIM card interface 195 is used to connect a SIM card.
  • the SIM card can be inserted into the SIM card interface 195 or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device.
  • the electronic device can support 2 or N SIM card interfaces, where N is a positive integer greater than 2.
  • the SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different.
  • the SIM card interface 195 can also be compatible with different types of SIM cards.
  • the SIM card interface 195 is also compatible with external memory cards.
  • the electronic device interacts with the network through the SIM card to realize functions such as call and data communication.
  • the electronic device employs an eSIM, ie: an embedded SIM card.
  • the eSIM card can be embedded in the electronic device and cannot be separated from the electronic device.
  • the charging management module 140 is used to receive charging input from the charger.
  • the charger may be a wireless charger or a wired charger.
  • the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .
  • the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device. While the charging management module 140 charges the battery 142 , it can also supply power to the electronic device through the power management module 141 .
  • the power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 .
  • the power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, the wireless communication module 160, and the like.
  • the power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, battery health status (leakage, impedance).
  • the power management module 141 may also be provided in the processor 110 .
  • the power management module 141 and the charging management module 140 may also be provided in the same device.
  • the wireless communication function of the electronic device may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem, the baseband processor, and the like.
  • Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals.
  • Each antenna in an electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.
  • the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
  • the mobile communication module 150 can provide a wireless communication solution including 2G/3G/4G/5G/(6G and other subsequent evolution standards) and other standards applied to the electronic device.
  • the mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), and the like.
  • the wireless communication module 160 includes wireless local area networks (WLAN) (such as Wi-Fi networks), bluetooth (BT), global navigation satellite system (GNSS), which can provide applications on electronic devices. ), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR) and other wireless communication solutions.
  • WLAN wireless local area networks
  • BT Bluetooth
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication
  • infrared technology infrared, IR
  • the antenna 1 of the electronic device is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device can communicate with the network and other devices through wireless communication technology.
  • the wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM and/or IR technology, etc.
  • the GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou satellite navigation system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).
  • GPS global positioning system
  • GLONASS global navigation satellite system
  • BDS Beidou satellite navigation system
  • BDS Beidou navigation satellite system
  • QZSS quasi-zenith satellite system
  • SBAS satellite based augmentation systems
  • the electronic device implements a display function through a GPU, a display screen 194, an application processor, and the like.
  • the display screen 194 is used to display images, videos, and the like.
  • Display screen 194 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode, or an active matrix organic light emitting diode (active-matrix organic light).
  • emitting diode, AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed
  • quantum dot light-emitting diode quantum dot light emitting diodes, QLED
  • the electronic device may include 1 or N display screens 194 , where N is a positive integer greater than 1.
  • the electronic device can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, and the application processor.
  • the ISP is used to process the data fed back by the camera 193 .
  • the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye.
  • ISP can also perform algorithm optimization on image noise, brightness and skin tone.
  • ISP can also optimize parameters such as exposure and color temperature of the shooting scene.
  • the ISP may be provided in the camera 193 .
  • the electronic device may include 1 or N cameras 193 , where N is a positive integer greater than 1.
  • the external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device.
  • the external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function.
  • the internal memory 121 includes an operating memory (memory) and a built-in memory.
  • the running memory can be used to store computer programs and/or data and the like.
  • the processor 110 executes various functional applications and data processing of the electronic device by executing the computer program stored in the operating memory.
  • running memory may include high-speed random access memory.
  • the built-in memory can also be called built-in external memory, etc., and can be used to store computer programs and/or data.
  • the built-in memory may store an operating system, application programs, and the like.
  • the electronic device usually loads the computer program and/or data in the built-in memory into the running memory, so that the processor 110 runs the corresponding computer program and/or data to realize the corresponding function.
  • the internal memory 121 may include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), and the like.
  • the electronic device can implement the audio function through the audio module 170 and the application processor and the like. Such as music playback, recording, etc.
  • the keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key.
  • the electronic device may receive key input and generate key signal input related to user settings and function control of the electronic device.
  • the indicator 192 can be an indicator light, which can be used to indicate a charging state, a change in power, or a message, a missed call, a notification, and the like.
  • Touch sensor also known as "touch panel”.
  • the touch sensor may be disposed on the display screen 194, and the touch sensor and the display screen 194 form a touch screen, also referred to as a "touch screen”.
  • a touch sensor is used to detect touch operations on or near it.
  • the touch sensor can pass the detected touch operation to the application processor to determine the type of touch event.
  • Visual output associated with touch operations can be provided through the display.
  • the touch sensor may also be disposed on the surface of the electronic device, which is different from where the display screen 194 is located.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device.
  • the electronic device may include more or less components than shown, or combine some components, or separate some components, or arrange different components.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • At least one in the embodiments of the present application refers to one or more.
  • "Plural” means two or more.
  • "And/or" which describes the association relationship of associated objects, indicates that there can be three kinds of relationships. For example, A and/or B can represent three cases where A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.
  • the character “/” generally indicates that the associated objects are an “or” relationship.
  • At least one (item) of the following” or its similar expression refers to any combination of these items, including any combination of single item (item) or plural item (item).
  • the following describes the cross-device interaction method of the present application by taking the main control device as an intelligent switch in an IoT device as an example.
  • the main interface 410 displayed on the display screen of the electronic device.
  • the user can click the icon 411 displaying the application on the main interface 410, and the electronic device displays the interface displaying the application on the display screen in response to the user's operation of clicking the icon 411 displaying the application.
  • the interface for displaying the application may be the interface 420 shown in B in FIG. 4 .
  • the display interface includes 3 function options: WiFi421, Bluetooth 422, Account 423, these 3 function options can be realized by the control.
  • the function option WiFi421 is used to control the display application to search for the main control device that is in the same WiFi network as the electronic device, and to enter the management interface of the main control device that is in the same WiFi network as the electronic device;
  • the function option Bluetooth 422 is used to control the display application search The master control device located within the Bluetooth coverage of the electronic device, and enter the management interface of the master control device located within the Bluetooth coverage of the electronic device;
  • the function option account number 423 is used to control the display application to communicate with the display application server through the cellular network, and from the display The application server obtains the master control device associated with the user's personal account, and enters the management interface of the master control device associated with the user's personal account.
  • the interface 420 for displaying the application may also include other functional options, such as a functional option for controlling exiting the displayed application, a functional option for configuring a user's personal account, and the like.
  • the main control device in the same WiFi network, and enter the management interface 510 of the main control device in the same WiFi network as the electronic device. If the smart switch is already in the device list 511, the user can directly click the "smart switch" in the device list 511 option, the display application can respond to the user's operation of clicking the "smart switch” option to establish a WiFi connection with the smart switch.
  • the user can click the search button 512 to display the application to respond to the user's operation of clicking the search button 512, re-search for the master device that is in the same WiFi as the electronic device, and search for the device list.
  • the master device in 511 is updated.
  • a service containing UI code is running in the smart switch.
  • the UI code running in the smart switch includes the background color of rgba(30,144,255,0.7); the style of function options (such as components) is "button", attribute For "closed”, supported (or associated) operation types including "click” and other information, the specific UI code can be as follows:
  • the UI parsing engine in the smart switch can describe the UI (ie UI code) of the smart switch into a domain specific language. DSL) and sent to the electronic device.
  • the smart switch can describe the hml, Css, and Js parts of the UI code to generate JSON format code, where the hml and Css parts mainly describe the interface information of the smart switch UI, and the Js part mainly describes the smart switch UI.
  • Types of operations such as single-click, double-click, etc.
  • functional options such as components.
  • JSON format code that generates (ie, converts) the above UI code description of the smart switch.
  • the display application on the electronic device After the display application on the electronic device receives the UI information from the smart switch and draws the UI of the smart switch, it can draw the UI of the smart switch by using the view as a unit. The graphics of the view, and then combine all the graphics of the view into a UI. Specifically, when building the view tree, the display application can consider the positional relationship between different views. Taking the UI information received by the display application as the above JSON format code as an example, the display application parses the above JSON format code, and the constructed view tree is shown in Figure 6. View1 corresponds to layer 1, view1 is located on the background, and the background can be used as A single view is the view root (viewroot), and view1 is the child node of viewroot.
  • the display application renders the layout of each view in the view tree according to the view tree to synthesize a UI.
  • the display application obtains the color corresponding to 512819123 according to the corresponding style as "background (div)" and the background color as "512819123", that is, rgba(30, 144, 255, 0.7), and sets the background color as rgba(30, 144, 255, 0.7).
  • rgb in rgba corresponds to red (red, R), green (green, G), and blue (blue, B)
  • a represents the alpha channel (alpha), which usually refers to transparency.
  • the displayed UI is displayed after the application performs layout rendering according to the UI information.
  • the display application can determine the function option corresponding to the user's operation on the UI according to the position of the operation, and determine whether the user operation is an operation type supported by the function option. , if yes, the display application generates a DSL from the description of the identification information of the function option and the type of user operation (such as single click, double click, etc.), and sends it to the smart switch.
  • the display application determines the corresponding operation on the UI according to the position of the operation.
  • the function option is the function option with the style of "button"
  • the click operation belongs to the operation type supported by the function option
  • the identification information (such as id) of the function option and the type of user operation by the application are displayed, such as "click operation (such as onclick)" description to generate a DSL and send it to the smart switch.
  • the specific DSL can be as follows.
  • the smart switch After receiving the DSL, the smart switch parses the DSL, calls the processing function corresponding to the click operation on the function option with the style "button”, and executes the opening corresponding to the click operation on the function option with the style "button” command, turn the smart switch on, and update the attribute of the function option with the style "button” to "on".
  • the UI parsing engine of the smart switch converts the updated attribute information of the function option with the style of "button” into a DSL, and sends it to the electronic device.
  • the DSL contains information to update the property of the feature option with id 2 of style "button” to "opened".
  • the display application updates the attribute of the function option styled as “button” in the UI from “closed” to “opened” according to the received DSL.
  • the overall information of the updated UI of the smart switch can also be sent to the display application, and the display application can send the information to the display application according to the smart switch. Switch the updated UI information to update the UI.
  • FIG. 9 is a schematic diagram of the architecture of a smart switch and an electronic device provided by an embodiment of the present application.
  • the interface layout description includes a code describing the interface information of the UI of the smart switch, and the logic code includes the UI describing the smart switch.
  • the code of the event information (such as the operation of the function option) of one or more function options in the interface, the code in the interface layout description is parsed by the UI parsing engine (such as the JS engine), and the smart switch can obtain the UI interface of the smart switch.
  • the UI parsing engine such as the JS engine
  • the smart switch can also obtain the interface information in the logic code (such as the operation type supported by one or more functional options), through the interface information of the UI of the smart switch and the interface information supported by one or more functional options in the UI of the smart switch
  • the operation type is encoded and converted into DSL to send to the display application of the electronic device.
  • the smart switch can also send UI-related resources, such as media (music, video, color, etc.) and images (such as pictures of function options, etc.) to the display application of the electronic device.
  • the display application parses the received DSL to obtain the interface information of the UI of the smart switch and the operation type supported by one or more function options in the UI of the smart switch, and displays the UI of the smart switch through layout rendering.
  • the display application detects a user operation for a certain target function option in the displayed UI, if the user operation belongs to the operation type supported by the target function option, the type of the user operation and the identification information of the target function option are encoded by encoding , converted into DSL and sent to the smart switch, the smart switch parses the received DSL, calls the processing function corresponding to the user operation on the target function option, executes the instruction corresponding to the user operation on the target function option, and The updated attribute information of the target function option is encoded into a DSL and fed back to the display application, so that the display application in the electronic device updates the displayed UI.
  • the steps performed by the smart switch in the implementation of the present application may also be implemented by an application in the smart switch, for example, by a switch application installed in the smart switch.
  • FIG. 10 shows a schematic diagram of a device interaction method provided by an embodiment of the present application. As shown in FIG. 10 , the flow of the method includes:
  • S1001 The electronic device establishes a communication connection with the master device in response to a second user operation of selecting the master device from the list of devices to be connected.
  • the electronic device can search for a master control device that is in the same WiFi network as the electronic device through WiFi, Bluetooth, etc., or search for a master control device located within the Bluetooth coverage of the electronic device, or log in to the user's personal account through
  • the server obtains the master control device associated with the user's personal account, and can establish a communication connection with the master control device selected by the user according to the second user operation of selecting the master control device from the device list to be connected.
  • the device list 511 in FIG. 5 is an example of a possible list of devices to be connected.
  • the electronic device can establish a communication connection with the master device selected by the user through the WiFi network.
  • a master device such as a smart switch
  • the main control device sends UI information to the electronic device, and the electronic device receives the UI information.
  • the UI corresponding to the UI information includes at least a first function option (eg, a component), and the first function option is associated with at least one user operation type (eg, single-click, double-click, etc.).
  • the UI parsing engine in the master device can generate a DSL from the UI (i.e., UI code) description of the master device, and send it to the electronic device.
  • UI i.e., UI code
  • S1003 The electronic device displays a corresponding UI in response to the UI information.
  • the electronic device may first construct a view tree based on the UI information. Specifically, when constructing a view tree, the electronic device can consider the positional relationship between different views, where each view in the view tree represents a functional option or background in the UI, and the view corresponding to the background can also be represented by a viewroot. As shown in Figure 11, view1 corresponds to layer 1, view2 corresponds to layer 2, and view3 corresponds to layer 3. view1, view2, and view3 are all located on viewroot, so view1, view2, and view3 can be used as child nodes of viewroot. And the positions of view1, view2, and view3 do not overlap, then view1, view2, and view3 can be used as the same layer. The electronic device renders each node (view) of the view tree in turn according to the layout of the view tree, so as to obtain a UI.
  • S1004 In response to a first user operation based on the first function option in the UI, the electronic device sends user operation information to the main control device, and the main control device receives the user operation information.
  • the user operation information includes at least the identification information of the first function option and the type of the first user operation, and the type of the first user operation belongs to (or supported by) the first function option User action type.
  • the electronic device A DSL may be generated from the identification information of the first functional option and the type description of the first user operation of the user, and sent to the main control device.
  • the first user operation may also be a text input operation or a voice input operation
  • the user operation information further includes user input information obtained by the electronic device based on the first user operation.
  • the user operation information sent by the electronic device to the main control device may also include information input by the user in the text box.
  • the main control device performs a control operation or a management operation corresponding to the operation information.
  • the main control device After receiving the operation information, the main control device can call the processing function corresponding to the user operation on the first function option, execute the instruction corresponding to the first user operation on the first function option, and perform the operation corresponding to the first function option.
  • the control operation or management operation corresponding to the information.
  • S1006 The main control device sends UI update information to the electronic device, and the electronic device receives the UI information.
  • the UI update information at least includes the change information of the UI after the main control device performs a control operation or a management operation corresponding to the user operation information.
  • the UI update information may only include UI change information after the main control device performs the control operation or management operation corresponding to the user operation information, such as the updated attribute information of the first function option, and may also include the main control device.
  • the overall information of the new UI after the control operation or management operation corresponding to the user operation information is performed, which is not limited in this application.
  • S1007 The electronic device updates the UI in response to the UI update information.
  • the electronic device converts the identification information of the function option and the information of the click operation into a DSL, and sends it to the intelligent switch (i.e. the master device).
  • the smart switch After receiving the DSL, the smart switch parses the DSL, calls the processing function corresponding to the click operation on the function option with the style "button”, and executes the opening corresponding to the click operation on the function option with the style "button” instruction, turn the smart switch on, and update the attribute of the function option with style "button” to "opened”, and update the UI update information containing the attribute of the function option with style "button” to "opened” Convert to DSL, send to electronic device. According to the received DSL, the electronic device updates the attribute of the function option styled as "button” in the UI from “closed” to "opened”.
  • the methods provided by the embodiments of the present application have been introduced from the perspective of the electronic device and the main control device as execution subjects.
  • the electronic device or the main control device may include a hardware structure and/or a software module, and implement the above-mentioned various functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Function. Whether one of the above functions is performed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.
  • An embodiment of the present application further provides an electronic device, as shown in FIG. 12 , including one or more processors 1201 and one or more memories 1202 .
  • One or more computer programs are stored in the memory 1202, and when the one or more computer programs are executed by the processor 1201, make the electronic device execute the cross-device interaction method provided by the embodiments of this application.
  • the electronic device may also include a display screen 1205 for displaying a graphical user interface, such as an interface for a media application.
  • display screen 1205 includes a display panel.
  • the display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • AMOLED organic light-emitting diode
  • FLED flexible light-emitting diode
  • Miniled MicroLed, Micro-oLed
  • quantum dot light-emitting diode quantum dot light emitting diodes, QLED
  • the electronic device may further include a camera 1203 and a microphone 1204 .
  • the electronic device in the embodiment of the present application may further include a speaker, a touch sensor, etc., which is not limited in the embodiment of the present application.
  • This embodiment of the present application further provides a main control device, as shown in FIG. 13 , including one or more processors 1301 and one or more memories 1302 .
  • One or more computer programs are stored in the memory 1302, and when the one or more computer programs are executed by the processor 1301, the master control device executes the cross-device interaction method provided by the embodiments of the present application.
  • the main control device may further include a camera 1303 , a microphone 1304 and a display screen 1305 .
  • the main control device in the embodiment of the present application may further include a speaker, a touch sensor, etc., which is not limited in the embodiment of the present application.
  • connection medium between the above-mentioned processor, memory, camera, microphone, and display screen is not limited in the embodiments of the present application.
  • the processor, the memory, the camera, the microphone, and the display screen may be connected through a bus, and the bus may be divided into an address bus, a data bus, a control bus, and the like.
  • the processor may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, which can implement or
  • a general purpose processor may be a microprocessor or any conventional processor or the like.
  • the steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.
  • the memory may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or may also be a volatile memory (volatile memory), for example Random-access memory (RAM).
  • Memory is, but is not limited to, any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.
  • the memory in this embodiment of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and/or data.
  • the terms “when” or “after” can be interpreted to mean “if” or “after” or “in response to determining" or “in response to detecting ...”.
  • the phrases “in determining" or “if detecting (the stated condition or event)” can be interpreted to mean “if determining" or “in response to determining" or “on detecting (the stated condition or event)” or “in response to the detection of (the stated condition or event)”.
  • the above-mentioned embodiments it may be implemented in whole or in part by software, hardware, firmware or any combination thereof.
  • software it can be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated.
  • the computer may be a general purpose computer, special purpose computer, computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission to another website site, computer, server, or data center is by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media.
  • the usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVD), or semiconductor media (eg, Solid State Disk (SSD)), and the like.
  • magnetic media eg, floppy disks, hard disks, magnetic tapes
  • optical media eg, DVD
  • semiconductor media eg, Solid State Disk (SSD)

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Telephone Function (AREA)

Abstract

本申请涉及终端技术领域，公开了一种跨设备交互方法及设备，用以实现跨设备的业务逻辑和UI分离，降低厂商的开发难度，减轻用户的使用成本，提高数据的安全性。该方法包括：电子设备响应于来自主控设备的用户界面UI信息，显示对应的UI，所述UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；所述电子设备响应于基于所述UI中所述第一功能选项的第一用户操作，向所述主控设备发送用户操作信息，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型，所述用户操作信息用于触发所述主控设备执行与所述用户操作信息对应的控制操作或管理操作。

Description

一种跨设备交互方法及设备
相关申请的交叉引用
本申请要求在2021年04月14日提交中国国家知识产权局、申请号为202110400031.6、申请名称为“一种跨设备交互方法及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及终端技术领域，尤其涉及一种跨设备交互方法及设备。
背景技术
物联网(internet of things，IoT)的发展，为人们的生活提供了极大的便利。目前，用户可以通过终端设备上安装的应用程序(application，APP)，实现对物联网设备的远程控制。以智能家居设备为例，如图1所示，用户可以打开终端设备上安装的智能家居APP，由智能家居APP从云平台上获取关联的智能家居设备的各项数据并显示，也可以通过智能家居APP对智能家居设备下达控制指令，由智能家居APP通过云平台将控制指令发送给智能家居设备，实现对智能家居设备的控制。
然而，当用户购买了多个厂商的物联网设备后，需要在终端设备上安装多个厂商的APP，使用繁琐。厂商面对不同的APP平台(如
Figure PCTCN2022085461-appb-000001
小程序等)时，也需要维护多个APP运维团队，运维成本高，并且物联网设备的数据在公网上传输也容易造成数据泄露与非法搜集。
发明内容
本申请实施例提供一种跨设备交互方法及设备，用以实现跨设备的业务逻辑和UI分离，降低厂商的开发难度，减轻用户的使用成本，提高数据的安全性。
第一方面，本申请实施例提供一种跨设备交互方法，该方法包括：电子设备响应于来自主控设备的用户界面UI信息，显示对应的UI，所述UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；所述电子设备响应于基于所述UI中所述第一功能选项的第一用户操作，向所述主控设备发送用户操作信息，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型，所述用户操作信息用于触发所述主控设备执行与所述用户操作信息对应的控制操作或管理操作。可选的，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。其中主控设备可以是IoT设备等。
在本申请实施例中，业务代码和UI代码全部运行在主控设备中，厂商无需为不同APP平台开发多个APP，可以降低开发难度、节约成本。另外主控设备无需向外暴露业务API，所有交互都通过UI，提高了数据的安全性，同时用户在电子设备上也无需下载不同厂商的APP，仅需一个显示应用即可以实现对不同厂商的主控设备的控制，有利于减轻用户的使用成本。
在一种可能的设计中，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
采用上述设计，可以使用DSL(如JSON格式代码等)在设备间传输UI与事件(如用户操作信息)，有利于设备间对UI与事件的解析和识别。
在一种可能的设计中，所述方法还包括：所述电子设备响应于来自主控设备的UI信息之前，还包括：所述电子设备响应于从待连接设备列表中选取所述主控设备的第二用户操作，与所述主控设备建立通信连接。
上述设计中，主控设备可以在检测到与电子设备建立连接后，向电子设备发送自身的UI信息，有助于实现跨主控设备和电子设备的业务逻辑和UI分离。
在一种可能的设计中，所述电子设备发送用户操作信息之后，还包括：所述电子设备接收来自所述主控设备的UI更新信息，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息；所述电子设备响应于所述UI更新信息，更新所述UI。
上述设计中，在主控设备执行与用户操作信息对应的控制操作或管理操作后，电子设备根据主控设备发送的UI更新信息，将显示的UI更新为主控设备执行对应的控制操作或管理操作后的UI，有利于保持主控设备的业务逻辑与电子设备显示的UI的一致性。
第二方面，本申请实施例提供一种跨设备交互方法，该方法包括：主控设备向电子设备发送用户界面UI信息，所述UI信息对应的UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；所述主控设备接收来自所述电子设备的用户操作信息，所述用户操作信息是所述电子设备基于所述UI中所述第一功能选项的第一用户操作获取到的，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型；所述主控设备执行与所述操作信息所对应的控制操作或管理操作。可选的，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。
在一种可能的设计中，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
在一种可能的设计中，所述主控设备向电子设备发送UI信息，包括：所述主控设备与所述电子设备建立通信连接后，向所述电子设备发送UI信息。
在一种可能的设计中，所述主控设备执行与所述操作信息所对应的控制操作或管理操作之后，还包括：所述主控设备向所述电子设备发送UI更新信息，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息。
第三方面，本申请实施例还提供一种电子设备，所述电子设备包括：存储器、处理器和显示屏；其中，所述处理器与所述存储器和显示屏连接；所述存储器中存储有一个或多个计算机程序，所述一个或多个计算机程序包括指令，当所述指令被所述电子设备执行时，使得所述电子设备执行上述第一方面或者第一方面的任一种可能的设计中所述的方法。
第四方面，本申请实施例还提供一种主控设备，所述主控设备包括：存储器和处理器；其中，所述处理器与所述存储器连接；所述存储器中存储有一个或多个计算机程序，所述一个或多个计算机程序包括指令，当所述指令被所述主控设备执行时，使得所述主控设备执行上述第二方面或者第二方面的任一种可能的设计中所述的方法。
第五方面，本申请实施例还提供一种电子设备，该电子设备中包括能够执行上述第一 方面或者第一方面的任一种可能的设计中方法的功能模块；这些功能模块可以通过硬件实现，也可以通过硬件执行相应的软件实现。
第六方面，本申请实施例还提供一种主控设备，该主控设备中包括能够执行上述第二方面或者第二方面的任一种可能的设计中方法的功能模块；这些功能模块可以通过硬件实现，也可以通过硬件执行相应的软件实现。
第七方面，本申请实施例还提供一种计算机存储介质，包括计算机指令，当所述计算机指令在计算机上运行时，使得所述计算机执行上述第一方面或者第一方面的任一种可能的设计中所述的方法，或执行上述第二方面或者第二方面的任一种可能的设计中所述的方法。
第八方面，本申请实施例还提供一种程序产品，当所述程序产品在计算机上运行时，使得所述计算机执行上述第一方面或者第一方面的任一种可能的设计中所述的方法，或执行上述第二方面或者第二方面的任一种可能的设计中所述的方法。
第九方面，本申请实施例提供的一种芯片，用于从存储器中调用并运行所述存储器中存储的计算机程序，执行上述第一方面或者第一方面的任一种可能的设计的方法，或执行上述第二方面或者第二方面的任一种可能的设计的方法。
上述第二方面至第九方面所能达到的技术效果请参照上述第一方面所能达到的技术效果，这里不再重复赘述。
附图说明
图1为智能家居设备控制方案示意图；
图2为本申请实施例提供的跨设备交互系统的架构示意图；
图3为本申请实施例提供的电子设备结构示意图之一；
图4为本申请实施例提供的电子设备界面示意图之一；
图5为本申请实施例提供的显示应用界面示意图；
图6为本申请实施例提供的view示意图之一；
图7为本申请实施例提供的UI示意图之一；
图8为本申请实施例提供的UI示意图之二；
图9为本申请实施例提供的电子设备和智能开关的架构示意图；
图10为本申请实施例提供的跨设备交互方法示意图；
图11为本申请实施例提供的view示意图之二；
图12为本申请实施例提供的电子设备的结构示意图之二；
图13为本申请实施例提供的主控设备的结构示意图。
具体实施方式
随着智能电灯、智能插座、智能水表等IoT设备的兴起，为人们的生活带来了诸多便利，但是目前用户购买了多个厂商的物联网设备后，需要在终端设备上安装多个厂商的APP，使用繁琐，厂商面对不同的APP平台也需要维护多个APP运维团队，运维成本高，并且IoT设备的数据在公网上传输也容易造成数据泄露与非法搜集。
有鉴于此，本申请实施例提供了一种跨设备交互方案，通过将业务逻辑和用户界面 (user interface，UI)拆分的方式，使得应用可以跨设备显示与交互，用户可以使用手表、平板等任一带屏电子设备控制IoT设备，无需厂商维护多个APP运维团队，用户也无需下载不同厂商的多个APP，降低了厂商开发难度，提高了用户体验。另外，所有交互都通过UI实现，IoT设备也无需对外暴露业务应用程序接口(application programming interface，API)，可以提高IoT设备数据的安全性。
如图2所示为本申请实施例提供的一种可能的跨设备交互系统的架构示意图，该系统包括多个主控设备(如智能开关、智能电灯、智能冰箱等)和电子设备，电子设备可以搭载不同的操作系统，并安装有同一款显示应用，电子设备和主控设备间可以通过WiFi、蓝牙、蜂窝网络等方式进行通信。通过将主控设备的业务逻辑和UI拆分的方式，将主控设备的UI在电子设备上进行显示，在主控设备和电子设备间传输UI和事件(如对UI中功能选项的操作)，可以实现跨主控设备和电子设备的交互，实现对主控设备进行控制。
下面将结合附图，对本申请实施例中的技术方案进行清楚、完整地描述。为了便于本领域技术人员理解，首先对本申请实施例中的部分用语进行解释说明。
1)、主控设备，本申请实施例中的主控设备可以为IoT设备，例如可以为智能电灯、智能插座、智能开关、智能空调、智能冰箱等智能家居设备，也可以是智能红绿灯、智能水表、路侧单元(road side unit，RSU)等其它IoT设备。此外，本申请实施例的主控设备还可以是电视机等设备。需要理解的是，本申请实施例中的主控设备可以为不带屏设备，也可以为带屏设备。
2)、JS对象简谱(java script object notation，JSON)是一种轻量级的数据交换格式。它基于欧洲计算机协会制定的JS规范的一个子集，采用完全独立于编程语言的文本格式来存储和表示数据。简洁和清晰的层次结构使得JSON成为理想的数据交换语言。易于人阅读和编写，同时也易于机器解析和生成，并有效地提升网络传输效率。
3)、计算机领域的事件驱动，所谓事件驱动，简单来说就是基于产生的事件，计算机执行相应的响应操作。从事件角度说，事件驱动程序的基本结构是由一个事件收集器、一个事件发送器和一个事件处理器组成。事件收集器专门负责收集所有事件，包括来自用户操作的事件(如鼠标、键盘事件等)、来自硬件的事件(如时钟事件等)和来自软件的事件(如操作系统、应用程序的逻辑处理产生的事件等)。事件发送器负责将收集器收集到的事件分发到目标对象中。事件处理器做具体的事件响应工作。
4)、电子设备，本申请实施例中的电子设备可以为便携式终端设备，例如手机、平板电脑、便携式计算机、可穿戴式电子设备(如智能手表、智能头盔、智能眼镜、智能手环等)、VR设备、增强现实(augmented reality，AR)设备等。示例性的，便携式终端包括但不限于搭载
Figure PCTCN2022085461-appb-000002
鸿蒙
Figure PCTCN2022085461-appb-000003
或其他操作系统。此外，本申请实施例的电子设备还可以不是便携式终端设备，例如电视、个人计算机(personal computer，PC)等带屏的显示设备。
示例的，如图3所示，为本申请实施例的一种电子设备的硬件结构示意图。具体的，如图3所示，电子设备包括处理器110、外部存储器接口120、内部存储器121、通用串行总线(universal serial bus，USB)接口130、充电管理模块140、电源管理模块141、电池142、天线1、天线2、移动通信模块150、无线通信模块160、音频模块170、传感器模块180、按键190、指示器192、摄像头193、显示屏194、以及用户标识模块(subscriber identification module，SIM)卡接口195等。其中，传感器模块180可以包括压力传感器、 陀螺仪传感器、加速度传感器、指纹传感器、触摸传感器等，音频模块170可以包括扬声器、麦克风、耳机接口等。
处理器110可以包括一个或多个处理单元。例如：处理器110可以包括应用处理器(application processor，AP)、调制解调器(modem)、图形处理器(graphics processing unit，GPU)、图像信号处理器(image signal processor，ISP)、控制器、视频编解码器、数字信号处理器(digital signal processor，DSP)、基带处理器、和/或神经网络处理器(neural-network processing unit，NPU)等。其中，不同的处理单元可以是独立的器件，或者，两个或更多个不同的处理单元也可以集成在一个器件中。
处理器110中还可以设置存储器，用于存储计算机程序和/或数据。在一些实施例中，处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的计算机程序和/或数据。如果处理器110需要再次使用该计算机程序和/或数据，可从所述存储器中直接调用。避免了重复存取，减少了处理器110的等待时间，因而提高了系统的效率。
在一些实施例中，处理器110可以包括一个或多个接口。例如，处理器110包括通用串行总线(universal serial bus，USB)接口130、用户标识模块(subscriber identity module，SIM)接口195。再例如，处理器110还可以包括集成电路(inter-integrated circuit，I2C)接口、集成电路内置音频(inter-integrated circuit sound，I2S)接口、脉冲编码调制(pulse code modulation，PCM)接口、通用异步收发传输器(universal asynchronous receiver/transmitter，UART)接口、移动产业处理器接口(mobile industry processor interface，MIPI)、和/或通用输入输出(general-purpose input/output，GPIO)接口等。
可以理解的是，本申请实施例示意的各模块间的接口连接关系，只是示意性说明，并不构成对电子设备的结构限定。在本申请另一些实施例中，电子设备也可以采用上述实施例中不同的接口连接方式，或多种接口连接方式的组合。
USB接口130是符合USB标准规范的接口，具体可以是Mini USB接口、Micro USB接口、USB Type C接口等。USB接口130可以用于连接充电器为电子设备充电，也可以用于电子设备与外围设备之间传输数据。也可以用于连接耳机，通过耳机播放音频。该接口还可以用于连接其他电子设备，例如增强现实(augmented reality，AR)设备等。
SIM卡接口195用于连接SIM卡。SIM卡可以通过插入SIM卡接口195，或从SIM卡接口195拔出，实现和电子设备的接触和分离。电子设备可以支持2个或N个SIM卡接口，N为大于2的正整数。SIM卡接口195可以支持Nano SIM卡、Micro SIM卡、SIM卡等。同一个SIM卡接口195可以同时插入多张卡。所述多张卡的类型可以相同，也可以不同。SIM卡接口195也可以兼容不同类型的SIM卡。SIM卡接口195也可以兼容外部存储卡。电子设备通过SIM卡和网络交互，实现通话以及数据通信等功能。在一些实施例中，电子设备采用eSIM，即：嵌入式SIM卡。eSIM卡可以嵌在电子设备中，不能和电子设备分离。
充电管理模块140用于从充电器接收充电输入。其中，充电器可以是无线充电器，也可以是有线充电器。在一些有线充电的实施例中，充电管理模块140可以通过USB接口130接收有线充电器的充电输入。在一些无线充电的实施例中，充电管理模块140可以通过电子设备的无线充电线圈接收无线充电输入。充电管理模块140为电池142充电的同时，还可以通过电源管理模块141为电子设备供电。
电源管理模块141用于连接电池142、充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入，为处理器110、内部存储器121、外部存储器、显示屏194、摄像头193和无线通信模块160等供电。电源管理模块141还可以用于监测电池容量、电池循环次数、电池健康状态(漏电、阻抗)等参数。在其他一些实施例中，电源管理模块141也可以设置于处理器110中。在另一些实施例中，电源管理模块141和充电管理模块140也可以设置于同一个器件中。
电子设备的无线通信功能可以通过天线1、天线2、移动通信模块150、无线通信模块160、调制解调器以及基带处理器等实现。
天线1和天线2用于发射和接收电磁波信号。电子设备中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用，以提高天线的利用率。例如：可以将天线1复用为无线局域网的分集天线。在另外一些实施例中，天线可以和调谐开关结合使用。
移动通信模块150可以提供应用在电子设备上的包括2G/3G/4G/5G/(6G等后续演进标准)等制式的无线通信的解决方案。移动通信模块150可以包括至少一个滤波器、开关、功率放大器、低噪声放大器(low noise amplifier，LNA)等。
无线通信模块160包括可以提供应用在电子设备上的包括无线局域网(wireless local area networks，WLAN)(如Wi-Fi网络)、蓝牙(bluetooth，BT)、全球导航卫星系统(global navigation satellite system，GNSS)、调频(frequency modulation，FM)、近距离无线通信技术(near field communication，NFC)、红外技术(infrared，IR)等无线通信的解决方案。
在一些实施例中，电子设备的天线1和移动通信模块150耦合，天线2和无线通信模块160耦合，使得电子设备可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications，GSM)、通用分组无线服务(general packet radio service，GPRS)、码分多址接入(code division multiple access，CDMA)、宽带码分多址(wideband code division multiple access，WCDMA)、时分码分多址(time-division code division multiple access，TD-SCDMA)、长期演进(long term evolution，LTE)、BT、GNSS、WLAN、NFC、FM和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system，GPS)、全球导航卫星系统(global navigation satellite system，GLONASS)、北斗卫星导航系统(beidou navigation satellite system，BDS)、准天顶卫星系统(quasi-zenith satellite system，QZSS)和/或星基增强系统(satellite based augmentation systems，SBAS)。
电子设备通过GPU、显示屏194以及应用处理器等实现显示功能。显示屏194用于显示图像、视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)、有机发光二极管(organic light-emitting diode，OLED)、有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)、柔性发光二极管(flex light-emitting diode，FLED)、Miniled、MicroLed、Micro-oLed、量子点发光二极管(quantum dot light emitting diodes，QLED)等。在一些实施例中，电子设备可以包括1个或N个显示屏194，N为大于1的正整数。
电子设备可以通过ISP、摄像头193、视频编解码器、GPU、显示屏194以及应用处理器等实现拍摄功能。ISP用于处理摄像头193反馈的数据。例如，拍照时，打开快门，光线通过镜头被传递到摄像头感光元件上，光信号转换为电信号，摄像头感光元件将所述电信号传递给ISP处理，转化为肉眼可见的图像。ISP还可以对图像的噪点，亮度，肤色进 行算法优化。ISP还可以对拍摄场景的曝光、色温等参数优化。在一些实施例中，ISP可以设置在摄像头193中。在一些实施例中，电子设备可以包括1个或N个摄像头193，N为大于1的正整数。
外部存储器接口120可以用于连接外部存储卡，例如Micro SD卡，实现扩展电子设备的存储能力。外部存储卡通过外部存储器接口120与处理器110通信，实现数据存储功能。
内部存储器121包括运行内存(memory)和内置存储器。其中，运行内存可以用于存储计算机程序和/或数据等。处理器110通过运行存储在运行内存的计算机程序，从而执行电子设备的各种功能应用以及数据处理。例如，运行内存可以包括高速随机存取存储器。而内置存储器又可以称之为内置外存等，可以用于存储计算机程序和/或数据。例如，内置存储器可以存储操作系统、应用程序等。电子设备通常将内置存储器中的计算机程序和/或数据加载到运行内存后，使得处理器110运行相应的计算机程序和/或数据，实现相应的功能。此外，内部存储器121可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、通用闪存存储器(universal flash storage，UFS)等。
电子设备可以通过音频模块170以及应用处理器等实现音频功能。例如音乐播放、录音等。
按键190包括开机键、音量键等。按键190可以是机械按键。也可以是触摸式按键。电子设备可以接收按键输入，产生与电子设备的用户设置以及功能控制有关的键信号输入。
指示器192可以是指示灯，可以用于指示充电状态、电量变化，也可以用于指示消息、未接来电、通知等。
触摸传感器，也称“触控面板”。触摸传感器可以设置于显示屏194，由触摸传感器与显示屏194组成触摸屏，也称“触控屏”。触摸传感器用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器，以确定触摸事件类型。可以通过显示屏提供与触摸操作相关的视觉输出。在另一些实施例中，触摸传感器也可以设置于电子设备的表面，与显示屏194所处的位置不同。
可以理解的是，本申请实施例示意的结构并不构成对电子设备的具体限定。在本申请另一些实施例中，电子设备可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
另外，应理解，本申请实施例中“至少一个”是指一个或者多个。“多个”是指两个或两个以上。“和/或”，描述关联对象的关联关系，表示可以存在三种关系。例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B的三种情况。其中A、B可以是单数或者复数。字符“/”一般表示前后关联对象是一种“或”的关系。“以下至少一(项)个”或其类似表达，是指的这些项中的任意组合，包括单项(个)或复数项(个)的任意组合。
在本申请中，“示例的”、“在一些实施例中”、“在另一些实施例中”等用于表示作例子、例证或说明。本申请中被描述为“示例”的任何实施例或设计方案不应被解释为比其它实施例或设计方案更优选或更具优势。确切而言，使用示例的一词旨在以具体方式呈现概念。
需要指出的是，本申请实施例中涉及的“第一”、“第二”等词汇，仅用于区分描述的目的，而不能理解为指示或暗示相对重要性，也不能理解为指示或暗示顺序。
下面以主控设备为IoT设备中的智能开关为例，介绍本申请的跨设备交互方法。
以电子设备上支持跨设备交互的应用为显示应用且主控设备为智能开关为例，如图4中的A所示，为电子设备的显示屏上显示的主界面410，当用户需要控制智能开关时，用户可以点击主界面410上的显示应用的图标411，电子设备响应于用户点击显示应用的图标411的操作，在显示屏上显示显示应用的界面。例如：显示应用的界面可以如图4中的B所示的界面420。显示界面上包括3个功能选项：WiFi421、蓝牙422、账号423，这3个功能选项可由控件实现。其中，功能选项WiFi421用于控制显示应用搜索与电子设备处于同一WiFi网络的主控设备，并进入与电子设备处于同一WiFi网络的主控设备的管理界面；功能选项蓝牙422用于控制显示应用搜索位于电子设备蓝牙覆盖范围内的主控设备，并进入位于电子设备蓝牙覆盖范围内的主控设备的管理界面；功能选项账号423用于控制显示应用与显示应用服务器通过蜂窝网络进行通信，从显示应用服务器获取用户个人账号下关联的主控设备，进入用户个人账户下关联的主控设备的管理界面。可以理解的是，显示应用的界面420上还可以包括其它功能选项，例如用于控制退出显示应用的功能选项、配置用户个人账号的功能选项等。
以用户点击如图4中的B所示的界面420中的功能选项WiFi421为例，如图5所示，显示应用(也即电子设备)响应于用户点击功能选项WiFi421的操作，搜索与电子设备处于同一WiFi网络的主控设备，并进入与电子设备处于同一WiFi网络的主控设备的管理界面510，如果智能开关已经在设备列表511中，用户可以直接点击设备列表511中的“智能开关”选项，显示应用即可响应用户点击“智能开关”选项的操作，与智能开关建立WiFi连接。如果智能开关未在主控设备的设备列表511中，用户可以点击搜索按键512，显示应用即可响应用户点击搜索按键512的操作，重新搜索与电子设备处于同一WiFi的主控设备，对设备列表511中的主控设备进行更新。
在智能开关中运行着一个包含UI的代码的服务，示例性的，智能开关中运行的UI代码中包括背景颜色为rgba(30,144,255,0.7)；功能选项(如组件)样式为“按钮”、属性为“已关闭”、支持的(或关联的)操作类型包括“单击”等信息，具体的UI代码可以如下所示：
Figure PCTCN2022085461-appb-000004
Figure PCTCN2022085461-appb-000005
当智能开关确定与电子设备建立通信连接或接收到来自电子设备的UI上报信息时，智能开关中的UI解析引擎可以将智能开关的UI(即UI代码)描述生成特定领域语言(domain specific language，DSL)，并发送到电子设备上。
作为一种示例，智能开关可以将UI代码中的hml、Css、Js部分描述生成JSON格式代码，其中hml和Css部分主要描述了智能开关的UI的界面信息，Js部分主要描述了智能开关的UI中一个或多个功能选项(如组件)支持的操作类型(如支持单击、双击等)。如下所示为将智能开关的上述UI代码描述生成(也即转换成)的JSON格式代码示例。
Figure PCTCN2022085461-appb-000006
电子设备上的显示应用在接收到来自智能开关的UI信息后，绘制智能开关的UI时，可以以视图(view)为单元进行绘制，如UI上一个图层对应一个view，显示应用绘制每 个view的图形，然后将所有view的图形合成一个UI。具体而言，构建view树时，显示应用可以考虑不同的view之间的位置关系。以显示应用接收到的UI信息为上述JSON格式代码为例，显示应用对上述JSON格式代码进行解析，构建的view树如图6所示，view1对应图层1，view1位于背景上，背景可以作为一个单独的view即视图根(viewroot)，view1为viewroot的子节点。其中由上述JSON格式代码可知，view1对应样式(也可以称为类型)为“按钮(button)”、属性为“已关闭”、支持的操作类型为“单击(onclick)”等信息，viewroot对应样式为“背景(div)”、背景色为“512819123”等。
显示应用根据view树，对view树中每个view进行布局渲染，合成一个UI。以viewroot为例，显示应用根据对应样式为“背景(div)”、背景色为“512819123”等，获取对应512819123的颜色，也即rgba(30，144，255，0.7)，设置背景的颜色为rgba(30，144，255，0.7)。其中，rgba中rgb对应红(red，R)、绿(green，G)、蓝(blue，B)，a代表阿尔法通道(alpha)，通常指透明度。如图7所示为显示应用根据UI信息进行布局渲染后，显示的UI。
当用户在电子设备显示的UI上进行单击、双击等操作后，显示应用可以根据操作的位置确定用户操作在UI上所对应的功能选项，并判断用户操作是否为该功能选项支持的操作类型，如果是，显示应用将该功能选项的标识信息和用户操作的类型(如单击、双击等)描述生成DSL，发送至智能开关。
作为一种示例，如图7所示，用户在电子设备显示的UI上对样式为“按钮”的功能选项进行单击操作后，显示应用根据该操作的位置确定该操作在UI上所对应的功能选项为样式为“按钮”的功能选项、单击操作属于该功能选项支持的操作类型，显示应用将该功能选项的标识信息(如id)和用户操作的类型，如“单击操作(如onclick)”描述生成DSL，发送至智能开关。具体的DSL可以如下所示。
{
“id”:“2-onclick”
}
智能开关接收到该DSL后，解析该DSL，并调用对样式为“按钮”的功能选项进行单击操作对应的处理函数，执行对样式为“按钮”的功能选项进行单击操作所对应的打开指令，将智能开关打开，并将样式为“按钮”的功能选项的属性更新为“已打开”。
click(){
api.open()；
this.btn＝“已打开；
}
进一步的，智能开关的UI解析引擎将样式为“按钮”的功能选项更新后的属性信息转化成DSL，并发送给电子设备。如下所示，DSL包含将id为2的样式为“按钮”的功能选项的属性更新为“已打开”的信息。
Figure PCTCN2022085461-appb-000007
如图8所示，显示应用根据接收到的DSL，将UI中样式为“按钮”的功能选项的属性由“已关闭”更新为“已打开”。
在另一种可能的实施中，智能开关将样式为“按钮”的功能选项的属性更新为“已打开”之后，也可以将智能开关更新后UI的整体信息发送给显示应用，显示应用根据智能开关更新后的UI信息，更新UI。
如图9所示为本申请实施例提供的一种智能开关和电子设备的架构示意图，界面布局描述中包括了描述智能开关的UI的界面信息的代码，逻辑代码中包括描述了智能开关的UI中一个或多个功能选项的事件信息(如对功能选项的操作)的代码，通过UI解析引擎(如JS引擎)对界面布局描述中的代码进行解析，智能开关可以获得智能开关的UI的界面信息，智能开关也可以获取逻辑代码中的接口信息(如一个或多个功能选项支持的操作类型)，通过对智能开关的UI的界面信息和智能开关的UI中一个或多个功能选项支持的操作类型进行编码，将其转换成DSL发送给电子设备的显示应用。另外，智能开关也可以将与UI相关的资源，如媒体(音乐、视频、颜色等)、图像(如功能选项的图片等)发送给电子设备的显示应用。
显示应用解析接收到的DSL获得智能开关的UI的界面信息和智能开关的UI中一个或多个功能选项支持的操作类型，通过布局(layout)渲染，显示智能开关的UI。
当显示应用检测到针对显示的UI中某一目标功能选项的用户操作时，如果所述用户操作属于目标功能选项支持的操作类型，将所述用户操作的类型和目标功能选项的标识信息通过编码，转换为DSL后发送给智能开关，智能开关解析收到的DSL，调用对目标功能选项进行所述用户操作对应的处理函数，执行与对目标功能选项执行所述用户操作所对应的指令，并将更新后的目标功能选项的属性信息通过编码，转化成DSL反馈给显示应用，使得电子设备中的显示应用更新显示的UI。
需要理解的是，本申请实施中智能开关所执行的步骤，也可以通过智能开关中的应用实现，如通过智能开关中安装的开关应用实现。
以下实施例介绍通过电子设备和主控设备进行的跨设备交互的可能实现方式。图10示出了本申请以实施例提供的设备交互方法示意图，如图10所示，该方法的流程包括：
S1001：电子设备响应于从待连接设备列表中选取主控设备的第二用户操作，与主控设备建立通信连接。
在一些实施例中，电子设备可以通过WiFi、蓝牙等方式搜索与电子设备处于同一WiFi网络的主控设备，或者搜索位于电子设备蓝牙覆盖范围内的主控设备，也可以登录用户个人的账号通过服务器获取用户个人的账号下关联的主控设备，并可以根据从待连接设备列表选取主控设备的第二用户操作，与用户选取的主控设备建立通信连接。以电子设备搜索与电子设备处于同一WiFi网络的主控设备为例，如图5中的设备列表511为一种可能的待连接设备列表示例。
作为一种示例，如图5所示，用户选取与电子设备处于同一WiFi网络的主控设备(如智能开关)后，电子设备可以通过WiFi网络与用户选取的主控设备建立通信连接。
S1002：所述主控设备向所述电子设备发送UI信息，所述电子设备接收所述UI信息。其中，所述UI信息对应的UI中至少包括第一功能选项(如组件)，所述第一功能选项关联至少一个用户操作类型(如单击、双击等)。
举例来说，当主控设备检测到与电子设备建立通信连接时，主控设备中的UI解析引 擎可以将主控设备的UI(即UI代码)描述生成DSL，并发送到电子设备上。具体实现可以参照上述智能开关向电子设备发送UI信息的实现，不再进行赘述。
S1003：所述电子设备响应于所述UI信息，显示对应的UI。
在一些实施中，电子设备接收到来自主控设备的UI信息后，可以先基于该UI信息构建view树。具体而言，构建view树时，电子设备可以考虑不同的view之间的位置关系，其中view树中每个view表示UI中的一个功能选项或背景，背景对应的view也可以用viewroot表示。如图11所示，view1对应图层1，view2对应图层2，view3对应图层3。view1、view2、view3都位于viewroot上，则view1、view2、view3可以作为viewroot的子节点。且view1、view2、view3所在的位置没有重叠，则view1、view2、view3可作为同一层。电子设备根据view树的布局，对view树的各个节点(view)依次来渲染，即可得到一个UI。
S1004：所述电子设备响应于基于所述UI中所述第一功能选项的第一用户操作，向所述主控设备发送用户操作信息，所述主控设备接收所述用户操作信息。
其中，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型，所述第一用户操作的类型属于所述第一功能选项关联的(或支持的)用户操作类型。
作为一种示例，当用户对UI中的第一功能选项进行第一用户操作时(如单击操作)，如果该第一用户操作的类型属于第一功能选项关联的用户操作类型，则电子设备可以将该第一功能选项的标识信息和用户的第一用户操作的类型描述生成DSL，发送至主控设备。
在一些可能的实施中，第一用户操作还可能是文本输入操作、语音输入操作，所述用户操作信息还包括电子设备基于所述第一用户操作获取到的用户输入信息。
作为一种示例，以第一功能选项为文本框、用户进行文本输入操作为例，电子设备向主控设备发送的用户操作信息中还可以包括用户在文本框中输入的信息。
S1005：所述主控设备执行与所述操作信息所对应的控制操作或管理操作。
主控设备接收到操作信息后，可以调用对第一功能选项进行所述用户操作对应的处理函数，执行与对第一功能选项执行所述第一用户操作所对应的指令，进行与所述操作信息所对应的控制操作或管理操作。
S1006：所述主控设备向所述电子设备发送UI更新信息，所述电子设备接收所述UI信息。
其中，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息。
需要理解的是，在UI更新信息中可以仅包括主控设备执行用户操作信息对应的控制操作或管理操作后UI的变化信息，如第一功能选项更新后的属性信息，还可以包括主控设备执行用户操作信息对应的控制操作或管理操作后新UI的整体信息，本申请对此不作限定。
S1007：所述电子设备响应于所述UI更新信息，更新所述UI。
作为一种示例，参照图7和图8，当用户单击显示的样式为“按钮”的功能选项时，电子设备将该功能选项的标识信息和单击操作的信息转换成DSL，发送至智能开关(即主控设备)。智能开关接收到该DSL后，解析该DSL，并调用对样式为“按钮”的功能选项进行单击操作对应的处理函数，执行对样式为“按钮”的功能选项进行单击操作所对应的 打开指令，将智能开关打开，并将样式为“按钮”的功能选项的属性更新为“已打开”，并将包含将样式为“按钮”的功能选项的属性更新为“已打开”的UI更新信息转换为DSL，发送给电子设备。电子设备根据接收到的DSL，将UI中样式为“按钮”的功能选项的属性由“已关闭”更新为“已打开”。
上述本申请提供的实施例中，从电子设备和主控设备作为执行主体的角度对本申请实施例提供的方法进行了介绍。为了实现上述本申请实施例提供的方法中的各功能，电子设备或主控设备可以包括硬件结构和/或软件模块，以硬件结构、软件模块、或硬件结构加软件模块的形式来实现上述各功能。上述各功能中的某个功能以硬件结构、软件模块、还是硬件结构加软件模块的方式来执行，取决于技术方案的特定应用和设计约束条件。
下面结合附图介绍本申请实施例提供的装置，以实现本申请上述方法实施例。
本申请实施例还提供了一种电子设备，如图12所示，包括一个或多个处理器1201、一个或多个存储器1202。存储器1202中存储有一个或多个计算机程序，当该一个或多个计算机程序被处理器1201执行时，使得电子设备执行本申请实施例提供的跨设备交互方法。
该电子设备还可以包括显示屏1205，用于显示图形用户界面，例如媒体应用的界面。示例的，显示屏1205包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，Miniled，MicroLed，Micro-oLed，量子点发光二极管(quantum dot light emitting diodes，QLED)等。
进一步的，在一些实施例中，该电子设备还可以包括摄像头1203和麦克风1204。
此外，本申请实施例中的电子设备还可以包括扬声器、触摸传感器等，本申请实施例对此不作限定。
本申请实施例还提供了一种主控设备，如图13所示，包括一个或多个处理器1301、一个或多个存储器1302。存储器1302中存储有一个或多个计算机程序，当该一个或多个计算机程序被处理器1301执行时，使得主控设备执行本申请实施例提供的跨设备交互方法。
进一步的，在一些实施例中，该主控设备还可以包括摄像头1303、麦克风1304和显示屏1305。
此外，本申请实施例中的主控设备还可以包括扬声器、触摸传感器等，本申请实施例对此不作限定。
本申请实施例中不限定上述处理器、存储器、摄像头、麦克风和显示屏之间的连接介质。例如，本申请实施例中处理器、存储器、摄像头、麦克风和显示屏之间可以通过总线连接，所述总线可以分为地址总线、数据总线、控制总线等。
在本申请实施例中，处理器可以是通用处理器、数字信号处理器、专用集成电路、现场可编程门阵列或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件，可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。
在本申请实施例中，存储器可以是非易失性存储器，比如硬盘(hard disk drive，HDD)或固态硬盘(solid-state drive，SSD)等，还可以是易失性存储器(volatile memory)，例如随机存取存储器(random-access memory，RAM)。存储器是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其他介质，但不限于此。本申请实施例中的存储器还可以是电路或者其它任意能够实现存储功能的装置，用于存储程序指令和/或数据。
以上实施例中所用，根据上下文，术语“当…时”或“当…后”可以被解释为意思是“如果…”或“在…后”或“响应于确定…”或“响应于检测到…”。类似地，根据上下文，短语“在确定…时”或“如果检测到(所陈述的条件或事件)”可以被解释为意思是“如果确定…”或“响应于确定…”或“在检测到(所陈述的条件或事件)时”或“响应于检测到(所陈述的条件或事件)”。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行所述计算机程序指令时，全部或部分地产生按照本发明实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。在不冲突的情况下，以上各实施例的方案都可以组合使用。
需要指出的是，本专利申请文件的一部分包含受著作权保护的内容。除了对专利局的专利文件或记录的专利文档内容制作副本以外，著作权人保留著作权。

Claims (23)

  1. 一种跨设备交互方法，其特征在于，包括：
    电子设备响应于来自主控设备的用户界面UI信息，显示对应的UI，所述UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；
    所述电子设备响应于基于所述UI中所述第一功能选项的第一用户操作，向所述主控设备发送用户操作信息，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型，所述用户操作信息用于触发所述主控设备执行与所述用户操作信息对应的控制操作或管理操作。
  2. 如权利要求1所述的方法，其特征在于，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。
  3. 如权利要求1或2所述的方法，其特征在于，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
  4. 如权利要求1-3中任一项所述的方法，其特征在于，所述电子设备响应于来自主控设备的UI信息之前，还包括：
    所述电子设备响应于从待连接设备列表中选取所述主控设备的第二用户操作，与所述主控设备建立通信连接。
  5. 如权利要求1-4中任一项所述的方法，其特征在于，所述电子设备发送用户操作信息之后，还包括：
    所述电子设备接收来自所述主控设备的UI更新信息，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息；
    所述电子设备响应于所述UI更新信息，更新所述UI。
  6. 一种跨设备交互方法，其特征在于，包括：
    主控设备向电子设备发送用户界面UI信息，所述UI信息对应的UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；
    所述主控设备接收来自所述电子设备的用户操作信息，所述用户操作信息是所述电子设备基于所述UI中所述第一功能选项的第一用户操作获取到的，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型；
    所述主控设备执行与所述操作信息所对应的控制操作或管理操作。
  7. 如权利要求6所述的方法，其特征在于，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。
  8. 如权利要求6或7所述的方法，其特征在于，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
  9. 如权利要求6-8中任一项所述的方法，其特征在于，所述主控设备向电子设备发送UI信息，包括：
    所述主控设备与所述电子设备建立通信连接后，向所述电子设备发送UI信息。
  10. 如权利要求6-9任一项所述的方法，其特征在于，所述主控设备执行与所述操作信息所对应的控制操作或管理操作之后，还包括：
    所述主控设备向所述电子设备发送UI更新信息，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息。
  11. 一种电子设备，其特征在于，所述电子设备包括：
    存储器、处理器和显示屏，其中所述处理器与所述存储器和显示屏连接；
    以及一个或多个计算机程序，其中所述一个或多个计算机程序被存储在所述存储器中，所述一个或多个计算机程序包括指令，当所述指令被所述电子设备执行时，使得所述电子设备执行以下步骤：
    响应于来自主控设备的用户界面UI信息，显示对应的UI，所述UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；
    响应于基于所述UI中所述第一功能选项的第一用户操作，向所述主控设备发送用户操作信息，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型，所述用户操作信息用于触发所述主控设备执行与所述用户操作信息对应的控制操作或管理操作。
  12. 如权利要求11所述的电子设备，其特征在于，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。
  13. 如权利要求11或12所述的电子设备，其特征在于，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
  14. 如权利要求11-13中任一项所述的电子设备，其特征在于，当所述指令被所述电子设备执行时，使得所述电子设备响应于来自主控设备的UI信息之前，还执行以下步骤：
    响应于从待连接设备列表中选取所述主控设备的第二用户操作，与所述主控设备建立通信连接。
  15. 如权利要求11-14中任一项所述的电子设备，其特征在于，当所述指令被所述电子设备执行时，使得所述电子设备发送用户操作信息之后，还执行以下步骤：
    接收来自所述主控设备的UI更新信息，所述UI更新信息中至少包括所述主控设备执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息；
    响应于所述UI更新信息，更新所述UI。
  16. 一种主控设备，其特征在于，所述主控设备包括：
    存储器和处理器，其中所述处理器与所述存储器连接；
    以及一个或多个计算机程序，其中所述一个或多个计算机程序被存储在所述存储器中，所述一个或多个计算机程序包括指令，当所述指令被所述主控设备执行时，使得所述主控设备执行以下步骤：
    向电子设备发送用户界面UI信息，所述UI信息对应的UI中至少包括第一功能选项，所述第一功能选项关联至少一个用户操作类型；
    接收来自所述电子设备的用户操作信息，所述用户操作信息是所述电子设备基于所述UI中所述第一功能选项的第一用户操作获取到的，所述用户操作信息至少包括所述第一功能选项的标识信息和所述第一用户操作的类型；
    执行与所述操作信息所对应的控制操作或管理操作。
  17. 如权利要求16所述的主控设备，其特征在于，所述用户操作信息还包括基于所述第一用户操作获取到的用户输入信息。
  18. 如权利要求16或17所述的主控设备，其特征在于，所述UI信息和所述用户操作信息为特定领域语言DSL格式的信息。
  19. 如权利要求16-18中任一项所述的主控设备，其特征在于，所述主控设备执行向电 子设备发送UI信息时，具体执行以下步骤：
    与所述电子设备建立通信连接后，向所述电子设备发送UI信息。
  20. 如权利要求16-19中任一项所述的主控设备，其特征在于，当所述指令被所述主控设备执行时，使得所述主控设备执行与所述操作信息所对应的控制操作或管理操作之后，还执行以下步骤：
    向所述电子设备发送UI更新信息，所述UI更新信息中至少包括执行与所述用户操作信息对应的控制操作或管理操作后所述UI的变化信息。
  21. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质包括计算机程序，当所述计算机程序在计算机上运行时，使得所述计算机执行如权利要求1-5中任一项所述的方法，或执行如权利要求6-10中任一项所述的方法。
  22. 一种芯片，其特征在于，所述芯片与存储器耦合，使得所述芯片在运行时调用所述存储器中存储的计算机程序，实现如权利要求1-5中任一项所述的方法，或实现如权利要求6-10中任一项所述的方法。
  23. 一种计算机程序产品，其特征在于，包括计算机程序或指令，当计算机程序或指令被执行时，实现如权利要求1-5中任一项所述的方法，或实现如权利要求6-10中任一项所述的方法。
PCT/CN2022/085461 2021-04-14 2022-04-07 一种跨设备交互方法及设备 WO2022218197A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110400031.6A CN115202546A (zh) 2021-04-14 2021-04-14 一种跨设备交互方法及设备
CN202110400031.6 2021-04-14

Publications (1)

Publication Number Publication Date
WO2022218197A1 true WO2022218197A1 (zh) 2022-10-20

Family

ID=83573723

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/085461 WO2022218197A1 (zh) 2021-04-14 2022-04-07 一种跨设备交互方法及设备

Country Status (2)

Country Link
CN (1) CN115202546A (zh)
WO (1) WO2022218197A1 (zh)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6089459A (en) * 1992-06-16 2000-07-18 Smartdiskette Gmbh Smart diskette device adaptable to receive electronic medium
CN105116734A (zh) * 2015-06-26 2015-12-02 青岛海信移动通信技术股份有限公司 一种智能家电的控制方法及装置、控制终端
CN106375409A (zh) * 2016-08-31 2017-02-01 杭州古北电子科技有限公司 一种自动生成智能设备控制界面的方法
CN111683278A (zh) * 2020-06-22 2020-09-18 深圳创维－Rgb电子有限公司 智能家电设备远程协助方法、装置和移动终端
CN111817930A (zh) * 2020-06-09 2020-10-23 四川虹美智能科技有限公司 设备控制方法、智能家电、智能终端及智能电子设备

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112231025B (zh) * 2019-03-06 2022-12-06 华为终端有限公司 Ui组件显示的方法及电子设备
CN112351412A (zh) * 2019-08-06 2021-02-09 华为技术有限公司 一种内容接续方法、系统及电子设备
CN111523095B (zh) * 2020-03-31 2024-03-15 华为技术有限公司 一种跨设备交互的方法和终端设备
CN111666075B (zh) * 2020-05-30 2022-04-12 荣耀终端有限公司 一种多设备交互的方法及系统

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6089459A (en) * 1992-06-16 2000-07-18 Smartdiskette Gmbh Smart diskette device adaptable to receive electronic medium
CN105116734A (zh) * 2015-06-26 2015-12-02 青岛海信移动通信技术股份有限公司 一种智能家电的控制方法及装置、控制终端
CN106375409A (zh) * 2016-08-31 2017-02-01 杭州古北电子科技有限公司 一种自动生成智能设备控制界面的方法
CN111817930A (zh) * 2020-06-09 2020-10-23 四川虹美智能科技有限公司 设备控制方法、智能家电、智能终端及智能电子设备
CN111683278A (zh) * 2020-06-22 2020-09-18 深圳创维－Rgb电子有限公司 智能家电设备远程协助方法、装置和移动终端

Also Published As

Publication number Publication date
CN115202546A (zh) 2022-10-18

Similar Documents

Publication Publication Date Title
WO2020244492A1 (zh) 一种投屏显示方法及电子设备
WO2020244495A1 (zh) 一种投屏显示方法及电子设备
WO2021052147A1 (zh) 一种数据传输的方法及相关设备
WO2021121052A1 (zh) 一种多屏协同方法、系统及电子设备
US11934352B2 (en) Card rendering method and electronic device
WO2021129253A1 (zh) 显示多窗口的方法、电子设备和系统
WO2021185244A1 (zh) 一种设备交互的方法和电子设备
WO2022100304A1 (zh) 应用内容跨设备流转方法与装置、电子设备
WO2019227450A1 (zh) 应用功能的实现方法及电子设备
WO2022083296A1 (zh) 一种显示方法及电子设备
CN113961157B (zh) 显示交互系统、显示方法及设备
WO2021244452A1 (zh) 一种多设备交互的方法及系统
WO2024001810A1 (zh) 设备交互方法、电子设备及计算机可读存储介质
WO2022111529A1 (zh) 一种应用程序的调试方法及电子设备
WO2022143310A1 (zh) 一种双路投屏的方法及电子设备
WO2021031862A1 (zh) 一种数据处理方法及其装置
WO2022218197A1 (zh) 一种跨设备交互方法及设备
WO2023029983A1 (zh) 一种控件内容的拖拽方法、电子设备及系统
WO2023273766A1 (zh) 一种编译优化方法和装置
WO2022042765A1 (zh) 一种显示方法及电子设备
WO2023020496A1 (zh) 一种显示方法、芯片与电子设备
WO2023179682A1 (zh) 一种设备协同方法
WO2024109481A1 (zh) 窗口控制方法及电子设备
WO2024046062A1 (zh) 一种卡片跨设备交互方法及电子设备
WO2023051036A1 (zh) 加载着色器的方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22787424

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22787424

Country of ref document: EP

Kind code of ref document: A1