CN113273151A - Intelligent device management method, mobile terminal and system - Google Patents

Abstract

The disclosure relates to an intelligent device management method, a mobile terminal and a system. The mobile terminal sends an intelligent equipment binding request to the server; the server sends a recording starting instruction to the mobile terminal; the server sends an operation control instruction to the intelligent equipment to instruct the intelligent equipment to execute at least one operation; the intelligent equipment executes the at least one operation according to the control operation instruction; the mobile terminal records the process of executing the at least one operation by the intelligent equipment, generates a media file and sends the media file to the server; and the server analyzes the media file and judges whether the media file is matched with the control operation instruction, and if the media file is matched with the control operation instruction, the intelligent equipment is associated with the user management account. The operation control instruction can be randomly generated by the server, and the user needs to record the operation execution process of the intelligent device in real time, so that the binding safety can be improved.

Description

Intelligent device management method, mobile terminal and system Technical Field

The disclosed embodiments relate to the field of intelligent terminals, and more particularly, to an intelligent device management method, a mobile terminal and a system.

Background

With the development of intelligent hardware, various intelligent devices (e.g., sound boxes, intelligent lamps, intelligent air conditioners, etc.) gradually enter people's lives. To implement the management and operation of the intelligent device, a binding association relationship between the intelligent device and the user account is established. The main purpose of the binding operation between the intelligent device and the user account is to confirm that the user holds the intelligent device, and establish an association relationship between the user account and the intelligent device on the cloud server side. Once the association relationship between the user account and the intelligent device is established, only the binding user has the right to access and control the intelligent device, so that higher security requirements are bound in the process of establishing the binding relationship between the user and the intelligent device.

In the prior art, a two-dimensional code printed on an intelligent device can be scanned through an application program installed on a mobile terminal and used for controlling the intelligent device, and a related certificate is generated through the two-dimensional code and sent to a cloud server so as to realize binding association between a user account and the intelligent device.

However, with this solution, if the smart device is too small (e.g., a light bulb, a switch, a button, etc.), there may not be enough space to print the two-dimensional code; or if the installation position of the smart device is too high (e.g., ceiling lamp), the user has difficulty in performing the operation of scanning the two-dimensional code. Moreover, the two-dimensional code for binding operation needs to be printed in advance before the intelligent device leaves the factory, and needs to be recorded in the cloud in advance, so that the process is complicated, and the data uniqueness, integrity and confidentiality guarantee difficulty is high. Therefore, such smart device binding techniques have a great disadvantage in terms of security or operability, resulting in difficulty in popularization and application.

Disclosure of Invention

An intelligent device management method, a mobile terminal and a system are described, which can provide a binding scheme with higher safety and simple operation.

In order to achieve the above purpose, the embodiment of the present application adopts the following technical solutions:

in a first aspect, the present application provides a method for intelligent device management. The intelligent device management method comprises the following steps: the method comprises the steps that the mobile terminal sends a first request to a server, wherein the first request comprises a first identifier of the intelligent device; the server responds to the first request, sends a first instruction to the intelligent device and sends a second instruction to the mobile terminal; the intelligent equipment executes at least one operation according to the first instruction; the mobile terminal responds to the second instruction, the recording intelligent equipment executes the process of the at least one operation and generates a recorded first file; the mobile terminal sends a first file to the server; the server determines whether at least one operation executed by the intelligent equipment is matched with the first instruction or not according to the first file; the server associates the first identification of the smart device with the user management account established at the server in response to determining that at least one operation performed by the smart device matches the first instruction.

To realize the management and operation of the intelligent device, a binding association relationship between the intelligent device and the user account is established. The main purpose of the binding operation of the intelligent device and the user account is to confirm that the user holds the intelligent device and establish the association relationship between the user account and the device.

In the method, the intelligent device may dynamically execute the corresponding operation based on the first instruction, and the user needs to record the process of executing the operation of the intelligent device through a camera or a microphone of the mobile terminal. Thus, the method can confirm that the user actually holds the smart device.

Also, the first instruction may be randomly generated by the server according to the operational capability of the smart device. The server can generate various combined operation sequences according to the capacity of the intelligent equipment, and the operation sequences are effective once and can reach quite high safety level.

By the method, the user of the mobile terminal can bind the intelligent equipment with the user account only by recording the process that the intelligent equipment executes at least one operation, and the operation is simple. Moreover, the intelligent device does not need to be pasted with an identification mark (such as a two-dimensional code), and the appearance is more attractive.

In some possible implementations, the first identifier of the smart device may be dynamically assigned by the server, rather than being written into the smart device in the generation process, which may improve the security of the binding process. For example, the mobile terminal sends first information of the intelligent device to the server; the server acquires a first identifier of the intelligent device according to the first information and sends the first identifier to the mobile terminal, wherein the first information is one or any combination of the following parameters: a serial number of the intelligent device, a media access control address.

In some possible implementations, the first identifier of the smart device is randomly generated by the server according to the first information. Moreover, the first identifier is used to uniquely identify the smart device. The first identification distributed by the server can be used for improving the uniqueness, the integrity and the confidentiality of data.

In some possible implementations, the server may randomly generate the first instruction according to an operation capability set of the smart device. The set of operational capabilities of the smart device represents some operational parameters that the smart device can perform, such as variation parameters of the color and brightness of the indicator lights, variation parameters of the timbre, pitch, intensity and duration of the sound, vibration parameters of the vibration motor, or motion pattern parameters. The first instruction may include a sequence of operational instructions (e.g., the sequence of operations is on, off, on). The server may send the instructions in the operation sequence to the smart device in sequence for execution at certain intervals (for example, send instructions for turning on, turning off, and turning on to the smart device in sequence at certain intervals for execution).

In some possible implementations, the at least one operation performed by the smart device according to the first instruction includes one or any combination of the following operations: voice operation, light operation, motion operation.

In some possible implementations, the sound operation includes at least one operation of timbre, pitch, intensity, or duration of sound.

In some possible implementations, the lighting operation includes a color operation or a brightness operation of the light.

In some possible implementations, the motion operation includes a vibration operation or a movement operation.

In some possible implementation manners, the server identifies the image information and the audio information in the first file to acquire the operation behavior characteristics of the intelligent device; and determining whether the operational behavior characteristic matches the second instruction.

In some possible implementations, the mobile terminal may control the smart device to perform the operational event.

In some possible implementations, the mobile terminal records a process of performing at least one operation by the smart device through at least one of a camera or a microphone.

In a second aspect, an embodiment of the present application provides another intelligent device management method, where the method includes: the method comprises the steps that the mobile terminal sends a first request to a server, wherein the first request comprises a first identifier of the intelligent device; the method comprises the steps that the intelligent equipment receives a first instruction sent by a server; responding to the first instruction, and executing at least one operation by the intelligent equipment; the mobile terminal receives a second instruction sent by the server; responding to the second instruction, the mobile terminal records a process that the intelligent equipment executes at least one operation and generates a recorded first file; the mobile terminal sends the first file to the server, and the first file is used by the server for determining whether at least one operation executed by the intelligent equipment is matched with the second instruction; the mobile terminal receives an association success notification from the server, the association success notification indicating: and after determining that at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.

In a third aspect, an embodiment of the present application provides an intelligent device management system, where the system includes the mobile terminal, the intelligent device, and the server according to the first aspect.

In a fourth aspect, an embodiment of the present application provides another intelligent device management system, where the system includes the mobile terminal and the intelligent device related to the second aspect.

In a fifth aspect, an embodiment of the present application provides an intelligent device management method, where the method includes: in response to the first touch input, displaying a first user interface, the first user interface including a first interface element, the first interface element for adding a smart device; receiving a second touch input selecting the first interface element; in response to the second touch input, displaying a second user interface, the second user interface containing an identification of the smart device; receiving a third touch input selecting the smart device identifier; in response to the third touch input, displaying a network configuration interface, wherein the network configuration interface is used for receiving the SSID and/or the password input by the user; the SSID and/or the password are/is sent to the intelligent equipment; starting a camera and recording a process of executing at least one operation by intelligent equipment and generating a first file; sending the first file to a server; and receiving an association success notification sent by the server, wherein the association success notification indicates that the server successfully associates the intelligent device with the user management account.

In a sixth aspect, an embodiment of the present application provides a mobile terminal, including: a touch screen; one or more processors; one or more memories; and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the mobile terminal, cause the mobile terminal to perform the method steps performed by the mobile terminal in the first, second or fifth aspect.

In a seventh aspect, an embodiment of the present application provides a computer program product containing instructions, which, when run on a mobile terminal, causes the mobile terminal to perform the method steps performed by the mobile terminal in the first aspect, the second aspect, or the fifth aspect.

In an eighth aspect, embodiments of the present application provide a computer-readable storage medium, which stores instructions that, when executed on a mobile terminal, cause the mobile terminal to perform the method steps performed by the mobile terminal in the first aspect, the second aspect, or the fifth aspect.

After the intelligent device completes registration, the cloud server can send an instruction to the intelligent device, so that the intelligent device can execute operations such as flashing, color changing or sounding; and finally, the user records the process that the intelligent equipment executes the operation and provides a recording certificate to the server, and the intelligent equipment binding is completed at the cloud server to obtain the complete control authority of the intelligent equipment. After the binding operation is completed, the user can use the management APP to obtain complete device control rights (e.g., control on/off of an air conditioner, an operation mode, a target temperature, a wind speed, etc.).

According to the scheme, the intelligent device can be used for binding the management account number only by the aid of state changes which can be observed by the outside, such as flashing of a lamp, color change of the lamp, icon change, character change, sounding of a buzzer or an observable motion track. The scheme does not need to intervene in the production or sale link of intelligent hardware, and is not limited by the size and the installation position of equipment.

In addition, for technical effects brought by any one of the design manners in the second aspect to the eighth aspect, reference may be made to technical effects brought by different design methods in the first aspect, and details are not described here.

Drawings

Fig. 1 is a schematic diagram of an intelligent device management system provided in an embodiment of the present application;

fig. 2 is a schematic diagram of a smart device registration process provided in an embodiment of the present application;

fig. 3A is a signaling interaction diagram of a smart device binding according to an embodiment of the present application;

fig. 3B is a signaling interaction diagram of another intelligent device binding provided in an embodiment of the present application;

4A, 4B, 4C, 4E, 4F, 4G show user interfaces during adding intelligent devices to the mobile terminal in the embodiment of the present application;

fig. 5, fig. 6, and fig. 7 are schematic diagrams of operations performed by the smart device according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a mobile terminal according to an embodiment of the present application;

fig. 9 is a schematic structural diagram of a communication device according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be described below with reference to the drawings in the embodiments of the present disclosure.

Fig. 1 illustrates an example of a smart device management system 100 to which one or more of the apparatus, methods, systems, and/or computer program products further described herein may be applied. The depicted smart device management system 100 includes one or more smart devices 118. The smart device 118 may receive a control command of the mobile terminal or the server, and perform a corresponding operation according to the control command. The smart devices 118 represent a variety of smart devices, such as the light bulb 102, the switch 104, the socket 106, the sound box 108 in FIG. 1. Hereinafter, smart devices 102, 104, 106, and 108 and other smart home devices (such as televisions, washing machines, fans, air conditioners, security systems, etc.) are collectively referred to as "smart devices". The light bulb 102 may control changes in the light, such as changes in the color and brightness of the light. The switch 104 may control the turning on or off of other smart devices, such as the turning on or off of the light bulb 102. In some implementations, the switch 104 can also detect ambient lighting conditions, thereby controlling the power and/or the light-dark state of one or more light bulbs 102. The receptacle 106 may control the power supply to one or more receptacles (e.g., receive a remote control command to turn the power supply on or off). Speaker 108 may be in voice communication with the user, for example, may receive voice control commands from the user to play songs that are preferred by the user. In some implementations, the speaker 108 may have an integrated voice assistant module that may provide interactive voice dialog or query functionality via a "wake up word" (e.g., "hello, art"). The smart device management system 100 also includes one or more mobile terminals 110, and the mobile terminals 110 may include smartphones, laptop computers, desktop computers, tablet computers, smartwatches, smartbands, and the like.

The smart device management system 100 also includes a wireless router 112. The smart devices can communicate with each other via the wireless router 112 (e.g., the light bulb 102 communicates with the switch 104 through the wireless router 112). Smart devices can also communicate with each other via a connection to a network, such as network 114. Through the network 114, the smart devices are able to communicate with the server 116, as well as with the mobile terminals 110. Of course, the smart device and the mobile terminal 110 may also communicate directly (e.g., via short-range communication technologies such as Wi-Fi, NFC, bluetooth, etc.). In some embodiments, the mobile terminal 110 and the smart device 118 may also communicate using other wireless communication technologies (e.g., ZigBee, infrared). Accordingly, the mobile terminal 110 and the smart device 118 may be connected to the network 114 through other types of smart home gateways (e.g., ZigBee gateways) to communicate with the server 116.

The server 116 can be used as a medium platform to realize communication between the inside of the home and external control equipment, and meet the requirements of remote control, detection and information exchange. The server 116 is associated with a manufacturer or service provider of the smart device. For example, the server 116 may automatically send software updates to the smart device or provide cloud services for the smart device. In some embodiments, the speakers 108 communicate with the server 116 over the network 114. For example, the user issues an instruction to the sound box 108 to "inquire about tickets a to B". The voice interaction system of the speaker 108 collects the voice, reduces noise, recognizes the wakeup word, converts the voice signal into a digital signal through the voice algorithm processing unit and the audio decoding unit, and then uploads the processed digital signal to the server 116. The server 116 will perform speech recognition and semantic understanding, and then call up the information in the ticket reservation database and pass it to the speaker 108. The sound box 108 restores the ticket booking information into a voice signal through the sound effect unit and plays the voice signal.

Examples of network 114 may include a Local Area Network (LAN) and a Wide Area Network (WAN), such as the Internet. The communication network 114 may be implemented using any known network protocol, including various wired or wireless protocols, such as, for example, ethernet, Universal Serial Bus (USB), global system for mobile communications (GSM), Enhanced Data GSM Environment (EDGE), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Long Term Evolution (LTE), fifth generation mobile communications (5G), bluetooth, wireless fidelity (Wi-Fi), or any other suitable communication protocol.

Mobile terminal 110 is capable of communicating over a Wi-Fi network provided by router 112. Mobile terminal 110 is capable of communicating with server 116 over network 114 via router 112. For example, a user may communicate with the server 116 through the mobile terminal 110 over the network 114 to view images captured by a smart camera in the home. Before returning home at night, the user can control the air conditioner to adjust to the appropriate temperature in advance through the mobile terminal 110.

In an embodiment of the application, the smart device may be bound or paired with a user management account registered on the server. Among other things, the server 116 is responsible for managing smart device management accounts for one or more users. The user may perform the binding or pairing operation when the smart device is powered up for the first unpacking or after a reset of the smart device system.

The user-managed account may be associated with a particular user or a group of particular users (e.g., a family or a group). The user can establish a user account through an account management function of an intelligent device management application (for example, an intelligent home APP), apply for a user management account and set a management password, and can request reasonable management permission as required. Also, the user management account can be associated with one or more smart devices. The user management account generally includes a set of data and services necessary for the operation of the smart-home device or the ecosystem of the smart-home device. In some embodiments, the user management account will be described as being associated with a cloud-based service provider having a "cloud server" (e.g., a computing device that utilizes cloud computing technology and large-scale distributed storage technology to meet various application requirements), which may be accessed by the smart device over the network 114, and may be accessed over the network 114 via a mobile terminal (e.g., tablet, smartphone, wearable device). A particular user-managed account is often uniquely identified by the user's email address or mobile phone number, other account designations may also be used (e.g., social networking site accounts).

For smart devices, the purpose of the binding or pairing process is to associate a unique identification of the smart device (e.g., a Media Access Control (MAC) address or other unique identifier of the smart device) with a user management account. After binding or pairing, the user may control the smart device 118 by managing the account. For example, turning on and off a light bulb, adjusting the temperature of an intelligent air conditioner, opening a door or playing music, or according to rules set by a user, stopping or starting a plurality of devices at the same time (for example, after the intelligent device management application is turned on in a leaving mode, all lights and appliances of a home are automatically turned off, or after the intelligent device management application is turned on in a returning mode, the lights, the air conditioner, a water heater and the like are adjusted to appropriate working states). Specific examples of this binding process are discussed in detail below.

The management method of the smart device will be described below by taking the smart device 118 (e.g., the smart device 102 (light bulb)) shown in fig. 1 as an example. To implement the binding between the smart device and the user management account, the smart device may first access the Wi-Fi network created by the router 112. Meanwhile, in order to ensure the validity of the smart device accessed to the smart home management platform (e.g., the server 116), registration authentication is required for the smart device accessed to the management platform through first networking, and fig. 2 shows a signaling interaction diagram for registering the smart device.

Referring to fig. 2, the mobile terminal 110 sends the network SSID and password of the Wi-Fi network to the smart device 118 at step 202.

The user may download and install the management application of the smart device 118 (or the smart home platform application of the cloud service provider of the smart device 118) in the mobile terminal 110. The user sets up the user management account through the management application of the smart device 118. Within the management application, the user may add a new smart device and send the network SSID and password of the Wi-Fi network to the smart device 118 at the beginning of the addition or binding process.

The user first activates the smart device 118 (e.g., connects it to the home power system and powers on). The user can select an option such as "add new device" in the management application or specifically specify the new device model he/she is adding (e.g., bulb, switch, socket, speaker, etc.). After the smart device 118 is selected for addition, the network SSID and password of the Wi-Fi network are transmitted to the smart device 118 in a variety of ways.

In some embodiments, the smart device 118 may establish a temporary Wi-Fi network after power-up, i.e., the smart device 118 is in an Access Point (AP) mode. The mobile terminal 110 is in a Station (STA) mode, and the mobile terminal 110 is connected to a temporary Wi-Fi network of the smart device 118 to form a local area network. The mobile terminal 110 sends the SSID of the Wi-Fi network and the password to the smart device 118 via the temporary Wi-Fi network.

In some embodiments, the mobile terminal 110 may establish a temporary Wi-Fi network, i.e., the mobile terminal 110 is in an Access Point (AP) mode. The smart device 118 is in a Station (STA) mode, and the smart device 118 is connected to a temporary Wi-Fi network of the mobile terminal 110 to form a local area network. The mobile terminal 110 sends the SSID of the Wi-Fi network and the password to the smart device 118 via the temporary Wi-Fi network.

In other embodiments, the smart device 118 listens for all messages in the network after power up. The mobile terminal 110 encodes the SSID and password of the Wi-Fi network into a UDP message, and sends the UDP message through a broadcast packet or a multicast packet, and the intelligent device 118 decodes the UDP message to obtain the correct SSID and password of the Wi-Fi network. Of course, the mobile terminal 110 may also pass the SSID and password of the Wi-Fi network to the smart device 118 via other communication protocols (e.g., NFC, bluetooth, etc. near field communication protocols).

In some embodiments, the mobile terminal 110 may send the encrypted SSID and password of the Wi-Fi network to the smart device 118 in step 202, thereby improving the security of data transmission. After receiving the encrypted SSID and password, the smart device 118 may obtain the SSID and password by decrypting with the corresponding key.

In step 204, the smart device 118 acquires the network SSID and password of the Wi-Fi network and then automatically connects to the router 112 identified by the network SSID of the Wi-Fi network.

At step 206, the mobile terminal 110 requests registration of the smart device 118. In step 206, the mobile terminal 110 may provide the server 116 with the first information of the smart device 118. The first information may be, for example, a serial number of the smart device 118 or a Media Access Control (MAC) address (e.g., an 802.11MAC address). In some embodiments, the smart device 118 may send the first information to the mobile terminal 110 over a temporary Wi-Fi network established by the smart device. In some embodiments, the smart device 118 may also transmit the first information to the mobile terminal 110 via other communication protocols (e.g., NFC, bluetooth, etc. close-range communication protocols).

In response to the request, the server 116 provides the configuration information of the smart device 118 to the mobile terminal 110 at step 208.

In some embodiments, the configuration information may include a first identification of the smart device 118. The first identification may be used to pair the smart device 118 with a user-managed account of the server 116. In some implementations, the first identification is generated by the server 116 based on the first information and may be a series of serial numbers or names that uniquely identify the smart device 118. In other implementations, the first identifier is obtained by the server 116 querying a database storing the correspondence between the first information of the smart device and the configuration information.

In some embodiments, the configuration information may also include a registration code for the server 116 to subsequently verify the identity of the smart device 118. The smart device 118 that completed the first power-up registration will then need to request login to the server 116 using the registration code each time it subsequently reconnects to the network. The configuration information (e.g., the first identification of the smart device) sent from the server 116 is dynamically assigned and therefore is more secure than the serial number or MAC address of the smart device 118 and is less susceptible to leakage and counterfeiting.

At step 210, the mobile terminal 110 sends the configuration information (e.g., the first identification and/or registration code of the smart device 118) to the smart device 118. Wherein the mobile terminal 110 may transmit the configuration information to the smart device 118 in an encrypted manner.

At step 212, the smart device 118 initiates registration with the server 116. After the smart device 118 is able to establish a communication connection with the server 116, the configuration information (e.g., the first identification and/or registration code of the smart device 118) is sent to the server 116 and a communication key is negotiated to encrypt future communications between the smart device 118 and the server 116.

Therefore, the intelligent device 118 completes the step of accessing the Wi-Fi network, establishes communication connection with the server through the Wi-Fi network, and lays a foundation for subsequently receiving the operation instruction of the server.

The following describes a specific smart device binding process. Fig. 3A illustrates a signaling interaction diagram for smart device binding according to some embodiments of the present application.

Referring to fig. 3A, at step 342, the mobile terminal 110 sends a first request to the server 116 for binding of the smart device 118. Wherein the first request may carry a first identification of the smart device 118. The first identifier may be dynamically generated by the server 116 and, therefore, not easily compromised and falsified. Optionally, prior to step 342, the mobile terminal 110 may send the first information of the smart device 118 to the server 116. The server 116 obtains a first identifier of the smart device 118 according to first information and sends the first identifier to the mobile terminal 110, wherein the first information includes a serial number or a media access control address of the smart device 118.

In response to the first request, server 116 sends a second instruction to mobile terminal 110, which instructs mobile terminal 110 to perform a recording operation (e.g., recording of audio or video), at step 344. Specifically, the server 116 may send a start recording instruction to the mobile terminal 110, where the start recording instruction instructs the mobile terminal to start a camera and/or a microphone to record an operation event of the smart device 118 (for example, record a brightness change, a color change, a sound change, or a motion change of the smart device 118).

In step 346, mobile terminal 110 initiates a recording operation in response to the second instruction. In some embodiments, the mobile terminal 110 receives a second instruction (e.g., a video recording start notification instruction) from the server 116, automatically starts a camera to capture an image and starts a microphone to acquire audio data, obtains the final video recording data, and stores the final video recording data in the first file. In some embodiments, the mobile terminal 110 receives a second instruction (e.g., a record start notification) from the server 116 to display a record prompt to start the intelligent device 118. The user activates the video recording function in the camera or video recorder application according to the video recording prompt, and the activation of the function may be regarded as inputting a video recording activation instruction, which is generally inputted (for example, clicking a video recording button displayed on a screen) by the user through an input device (physical keyboard, button, or touch screen) of the mobile terminal 110.

At step 348, the server 116 sends a first instruction, which may be a sequence of operations, to the smart device 118 instructing the smart device to perform at least one operation. In some embodiments, the server 116 generates the first instructions according to the operational capabilities of the smart device 118. For example, for the light bulb 102, it may exhibit variations in lighting color (red, yellow, green), brightness (brightness level 1, brightness level 2, brightness level 3), etc., and its operational capability may be expressed as { red, yellow, green, brightness level 1, brightness level 2, brightness level 3 }. The first command may comprise a sequence of operations that may be a sequence of lights on and off. For example, a plurality of operation sequences may be formed in a combination of the arrangement of the light-on and light-off control operations. For example, the operational sequence 1: light, extinguish and light.

At step 350, the smart device 118 performs at least one operation according to the first instructions. In some embodiments, each of the at least one operation comprises one or any combination of the following: a sound adjustment operation, a light adjustment operation, or a motion operation. Wherein the sound adjusting operation comprises one or any combination of the following operations: tone color adjustment, tone intensity adjustment, or duration adjustment. The lighting operation includes color adjustment or brightness adjustment of the light. The motion operation includes vibration or movement. For example, the bulb 102 performs operations of lighting on, off, and on according to the operation sequence 1.

At step 352, mobile terminal 110 sends the first file to server 116.

At step 354, the server 116 determines whether at least one operation performed by the smart device matches the first instruction based on the first file. In some embodiments, the server 116 identifies the image information and the audio information in the first file to obtain operational behavior characteristics of the smart device 118; it is then determined whether the operational behavior feature matches the first instruction.

At step 356, the server 116 associates the smart device with the user-managed account in response to determining that the at least one operation performed by the smart device 118 matches the first instruction. In some embodiments, the server 116 associates the first identification of the smart device 118 with a user management account established at the server.

At step 358, server 116 sends an association success notification to mobile terminal 110. Thereafter, the mobile terminal may control the smart device to perform an operation event (e.g., control the smart device to turn on or off, adjust the brightness of the light of the smart device, etc. through the smart device management application).

Fig. 3B illustrates a signaling interaction diagram for smart device binding according to some embodiments of the present application. At step 302, the mobile terminal 110 sends a first request to the server 116 for binding of the smart device 118. Wherein the first request may carry a first identification of the smart device 118 (e.g., the first identification of the smart device 118 mentioned above). The first identifier may be dynamically generated by the server 116 and, therefore, not easily compromised and falsified.

Of course, the mobile terminal 110 may also send the aforementioned information such as the serial number or MAC (e.g., 802.11MAC address) address of the smart device to the server 116 to bind the smart device 118 in step 302.

In some embodiments, step 302 may be preceded by at least one of the following steps:

(1) the mobile terminal 110 displays a first user interface (e.g., the user interface 402 shown in fig. 4A) in response to the first touch input. The first user interface includes a smart sweeper 406 and a smart speaker 408. The first user interface may also contain a smart device add interface element (plus sign icon in the upper right corner of fig. 4A). For example, a user opens the user interface 402 as shown in fig. 4A through the management application of the smart device 118 (or the smart home platform application of the smart device 118 cloud service provider);

(2) receiving a second touch input for selecting the intelligent equipment adding interface element; in response to the second touch input, a second user interface is displayed, the second user interface containing a list of smart devices containing an identification of the first smart device (e.g., an identification of a nearby smart device to be bound, "smart bulb a"). For example, after the user clicks the add smart device button 404 (plus sign icon in the upper right corner of fig. 4A), a second user interface 410 as shown in fig. 4B is displayed, and the second user interface 410 contains the smart bulb a412 to be bound. The list of smart devices displayed by the second user interface may include one or more smart devices. In some implementations, the second user interface also includes a "scan devices" interface element, and the user can select a "scan devices" button to discover surrounding bindable devices. In some implementations, the second interface also includes a "manually add device" interface element that the user can select to add the smart device by selecting a "manually add" button.

(3) Receiving a third touch input selecting the first smart device; and responding to the third touch input, and displaying a network configuration interface, wherein the network configuration interface is used for receiving the SSID and/or the password input by the user. Similar to the method of communicating SSIDs and passwords described in step 202 of fig. 2, a user may enter/select the SSID and/or password of their Wi-Fi network, which is then sent to the smart device 118, e.g., the user enters the SSID of their Wi-Fi network via the interface in fig. 4C: HomeNetwork 416 and password 418; in some implementations, the mobile terminal stores an SSID of a Wi-Fi network, and the user may directly select a network SSID that the user wants to send to the smart device 118 on the user interface, i.e., send the corresponding SSID and password to the smart device.

(4) A recording interface, such as recording interface 420 shown in fig. 4D, is displayed. The recording interface may include a recording box 422 and may also include a prompt 424 for prompting to record the smart device 118 and for prompting the user to point the camera at the smart device 118;

(5) detecting whether the camera is aligned to the smart device 118, wherein the mobile terminal 110 may perform image content recognition on the image captured by the camera, determine whether the currently captured image content matches with the smart device 118 of a certain model selected by the user, and if so, execute step 302. In some implementations, the mobile terminal 110 may perform image content recognition on the image captured by the camera, extract image features to describe the visual content of the image, such as color, shape, texture, and the like, and if the extracted image features match features of the smart device 118 stored in advance by the mobile terminal 110, step 302 is performed. In other implementations, the mobile terminal 110 may send the image captured by the camera to an image recognition server for image content recognition, where the image recognition server extracts image features to describe visual content of the image, such as color, shape, texture, and the like, and if the extracted image features match features of the smart device 118 stored in advance by the image recognition server, step 302 is executed.

At step 304, the server 116 determines whether the smart device is unbound. If the smart device is not bound, step 306 is performed. The smart device must have completed registration (e.g., the registration process shown in fig. 2) and not yet bound to request binding. If the smart device 118 has bound to a certain user management account, the mobile terminal 110 is directly notified of the binding failure.

At step 306, the server 116 sends a second command to the mobile terminal 110, informing the mobile terminal 110 to start the operation process of the recording intelligent device 118. Specifically, the server 116 may send a recording start instruction to the mobile terminal 110, where the recording start instruction instructs the mobile terminal to start a camera and/or a microphone to record an operation event of the smart device.

In some embodiments, the server 116 may also notify the mobile terminal 110 to obtain other operational information of the smart device 118 in step 304. For example, the mobile terminal 110 acquires the bluetooth broadcast information of the smart device 118 through bluetooth, or acquires the specific information of the smart device 118 through NFC.

At step 308, the mobile terminal 110 begins the process of recording at least one action by the smart device 118.

In some embodiments, the mobile terminal 110 receives the video recording start notification from the server 116 in step 306, automatically starts a camera to capture image frames and starts a microphone to acquire audio data, so as to obtain the final video recording data. In other embodiments, step 306 is optional, and mobile terminal 110 may perform the recording process directly after step 302, i.e., without receiving a notification sent by server 116 to initiate recording. In some implementations, mobile terminal 110 displays a record start prompt for smart device 118 after receiving the record start notification from server 116 in step 306. The user activates the video recording function in the camera or video recorder application according to the video recording prompt, and the activation of the function may be regarded as inputting a video recording activation instruction, which is generally inputted (for example, clicking a video recording button displayed on a screen) by the user through an input device (physical keyboard, button, or touch screen) of the mobile terminal 110.

At step 310, the server 116 generates a first instruction according to the operational capability of the smart device 118, the first instruction instructing the smart device 118 to perform at least one operation.

In some embodiments, the server 116 queries the smart device 118 for the device model number based on the first identification of the smart device 118. The server 116 obtains a set of operational capabilities of the smart device 118 that matches the device model of the smart device 118 and generates a first instruction based on the set of operational capabilities of the smart device 118.

In other embodiments, the server 116 maintains a first instruction that matches the set of operational capabilities of the smart device 118. The server 116 may query for a first instruction matching the set of operational capabilities of the smart device 118 and send the first instruction to the smart device 118. The first instruction may include at least one operation instruction during the binding operation, each operation instruction corresponding to an operation (e.g., a brightness adjustment instruction corresponding to a brightness adjustment operation, a color adjustment instruction corresponding to a color adjustment operation, a sound adjustment instruction corresponding to a sound adjustment operation). The first instruction may be a sequence of operations instructing the smart device to perform at least one operation.

In some embodiments, the set of operational capabilities indicates a set of control operations that the smart device 118 is capable of performing. For example, the set of operational capabilities of the smart device 118 includes parameters of variations in lighting color and brightness (e.g., the color displayable by the smart device 118 includes 3 colors: red, yellow, and green, the brightness variation may present 3 different brightness levels, etc., and then the set of operational capabilities of the smart device 118 may be represented as { red, yellow, and green, brightness level 1, brightness level 2, and brightness level 3 }); the set of operational capabilities of the smart device 118 may include parameters of variation of the timbre, pitch, intensity and duration of sound, and of course may also include parameters of vibration of the vibration motor of the smart device 118, in addition to the parameters of variation of the color and brightness of the indicator lights. The set of operational capabilities of the smart device 118 may also be limited to only the motion pattern parameters of the device 118. The server may query a particular database to obtain the capability sets of the smart devices 118, where different smart device 118 models and their corresponding sets of operational capabilities are recorded.

In some embodiments, it is directed to smart devices having indicator lights or other light capabilities. For example, the light bulb 102 may illuminate, a power indicator light for the sound box 108 may illuminate, and the switch 104 or the socket 106 may also have an indicator light. The operation sequence may be a sequence of on/off commands for a set of lights. For example, a plurality of operation sequences may be formed in a combination of the arrangement of the light-on and light-off control operations. For example, the operational sequence 1: bright, go out and bright; operation sequence 2: light, extinguish, light and extinguish. The length of the display time of the on/off state may also be added as a parameter to the operation sequence, for example, the operation sequence 3: bright (1 second), off (2 seconds), bright (1 second); operation sequence 4: bright (1 second), off (1 second), bright (2 seconds), bright (1 second); and (5) an operation sequence: bright (2 seconds), off (2 seconds), bright (1 second), off (1 second). As shown in fig. 5, the bulb includes a bulb 502 and a screw-in base 504. In response to the sequence of operations, the bulb 502 may produce a light intensity change.

In other embodiments, the smart device is configured to have an indicator light or other lighting capability. The sequence of operations may be a color change of a set of lights, for example, the indicator lights of the smart device 118 may appear in four different colors, i.e., red, green, blue, yellow. Various operation sequences can be formed according to the arrangement and combination of the light colors. For example, the operational sequence 1: red, green, blue, yellow; operation sequence 2: red, green, yellow, blue; operation sequence 3: red, yellow, blue, green. It is also possible to add the duration of each light color display as a parameter to the operation sequence, for example, operation sequence 4: red (display 2 seconds), yellow (display 1 second), blue (display 3 seconds), green (display 2 seconds); and (5) an operation sequence: red (display for 1 second), yellow (display for 3 seconds), blue (display for 3 seconds), green (display for 2 seconds); operation sequence 6: red (display 2 seconds), yellow (display 2 seconds), blue (display 3 seconds), green (display 2 seconds). As shown in fig. 5, the light bulb 502 may produce a light color change in response to a sequence of operations.

In some embodiments, for smart devices with speakers. The sequence of operations may indicate a change in sound for a group of speakers, e.g., the speakers of the smart device 118 may emit sounds of different timbre, pitch, intensity, and duration. Various operation sequences can be formed according to different combinations of tone color, tone, sound intensity and sound length. For example, the sequence of operations: frequency 1(10MHz for 2 seconds), frequency (12MHz for 3 seconds), frequency (13MHz for 2 seconds), frequency (15MHz for 2 seconds). In some implementations, the screw-in base 504 of the light bulb in fig. 5 can include a speaker that can perform the sound operations indicated by the sequence of operations described above, and the user can also hear the sound played by the light bulb.

In some embodiments, for smart devices having a vibration motor or electric machine. The sequence of operations may indicate a set of motions, for example, the smart device 118 having a vibration motor, may simulate a vibration feedback, and cause the smart device 118 to perform the vibration motion. The vibration motor may form a variety of operating sequences in different combinations of vibration frequency and amplitude. For example, the sequence of operations: frequency 1 (duration 2 seconds), frequency 2 (duration 3 seconds), frequency 3 (duration 2 seconds), frequency 4 (duration 2 seconds), wherein frequency 1, frequency 2, frequency 3, frequency 4 represent four different vibration frequencies. As another example, the smart device 118 is a smart sweeper with mobility capability that is free to move across the floor. The sequence of operations may comprise a motion trajectory, for example the sequence of operations 1 may be: the intelligent sweeper in fig. 6 can perform corresponding moving operation according to the arrow shown in the figure by moving 30 cm forwards, 10 cm leftwards and 10 cm backwards. The operation sequence 2 may be a movement in an S-shaped curve for ten seconds, and an example of a trajectory of the S-shaped curve movement is also shown in fig. 6.

In some embodiments, for smart devices with display capabilities. The sequence of operations may indicate a change in an image of the display screen of the smart device 118. For example, the smart device 118 is a speaker with a display screen that may display characters, pictures, or other patterns. The sequence of operations may include displaying at least one pattern, such as: a string of random serial numbers, a picture, a string of words, etc. The sequence of operations may include displaying a combination of at least one pattern and sound. For example, sound box 700 in fig. 7 may display a fish pattern in response to a sequence of operations. In some implementations, the smart device 118 may further generate and display a two-dimensional code according to the first instruction received in step 308, and the mobile terminal 110 may scan the two-dimensional code displayed by the smart device 118 to perform a binding operation. Of course, there are various combinations of operation sequences, which are not listed here.

In some embodiments, the sequence of operations may be randomly generated by the server 116. For a smart socket device, the sequence of operations may correspond to a sequence of on and off commands for a set of power indicator lights. For example, the command sequence includes 16 on/off control operations, and the combination of the state command sequence changes can reach 65536 types (i.e. 2 types)¹⁶And power 16 of 2). For the smart sockets to be paired, the operation sequence is effective once, and a quite high safety level can be achieved.

At step 312, the server 116 sends a first instruction to the smart device 118. In some implementations, the server 116 issues instructions to the smart device 118 for execution in sequence of operations.

At step 314, the smart device 118 executes the first instruction. As described above, the smart device 118 may perform a set of operations such as the operation of changing the brightness of the light, the color, and the operation of playing the sound shown in fig. 5, the operation of moving shown in fig. 6, and the operation of displaying shown in fig. 7.

At step 316, the smart device 118 returns the first instruction execution result to the server 116 to notify the server 116 that the first instruction has been executed. In some implementations, the server 116 sends the instructions to the smart device 118 in sequence for execution according to the operation sequence, the smart device returns the execution result of each instruction to the server 116, and after the server 116 confirms that the instruction is successfully executed, the server sends the next instruction according to the interval T.

Server 116 notifies mobile terminal 110 to submit the recorded file at step 318. For example, in response to receiving the first instruction execution result transmitted in step 316, the server 116 sends a transfer media file notification to the mobile terminal 110.

At step 320, mobile terminal 110 stops recording.

In step 322, mobile terminal 110 sends the first file to server 116. In some embodiments, the first file may be a recorded media file, and the first file may include image and/or audio data. In some implementations, the mobile terminal 110 may also transmit feature information extracted from the recorded media file, such as image feature information such as color, shape, and texture, or sound feature information such as sound frequency, sound intensity, and the like.

At step 324, the server 116 may analyze the image and/or audio data contained in the first file to extract operational behavior characteristics that reflect a progress of the smart device 118 performing at least one operation indicated by the first instruction. Analyzing the image may include analyzing image features extracted from the image captured by the camera of mobile terminal 110 to describe the visual content of the image, such as color, shape, texture, etc., each feature having multiple ways of description. Each feature computationally describes the visual content of the image from a different perspective. The server 116 performs speech recognition to extract useful features from the audio data, and the recognition method includes a template matching method or an artificial intelligence network (ANN) method, etc. The server 116 may also implement image and/or audio data analysis with machine learning and deep learning.

At step 326, the server 116 determines whether the operational behavior characteristics of the smart device 118 match the first instructions of step 308. For example, the server 116 may analyze the image and compare image features to standard features that should be present in the image. For example, if the image sequence and generated operational mode match confidence is below a threshold, mobile terminal 110 may be notified of a binding failure. If the image sequence and the sequence of operational instructions match with a confidence level above the threshold, step 328 is performed.

In some embodiments, when the sequence of operations includes a set of light-on and light-off control instructions, the server 116 parses the media file to determine whether the smart device 118 has undergone a light-on and light-off change indicated in the sequence of operations; in some embodiments, when the operational sequence includes a set of color control instructions for the light, the server 116 parses the media file to determine whether the smart device 118 has undergone a change in the color of the light indicated in the operational sequence; in some embodiments, when the sequence of operations includes a set of sound control instructions, the server 116 parses the media file to determine whether the smart device 118 emitted the sound indicated in the sequence of operations; in some embodiments, when the sequence of operations includes a set of motion control instructions, the server 116 parses the media file to determine whether the smart device 118 made a corresponding motion (e.g., vibration or movement) in the sequence of operations; in some embodiments, when the sequence of operations includes a set of display control instructions, the server 116 parses the media file to determine whether the display device of the smart device 118 displays the corresponding pattern content (e.g., the fish shown in FIG. 7). Of course, the operation sequence may be a combination of at least one of the above first instructions, such as a combination of a light color and a brightness, or a combination of a screen display control instruction and a sound control instruction.

At step 328, the server 116 may associate the first identification of the smart device 118 with the user management account. After the association is successful, the user may be allowed to manage the account to access the smart device 118, for example, allowing the user to manage remote control of the account (e.g., remotely turn on the smart device 118 while the company is working), or receive a message notification from the smart device 118 (e.g., when the company is working, the outdoor weather sensing system on the balcony monitors outdoor weather conditions in real time, and rain and snow weather may automatically close a window in conjunction with a window pusher and send a message notification to the user).

Steps 324, 326 and 328 require processing by the server 116 and the user may need to wait for the server 116 processing to complete. In some embodiments, after step 320, mobile terminal 110 may display progress interface 426 shown in fig. 4E. Progress interface 426 may include a current operational progress 428 (progress 64 in FIG. 4E), and may also include a prompt 430 for prompting the user to ensure that the network is in good condition.

At step 330, the server 116 notifies the mobile terminal 110 of the successful binding of the smart device 118. Mobile terminal 110 may display connection success interface 432 shown in fig. 4F. The connection success interface 432 may include a success indication 434. The user may also select a location category (e.g., primary, secondary, living room, kitchen, bathroom, etc.) for the smart device 118 at the connection success interface 432. The mobile terminal 110 may then display the corresponding smart device 118 in the corresponding location category.

After step 330, in response to the user's operation of viewing the list of smart devices, mobile terminal 110 displays a user interface as shown in fig. 4G. Fig. 4G shows a user interface 436 that includes icons 442 for newly added smart devices 118 in addition to icons for a smart sweeper 438 and a smart sound box 440. Thereafter, the user may control the smart device 118 through a management application of the smart device 118 (or a smart home platform application of a cloud service provider of the smart device 118). For example, the light bulb 102 is controlled to be turned on or off, or the sound box 108 is controlled to play favorite music or inquire weather conditions through voice commands.

Fig. 3A and 3B illustrate in detail a flow chart containing the binding operations performed by the mobile terminal 110, the smart device 118, and the server 116. The structure of the mobile terminal 110, the smart device 118, and the server 116 that perform the above-described processes will now be described continuously.

Fig. 8 is a schematic structural diagram of a mobile terminal 110 according to an embodiment of the present application, where the mobile terminal 110 includes: radio Frequency (RF) circuitry 810, a power supply 820, a processor 830, a memory 840, an input unit 850, a display unit 860, a sensor 870, audio circuitry 880, and a wireless-fidelity (Wi-Fi) module 890. Those skilled in the art will appreciate that the handset configuration shown in fig. 8 is not intended to be limiting and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

The following describes each component of the mobile terminal 110 in detail with reference to fig. 8:

the RF circuit 810 may be used for transmitting and receiving information or for receiving and transmitting signals during a call. For example: RF circuitry 810 may send downlink data received from the base station to processor 830 for processing and may send uplink data to the base station.

In general, RF circuits include, but are not limited to, an RF chip, an antenna, at least one amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, a radio frequency switch, and the like. In addition, the RF circuit 810 may also communicate wirelessly with networks and other devices. The wireless communication may use any communication standard or protocol, including but not limited to global system for mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Message Service (SMS), etc.

The memory 840 may be used to store software programs and modules, and the processor 830 may execute various functional applications and data processing of the mobile terminal 110 by operating the software programs and modules stored in the memory 840. The memory 840 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the mobile terminal 110, and the like. Further, the memory 840 may include high speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. The memory 840 may also store a knowledge base, a tag base, and an algorithm base.

The input unit 850 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal 110. Specifically, the input unit 850 may include a touch panel 851 and other input devices 852. The touch panel 851, also referred to as a touch screen, can collect touch operations by a user (such as operations by the user on the touch panel 851 or near the touch panel 851 using any suitable object or accessory such as a finger, a stylus, etc.) and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 851 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 830, and can receive and execute commands sent by the processor 830. In addition, the touch panel 851 may be implemented by various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 850 may include other input devices 852 in addition to the touch panel 851. In particular, other input devices 852 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 860 may be used to display information input by a user or information provided to the user and various menus of the cellular phone 800. The display unit 860 may include a display panel 861, and the display panel 861 may be configured, optionally, in the form of a Liquid Crystal Display (LCD), an electromechanical laser display (OLED), or the like. Further, the touch panel 851 may cover the display panel 861, and when the touch panel 851 detects a touch operation on or near the touch panel 851, the touch operation is transmitted to the processor 830 to determine the type of the touch event, and then the processor 830 provides a corresponding visual output on the display panel 861 according to the type of the touch event. Although in fig. 8, the touch panel 851 and the display panel 861 are two separate components to implement the input and output functions of the mobile phone 800, in some embodiments, the touch panel 851 and the display panel 861 may be integrated to implement the input and output functions of the mobile terminal 110.

The mobile terminal 110 may also include at least one sensor 870, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 861 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 861 and/or backlight when the mobile terminal 110 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), and the like. The mobile terminal 110 may further include other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which are not described herein again.

The audio circuitry 880, speaker 881, and microphone 882 may provide an audio interface between the user and the mobile terminal 110. The audio circuit 880 may transmit the electrical signal converted from the received audio data to the speaker 881, and the electrical signal is converted into a sound signal by the speaker 881 and output; on the other hand, the microphone 882 converts the collected sound signals into electrical signals, which are received by the audio circuitry 880 for conversion into audio data, which is then output to the RF circuitry 810 for transmission to, for example, another cell phone, or to the memory 840 for further processing.

Wi-Fi belongs to short-range wireless transmission technology, and the mobile terminal 110 can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the Wi-Fi module 890, and provides wireless broadband Internet access for the user. Although fig. 8 shows the Wi-Fi module 890, it is understood that it does not belong to the essential constitution of the mobile terminal 110, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 830 is a control center of the mobile terminal 110, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the mobile terminal 110 and processes data by operating or executing software programs and/or modules stored in the memory 840 and calling data stored in the memory 840, thereby implementing various services based on the mobile phone. Optionally, processor 830 may include one or more processing units; preferably, the processor 830 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 830.

In the embodiment of the present application, the processor 830 can execute the program instructions stored in the memory 840 to implement the method shown in the above embodiment. For example, program instructions for executing the methods shown in fig. 2 or fig. 3A, 3B described above on the side of mobile terminal 110 are stored in memory 840, and processor 830, when executing the program instructions, causes mobile terminal 110 to perform the method steps performed by mobile terminal 110 in fig. 2, 3A, or 3B. For example, steps 202, 206, 208, 210 in fig. 2, steps 342, 346, 352, 358 shown in fig. 3A, and steps 302, 308, 320, 322, and 330 shown in fig. 3B. The camera in the mobile terminal 110 is used for taking pictures and recording videos for the intelligent device, and the microphone is used for acquiring sound information of the intelligent device. The mobile terminal 110 may also encode the captured image through a video encoder and encode the sound collected by the microphone through an audio encoder. The memory 840 is also used for storing recorded audio-video data. The display unit 860 is used to display the user interfaces shown in fig. 4A to 4G.

Mobile terminal 110 also includes a power supply 820 (e.g., a battery) for powering the various components, which may preferably be logically coupled to processor 830 via a power management system that may enable managing charging, discharging, and power consumption via the power management system.

Although not shown, the mobile terminal 110 may further include a camera, a bluetooth module, and the like, which are not described in detail herein.

Fig. 9 shows a schematic diagram of a communication device, which may be the smart device 118 or the server 116 in fig. 1. As shown in fig. 9, the communication device may include a communication module 910, a processor 920, and a memory 930.

The memory 930 may be used to store data, software programs, and software modules, and may be a volatile memory (volatile memory), such as a random-access memory (RAM); or a non-volatile memory (non-volatile memory), such as a read-only memory (ROM), a flash memory (flash memory), a Hard Disk Drive (HDD) or a solid-state drive (SSD), or a combination of the above types of memories. Specifically, the memory 930 may store a program code, which is used for causing the processor 920 to execute the intelligent device management method provided by the embodiment of the present application by executing the program code.

For example, for the smart device 118, the memory 930 may have stored therein program code for causing the processor 920 to perform the method steps (e.g., steps 202, 204, 212 in fig. 2, step 350 in fig. 3A, step 314 in fig. 3B) performed on the smart device 118 side in fig. 2 or fig. 3A or fig. 3B by executing the program code. The smart device 118 is mainly responsible for receiving the first instruction sent by the server 116, and executing at least one operation (for example, turning on or off an indicator light, changing a color of the light, displaying a specific character/icon, sounding a buzzer, or observing a motion by human eyes, etc.) according to the first instruction, and sending an execution result to the server.

For the server 116, the memory 930 may store program code for causing the processor 920 to perform method steps (e.g., step 208 in fig. 2, steps 344, 348, 354, 356, 358 in fig. 3A, steps 304, 306, 310, 312, 318, 324, 326, 328, 330 in fig. 3B) performed on the server 116 side in fig. 2 or fig. 3A or fig. 3B by executing the program code. The server 116 is further configured to perform user identity authentication and record a login state of a user account; managing the binding relationship between the user account and the equipment; device registration information, a device capability set are managed.

The memory 930 may include both fast read and write memory for storing programs and data during operation of the communication device as well as persistent memory for storing instructions and data as needed. The memory 930 may also store data to be transmitted from or received by the communication device.

The communication module 910 may include a wireless communication module and/or a wired communication module, such as one or more of a Wi-Fi module, a bluetooth module, an infrared module, a GPRS module, an ethernet communication module, etc., and may be configured to transmit and receive information and may further send the received information to the processor 920 for processing; alternatively, signals generated by processor 920 are transmitted. The communication module 910 may receive wireless control signals of a desired operation, which may originate from a remote communication device, such as the mobile terminal 110.

The processor 920 is coupled to the communication module 910, the memory 930. Processor 920 can be any processing device including, but not limited to, an mips (microprocessor with interleaved stages) processor, a microprocessor, a digital signal processor, an application specific integrated circuit, a microcontroller, a state machine, or any type of programmable logic array. The processor 920 is a control center of the communication device, connects various parts of the entire communication device using various interfaces and lines, and performs various functions of the communication device and processes data by operating or executing software programs and/or modules stored in the memory 930 and calling data stored in the memory 930.

In addition, it should be noted that the structure shown in fig. 9 is not limited to the structure of the communication device related to the embodiment of the present application, and the communication device may further include other modules, such as a sensor and a power module, and details are not repeated here.

In the embodiment of the present application, the user equipment may be divided into the functional modules according to the method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. It should be noted that, in the embodiment of the present application, the division of the module is schematic, and is only one logic function division, and there may be another division manner in actual implementation.

The embodiment of the present application further provides a system for binding an intelligent device with a user management account established in a server, where the system includes the mobile terminal 110, the intelligent device 118, and the server 116 shown in fig. 1, and the system may implement the intelligent device management method shown in fig. 2, fig. 3A, and fig. 3B, where the mobile terminal 110 is configured to execute the method steps executed by the mobile terminal 110 side shown in fig. 2 or fig. 3A or fig. 3B, the intelligent device 118 is configured to execute the method steps executed by the intelligent device 118 shown in fig. 2 or fig. 3A or fig. 3B, and the server 116 is configured to execute the method steps executed by the server 116 shown in fig. 2 or fig. 3.

The embodiment of the present application further provides another system for binding an intelligent device with a user management account established in a server, where the system includes the mobile terminal 110 and the intelligent device 118 shown in fig. 1, and the system may implement the intelligent device management method steps shown in fig. 2 and fig. 3A or 3B, where the mobile terminal 110 is configured to execute the method steps executed by the mobile terminal 110 side shown in fig. 2, fig. 3A or fig. 3B, and the intelligent device 118 is configured to execute the method steps executed by the intelligent device 118 shown in fig. 2, fig. 3A or fig. 3B.

Embodiments of the present application also provide a computer storage medium that, when run on a mobile terminal, causes the mobile terminal to perform the relevant method steps in fig. 2, fig. 3A, or fig. 3B.

Embodiments of the present application also provide a computer program product, which when run on a mobile terminal, causes the mobile terminal to perform the relevant method steps in fig. 2, fig. 3A or fig. 3B.

In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware or any combination thereof. When implemented using a software program, may appear in whole or in part as a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website site, computer, server, or data center to another website site, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

The above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (47)

1. An intelligent device management method, comprising:

   the mobile terminal sends a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

   the server responds to the first request, sends a first instruction to the intelligent device and sends a second instruction to the mobile terminal;

   the intelligent device executes at least one operation according to the first instruction;

   the mobile terminal responds to the second instruction, records the process that the intelligent equipment executes the at least one operation and generates a recorded first file;

   the mobile terminal sends the first file to the server;

   the server determines whether the at least one operation executed by the intelligent device is matched with the first instruction according to the first file;

   the server associates a first identification of the smart device with a user management account established at the server in response to determining that the at least one operation performed by the smart device matches the first instruction.
2. The method of claim 1, wherein the method further comprises:

   the mobile terminal sends first information of the intelligent equipment to the server;

   and the server acquires a first identifier of the intelligent equipment according to the first information and sends the first identifier to the mobile terminal, wherein the first information comprises a serial number of the intelligent equipment or a media access control address of the intelligent equipment.
3. The method of claim 2, wherein the first identification is randomly generated by the server from the first information.
4. The method of any of claims 1 to 3, wherein said sending a first instruction to the smart device comprises:

   the server generates the first instruction according to the operation capacity of the intelligent equipment;

   and the server sends the first instruction to the intelligent equipment.
5. The method of any of claims 1 to 4, wherein each of the at least one operation comprises one or any combination of the following operations: a sound adjustment operation, a light adjustment operation, or a motion operation.
6. The method of claim 5, wherein the sound adjustment operation comprises one or any combination of the following operations: tone color adjustment, tone intensity adjustment, or duration adjustment.
7. The method of claim 5, wherein the lighting operation comprises a color adjustment or a brightness adjustment of the lighting.
8. The method of claim 5, wherein the motion operation comprises vibration or movement.
9. The method of any of claims 1 to 8, wherein the server determining from the first file whether the at least one operation performed by the smart device matches the first instruction comprises:

   the server identifies the image information and the audio information in the first file to acquire the operation behavior characteristics of the intelligent equipment;

   the server determines whether the operational behavior characteristics of the smart device match the first instructions.
10. The method of any of claims 1 to 9, further comprising:

    and the mobile terminal controls the intelligent equipment to execute the operation event.
11. The method of any of claims 1 to 10, wherein said recording said at least one operation performed by said smart device comprises:

    and the mobile terminal records the process of executing the at least one operation by the intelligent equipment through at least one of a camera or a microphone.
12. An intelligent device management method, comprising:

    the mobile terminal sends a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

    the intelligent equipment receives a first instruction sent by the server;

    in response to the first instruction, the smart device performs at least one operation;

    the mobile terminal receives a second instruction sent by the server;

    responding to the second instruction, the mobile terminal records the process that the intelligent equipment executes the at least one operation and generates a recorded first file;

    the mobile terminal sends the first file to the server, and the first file is used by the server to determine whether the at least one operation executed by the intelligent device is matched with the second instruction;

    the mobile terminal receives an association success notification from the server, the association success notification indicating: and after determining that the at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.
13. The method of claim 12, wherein the method further comprises:

    the mobile terminal sends first information of the intelligent equipment to the server;

    and the mobile terminal receives a first identifier of the intelligent device, which is acquired by the server according to the first information, wherein the first information comprises a serial number of the intelligent device or a media access control address of the intelligent device.
14. The method of claim 13, wherein the first identifier is randomly generated by the server based on the first information.
15. The method of any of claims 12 to 14, wherein the first instruction is generated by the server based on operational capabilities of the smart device.
16. The method of any of claims 12 to 15, wherein each of the at least one operation comprises one or any combination of the following operations: a sound adjustment operation, a light adjustment operation, or a motion operation.
17. The method of claim 16, wherein the sound adjustment operation comprises one or any combination of the following operations: tone color adjustment, tone intensity adjustment, or duration adjustment.
18. The method of claim 16, wherein the lighting operation comprises a color adjustment or a brightness adjustment of the lighting.
19. The method of claim 16, wherein the motion operation comprises vibration or movement.
20. The method of any of claims 12 to 19, further comprising:

    and the mobile terminal controls the intelligent equipment to execute the operation event.
21. The method of any of claims 12 to 20, wherein said recording said at least one operation performed by said smart device comprises:

    and the mobile terminal records the process of executing the at least one operation by the intelligent equipment through at least one of a camera or a microphone.
22. A system, characterized in that the system comprises:

    the mobile terminal is configured to send a first request to the server, wherein the first request comprises a first identifier of the intelligent device;

    the server is configured to respond to the first request, send a first instruction to the intelligent device and send a second instruction to the mobile terminal;

    the intelligent device is configured to execute at least one operation according to the first instruction;

    the mobile terminal is further configured to respond to the second instruction, record a process that the intelligent device executes the at least one operation, generate a recorded first file, and send the first file to the server;

    the server is further configured to determine from the first file whether the at least one operation performed by the smart device matches the first instruction, and associate a first identification of the smart device with a user management account established at the server in response to determining that the at least one operation performed by the smart device matches the first instruction.
23. The system of claim 22, wherein the mobile terminal is further configured to send first information of the smart device to the server, the server further configured to generate a first identification of the smart device from the first information and send the first identification to the mobile terminal.
24. The system of claim 22 or 23, wherein the first instructions are generated by the server based on operational capabilities of the smart device.
25. The system according to any one of claims 22 to 24, wherein each of the at least one operation comprises one or any combination of the following operations: a sound adjustment operation, a light adjustment operation, or a motion operation.
26. The system of claim 25, wherein the sound operation comprises an operational change in at least one of timbre, pitch, intensity, and duration of sound.
27. The system of claim 25, wherein the lighting operation comprises a color adjustment or a brightness adjustment of the light.
28. The system of claim 25, wherein the motion operation comprises vibration or movement.
29. The system of any of claims 22 to 28, wherein the determining from the first file whether the at least one operation performed by the smart device matches the first instruction comprises: identifying image information and audio information in the first file to obtain operation behavior characteristics of the intelligent equipment; determining whether the operational behavior characteristics of the smart device match the first instructions.
30. The system of any of claims 22 to 29, wherein the mobile terminal is further configured to control the smart device to perform operational events.
31. The system according to any one of claims 22 to 30, wherein said recording said at least one operation performed by said smart device comprises:

    and recording the process of executing the at least one operation by the intelligent equipment through at least one of a camera or a microphone of the mobile terminal.
32. A system, characterized in that the system comprises:

    a mobile terminal configured to: sending a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

    the smart device configured to: receiving a first instruction sent by the server, and responding to the first instruction, executing at least one operation by the intelligent equipment;

    the mobile terminal is further configured to: receiving a second instruction sent by the server, responding to the second instruction, recording the process of executing the at least one operation by the intelligent equipment, generating a recorded first file, and sending the first file to the server; the first file is used by the server to determine whether the at least one operation performed by the smart device matches the second instruction; receiving an association success notification from the server, the association success notification indicating: and after determining that the at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.
33. The system of claim 32, wherein the mobile terminal is further configured to send first information of the smart device to the server, receive a first identifier of the smart device obtained by the server according to the first information, wherein the first information comprises a serial number of the smart device or a media access control address of the smart device.
34. The system of claim 32 or 33, wherein the first instructions are generated by the server based on operational capabilities of the smart device.
35. The system according to any of claims 32 to 34, wherein each of the at least one operation comprises one or any combination of the following operations: a sound adjustment operation, a light adjustment operation, or a motion operation.
36. The system of claim 35, wherein the sound operation comprises an operational change in at least one of timbre, pitch, intensity, and duration of sound.
37. The system of claim 35, wherein the lighting operation comprises a color adjustment or a brightness adjustment of the light.
38. The system of claim 35, wherein the motion operation comprises vibration or movement.
39. The system of any of claims 32 to 38, wherein the mobile terminal is further configured to control the smart device to perform operational events.
40. The system according to any one of claims 32 to 39, wherein said recording said at least one operation performed by said smart device comprises:

    and recording the process of executing the at least one operation by the intelligent equipment through at least one of a camera or a microphone of the mobile terminal.
41. A mobile terminal, comprising:

    a touch screen, wherein the touch screen comprises a touch sensitive surface and a display screen;

    one or more processors;

    one or more memories;

    and one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs comprising instructions which, when executed by the mobile terminal, cause the mobile terminal to perform the steps of:

    sending a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

    receiving a second instruction sent by the server;

    responding to the second instruction, recording a process of executing at least one operation by the intelligent equipment and generating a recorded first file, wherein the intelligent equipment responds to the first instruction sent by the server to execute the at least one operation;

    sending the first file to the server, wherein the first file is used by the server to determine whether the at least one operation executed by the intelligent device matches the second instruction;

    receiving an association success notification from the server, the association success notification indicating: and after determining that the at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.
42. The mobile terminal of claim 41, wherein the instructions, when executed by the mobile terminal, further cause the mobile terminal to perform the steps of:

    sending first information of the intelligent equipment to the server;

    and receiving a first identifier of the intelligent device obtained by the server according to the first information, wherein the first information comprises a serial number of the intelligent device or a media access control address of the intelligent device.
43. The mobile terminal of claim 42, wherein the first identifier is randomly generated by the server based on the first information.
44. The mobile terminal according to any of claims 41 to 43, wherein the first instruction is generated by the server according to the operating capability of the smart device.
45. The mobile terminal according to any of claims 41 to 44, wherein each of said at least one operation comprises one or any combination of the following operations: a sound adjustment operation, a light adjustment operation, or a motion operation.
46. A computer readable storage medium having instructions stored therein, which when run on a mobile terminal, cause the mobile terminal to perform the steps of:

    sending a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

    receiving a second instruction sent by the server;

    responding to the second instruction, recording a process of executing at least one operation by the intelligent equipment and generating a recorded first file, wherein the intelligent equipment responds to the first instruction sent by the server to execute the at least one operation;

    sending the first file to the server, wherein the first file is used by the server to determine whether the at least one operation executed by the intelligent device matches the second instruction;

    receiving an association success notification from the server, the association success notification indicating: and after determining that the at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.
47. A computer program product comprising instructions which, when run on a mobile terminal, cause the mobile terminal to perform the steps of:

    sending a first request to a server, wherein the first request comprises a first identifier of the intelligent device;

    receiving a second instruction sent by the server;

    responding to the second instruction, recording a process of executing at least one operation by the intelligent equipment and generating a recorded first file, wherein the intelligent equipment responds to the first instruction sent by the server to execute the at least one operation;

    sending the first file to the server, wherein the first file is used by the server to determine whether the at least one operation executed by the intelligent device matches the second instruction;

    receiving an association success notification from the server, the association success notification indicating: and after determining that the at least one operation executed by the intelligent device is matched with the second instruction, the server successfully associates the first identification of the intelligent device with the user management account established in the server.

Images