CN109218145B - IOT equipment control interface display method, system, equipment and storage medium - Google Patents
IOT equipment control interface display method, system, equipment and storage medium Download PDFInfo
- Publication number
- CN109218145B CN109218145B CN201810971551.0A CN201810971551A CN109218145B CN 109218145 B CN109218145 B CN 109218145B CN 201810971551 A CN201810971551 A CN 201810971551A CN 109218145 B CN109218145 B CN 109218145B
- Authority
- CN
- China
- Prior art keywords
- image
- identification number
- equipment
- control center
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2807—Exchanging configuration information on appliance services in a home automation network
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
Abstract
The invention provides a display method, a system, equipment and a storage medium of a control interface of IOT equipment, wherein the method comprises the steps that a user side obtains a first image related to the IOT equipment and sends the first image to a control center, and the control center identifies an equipment identification number through a set machine learning algorithm model; if the control center successfully identifies, the control center returns the equipment identification number, and the user side displays a corresponding control interface; if the control center fails to recognize, the control center returns an equipment identification number list, the user side selects an equipment identification number according to a user instruction and sends a first image storage signal to the control center, the control center stores a first image recognized by the user into an image training set, and the image training set comprises images marked with the equipment identification number and is used for training a machine learning algorithm model. The invention automatically displays the control interface of the IOT equipment through image recognition, and solves the problem that the manual searching of the equipment control interface is difficult and complicated caused by the increase of the IOT equipment.
Description
Technical Field
The invention relates to the field of Internet of things, in particular to a display method, a display system, a display device and a storage medium of an IOT device control interface.
Background
IOT (Internet of things) is an important component of new-generation information technology and is also an important development stage of the 'informatization' era. The development of IOT makes it possible to use a home as a platform and to use a smart home system integrated with facilities related to home life, such as an integrated wiring technology, a network communication technology, a security technology, an automatic control technology, and the like.
The smart home system can construct an efficient management system for residential facilities (IOT devices) and family schedule things, wherein the IOT devices include smart home appliances: such as air conditioners, central air conditioning systems, refrigerators, modern kitchens and bathrooms, integrated kitchens, electric rice cookers, televisions, home entertainment, home cinemas, central background music systems, hot water showers, integrated bathrooms, washing machines, dust collectors, electric heating systems and the like; still include intelligent house network sensor: such as a wireless combustible gas leakage detector, a gas composition sensor, a temperature sensor, a humidity sensor, a wireless door magnetic sensor and a window magnetic sensor. Still include video sensing intelligent system: the system comprises an access control video intelligent system, an old and young life safety video intelligent system, a garage video intelligent system and the like; still include intelligent energy power switch system: intelligent power box, intelligent electric current socket, but remote control and management electric parameter: current, voltage, power consumption; electric gas valves, electric water valves, electric valves of heating heat source systems and the like, and even intelligent building and environment monitoring can be included.
Generally, the IOT devices of the smart home system are managed by the user side, such as the APP of the mobile phone, the PAD side or the PC side, but with the increase of the IOT devices, the manual searching and displaying of the management APPs of different devices and the multiple operation interfaces of the same device take time and are tedious, and therefore, a high-efficiency display method for the control interface of the automatic IOT device is urgently needed to meet the requirement of smart home development.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present invention and therefore may include information that does not constitute prior art known to a person of ordinary skill in the art.
Disclosure of Invention
Aiming at the problems in the prior art, the invention aims to provide a display method, a display system, a display device and a storage medium for an IOT device control interface, which solve the problem that the manual searching of the device control interface is difficult and tedious due to the increase of IOT devices.
The embodiment of the invention provides a display method of an IOT equipment control interface, which is characterized by comprising the following steps:
a user side acquires a first image related to IOT equipment and sends the first image to a control center, and the control center is provided with a machine learning algorithm model with classification capability;
the control center receives the first image and identifies the equipment identification number through the machine learning algorithm model;
if the control center successfully identifies, the control center returns an equipment identification number to the user side, and the user side displays a control interface corresponding to the equipment identification number;
if the control center fails in identification, the control center returns an equipment identification number list to the user side, the user side selects an equipment identification number according to the user instruction, the user side sends a first image storage signal to the control center, the control center stores the first image identified by the user into an image training set, and the image training set comprises images marked with the equipment identification number and is used for training the machine learning algorithm model.
Preferably, the machine learning algorithm model comprises at least any one of a first model, a second model and a third model;
the input of the first model is an image containing equipment, and the output is a corresponding equipment identification number;
the input of the second model is an image containing equipment and a background, and the output is a corresponding equipment identification number;
the input of the third model is an image containing a scene corresponding to the equipment, and the output is a corresponding equipment identification number.
Preferably, the machine learning algorithm model is trained using the training set of images by:
and extracting image features of the image training set, wherein the image features are expressed in a vector form to obtain model parameters of a machine learning algorithm model.
Preferably, the control center may set a frequency of training the machine learning algorithm model or a condition for starting training.
Preferably, the step of identifying the device identification number by the control center through the machine learning algorithm model comprises the following steps:
and the control center extracts the first image characteristics of the received first image, and compares the first image characteristics with the image characteristics in the machine learning algorithm model to obtain the probability that the first image identifier is the identification number of each device.
Preferably, the control center determines the maximum probability of the probabilities and sets a specific value;
when the maximum probability is larger than or equal to the specific numerical value, in order to identify a successful state, returning an equipment identification number corresponding to the maximum probability to the user side as an equipment identification number corresponding to the image;
and when the maximum probability is smaller than the specific numerical value, the control center returns an equipment identification number list to the user side for identifying a failure state.
Preferably, the control center returns a list of device identification numbers to the user side, where the list is the device identification numbers arranged according to the probability.
The embodiment of the invention also provides a display system of the control interface of the IOT equipment, which is characterized by comprising a user side and a control center;
the user side has a camera shooting function;
the control center comprises a control unit, an analysis and calculation unit, an image unit and a training unit;
the user side acquires a first image related to the IOT equipment and sends the first image to the control unit, and the control unit is provided with a machine learning algorithm model with classification capability;
the control unit receives the first image, and the analysis and calculation unit identifies the equipment identification number through the machine learning algorithm model;
if the analysis and calculation unit successfully identifies, the control unit returns an equipment identification number to the user side, and the user side displays a control interface corresponding to the equipment identification number;
if the analysis and calculation unit fails to recognize, the control unit returns an equipment identification number list to the user side, the user side selects an equipment identification number according to the user instruction, the user side sends a first image storage signal to the control unit, the control unit stores the first image recognized by the user into an image training set in the image unit, the image training set comprises images marked with the equipment identification number, and the training unit trains the machine learning algorithm model by using the image training set.
An embodiment of the present invention further provides a display device for an IOT device control interface, where the display device includes:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the display method of the IOT device control interface via execution of the executable instructions.
An embodiment of the present invention is a computer-readable storage medium for storing a program, where the program is configured to implement the steps of the display method of the IOT device control interface when executed.
The invention automatically displays the control interface of the IOT equipment through image recognition, and solves the problem that the manual searching of the equipment control interface is difficult and complicated caused by the increase of the IOT equipment.
Drawings
Other features, objects, and advantages of the invention will be apparent from the following detailed description of non-limiting embodiments, which proceeds with reference to the accompanying drawings and which is incorporated in and constitutes a part of this specification, illustrating embodiments consistent with the present application and together with the description serve to explain the principles of the application. It is obvious that the drawings in the following description are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic view of an application scenario of a display method of an IOT device control interface according to an embodiment of the present invention;
FIG. 2 is a flowchart of a method for displaying a control interface of an IOT device according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an interface for initial setup according to an embodiment of the present invention;
FIG. 4 is a diagram illustrating an IOT device control interface displayed after successful recognition according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a client-side interactive interface after a failure in recognition according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a display system of an IOT device control interface according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a display device of an IOT device control interface according to an embodiment of the present invention; and
fig. 8 is a schematic structural diagram of a computer-readable storage medium according to an embodiment of the present invention.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
Fig. 1 is a schematic structural diagram of an intelligent home system, which includes: a user terminal 10, a control center 20, and a plurality of IOT devices 30.
The user terminal 10 and the control center 20, and the control center 20 and the plurality of IOT devices 30 are connected through a wired network or a wireless network.
The network protocols employed by the IOT device 30 include, but are not limited to, at least one of the following:
a network protocol based on Zigbee (Zigbee) protocol;
a network protocol based on a wireless networking specification Z-Wave;
a network protocol based on Wi-Fi (Wireless Fidelity) protocol;
a network protocol based on a BLE (Bluetooth Low Energy) protocol;
a network protocol based on an RF (Radio Frequency) 433 protocol, which uses a 433Mhz Frequency band;
a network protocol based on RF2.4G protocol, wherein the network protocol uses a 2.4Ghz frequency band;
a network protocol based on the radio frequency RF5G protocol, which uses the 5Ghz band.
Fig. 2 is a flowchart of a display method of an IOT device control interface according to an embodiment of the present invention, where the method specifically includes the following steps:
s100 a user acquires a first image related to the IOT device, where the user may be a mobile phone terminal 11, a Pad terminal 12, or a PC terminal 13 with a camera function.
S100, in the process of shooting IOT equipment by a user, the user side sends the first image to a control center, and the control center is provided with a machine learning algorithm model with classification capability;
s200, the control center receives a first image;
s300, the control center identifies equipment identification numbers through the machine learning algorithm model, wherein the equipment identification numbers can be numbers of the IOT equipment and have the function of identifying the IOT equipment;
s400, if the control center successfully identifies, the control center returns the device identification number to the user side, in this embodiment, the user side may pre-store a mapping relationship table between the device identification number and the corresponding control interface, and the user side determines and displays the corresponding control interface through the display mapping relationship table.
TABLE 1 mapping relationship table of device identification number and corresponding control interface
| Device identification number | IOT equipment corresponding control interface |
| ID0001 | Control interface of refrigerator |
| ID0002 | Control interface of air conditioner |
| IDXXXX | Control interface of sound box |
S500, if the control center fails to recognize, the control center returns an equipment identification number list to the user side, the user side selects an equipment identification number according to the user instruction, the user side sends a first image storage signal to the control center, the control center stores the first image recognized by the user into an image training set, and the image training set comprises images marked with the equipment identification number and is used for training the machine learning algorithm model.
The initial image training set can be a general image training set pre-stored by the control center and related to the equipment, and the machine learning algorithm model with the classification capability can be a machine learning algorithm model trained well by the pre-stored general image training set.
In the embodiment of the present invention, the image training set may also be set individually by the user, and the user side of the present invention may provide an interactive interface for setting the initial image training set, as shown in fig. 3. Specifically, a user selects an IOT device on the interactive interface, such as the device 001 in the option living room, the user enters a shooting mode to shoot the device 001, the daily use habit can be considered during shooting, the same device shoots from multiple angles and multiple distances to obtain images of each viewing angle of the device, the images are uploaded to an image training set of the control center, the image user end which shoots after selecting the device automatically sets an image corresponding device identification number, namely, the image training set stores images with device identification numbers, and the user can repeat the steps to complete image identification classification of each IOT device in a home environment. The method for inputting the set image training set at the user end is not limited to this, and for example, the image may be captured first, and the user may manually fill in the device information.
After the user-customized image training set is established, the machine learning algorithm model extracts the image features of the image training set, the image features are expressed in a vector form to obtain model parameters of the machine learning algorithm model, and the machine learning algorithm model is trained.
In actual use, when the device has the same or similar shape, the recognition rate is low only by the device shape recognition device. Therefore, the machine learning algorithm model of the present invention may include models of different image recognition modes, such as a first model, a second model, a third model, and the like. In our embodiment, the input of the first model is defined as an image containing equipment, and the output is the corresponding equipment identification number; the input of the second model is an image containing equipment and a background, and the output is a corresponding equipment identification number; the input of the third model is an image containing a scene corresponding to the equipment, and the output is a corresponding equipment identification number. For devices with the same or similar shapes, the shooting device can contain a background, such as a surrounding partial decorative scene; if the background of the device is the same or similar, some recognizable decoration can be attached to the device to improve the recognition accuracy.
It should be noted that the present invention can not only determine the device identification number by identifying the device in the image, but also can set the image of the scene corresponding to the specific device to identify the device, for example, when the sweeping robot is sweeping the bottom of the bed, the user shoots the ground in the living room, the control center identifies the image of the ground, finds the sweeping robot identification number corresponding to the ground, and the user side can call out the control interface of the sweeping robot, thereby controlling the sweeping robot. The number of the machine learning algorithm models and the recognition mode of the models to the images are not limited to the above explanation, and can be determined according to actual needs.
S300, the control center identifies the device identification number through the machine learning algorithm model, and in an embodiment, the method specifically includes the following steps:
the control center extracts and receives first image features of the first image, compares the first image features with image features in the machine learning algorithm model, compares the image features with a plurality of used vectors, and obtains the probability that the first image identifier is the identification number of each device according to the similarity between the vectors.
The control center determines the maximum probability in the probabilities and sets a specific numerical value; and when the maximum probability is larger than or equal to the specific numerical value, in order to identify a successful state, returning the equipment identification number corresponding to the maximum probability to the user side as the equipment identification number corresponding to the image. Fig. 4 is a schematic diagram of the user terminal 10 displaying the IOT device control interface after successful recognition according to an embodiment of the present invention, through which the user controls the IOT device, which is a control interface of a sound box in this embodiment.
And when the maximum probability is smaller than the specific numerical value, the control center returns an equipment identification number list to the user side for identifying a failure state. Fig. 5 is a schematic diagram of the interactive interface of the user terminal 10 after the identification failure according to an embodiment of the present invention, where the list is the device identification numbers arranged according to the probability. The device identifiers in the actual list may be more than shown in fig. 5, see the return list area 8 of fig. 5. For example, if the user selects "air conditioner" in fig. 5 as the device corresponding to the first image with failed identification, the user marks the first image as the device identification number corresponding to the "air conditioner" and sends a first image storage signal to the control center, and the control center stores the first image in the image training set, which is actually a process of updating the image training set by the user. The control center may set a frequency of training the machine learning algorithm model or a condition for initiating training. For example, the machine learning algorithm model may be trained periodically according to an image training set, or may be trained after a certain number of images are added to the image training set.
For convenience, the interactive interface of the user terminal 10 may also provide an option of acquiring the image related to the IOT device again when the identification fails, see the functional area 9 in fig. 5, so as to return to step S100 of the present invention and repeat the flow of the display method of the present invention.
Fig. 6 is a schematic diagram of a display system of an IOT device control interface according to an embodiment of the present invention, where the system includes a user terminal 10 and a control center 20; the user terminal 10 has a camera shooting function, and the control center comprises a control unit 21, an analysis and calculation unit 22, an image unit 23 and a training unit 24;
the user side acquires a first image related to the IOT equipment and sends the first image to the control unit 21, and the control unit 21 is provided with a machine learning algorithm model with classification capability;
the control unit 21 receives the first image, and the analysis and calculation unit 22 identifies the equipment identification number through the machine learning algorithm model;
if the analysis and calculation unit 22 successfully identifies, the control unit 21 returns an equipment identification number to the user side, and the user side displays a control interface corresponding to the equipment identification number;
if the analysis and calculation unit 22 fails to perform recognition, the control unit 21 returns a device identification number list to the user side, the user side selects a device identification number according to the user instruction, the user side sends a first image storage signal to the control unit 21, the control unit 21 stores the first image recognized by the user into an image training set in the image unit 23, the image training set includes images marked with device identification numbers, and the training unit 24 trains the machine learning algorithm model by using the image training set.
The embodiment of the invention also provides the IOT equipment control interface equipment, which comprises a processor; a memory having stored therein executable instructions of the processor. Wherein the processor is configured to perform the steps of the IOT device control interface method via execution of executable instructions.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" platform.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 7. The electronic device 600 shown in fig. 7 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 7, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one memory unit 620, a bus 630 connecting the different platform components (including the memory unit 620 and the processing unit 610), a display unit 640, etc.
Wherein the storage unit stores program code executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of the present specification. For example, processing unit 610 may perform the steps as shown in fig. 2.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RA identification information systems, tape drives, and data backup storage platforms, etc.
An embodiment of the present invention further provides a computer-readable storage medium for storing a program, where the program is executed to implement the steps of the display method for the IOT device control interface. In some possible embodiments, the aspects of the present invention may also be implemented in the form of a program product comprising program code for causing a terminal device to perform the steps according to various exemplary embodiments of the present invention described in the above-mentioned electronic prescription flow processing method section of this specification, when the program product is run on the terminal device.
Referring to fig. 8, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
In summary, the present invention provides a method for displaying a control interface of an IOT device, where the method includes that a user side obtains a first image related to the IOT device and sends the first image to a control center, and the control center is provided with a machine learning algorithm model with classification capability; the control center receives the first image and identifies the equipment identification number through the machine learning algorithm model; if the control center successfully identifies, the control center returns an equipment identification number to the user side, and the user side displays a control interface corresponding to the equipment identification number; if the control center fails to recognize, the control center returns an equipment identification number list to the user side, the user side selects an equipment identification number according to the user instruction, the user side sends a first image storage signal to the control center, the control center stores the first image recognized by the user into an image training set, and the image training set comprises images marked with the equipment identification numbers and is used for training the machine learning algorithm model. The invention automatically displays the control interface of the IOT equipment through image recognition, and solves the problem that the manual searching of the equipment control interface is difficult and complicated caused by the increase of the IOT equipment.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.
Claims (6)
1. A display method of an IOT device control interface is characterized by comprising the following steps:
a user side acquires a first image related to IOT equipment and sends the first image to a control center, and the control center is provided with a machine learning algorithm model with classification capability;
the control center receives the first image and identifies the equipment identification number through the machine learning algorithm model;
the machine learning algorithm model at least comprises any one of a first model, a second model and a third model;
the input of the first model is an image containing equipment, and the output is a corresponding equipment identification number;
the input of the second model is an image containing equipment and a background, and the output is a corresponding equipment identification number;
the input of the third model is an image containing a scene corresponding to equipment, and the output of the third model is a corresponding equipment identification number;
the control center identifies the equipment identification number through the machine learning algorithm model and comprises the following steps:
the control center extracts and receives first image characteristics of the first image, and compares the first image characteristics with image characteristics in the machine learning algorithm model to obtain the probability that the first image identification is the identification number of each device;
the control center determines the maximum probability in the probabilities and sets a specific numerical value;
when the maximum probability is larger than or equal to the specific numerical value, in order to identify a successful state, returning an equipment identification number corresponding to the maximum probability to the user side as an equipment identification number corresponding to the image, and displaying a control interface corresponding to the equipment identification number by the user side;
when the maximum probability is smaller than the specific numerical value, in a recognition failure state, the control center returns an equipment identification number list to the user side, the user side selects an equipment identification number according to a user instruction, the user side sends a first image storage signal to the control center, the control center stores the first image identified by the user into an image training set, and the image training set comprises images marked with the equipment identification number and is used for training the machine learning algorithm model;
the control center sets a frequency of training the machine learning algorithm model or a condition for starting training.
2. The method of displaying an IOT device control interface of claim 1, wherein the machine learning algorithm model is trained using the training set of images by:
and extracting image features of the image training set, wherein the image features are expressed in a vector form to obtain model parameters of a machine learning algorithm model.
3. The method of displaying an IOT device control interface of claim 1, wherein the control center returns a list of device identification numbers to the user side, the list being a list of device identification numbers arranged according to a probability.
4. The display system of the IOT equipment control interface is characterized by comprising a user side (10) and a control center (20);
the user terminal (10) has a camera shooting function;
the control center comprises a control unit (21), an analysis and calculation unit (22), an image unit (23) and a training unit (24);
the user side acquires a first image related to the IOT equipment and sends the first image to the control unit (21), and the control unit (21) is provided with a machine learning algorithm model with classification capability;
the machine learning algorithm model at least comprises any one of a first model, a second model and a third model;
the input of the first model is an image containing equipment, and the output is a corresponding equipment identification number;
the input of the second model is an image containing equipment and a background, and the output is a corresponding equipment identification number;
the input of the third model is an image containing a scene corresponding to equipment, and the output of the third model is a corresponding equipment identification number;
the control unit (21) receives a first image, and the analysis and calculation unit (22) identifies an equipment identification number through the machine learning algorithm model;
the control center identifies the equipment identification number through the machine learning algorithm model and comprises the following steps:
the control center extracts and receives first image characteristics of the first image, and compares the first image characteristics with image characteristics in the machine learning algorithm model to obtain the probability that the first image identification is the identification number of each device;
the control center determines the maximum probability in the probabilities and sets a specific numerical value;
when the maximum probability is larger than or equal to the specific numerical value, if the analysis and calculation unit (22) successfully identifies, the control unit (21) returns an equipment identification number to the user side, and the user side displays a control interface corresponding to the equipment identification number;
when the maximum probability is smaller than the specific numerical value, if the analysis and calculation unit (22) fails to perform recognition, the control unit (21) returns an equipment identification number list to the user side, the user side selects an equipment identification number according to a user instruction, the user side sends a first image storage signal to the control unit (21), the control unit (21) stores the first image recognized by the user into an image training set in the image unit (23), the image training set comprises images marked with the equipment identification numbers, and the training unit (24) trains the machine learning algorithm model by using the image training set;
the control center sets a frequency of training the machine learning algorithm model or a condition for starting training.
5. A display device based on an IOT device control interface, comprising:
a processor;
a memory having stored therein executable instructions of the processor;
wherein the processor is configured to perform the steps of the method of displaying the IOT device control interface of any of claims 1-3 via execution of the executable instructions.
6. A computer readable storage medium storing a program which when executed performs the steps of the IOT device control interface method of any of claims 1-3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810971551.0A CN109218145B (en) | 2018-08-24 | 2018-08-24 | IOT equipment control interface display method, system, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810971551.0A CN109218145B (en) | 2018-08-24 | 2018-08-24 | IOT equipment control interface display method, system, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109218145A CN109218145A (en) | 2019-01-15 |
| CN109218145B true CN109218145B (en) | 2021-10-08 |
Family
ID=64989301
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810971551.0A Active CN109218145B (en) | 2018-08-24 | 2018-08-24 | IOT equipment control interface display method, system, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109218145B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP7278847B2 (en) * | 2019-04-19 | 2023-05-22 | 東芝ライフスタイル株式会社 | Remote control system, remote control terminal, remote control program |
| CN110457101B (en) * | 2019-07-26 | 2023-03-21 | 联想(北京)有限公司 | Information processing method, electronic equipment and computer readable storage medium |
| CN112953961B (en) * | 2021-03-14 | 2022-05-17 | 国网浙江省电力有限公司电力科学研究院 | Equipment type identification method in power distribution room Internet of things |
| EP4089581A1 (en) * | 2021-05-10 | 2022-11-16 | Carrier Corporation | Machine learning assisted contactless control of a physical device through a user device |
| CN113703382B (en) * | 2021-07-13 | 2023-05-16 | 特科能(株洲)科技有限公司 | Workpiece identification system of foreroom pre-vacuumizing multipurpose atmosphere nitriding furnace |
| CN113504831A (en) * | 2021-07-23 | 2021-10-15 | 电光火石(北京)科技有限公司 | IOT (input/output) equipment control method based on facial image feature recognition, IOT and terminal equipment |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011145709A1 (en) * | 2010-05-21 | 2011-11-24 | 日本電気株式会社 | Device cooperation system, operation instructing device, device user interface configuring means, method for providing cooperation of devices and program |
| CN105204742A (en) * | 2015-09-28 | 2015-12-30 | 小米科技有限责任公司 | Control method and device of electronic equipment and terminal |
| CN105739315A (en) * | 2016-02-02 | 2016-07-06 | 杭州鸿雁电器有限公司 | Indoor user electric appliance equipment control method and device |
| CN107038462A (en) * | 2017-04-14 | 2017-08-11 | 广州机智云物联网科技有限公司 | Equipment control operation method and system |
| US9753687B1 (en) * | 2014-01-03 | 2017-09-05 | Sony Interactive Entertainment America Llc | Wearable computer using programmed local tag |
| CN108010302A (en) * | 2016-10-31 | 2018-05-08 | 深圳市掌网科技股份有限公司 | Remote controlling system for electric appliances and method based on augmented reality |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10379514B2 (en) * | 2016-07-27 | 2019-08-13 | Ademco Inc. | Systems and methods for controlling a home automation system based on identifying a user location via a wi-fi fingerprint |
| CN107480725A (en) * | 2017-08-23 | 2017-12-15 | 京东方科技集团股份有限公司 | Image-recognizing method, device and computer equipment based on deep learning |
-
2018
- 2018-08-24 CN CN201810971551.0A patent/CN109218145B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011145709A1 (en) * | 2010-05-21 | 2011-11-24 | 日本電気株式会社 | Device cooperation system, operation instructing device, device user interface configuring means, method for providing cooperation of devices and program |
| US9753687B1 (en) * | 2014-01-03 | 2017-09-05 | Sony Interactive Entertainment America Llc | Wearable computer using programmed local tag |
| CN105204742A (en) * | 2015-09-28 | 2015-12-30 | 小米科技有限责任公司 | Control method and device of electronic equipment and terminal |
| CN105739315A (en) * | 2016-02-02 | 2016-07-06 | 杭州鸿雁电器有限公司 | Indoor user electric appliance equipment control method and device |
| CN108010302A (en) * | 2016-10-31 | 2018-05-08 | 深圳市掌网科技股份有限公司 | Remote controlling system for electric appliances and method based on augmented reality |
| CN107038462A (en) * | 2017-04-14 | 2017-08-11 | 广州机智云物联网科技有限公司 | Equipment control operation method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109218145A (en) | 2019-01-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109218145B (en) | IOT equipment control interface display method, system, equipment and storage medium | |
| US10985936B2 (en) | Customized interface based on vocal input | |
| CN110687817B (en) | Intelligent household control method and device, terminal and computer readable storage medium | |
| EP2908469B1 (en) | System and method for commissioning wireless building system devices | |
| CN105471705B (en) | Intelligent control method, equipment and system based on instant messaging | |
| CN103116336B (en) | Method and device for automatic management of controlled device through intelligent home control terminal | |
| CN106842968B (en) | Control method, device and system | |
| US20150198938A1 (en) | Systems, devices, methods and graphical user interface for configuring a building automation system | |
| US11929844B2 (en) | Customized interface based on vocal input | |
| US20140070919A1 (en) | User Identification and Location Determination in Control Applications | |
| US9100207B2 (en) | Systems, devices, and methods for mapping devices to realize building automation and energy management | |
| US11781771B2 (en) | Operating system, information processing device, control system, and infrared output device | |
| CN108061359B (en) | Air conditioner control method and device | |
| CN108989162B (en) | Household intelligent robot management system | |
| KR20170115802A (en) | Electronic apparatus and IOT Device Controlling Method thereof | |
| JP2017046295A (en) | Equipment operation confirmation system and remote control device | |
| CN111936981A (en) | Device control system and device control method | |
| JP2016063415A (en) | Network system, audio output method, server, device and audio output program | |
| CN111580406A (en) | Intelligent home control system based on Internet of things | |
| WO2017119163A1 (en) | Information processing apparatus, electronic device, method, and program | |
| CN112331190A (en) | Intelligent equipment and method and device for self-establishing voice command thereof | |
| KR101958068B1 (en) | Method of servicing smart home automation and apparatuses performing the same | |
| CN111964222B (en) | Air conditioner control method and device, equipment and storage medium | |
| US20200041151A1 (en) | Air conditioning control device, air conditioning control method, and program | |
| Okemiri et al. | Development of a Smart Home Control System |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |