CN110308661B

CN110308661B - Intelligent device control method and device based on machine learning

Info

Publication number: CN110308661B
Application number: CN201910490889.9A
Authority: CN
Inventors: 樊思远
Original assignee: Midea Group Co Ltd
Current assignee: Midea Group Co Ltd
Priority date: 2019-06-06
Filing date: 2019-06-06
Publication date: 2020-12-22
Anticipated expiration: 2039-06-06
Also published as: CN110308661A

Abstract

The embodiment of the invention provides an intelligent device control method and device based on machine learning, wherein the method comprises the following steps: if detecting that an application management program on the mobile terminal is started, acquiring the position information and the current operation time information of the mobile terminal; inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment; and controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays a control interface for remotely operating the conventional intelligent equipment. The embodiment of the invention is based on a machine learning mode, and automatically starts the control interface of the corresponding intelligent equipment according to the habit of the user, so that the trouble that the user searches for the intelligent equipment to be controlled from a plurality of intelligent equipment stored in an application management program can be saved, the operation of the user is simple and convenient, and the intelligent equipment to be controlled can be directly and quickly controlled, thereby improving the user experience.

Description

Intelligent device control method and device based on machine learning

Technical Field

The invention relates to the technical field of intelligent control, in particular to an intelligent device control method and device based on machine learning.

Background

With the development of the internet of things technology, smart home devices are also becoming popular. The number of intelligent home devices in the user office or the family is also increasing continuously, and the user can remotely operate each intelligent home device by using the human-computer interaction interface of the APP installed on the mobile terminal so as to obtain more automatic and intelligent scene control functions.

At present, the man-machine interaction interface of the APP installed on the mobile terminal generally presents the smart home device to be controlled by the following method: (1) listing all intelligent household equipment of a user; (2) and displaying the intelligent household equipment in a classified manner according to the category or the online/offline state of the intelligent household equipment.

For both of the above-mentioned presentation modes, the following problems exist: when a user needs to control a certain household device, the household device to be controlled needs to be recognized in the whole intelligent household device list, which causes that the user operation is complex and cumbersome, and thus the user experience is poor.

Disclosure of Invention

Aiming at the problems in the prior art, the embodiment of the invention provides an intelligent device control method and device based on machine learning.

In order to solve the above technical problems, embodiments of the present invention provide the following technical solutions:

in a first aspect, an embodiment of the present invention provides a machine learning-based intelligent device control method, including:

if detecting that an application management program on a mobile terminal is started, acquiring the position information and the current operation time information of the mobile terminal;

inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment;

controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays control information for remotely operating the conventional intelligent equipment;

the conventional intelligent equipment recognition model is obtained by training based on a machine learning algorithm according to historical operation data; the historical operation data comprises position information and operation time information of the mobile terminal and identification information of any intelligent device when the mobile terminal utilizes the application management program to remotely control any intelligent device before the current operation time.

Further, before the position information and the current operation time information are input into a conventional smart device recognition model and identification information of a conventional smart device is output, the method for controlling a smart device based on machine learning further includes: establishing the conventional intelligent equipment identification model;

wherein the establishing of the familiar intelligent device identification model comprises:

when the fact that the mobile terminal completes remote control on any intelligent device through the application management program is detected every time, position information, operation time information and identification information of any intelligent device where the mobile terminal is located are obtained, the obtained position information and the obtained operation time information of the mobile terminal are used as sample input data, the obtained identification information of any intelligent device is used as sample output data, model training is conducted on the basis of a machine learning algorithm, and the conventional intelligent device recognition model is obtained.

Further, the intelligent device control method based on machine learning further includes:

in the process of establishing the conventional intelligent equipment identification model, when the mobile terminal is detected to finish remote control on any intelligent equipment by using the application management program every time, operation content information of any intelligent equipment is also acquired; correspondingly, the acquired position information and the acquired operation time information of the mobile terminal are used as sample input data, the identification information of any intelligent equipment and the operation content information are used as sample output data, and model training is carried out on the basis of a machine learning algorithm to obtain the conventional intelligent equipment identification model;

correspondingly, when the position information and the current operation time information are input into a conventional intelligent equipment recognition model and identification information of conventional intelligent equipment is output, corresponding operation content information is also output;

correspondingly, the application management program is controlled according to the identification information of the conventional intelligent device, so that when the application management program displays the control information for remotely operating the conventional intelligent device, the control information also comprises the corresponding operation content information.

Further, the acquiring the location information of the mobile terminal specifically includes:

acquiring a wireless signal strength indication (RSSI) value of preset appointed intelligent equipment monitored by the mobile terminal as position information of the mobile terminal; the preset appointed intelligent equipment is fixed intelligent equipment, and the number of the preset appointed intelligent equipment is more than or equal to 2;

correspondingly, in the process of establishing the conventional intelligent device identification model, when the mobile terminal is detected to complete remote control on any intelligent device by using the application management program every time, the RSSI value of the preset specified intelligent device monitored by the mobile terminal is obtained as the position information of the mobile terminal, the RSSI value and the operation time information of the preset specified intelligent device monitored by the mobile terminal are used as sample input data, the identification information of any intelligent device is used as sample output data, and model training is carried out on the basis of a machine learning algorithm to obtain the conventional intelligent device identification model.

acquiring the current position information of the mobile terminal through positioning software installed on the mobile terminal;

or, acquiring the current position information of the mobile terminal through a Global Positioning System (GPS) locator;

or, acquiring the current position information of the mobile terminal through a plurality of preset pyroelectric infrared sensor nodes;

or, acquiring the current position information of the mobile terminal through an image processing algorithm based on computer machine vision.

In a second aspect, an embodiment of the present invention further provides an intelligent device control apparatus based on machine learning, including:

the mobile terminal comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring the position information and the current operation time information of the mobile terminal if detecting that an application management program on the mobile terminal is started;

the processing module is used for inputting the position information and the current operation time information into a conventional intelligent equipment recognition model and outputting identification information of conventional intelligent equipment;

the control module is used for controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays control information for remotely operating the conventional intelligent equipment;

Further, the intelligent device control apparatus based on machine learning further includes: a model building module;

wherein the model building module is specifically configured to:

Further, in the process of establishing the conventional intelligent device identification model, the model construction module further acquires operation content information of any intelligent device each time when detecting that the mobile terminal completes remote control on any intelligent device by using the application management program; correspondingly, the model construction module takes the acquired position information and the acquired operation time information of the mobile terminal as sample input data, takes the identification information and the operation content information of any intelligent device as sample output data, and performs model training based on a machine learning algorithm to obtain the conventional intelligent device identification model;

correspondingly, the processing module also outputs corresponding operation content information when inputting the position information and the current operation time information into a conventional intelligent equipment recognition model and outputting identification information of conventional intelligent equipment;

correspondingly, the control module controls the application management program according to the identification information of the conventional intelligent device, so that when the application management program displays the control information for remotely operating the conventional intelligent device, the control information also comprises the corresponding operation content information.

Further, the obtaining module is specifically configured to:

correspondingly, in the process of establishing the conventional intelligent device identification model, each time when the mobile terminal is detected to complete remote control on any intelligent device by using the application management program, the model construction module acquires the RSSI value of the preset designated intelligent device monitored by the mobile terminal as the position information of the mobile terminal, takes the RSSI value and the operation time information of the preset designated intelligent device monitored by the mobile terminal as sample input data, and takes the identification information of any intelligent device as sample output data, and performs model training based on a machine learning algorithm to acquire the conventional intelligent device identification model.

Further, the obtaining module is specifically configured to:

the acquiring the location information of the mobile terminal specifically includes:

In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the method for controlling an intelligent device based on machine learning according to the first aspect when executing the program.

In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored, where the computer program, when executed by a processor, implements the steps of the method for controlling a smart device based on machine learning according to the first aspect.

It can be known from the foregoing technical solutions that, according to the method and apparatus for controlling an intelligent device based on machine learning provided in the embodiments of the present invention, based on a machine learning manner, a control interface of a corresponding intelligent device is automatically started according to a user habit (at what location and at what time the user is accustomed to operating the intelligent device), so that a trouble that the user searches for an intelligent device to be controlled from among a plurality of intelligent devices stored in an application management program can be saved, and thus, the user operation becomes simple and convenient, and the intelligent device to be controlled can be directly and quickly controlled, thereby improving user experience.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a flowchart of a method for controlling an intelligent device based on machine learning according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of an intelligent device habitually operated at a preset location according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a control interface provided by an embodiment of the present invention;

FIG. 4 is a schematic view of another control interface provided by an embodiment of the present invention;

FIG. 5 is a schematic view of yet another control interface provided by an embodiment of the present invention;

fig. 6 is a schematic diagram of a CNN model according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an intelligent device control apparatus based on machine learning according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

After independent research and big data analysis based on statistics of user operation habit data, the inventor finds that the user utilizes the mobile terminal to remotely control a specific intelligent household appliance and the position and time of the user have certain correlation. For example, when a user group is in the sofa position of a living room between 21 o 'clock and 22 o' clock of a working day or controls an intelligent household appliance by a mobile phone in front of an intelligent television, the operation has a high probability for controlling the intelligent television, and when the user group is near the bed of a bedroom between 23 o 'clock and 1 o' clock of the next day, the operation has a high probability for controlling the intelligent air conditioner.

Based on the knowledge, the application provides an intelligent device control method based on machine learning, and a high-precision user habit prediction model (namely, a subsequent familiar intelligent device identification model) capable of outputting a specific intelligent device (or a specific operation function) to be controlled by a user can be trained by using position information and current operation time information as input data and utilizing a well-known machine learning algorithm such as a neural network, so that the control of the corresponding intelligent device is automatically acquired according to the user habit.

Based on the above analysis, the working principle and the working process of the intelligent device control method based on machine learning provided by the present application will be explained in detail through specific embodiments.

Fig. 1 is a flowchart illustrating a method for controlling an intelligent device based on machine learning according to an embodiment of the present invention, and referring to fig. 1, the method for controlling an intelligent device based on machine learning according to an embodiment of the present invention includes:

step 101: and if the application management program on the mobile terminal is detected to be started, acquiring the position information and the current operation time information of the mobile terminal.

In this embodiment, the location information may be absolute location information (e.g., GPS geographic location coordinates) or relative location information (e.g., the relative location information of the mobile terminal in the room may be determined according to distance information between the mobile terminal and several preset fixed points in the room).

In this embodiment, the current operation time information refers to time information corresponding to the detected start of the application management program on the mobile terminal. The time information may refer to working day time and weekend time, or to noon time and evening time, or may refer to any time period in a day, or may refer to other time information including date information and clock information, or may refer to other time information in various forms, which is not limited in this embodiment.

In this embodiment, the application management program refers to APPs for performing management control on the smart device, and these APPs are generally installed on the mobile terminal for the user to remotely operate the corresponding smart device.

In this embodiment, the step of acquiring the location information of the mobile terminal may be performed automatically when detecting that the application management program on the mobile terminal is started, or may be performed manually by a user when detecting that the application management program on the mobile terminal is started.

In this embodiment, the mobile terminal may refer to a smart phone, a smart remote controller, and the like. The intelligent device can be various household appliances or office equipment, such as a refrigerator, a television, an air conditioner, a washing machine and the like.

Step 102: and inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment.

In this step, the conventional intelligent device recognition model is obtained by training based on a machine learning algorithm according to historical operation data, such as location information, operation time information, and identification information of any intelligent device, of the mobile terminal when the mobile terminal completes remote control on any intelligent device by using the application management program before the current operation time. Therefore, after the position information and the current operation time information are acquired in step 101, the position information and the current operation time information are input into the conventional smart device recognition model as input data, so that the identification information of the conventional smart device can be output.

Step 103: and controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays control information for remotely operating the conventional intelligent equipment.

In this embodiment, after the identification information of the smart device (also referred to as the identification code of the smart device) is obtained in step 102, the application management program is controlled according to the identification information of the smart device obtained in step 102, so that the application management program displays control information for remotely operating the smart device, where the control information may be a control button for remotely operating the smart device, a control instruction for remotely operating the smart device, a control interface including a control button or a control instruction, and the like.

For example, referring to fig. 2, assuming that the user is accustomed to controlling the air conditioner 1 at position a1 during noon hours and accustomed to controlling the television at position a1 during evening hours based on historical operating data, a custom smart device recognition model may be trained based on machine learning algorithms based on these historical operating data. If the acquired position information of the mobile terminal is position a1 and the current operation time (the time for starting the application management program) is noon time when the application management program on the mobile terminal is detected to be started, the position information (position a1) and the current operation time information (the noon time is 12: 30) of the mobile terminal are input into the conventional intelligent device recognition model, then the identification information of the corresponding conventional intelligent device (air conditioner 1) is output by the conventional intelligent device recognition model, and then the application management program is controlled based on the identification information of the air conditioner 1, so that the control information displayed by the application management program is as shown in fig. 3, wherein fig. 3, and the following fig. 4 and fig. 5 are described by taking a control interface as an example. As can be seen from fig. 3, a control interface of the intelligent device (air conditioner 1) that the user is accustomed to operating at the current operation time and the current position is displayed on the application management program, so that the user can control the air conditioner 1 accordingly. Here, the noon time and the evening time may be specifically set, and generally, the noon time is 12-14 o 'clock, and the evening time is 18-22 o' clock.

For another example, assume that it is found that the user is accustomed to controlling the air conditioner 2 at the location A3 in the evening in summer according to the history data, therefore, when it is detected that the application management program on the mobile terminal is started, the obtained location information of the mobile terminal is the location A3, and the current operation time is the evening in summer, the location information (the location A3) and the current operation time information (the evening in summer is 20 o 'clock of 6 months and 22 o' clock) of the mobile terminal are input into the familiar smart device identification model, then the identification information of the corresponding familiar smart device (the air conditioner 2) is output by the familiar smart device identification model, and then the application management program is controlled so that the control interface displayed by the application management program is as shown in fig. 4. As can be seen from fig. 4, a control interface of the intelligent device (air conditioner 2) that is used by the user to operate at the current operation time and the current position is displayed on the application management program (various virtual keys for controlling the air conditioner 2 are displayed in the control interface, such as mode selection, temperature setting, power on/off control, and the like are available), so that the user can control the air conditioner 2 accordingly.

In addition, it should be noted that the control interface displayed by the application management program is not limited to the control interface shown in fig. 3 or fig. 4 that only includes one or one smart device, and may also be a control interface shown in fig. 5 that includes multiple smart devices. For example, assuming that a user is accustomed to operating both air conditioner 1 and television at location a1 during the evening hours of the summer, a custom smart device recognition model may be trained based on machine learning algorithms based on these historical operating data. If it is detected that the application management program on the mobile terminal is started, the acquired location information of the mobile terminal is location a1, and the current operation time is summer evening time, the location information (location a1) and the current operation time information (evening time) of the mobile terminal can be input into the conventional intelligent device recognition model, then the identification information of the corresponding conventional intelligent device (air conditioner 1 and television) is output by the conventional intelligent device recognition model, and then the application management program is controlled based on the identification information of the air conditioner 1 and the television, so that the control interface displayed by the application management program is as shown in fig. 5, thereby facilitating the user to correspondingly control the air conditioner 1 and the television. It should be noted that, because the number of the smart devices in a general home is not less than 10, by using the control method provided in this embodiment, the trouble of the user searching for the smart device to be controlled from the plurality of smart devices stored in the application management program can be eliminated, so that the user can directly see the control interface of the smart device to be controlled, thereby improving the user experience.

In this embodiment, it should be noted that the conventional smart device identification model is generated based on machine learning algorithm training according to historical operation data (data of user history for performing remote control on the smart device through the application management program) before the current operation time. When the conventional intelligent device recognition model is generated based on machine learning algorithm training, the position information and the corresponding operation time information of the intelligent device under historical control of the mobile terminal by using an application management program are generally used as sample input data, the corresponding identification information of the controlled intelligent device is used as sample output data, the initial machine learning model is trained until a model convergence condition is met, and then the conventional intelligent device recognition model is generated. In this embodiment, a CNN or RNN machine learning model may be employed for model training. In addition, when the conventional smart device recognition model is generated based on machine learning algorithm training according to historical operation data, the used historical operation data is the latest historical operation data, for example, the historical operation data may be the historical operation data with the current operation time being closer, such as the latest week, the latest month, the latest three months or the latest half year.

It should be noted that, in the embodiment, when the control interface of the corresponding intelligent device is automatically started according to the habit of the user, not only what intelligent device the user is accustomed to operate in what location area, but also what intelligent device the user is accustomed to operate in what location area and what intelligent device the user is accustomed to operate in what time are considered, so that the control interface of the intelligent device the user currently wants to control is automatically and accurately provided for the user, and therefore, the trouble that the user searches for the intelligent device which is frequently used at the current time and at the location from a plurality of intelligent devices stored in the application management program is omitted, and the user experience is improved.

In addition, it should be noted that the intelligent device control method based on machine learning provided in this embodiment may also be used to control what intelligent device a user is accustomed to operate in what location area, that is, with respect to the above-mentioned content, when inputting data to the familiar intelligent device identification model, only the location information of the mobile terminal needs to be input, and the current operation time information does not need to be input. Accordingly, when the conventional intelligent device recognition model is trained, the sample data does not need to contain the time information. Referring to fig. 2, for example, assume that the user has been accustomed to controlling the air conditioner 1 and the television at position a1 for the past time, based on these historical operating data, a custom smart device recognition model can be trained based on machine learning algorithms, then, when the mobile terminal is detected to be located at the position A1, the position information A1 of the mobile terminal can be input into the conventional intelligent device identification model, then the identification information of the corresponding conventional intelligent equipment (the air conditioner 1 and the television) is output by the conventional intelligent equipment identification model, and then controlling the application management program based on the identification information of the air conditioner 1 and the television, so that the control interface displayed by the application management program comprises the control interface of the intelligent equipment air conditioner 1 and the television which are used to operate by the user at the position, and the user can conveniently control the air conditioner 1 and the television correspondingly.

In this embodiment, it should be noted that the source of the training data set used in the training based on the machine learning algorithm may be in various manners, and this embodiment does not limit this, for example, it may be: a: only from the historical operating data of the user collected in advance; b: the training data set is only from a preset training data set, the preset training data set is generated by the inventor according to the usage habit data of a plurality of other users collected in advance, for example, a matched training data set can be determined for the user according to registration information such as the age and the sex registered by the user at the time of purchase, the type of household appliances purchased by the corresponding family of the user, and the like; c: and further training the obtained data set by utilizing the operation habit of the user on the basis of presetting the training data set.

It can be known from the foregoing technical solutions that, in the intelligent device control method provided in the embodiments of the present invention, based on a machine learning manner, a control interface of a corresponding intelligent device is automatically started according to a user habit (at what location and at what time and in what area, what intelligent device is operated), so that a trouble that a user searches for an intelligent device to be controlled from among a plurality of intelligent devices stored in an application management program can be saved, and thus, the user operation becomes simple and convenient, and the intelligent device to be controlled can be directly and quickly controlled, thereby improving user experience.

It should be noted that the intelligent device control method provided in this embodiment may be executed by a mobile terminal, may also be executed by a server, and may also be executed by both the mobile terminal and the server in an information interaction manner.

For example, in one implementation manner, the smart device control method provided by the present embodiment may be executed by a mobile terminal. In this implementation, the training of the conventional smart device recognition model is performed by the mobile terminal. Meanwhile, when detecting that an application management program used for controlling the intelligent equipment is started, the mobile terminal acquires position information and current operation time information of the mobile terminal, inputs the position information and the current operation time information into the conventional intelligent equipment identification model, and acquires identification information of the corresponding intelligent equipment from an output end of the conventional intelligent equipment identification model, so that the application management program is controlled according to the identification information of the intelligent equipment, the application management program displays a control interface or a control command for remotely operating the intelligent equipment, a user using the mobile terminal can see the control interface (or the control command) of the intelligent equipment which is operated in the position at the first time, the corresponding intelligent equipment is controlled, and the situation that the intelligent equipment which is frequently used in the position is searched from more intelligent equipment stored in the application management program is omitted Is troublesome. In addition, it should be specifically noted that the intelligent device control method provided in this embodiment is different from some control methods that display a list of intelligent device information located near or around the mobile terminal on the application management program of the mobile terminal according to the location information of the mobile terminal, because the core inventive idea of this embodiment is to start the control interface of the corresponding intelligent device for the user according to the user's habit, rather than to provide the control interface of the intelligent device near the location for the user according to the location of the user. Finally, it should be emphasized that the present embodiment is directed to embody the core inventive idea of automatically starting the control interface of the corresponding smart device according to the user habit (at what location and at what time the smart device is operated).

For another example, in an implementation manner, the intelligent device control method provided by this embodiment may be executed by a server. In this implementation, the training of the conventional smart device recognition model is performed by a server. Correspondingly, when detecting that an application management program on the mobile terminal is started, the server acquires the position information and the current operation time information of the mobile terminal in a certain mode, inputs the position information and the current operation time information into the conventional intelligent device identification model, acquires the identification information of the corresponding intelligent device from the output end of the conventional intelligent device identification model, and controls the application management program installed on the mobile terminal in a certain mode according to the identification information of the intelligent device, so that the application management program displays a control interface or a control command for remotely operating the intelligent device.

In this implementation, the server may detect whether the application management program for controlling the smart device on the mobile terminal is started in various ways, for example, such processing logic may be preset: and when the monitoring software monitors that an application management program for controlling the intelligent equipment on the mobile terminal is started, the monitoring software sends prompt information to the server. The prompt information also comprises the current position information and the current operation time information of the mobile terminal, which are acquired by the monitoring software. For another example, the method may also be implemented in other ways, such as presetting such processing logic: and when the mobile terminal detects that an application management program used for controlling the intelligent equipment on the mobile terminal is started, sending prompt information to the server. The prompt message also includes the current position information and the current operation time information acquired by the mobile terminal.

Similarly, after the server obtains the identification information of the intelligent device from the output end of the conventional intelligent device identification model in the local server, the application management program installed on the mobile terminal can be controlled through the monitoring software, so that the application management program displays a control interface or a control command for remotely operating the intelligent device. Or after acquiring the identification information of the intelligent device, the server may send the identification information of the intelligent device to the mobile terminal, and the mobile terminal controls the application management program, so that the application management program displays a control interface or a control command for remotely operating the intelligent device.

In this implementation manner, similarly, when the server trains and generates the conventional smart device recognition model based on the machine learning, the server needs to acquire sample data for training from the mobile terminal, where the sample data includes: and in the historical time, the user carries out remote control on the intelligent device through the application management program, and the position information, the operation time information and the corresponding identification information of the operated intelligent device are obtained. The server may also use the monitoring software related technology described above when acquiring the sample data from the mobile terminal. For example, monitoring whether the application management program on the mobile terminal completes remote control operation on the intelligent device or not by using monitoring software, and when the application management program on the mobile terminal is monitored to complete remote control operation on the intelligent device, acquiring the position information, the operation time information and the identification information of the operated intelligent device corresponding to the remote control, and sending the acquired information to the server, so that the server can complete the training process of the conventional intelligent device identification model locally on the server.

For another example, in an implementation manner, the intelligent device control method provided by this embodiment may be executed by the mobile terminal and the server together in an information interaction manner. In this implementation, the training of the conventional smart device recognition model is performed by a server. Accordingly, whether an application management program for controlling the smart device is started or not is detected by the mobile terminal, and when the start is detected, the position information and the current operation time information of the mobile terminal are acquired, then the position information and the current operation time information are sent to a server, so that the server inputs the position information and the current operation time information into the conventional intelligent equipment identification model, then the identification information of the corresponding intelligent device is output by the conventional intelligent device identification model, after the server acquires the identification information of the intelligent device, sending the identification information of the intelligent device to a mobile terminal, controlling the application management program by the mobile terminal according to the identification information of the intelligent device, causing the application manager to display a control interface (or control command) for remotely operating the smart device.

Further, based on the content of the foregoing embodiment, in this embodiment, before the step 101 or the step 102, the method for controlling an intelligent device based on machine learning further includes:

step 100: and establishing the conventional intelligent equipment identification model.

In this step, the establishing of the conventional smart device recognition model includes:

In this embodiment, it should be noted that the conventional intelligent device recognition model is a dynamic training process, and is obtained by continuously training a model based on a machine learning algorithm according to location information, operation time information and identification information of a corresponding intelligent device included in a continuously-occurring remote control behavior of the intelligent device, as well as the conventional intelligent device recognition model is not a constant one. The conventional intelligent device recognition model can be continuously updated according to the recent user operation behaviors, so that the recognition accuracy of the conventional intelligent device recognition model is improved. In this embodiment, in order to keep the conventional intelligent device recognition model updated continuously, when it is detected that the mobile terminal performs remote control on any intelligent device by using the application management program every time, it is required to acquire the location information and the operation time information of the mobile terminal and the identification information of any intelligent device, use the acquired location information and operation time information of the mobile terminal as sample input data, use the identification information of any intelligent device as sample output data, and perform model training based on a machine learning algorithm to acquire the conventional intelligent device recognition model.

For example, referring to fig. 2, it is assumed that in months 1-3, a user is accustomed to remotely control a television at position a1 using a mobile terminal, but since month 4, the user is accustomed to remotely control the television at position a2 using the mobile terminal (it may be that the owner of the family is on a business trip, and the other friend is staying at the family and is accustomed to remotely controlling the television at position a 2), therefore, according to several operation behaviors of the friend, the model can be continuously trained and updated based on a machine learning algorithm, so that the recognition accuracy of the familiar smart device recognition model is improved and gradually conforms to the current user's habit.

In this embodiment, when performing model training by machine learning, a CNN or RNN model may be used. In the following, a CNN model is taken as an example to be described with reference to fig. 6, it should be noted that fig. 6 is only a schematic model, where only two convolutional layers and two pooling layers are simply illustrated, and in practical applications, the number of convolutional layers and pooling layers is generally greater than 2. Specifically, the structure of the CNN model mainly includes: an input layer, n convolutional layers, n pooling layers, m full-link layers, and an output layer; the input of the input layer is sample input data containing position information and operation time information of the mobile terminal, and the input layer is connected with the convolutional layer C1; the convolutional layer C1 contains k1 convolutional kernels with the size of a1 × a1, sample input data of the input layer passes through the convolutional layer C1 to obtain k1 feature maps, and the obtained feature maps are transmitted to the pooling layer P1; the pooling layer P1 pools the feature map generated by the convolutional layer C1 with a sampling size of b1 × b1 to obtain corresponding k1 sampled feature maps, and then transmits the obtained feature maps to the next convolutional layer C2; the n convolutional layers and the pooling layer pairs are sequentially connected to continuously extract sampling characteristics of sample input data deep levels, and the last pooling layer Pn is connected with a full-connection layer F1, wherein the convolutional layers Ci contain ki convolutional kernels with the sizes of ai and ai, the sampling size of the pooling layer Pj is bj and bj, Ci represents the ith convolutional layer, and Pj represents the jth pooling layer; the full-connection layer F1 is a one-dimensional layer formed by mapping pixel points of all kn feature maps obtained by the last pooling layer Pn, each pixel represents a neuron node of the full-connection layer F1, and all neuron nodes of the F1 layer are fully connected with neuron nodes of the next full-connection layer F2; the output layer is connected with the output layer through m full-connection layers in sequence, and the last full-connection layer Fm is connected with the output layer in a full-connection mode; the output layer outputs sample output data containing identification information of a legacy smart device. In this embodiment, the CNN model is trained based on a machine learning algorithm by using sample input data including location information and operation time information of the mobile terminal and sample output data including identification information of a conventional smart device until the CNN model converges, so as to obtain the conventional smart device recognition model.

For example, when performing model training based on a machine learning algorithm, it is assumed that sample data for training a model is obtained through recording of historical operation data as shown in table 1 below.

TABLE 1

For the above table 1, it is assumed that the position Ax is a sofa in a home, the first time period is 12:00-14:00, the second time period is 19:00-22:00, the smart device a is an air conditioner, the smart device b is a washing machine, and the smart device c is a television, because the user is used to turn on the television whenever the user is on the sofa, the air conditioner is turned on only when the user is hot at noon, and the user can wash clothes with the washing machine only when the user is in the evening. Therefore, after the model is trained by adopting the sample data, the identification result which is more matched with the user requirements can be provided for the familiar intelligent equipment identification model which can be obtained by training, and then a control interface which is more matched with the user habits can be provided for the user, so that the user experience can be improved. In addition, it should be noted that table 1 is only an illustration for convenience of example, and the data amount is much larger than that shown in table 1 when actually performing sample training.

Further, based on the content of the foregoing embodiment, in this embodiment, the method for controlling an intelligent device based on machine learning further includes:

It should be noted that, in the present embodiment, on the basis of the above-mentioned embodiment, a content of displaying corresponding operation content information is added, so that the content displayed on the control interface more matches the requirement of the user, thereby further saving the operation time of the user, and further improving the user experience.

For example, assume that a user is accustomed to watching program 1 (e.g., lunch news) in television at position Ax during noon hours (first time period), and is accustomed to watching program 2 (e.g., football game) at position Ax during evening hours (second time period). Therefore, when the user is currently located at the position Ax and the current time is at night, and the application management program is started, the application management program not only displays a control interface for remotely operating the television, but also displays a trigger or control button for the program 2, so that the user can quickly start the program 2.

In this embodiment, the operation content information may be information of a control mode, a working channel, working details, and the like of the smart device, which is not limited in this embodiment. For example, the television may be channel information such as a drama channel, a movie channel, and a music channel, or may be specific program information. In addition, for the refrigerator, the operation content may be an operation mode of the refrigerator, such as a power saving mode (relatively suitable for working days) and a normal mode (relatively suitable for weekends). In addition, for the air conditioner, the operation content may be a cooling mode, a heating mode, a dehumidification mode, and the like of the air conditioner, and further may be a specific set temperature in the cooling mode, and the like. The present embodiment is not illustrated for specific meanings of the operation contents.

In this embodiment, when performing model training based on a machine learning algorithm, it is assumed that sample data for training a model obtained by recording historical operation data is as shown in table 2 below.

TABLE 2

For the above table 2, it is assumed that the position Ax is a sofa in a home, the first time period is 12:00-14:00, the second time period is 19:00-22:00, and the smart device a is a television, because the user is accustomed to watching the program 1 (such as the lunch news) at the noon time (the first time period), and is accustomed to watching the program 2 (such as the football match) at the evening time (the second time period), it can be seen that after the model is trained by using the sample data, the trained familiar smart device identification model can provide an identification result more matching with the user requirements, and further, a control interface more matching with the user habits can be provided for the user, so that the user experience can be improved. In addition, it should be noted that table 2 above is only an illustration for convenience of example, and the data amount and data content during actual sample training are much larger than those shown in table 2 above.

Further, based on the content of the foregoing embodiment, in an optional implementation manner, the acquiring the location information of the mobile terminal specifically includes:

and acquiring the current position information of the mobile terminal through positioning software installed on the mobile terminal.

In this embodiment, the current location information of the mobile terminal may be directly obtained through positioning software installed on the mobile terminal. It should be noted that, since the location information obtained in this embodiment is generally specific location information in a room or an office area, the location accuracy of the location software is required to be high enough to be able to distinguish different area locations in the room or the office area as a minimum requirement. For example, the positioning accuracy of the positioning software needs to meet the minimum requirement of being able to distinguish the living room position from the bedroom position based on a room of 90 square meters.

In addition, the current location information of the mobile terminal may also be obtained through a global Positioning system (gps) locator. In addition, the current position information of the mobile terminal can be acquired through a human body infrared sensor (pyroelectric infrared sensor). In addition, the positioning can be performed by computer machine vision related technology (such as image processing algorithm based on computer machine vision), and since the part of the content can adopt the currently mature positioning technology, the detailed description is omitted here. It should be noted that the positioning result obtained in this way is generally a direct positioning result, that is, the obtained position information refers to the absolute position information of the mobile terminal currently located.

Further, based on the content of the foregoing embodiment, in another optional implementation manner, the acquiring the location information of the mobile terminal specifically includes:

In this embodiment, when the location information of the mobile terminal is obtained, the absolute location of the mobile terminal is not obtained by using positioning software, but a preset specified smart device in a room is used as a positioning reference point, and then the location of the mobile terminal is determined according to the RSSI value, which is monitored by the mobile terminal, of the wireless signal strength indication of the preset specified smart device.

In this embodiment, the preset designated smart devices are fixed smart devices, and the number of the preset designated smart devices is greater than or equal to 2, so that the number of the preset designated smart devices is required to be greater than or equal to 2 to ensure the positioning accuracy. For example, 3 preset specified intelligent devices can be set, and as the three-point positioning is more accurate, the positioning accuracy can be ensured through the setting of the 3 preset specified intelligent devices.

It should be noted that a plurality of smart devices (smart home devices, smart home appliances) having a wireless communication function are installed in a home or an office of a general user, and include fixed-location smart devices and non-fixed-location smart devices. The intelligent equipment with fixed positions refers to an intelligent household appliance which cannot be moved easily after being set, such as an air conditioner, a refrigerator and the like, and is suitable for being used as a reference point for positioning the mobile terminal. Correspondingly, small-sized smart devices such as smart speakers are not suitable as reference points because the set positions may be frequently changed during use. For example, the preset designated intelligent device may be a refrigerator, a television, an air conditioner 1, an air conditioner 2, an air conditioner 3, and a washing machine in fig. 2, and the intelligent devices at the fixed positions may be used as the preset designated intelligent device, so that the position of the mobile terminal may be accurately determined according to the RSSI value, monitored by the mobile terminal, of the wireless signal strength indication of the preset designated intelligent device. Here, the location of the mobile terminal may be understood as the location of the user.

It should be noted that, the RSSI value for wireless signal strength indication mentioned in this embodiment may be a WiFi RSSI value or a bluetooth RSSI value.

In addition, in this embodiment, the RSSI value monitoring may be used in combination with the human body infrared sensor, for example, the RSSI value may be obtained after the human body infrared sensor detects a human body, thereby reducing power consumption and computation.

It should be noted that, when monitoring the RSSI value of the preset specified smart device, the mobile terminal may read the data packet of each smart device in the promiscuous mode. Or all the intelligent devices under the user account are instructed to broadcast beacon frames for measuring the RSSI value, so that the RSSI value of the preset designated intelligent device is obtained. The preset appointed intelligent device can be manually set by a user or can be obtained by the mobile terminal through identification code recognition of the intelligent device.

For example, when a user starts a smart device management APP in a room by using a smart phone (or a smart remote controller), after the APP logs in a user account, the smart phone monitors (for example, in a hybrid mode) a beacon frame (a standard beacon frame or a custom common data frame with a beacon function) sent by a fixed smart appliance, i.e., a refrigerator, a television, an air conditioner 1, an air conditioner 2, an air conditioner 3, and a washing machine, through a WiFi module, and acquires an RSSI value of a location where the smart phone is located. It should be noted that, if the smart phone monitors the RSSI values of the smart appliances through the bluetooth module, the normal use of the WiFi network of the phone may not be affected.

Although the present embodiment refers to the RSSI location fingerprint positioning technology, this embodiment does not need to be divided into an offline sampling stage and a real-time positioning stage, and each operation of the user is recorded in the database as a basis for positioning (the fingerprint positioning method needs to perform offline sampling in advance, and only records the RSSI value in the offline sampling stage, and the RSSI value in the real-time positioning stage is only used for positioning, and the RSSI value cannot be recorded in the database), thereby greatly reducing the learning cost of the user. In addition, the embodiment also does not need to establish a positioning coordinate system and an access point layout, and does not need the positioning precision required by the fingerprint positioning technology, so the computation amount and the precision requirement are reduced.

In addition, in a preferred embodiment, before the RSSI value of the location of the mobile terminal is collected, a group of the smart devices may be filtered according to the current GPS location of the mobile terminal or an SSID or a network segment of a connected WiFi hotspot (for example, the smart devices placed in an office are an office group, and the smart devices placed in a home are a home group), so that a smart device group adapted to the current scene is left in the APP. For example, when the mobile terminal is located at home, office groups are filtered out, and when the mobile terminal is located at office, home groups are filtered out. The advantages of this treatment are: and the user can conveniently and manually set the preset appointed intelligent equipment. In addition, the preset specified intelligent device may be an intelligent device bound with the user account, or may be an intelligent device not bound with the user account.

Fig. 7 is a schematic structural diagram of an intelligent device control apparatus based on machine learning according to an embodiment of the present invention, and referring to fig. 7, the intelligent device control apparatus based on machine learning according to an embodiment of the present invention includes: an acquisition module 21, a processing module 22 and a control module 23, wherein:

an obtaining module 21, configured to obtain location information and current operation time information of a mobile terminal if it is detected that an application management program on the mobile terminal is started;

the processing module 22 is configured to input the location information and the current operation time information into a conventional intelligent device recognition model, and output identification information of a conventional intelligent device;

the control module 23 is configured to control the application management program according to the identification information of the conventional intelligent device, so that the application management program displays control information for remotely operating the conventional intelligent device;

Further, based on the content of the foregoing embodiment, in this embodiment, the intelligent device control apparatus based on machine learning further includes: a model building module;

wherein the model building module is specifically configured to:

Further, based on the content of the foregoing embodiment, in this embodiment, in the process of establishing the conventional intelligent device identification model, each time it is detected that the mobile terminal completes remote control over any intelligent device by using the application management program, the model construction module further obtains operation content information of any intelligent device; correspondingly, the model construction module takes the acquired position information and the acquired operation time information of the mobile terminal as sample input data, takes the identification information and the operation content information of any intelligent device as sample output data, and performs model training based on a machine learning algorithm to obtain the conventional intelligent device identification model;

Further, based on the content of the foregoing embodiment, in an optional implementation manner, the obtaining module is specifically configured to:

Further, based on the content of the foregoing embodiment, in another optional implementation manner, the obtaining module is specifically configured to:

Since the machine learning-based smart device control apparatus provided in this embodiment can be used to execute the machine learning-based smart device control method according to the second embodiment, and the operation principle and the beneficial effect are similar, detailed descriptions are omitted here, and specific contents can be referred to the description of the above embodiment.

Based on the same inventive concept, another embodiment of the present invention provides an electronic device, which specifically includes the following components, with reference to fig. 8: a processor 601, a memory 602, a communication interface 603, and a communication bus 604;

the processor 601, the memory 602 and the communication interface 603 complete mutual communication through the communication bus 604; the communication interface 603 is used for realizing information transmission among related devices such as modeling software, an intelligent manufacturing equipment module library and the like;

the processor 601 is configured to call a computer program in the memory 602, and when the processor executes the computer program, the processor implements all the steps of the above-mentioned intelligent device control method based on machine learning, for example, when the processor executes the computer program, the processor implements the following steps: if detecting that an application management program on a mobile terminal is started, acquiring the position information and the current operation time information of the mobile terminal; inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment; controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays control information for remotely operating the conventional intelligent equipment; the conventional intelligent equipment recognition model is obtained by training based on a machine learning algorithm according to historical operation data; the historical operation data comprises position information and operation time information of the mobile terminal and identification information of any intelligent device when the mobile terminal utilizes the application management program to remotely control any intelligent device before the current operation time.

It should be noted that the electronic device mentioned in this embodiment may be a mobile terminal, and may also be a cloud server.

Based on the same inventive concept, another embodiment of the present invention provides a non-transitory computer-readable storage medium, having a computer program stored thereon, which when executed by a processor implements all the steps of the above-mentioned machine learning-based smart device control method, for example, when the processor executes the computer program, the processor implements the following steps: if detecting that an application management program on a mobile terminal is started, acquiring the position information and the current operation time information of the mobile terminal; inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment; controlling the application management program according to the identification information of the conventional intelligent equipment, so that the application management program displays control information for remotely operating the conventional intelligent equipment; the conventional intelligent equipment recognition model is obtained by training based on a machine learning algorithm according to historical operation data; the historical operation data comprises position information and operation time information of the mobile terminal and identification information of any intelligent device when the mobile terminal utilizes the application management program to remotely control any intelligent device before the current operation time.

In addition, the logic instructions in the memory may be implemented in the form of software functional units and may be stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention. One of ordinary skill in the art can understand and implement it without inventive effort.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the technical solutions mentioned above may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the machine learning-based smart device control method according to the embodiments or some parts of the embodiments.

In the description of the present invention, it should be noted that the terms "upper", "lower", and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the drawings, which are merely for convenience in describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and operate, and thus, should not be construed as limiting the present invention. Unless expressly stated or limited otherwise, the terms "mounted," "connected," and "connected" are intended to be inclusive and mean, for example, that they may be fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present invention can be understood by those skilled in the art according to specific situations.

It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims

1. A machine learning-based intelligent device control method is characterized by comprising the following steps:

inputting the position information and the current operation time information into a conventional intelligent equipment recognition model, and outputting identification information of conventional intelligent equipment; the identification information of the conventional intelligent equipment is the identification of the intelligent equipment which is habitually operated by the mobile terminal at the current operation time; the position range of the intelligent equipment operated by the mobile terminal at the current operation time habit includes a whole house range, and the judgment standard of the intelligent equipment operated by the mobile terminal at the current operation time habit is the operation habit at the corresponding position at the current operation time;

2. The machine-learning-based smart device control method according to claim 1, wherein before inputting the location information and the current operation time information into a familiar smart device recognition model and outputting identification information of a familiar smart device, the machine-learning-based smart device control method further comprises: establishing the conventional intelligent equipment identification model;

3. The machine-learning-based smart device control method according to claim 2, further comprising:

4. The machine learning-based intelligent device control method according to claim 2, wherein the acquiring the location information of the mobile terminal specifically includes:

5. The machine learning-based intelligent device control method according to claim 1, wherein the acquiring the location information of the mobile terminal specifically includes:

6. An intelligent device control apparatus based on machine learning, comprising:

the processing module is used for inputting the position information and the current operation time information into a conventional intelligent equipment recognition model and outputting identification information of conventional intelligent equipment; the identification information of the conventional intelligent equipment is the identification of the intelligent equipment which is habitually operated by the mobile terminal at the current operation time; the position range of the intelligent equipment operated by the mobile terminal at the current operation time habit includes a whole house range, and the judgment standard of the intelligent equipment operated by the mobile terminal at the current operation time habit is the operation habit at the corresponding position at the current operation time;

7. The machine-learning-based smart device control apparatus of claim 6, further comprising: a model building module;

wherein the model building module is specifically configured to:

8. The intelligent device control apparatus based on machine learning of claim 7, wherein the model building module further obtains operation content information of any intelligent device each time it is detected that the mobile terminal completes remote control of any intelligent device by using the application management program in the process of building the conventional intelligent device identification model; correspondingly, the model construction module takes the acquired position information and the acquired operation time information of the mobile terminal as sample input data, takes the identification information and the operation content information of any intelligent device as sample output data, and performs model training based on a machine learning algorithm to obtain the conventional intelligent device identification model;

9. The machine-learning-based smart device control apparatus of claim 7, wherein the obtaining module is specifically configured to:

10. The machine-learning-based smart device control apparatus of claim 6, wherein the obtaining module is specifically configured to:

11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the machine learning-based intelligent device control method according to any one of claims 1 to 5 are implemented when the processor executes the program.

12. A non-transitory computer readable storage medium, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the steps of the machine learning-based smart device control method according to any one of claims 1 to 5.