CN117608210A

CN117608210A - Equipment control method and device, storage medium and electronic equipment

Info

Publication number: CN117608210A
Application number: CN202311478484.6A
Authority: CN
Inventors: 李小庆
Original assignee: Shenzhen TCL New Technology Co Ltd
Current assignee: Shenzhen TCL New Technology Co Ltd
Priority date: 2023-11-07
Filing date: 2023-11-07
Publication date: 2024-02-27

Abstract

The application discloses a device control method, a device, a storage medium and electronic equipment, and relates to the technical field of Internet of things, wherein the method comprises the following steps: acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; acquiring an equipment environment image, wherein the equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled; analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; and obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, wherein the target identification information is used for controlling the equipment to be controlled. The method and the device can effectively reduce the time consumption of identification information identification of the controlled device (to-be-controlled device), improve the device control efficiency and improve the user experience.

Description

Equipment control method and device, storage medium and electronic equipment

Technical Field

The application relates to the technical field of internet of things, in particular to a device control method, a device, a storage medium and electronic equipment.

Background

When the internet of things equipment is becoming more and more popular, in the internet of things control system, a controlled device (to be controlled) needs to be determined on a master control device (such as an APP, an applet, virtual reality glasses and helmets, mixed reality glasses and helmets, enhanced display glasses and helmets, etc.), and the determining process needs more steps, is troublesome to operate, and affects user experience.

Currently, in many internet of things main control devices, a controlled device is generally required to be found in the main control device through manual selection or eye movement, etc., through a layer-by-layer menu selection or list selection, so as to determine identification information of the controlled device (to-be-controlled device), and then the controlled device can be controlled.

In the current mode, identification information of the controlled equipment (equipment to be controlled) in the process is high in time consumption, and particularly, a scene of a large number of controlled equipment is bound in main control equipment such as enterprises or factories, and the time consumption is especially high, so that the equipment control efficiency is low, and inconvenience is brought to users.

Disclosure of Invention

The embodiment of the application provides a scheme, which can effectively reduce the identification information identification time consumption of the controlled equipment (equipment to be controlled), improve the equipment control efficiency and improve the user experience.

The embodiment of the application provides the following technical scheme:

according to one embodiment of the present application, a device control method includes: acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; acquiring an equipment environment image, wherein the equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled; analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; and obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, wherein the target identification information is used for controlling the equipment to be controlled.

In some embodiments of the present application, the analyzing the device environment image using a preset model identification model to obtain a target device model includes: performing probability screening setting on the preset model identification model to obtain a set model identification model, wherein the probability identification of the set model identification model on other models except the preset equipment model included in the candidate equipment list is zero; analyzing the equipment environment image by adopting the set model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model; and obtaining the target equipment model according to the at least one predicted equipment model and the probability corresponding to each predicted equipment model.

In some embodiments of the present application, the preset model identification model is trained as follows: acquiring a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are acquired equipment surrounding environment images containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of corresponding sample equipment; analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image; and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

In some embodiments of the present application, the recognition model to be trained is constructed as follows: acquiring a pre-trained image classification model; replacing the full-connection layer of the last layer of the image classification model with a new full-connection layer to obtain an updated image classification model; initializing the weight in the updated image classification model into the weight of the image classification model, and freezing the weights of all layers except the selected layer in the updated image classification model to obtain the recognition model to be trained.

In some embodiments of the present application, the candidate device list includes a first binding device list; the obtaining the candidate device list includes: transmitting control user information in the main control equipment to a server; receiving first binding equipment information sent by the server, wherein the first binding equipment information comprises equipment related information of candidate equipment bound by the control user information, and the first binding equipment information is inquired by the server according to the control user information; and obtaining a first binding device list according to the first binding device information, wherein the first binding device list comprises corresponding preset device models and preset identification information of the candidate devices bound by the control user information.

In some embodiments of the present application, the candidate device list includes a second binding device list; the obtaining the candidate device list includes: sending control scenes and control user information to a server; receiving second binding equipment information sent by the server, wherein the second binding equipment information comprises equipment related information of candidate equipment which is bound by the control user information and accords with the control scene, and the second binding equipment information is inquired by the server according to the control scene and the control user information; and obtaining a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene.

In some embodiments of the present application, the obtaining the target identification information according to preset identification information corresponding to a preset equipment model matched with the target equipment model in the candidate equipment list includes: if the preset identification information corresponding to the preset equipment model matched with the target equipment model is one, determining the preset identification information corresponding to the preset equipment model matched with the target equipment model as the target identification information; if the number of the preset identification information corresponding to the preset equipment model matched with the target equipment model is at least two, prompting a user to select, and obtaining the target identification information according to the user selection.

According to one embodiment of the present application, an apparatus control device, the device comprises: the list acquisition module is used for acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; the image acquisition module is used for acquiring an equipment environment image, wherein the equipment environment image is acquired equipment surrounding environment image containing equipment to be controlled; the model analysis module is used for analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; the identification determining module is used for obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, and the target identification information is used for controlling the equipment to be controlled.

In some embodiments of the present application, the model analysis module is configured to: performing probability screening setting on the preset model identification model to obtain a set model identification model, wherein the probability identification of the set model identification model on other models except the preset equipment model included in the candidate equipment list is zero; analyzing the equipment environment image by adopting the set model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model; and obtaining the target equipment model according to the at least one predicted equipment model and the probability corresponding to each predicted equipment model.

In some embodiments of the present application, the apparatus further comprises a training module for: acquiring a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are acquired equipment surrounding environment images containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of corresponding sample equipment; analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image; and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

In some embodiments of the present application, the training module is configured to: acquiring a pre-trained image classification model; replacing the full-connection layer of the last layer of the image classification model with a new full-connection layer to obtain an updated image classification model; initializing the weight in the updated image classification model into the weight of the image classification model, and freezing the weights of all layers except the selected layer in the updated image classification model to obtain the recognition model to be trained.

In some embodiments of the present application, the candidate device list includes a first binding device list; the list acquisition module is configured to: transmitting control user information in the main control equipment to a server; receiving first binding equipment information sent by the server, wherein the first binding equipment information comprises equipment related information of candidate equipment bound by the control user information, and the first binding equipment information is inquired by the server according to the control user information; and obtaining a first binding device list according to the first binding device information, wherein the first binding device list comprises corresponding preset device models and preset identification information of the candidate devices bound by the control user information.

In some embodiments of the present application, the candidate device list includes a second binding device list; the list acquisition module is configured to: sending control scenes and control user information to a server; receiving second binding equipment information sent by the server, wherein the second binding equipment information comprises equipment related information of candidate equipment which is bound by the control user information and accords with the control scene, and the second binding equipment information is inquired by the server according to the control scene and the control user information; and obtaining a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene.

In some embodiments of the present application, the identification determination module is configured to: if the preset identification information corresponding to the preset equipment model matched with the target equipment model is one, determining the preset identification information corresponding to the preset equipment model matched with the target equipment model as the target identification information; if the number of the preset identification information corresponding to the preset equipment model matched with the target equipment model is at least two, prompting a user to select, and obtaining the target identification information according to the user selection.

According to another embodiment of the present application, a storage medium has stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform the method described in the embodiments of the present application.

According to another embodiment of the present application, an electronic device may include: a memory storing a computer program; and the processor reads the computer program stored in the memory to execute the method according to the embodiment of the application.

According to another embodiment of the present application, a computer program product or computer program includes computer instructions stored in a computer readable storage medium. The computer instructions are read from the computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternative implementations described in the embodiments of the present application.

In the embodiment of the application, a candidate device list is obtained, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; acquiring an equipment environment image, wherein the equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled; analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; and obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, wherein the target identification information is used for controlling the equipment to be controlled.

In this way, the device environment image is acquired, the device environment image is analyzed by adopting the preset model identification model to obtain the target device model, then, the preset identification information corresponding to the preset device model matched with the target device model can be quickly and accurately matched from the candidate device list in a model matching mode, and further, the target identification information of the device to be controlled can be effectively and accurately identified in the main control device, so that the device to be controlled is effectively controlled. The identification information identification time consumption of the controlled equipment (equipment to be controlled) can be effectively reduced, the equipment control efficiency is improved, and the user experience is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly introduced below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 shows a flow chart of a device control method according to an embodiment of the present application.

FIG. 2 illustrates a model building and training flow diagram according to one embodiment of the present application.

FIG. 3 illustrates a model training flow diagram according to one embodiment of the present application.

Fig. 4 shows a flowchart of identification information identification according to an embodiment of the present application in one scenario.

Fig. 5 shows a block diagram of a device control apparatus according to one embodiment of the present application.

Fig. 6 shows a block diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The present disclosure is further described in detail below with reference to the drawings and examples. It should be understood that the examples provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are some of the embodiments for implementing the present disclosure, and not all of the embodiments for implementing the present disclosure, and the technical solutions described in the embodiments of the present disclosure may be implemented in any combination without conflict.

It should be noted that, in the embodiments of the present disclosure, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a method or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such method or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other related elements (e.g., a step in a method or a unit in an apparatus, e.g., a unit may be a part of a circuit, a part of a processor, a part of a program or software, etc.) in a method or apparatus comprising the element.

For example, the apparatus control method provided in the embodiment of the present disclosure includes a series of steps, but the apparatus control method provided in the embodiment of the present disclosure is not limited to the described steps, and similarly, the apparatus control device provided in the embodiment of the present disclosure includes a series of units, but the apparatus provided in the embodiment of the present disclosure is not limited to including the explicitly described units, and may include units that are required to be set for acquiring related information or performing processing based on the information.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure.

Fig. 1 schematically shows a flow chart of a device control method according to an embodiment of the present application. The execution main body of the device control method can be any main control device with processing capability, such as a television, a computer, a mobile phone, a smart watch, a household appliance and the like.

The master device may perform a device control method as shown in fig. 1, which may include steps S110 to S140.

Step S110, a candidate device list is obtained, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; step S120, acquiring an equipment environment image, wherein the equipment environment image is acquired surrounding environment images of equipment containing equipment to be controlled; step S130, analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; step S140, obtaining target identification information according to preset identification information corresponding to a preset device model matched with the target device model in the candidate device list, where the target identification information is used to control the device to be controlled.

The master device may acquire a candidate device list, where the candidate device list may be acquired from a predetermined location such as a server or a local cache according to an actual situation. The candidate device list includes preset device types and preset identification information corresponding to a plurality of candidate devices, wherein the preset device types (DeviceModelNumber) are the types of the candidate devices, the preset identification information is the identification information of the candidate devices, the preset identification information can include preset device IDs (deviceids), and the preset identification information can also include preset device names (devicenames).

The main control equipment can acquire equipment environment images, wherein the main control equipment can acquire the equipment environment images through a camera of the main control equipment under the operation of a user, and the main control equipment can also receive the equipment environment images transmitted by other equipment. The equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled, namely, the equipment environment image at least can display the equipment to be controlled; the environment of the equipment to be controlled can be displayed in the equipment environment image according to actual conditions, and the accuracy of the model of the target equipment can be further improved.

The device environment image is input into the preset model identification model, the preset model identification model can analyze the device environment image to obtain at least one predicted device model, and the target device model can be obtained according to the at least one predicted device model.

Further, the preset device model matched with the target device model can be obtained from the candidate device list, and the preset device model matched with the target device model is the preset device model with the same target device model. For example, the target device model is 2000000000N, and the preset device model to which the target device model matches, namely 2000000000N.

The preset identification information corresponding to the preset equipment model matched with the target equipment model can be determined from the candidate equipment list, the target identification information can be obtained according to the preset identification information corresponding to the preset equipment model matched with the target equipment model, and the target identification information can be used for controlling equipment to be controlled.

The device to be controlled can be controlled based on the target identification information, specifically, if the target identification information is the identification information corresponding to the device to be controlled bound by the main control device, the main control device can automatically enter a control page or enter the control page under the operation of a user, the target identification information is displayed or hidden in the control page, and related controls in the control page can be operated to control the device to be controlled. In some cases, if the target identification information is identification information corresponding to the to-be-controlled device that is not bound by the master control device, the master control device may first bind the device according to the target identification information and then control the device.

In this way, based on the steps S110 to S140, the device environment image is acquired, the device environment image is analyzed by using the preset model identification model, so as to obtain the target device model, then, by means of model matching, the preset identification information corresponding to the preset device model matched with the target device model can be quickly and accurately matched from the candidate device list, and further, the target identification information of the device to be controlled can be effectively and accurately identified in the master control device, so that the device to be controlled is effectively controlled. The identification information identification time consumption of the controlled equipment (equipment to be controlled) can be effectively reduced, the equipment control efficiency is improved, and the user experience is improved.

Further alternative embodiments of the steps performed when performing plant control under the embodiment of fig. 1 are described below.

In one embodiment, the candidate device list includes a first binding device list; the obtaining the candidate device list includes: transmitting control user information in the main control equipment to a server; receiving first binding equipment information sent by the server, wherein the first binding equipment information comprises equipment related information of candidate equipment bound by the control user information, and the first binding equipment information is inquired by the server according to the control user information; and obtaining a first binding device list according to the first binding device information, wherein the first binding device list comprises corresponding preset device models and preset identification information of the candidate devices bound by the control user information.

Specifically, the master device may send control user information (for example, a user ID of a user logged in the master device) in the master device to a server (for example, a cloud server or a physical server, etc.), and the server may query and obtain first binding device information according to the control user information (for example, the user ID), where the first binding device information includes device related information of candidate devices bound by the control user information.

For example, the first binding device information is shown below, where DeviceModelNumber is a preset device model, deviceID is a preset device identifier, and DeviceName is a preset device name. Wherein, the server classifies the related information according to different preset equipment models.

The master control device may receive the first binding device information sent by the server, and obtain a first binding device list according to the first binding device information, where the first binding device list includes a corresponding preset device model and preset identification information of the candidate device bound by the control user information.

For example, the information in the first binding device list may be specifically as follows:

"2000000000N":

[ { "DeviceID": "10000001", "DeviceName": "main air conditioner" },

{ "DeviceID": "10000002", "DeviceName": "recumbent air conditioner" },

{ "DeviceID": "10000003", "DeviceName": "study air conditioner" })

"2000000000S":

[ { "DeviceID": "10000004", "DeviceName": "Living room air conditioner" ]

"3000000000T":

[ { "DeviceID": "10000004", "DeviceName": "living room television" ].

Further, in some manners, the master device may send the control user information (e.g., the user ID of the user logged in the master device) in the master device together with the authentication information (e.g., a credential token, session, etc.) to the server (e.g., a cloud server or a physical server, etc.), and the server may query the first binding device information after the authentication information is authenticated.

Optionally, in another embodiment, the candidate device list includes a second binding device list; the obtaining the candidate device list includes: sending control scenes and control user information to a server; receiving second binding equipment information sent by the server, wherein the second binding equipment information comprises equipment related information of candidate equipment which is bound by the control user information and accords with the control scene, and the second binding equipment information is inquired by the server according to the control scene and the control user information; and obtaining a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene.

In this embodiment, the master control device may send control user information (e.g., user ID of a user logged in the master control device) and the control scene in the master control device to the server (e.g., cloud server or physical server), and the server may query the control user information (e.g., user ID) to obtain first binding device information and then determine binding device information corresponding to the control scene from the first binding device information to obtain second binding device information.

For example, if the control scenario is refrigeration, the second binding device information may be specifically shown as follows, where DeviceModelNumber is a preset device model, deviceID is a preset device identifier, and DeviceName is a preset device name. Wherein, the server classifies the related information according to different preset equipment models.

The master control device can receive second binding device information sent by the server, and obtain a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene. Therefore, the range of the candidate equipment can be further reduced, and the identification efficiency of the equipment identification information is further improved.

For example, the information in the second binding device list may specifically be as follows:

"2000000000N":

[ { "DeviceID": "10000001", "DeviceName": "main air conditioner" },

{ "DeviceID": "10000002", "DeviceName": "recumbent air conditioner" },

{ "DeviceID": "10000003", "DeviceName": "study air conditioner" })

"2000000000S":

[ { "DeviceID": "10000004", "DeviceName": "living room air conditioner" } ].

In one embodiment, the analyzing the device environment image by using a preset model identification model to obtain a target device model includes: performing probability screening setting on the preset model identification model to obtain a set model identification model, wherein the probability identification of the set model identification model on other models except the preset equipment model included in the candidate equipment list is zero; analyzing the equipment environment image by adopting the set model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model; and obtaining the target equipment model according to the at least one predicted equipment model and the probability corresponding to each predicted equipment model.

In this embodiment, probability screening is performed on the preset model identification model to obtain a set model identification model, probability that the set model identification model only identifies the preset device model included in the candidate device list is set through a probability screening technology, and probability identification of other models except the preset device model included in the candidate device list is zero.

And further, analyzing the equipment environment image by using the set model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model, wherein the predicted equipment models are all preset equipment models included in the candidate equipment list.

According to at least one predicted equipment model and the probability corresponding to each predicted equipment model, one predicted equipment model with the highest probability can be used as the obtained target equipment model. With such an embodiment, the recognition of the model in a wide model range is avoided, and the recognition efficiency is further improved only in the model range in the candidate device list, so that the recognition efficiency of the identification information is further improved as a whole.

Optionally, in other embodiments, the analyzing the device environment image by using a preset model identification model to obtain a target device model includes: directly analyzing the equipment environment image by adopting the preset model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model; and according to the at least one predicted equipment model and the probability corresponding to each predicted equipment model, taking one predicted equipment model with the highest probability as the obtained target equipment model.

In one embodiment, the preset model identification model in the foregoing embodiment may be specifically trained in the following manner: acquiring a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are acquired equipment surrounding environment images containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of corresponding sample equipment; analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image; and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

The method comprises the steps of pre-collecting a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are collected surrounding environment images of equipment containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of the corresponding sample equipment. For example, if a sample device environment image includes a sample device S, a sample device model corresponding to the sample device S may be calibrated for the sample device environment image, and the sample device model is a model tag.

Analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image, wherein the predicted sample equipment model can be specifically: sample equipment environment images in the training set can be respectively input into a recognition model to be trained, and the recognition model to be trained outputs a predicted sample equipment model corresponding to each sample equipment environment image and a probability corresponding to the predicted sample equipment model through analysis.

According to the model number and the corresponding probability of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model number of the sample equipment calibrated by each sample equipment environment image, parameter adjustment can be performed on the recognition model to be trained until a preset condition is met (for example, training is performed for a preset number of times or the error of the model number of the predicted sample equipment output by the model and the model number of the sample equipment is smaller than a preset threshold value, etc.), so as to obtain a trained preset model number recognition model. Furthermore, the device environment image can be analyzed based on the trained preset model identification model, so as to obtain at least one predicted device model and the probability corresponding to each predicted device model.

In one embodiment, the recognition model to be trained may be specifically constructed in the following manner: and constructing a brand new model sequentially comprising an image feature extraction network layer and an image feature classification network layer as a recognition model to be trained.

Further, in an embodiment, the recognition model to be trained may be specifically constructed according to the following manner: acquiring a pre-trained image classification model; replacing the full-connection layer of the last layer of the image classification model with a new full-connection layer to obtain an updated image classification model; initializing the weight in the updated image classification model into the weight of the image classification model, and freezing the weights of all layers except the selected layer in the updated image classification model to obtain the recognition model to be trained.

In this embodiment, a pre-trained image classification model (such as VGG, resNet, inception) is selected and obtained, and because the pre-trained image classification model is already trained on a large-scale image dataset (such as ImageNet) and has good performance, the model is further constructed as a recognition model to be trained based on the method of the embodiment, and a preset model recognition model with high performance meeting the requirements of the application can be obtained based on the recognition model to be trained in an efficient manner.

Specifically, referring to fig. 2, in one embodiment, the step of training to obtain the preset model identification model may specifically include steps S210 to S270.

In step S210, a pre-trained image classification model, such as VGG, resNet, acceptance, etc., is selected. The pre-trained image classification model has been trained on large-scale image datasets (such as ImageNet) and has good performance.

Step S220, replacing the full connection layer. Specifically, the full-connection layer of the last layer of the image classification model is replaced by a new full-connection layer, so that an updated image classification model is obtained, wherein the full-connection layer is replaced by the new full-connection layer to adapt to a training set implemented by the application;

step S230, initializing model weights. Specifically, the weights in the updated image classification model are initialized to the weights of the image classification model. The weights of the model refer to the parameters of the trained model between the various layers, which determine the learning and prediction capabilities of the model.

Step S240, the parameters are frozen. Specifically, the weights of all layers except the selected layer in the updated image classification model are frozen so as to avoid resetting the weights of the pre-trained image classification model, and the recognition model to be trained is obtained after freezing.

In step S250, training super-parameters such as learning rate, batch size, etc. may be adjusted, and cross-validation techniques may be used to select appropriate super-parameters.

Step S260, training. Specifically, a to-be-trained identification model is adopted to analyze a plurality of sample equipment environment images, and a predicted sample equipment model corresponding to each sample equipment environment image is obtained.

Step S270, evaluating model performance. Specifically, calculating and analyzing the error of the model of the sample equipment corresponding to each sample equipment environment image and the model of the sample equipment calibrated by each sample equipment environment image.

Step S280, judging whether the expected situation is met. Specifically, it may be determined whether an error corresponding to the environmental image of the sample device in the predetermined proportion is smaller than a predetermined threshold, if so, the expected error is satisfied, and if not, the expected error is not satisfied.

In step S290, the model parameters are fine-tuned. In particular, if not expected, the parameters of the "selected layer" in the recognition model to be trained may be fine-tuned, which may typically include the last new fully connected layer and the penultimate layer in particular. Then, the process advances to step S260.

In step S2100, a model is derived. Specifically, if the model is in accordance with the expectation, a trained recognition model to be trained is derived and used as a preset model recognition model.

Further, in some embodiments, the obtaining the target identification information according to preset identification information corresponding to a preset equipment model matched with the target equipment model in the candidate equipment list includes: if the preset identification information corresponding to the preset equipment model matched with the target equipment model is one, determining the preset identification information corresponding to the preset equipment model matched with the target equipment model as the target identification information; if the number of the preset identification information corresponding to the preset equipment model matched with the target equipment model is at least two, prompting a user to select, and obtaining the target identification information according to the user selection.

For example, if the target device model is 2000000000N, the preset identification information corresponding to the preset device model 2000000000N matched with the target device model 2000000000N includes "{" DeviceID ": 10000001", "DeviceName": "main air conditioner" }, { "DeviceID": 10000002"," DeviceName ":" sub-air conditioner "}, {" DeviceID ": 10000003", "DeviceName": "study air conditioner" }. The user can select the target identification information through popup window, page switching or other prompting modes, and the preset identification information selected by the user can be determined to be the target identification information.

For example, if the target device model is 2000000000S, the preset identification information corresponding to the preset device model 2000000000S matched with the target device model 2000000000S includes { "DeviceID": "10000004", "DeviceName": "living room air conditioner" }, and { "DeviceID": "10000004", "DeviceName": "living room air conditioner" } may be determined as the target identification information.

In order to better implement the device control method provided in the embodiments of the present application, the foregoing embodiments are further described below in conjunction with a flow of device control in one scenario. In this scenario, the device control flow is performed by applying some embodiments of the present application, referring to fig. 3 and fig. 4, where the device control flow includes steps S310 to S390.

Step S310, a training set is acquired.

Specifically, a training set is collected in advance, the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are collected and comprise equipment surrounding environment images of sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of a corresponding sample equipment.

Step S320, training.

Specifically, analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image; and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

Step S330, start identification. The user can trigger the process of identifying the equipment identification information of the equipment to be controlled on the main control equipment through related operation.

Step S340, querying the user' S list of bound devices.

Specifically, in some modes, control user information in the master control equipment is sent to a server; receiving first binding equipment information sent by the server, wherein the first binding equipment information comprises equipment related information of candidate equipment bound by the control user information, and the first binding equipment information is inquired by the server according to the control user information; and obtaining a first binding device list according to the first binding device information, wherein the first binding device list comprises corresponding preset device models and preset identification information of the candidate devices bound by the control user information.

Optionally, in other modes, sending a control scene and control user information to the server; receiving second binding equipment information sent by the server, wherein the second binding equipment information comprises equipment related information of candidate equipment which is bound by the control user information and accords with the control scene, and the second binding equipment information is inquired by the server according to the control scene and the control user information; and obtaining a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene.

Step S350, caching the candidate device list, i.e. caching the candidate device list locally.

Step S360, probability screening setting.

Specifically, probability screening setting is performed on the preset model identification model, so that a set model identification model is obtained, and the probability identification of the set model identification model to other models except the preset equipment model included in the candidate equipment list is zero.

In step S370, the camera takes an image, and in this scenario, the master control device may acquire an environmental image of the multi-frame device through its own camera under the operation of the user.

Step S380, a frame is read, and a frame of equipment environment image is read from the acquired multi-frame equipment environment image.

Step S390, recognition. Specifically, the set model identification model is adopted to analyze the environment image of the equipment.

Step S3100, determining whether there is a result, specifically, determining whether the model identification model after setting analyzes the device environment image, and obtaining the predicted device model.

Step S3110, determining a target device model, specifically, if the set model identification model is used to analyze the device environment image to obtain at least one predicted device model and a probability corresponding to each predicted device model, then obtaining the target device model according to the at least one predicted device model and the probability corresponding to each predicted device model, and specifically, using a predicted device model with the highest probability as the obtained target device model.

In step S3120, the device list is queried, specifically, preset identification information corresponding to a preset device model matched with the target device model is queried from the candidate device list.

In step S3130, it is determined whether the number of the list is equal to 1, specifically, whether the preset identification information corresponding to the preset device model matched with the target device model is one.

In step S3140, the target identification information is directly determined. Specifically, if the preset identification information corresponding to the preset equipment model matched with the target equipment model is one, the preset identification information corresponding to the preset equipment model matched with the target equipment model is determined to be the target identification information.

In step S3150, prompt selection, specifically, if the preset identification information corresponding to the preset device model matched with the target device model is at least two, prompt the user to select, and obtain the target identification information according to the user selection.

In step S3160, identification information is returned, and specifically, the target identification information may be displayed in a predetermined page.

In this scene, through the equipment control of some embodiments that apply this application, can effectively reduce the identification information discernment of controlled equipment (waiting to control equipment) consuming time on the whole, promote equipment control efficiency, promote user experience.

In order to facilitate better implementation of the device control method provided by the embodiment of the application, the embodiment of the application also provides a device control device based on the device control method. Where the meaning of the terms is the same as in the above-described device control method, specific implementation details may be referred to in the description of the method embodiments. Fig. 5 shows a block diagram of a device control apparatus according to one embodiment of the present application.

As shown in fig. 5, the device control apparatus 400 may include: the list obtaining module 410 may be configured to obtain a candidate device list, where the candidate device list includes preset device models and preset identification information corresponding to a plurality of candidate devices; the image obtaining module 420 may be configured to obtain an equipment environment image, where the equipment environment image is an acquired equipment ambient environment image including equipment to be controlled; the model analysis module 430 may be configured to analyze the device environment image by using a preset model identification model to obtain a target device model; the identification determining module 440 may be configured to obtain target identification information according to preset identification information corresponding to a preset device model matched with the target device model in the candidate device list, where the target identification information is used to control the device to be controlled.

It should be noted that although in the above detailed description several modules or units of a device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functions of two or more modules or units described above may be embodied in one module or unit, in accordance with embodiments of the present application. Conversely, the features and functions of one module or unit described above may be further divided into a plurality of modules or units to be embodied.

In addition, the embodiment of the application further provides an electronic device, as shown in fig. 6, which shows a schematic structural diagram of the electronic device according to the embodiment of the application, specifically:

The electronic device may include one or more processing cores 'processors 501, one or more computer-readable storage media's memory 502, a power supply 503, and an input unit 504, among other components. It will be appreciated by those skilled in the art that the electronic device structure shown in fig. 6 is not limiting of the electronic device and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

Wherein:

the processor 501 is a control center of the electronic device, and connects various parts of the entire computer device using various interfaces and lines, and performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 502, and calling data stored in the memory 502, thereby performing overall monitoring of the electronic device. Optionally, processor 501 may include one or more processing cores; preferably, the processor 501 may integrate an application processor and a modem processor, wherein the application processor primarily handles operating systems, user pages, applications, etc., and the modem processor primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 501.

The memory 502 may be used to store software programs and modules, and the processor 501 executes various functional applications and data processing by executing the software programs and modules stored in the memory 502. The memory 502 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide access to the memory 502 by the processor 501.

The electronic device further comprises a power supply 503 for powering the various components, preferably the power supply 503 is logically connected to the processor 501 via a power management system, whereby the functions of managing charging, discharging, and power consumption are performed by the power management system. The power supply 503 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.

The electronic device may further comprise an input unit 504, which input unit 504 may be used for receiving input digital or character information and for generating keyboard, mouse, joystick, optical or trackball signal inputs in connection with user settings and function control.

Although not shown, the electronic device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 501 in the electronic device loads executable files corresponding to the processes of one or more computer programs into the memory 502 according to the following instructions, and the processor 501 executes the computer programs stored in the memory 502, so as to implement the functions in the foregoing embodiments of the present application, where the processor 501 may perform the following steps:

acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices; acquiring an equipment environment image, wherein the equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled; analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model; and obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, wherein the target identification information is used for controlling the equipment to be controlled.

In some embodiments of the present application, further comprising: acquiring a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are acquired equipment surrounding environment images containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of corresponding sample equipment; analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image; and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

In some embodiments of the present application, further comprising: acquiring a pre-trained image classification model; replacing the full-connection layer of the last layer of the image classification model with a new full-connection layer to obtain an updated image classification model; initializing the weight in the updated image classification model into the weight of the image classification model, and freezing the weights of all layers except the selected layer in the updated image classification model to obtain the recognition model to be trained.

It will be appreciated by those of ordinary skill in the art that all or part of the steps of the various methods of the above embodiments may be performed by a computer program, or by computer program control related hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.

To this end, the present embodiments also provide a storage medium having stored therein a computer program that can be loaded by a processor to perform the steps of any of the methods provided by the embodiments of the present application.

Wherein the storage medium may be a computer-readable storage medium, the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.

Since the computer program stored in the storage medium may perform any of the steps in the method provided in the embodiment of the present application, the beneficial effects that can be achieved by the method provided in the embodiment of the present application may be achieved, which are detailed in the previous embodiments and are not described herein.

Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of the application following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the application pertains.

It will be understood that the present application is not limited to the embodiments that have been described above and shown in the drawings, but that various modifications and changes can be made without departing from the scope thereof.

Claims

1. A device control method, characterized by comprising:

acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices;

acquiring an equipment environment image, wherein the equipment environment image is an acquired equipment surrounding environment image containing equipment to be controlled;

analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model;

and obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, wherein the target identification information is used for controlling the equipment to be controlled.

2. The method of claim 1, wherein the analyzing the device environment image using the preset model identification model to obtain the target device model comprises:

performing probability screening setting on the preset model identification model to obtain a set model identification model, wherein the probability identification of the set model identification model on other models except the preset equipment model included in the candidate equipment list is zero;

Analyzing the equipment environment image by adopting the set model identification model to obtain at least one predicted equipment model and the probability corresponding to each predicted equipment model;

and obtaining the target equipment model according to the at least one predicted equipment model and the probability corresponding to each predicted equipment model.

3. Method according to claim 1 or 2, characterized in that the pre-set model identification model is trained in the following way:

acquiring a training set, wherein the training set comprises a plurality of sample equipment environment images, the sample equipment environment images are acquired equipment surrounding environment images containing sample equipment, and each sample equipment environment image is used for calibrating a sample equipment model of corresponding sample equipment;

analyzing the plurality of sample equipment environment images by adopting an identification model to be trained to obtain a predicted sample equipment model corresponding to each sample equipment environment image;

and carrying out parameter adjustment on the recognition model to be trained according to the model of the predicted sample equipment corresponding to each sample equipment environment image and the error of the model of the sample equipment calibrated by each sample equipment environment image until the recognition model to be trained meets the preset condition, and obtaining the trained recognition model of the preset model.

4. A method according to claim 3, wherein the recognition model to be trained is constructed in the following manner:

acquiring a pre-trained image classification model;

replacing the full-connection layer of the last layer of the image classification model with a new full-connection layer to obtain an updated image classification model;

initializing the weight in the updated image classification model into the weight of the image classification model, and freezing the weights of all layers except the selected layer in the updated image classification model to obtain the recognition model to be trained.

5. The method of claim 1, wherein the list of candidate devices comprises a first list of bound devices; the obtaining the candidate device list includes:

transmitting control user information in the main control equipment to a server;

receiving first binding equipment information sent by the server, wherein the first binding equipment information comprises equipment related information of candidate equipment bound by the control user information, and the first binding equipment information is inquired by the server according to the control user information;

and obtaining a first binding device list according to the first binding device information, wherein the first binding device list comprises corresponding preset device models and preset identification information of the candidate devices bound by the control user information.

6. The method of claim 1, wherein the list of candidate devices comprises a second list of bound devices; the obtaining the candidate device list includes:

sending control scenes and control user information to a server;

receiving second binding equipment information sent by the server, wherein the second binding equipment information comprises equipment related information of candidate equipment which is bound by the control user information and accords with the control scene, and the second binding equipment information is inquired by the server according to the control scene and the control user information;

and obtaining a second binding device list according to the second binding device information, wherein the second binding device list comprises corresponding preset device models and preset identification information of candidate devices which are bound by the control user information and accord with the control scene.

7. The method according to claim 1, wherein the obtaining the target identification information according to the preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list includes:

if the preset identification information corresponding to the preset equipment model matched with the target equipment model is one, determining the preset identification information corresponding to the preset equipment model matched with the target equipment model as the target identification information;

If the number of the preset identification information corresponding to the preset equipment model matched with the target equipment model is at least two, prompting a user to select, and obtaining the target identification information according to the user selection.

8. An apparatus control device, comprising:

the list acquisition module is used for acquiring a candidate device list, wherein the candidate device list comprises preset device models and preset identification information corresponding to a plurality of candidate devices;

the image acquisition module is used for acquiring an equipment environment image, wherein the equipment environment image is acquired equipment surrounding environment image containing equipment to be controlled;

the model analysis module is used for analyzing the equipment environment image by adopting a preset model identification model to obtain a target equipment model;

the identification determining module is used for obtaining target identification information according to preset identification information corresponding to the preset equipment model matched with the target equipment model in the candidate equipment list, and the target identification information is used for controlling the equipment to be controlled.

9. A storage medium having stored thereon a computer program which, when executed by a processor of a computer, causes the computer to perform the method of any of claims 1 to 7.

10. An electronic device, comprising: a memory storing a computer program; a processor reading a computer program stored in a memory to perform the method of any one of claims 1 to 7.