CN113194292A - Coordination management method, device, equipment and storage medium for intelligent equipment - Google Patents
Coordination management method, device, equipment and storage medium for intelligent equipment Download PDFInfo
- Publication number
- CN113194292A CN113194292A CN202110480244.4A CN202110480244A CN113194292A CN 113194292 A CN113194292 A CN 113194292A CN 202110480244 A CN202110480244 A CN 202110480244A CN 113194292 A CN113194292 A CN 113194292A
- Authority
- CN
- China
- Prior art keywords
- picture
- scene
- coordination
- video data
- image processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000007726 management method Methods 0.000 title claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 87
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000008859 change Effects 0.000 claims abstract description 34
- 238000012544 monitoring process Methods 0.000 claims abstract description 24
- 230000005540 biological transmission Effects 0.000 claims abstract description 22
- 238000010606 normalization Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 8
- 238000004458 analytical method Methods 0.000 claims description 6
- 239000000126 substance Substances 0.000 claims description 2
- 238000005516 engineering process Methods 0.000 abstract description 2
- 230000006870 function Effects 0.000 description 19
- 230000004913 activation Effects 0.000 description 10
- 238000004891 communication Methods 0.000 description 10
- 238000012806 monitoring device Methods 0.000 description 7
- 238000012549 training Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 238000003062 neural network model Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000001914 filtration Methods 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/28—Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
- H04L12/2803—Home automation networks
- H04L12/2816—Controlling appliance services of a home automation network by calling their functionalities
Abstract
The invention relates to a cloud transmission technology, and discloses a coordination management method of intelligent equipment, which comprises the following steps: receiving video data shot by monitoring equipment in real time, and judging whether picture element change occurs in the video data; intercepting the video data according to a preset intercepting frequency within a time period when picture elements in the video data change to obtain a picture set; identifying picture characteristics in the picture set by using the image processing model, and analyzing to obtain an event scene corresponding to the picture set according to the picture characteristics; and inquiring a coordination command corresponding to the event scene in the scene coordination rule according to the scene coordination rule, and sending the coordination command to preset intelligent equipment. The invention also relates to blockchain techniques, the picture features may be stored in blockchain nodes. The invention also provides a coordination management device of the intelligent equipment, the equipment and a computer readable storage medium. The invention aims to solve the problem that the traditional gateway box cannot perform cooperative management on a large number of intelligent devices.
Description
Technical Field
The invention relates to the technical field of cloud transmission, in particular to a coordination management method and device for intelligent equipment, electronic equipment and a computer-readable storage medium.
Background
With the development of the times, intelligent devices gradually acquire the capability of recognizing event scenes. For example, the intelligent home can judge the character or gesture information of a door opener according to a camera in a corridor, select to open a door or give an alarm, and select to automatically open indoor light according to a time scene after the door is opened. The intelligent device generally identifies the event scene through a gateway box, wherein the gateway box comprises various data conversion protocols, data in various types of monitoring devices can be converted into a specified data format, the gateway box can transmit the data to a cloud terminal in a wired and wireless mode to perform deep processing on the data, and converts a command sent by the cloud terminal and then sends the command to the intelligent device to perform command execution.
However, the gateway box needs to transmit video data shot by the monitoring devices to the cloud in real time for cloud identification or storage, and when the amount of video data received from the monitoring devices increases or the number of connected monitoring devices increases, the bandwidth occupied by data transmission in the gateway box gradually increases, so that the data uploading or downloading rate is reduced, and some intelligent devices cannot receive commands in time, and therefore the traditional gateway box can only manage few intelligent devices, and is not convenient for simultaneous management of multiple intelligent devices.
Disclosure of Invention
The invention provides a coordination management method and device for intelligent equipment, electronic equipment and a computer readable storage medium, and aims to solve the problem that a traditional gateway box cannot perform coordination management on a large number of intelligent equipment.
In order to achieve the above object, the present invention provides a method for coordinating and managing an intelligent device, where the method is applied in a gateway box and includes:
receiving video data shot by monitoring equipment in real time, and judging whether picture element change occurs in the video data;
intercepting the video data according to a preset interception frequency within a time period when picture elements in the video data change to obtain a picture set;
identifying picture features in the picture set by using a pre-trained image processing model, and analyzing and obtaining an event scene corresponding to the picture set according to the picture features;
and inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule, and sending the coordination command to a preset intelligent device.
Optionally, the determining whether a picture element change occurs in the video data includes:
monitoring the data flow of the video data by using a data transmission monitoring service at any moment;
and when the data flow changes, judging that the picture elements in the video data change.
Optionally, the analyzing the event scene corresponding to the picture set according to the picture features includes:
judging the probability of the picture features belonging to each scene feature according to the scene features corresponding to each preset event scene, and carrying out normalization operation on the probability to obtain a normalization value set;
judging whether the maximum normalized value in the normalized value set is greater than or equal to a preset first threshold value or not;
when the maximum normalization value is smaller than the first threshold value, judging that the picture features do not belong to the scene features;
and when the maximum normalization value is larger than or equal to the first threshold value, judging that the picture features belong to the scene features corresponding to the maximum normalization value, and obtaining the event scene corresponding to the picture set according to the scene features.
Optionally, the querying, according to a pre-constructed scene collaboration rule, a collaboration command corresponding to the event scene in the scene collaboration rule includes:
analyzing a collaborative method corresponding to the event scene;
and inquiring one or more collaboration commands corresponding to the collaboration method from the collaboration command set corresponding to the scene collaboration rule.
Optionally, before the image features in the image set are identified by using the pre-trained image processing model, the method further includes:
step I, acquiring a local image set;
step II, performing feature extraction on the local image set by using a data input layer of a pre-constructed image processing model to obtain a local image feature sequence set;
step III, carrying out feature analysis on the local image feature sequence set by utilizing a hidden layer of the image processing model to obtain a prediction result set;
step IV, calculating the difference value of the scene labels corresponding to the prediction result set and the local image set to obtain an error value;
and V, when the error value is judged not to be converged, updating the parameters of the image processing model, and returning to the step II until the error value is converged to obtain the trained image processing model.
Optionally, after obtaining the trained image processing model, the method further includes:
connecting the image processing model into the gateway box using an application programming interface.
Optionally, after the picture features in the picture set are identified by using the pre-trained image processing model, the method further includes:
and sending the picture characteristics to a pre-constructed cloud Internet of things platform for storage.
In order to solve the above problem, the present invention further provides a coordination management apparatus for an intelligent device, the apparatus including:
the data acquisition module is used for receiving video data shot by monitoring equipment in real time, judging whether picture elements in the video data change or not, and intercepting the video data according to a preset interception frequency in a time period when the picture elements in the video data change to obtain a picture set;
the data processing module is used for identifying picture characteristics in the picture set by using a pre-trained image processing model and analyzing and obtaining an event scene corresponding to the picture set according to the picture characteristics;
and the scene coordination module is used for inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule and sending the coordination command to a preset intelligent device.
In order to solve the above problem, the present invention also provides an electronic device, including:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores computer program instructions executable by the at least one processor to enable the at least one processor to perform the method of coordinating management of smart devices described above.
In order to solve the above problem, the present invention further provides a computer-readable storage medium including a storage data area and a storage program area, the storage data area storing created data, the storage program area storing a computer program; wherein, the computer program realizes the coordination management method of the intelligent device when being executed by a processor.
The embodiment of the invention only intercepts the video data with changed picture elements, reduces the size of the video data acquired from the monitoring equipment and relieves the occupation amount of transmission bandwidth. Through the image processing model loaded in advance, event scene recognition and local scene cooperative processing can be carried out on the intercepted image set, and only the image features after local processing need to be transmitted to the cloud, so that the data volume is further reduced, and the occupation of data transmission on transmission bandwidth is reduced. Therefore, the embodiment of the invention can solve the problem that the traditional gateway box cannot perform cooperative management on a large number of intelligent devices.
Drawings
Fig. 1 is a schematic flowchart of a coordination management method for an intelligent device according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating a detailed step of the coordination management method for the intelligent device shown in FIG. 1;
fig. 3 is a schematic block diagram of a coordination management apparatus of an intelligent device according to an embodiment of the present invention;
fig. 4 is a schematic internal structural diagram of an electronic device implementing a coordination management method for an intelligent device according to an embodiment of the present invention;
the implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
The embodiment of the application provides a coordination management method of intelligent equipment. The execution subject of the coordination management method of the intelligent device includes, but is not limited to, at least one of electronic devices that can be configured to execute the method provided by the embodiments of the present application, such as a server, a terminal, and the like. In other words, the coordination management method of the intelligent device may be performed by software or hardware installed in the terminal device or the server device, and the software may be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Fig. 1 is a schematic flowchart of a coordination management method for an intelligent device according to an embodiment of the present invention. In this embodiment, the method for coordinating and managing the smart device is applied to a gateway box. The gateway box is a device which has the functions of carrying out data protocol conversion, edge calculation, information transmission and the like on edge data. The gateway box comprises an access layer, a processing layer and a transmission layer, wherein the access layer is provided with a plurality of communication interfaces and used for connecting different terminal devices such as monitoring devices or intelligent devices, the processing layer is provided with an operation engine and a resource space and used for carrying out local calculation, the transmission layer is provided with a data conversion protocol and can convert the data protocol of the monitoring devices and a cloud protocol and transmit the converted data to a cloud for storage.
In one embodiment of the present invention, the method for coordinating and managing the intelligent device includes:
and S1, receiving the video data shot by the monitoring equipment in real time, and judging whether picture element change occurs in the video data.
In the embodiment of the invention, whether the picture element change occurs in the video data can be judged through a frequency dynamic intercepting interface. The video dynamic intercepting interface is a data receiving interface obtained by adding a judgment function on a communication interface of the gateway box.
In the embodiment of the invention, the monitoring equipment can acquire the video data in real time, and when picture elements in the video data change, the data flow of data transmission changes due to the change of the picture elements. The embodiment of the invention constantly observes the change of the data flow through the video dynamic intercepting interface, and judges that the picture element of the video data changes when the fluctuation condition of the data flow changes suddenly.
In detail, the determining whether a picture element change occurs in the video data includes:
monitoring the data flow of the video data by using a data transmission monitoring service at any moment;
and when the data flow changes, judging that the picture elements in the video data change.
The embodiment of the invention can distinguish the video data shot by the monitoring equipment by utilizing the video dynamic intercepting interface, thereby filtering a large amount of useless static video data, only reserving the video data with changed picture elements and reducing the data volume of subsequent local processing.
And S2, intercepting the video data according to a preset intercepting frequency in a time period when picture elements in the video data change to obtain a picture set.
After the video image with the changed picture elements is obtained, the video data is intercepted according to the intercepting frequency of intercepting once every 0.5 second, and the picture set is obtained.
According to the image processing method and device, the data processing amount of subsequent image recognition can be reduced through the image set.
S3, recognizing picture features in the picture set by using a pre-trained image processing model, and analyzing to obtain an event scene corresponding to the picture set according to the picture features.
In the embodiment of the invention, the image processing model is a neural network model which can extract the characteristic information in the picture and output the characteristic information by scene classification. For example, in the embodiment of the present invention, the image processing model is a neural network model of a TransForm model framework.
In detail, as shown in fig. 2, in the embodiment of the present invention, the analyzing the event scene corresponding to the picture set according to the picture feature includes:
s31, judging the probability that the picture features belong to each scene feature according to the scene features corresponding to each preset event scene, and carrying out normalization operation on the probability to obtain a normalization value set;
s32, judging whether the maximum normalized value in the normalized value set is larger than or equal to a preset first threshold value or not;
s33, when the maximum normalization value is smaller than the first threshold value, judging that the picture features do not belong to the scene features;
and S33, when the maximum normalization value is larger than or equal to the first threshold value, judging that the picture features belong to the scene features corresponding to the maximum normalization value, and obtaining the event scene corresponding to the picture set according to the scene features.
Specifically, in the embodiment of the present invention, the image processing model is used to perform feature extraction analysis on the image set to obtain an image feature set corresponding to the image set, and the image feature set is compared with scene features corresponding to a preset event scene to obtain probabilities that the image feature set belongs to each scene feature, the probabilities are ranked, and the event scene corresponding to the image set is obtained according to the scene features corresponding to the maximum probabilities.
In detail, in an embodiment of the present invention, before the identifying, by using the pre-trained image processing model, the features appearing in the picture set, the method further includes:
step I, acquiring a local image set;
and II, performing feature extraction on the local image set by using a data input layer of a pre-constructed image processing model to obtain a local image feature sequence set.
According to the embodiment of the invention, a local image is obtained from the local image set through traversal operation, the local image is input into an activation function of a layer, and the local image is subjected to feature extraction to obtain a local image feature sequence set corresponding to the local image.
And III, performing feature analysis on the local image feature sequence set by utilizing a hidden layer of the image processing model to obtain a prediction result set.
The embodiment of the invention firstly arranges and identifies the local image feature sequence through an activation function of a hidden layer of the image processing model to obtain a preliminary scene of the local image feature sequence. And carrying out more detailed division on the preliminary scene by utilizing an activation function of the image processing model output layer, wherein the division types are types of various preset scene characteristics.
Step IV, calculating the difference value of the scene labels corresponding to the prediction result set and the local image set to obtain an error value;
and V, when the error value is judged not to be converged, updating the parameters of the image processing model, and returning to the step II until the error value is converged to obtain the trained image processing model.
The embodiment of the invention can judge the training progress of the image processing model to be trained by judging the error value, the loss function in the image processing model to be trained can update the parameters of the activation function in the image processing model to be trained by using the error value, when the error value is converged, the activation function is shown to be converged, the training process is finished, and the image processing model after training is obtained.
Further, in the embodiment of the present invention, after obtaining the trained image processing model, the method may further include:
connecting the image processing model into the gateway box using an application programming interface.
The embodiment of the present invention introduces the image processing model into a processing layer loaded in the gateway box through the Application Programming Interface, and enables the image processing model to be connected to a local operation engine and a resource space of the gateway box.
S4, inquiring a coordination command corresponding to the event scene in the scene coordination rule according to the pre-constructed scene coordination rule, and sending the coordination command to a preset intelligent device.
In the embodiment of the invention, the local scene rule is a pre-constructed rule table for executing different actions in different event scenes, for example, when the door is opened, a character and a gesture of the character are identified, and when the identification is passed, the door is opened and the household appliance is switched to a specified state according to a preset cooperation rule, for example, lights of a living room and an air conditioner are opened; when the images do not pass the identification, the images are uploaded to the cloud end for storage, and a user can check the door opening dynamic state through a pre-constructed mobile terminal.
In detail, in the embodiment of the present invention, the querying, according to the pre-constructed scene collaboration rule, the collaboration command corresponding to the event scene in the scene collaboration rule includes:
analyzing a collaborative method corresponding to the event scene;
and inquiring one or more collaboration commands corresponding to the collaboration method from the collaboration command set corresponding to the scene collaboration rule.
In the embodiment of the present invention, a coordination command set corresponding to the scene coordination rule is obtained, when it is determined that the picture set triggers an event scene in the scene coordination rule, a coordination method corresponding to the event scene is obtained, and according to the coordination method, one or more coordination commands corresponding to the coordination method are/is queried from the coordination command set.
The embodiment of the invention only intercepts the video data with changed picture elements, reduces the size of the video data acquired from the monitoring equipment and relieves the occupation amount of transmission bandwidth. Through the image processing model loaded in advance, event scene recognition and local scene cooperative processing can be carried out on the intercepted image set, and only the image features after local processing need to be transmitted to the cloud, so that the data volume is further reduced, and the occupation of data transmission on transmission bandwidth is reduced. Therefore, the embodiment of the invention can solve the problem that the traditional gateway box cannot perform cooperative management on a large number of intelligent devices.
Fig. 3 is a schematic block diagram of a coordination management apparatus for an intelligent device according to an embodiment of the present invention.
The coordination management apparatus 100 of the smart device according to the present invention may be installed in an electronic device. According to the implementation steps, the coordination management apparatus of the intelligent device may include a data acquisition module 101, a data processing module 102, and a scene coordination module 103. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and perform certain steps, and that are stored in a memory of the electronic device.
In this embodiment, the steps for each module/unit are as follows:
the data acquisition module 101 is configured to receive video data shot by a monitoring device in real time, determine whether a picture element in the video data changes, and capture the video data according to a preset capture frequency in a time period in which the picture element in the video data changes, so as to obtain a picture set.
In the apparatus of the present invention, the data acquisition module 101 includes a short video acquisition module and a picture acquisition module.
The short video acquisition module is used for receiving video data shot by monitoring equipment in real time and judging whether picture element change occurs in the video data.
In the embodiment of the invention, the video dynamic intercepting interface is a data receiving interface obtained by adding a judging function on a communication interface of a gateway box, and whether the change of picture elements occurs in the video data can be judged through the video dynamic intercepting interface.
In the embodiment of the invention, the monitoring equipment can acquire the video data in real time, and when picture elements in the video data change, the data flow of data transmission changes due to the change of the picture elements. The embodiment of the invention constantly observes the change of the data flow through the video dynamic intercepting interface, and judges that the picture element of the video data changes when the fluctuation condition of the data flow shakes.
In detail, in the embodiment of the present invention, when determining whether a picture element change occurs in the video data, the data obtaining module 101 specifically executes:
monitoring the data flow of the video data by using a data transmission monitoring service at any moment;
and when the data flow changes, judging that the picture elements in the video data change.
The embodiment of the invention can distinguish the video data shot by the monitoring equipment by utilizing the video dynamic intercepting interface, thereby filtering a large amount of useless static video data, only reserving the video data with changed picture elements and reducing the data volume of subsequent local processing.
The picture acquisition module is used for intercepting the video data according to a preset interception frequency in a time period when picture elements in the video data change, so as to obtain a picture set.
After the video image with the changed picture elements is obtained, the video data is intercepted according to the intercepting frequency of intercepting once every 0.5 second, and the picture set is obtained.
According to the image processing method and device, the data processing amount of subsequent image recognition can be reduced through the image set.
The data processing module 102 is configured to identify picture features in the picture set by using a pre-trained image processing model, and analyze an event scene corresponding to the picture set according to the picture features.
In the embodiment of the invention, the image processing model is a neural network model which can extract the characteristic information in the picture and output the characteristic information by scene classification. For example, the image processing model is a neural network model of a TransForm model framework.
In detail, in the embodiment of the present invention, when the data processing module 102 analyzes and obtains the event scene corresponding to the picture set according to the picture feature, specifically:
judging the probability of the picture features belonging to each scene feature according to the scene features corresponding to each preset event scene, and carrying out normalization operation on the probability to obtain a normalization value set;
judging whether the maximum normalized value in the normalized value set is greater than or equal to a preset first threshold value or not;
when the maximum normalization value is smaller than the first threshold value, judging that the picture features do not belong to the scene features;
and when the maximum normalization value is larger than or equal to the first threshold value, judging that the picture features belong to the scene features corresponding to the maximum normalization value, and obtaining the event scene corresponding to the picture set according to the scene features.
Specifically, in the embodiment of the present invention, the image processing model is used to perform feature extraction analysis on the image set to obtain an image feature set corresponding to the image set, and the image feature set is compared with scene features corresponding to a preset event scene to obtain probabilities that the image feature set belongs to each scene feature, the probabilities are ranked, and the event scene corresponding to the image set is obtained according to the scene features corresponding to the maximum probabilities.
In detail, in this embodiment of the present invention, before the data processing module 102 identifies the features appearing in the picture set by using the pre-trained image processing model, the following steps are further performed:
step A, obtaining a local image set;
and B, performing feature extraction on the local image set by using a data input layer of a pre-constructed image processing model to obtain a local image feature sequence set.
According to the embodiment of the invention, a local image is obtained from the local image set through traversal operation, the local image is input into an activation function of a layer, and the local image is subjected to feature extraction to obtain a local image feature sequence set corresponding to the local image.
And C, carrying out feature analysis on the local image feature sequence set by utilizing a hidden layer of the image processing model to obtain a prediction result set.
According to the embodiment of the invention, the activation function of the hidden layer is firstly used for sorting and identifying the local image feature sequence to obtain the initial scene of the local image feature sequence. And performing more detailed division on the preliminary scene by using the activation function of the output layer, wherein the division types are types of various preset scene characteristics.
D, performing difference value calculation on the scene labels corresponding to the prediction result set and the local image set to obtain an error value;
and E, when the error value is judged not to be converged, updating the parameters of the image processing model, and returning to the step B until the error value is converged, so as to obtain the trained image processing model.
The embodiment of the invention can judge the training progress of the image processing model to be trained by judging the error value, the loss function in the image processing model to be trained can update the parameters of the activation function in the image processing model to be trained by using the error value, when the error value is converged, the activation function is shown to be converged, the training process is finished, and the image processing model after training is obtained.
Further, in this embodiment of the present invention, after obtaining the trained image processing model, the data processing module 102 further needs to perform:
connecting the image processing model into the gateway box using an application programming interface.
The embodiment of the present invention introduces the image processing model into a processing layer loaded in the gateway box through the Application Programming Interface, and enables the image processing model to be connected to a local operation engine and a resource space of the gateway box.
The scene collaboration module 103 is configured to query, according to a pre-constructed scene collaboration rule, a collaboration command corresponding to the event scene in the scene collaboration rule, and send the collaboration command to a preset intelligent device.
In the embodiment of the invention, the local scene rule is a pre-constructed rule table for executing different actions according to different event scenes, for example, when the door is opened, a character and a gesture of the character are identified, and when the identification is passed, the door is opened and the household appliance is switched to a specified state according to the preset collaborative rule, for example, light of a living room and an air conditioner are opened; when the images do not pass the identification, the images are uploaded to the cloud end for storage, and a user can check the door opening dynamic state through a pre-constructed mobile terminal.
In detail, in the embodiment of the present invention, when querying, according to a pre-constructed scene collaboration rule, a collaboration command corresponding to the event scene in the scene collaboration rule, the scene collaboration module 103 specifically executes:
analyzing a collaborative method corresponding to the event scene;
and inquiring one or more collaboration commands corresponding to the collaboration method from the collaboration command set corresponding to the scene collaboration rule.
In the embodiment of the present invention, a coordination command set corresponding to the scene coordination rule is obtained, when it is determined that the picture set triggers an event scene in the scene coordination rule, a coordination method corresponding to the event scene is obtained, and according to the coordination method, one or more coordination commands corresponding to the coordination method are/is queried from the coordination command set.
The embodiment of the invention only intercepts the video data with changed picture elements, reduces the size of the video data acquired from the monitoring equipment and relieves the occupation amount of transmission bandwidth. Through the image processing model loaded in advance, event scene recognition and local scene cooperative processing can be carried out on the intercepted image set, and only the image features after local processing need to be transmitted to the cloud, so that the data volume is further reduced, and the occupation of data transmission on transmission bandwidth is reduced. Therefore, the embodiment of the invention can solve the problem that the traditional gateway box cannot perform cooperative management on a large number of intelligent devices.
Fig. 4 is a schematic structural diagram of an electronic device implementing the coordination management method for an intelligent device according to the present invention.
The electronic device may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may further include a computer program, such as a coordination management program of an intelligent device, stored in the memory 11 and executable on the processor 10.
In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same function or different functions, and includes one or more Central Processing Units (CPUs), a microprocessor, a digital Processing chip, a graphics processor, a combination of various control chips, and the like. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device by running or executing programs or modules (e.g., executing a coordination management program of a smart device, etc.) stored in the memory 11 and calling data stored in the memory 11.
The memory 11 includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device. The memory 11 may be used to store not only application software installed in the electronic device and various types of data, such as codes of a coordination management program of the smart device, but also temporarily store data that has been output or will be output.
The communication bus 12 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
The communication interface 13 is used for communication between the electronic device and other devices, and includes a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), which are typically used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable, among other things, for displaying information processed in the electronic device and for displaying a visualized user interface.
Fig. 4 shows only an electronic device having components, and those skilled in the art will appreciate that the structure shown in fig. 4 does not constitute a limitation of the electronic device, and may include fewer or more components than those shown, or some components may be combined, or a different arrangement of components.
For example, although not shown, the electronic device may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management and the like are realized through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The coordination management program of the intelligent device stored in the memory 11 of the electronic device is a combination of a plurality of computer programs, and when running in the processor 10, can realize:
receiving video data shot by monitoring equipment in real time, and judging whether picture element change occurs in the video data;
intercepting the video data according to a preset interception frequency within a time period when picture elements in the video data change to obtain a picture set;
identifying picture features in the picture set by using a pre-trained image processing model, and analyzing and obtaining an event scene corresponding to the picture set according to the picture features;
and inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule, and sending the coordination command to a preset intelligent device.
Specifically, the processor 10 may refer to the description of the relevant steps in the embodiment corresponding to fig. 1 for a specific implementation method of the computer program, which is not described herein again.
Further, the electronic device integrated module/unit, if implemented in the form of a software functional unit and sold or used as a separate product, may be stored in a non-volatile computer-readable storage medium. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
receiving video data shot by monitoring equipment in real time, and judging whether picture element change occurs in the video data;
intercepting the video data according to a preset interception frequency within a time period when picture elements in the video data change to obtain a picture set;
identifying picture features in the picture set by using a pre-trained image processing model, and analyzing and obtaining an event scene corresponding to the picture set according to the picture features;
and inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule, and sending the coordination command to a preset intelligent device.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
The block chain is a novel application mode of computer technologies such as distributed data storage, point-to-point transmission, a consensus mechanism, an encryption algorithm and the like. A block chain (Blockchain), which is essentially a decentralized database, is a series of data blocks associated by using a cryptographic method, and each data block contains information of a batch of network transactions, so as to verify the validity (anti-counterfeiting) of the information and generate a next block. The blockchain may include a blockchain underlying platform, a platform product service layer, an application service layer, and the like.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A coordination management method for intelligent equipment is characterized in that the method is applied to a gateway box and comprises the following steps:
receiving video data shot by monitoring equipment in real time, and judging whether picture element change occurs in the video data;
intercepting the video data according to a preset interception frequency within a time period when picture elements in the video data change to obtain a picture set;
identifying picture features in the picture set by using a pre-trained image processing model, and analyzing and obtaining an event scene corresponding to the picture set according to the picture features;
and inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule, and sending the coordination command to a preset intelligent device.
2. The method for coordinating and managing smart device according to claim 1, wherein the determining whether the picture element change occurs in the video data comprises:
monitoring the data flow of the video data by using a data transmission monitoring service at any moment;
and when the data flow changes, judging that the picture elements in the video data change.
3. The method for coordinating and managing smart device according to claim 1, wherein the analyzing the event scene corresponding to the picture set according to the picture features comprises:
judging the probability of the picture features belonging to each scene feature according to the scene features corresponding to each preset event scene, and carrying out normalization operation on the probability to obtain a normalization value set;
judging whether the maximum normalized value in the normalized value set is greater than or equal to a preset first threshold value or not;
when the maximum normalization value is smaller than the first threshold value, judging that the picture features do not belong to the scene features;
and when the maximum normalization value is larger than or equal to the first threshold value, judging that the picture features belong to the scene features corresponding to the maximum normalization value, and obtaining the event scene corresponding to the picture set according to the scene features.
4. The method for coordinating and managing smart device according to claim 1, wherein the querying, according to the pre-constructed scenario collaboration rule, the collaboration command corresponding to the event scenario in the scenario collaboration rule includes:
analyzing a collaborative method corresponding to the event scene;
and inquiring one or more collaboration commands corresponding to the collaboration method from the collaboration command set corresponding to the scene collaboration rule.
5. The method for coordinated management of smart devices as recited in claim 1, wherein prior to identifying picture features in the picture set using a pre-trained image processing model, the method further comprises:
step I, acquiring a local image set;
step II, performing feature extraction on the local image set by using a data input layer of a pre-constructed image processing model to obtain a local image feature sequence set;
step III, carrying out feature analysis on the local image feature sequence set by utilizing a hidden layer of the image processing model to obtain a prediction result set;
step IV, calculating the difference value of the scene labels corresponding to the prediction result set and the local image set to obtain an error value;
and V, when the error value is judged not to be converged, updating the parameters of the image processing model, and returning to the step II until the error value is converged to obtain the trained image processing model.
6. The method for coordinated management of intelligent devices according to claim 5, wherein after obtaining the trained image processing model, the method further comprises:
connecting the image processing model into the gateway box using an application programming interface.
7. The method for coordinated management of intelligent devices according to any one of claims 1 to 6, wherein after identifying picture features in the picture set using a pre-trained image processing model, the method further comprises:
and sending the picture characteristics to a pre-constructed cloud Internet of things platform for storage.
8. The coordination management device of the intelligent equipment is characterized by comprising:
the data acquisition module is used for receiving video data shot by monitoring equipment in real time, judging whether picture elements in the video data change or not, and intercepting the video data according to a preset interception frequency in a time period when the picture elements in the video data change to obtain a picture set;
the data processing module is used for identifying picture characteristics in the picture set by using a pre-trained image processing model and analyzing and obtaining an event scene corresponding to the picture set according to the picture characteristics;
and the scene coordination module is used for inquiring a coordination command corresponding to the event scene in the scene coordination rule according to a pre-constructed scene coordination rule and sending the coordination command to a preset intelligent device.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores computer program instructions executable by the at least one processor to enable the at least one processor to perform a method of coordinated management of a smart device according to any one of claims 1 to 7.
10. A computer-readable storage medium comprising a storage data area storing created data and a storage program area storing a computer program; characterized in that said computer program, when executed by a processor, implements a method of coordinated management of an intelligent device according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110480244.4A CN113194292A (en) | 2021-04-30 | 2021-04-30 | Coordination management method, device, equipment and storage medium for intelligent equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110480244.4A CN113194292A (en) | 2021-04-30 | 2021-04-30 | Coordination management method, device, equipment and storage medium for intelligent equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113194292A true CN113194292A (en) | 2021-07-30 |
Family
ID=76982988
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110480244.4A Pending CN113194292A (en) | 2021-04-30 | 2021-04-30 | Coordination management method, device, equipment and storage medium for intelligent equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113194292A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114745229A (en) * | 2022-02-15 | 2022-07-12 | 成都臻识科技发展有限公司 | Embedded AI video gateway and implementation method thereof |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107578007A (en) * | 2017-09-01 | 2018-01-12 | 杭州电子科技大学 | A kind of deep learning face identification method based on multi-feature fusion |
| CN107909095A (en) * | 2017-11-07 | 2018-04-13 | 江苏大学 | A kind of image-recognizing method based on deep learning |
| CN109447140A (en) * | 2018-10-19 | 2019-03-08 | 广州四十五度科技有限公司 | A method of the image recognition based on neural network deep learning simultaneously recommends cognition |
| CN109783047A (en) * | 2019-01-18 | 2019-05-21 | 三星电子（中国）研发中心 | Intelligent volume control method and device in a kind of terminal |
| CN109922310A (en) * | 2019-01-24 | 2019-06-21 | 北京明略软件系统有限公司 | The monitoring method of target object, apparatus and system |
| CN110222696A (en) * | 2019-04-16 | 2019-09-10 | 天津科技大学 | A method of plant leaf blade disease identification is carried out using GPCNNs and ELM |
| CN110971942A (en) * | 2018-09-28 | 2020-04-07 | 杭州海康威视数字技术股份有限公司 | Data transmission method and device |
| CN111752165A (en) * | 2020-07-10 | 2020-10-09 | 广州博冠智能科技有限公司 | Intelligent equipment control method and device of intelligent home system |
| CN112418135A (en) * | 2020-12-01 | 2021-02-26 | 深圳市优必选科技股份有限公司 | Human behavior recognition method and device, computer equipment and readable storage medium |
-
2021
- 2021-04-30 CN CN202110480244.4A patent/CN113194292A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107578007A (en) * | 2017-09-01 | 2018-01-12 | 杭州电子科技大学 | A kind of deep learning face identification method based on multi-feature fusion |
| CN107909095A (en) * | 2017-11-07 | 2018-04-13 | 江苏大学 | A kind of image-recognizing method based on deep learning |
| CN110971942A (en) * | 2018-09-28 | 2020-04-07 | 杭州海康威视数字技术股份有限公司 | Data transmission method and device |
| CN109447140A (en) * | 2018-10-19 | 2019-03-08 | 广州四十五度科技有限公司 | A method of the image recognition based on neural network deep learning simultaneously recommends cognition |
| CN109783047A (en) * | 2019-01-18 | 2019-05-21 | 三星电子（中国）研发中心 | Intelligent volume control method and device in a kind of terminal |
| CN109922310A (en) * | 2019-01-24 | 2019-06-21 | 北京明略软件系统有限公司 | The monitoring method of target object, apparatus and system |
| CN110222696A (en) * | 2019-04-16 | 2019-09-10 | 天津科技大学 | A method of plant leaf blade disease identification is carried out using GPCNNs and ELM |
| CN111752165A (en) * | 2020-07-10 | 2020-10-09 | 广州博冠智能科技有限公司 | Intelligent equipment control method and device of intelligent home system |
| CN112418135A (en) * | 2020-12-01 | 2021-02-26 | 深圳市优必选科技股份有限公司 | Human behavior recognition method and device, computer equipment and readable storage medium |
Non-Patent Citations (2)
| Title |
|---|
| 余胜生等: "视频监控应用中的一种流控策略" * |
| 王蓉等: "一种应用于智能物流园区中的自适应视频传输控制方法" * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114745229A (en) * | 2022-02-15 | 2022-07-12 | 成都臻识科技发展有限公司 | Embedded AI video gateway and implementation method thereof |
| CN114745229B (en) * | 2022-02-15 | 2024-04-12 | 成都臻识科技发展有限公司 | Embedded AI video gateway and implementation method thereof |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111723727A (en) | Cloud monitoring method and device based on edge computing, electronic equipment and storage medium | |
| CN110569298B (en) | Data docking and visualization method and system | |
| CN111835582B (en) | Configuration method and device of Internet of things inspection equipment and computer equipment | |
| CN112447189A (en) | Voice event detection method and device, electronic equipment and computer storage medium | |
| CN110414376B (en) | Method for updating face recognition model, face recognition camera and server | |
| US20200286100A1 (en) | Payment complaint method, device, server and readable storage medium | |
| CN111459439B (en) | Information display method, intelligent home server and storage medium | |
| CN109636582A (en) | Credit information management method, device, equipment and storage medium | |
| CN111695594A (en) | Image category identification method and device, computer equipment and medium | |
| CN110848897A (en) | Intelligent air conditioner adjusting method and computer readable storage medium | |
| CN113194292A (en) | Coordination management method, device, equipment and storage medium for intelligent equipment | |
| CN110674123B (en) | Data preprocessing method, device, equipment and medium | |
| CN111007735A (en) | Intelligent household equipment linkage method and system based on community license plate recognition | |
| CN114693435A (en) | Intelligent return visit method and device for collection list, electronic equipment and storage medium | |
| CN116089446A (en) | Optimization control method and device for structured query statement | |
| CN115147894A (en) | Image processing method, image processing apparatus, electronic device, and medium | |
| CN114691782A (en) | Database table increment synchronization method and device and storage medium | |
| CN112560789A (en) | Image object detection method and device, electronic equipment and storage medium | |
| CN113419951A (en) | Artificial intelligence model optimization method and device, electronic equipment and storage medium | |
| CN112445811A (en) | Data service method, device, storage medium and component based on SQL configuration | |
| CN114723400B (en) | Service authorization management method, device, equipment and storage medium | |
| CN116719969B (en) | Intelligent home interaction data analysis method and system based on Internet of things | |
| CN109064570A (en) | A kind of managing system of car parking | |
| WO2023141778A1 (en) | Information transmission method and apparatus, and device, storage medium and program product | |
| CN112801130B (en) | Image clustering quality evaluation method, system, medium, and apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210730 |