Intelligent image recognition system and method for power equipment based on cloud edge cooperation technology
Technical Field
The invention relates to the field of electrical engineering science, in particular to an intelligent image recognition system and method for power equipment based on a cloud edge cooperation technology.
Background
Along with the development of related artificial intelligence at home and abroad, except that the human face recognition technology and the license plate recognition technology tend to be mature and start to be commercially used in a large scale in each subdivision field, other related technologies are slowly developed due to various factors, particularly countless subdivision application scenes exist in countless subdivision industry fields, and the development speed of the technology is greatly limited by the customization difficulty and the large-scale market effect.
With the deep development of the power internet of things technology, tens of thousands of power devices generate massive data, especially in the field of computer vision, the amount of pictures and video data generated by front-end devices is huge, if all the data are gathered to a cloud for intelligent analysis, infinite pressure is brought to the bandwidth requirement, the real-time requirement and the like of communication, especially in some scenes with high requirements on real-time performance and privacy safety, for example, it is unrealistic to transmit a large amount of video data to the cloud for processing and then return decision results, so that we are required to provide edge intelligent services nearby on a device node side or a network edge side close to a data source, and move AI calculation power (inference power) from the cloud to the edge side gradually. The cloud is focused on the training of the non-real-time and large-data-volume deep learning algorithm, optimizes the algorithm model, and provides services such as algorithm scheduling for the edge. The technology combining edge computing and cloud computing is more and more emphasized, and the edge computing and the cloud computing are widely applied to various application scenes in various AI + IOT scenes.
According to the safety standard of the electric power construction site, all constructors must wear safety helmets and patrol according to the specified lines. In order to supervise whether all constructors meet the requirements of safe construction, the current methods generally include: firstly, a manual monitoring method: transmitting the monitoring video of the personnel in the construction site to a supervision center through a camera installed on site, and manually checking the video; secondly, a monitoring method based on cloud intelligence comprises the following steps: the method comprises the steps of installing a camera on site, transmitting a monitoring video of a construction site to a supervision center, deploying a computing server in the supervision center, processing the received video through an AI algorithm deployed on the server, analyzing whether a constructor meets the safety management requirement and giving an alarm to the supervisors.
The first method completely does not adopt an intelligent means, and completely depends on manual identification, so that the first method has the following defects: 1) manual inspection is carried out, and the efficiency is low; 2) when a plurality of construction sites are simultaneously carved, monitoring omission or inspection by more supervision personnel is possibly caused, so that the cost is high and the effect is poor; 3) the construction site and the monitoring center generally adopt wireless transmission real-time video, the bandwidth requirement is high, the video quality is difficult to guarantee, and meanwhile, a large amount of redundant video causes bandwidth waste. In the second method, artificial intelligence recognition processing is added on the basis of the first method, so that efficiency is greatly improved, but the cost is high due to the adoption of a cloud server, and meanwhile, a complete video needs to be transmitted to a cloud in real time on a construction site, so that the defects of long time delay, waste of bandwidth and the like exist.
Disclosure of Invention
Aiming at the problems, the invention provides an intelligent image recognition system and method for power equipment based on a cloud edge cooperation technology.
In order to achieve the object of the present invention, the present invention provides an intelligent image recognition system for an electrical device based on a cloud-edge coordination technology, comprising: the system comprises image acquisition equipment, edge equipment and a cloud platform, wherein a ZEUS AI intelligent computing hardware platform and an embedded target detection algorithm software platform are arranged on the edge equipment, and the image acquisition equipment, the edge equipment and the cloud platform transmit data through a network.
Further, the invention provides an intelligent image recognition method for power equipment based on a cloud edge coordination technology, which comprises the following steps:
s1: the image acquisition equipment acquires and analyzes image sample data of the power equipment;
s2: selecting a detection recognition model, and training the model;
s3: summarizing a monitoring range of the operation and maintenance of the power equipment according to the analysis of the application scene;
s4: calibrating the model training result according to the acquired image sample, and performing deep learning on a ZEUS AI intelligent computing hardware platform of the edge device and the embedded target detection algorithm software platform;
s5: carrying out model training again in S2 according to the deep learning to obtain a model training result;
s6: and adjusting parameters according to the model training result to complete the simulation test.
Further, the implementation process of the embedded target detection algorithm includes:
extracting an input video stream, and selecting a frame of data to perform preprocessing operation;
inputting the data after the preprocessing operation into a pre-trained classification network, and fixing corresponding network parameters;
and performing the following operation on the obtained feature map through the last convolutional layer of the pre-training classification network:
a. performing region generation network operation on the feature map and obtaining a corresponding region of interest;
b. acquiring a plurality of position sensitivity score mapping graphs with K X K (C +1) dimensions on the feature graph and carrying out classification operation;
c. acquiring a plurality of position sensitivity score maps with dimensions of 4 x K on the feature map and performing regression operation;
d. and respectively executing position sensitive ROI pooling operation on the K-by-K (C +1) -dimensional position sensitive score mapping graph and the 4-by-K-dimensional position sensitive score mapping graph, and acquiring corresponding category and position information, wherein K is the size of the sub-region of the region of interest, and C is the number of the identified target categories.
Further, K is 3 or more and 5 or less; c is not less than 2 and not more than 3.
Compared with the prior art, the invention has the following beneficial technical effects:
based on a high-performance ZEUS AI intelligent computing hardware platform and a CAFFE algorithm framework (an embedded target detection algorithm framework), the invention completes image acquisition, image processing, target detection, deep learning, target key feature extraction, structured video extraction and other processing at an equipment end through an advanced embedded target detection algorithm running at the equipment end, completes safety helmet detection, personnel falling down and track tracking at the equipment end and a cloud platform in real time according to the acquired video at the site, starts alarm linkage in real time for the situation that the safety management is not met, transmits an identification result and a structured alarm video to the cloud platform for display processing, and supervisors only pay attention to the alarm result and the alarm segment without checking a large amount of redundant videos, thereby greatly improving the management efficiency. The artificial intelligence video detection method has the advantages of intelligence, high efficiency, real time, low occupied transmission bandwidth and the like, is very convenient to deploy, and is particularly suitable for mobile or fixed deployment in various power construction places.
Drawings
Fig. 1 is a schematic flowchart of an intelligent image recognition method for an electrical device based on a cloud edge coordination technology according to an embodiment;
FIG. 2 is a schematic diagram of an embedded target detection algorithm framework of an embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.
Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is explicitly and implicitly understood by one skilled in the art that the embodiments described herein can be combined with other embodiments.
The invention provides an intelligent image recognition system of power equipment based on a cloud edge coordination technology, which comprises the following components: the system comprises image acquisition equipment, edge equipment and a cloud platform, wherein a ZEUS AI intelligent computing hardware platform and an embedded target detection algorithm software platform are arranged on the edge equipment, and the image acquisition equipment, the edge equipment and the cloud platform transmit data through a network.
In one embodiment, an intelligent image recognition method for an electric power device based on a cloud edge coordination technology is provided, as shown in fig. 1, including the following steps:
s1: the image acquisition equipment acquires and analyzes image sample data of the power equipment;
s2: selecting a detection recognition model, and training the model;
s3: summarizing a monitoring range of the operation and maintenance of the power equipment according to the analysis of the application scene, thereby determining the activity range of the monitored operation and maintenance personnel;
s4: calibrating the model training result according to the acquired image sample, and performing deep learning on a ZEUS AI intelligent computing hardware platform of the edge device and the embedded target detection algorithm software platform;
s5: carrying out model training again in S2 according to the deep learning to obtain a model training result;
s6: and adjusting parameters according to the model training result to complete the simulation test.
In one embodiment, as shown in fig. 2, the implementation process of the embedded target detection algorithm includes:
extracting an input video stream, and selecting a frame of data to perform preprocessing operation;
inputting the data after the preprocessing operation into a pre-trained classification network, and fixing corresponding network parameters;
and performing the following operation on the obtained feature map through the last convolutional layer of the pre-training classification network:
a. performing region generation network operation on the feature map and obtaining a corresponding region of interest;
b. acquiring a plurality of position sensitivity score mapping graphs with K X K (C +1) dimensions on the feature graph and carrying out classification operation;
c. acquiring a plurality of position sensitivity score maps with dimensions of 4 x K on the feature map and performing regression operation;
d. and respectively executing position sensitive ROI pooling operation on the K-by-K (C +1) -dimensional position sensitive score mapping graph and the 4-by-K-dimensional position sensitive score mapping graph, and acquiring corresponding category and position information, wherein K is the size of the sub-region of the region of interest, and C is the number of the identified target categories.
The embedded target detection algorithm is a full convolution neural network framework algorithm based on a region, when the algorithm is identified, firstly, image features are extracted and a feature map is generated, then, the region is used for generating a network RPN, regions of interest with different proportions are generated on the feature map, meanwhile, a position sensitivity score mapping map is generated by using the feature map, and the algorithm identifies different types of targets according to the position sensitivity score mapping map.
In one embodiment, according to the characteristics of an electric power application scene, the value range of K is set to be 3-5, and the value range of C is set to be 2-3.
The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
It should be noted that the terms "first \ second \ third" referred to in the embodiments of the present application merely distinguish similar objects, and do not represent a specific ordering for the objects, and it should be understood that "first \ second \ third" may exchange a specific order or sequence when allowed. It should be understood that "first \ second \ third" distinct objects may be interchanged under appropriate circumstances such that the embodiments of the application described herein may be implemented in an order other than those illustrated or described herein.
The terms "comprising" and "having" and any variations thereof in the embodiments of the present application are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device that comprises a list of steps or modules is not limited to the listed steps or modules but may alternatively include other steps or modules not listed or inherent to such process, method, product, or device.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.