CN112804302A - Power video image analysis system and method based on cloud edge cooperation - Google Patents
Power video image analysis system and method based on cloud edge cooperation Download PDFInfo
- Publication number
- CN112804302A CN112804302A CN202011619666.7A CN202011619666A CN112804302A CN 112804302 A CN112804302 A CN 112804302A CN 202011619666 A CN202011619666 A CN 202011619666A CN 112804302 A CN112804302 A CN 112804302A
- Authority
- CN
- China
- Prior art keywords
- video image
- analysis
- data
- edge
- platform
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 238000010191 image analysis Methods 0.000 title claims abstract description 14
- 238000004458 analytical method Methods 0.000 claims abstract description 148
- 238000012545 processing Methods 0.000 claims abstract description 82
- 238000013473 artificial intelligence Methods 0.000 claims abstract description 21
- 238000004519 manufacturing process Methods 0.000 claims abstract description 15
- 238000001514 detection method Methods 0.000 claims description 86
- 238000012544 monitoring process Methods 0.000 claims description 40
- 238000003745 diagnosis Methods 0.000 claims description 16
- 239000000779 smoke Substances 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 8
- 230000008014 freezing Effects 0.000 claims description 4
- 238000007710 freezing Methods 0.000 claims description 4
- 238000003703 image analysis method Methods 0.000 claims description 4
- 238000004422 calculation algorithm Methods 0.000 abstract description 33
- 238000009826 distribution Methods 0.000 abstract description 2
- 238000012549 training Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 10
- 230000002159 abnormal effect Effects 0.000 description 9
- 238000003062 neural network model Methods 0.000 description 9
- 238000004590 computer program Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000004364 calculation method Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 238000013528 artificial neural network Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 4
- 239000000463 material Substances 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 239000004020 conductor Substances 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000007547 defect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000011981 development test Methods 0.000 description 1
- 238000012774 diagnostic algorithm Methods 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000007726 management method Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000013468 resource allocation Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 239000013589 supplement Substances 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/02—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]
- H04L67/025—Protocols based on web technology, e.g. hypertext transfer protocol [HTTP] for remote control or remote monitoring of applications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1001—Protocols in which an application is distributed across nodes in the network for accessing one among a plurality of replicated servers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computing Systems (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Molecular Biology (AREA)
- Multimedia (AREA)
- Medical Informatics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a power video image analysis system and method based on cloud edge cooperation, wherein the system comprises the following steps: the system comprises an electric power video image platform, a service processing system, an artificial intelligence cloud platform and a plurality of edge analysis devices, wherein the edge analysis devices are used for acquiring electric power video image data in an electric power production environment and performing first-layer analysis processing on the electric power video image data; and the artificial intelligence cloud platform is used for carrying out second-layer analysis processing on the video image data. The invention aims at solving the problems of instantaneity of data processing, unreasonable occupation of analysis resources and the like by computing the computing power distribution of the cloud end and the edge end through the analysis of the analysis power of the computing cloud end, the analysis power of the edge end, the algorithm complexity and the algorithm calculated amount in the existing cloud end centralized processing mode.
Description
Technical Field
The invention belongs to the technical field of video image processing, and particularly relates to a power video image analysis system based on cloud-edge coordination, and further relates to a power video image analysis method based on cloud-edge coordination.
Background
With the integration and application of power enterprises and the internet, the concept of establishing a power ubiquitous internet of things is proposed, the digital technology, the digital economy and the vitality and competitiveness of the power enterprises have an inseparable relationship, and different cloud platforms are established in a dispute for each large power group enterprise to manage the production process and ensure the production safety. For a traditional cloud platform, a lot of problems are faced in the process of power production management, and when the coverage user area is wide enough, problems of too many access devices, large information amount, obvious service cycle peak value and the like can be encountered. Although the cloud computing technology can solve the problems of wide service objects, peak value of service period and the like in practical application, due to the reasons of large coverage range, wide range and the like in the power industry, if all data are uploaded to the cloud end for processing, computing pressure can be brought to a cloud platform, and huge burden can be caused to network bandwidth resources. In addition, when equipment is deployed in the power industry, the problems of remote equipment location, difficulty in deployment and the like often exist, and network transmission conditions are difficult to meet the requirement of transmitting a large amount of information data of large-scale equipment, so that the problem is difficult to solve only by using a cloud platform in the power industry, and the cloud computing platform and the edge computing platform are required to be cooperatively used.
The cloud computing platform and the edge computing platform are two concepts which are relatively independent, and are a unified whole which is mutually cooperated. The cloud platforms are positioned in the center of the whole computing frame, a plurality of edge platforms can exist, and the edge platforms are respectively deployed in edge areas and directly process initial information transmitted by equipment; compared with an edge computing platform, the cloud computing platform has stronger analysis capability and computing capability of processing a large amount of data, and the edge computing platform is generally used for relatively simple computing of a small amount of data compared with the cloud platform. Edge computing is one of the related prior arts, and means that an edge computing platform integrating storage and computing is adopted near data sources such as video equipment, and the nearest-end service is provided nearby. The application program is initiated at the edge side, faster network service response is generated, long-distance data transmission is not needed, data leakage is not easy to occur, and the basic requirements of the industry on real-time business, application intelligence, safety, privacy protection and the like are met.
For an electric power video image platform, in the whole video monitoring system, a video stream and an image are taken as objects, real-time image capture is carried out on the objects, a key frame of the video stream is captured and stored, and then various image recognition processing works are carried out, such as analysis of dangerous factors of an image picture, diagnosis of defects of the image picture and the like. Because the coverage of the power video image platform is wide, the data transmitted by the equipment is huge, and the image video is analyzed and calculated only by using the cloud platform, the information flow is easy to be large, and huge pressure is brought to the bandwidth and the cloud platform. If only the traditional cloud edge cooperation is used, problems of uneven task allocation, overlong task queuing time, vacant resources and resource waste are easily caused.
Therefore, the invention provides the power video image analysis framework based on cloud edge cooperation, and the image recognition function of the power video image platform is optimized.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides a power video image analysis system and method based on cloud-edge coordination.
In order to solve the technical problem, the invention provides a power video image analysis system based on cloud edge coordination, which comprises:
electric power video image platform, business processing system, artificial intelligence cloud platform and a plurality of edge analysis equipment, wherein:
the edge analysis equipment is used for acquiring power video image data in a power production environment, performing first-layer analysis processing on the power video image data, and uploading the acquired power video image data and an analysis result to the power video image platform;
the electric power video image platform is used for distributing analysis tasks for the edge analysis equipment and the artificial intelligent cloud platform; uploading the collected power video image data to a cloud platform, and reporting an analysis result to a service processing system;
and the artificial intelligence cloud platform is used for carrying out second-layer analysis processing on the video image data and sending an analysis result to the power video image platform.
Further, the first layer of analysis processing comprises work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis.
Further, the second layer of analysis processing comprises smoke and fire detection, large-scale mechanical detection, wire galloping, wire sag and wire windage yaw detection.
Further, the device diagnosis includes: signal loss, image blur, low contrast, image over-brightness, image over-darkness, image color cast, noise interference, streak interference, black and white images, video occlusion, picture freezing, video sharp change, video jitter, and scene change.
Further, the edge analysis apparatus includes: the device comprises a data acquisition module, a data caching module, a data processing module and a data sending module;
a data acquisition module: the system comprises a data cache module, a data acquisition module, a data processing module and a data processing module, wherein the data cache module is used for acquiring power video image data in a power production environment and transmitting the acquired data to the data cache module;
the data caching module is used for caching the data acquired by the data acquisition module and regularly clearing out-of-date data;
a data processing module: performing a first layer of analysis processing on the power video image data, wherein the first layer of analysis processing comprises work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis;
and the data sending module is used for uploading the acquired electric power video image data and the analysis result to the electric power video image platform.
Correspondingly, the invention also provides a power video image analysis method based on cloud edge cooperation, which comprises the following steps:
monitoring and recording the power production environment by the edge analysis equipment to obtain video image data;
the method comprises the steps that edge analysis equipment carries out first-layer analysis processing on video image data, wherein the first-layer analysis processing comprises working clothes wearing detection, personnel off-post detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis;
the edge analysis equipment transmits the analysis result and the video image data to the power video image platform;
the electric power video image platform receives video image data and simultaneously sends the video image data to the artificial intelligent cloud platform;
the artificial intelligence cloud platform carries out a second layer of analysis processing on the video image data, the second layer of analysis processing comprises smoke and fire detection, large-scale mechanical detection, wire galloping, wire sag and wire windage yaw detection, and sends the analysis result to the power video image platform,
and the electric power video image platform reports the analysis result to the service processing system.
Further, the method also comprises the following steps:
and counting the occupancy rates of a CPU and a GPU of the edge analysis equipment in real time by the power video image platform, and if the occupancy rates exceed a set threshold value, directly distributing the acquired video image data to the cloud platform for first-layer and second-layer analysis processing.
Further, the method also comprises the following steps:
the electric power video image platform counts the occupancy rates of a CPU and a GPU of the cloud platform cluster in real time, and if the occupancy rate of a certain cloud platform exceeds a set threshold value, the collected video image data is distributed to other cloud platforms for analysis and processing.
Compared with the prior art, the invention has the following beneficial effects:
1) according to the invention, a traditional cloud platform analysis mode is replaced by a cloud-edge cooperation mode, the information flow required to be transmitted is reduced, the problem of high data transmission cost is solved, the burden of bandwidth is reduced, the operation pressure of cloud platform calculation is relieved, and the timeliness and robustness of electric power video image analysis are improved;
2) according to the invention, invalid alarms of the cloud platform and the edge analysis platform on the power video image are analyzed and counted, machine learning is provided for the cloud platform and the edge analysis platform, an abnormal result of monitoring data is alarmed, operation and maintenance personnel process effective alarm information after receiving the alarms and feed back invalid alarm information, and the cloud platform and the edge analysis platform perform deep learning again to form a virtuous circle for solving the problems.
3) According to the cloud-edge collaborative analysis method, the power video image is analyzed through the cloud-edge collaborative analysis framework, cloud-edge resources are reasonably distributed, and the problems that resources are idle and wasted and task queuing time is too long due to cloud-edge capability difference possibly caused by traditional cloud-edge collaboration are solved.
Drawings
FIG. 1 is a block diagram of the framework of the system of the present invention;
FIG. 2 is a flow diagram of a business processing system;
FIG. 3 is a schematic flow chart of the method of the present invention.
Detailed Description
The invention is further described below with reference to the accompanying drawings. The following examples are only for illustrating the technical solutions of the present invention more clearly, and the protection scope of the present invention is not limited thereby.
The invention discloses a cloud-edge-collaboration-based power video image analysis system, a framework of which is shown in fig. 1, and the system comprises a power video image platform, a service processing system, an artificial intelligence cloud platform (called a cloud platform for short) and a plurality of edge analysis devices, wherein:
the edge analysis equipment is used for acquiring power video image data in a power production environment, performing first-layer analysis processing on the power video image data, and uploading the acquired power video image data and an analysis result to the power video image platform;
the electric power video image platform is used for uploading the collected electric power video image data to the cloud platform and reporting the analysis result to the service processing system;
and the cloud platform is used for carrying out second-layer analysis processing on the video image data and sending an analysis result to the power video image platform.
The number of the edge analysis devices may be one or a plurality of them may be arranged.
Services deployed on a cloud platform include:
IaaS is an infrastructure service layer, physical resources of an artificial intelligence cloud platform are deployed, cloud platform processing services are integrated through a virtualization technology, and online physical resources required by computing power are improved;
the PaaS provides development service for the platform, provides a complete development test flow and code structure optimization system for the subsequent development of the platform, and facilitates the subsequent supplement and improvement of a platform algorithm;
the SaaS provides services for cloud computing, receives monitoring materials such as video images and the like transmitted from the data acquisition module, and performs algorithm analysis on the monitoring materials.
In one embodiment of the invention, the video image analysis algorithms collectively comprise: the method comprises the following steps of working clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection, equipment diagnosis, smoke and fire detection, large-scale mechanical detection, wire galloping, wire sag and wire windage deviation detection.
In the embodiment of the invention, in the same analysis device, different analysis algorithm tasks are arranged, the time t1 taken by the completion of the analysis tasks is counted and recorded, a minimum threshold t0 is set, and when t1> t0, the algorithm is set as the first layer of analysis processing; when t0< t1, the algorithm is set to the second layer analysis process.
The computing power comprehensive evaluation is carried out on different edge devices and cloud service nodes, computing tasks are respectively arranged on the CPU and GPU clusters of the edge devices and the cloud service nodes, the time consumption is compared, the computing power ratio is obtained, algorithms with different complexities are arranged according to different ratios, and the problem that the overall task processing flow is affected due to the fact that the computing time is too long due to the fact that the algorithms are too complex is avoided. In one embodiment, smoke and fire detection, large-scale mechanical detection, conductor galloping, conductor sag and conductor windage yaw detection algorithm models have higher requirements on CPU resources and GPU resources, and longer time is consumed for task completion, and the algorithm models (called complex algorithms) are subjected to calculation analysis on a cloud computing platform; the requirements of work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis on CPU resources and GPU resources are low, the time consumed for completing tasks is short, and the algorithm model (called as a simple algorithm) is subjected to calculation analysis in edge analysis equipment.
In one example, the first layer of analytical processing includes work clothes wear detection, personnel off Shift detection, personnel loitering detection, border crossing detection, zone detection, and equipment diagnostics; the second layer of analysis processing comprises smoke and fire detection, large mechanical detection, wire galloping, wire sag and wire windage yaw detection.
The neural network model and algorithm of the edge analysis equipment are respectively work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis, wherein the work clothes wearing detection comprises whether workers wear safety helmets or not, whether the workers wear short-sleeve shorts to expose skin or not and the equipment diagnosis has fourteen diagnosis algorithms: the method comprises the steps of signal loss, image blurring, low contrast, over-bright image, over-dark image, image color cast, noise interference, stripe interference, black and white image, video shielding, picture freezing, video sharp change, video dithering and scene change. The method comprises the steps of wearing the working clothes and detecting the state of personnel, carrying out related sample training on a sample by adopting a fast-rcnn model of a deep neural network in an area and an out-of-range area to obtain a related model, carrying out safety helmet sample training on the sample by adopting the fast-rcnn model of the deep neural network when the safety helmet is detected to obtain a safety helmet model, and then detecting whether the safety helmet exists in a picture to be detected by using the trained safety helmet model. Wherein the device diagnostic algorithm trains algorithm parameters through differences in pixel thresholds and differences between successive multi-frame pictures.
The edge analysis equipment trains the neural network model and the algorithm parameters according to the unified sample database, so that the neural network model and the algorithm parameters contained in the edge analysis equipment provided by the embodiment of the invention are the same.
The power video image platform arranges the same intelligent analysis algorithm task for all the edge analysis devices, analyzes the task by using different analysis devices, counts and records the time t2 for completing the task analysis, and counts the reciprocal of the time taken according to the time taken by the different analysis devices to obtain the calculation capability cap values of the different analysis devices.
And the power video image platform performs task allocation on different edge analysis equipment according to the cap values of the different edge analysis equipment.
The neural network model and algorithm of the artificial intelligent cloud platform comprise smoke and fire detection, large-scale mechanical detection, wire waving, wire sag and wire windage yaw detection, an image database containing multiple types of influence factors is established firstly, the large-scale mechanical detection and the smoke and fire detection are respectively carried out on the image database, target detection is carried out on core factors in images of each type of images, unique pixel points of the images are marked, deep learning is carried out on the model, filtering is carried out by adopting a NMS non-maximum suppression algorithm, and the type of target hidden danger and the position of a rectangular frame are obtained. The firework detection method comprises the following steps that a fast-rcnn model of a deep neural network is adopted to conduct firework sample training on a sample to obtain a firework model, and then the trained firework model is used to detect pictures to be tested to determine whether fireworks exist; the large-scale machine detection adopts a fast-rcnn model of a deep neural network to train a large-scale machine sample on the sample to obtain a large-scale machine model, and then the trained large-scale machine model is used for detecting a picture to be tested to determine whether a large-scale machine exists; the method comprises the steps of firstly removing noise interference information around a wire by adopting a high-pass filtering method, then detecting whether the wire has galloping, sag or windage yaw by adopting a dense optical flow method, and calculating the amplitude of the wire galloping, sag or windage yaw.
In order to realize resource allocation, neural network models and algorithms in all edge analysis devices are also deployed in the artificial intelligence cloud platform. The edge analysis equipment counts the occupancy rates of a CPU and a GPU of the equipment in real time and informs an electric power video image platform, if the resource occupancy rate is lower than 90%, the image is collected and then analyzed and processed in the edge analysis equipment, and if the resource occupancy rate is higher than 90%, collected data are uploaded to the electric power video image platform and then uploaded to a cloud platform for analysis and processing.
Meanwhile, the electric power video image platform counts the occupancy rates of the CPU and the GPU of the cloud platform cluster in real time, and if the occupancy rate of one cloud platform exceeds 90%, the collected video image data is distributed to other cloud platforms for analysis and processing.
The frame of the edge analysis device is shown in fig. 1 and mainly divided into four modules: the device comprises a data acquisition module, a data caching module, a data processing module and a data sending module.
A data acquisition module: the system comprises a data cache module, a data acquisition module, a data processing module and a data processing module, wherein the data cache module is used for acquiring power video image data in a power production environment and transmitting the acquired data to the data cache module;
and the data caching module is used for caching the data collected by the data collecting module and regularly clearing out the expired data (deleting the data information when the data acquisition date exceeds 30 days).
A data processing module: processing the collected power video images, wherein the videos are extracted in a key frame extraction mode and transmitted into an algorithm for processing, the algorithm of a data processing module is respectively work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis, wherein the work clothes wearing detection comprises whether workers wear safety helmets or not and whether the workers wear short-sleeve shorts to expose skin or not, and the equipment diagnosis has fourteen diagnosis algorithms: the method comprises the steps of signal loss, image blurring, low contrast, over-bright image, over-dark image, image color cast, noise interference, stripe interference, black and white image, video shielding, picture freezing, video sharp change, video dithering and scene change.
When the data processing module detects that monitoring data is abnormal, such as a safety helmet, a work clothes or a worker does not wear the safety helmet or passes through a safety yellow line, or black and white pictures or stripe interference and the like appear in equipment, the error-reporting monitoring data and the error-reporting information are sent to the data sending module;
and the data sending module is used for sending the normal monitoring data, the error reporting monitoring data and the error reporting information to the power video image platform.
The edge analysis equipment transmits monitoring data to the power video image platform and is divided into two types, one type is the monitoring data of error detection and the other type is the monitoring data of normal detection of the edge analysis equipment, the power video image platform classifies different monitoring data into a library for statistics, and then transmits the received monitoring data to a SaaS layer of the cloud platform for analysis.
The cloud platform detects monitoring data analyzed by the edge analysis equipment, and the monitoring data comprises smoke and fire detection, large-scale mechanical detection, wire galloping, wire sag and wire windage yaw detection. And the cloud computing platform detects the monitoring data and then sends error reporting monitoring data and error reporting information to the power video image platform.
The electric power video image platform counts monitoring data transmitted by the cloud computing platform and transmits the monitoring data reported by the cloud computing platform or the edge analysis equipment and error information to business personnel, the related logic of the monitoring data reported by the cloud computing platform or the edge analysis equipment and the error information are shown in fig. 2, the business personnel process the monitoring data reported by the error and transmit an effective alarm to a front-end business system for processing, and an invalid alarm is fed back to the cloud platform and the edge equipment for deep learning.
Based on the above system, the power video image analysis method based on cloud edge coordination according to the present invention, as shown in fig. 3, includes the following processes:
s101, monitoring and recording the power production environment by edge analysis equipment to obtain video image data;
specifically, the edge analysis device monitors and records the corresponding power production environment to obtain corresponding pictures and videos for caching, for example, the snapshot interval of the edge analysis device is one hour, the video recording time is 8 am to six pm, and the video recording time is 24 hours after expiration and is cleaned.
And S102-S104, judging the occupancy rates of the CPU and the GPU of the equipment by the edge analysis equipment, informing the power video image platform, if the resource occupancy rate is lower than 90%, analyzing corresponding video image data by using a trained neural network model and algorithm, and if the resource occupancy rate is higher than 90%, uploading the acquired data to the power video image platform, and uploading the acquired data to the cloud platform by the power video image platform for analysis processing.
It should be noted that, if the monitoring data is a video record, the key frame of the video record is extracted for analysis, and if the monitoring data is a picture, the picture analysis is directly invoked. And obtaining the potential safety hazard type of the corresponding monitoring data and the image coordinate of the potential safety hazard.
And S105, the edge analysis equipment transmits the analyzed pictures and videos to the power video image platform.
Specifically, the edge analysis device transmits the analyzed pictures and videos including abnormal and normal pictures and carrying abnormal information to the power video image platform if the result is that the abnormal pictures carry abnormal information, wherein the pictures are connected through UDP and TCP and transmitted by using an I1 protocol; the video is transmitted by using an enterprise standard a interface through UDP connection, and the monitoring data needs to provide the real-time position of a shooting place, the temperature of the surrounding environment, the local time and the residual electric quantity information of the equipment. It should be noted that the present embodiment is not limited to the connection method and the protocol used for transmission.
And S106, the electric power video image platform receives the pictures and videos from the edge analysis equipment, simultaneously sends the pictures and videos to the artificial intelligence cloud platform for secondary analysis, and if the edge analysis equipment does not perform the first-layer analysis, the artificial intelligence cloud platform performs the first-layer analysis and the second-layer analysis.
It should be noted that the electric power video image platform counts the occupancy rates of the CPU and the GPU of the cloud platform cluster in real time, and if the occupancy rate of a certain cloud platform exceeds 90%, the acquired video image data is distributed to other cloud platforms for analysis and processing.
And S107, analyzing corresponding monitoring data by using the trained neural network model and algorithm by using the artificial intelligence cloud platform.
When the monitoring data is a video, the artificial intelligence cloud platform extracts a key frame of the video for analysis, because the edge device extracts and marks the key frame, the artificial intelligence cloud platform directly calls the key frame for analysis, when the monitoring data is a picture, the artificial intelligence cloud platform directly calls the picture for analysis, and then transmits the abnormal picture and video after secondary analysis and the image coordinates of the potential safety hazard type and the potential safety hazard into the power video image platform.
And S108, the electric power video image platform marks the abnormal picture and the error reporting video according to the potential safety hazard types and the potential safety hazard image coordinates of the abnormal picture and the error reporting video transmitted by the artificial intelligence cloud platform and records a database.
S109, the electric power video image platform transmits the marked monitoring data to the service processing system.
And S110-S113, the service processing system allocates service personnel to audit and screen the marked pictures and videos, judges whether the marked alarm monitoring materials and alarm information are invalid alarms or not, transmits the valid alarm monitoring materials and alarm information to the front-end service system to be handed to maintenance personnel for processing, feeds the invalid alarms back to the cloud platform and the edge analysis equipment for deep learning, and deeply trains the algorithm model of the artificial intelligent platform.
The artificial intelligence cloud platform and the edge analysis equipment are used for training an RPN model by collecting invalid alarm pictures and video key frames as explosals, and then training a fast-rcnn model by using the RPN. It should be noted that the basic training sample of the fast-rcnn model for the first training is obtained by fusing the hidden danger-free monitoring data and the hidden danger monitoring data obtained in the next iteration to obtain a new training sample. That is to say, the fast-rcnn model of the iterative training uses the sample of the standard sample database as the basic training sample, and is fused with the positive sample and the negative sample to obtain the sample of the iterative training, where it is required to say that the positive sample is the correctly labeled monitoring data, and the negative sample is the monitoring data of the invalid alarm. And after the iteration cycle is completed, performing the next round of faster-rcnn model training iteration. And the artificial intelligence cloud platform tests the updated neural network model, and updates the corresponding successfully-tested neural network model to the SaaS layer algorithm analysis service and all the edge analysis equipment of the artificial intelligence cloud platform.
In the embodiment of the invention, the cloud-side tasks are distributed according to the algorithm complexity and the algorithm calculation amount, and in order to ensure the real-time property of task processing, the edge analysis equipment can distribute the tasks again on the power video image platform after acquiring the monitoring data, and the specific distribution method comprises the following steps:
counting the occupancy rates of a CPU and a GPU of the edge analysis equipment in real time:
if the occupancy rate of the edge analysis equipment exceeds a set threshold (for example, 90%), the acquired monitoring data are directly distributed to the cloud platform for the first and second layers of analysis processing, so that resource vacancy is avoided.
The available resources of different analysis devices are counted in real time, including the GPU occupancy rate and the CPU occupancy rate of the different analysis devices, and the power video image platform distributes and analyzes tasks according to the real-time resource occupancy rate to ensure that the resources are not idle and the tasks are less queued.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (9)
1. A power video image analysis system based on cloud edge cooperation is characterized by comprising:
electric power video image platform, business processing system, artificial intelligence cloud platform and a plurality of edge analysis equipment, wherein:
the edge analysis equipment is used for acquiring power video image data in a power production environment, performing first-layer analysis processing on the power video image data, and uploading the acquired power video image data and an analysis result to the power video image platform;
the electric power video image platform is used for distributing analysis tasks for the edge analysis equipment and the artificial intelligent cloud platform; uploading the collected power video image data to a cloud platform, and reporting an analysis result to a service processing system;
and the artificial intelligence cloud platform is used for carrying out second-layer analysis processing or first-layer analysis processing and second-layer analysis processing on the video image data and sending an analysis result to the power video image platform.
2. The cloud-edge coordination based power video image analysis system according to claim 1, wherein the first layer of analysis processing includes work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis.
3. The cloud-edge-collaboration-based power video image analysis system as claimed in claim 1, wherein the second layer of analysis processing comprises smoke and fire detection, large machinery detection, wire galloping, wire sag and wire windage detection.
4. The system according to claim 2, wherein the device diagnosis comprises: signal loss, image blur, low contrast, image over-brightness, image over-darkness, image color cast, noise interference, streak interference, black and white images, video occlusion, picture freezing, video sharp change, video jitter, and scene change.
5. The system according to claim 1, wherein the edge analysis device comprises: the device comprises a data acquisition module, a data caching module, a data processing module and a data sending module;
a data acquisition module: the system comprises a data cache module, a data acquisition module, a data processing module and a data processing module, wherein the data cache module is used for acquiring power video image data in a power production environment and transmitting the acquired data to the data cache module;
the data caching module is used for caching the data acquired by the data acquisition module and regularly clearing out-of-date data;
a data processing module: performing first-layer analysis processing on the power video image data;
and the data sending module is used for uploading the acquired electric power video image data and the analysis result to the electric power video image platform.
6. The system according to any one of claims 1 to 5, wherein the power video image analysis method based on cloud edge coordination comprises the following steps:
monitoring and recording the power production environment by the edge analysis equipment to obtain video image data;
if the occupancy rates of the CPU and the GPU of the edge analysis equipment are lower than a set threshold value, the edge analysis equipment carries out first-layer analysis processing on the video image data;
the edge analysis equipment transmits the analysis result and the video image data to the power video image platform;
the electric power video image platform receives video image data and simultaneously sends the video image data to the artificial intelligent cloud platform;
the artificial intelligence cloud platform carries out second-layer analysis processing on the video image data and sends the analysis result to the power video image platform,
and the electric power video image platform reports the analysis result to the service processing system.
7. The method for power video image analysis based on cloud-edge collaboration as claimed in claim 6, wherein the first layer of analysis processing comprises work clothes wearing detection, personnel off duty detection, personnel loitering detection, border crossing detection, area detection and equipment diagnosis;
the second layer of analysis processing comprises smoke and fire detection, large-scale mechanical detection, wire galloping, wire sag and wire windage yaw detection;
the time for completing each analysis task in the second layer of analysis processing is longer than that in the first layer of analysis processing.
8. The method for analyzing the power video image based on the cloud-edge collaboration as claimed in claim 6, further comprising:
and if the CPU and GPU occupancy rates of the edge analysis equipment exceed the set threshold values, directly distributing the acquired video image data to the cloud platform for the first-layer and second-layer analysis processing.
9. The method for analyzing the power video image based on the cloud-edge collaboration as claimed in claim 6, further comprising:
the electric power video image platform counts the occupancy rates of a CPU and a GPU of the cloud platform cluster in real time, and if the occupancy rate of a certain cloud platform exceeds a set threshold value, the collected video image data is distributed to other cloud platforms for analysis and processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011619666.7A CN112804302B (en) | 2020-12-30 | 2020-12-30 | Cloud edge cooperation-based power video image analysis system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011619666.7A CN112804302B (en) | 2020-12-30 | 2020-12-30 | Cloud edge cooperation-based power video image analysis system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112804302A true CN112804302A (en) | 2021-05-14 |
| CN112804302B CN112804302B (en) | 2024-05-28 |
Family
ID=75806038
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011619666.7A Active CN112804302B (en) | 2020-12-30 | 2020-12-30 | Cloud edge cooperation-based power video image analysis system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112804302B (en) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113835884A (en) * | 2021-09-08 | 2021-12-24 | 深圳供电局有限公司 | Cloud edge cooperative management and control system and method |
| CN113937894A (en) * | 2021-10-22 | 2022-01-14 | 浙江九州云信息科技有限公司 | Cloud edge cooperation-based electric intelligent terminal management system and method |
| CN113949703A (en) * | 2021-09-18 | 2022-01-18 | 国网上海市电力公司 | Cloud edge cooperative system for electric power artificial intelligence |
| CN114140447A (en) * | 2021-12-06 | 2022-03-04 | 国网新疆电力有限公司信息通信公司 | Cloud edge cooperation technology-based power equipment image identification method and system |
| CN114205645A (en) * | 2021-12-10 | 2022-03-18 | 北京凯视达信息技术有限公司 | Distributed video content auditing method and device |
| CN114501131A (en) * | 2021-12-22 | 2022-05-13 | 天翼云科技有限公司 | Video analysis method and device, storage medium and electronic equipment |
| CN114501037A (en) * | 2021-12-24 | 2022-05-13 | 泰华智慧产业集团股份有限公司 | Video analysis method and system based on edge cloud cooperative computing under smart pole scene |
| CN116260699A (en) * | 2023-04-03 | 2023-06-13 | 中国电子技术标准化研究院 | Industrial Internet system based on cloud edge end cooperation and implementation method |
| CN117173711A (en) * | 2023-08-18 | 2023-12-05 | 安徽工程大学产业创新技术研究有限公司 | Automobile tire parameter identification and detection method and service platform |
| CN117596362A (en) * | 2023-11-04 | 2024-02-23 | 无锡金乌山集成科技有限公司 | Intelligent comprehensive monitoring system based on big data |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110557419A (en) * | 2018-06-01 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | task processing method and device and cloud computing system |
| CN111131480A (en) * | 2019-12-30 | 2020-05-08 | 南京德赛尔信息技术有限公司 | Cloud edge cooperative service system for smart power plant |
| CN111698470A (en) * | 2020-06-03 | 2020-09-22 | 河南省民盛安防服务有限公司 | Security video monitoring system based on cloud edge cooperative computing and implementation method thereof |
| CN112037365A (en) * | 2020-09-01 | 2020-12-04 | 枣庄学院 | Vehicle fire accident detection and alarm system based on edge calculation and oriented to automobile data recorder |
-
2020
- 2020-12-30 CN CN202011619666.7A patent/CN112804302B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110557419A (en) * | 2018-06-01 | 2019-12-10 | 杭州海康威视数字技术股份有限公司 | task processing method and device and cloud computing system |
| CN111131480A (en) * | 2019-12-30 | 2020-05-08 | 南京德赛尔信息技术有限公司 | Cloud edge cooperative service system for smart power plant |
| CN111698470A (en) * | 2020-06-03 | 2020-09-22 | 河南省民盛安防服务有限公司 | Security video monitoring system based on cloud edge cooperative computing and implementation method thereof |
| CN112037365A (en) * | 2020-09-01 | 2020-12-04 | 枣庄学院 | Vehicle fire accident detection and alarm system based on edge calculation and oriented to automobile data recorder |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113835884A (en) * | 2021-09-08 | 2021-12-24 | 深圳供电局有限公司 | Cloud edge cooperative management and control system and method |
| CN113949703A (en) * | 2021-09-18 | 2022-01-18 | 国网上海市电力公司 | Cloud edge cooperative system for electric power artificial intelligence |
| CN113937894A (en) * | 2021-10-22 | 2022-01-14 | 浙江九州云信息科技有限公司 | Cloud edge cooperation-based electric intelligent terminal management system and method |
| CN114140447A (en) * | 2021-12-06 | 2022-03-04 | 国网新疆电力有限公司信息通信公司 | Cloud edge cooperation technology-based power equipment image identification method and system |
| CN114205645A (en) * | 2021-12-10 | 2022-03-18 | 北京凯视达信息技术有限公司 | Distributed video content auditing method and device |
| CN114501131A (en) * | 2021-12-22 | 2022-05-13 | 天翼云科技有限公司 | Video analysis method and device, storage medium and electronic equipment |
| CN114501131B (en) * | 2021-12-22 | 2023-08-08 | 天翼云科技有限公司 | Video analysis method and device, storage medium and electronic equipment |
| CN114501037A (en) * | 2021-12-24 | 2022-05-13 | 泰华智慧产业集团股份有限公司 | Video analysis method and system based on edge cloud cooperative computing under smart pole scene |
| CN114501037B (en) * | 2021-12-24 | 2024-03-01 | 泰华智慧产业集团股份有限公司 | Video analysis method and system based on Bian Yun cooperative computing in intelligent pole scene |
| CN116260699A (en) * | 2023-04-03 | 2023-06-13 | 中国电子技术标准化研究院 | Industrial Internet system based on cloud edge end cooperation and implementation method |
| CN117173711A (en) * | 2023-08-18 | 2023-12-05 | 安徽工程大学产业创新技术研究有限公司 | Automobile tire parameter identification and detection method and service platform |
| CN117596362A (en) * | 2023-11-04 | 2024-02-23 | 无锡金乌山集成科技有限公司 | Intelligent comprehensive monitoring system based on big data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112804302B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112804302B (en) | Cloud edge cooperation-based power video image analysis system and method | |
| CN111582016A (en) | Intelligent maintenance-free power grid monitoring method and system based on cloud edge collaborative deep learning | |
| CN113705372B (en) | AI identification system for join in marriage net job site violating regulations | |
| CN103108159B (en) | Electric power intelligent video analyzing and monitoring system and method | |
| CN111432179A (en) | Intelligent coal conveying belt inspection system and method based on computer vision | |
| CN102740121B (en) | Be applied to video quality diagnostic control system and the method for video surveillance network | |
| CN109460744B (en) | Video monitoring system based on deep learning | |
| CN103856774A (en) | Intelligent detection system and method for video surveillance | |
| CN112950400A (en) | Data processing platform | |
| CN211906170U (en) | Based on electric wire netting field operation data management and control device | |
| CN111223263A (en) | Full-automatic comprehensive fire early warning response system | |
| CN109509190A (en) | Video monitoring image screening technique, device, system and storage medium | |
| CN103974029A (en) | Video monitoring method, video monitoring system and video monitoring device | |
| CN113239750A (en) | System, method, equipment and application for detecting personnel behaviors in electric power business hall | |
| CN111079731A (en) | Configuration system, method, equipment and medium based on safety helmet identification monitoring system | |
| CN108230669B (en) | Road vehicle violation detection method and system based on big data and cloud analysis | |
| CN211184122U (en) | Intelligent video analysis system for linkage of railway operation safety prevention and control and large passenger flow early warning | |
| CN111178241A (en) | Intelligent monitoring system and method based on video analysis | |
| CN115002448A (en) | Video image quality diagnosis method and system applied to security monitoring | |
| CN112929604A (en) | Office image acquisition management system | |
| CN118351487A (en) | Employee violation behavior identification method based on AI | |
| CN115880631A (en) | Power distribution station fault identification system, method and medium | |
| CN113111866B (en) | Intelligent monitoring management system and method based on video analysis | |
| CN118211188A (en) | Intelligent safety emergency management data identification system | |
| CN109324911A (en) | User behavior detects smart screen automatically and grabs screen system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |