CN112861681B - Pipe gallery video intelligent analysis method and system based on cloud processing - Google Patents
Pipe gallery video intelligent analysis method and system based on cloud processing Download PDFInfo
- Publication number
- CN112861681B CN112861681B CN202110128858.6A CN202110128858A CN112861681B CN 112861681 B CN112861681 B CN 112861681B CN 202110128858 A CN202110128858 A CN 202110128858A CN 112861681 B CN112861681 B CN 112861681B
- Authority
- CN
- China
- Prior art keywords
- abnormal
- pipe gallery
- result
- threshold value
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 44
- 238000004458 analytical method Methods 0.000 title claims abstract description 39
- 230000002159 abnormal effect Effects 0.000 claims abstract description 80
- 238000013528 artificial neural network Methods 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 28
- 238000012549 training Methods 0.000 claims abstract description 21
- 238000001514 detection method Methods 0.000 claims description 44
- 238000012216 screening Methods 0.000 claims description 20
- 230000001965 increasing effect Effects 0.000 claims description 8
- 238000004364 calculation method Methods 0.000 claims description 5
- 238000010606 normalization Methods 0.000 claims description 5
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000012423 maintenance Methods 0.000 abstract description 8
- 238000004422 calculation algorithm Methods 0.000 abstract description 5
- 238000007781 pre-processing Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 7
- 230000006872 improvement Effects 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000007689 inspection Methods 0.000 description 5
- 230000009466 transformation Effects 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- 230000005856 abnormality Effects 0.000 description 2
- 238000005520 cutting process Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 238000013519 translation Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/285—Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Mathematical Physics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a method and a system for intelligently analyzing a pipe gallery video based on cloud processing, wherein the method comprises the following steps: acquiring normal and abnormal learning sample sets of a pipe gallery; training a deep neural network classifier; adopting a trained deep neural network classifier to perform intelligent analysis on newly acquired images to be recognized; carrying out data enhancement processing to eliminate frequency non-uniform interference among sample categories; preprocessing the image with too dark brightness by using a picture enhancement algorithm to eliminate the interference of a weak light source; and (5) carrying out time continuity analysis and eliminating the shaking influence of the camera. The method can effectively identify and analyze the pipe gallery video, can quickly perform abnormity early warning, and improves the efficiency of daily operation, maintenance and management of the pipe gallery.
Description
Technical Field
The invention belongs to the technical field of computer digital image processing and pattern recognition, relates to the field of intelligent analysis of pipe gallery videos, and particularly relates to a pipe gallery video intelligent analysis method and system based on cloud processing.
Background
The utility tunnel (also called as 'common ditch' or 'common pipeline') is an underground city pipeline utility tunnel, namely, a tunnel space is built underground, various engineering pipelines such as electric power, communication, gas, heat supply, water supply and drainage and the like are integrated, a special access hole, a lifting hole and a monitoring system are arranged, unified planning, unified design, unified construction and management are implemented, and the utility tunnel is an important infrastructure and a 'lifeline' for guaranteeing city operation. The construction meaning of piping lane is significant, and it is in daily piping lane maintenance, mainly relies on artifical inspection or the robot to patrol artifical observation monitoring video analysis and judgement. After the construction of the urban pipe gallery reaches a certain scale, the method is limited in efficiency and cost depending on a manual inspection mode. To sum up, the demand that efficiency improvement was patrolled and examined to the piping lane is compelled to be up to.
With the development of the technology, how to realize the automatic monitoring mode of the comprehensive pipe gallery is a problem which needs to be solved urgently. The video analysis method of the existing implementation method generally has the following problems: 1. the accuracy is low, the robustness is poor, and the method has a good effect only under certain specific scenes; 2. the intelligent degree is low, can only simply must detect the target in the piping lane, and whether this scene is normal can't intelligent recognition, in case take place unusual autoalarm.
Disclosure of Invention
The invention aims to provide a method and a system for intelligently analyzing a pipe gallery video based on cloud processing, so as to solve one or more technical problems. The patrol robot monitoring system can monitor and analyze in real time in the driving process of the patrol robot, can improve the maintenance and operation efficiency of a pipe gallery, and saves the manpower analysis cost.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention discloses a cloud processing-based intelligent analysis method for a pipe gallery video, which comprises the following steps of:
step 1, acquiring normal and abnormal learning sample sets of a pipe gallery;
step 2, training the pre-constructed deep neural network by using the normal and abnormal learning sample sets of the pipe gallery obtained in the step 1 to obtain a trained network model;
step 3, carrying out anomaly detection on the image to be recognized by using the trained network model in the step 2 to obtain an anomaly detection result;
step 4, screening the abnormity detection result obtained in the step 3 by using an abnormity threshold value, eliminating fuzzy sample interference and obtaining a screened result;
step 5, performing image enhancement processing on the result obtained in the step 4, reducing interference of a weak light source condition, and obtaining a result after enhancement processing;
and 6, performing time domain factor limitation on the result obtained in the step 5, reducing the shaking interference of the camera, and completing intelligent analysis of the pipe gallery video.
The invention has the further improvement that in the step 1, the normal and abnormal learning sample sets of the pipe gallery comprise: a positive sample and a negative sample; wherein, positive sample includes the abnormal target of pipe gallery that awaits measuring, and negative sample includes the normal image of pipe gallery.
The invention has the further improvement that the step 1 specifically comprises the following steps: the on-site videos of a plurality of pipe racks are collected through the patrol robots installed on the pipe rack top runway, and a normal and abnormal learning sample set of the pipe racks is formed.
In a further improvement of the present invention, in step 2, the pre-constructed deep neural network is Resnet, densnet, VGG or Alexnet.
In step 2, the step of training the pre-constructed deep neural network comprises the following steps:
oversampling and increasing the sample set affine transformation balances the frequency between sample classes;
random translation and rotation transformation are carried out on the training images of the sample set, the camera shaking condition is simulated, and the network generalization performance is improved.
The invention has the further improvement that the step 3 specifically comprises the following steps: and analyzing the normal and abnormal conditions of the corridor by adopting a multi-scale analysis mechanism to obtain an abnormal detection result.
The invention has the further improvement that the step 4 specifically comprises the following steps: and (4) screening threshold values, carrying out softmax normalization processing on the results, and displaying the abnormal confidence coefficient greater than the abnormal threshold value.
The invention has the further improvement that the step 5 specifically comprises the following steps:
5.1 Luminance calculation is performed on the image, and an image smaller than a predetermined threshold is regarded as a dark image;
5.2 Brightness enhancement is performed on the dim image and then detection is performed, and the output is used as the detection result.
The invention is further improved in that the step 6 specifically comprises:
6.1 When a normal or abnormal result is detected in the first frame, the counter starts counting and saves each result without outputting;
6.2 When the subsequent frame detects the corresponding category result, the corresponding counter is accumulated;
6.3 When a frame with a detection result different from that before occurs, performing counter analysis; when the count is greater than the confidence coefficient threshold value, turning down the abnormal display threshold value; and when the confidence coefficient is less than or equal to the confidence coefficient threshold value, the abnormal display threshold value is increased.
The invention discloses a pipe gallery video intelligent analysis system based on cloud processing, which comprises:
the sample set acquisition module is used for acquiring normal and abnormal learning sample sets of the pipe gallery;
the network model acquisition module is used for training the pre-constructed deep neural network by using the normal and abnormal learning sample sets of the pipe gallery to obtain a trained network model;
the anomaly detection result acquisition module is used for carrying out anomaly detection on the image to be recognized by using the trained network model to obtain an anomaly detection result;
the screening module is used for screening the abnormal detection result by using the abnormal threshold value, eliminating the interference of the fuzzy sample and obtaining the screened result;
the enhancement module is used for carrying out image enhancement processing on the result obtained by the screening module, reducing the interference of the condition of a weak light source and obtaining the result after the enhancement processing;
and the time domain factor limiting module is used for limiting time domain factors of the result obtained by the enhancing module, reducing the shaking interference of the camera and completing intelligent analysis of the pipe gallery video.
Compared with the prior art, the invention has the following beneficial effects:
according to the intelligent analysis method for the pipe gallery video based on cloud processing, disclosed by the invention, various normal and abnormal videos are acquired on line through a patrol robot, and the videos are processed into pictures to form a pipe gallery abnormal image learning sample set; generating a deep learning network classifier based on sample training learning; detecting abnormal conditions of newly acquired images to be identified by adopting a deep neural network classifier; and finally, performing time domain analysis through continuous frame detection results to accurately identify abnormal results. The invention can effectively identify and analyze various abnormal conditions of the pipe gallery, can quickly carry out daily operation and maintenance of the pipe gallery, and enhances the utilization efficiency of urban underground space. In conclusion, the robot collects the video of the pipe gallery, and the AI algorithm identifies and analyzes the video, so that irregular laying and abnormal state detection of the pipe gallery in the visual field are realized, high-efficiency enabling of maintaining the pipe gallery by the pipe gallery can be realized, and the problems of untimely manual visual identification and personal experience judgment, non-systematization, incomprehension, easy fatigue of manual work, easy error and the like are solved to a great extent.
The system disclosed by the invention is superior to the traditional offline video analysis, can process the abnormal conditions of the pipe gallery in real time during operation, can monitor and analyze the abnormal conditions in real time during the running process of the patrol robot, can improve the maintenance and operation efficiency of the pipe gallery and save the manpower analysis cost. In conclusion, the robot inspection system can seamlessly migrate to various pipe gallery scenes for automatic inspection alarm by the aid of the robot inspection and the AI (artificial intelligence) analysis technology, and has the accuracy rate of over 95 percent.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art are briefly introduced below; it is obvious that the drawings in the following description are some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic flowchart of an intelligent analysis method for a pipe gallery video based on cloud processing according to an embodiment of the present invention;
FIG. 2 is a sample set diagram for deep neural network training in an embodiment of the present invention;
FIG. 3 is a schematic view of the tube lane image viewing mode in the embodiment of the present invention;
FIG. 4 is a diagram illustrating an anomaly detection result according to an embodiment of the present invention.
Detailed Description
In order to make the purpose, technical effect and technical solution of the embodiments of the present invention clearer, the following clearly and completely describes the technical solution of the embodiments of the present invention with reference to the drawings in the embodiments of the present invention; it is to be understood that the described embodiments are only some of the embodiments of the present invention. Other embodiments, which can be derived by one of ordinary skill in the art from the disclosed embodiments without inventive faculty, are intended to be within the scope of the invention.
Referring to fig. 1, a method for intelligently analyzing a pipe gallery video based on cloud processing according to an embodiment of the present invention includes the following steps:
step 1: acquiring normal and abnormal learning sample sets of a pipe gallery;
and 2, step: training the deep neural network by using the data of the sample set; specifically, neural networks which can be adopted by the method of the invention include Resnet, densenet, VGG and Alexnet, and the trained network model is stored on a local hard disk;
and step 3: carrying out anomaly detection on the newly acquired image to be identified by adopting a trained deep neural network classification model to obtain an anomaly detection result;
and 4, step 4: screening an abnormality detection result by using an abnormality threshold value to eliminate fuzzy sample interference;
and 5: carrying out image enhancement processing on the result of the step 4, and reducing the interference of the condition of a weak light source;
step 6: and (5) limiting time domain factors on the result of the step (5) and reducing the shaking interference of the camera.
In the embodiment of the invention, the normal and abnormal image learning sample sets of the pipe gallery obtained in the step 1 are neural network learning sample sets: the method is used for training a deep neural network classifier and comprises positive samples and negative samples; the positive sample contains the abnormal target of the pipe gallery to be detected, and the negative sample is a normal image of the pipe gallery.
In the embodiment of the invention, step 2 trains the network. The method comprises the following specific steps:
2.1 Add affine transformation before training, oversample all class pictures to a certain value. Avoid the influence of uneven frequency distribution among all classes.
2.2 The images are subjected to random translation transformation before training, the camera shaking condition is simulated to a certain extent, and the network generalization is increased.
In the embodiment of the invention, the step 3 of performing anomaly detection on the newly acquired image to be identified by adopting the deep neural network model comprises the following steps:
3.1 Cutting the image of the tube corridor in the visual field to respectively represent the conditions of the left side, the right side, the middle and the top of the tube corridor;
3.2 Respectively analyzing the four regions to obtain the result of whether the region is abnormal or not;
in the embodiment of the invention, threshold value screening is adopted in the step 4, firstly, softmax normalization processing is carried out on the detection result to obtain confidence degrees of various types, and the abnormal confidence degrees which are larger than the abnormal threshold value are displayed.
The operation process of softmax is as follows:
where V represents the output matrix and Vi represents the ith element in V
In the embodiment of the invention, the step 5 adopts image enhancement processing to reduce interference results of weak light source conditions and screen out the interference results, and comprises the following steps:
5.1 Brightness calculation is performed on the test image, and a dark image is considered when the brightness is less than a certain threshold value;
5.2 Brightness enhancement and detection of the dim image and output as the final result;
in the embodiment of the invention, step 6 adopts time domain factors for limitation, reduces the shaking influence of the camera, and comprises the following steps:
6.1 When a normal or abnormal result is detected in the first frame, the counter starts counting and saves each result without outputting;
6.2 When the subsequent frame detects the corresponding category result, the corresponding counter is accumulated;
6.3 When a frame with a different detection result from the previous frame appears, the counter analyzes, and when the count is greater than the confidence threshold, the abnormal display threshold is reduced; and when the confidence coefficient is smaller than the confidence coefficient threshold value, the abnormal display threshold value is increased.
The embodiment of the invention provides a cloud processing-based intelligent analysis method for a pipe gallery video, which comprises the steps of collecting various normal and abnormal videos on line through a patrol robot, processing the videos into pictures, and forming a pipe gallery abnormal image learning sample set; generating a deep learning network classifier based on sample training learning; detecting abnormal conditions of newly acquired images to be identified by adopting a deep neural network classifier; and finally, performing time domain analysis through continuous frame detection results to accurately identify abnormal results. The invention can effectively identify and analyze various abnormal conditions of the pipe gallery, can quickly carry out daily operation and maintenance of the pipe gallery, and enhances the utilization efficiency of urban underground space.
Referring to fig. 1, a method for intelligently analyzing a pipe gallery video according to an embodiment of the present invention includes the following steps:
step 1, obtain the normal and unusual study sample set of piping lane, include:
in the embodiment of the invention, the positive and negative sample sets of the pipe gallery are used as the deep neural network learning sample set and used for training the deep neural network classifier. Specifically, a plurality of on-site videos of the pipe gallery are collected through the patrol robot arranged on the runway on the top of the pipe gallery, and a normal and abnormal learning sample set of the pipe gallery is formed. The deep neural network learning sample set comprises a positive sample and a negative sample, the positive sample data size is 11486, the negative sample is 31955, and fig. 2 is a partial deep neural network learning sample set. The positive sample must contain the tube lane abnormal target to be detected, and the negative sample contains various normal types.
Step 2, training a deep neural network classifier, comprising:
according to the embodiment of the invention, a sample set is learned through a deep neural network, the prediction result of the network is similar to the real result by continuously reducing the error between the real label and the prediction label, and a deep neural network classifier is trained by adopting a deep learning algorithm.
The method specifically comprises the following steps:
(1) Carrying out affine transformation on the data set, oversampling all samples to 7000, and solving the problem of inter-class frequency imbalance;
(2) And the data set is randomly translated, camera shaking is simulated, shaking influence is reduced, and network generalization is increased.
(3) Training 10 epochs by using a deep neural network to generate a deep neural network classifier, and selecting and storing a model with the best performance from the 10 epochs.
Step 3, analyzing the abnormal condition of the newly acquired image to be recognized by adopting a trained deep neural network classifier;
the embodiment of the invention adopts a multi-scale analysis mechanism to analyze the normal and abnormal conditions of the corridor, and comprises the following specific steps:
1) Cutting the image of the tube corridor in the field of vision, as shown in fig. 3, respectively representing the conditions of the left side, the right side, the middle and the top of the tube corridor; each box can be uniquely determined by the coordinates of two points: the upper left corner and the lower right corner. By using the upper left corner of the image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, a coordinate system is established, and then the positions of the four frames can be expressed as:
left rectangular frame: (80, 150), (680, 1070), middle matrix box: (800, 700), (1100, 1080), right matrix box: (1100, 150), (1800, 950), top rectangular box: (690, 40), (1090, 320);
2) And analyzing the four regions respectively to obtain the result of whether the region is abnormal or not.
And 4, screening threshold values, carrying out softmax normalization processing on the detection result, and displaying the frames with the abnormal confidence degrees larger than the abnormal threshold values.
And 5, adopting picture enhancement processing to reduce interference results of weak light source conditions and screen out, wherein the method specifically comprises the following steps:
1) Performing brightness calculation on the test image, and regarding the test image smaller than a certain threshold value as a dark image;
2) Carrying out brightness enhancement on the dim image, then detecting, and outputting as a final result;
and 6, limiting by adopting time domain factors, and reducing the shaking influence of the camera, wherein the method comprises the following steps:
1) When the first frame detects a normal or abnormal result, the counter starts counting, and each result is stored without being output;
2) When the subsequent frame detects the corresponding category result, the corresponding counter is accumulated;
3) When a frame with a different detection result from the previous frame appears, the counter analyzes, and when the count is greater than the confidence coefficient threshold, the abnormal display threshold is reduced; and when the confidence coefficient is smaller than the confidence coefficient threshold value, the abnormal display threshold value is increased.
The experimental result of the embodiment of the invention shows that, as shown in fig. 4, by the technical scheme of the invention, the tube corridor abnormal conditions which randomly appear under the complex background can be accurately analyzed, the detection speed of more than 25 frames per second is achieved, and the accuracy (correct classification quantity/total image quantity) is more than 95%. Compared with the existing method, the method has the advantages that the recognition accuracy is higher, the robustness of the algorithm is enhanced, and the real-time capture and analysis of the abnormal conditions of the pipe gallery are realized. The invention departs from the traditional off-line video analysis, can process the abnormal condition of the pipe gallery in real time during operation, can monitor and analyze in real time during the running process of the patrol robot, can improve the maintenance and operation efficiency of the pipe gallery and save the manpower analysis cost.
At present, the deep neural network has made great progress in the field of image classification, and the invention can adopt various deep neural networks to carry out video analysis, wherein: resnet, densenet, VGG, alexnet. Through a plurality of experimental tests, resnet obtains comparative balance between speed and accuracy. The invention achieves the best effect when the Resnet deep neural network is adopted, but is not limited to the Resnet deep neural network. The Resnet network is a classic deep residual network structure, proposed in 2015 by He Kaiming of microsoft research institute, and is described in detail in the literature: he K, zhang X, ren S, et al. Deep reactive Learning for Image Recognition [ J ].2015:770-778 the basic idea is to propose a residual learning framework to simplify the training of much deeper networks than previously used. Layers are redefined to learn residual functions with reference to layer inputs, rather than learning unreferenced functions. Experiments prove that the deeper network has better performance, and the resnet can realize network training deeper than other structures.
The embodiment of the invention provides a pipe gallery video intelligent analysis system based on cloud processing, which comprises:
the sample set acquisition module is used for acquiring normal and abnormal learning sample sets of the pipe gallery;
the network model acquisition module is used for training the pre-constructed deep neural network by using the normal and abnormal learning sample sets of the pipe gallery to obtain a trained network model;
the anomaly detection result acquisition module is used for carrying out anomaly detection on the image to be recognized by using the trained network model to obtain an anomaly detection result;
the screening module is used for screening the abnormal detection result by using the abnormal threshold value, eliminating the interference of the fuzzy sample and obtaining the screened result;
the enhancement module is used for carrying out image enhancement processing on the result obtained by the screening module, reducing the interference of the weak light source condition and obtaining the result after the enhancement processing;
and the time domain factor limiting module is used for limiting time domain factors of the result obtained by the enhancing module, reducing the shaking interference of the camera and completing intelligent analysis of the pipe gallery video.
The invention discloses an intelligent analysis method for a pipe gallery video, which comprises the following steps: acquiring normal and abnormal learning sample sets of a pipe gallery; training a deep neural network classifier; adopting a trained deep neural network classifier to perform intelligent analysis on newly acquired images to be recognized; carrying out data enhancement processing to eliminate frequency non-uniform interference among sample categories; preprocessing the image with too dark brightness by using a picture enhancement algorithm to eliminate the interference of a weak light source; and (5) carrying out time continuity analysis and eliminating the camera shaking influence. The method can effectively identify and analyze the pipe gallery video, can quickly perform abnormity early warning, and improves the efficiency of daily operation, maintenance and management of the pipe gallery.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Therefore, the temperature of the molten metal is controlled, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can make modifications and equivalents to the embodiments of the present invention without departing from the spirit and scope of the present invention, which is set forth in the claims of the present application.
Claims (5)
1. A pipe gallery video intelligent analysis method based on cloud processing is characterized by comprising the following steps:
step 1, acquiring normal and abnormal learning sample sets of a pipe gallery;
step 2, training the pre-constructed deep neural network by using the normal and abnormal learning sample sets of the pipe gallery obtained in the step 1 to obtain a trained network model;
step 3, carrying out anomaly detection on the image to be recognized by using the trained network model in the step 2 to obtain an anomaly detection result;
step 4, screening the abnormity detection result obtained in the step 3 by using an abnormity threshold value, eliminating fuzzy sample interference and obtaining a screened result;
step 5, performing image enhancement processing on the result obtained in the step 4, reducing interference of a weak light source condition, and obtaining a result after enhancement processing;
step 6, limiting time domain factors on the result obtained in the step 5, and reducing camera shaking interference;
wherein, step 4 specifically includes: threshold value screening is adopted, softmax normalization processing is carried out on the result, and the abnormal confidence coefficient is displayed when being larger than the abnormal threshold value;
the step 5 specifically comprises the following steps:
5.1 Luminance calculation is performed on the image, and an image smaller than a predetermined threshold is regarded as a dark image;
5.2 Brightness enhancement is carried out on the dim image, and then detection is carried out, and the output is used as the detection result;
the step 6 specifically comprises the following steps:
6.1 When a normal or abnormal result is detected in the first frame, the counter starts counting and saves each result without outputting;
6.2 When the subsequent frame detects the corresponding category result, the corresponding counter is accumulated;
6.3 When a frame with a detection result different from that before occurs, performing counter analysis; when the count is greater than the confidence coefficient threshold value, turning down the abnormal display threshold value; and when the confidence coefficient is less than or equal to the confidence coefficient threshold value, the abnormal display threshold value is increased.
2. The intelligent analysis method for the pipe gallery video based on cloud processing according to claim 1, wherein in the step 1, the normal and abnormal learning sample sets of the pipe gallery comprise: a positive sample and a negative sample; wherein, positive sample includes the abnormal target of pipe gallery that awaits measuring, and negative sample includes the normal image of pipe gallery.
3. The intelligent pipe gallery video analysis method based on cloud processing as claimed in claim 2, wherein step 1 specifically includes: a plurality of on-the-spot videos of the pipe gallery are collected through a patrol robot installed on a runway on the top of the pipe gallery, and a normal and abnormal learning sample set of the pipe gallery is formed.
4. The cloud-processing-based pipe gallery video intelligent analysis method according to claim 1, wherein in step 2, the pre-constructed deep neural network is Resnet, densnet, VGG, or Alexnet.
5. The utility model provides a piping lane video intelligence analytic system based on high in clouds is handled which characterized in that includes:
the sample set acquisition module is used for acquiring normal and abnormal learning sample sets of the pipe gallery;
the network model acquisition module is used for training the pre-constructed deep neural network by using the normal and abnormal learning sample sets of the pipe gallery to obtain a trained network model;
the anomaly detection result acquisition module is used for carrying out anomaly detection on the image to be recognized by using the trained network model to obtain an anomaly detection result;
the screening module is used for screening the abnormal detection result by using the abnormal threshold value, eliminating the interference of the fuzzy sample and obtaining the screened result;
the enhancement module is used for carrying out image enhancement processing on the result obtained by the screening module, reducing the interference of the condition of a weak light source and obtaining the result after the enhancement processing;
the time domain factor limiting module is used for limiting the time domain factor of the result obtained by the enhancing module and reducing the shaking interference of the camera;
the method comprises the following steps of screening an abnormal detection result by using an abnormal threshold value, eliminating fuzzy sample interference, and obtaining a screened result, wherein the steps of screening the abnormal detection result by using the abnormal threshold value specifically comprise: threshold value screening is adopted, softmax normalization processing is carried out on the result, and the abnormal confidence coefficient is displayed when being larger than the abnormal threshold value;
the image enhancement processing is carried out on the result obtained by the screening module, the interference of the condition of a weak light source is reduced, and the step of obtaining the result after the enhancement processing specifically comprises the following steps:
5.1 Luminance calculation is performed on the image, and an image smaller than a predetermined threshold is regarded as a dark image;
5.2 Brightness enhancement is carried out on the dim image, and then detection is carried out, and the output is used as the detection result;
the step of limiting time domain factors on the result obtained by the enhancement module and reducing the camera shaking interference specifically comprises the following steps:
6.1 When a normal or abnormal result is detected in the first frame, the counter starts counting and saves each result without outputting;
6.2 When the subsequent frame detects the corresponding category result, the corresponding counter is accumulated;
6.3 When a frame with a detection result different from that before occurs, performing counter analysis; when the count is greater than the confidence coefficient threshold value, turning down the abnormal display threshold value; and when the confidence coefficient is less than or equal to the confidence coefficient threshold value, the abnormal display threshold value is increased.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110128858.6A CN112861681B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent analysis method and system based on cloud processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110128858.6A CN112861681B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent analysis method and system based on cloud processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112861681A CN112861681A (en) | 2021-05-28 |
| CN112861681B true CN112861681B (en) | 2023-04-07 |
Family
ID=75986983
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110128858.6A Active CN112861681B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent analysis method and system based on cloud processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112861681B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113989257A (en) * | 2021-11-09 | 2022-01-28 | 国网江苏省电力有限公司南通供电分公司 | Electric power comprehensive pipe gallery settlement crack identification method based on artificial intelligence technology |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB1396446A (en) * | 1972-09-25 | 1975-06-04 | Prb Sa | Process and apparatus for creating a triggered temporary barrier for inhibiting the spread of an explosion and or fire in a mine or underground gallery |
| CN109460744A (en) * | 2018-11-26 | 2019-03-12 | 南京邮电大学 | A kind of video monitoring system based on deep learning |
| CN111401131A (en) * | 2020-02-13 | 2020-07-10 | 深圳供电局有限公司 | Image processing method and device for tunnel pipe gallery, computer equipment and storage medium |
| CN111462326A (en) * | 2020-03-31 | 2020-07-28 | 武汉大学 | Low-cost 360-degree panoramic video camera urban pipeline three-dimensional reconstruction method and system |
| CN111612018A (en) * | 2020-04-24 | 2020-09-01 | 国网河北省电力有限公司雄安新区供电公司 | Underground pipe gallery monitoring method and system based on multi-source heterogeneous data fusion |
| CN111858926A (en) * | 2020-06-08 | 2020-10-30 | 北京市基础设施投资有限公司(原北京地铁集团有限责任公司) | Pipe gallery equipment fault prediction method and system based on multi-element heterogeneous data |
| CN111950585A (en) * | 2020-06-29 | 2020-11-17 | 广东技术师范大学 | XGboost-based underground comprehensive pipe gallery safety condition assessment method |
-
2021
- 2021-01-29 CN CN202110128858.6A patent/CN112861681B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB1396446A (en) * | 1972-09-25 | 1975-06-04 | Prb Sa | Process and apparatus for creating a triggered temporary barrier for inhibiting the spread of an explosion and or fire in a mine or underground gallery |
| CN109460744A (en) * | 2018-11-26 | 2019-03-12 | 南京邮电大学 | A kind of video monitoring system based on deep learning |
| CN111401131A (en) * | 2020-02-13 | 2020-07-10 | 深圳供电局有限公司 | Image processing method and device for tunnel pipe gallery, computer equipment and storage medium |
| CN111462326A (en) * | 2020-03-31 | 2020-07-28 | 武汉大学 | Low-cost 360-degree panoramic video camera urban pipeline three-dimensional reconstruction method and system |
| CN111612018A (en) * | 2020-04-24 | 2020-09-01 | 国网河北省电力有限公司雄安新区供电公司 | Underground pipe gallery monitoring method and system based on multi-source heterogeneous data fusion |
| CN111858926A (en) * | 2020-06-08 | 2020-10-30 | 北京市基础设施投资有限公司(原北京地铁集团有限责任公司) | Pipe gallery equipment fault prediction method and system based on multi-element heterogeneous data |
| CN111950585A (en) * | 2020-06-29 | 2020-11-17 | 广东技术师范大学 | XGboost-based underground comprehensive pipe gallery safety condition assessment method |
Non-Patent Citations (3)
| Title |
|---|
| 卓金武.小波分析程序设计综合案例.《MATLAB在数学建模中的应用 第2版》.北京航空航天大学出版社,2014, * |
| 柳林.仿射变换.《基于OpenCV的数字图像处理技术》.浙江大学出版社,2020,第82-83页. * |
| 王以中等.综合管廊风险防控措施.《城市生命线风险防控》.同济大学出版社,2019,第422-423页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112861681A (en) | 2021-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107123131B (en) | Moving target detection method based on deep learning | |
| CN109902633A (en) | Accident detection method and device based on the camera supervised video of fixed bit | |
| CN107782733A (en) | Image recognition the cannot-harm-detection device and method of cracks of metal surface | |
| CN111080634A (en) | Transformer appearance defect identification method based on inspection robot and Cascade RCNN algorithm | |
| CN113903081A (en) | Visual identification artificial intelligence alarm method and device for images of hydraulic power plant | |
| CN112541393A (en) | Transformer substation personnel detection method and device based on deep learning | |
| CN111339907A (en) | Pollution discharge identification method and device based on image identification technology | |
| CN111767826B (en) | Timing and fixed-point scene anomaly detection method | |
| CN116229052B (en) | Method for detecting state change of substation equipment based on twin network | |
| CN112861681B (en) | Pipe gallery video intelligent analysis method and system based on cloud processing | |
| CN113469938B (en) | Pipe gallery video analysis method and system based on embedded front-end processing server | |
| CN112766795B (en) | Cloud processing-based pipe gallery intelligent information management method and system | |
| CN116486240A (en) | Application of image recognition algorithm in intelligent inspection method of unmanned aerial vehicle of power transmission line | |
| CN116993665A (en) | Intelligent detection method for construction progress of construction engineering working face based on computer vision | |
| CN114241522A (en) | Method, system, equipment and storage medium for field operation safety wearing identification | |
| Zhang et al. | Belt deviation detection system based on deep learning under complex working conditions | |
| CN112784914B (en) | Pipe gallery video intelligent attribute detection method and system based on cloud processing | |
| CN114881665A (en) | Method and system for identifying electricity stealing suspected user based on target identification algorithm | |
| Daogang et al. | Anomaly identification of critical power plant facilities based on YOLOX-CBAM | |
| CN112633157A (en) | AGV working area safety real-time detection method and system | |
| CN112070740A (en) | BIM-based port storage yard accumulated water imaging method, system and control equipment | |
| CN112818836B (en) | Method and system for detecting personnel target of transformer substation scene | |
| CN114387564A (en) | Head-knocking engine-off pumping-stopping detection method based on YOLOv5 | |
| Wang et al. | Visual defect detection for substation equipment based on joint inspection data of camera and robot | |
| CN113850144A (en) | Video reliability automatic inspection system based on image recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: An Intelligent Analysis Method and System for Pipe Gallery Video Based on Cloud Processing Effective date of registration: 20231010 Granted publication date: 20230407 Pledgee: Zhejiang Tailong Commercial Bank Co.,Ltd. Huzhou Changxing sub branch Pledgor: Changxing yunshang Technology Co.,Ltd. Registration number: Y2023980060343 |