CN112784914A - Pipe gallery video intelligent attribute detection method and system based on cloud processing - Google Patents
Pipe gallery video intelligent attribute detection method and system based on cloud processing Download PDFInfo
- Publication number
- CN112784914A CN112784914A CN202110130569.XA CN202110130569A CN112784914A CN 112784914 A CN112784914 A CN 112784914A CN 202110130569 A CN202110130569 A CN 202110130569A CN 112784914 A CN112784914 A CN 112784914A
- Authority
- CN
- China
- Prior art keywords
- pipe gallery
- attribute
- image
- result
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 81
- 238000012545 processing Methods 0.000 title claims abstract description 47
- 238000000034 method Methods 0.000 claims abstract description 32
- 238000012216 screening Methods 0.000 claims abstract description 23
- 238000012549 training Methods 0.000 claims abstract description 17
- 238000002372 labelling Methods 0.000 claims abstract description 8
- 238000004458 analytical method Methods 0.000 claims description 32
- 238000003062 neural network model Methods 0.000 claims description 21
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000010606 normalization Methods 0.000 claims description 4
- 230000011218 segmentation Effects 0.000 claims description 4
- 238000012163 sequencing technique Methods 0.000 claims description 4
- 238000013528 artificial neural network Methods 0.000 abstract description 27
- 238000012423 maintenance Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 14
- 238000004590 computer program Methods 0.000 description 7
- 238000007689 inspection Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 238000012360 testing method Methods 0.000 description 6
- 238000013473 artificial intelligence Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 238000012544 monitoring process Methods 0.000 description 5
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000007726 management method Methods 0.000 description 4
- 238000010276 construction Methods 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- OLBCVFGFOZPWHH-UHFFFAOYSA-N propofol Chemical compound CC(C)C1=CC=CC(C(C)C)=C1O OLBCVFGFOZPWHH-UHFFFAOYSA-N 0.000 description 1
- 229960004134 propofol Drugs 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a method and a system for detecting intelligent attributes of a pipe gallery video based on cloud processing, wherein the method comprises the following steps: labeling various attribute learning sample sets of the pipe gallery; constructing a deep neural network, and training the neural network by using data of a sample set; carrying out attribute detection on the newly acquired image to be identified by using the trained network model; screening the attribute detection result by using a confidence threshold value to eliminate fuzzy target interference; the support is used as a background landmark object for positioning the pipeline, and the positioning dividing line can be adjusted in a self-adaptive manner along with the movement of the camera; and time domain factor limitation is carried out, camera shaking interference is reduced, and the influence of error detection results is weakened. The method can effectively identify and analyze the pipe gallery video, can quickly detect the attribute of the pipe gallery, and improves the efficiency of daily operation, maintenance and management of the pipe gallery.
Description
Technical Field
The invention belongs to the technical field of computer digital image processing and pattern recognition, relates to the field of intelligent analysis of pipe gallery videos, and particularly relates to a method and a system for detecting intelligent attributes of pipe gallery videos based on cloud processing.
Background
The utility tunnel (also called as 'common ditch' or 'common pipeline') is an underground city pipeline utility tunnel, namely, a tunnel space is built underground, various engineering pipelines such as electric power, communication, gas, heat supply, water supply and drainage and the like are integrated, a special access hole, a lifting hole and a monitoring system are arranged, unified planning, unified design, unified construction and management are implemented, and the utility tunnel is an important infrastructure and a 'lifeline' for guaranteeing city operation. The construction meaning of piping lane is significant, and it is in daily piping lane maintenance, mainly relies on artifical inspection or the robot to patrol artifical observation monitoring video analysis and judgement. After the construction of the urban pipe gallery reaches a certain scale, the method is limited in efficiency and cost depending on a manual inspection mode. To sum up, the demand that efficiency improvement was patrolled and examined to the piping lane is compelled to be up to.
In the traditional execution process, the management of the pipeline attributes in the pipe rack mainly depends on the manual judgment of workers in the inspection process, and the number of the optical cables and cables in the pipe rack is counted and reported to a data center for unified management, but the method has the following problems: 1. the efficiency is low, in order to check the number of cables in one pipe gallery, workers are required to walk through the whole pipe gallery, and records are carried out at each fork of the pipe gallery; 2. the accuracy is low, the number of general pipelines cannot be changed suddenly, but the disconnection or the theft of a certain pipeline is not excluded, and the situation cannot be identified according to the traditional method of counting only at each fork, so that great trouble is caused to operation and maintenance.
Disclosure of Invention
The invention aims to provide a method and a system for detecting intelligent attributes of a pipe gallery video based on cloud processing, so as to solve one or more technical problems. The intelligent attribute detection method for the video of the pipe gallery can detect and record the attributes and position conditions of various pipelines of the pipe gallery in the patrol process of the robot.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention discloses a pipe gallery video intelligent attribute detection method based on cloud processing, which comprises the following steps of:
step 2, training the pre-constructed deep neural network model by using the attribute learning sample set obtained in the step 1 to obtain a trained pipe gallery attribute analysis deep neural network model;
step 3, using the pipe gallery attribute analysis deep neural network model trained in the step 2 to perform attribute detection on the image to be recognized, comprising the following steps:
1) carrying out image enhancement processing on an image to be identified, wherein the image enhancement processing is used for reducing interference of a weak light source condition;
2) performing attribute analysis detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model to obtain an attribute detection result;
3) screening the attribute detection result by using a confidence threshold value, and eliminating fuzzy target interference to obtain a screening result;
4) based on the screening result, using the bracket as a background landmark object to perform pipeline positioning to obtain a positioning result; the positioning dividing line can be adjusted in a self-adaptive manner along with the movement of the camera;
5) and time domain factor limitation is carried out on the positioning result, so that the camera shaking interference is reduced, and the influence of error detection results is weakened.
A further refinement of the invention is that in step 1 the property learning sample set for the pipe gallery comprises an xml suffix tag file labelled with labelImg.
In step 2, the deep neural network model is YOLO, Fast-RCNN or SSD.
The further improvement of the present invention is that, in step 2, the training of the deep neural network model includes: and converting the learning sample set into a VOC standard data set format.
The invention is further improved in that step 1) of step 3 specifically comprises:
1.1) carrying out brightness calculation on the image, and regarding the image smaller than a preset threshold as a dark image;
1.2) carrying out brightness enhancement on the dim image, then detecting until the brightness is greater than or equal to a preset threshold value, and outputting the brightness as a final result.
The invention is further improved in that step 2) of step 3 specifically comprises:
2.1) cutting the image of the pipe gallery in the visual field to obtain a left area and a right area which respectively represent the left condition and the right condition of the pipe gallery;
2.2) analyzing the two areas respectively to obtain the attribute detection results of the left area and the right area.
A further improvement of the present invention is that step 3) of step 3 specifically comprises: and (4) performing softmax normalization processing on the detection result by adopting threshold value screening, and displaying the detection confidence coefficient which is greater than the confidence threshold value.
The invention is further improved in that step 4) of step 3 specifically comprises:
4.1) detecting the supports in the visual field, respectively carrying out sequencing treatment according to the visual field positions, and screening a row of supports closest to the camera as positioning reference points;
4.2) adaptively adjusting the position of the segmentation line according to the positioning reference point;
4.3) linear programming to determine the specific area of the pipeline.
The invention is further improved in that step 5) of step 3 specifically comprises:
5.1) when the first frame detects the corresponding pipe gallery attribute, the counter starts counting, and each result is stored without output;
5.2) when the attribute detection is continuously carried out on the subsequent frames, the corresponding counters are accumulated, and the frequency of the same result is accumulated;
and 5.3) when the counter reaches a threshold value, counting frequency counts corresponding to different results, and taking the result with the highest frequency count as a final attribute detection result.
The invention discloses a pipe gallery video intelligent attribute detection system based on cloud processing, which comprises:
the sample acquisition module is used for marking various attribute learning sample sets of the pipe gallery;
the model acquisition module is used for training the pre-constructed deep neural network model by using the obtained attribute learning sample set to obtain a trained pipe gallery attribute analysis deep neural network model;
the attribute analysis module is used for carrying out attribute detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model, and comprises the following steps:
1) carrying out image enhancement processing on an image to be identified, wherein the image enhancement processing is used for reducing interference of a weak light source condition;
2) performing attribute analysis detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model to obtain an attribute detection result;
3) screening the attribute detection result by using a confidence threshold value, and eliminating fuzzy target interference to obtain a screening result;
4) based on the screening result, using the bracket as a background landmark object to perform pipeline positioning to obtain a positioning result; the positioning dividing line can be adjusted in a self-adaptive manner along with the movement of the camera;
5) and time domain factor limitation is carried out on the positioning result, so that the camera shaking interference is reduced, and the influence of error detection results is weakened.
Compared with the prior art, the invention has the following beneficial effects:
the intelligent attribute detection method for the pipe gallery video based on cloud processing comprises the steps of collecting various types of monitoring videos on line through a patrol robot, processing the videos into pictures, and manually marking the pictures to form a pipe gallery attribute image learning sample set; generating a deep learning network target detector based on sample training learning; adopting a deep neural network target detector to perform attribute detection on the newly acquired image to be identified; and eliminating the interference of a weak light source by combining a target detection result and a digital image processing technology, and finally performing time domain analysis through continuous frame detection results to accurately analyze the properties of the pipe gallery. The invention can effectively identify and analyze the quantity and the attributes and the positions of various pipelines in the pipe gallery, can quickly carry out daily operation and maintenance of the pipe gallery, and enhances the utilization efficiency of urban underground space.
The invention provides a novel cloud-processing-based pipe gallery video intelligent attribute detection system by robot inspection and AI (artificial intelligence) analysis technology, inspection is carried out by replacing manpower by an inspection robot, meanwhile, the pipelines in a pipe gallery are continuously analyzed and counted, the result is uploaded to the cloud in real time and BIM reconstruction is carried out, the pipeline condition of the whole pipe gallery can be counted, the pipeline of each road section is recorded, and once a problem occurs, an error area can be positioned at a glance.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art are briefly introduced below; it is obvious that the drawings in the following description are some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
Fig. 1 is a schematic flowchart of a pipe gallery video intelligent attribute detection method based on cloud processing according to an embodiment of the present invention;
FIG. 2 is a sample set diagram for deep neural network training in an embodiment of the present invention; wherein, fig. 2 (a) is a schematic diagram of a sample at the upper left side of the pipe gallery, fig. 2 (b) is a schematic diagram of a sample at the right side of the pipe gallery, and fig. 2 (c) is a schematic diagram of a picture and a corresponding name labeling file thereof;
FIG. 3 is a schematic view of the tube lane image viewing mode in the embodiment of the present invention;
FIG. 4 is a diagram illustrating an attribute detection result according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a logging result after time domain statistics in the embodiment of the present invention.
Detailed Description
In order to make the purpose, technical effect and technical solution of the embodiments of the present invention clearer, the following clearly and completely describes the technical solution of the embodiments of the present invention with reference to the drawings in the embodiments of the present invention; it is to be understood that the described embodiments are only some of the embodiments of the present invention. Other embodiments, which can be derived by one of ordinary skill in the art from the disclosed embodiments without inventive faculty, are intended to be within the scope of the invention.
Referring to fig. 1, a method for detecting intelligent attributes of a pipe gallery video according to an embodiment of the present invention includes the following steps:
step 1: labeling various attribute learning sample sets of the pipe gallery;
step 2: constructing a deep neural network, and training the neural network by using data of a sample set; specifically, neural networks which can be adopted by the method of the invention include YOLO, Fast-RCNN and SSD, and the trained network model is stored on a local hard disk;
and step 3: the method comprises the steps of directly obtaining images from a camera or a video file on a local hard disk and carrying out image enhancement processing, so that interference of a weak light source condition is reduced;
and 4, step 4: carrying out attribute detection on the newly acquired image to be identified by using the trained network model;
and 5: screening the attribute detection result by using a confidence threshold value to eliminate fuzzy target interference;
step 6: the support is used as a background landmark object for positioning the pipeline, and the positioning dividing line can be adjusted in a self-adaptive mode along with the movement of the camera.
And 7: and (4) time domain factor limitation is carried out on the result of the step (6), so that the shaking interference of the camera is reduced, and the influence of an error detection result is weakened.
In the embodiment of the invention, the labeled corridor attribute learning sample set in the step 1 is an SSD neural network learning sample set: for training SSD deep neural network target detectors.
In the embodiment of the invention, step 2 trains the network. The selectable networks are: YOLO, Fast-RCNN, SSD. The learning sample set is converted into a VOC standard data set format, sample generalizability is improved, and model training efficiency is improved.
In the embodiment of the invention, the step 3 adopts image enhancement processing to reduce the interference of the weak light source condition, and comprises the following steps:
3.1) performing brightness calculation on the test image, and regarding the test image smaller than a certain threshold as a dark image;
3.2) carrying out brightness enhancement on the dim image, then detecting the dim image, and outputting the dim image as a final result.
In the embodiment of the invention, in the step 4, a deep neural network target detector is adopted to perform target detection on the newly acquired image to be identified, and the steps are as follows:
4.1) cutting the image of the pipe gallery in the visual field, wherein the image represents the left and right conditions of the pipe gallery respectively;
and 4.2) analyzing the two regions respectively to obtain the attribute detection results of the left region and the right region.
In the embodiment of the invention, threshold value screening is adopted in the step 5, firstly, softmax normalization processing is carried out on the detection result to obtain confidence degrees of various categories, and the detection confidence degrees which are greater than the threshold values are displayed.
The operation process of softmax is as follows:
where V represents the output matrix and Vi represents the ith element in V
In the embodiment of the invention, in the step 6, the bracket is used as a landmark object to position the pipeline, and the steps comprise:
6.1) detecting the supports in the visual field, respectively carrying out sequencing processing according to the visual field positions, and screening a row of supports closest to the camera as positioning reference points;
6.2) adaptively adjusting the position of the segmentation line according to the positioning reference point;
6.3) linear programming to determine the specific area of the pipeline.
In the embodiment of the invention, step 7 adopts time domain factors for limitation, reduces the shaking influence of the camera, and comprises the following steps:
7.1) when the first frame detects the corresponding pipe gallery attribute, the counter starts counting, and each result is stored without output;
7.2) when the attribute detection is continuously carried out on the subsequent frames, the corresponding counters are accumulated, and the frequency of the same result is accumulated;
and 7.3) when the counter reaches a threshold value, counting frequency counts corresponding to different results, and outputting the highest frequency count as a final attribute detection result. And saving the detection result, resetting the counter and the frequency count, and starting the next round of detection.
According to the intelligent attribute detection method for the pipe gallery video based on cloud processing, provided by the embodiment of the invention, various types of monitoring videos are collected on line through a patrol robot, the videos are processed into pictures, and a pipe gallery attribute image learning sample set is formed through manual labeling; generating a deep learning network target detector based on sample training learning; adopting a deep neural network target detector to perform attribute detection on the newly acquired image to be identified; and eliminating the interference of a weak light source by combining a target detection result and a digital image processing technology, and finally performing time domain analysis through continuous frame detection results to accurately analyze the properties of the pipe gallery. The invention can effectively identify and analyze the quantity and the attributes and the positions of various pipelines in the pipe gallery, can quickly carry out daily operation and maintenance of the pipe gallery, and enhances the utilization efficiency of urban underground space.
With the development of technology, a mode of realizing the automatic monitoring of the utility tunnel through a robot patrol + AI (artificial intelligence) analysis technology has appeared now. The robot collects the video of the pipe gallery, and the AI algorithm identifies and analyzes to detect various pipeline attributes in the pipe gallery in the view. The high-efficient enabling that combines location coordinate and warning record, realization piping lane maintained solves "artifical naked eye discernment + personal experience judges" untimely, not systematized, and not incomprehensible and artifical easy fatigue, easy mistake scheduling problem to a great extent.
The deep neural network has made great progress in the field of image target detection, and the invention can adopt various deep neural networks to carry out video analysis, wherein: YOLO, Fast-RCNN, SSD. Through many experimental tests, the SSD obtains a comparative balance between speed and accuracy. The invention achieves the best effect when the SSD deep neural network is adopted, but is not limited to the SSD deep neural network.
The SSD network is an object detection algorithm that directly predicts coordinates and categories of bounding box, and was proposed by liu wei of UNC Chapel Hill (church mountain school of university of north carolina) in 2016, and is described in detail in the literature: liu W, Anguelov D, Erhan D, et al ssd: a Single Shot MultiBox Detector [ J ].2016. the basic idea is to propose to cancel the process of generating propofol, and to use feature maps of different convolutional layers for synthesis to achieve the same effect. Experiments prove that the SSD has better performance by removing a bounding box proxy and a subsequent suppressing step of pixel or feature, and the SSD is obviously improved in speed compared with other networks.
Referring to fig. 1, a method for detecting intelligent attributes of a pipe gallery video according to an embodiment of the present invention includes the following steps:
in the embodiment, the tube corridor attribute labeling sample set is used as a deep neural network learning sample set and is used for training a deep neural network target detector. Specifically, a plurality of on-site videos of the pipe gallery are collected through a patrol robot arranged on a runway on the top of the pipe gallery, and various attribute learning sample sets of the pipe gallery are marked and formed. The deep neural network learning sample set includes cable and optical cable targets and positions, the sample data amount is 1012, and fig. 2 is a partial deep neural network learning sample set. The specimen must contain the tube lane target to be inspected, as well as the attributes and coordinate locations.
Step 2: training a deep neural network target detector;
in the embodiment, a sample set is learned through a deep neural network, the predicted result of the network is similar to the real result by continuously reducing the error between the real attribute and the predicted attribute, and a deep neural network target detector is trained by adopting a deep learning algorithm. The method comprises the following steps:
1) converting the data set into a VOC standard data set format for computer reading,
2) training with deep neural network for 200000 times to generate deep neural network target detector, and performing model test every 500 times to select and store a model with best performance.
And step 3: carrying out image enhancement processing on the newly acquired image to be identified;
1) performing brightness calculation on the test image, and regarding the test image smaller than a certain threshold value as a dark image;
2) carrying out brightness enhancement on the dim image, then detecting, and outputting as a final result;
and 4, step 4: analyzing the abnormal conditions of the processed images by adopting a trained deep neural network target detector;
in the embodiment, a multi-scale analysis mechanism is adopted to detect and analyze various attribute positions of the pipe gallery, and the steps are as follows:
1) cutting the image of the pipe gallery in the field of vision, as shown in fig. 3, respectively representing the left and right conditions of the pipe gallery; each box can be uniquely determined by the coordinates of two points: the upper left corner and the lower right corner. By using the upper left corner of the image as the origin, the horizontal direction as the x-axis, and the vertical direction as the y-axis, a coordinate system is established, and then the positions of the two frames can be expressed as:
left rectangular frame: (80, 150), (680, 1070), right matrix box: (1100, 150), (1800, 950);
2) analyzing the two regions respectively to obtain the result of the region attribute detection;
and 5: and (4) screening by adopting a threshold value, carrying out softmax normalization processing on the detection result, and displaying the target with the reliability greater than the threshold value.
Step 6: the support is used as a background landmark object for positioning the pipeline, and the positioning dividing line can be adjusted in a self-adaptive mode along with the movement of the camera. The method comprises the following steps:
1) detecting the supports in the visual field, respectively carrying out sequencing processing according to the visual field positions, and screening a row of supports closest to the camera as positioning reference points;
2) adaptively adjusting the position of the segmentation line according to the positioning reference point;
3) the linear programming determines the specific area in which the pipeline is located.
And 7: the time domain factor is adopted for limitation, the camera shaking influence is reduced, and the method comprises the following steps:
1) when the first frame detects the attribute of the corresponding pipe gallery, the counter starts counting, and each result is stored without being output;
2) when the attribute detection is continuously carried out on the subsequent frames, the corresponding counters accumulate, and the frequency of the same result accumulates;
3) and when the counter reaches a threshold value, counting frequency counts corresponding to different results, and outputting the frequency count with the highest frequency as a final attribute detection result. And saving the detection result, resetting the counter and the frequency count, and starting the next round of detection.
Referring to fig. 4 and 5, experimental results of the embodiment of the present invention show that, according to the technical solution, accurate attribute analysis can be performed on a pipe gallery which randomly appears in a complex background, and the number and types of pipes on each layer of the support can be output. A detection speed of more than 20 frames per second is achieved, with an accuracy (number of correct attribute detections/total number of images) of more than 91%. Compared with the existing method, the method has the advantages that the identification accuracy is higher, the robustness of the algorithm is enhanced, and the real-time capture and analysis of various pipeline conditions in the pipe gallery are realized. The invention provides a brand-new method based on a deep neural network model and time continuity analysis, which breaks out the traditional off-line video analysis and can detect various attributes of the pipe gallery in real time during operation. Can go in-process real-time supervision and analysis at patrol robot, can improve the piping lane and maintain operation efficiency, practice thrift the human analysis cost.
In summary, the invention discloses a method and a system for detecting intelligent attributes of a pipe gallery video, wherein the method comprises the following steps: labeling various attribute learning sample sets of the pipe gallery; constructing a deep neural network, training the neural network by using data of a sample set, and storing a trained network model on a local hard disk; the method comprises the steps of directly obtaining images from a camera or a video file on a local hard disk and carrying out image enhancement processing, so that interference of a weak light source condition is reduced; carrying out attribute detection on the newly acquired image to be identified by using the trained network model; screening the attribute detection result by using a confidence threshold value to eliminate fuzzy target interference; the support is used as a background landmark object for positioning the pipeline, and the positioning dividing line can be adjusted in a self-adaptive mode along with the movement of the camera. And time domain factor limitation is carried out, camera shaking interference is reduced, and the influence of error detection results is weakened. The method can effectively identify and analyze the pipe gallery video, can quickly detect the attribute of the pipe gallery, and improves the efficiency of daily operation, maintenance and management of the pipe gallery.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can make modifications and equivalents to the embodiments of the present invention without departing from the spirit and scope of the present invention, which is set forth in the claims of the present application.
Claims (10)
1. A pipe gallery video intelligent attribute detection method based on cloud processing is characterized by comprising the following steps:
step 1, labeling various attribute learning sample sets of a pipe gallery;
step 2, training the pre-constructed deep neural network model by using the attribute learning sample set obtained in the step 1 to obtain a trained pipe gallery attribute analysis deep neural network model;
step 3, using the pipe gallery attribute analysis deep neural network model trained in the step 2 to perform attribute detection on the image to be recognized, comprising the following steps:
1) carrying out image enhancement processing on an image to be identified, wherein the image enhancement processing is used for reducing interference of a weak light source condition;
2) performing attribute analysis detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model to obtain an attribute detection result;
3) screening the attribute detection result by using a confidence threshold value, and eliminating fuzzy target interference to obtain a screening result;
4) based on the screening result, using the bracket as a background landmark object to perform pipeline positioning to obtain a positioning result; the positioning dividing line can be adjusted in a self-adaptive manner along with the movement of the camera;
5) and time domain factor limitation is carried out on the positioning result, so that the camera shaking interference is reduced, and the influence of error detection results is weakened.
2. The cloud-processing-based pipe gallery video intelligent attribute detection method according to claim 1, wherein in step 1, the attribute learning sample set of the pipe gallery includes an xml suffix tag file labeled with labelImg.
3. The cloud-processing-based pipe gallery video intelligent attribute detection method of claim 1, wherein in step 2, the deep neural network model is YOLO, Fast-RCNN or SSD.
4. The method for detecting intelligent attributes of pipe gallery videos based on cloud processing according to claim 1, wherein in the step 2, the training of the deep neural network model includes: and converting the learning sample set into a VOC standard data set format.
5. The cloud processing-based pipe gallery video intelligent attribute detection method according to claim 1, wherein step 1) of step 3 specifically comprises:
1.1) carrying out brightness calculation on the image, and regarding the image smaller than a preset threshold as a dark image;
1.2) carrying out brightness enhancement on the dim image, then detecting until the brightness is greater than or equal to a preset threshold value, and outputting the brightness as a final result.
6. The cloud processing-based pipe gallery video intelligent attribute detection method according to claim 5, wherein step 2) of step 3 specifically comprises:
2.1) cutting the image of the pipe gallery in the visual field to obtain a left area and a right area which respectively represent the left condition and the right condition of the pipe gallery;
2.2) analyzing the two areas respectively to obtain the attribute detection results of the left area and the right area.
7. The cloud processing-based pipe gallery video intelligent attribute detection method according to claim 6, wherein step 3) of step 3 specifically comprises: and (4) performing softmax normalization processing on the detection result by adopting threshold value screening, and displaying the detection confidence coefficient which is greater than the confidence threshold value.
8. The cloud processing-based pipe gallery video intelligent attribute detection method according to claim 7, wherein step 4) of step 3 specifically comprises:
4.1) detecting the supports in the visual field, respectively carrying out sequencing treatment according to the visual field positions, and screening a row of supports closest to the camera as positioning reference points;
4.2) adaptively adjusting the position of the segmentation line according to the positioning reference point;
4.3) linear programming to determine the specific area of the pipeline.
9. The cloud processing-based pipe gallery video intelligent attribute detection method according to claim 8, wherein step 5) of step 3 specifically includes:
5.1) when the first frame detects the corresponding pipe gallery attribute, the counter starts counting, and each result is stored without output;
5.2) when the attribute detection is continuously carried out on the subsequent frames, the corresponding counters are accumulated, and the frequency of the same result is accumulated;
and 5.3) when the counter reaches a threshold value, counting frequency counts corresponding to different results, and taking the result with the highest frequency count as a final attribute detection result.
10. The utility model provides a piping lane video intelligence attribute detecting system based on high in clouds processing which characterized in that includes:
the sample acquisition module is used for marking various attribute learning sample sets of the pipe gallery;
the model acquisition module is used for training the pre-constructed deep neural network model by using the obtained attribute learning sample set to obtain a trained pipe gallery attribute analysis deep neural network model;
the attribute analysis module is used for carrying out attribute detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model, and comprises the following steps:
1) carrying out image enhancement processing on an image to be identified, wherein the image enhancement processing is used for reducing interference of a weak light source condition;
2) performing attribute analysis detection on the image to be recognized by using the trained pipe gallery attribute analysis deep neural network model to obtain an attribute detection result;
3) screening the attribute detection result by using a confidence threshold value, and eliminating fuzzy target interference to obtain a screening result;
4) based on the screening result, using the bracket as a background landmark object to perform pipeline positioning to obtain a positioning result; the positioning dividing line can be adjusted in a self-adaptive manner along with the movement of the camera;
5) and time domain factor limitation is carried out on the positioning result, so that the camera shaking interference is reduced, and the influence of error detection results is weakened.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110130569.XA CN112784914B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent attribute detection method and system based on cloud processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110130569.XA CN112784914B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent attribute detection method and system based on cloud processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112784914A true CN112784914A (en) | 2021-05-11 |
| CN112784914B CN112784914B (en) | 2023-04-07 |
Family
ID=75760049
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110130569.XA Active CN112784914B (en) | 2021-01-29 | 2021-01-29 | Pipe gallery video intelligent attribute detection method and system based on cloud processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112784914B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113899809A (en) * | 2021-08-20 | 2022-01-07 | 中海石油技术检测有限公司 | In-pipeline detector positioning method based on CNN classification and RNN prediction |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5615734A (en) * | 1994-11-16 | 1997-04-01 | Westinghouse Electric Corporation | Sludge lance inspection and verification system |
| CN206877112U (en) * | 2017-07-03 | 2018-01-12 | 詹皇源 | A kind of underground pipe gallery monitoring and operation managing system |
| CN108356818A (en) * | 2018-01-15 | 2018-08-03 | 史聪灵 | A kind of pipe gallery cruising inspection system and interactive system based on robot |
| CN210112054U (en) * | 2019-03-12 | 2020-02-21 | 上海电器科学研究所（集团）有限公司 | Utility tunnel security protection communication integration system based on wisdom line |
| CN111401131A (en) * | 2020-02-13 | 2020-07-10 | 深圳供电局有限公司 | Image processing method and device for tunnel pipe gallery, computer equipment and storage medium |
| CN111462326A (en) * | 2020-03-31 | 2020-07-28 | 武汉大学 | Low-cost 360-degree panoramic video camera urban pipeline three-dimensional reconstruction method and system |
| CN111612018A (en) * | 2020-04-24 | 2020-09-01 | 国网河北省电力有限公司雄安新区供电公司 | Underground pipe gallery monitoring method and system based on multi-source heterogeneous data fusion |
| CN211478895U (en) * | 2020-01-20 | 2020-09-11 | 重庆精聚达建筑工程咨询有限公司 | City utility tunnel wisdom supervisory systems |
| CN111986188A (en) * | 2020-08-27 | 2020-11-24 | 深圳市智源空间创新科技有限公司 | Capsule robot drainage pipe network defect identification method based on Resnet and LSTM |
| CN112258496A (en) * | 2020-11-02 | 2021-01-22 | 郑州大学 | Underground drainage pipeline disease segmentation method based on full convolution neural network |
-
2021
- 2021-01-29 CN CN202110130569.XA patent/CN112784914B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5615734A (en) * | 1994-11-16 | 1997-04-01 | Westinghouse Electric Corporation | Sludge lance inspection and verification system |
| CN206877112U (en) * | 2017-07-03 | 2018-01-12 | 詹皇源 | A kind of underground pipe gallery monitoring and operation managing system |
| CN108356818A (en) * | 2018-01-15 | 2018-08-03 | 史聪灵 | A kind of pipe gallery cruising inspection system and interactive system based on robot |
| CN210112054U (en) * | 2019-03-12 | 2020-02-21 | 上海电器科学研究所（集团）有限公司 | Utility tunnel security protection communication integration system based on wisdom line |
| CN211478895U (en) * | 2020-01-20 | 2020-09-11 | 重庆精聚达建筑工程咨询有限公司 | City utility tunnel wisdom supervisory systems |
| CN111401131A (en) * | 2020-02-13 | 2020-07-10 | 深圳供电局有限公司 | Image processing method and device for tunnel pipe gallery, computer equipment and storage medium |
| CN111462326A (en) * | 2020-03-31 | 2020-07-28 | 武汉大学 | Low-cost 360-degree panoramic video camera urban pipeline three-dimensional reconstruction method and system |
| CN111612018A (en) * | 2020-04-24 | 2020-09-01 | 国网河北省电力有限公司雄安新区供电公司 | Underground pipe gallery monitoring method and system based on multi-source heterogeneous data fusion |
| CN111986188A (en) * | 2020-08-27 | 2020-11-24 | 深圳市智源空间创新科技有限公司 | Capsule robot drainage pipe network defect identification method based on Resnet and LSTM |
| CN112258496A (en) * | 2020-11-02 | 2021-01-22 | 郑州大学 | Underground drainage pipeline disease segmentation method based on full convolution neural network |
Non-Patent Citations (2)
| Title |
|---|
| 王薇等: "城市地下综合管廊管线的数据采集及故障诊断", 《北方建筑》 * |
| 耿永常: "《地下空间规划与建筑设计》", 31 July 2019, 哈尔滨工程大学出版社 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113899809A (en) * | 2021-08-20 | 2022-01-07 | 中海石油技术检测有限公司 | In-pipeline detector positioning method based on CNN classification and RNN prediction |
| CN113899809B (en) * | 2021-08-20 | 2024-02-27 | 中海石油技术检测有限公司 | In-pipeline detector positioning method based on CNN classification and RNN prediction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112784914B (en) | 2023-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108037133B (en) | Intelligent electric power equipment defect identification method and system based on unmanned aerial vehicle inspection image | |
| CN111881730A (en) | Wearing detection method for on-site safety helmet of thermal power plant | |
| CN109858367B (en) | Visual automatic detection method and system for worker through supporting unsafe behaviors | |
| CN111080634A (en) | Transformer appearance defect identification method based on inspection robot and Cascade RCNN algorithm | |
| CN109902633A (en) | Accident detection method and device based on the camera supervised video of fixed bit | |
| CN112200011B (en) | Aeration tank state detection method, system, electronic equipment and storage medium | |
| CN111881970A (en) | Intelligent outer broken image identification method based on deep learning | |
| CN111178424A (en) | Petrochemical production site safety compliance real-time detection system and method | |
| CN111044149A (en) | Method and device for detecting temperature abnormal point of voltage transformer and readable storage medium | |
| CN112784914B (en) | Pipe gallery video intelligent attribute detection method and system based on cloud processing | |
| CN108471497A (en) | A kind of ship target real-time detection method based on monopod video camera | |
| CN112766795B (en) | Cloud processing-based pipe gallery intelligent information management method and system | |
| CN106570440A (en) | People counting method and people counting device based on image analysis | |
| CN111767826A (en) | Timing fixed-point scene abnormity detection method | |
| CN115171045A (en) | YOLO-based power grid operation field violation identification method and terminal | |
| CN113469938A (en) | Pipe gallery video analysis method and system based on embedded front-end processing server | |
| CN110738272A (en) | method for labeling visualized mechanical continuous alarm samples of power transmission line channel | |
| CN112861681B (en) | Pipe gallery video intelligent analysis method and system based on cloud processing | |
| CN112633157A (en) | AGV working area safety real-time detection method and system | |
| CN112070740A (en) | BIM-based port storage yard accumulated water imaging method, system and control equipment | |
| CN115953405A (en) | Bridge crack identification method and device for augmented reality equipment | |
| CN115294774B (en) | Non-motor vehicle road stopping detection method and device based on deep learning | |
| CN112818836B (en) | Method and system for detecting personnel target of transformer substation scene | |
| CN115100546A (en) | Mobile-based small target defect identification method and system for power equipment | |
| CN114387564A (en) | Head-knocking engine-off pumping-stopping detection method based on YOLOv5 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A Cloud Based Intelligent Attribute Detection Method and System for Pipe Gallery Video Effective date of registration: 20231010 Granted publication date: 20230407 Pledgee: Zhejiang Tailong Commercial Bank Co.,Ltd. Huzhou Changxing sub branch Pledgor: Changxing yunshang Technology Co.,Ltd. Registration number: Y2023980060343 |