CN109712140B - Method and device for training fully-connected classification network for leakage detection - Google Patents
Method and device for training fully-connected classification network for leakage detection Download PDFInfo
- Publication number
- CN109712140B CN109712140B CN201910002906.XA CN201910002906A CN109712140B CN 109712140 B CN109712140 B CN 109712140B CN 201910002906 A CN201910002906 A CN 201910002906A CN 109712140 B CN109712140 B CN 109712140B
- Authority
- CN
- China
- Prior art keywords
- network
- training
- convolutional neural
- area
- bounding box
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Image Analysis (AREA)
Abstract
The embodiment of the invention provides a method and a device for training a fully-connected classification network for leakage detection, relates to the technical field of image recognition, and solves the problems of low detection speed and low detection precision in the prior art under the condition that a leakage target to be detected dynamically changes in an image. According to one embodiment, the method comprises: determining network parameters through transfer learning by using the sample images; initializing a convolutional neural network and a regional suggestion network according to network parameters; training a convolutional neural network according to the regional suggestion network; optimizing an area suggestion network according to the trained convolutional neural network; training a fully-connected classification network according to the sample characteristic graph of the sample image and the candidate bounding box of the sample image; wherein the candidate bounding box of the sample image is extracted through the optimized region suggestion network.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to a method and a device for training a fully-connected classification network for leakage detection.
Background
The leakage, the overflow, the dripping and the liquid leakage (gas leakage, water leakage, liquid dripping and liquid leakage) are the enemies of modern safe civilized production, and in the long-term continuous production process, various sealing failures can be caused due to the influence of a plurality of factors such as vibration, stress, deformation, impact, scouring, corrosion, temperature, pressure, environment, season, human factors, material defects and the like, so that medium leakage is generated. If the leakage cannot be timely treated, the leakage can be rapidly enlarged under the scouring of the medium, so that the loss of materials and the damage of the production environment are caused, and if the leakage is toxic, harmful, flammable and explosive medium, serious accidents such as personnel poisoning, fire hazard and explosion can be caused. Therefore, how to rapidly manage leakage without influencing production is always a topic of interest for enterprise equipment managers.
In the prior art, when identifying a leakage target, a Region Convolutional Neural Network (hereinafter referred to as R-CNN) and a regression model are generally selected for target detection and positioning. Wherein R-CNN comprises: a Region Fast Convolutional Neural Network (hereinafter referred to as Fast R-CNN), a Region Faster Convolutional Neural Network (hereinafter referred to as Fast R-CNN), a Region Mask Convolutional Neural Network (hereinafter referred to as Mask R-CNN), and so on. The regression model includes: the YOLO (English full name: You Only Look one) algorithm, the SSD (Single Shot Multi Box Detector) algorithm, and the like.
However, the R-FCN, Fast R-CNN and Mask R-CNN all belong to target detection based on candidate areas, and are characterized by high detection precision, but the speed cannot meet the application real-time requirement; the above-mentioned YOLO algorithm and SSD algorithm are regression-based target detection, and are characterized by a high speed, but the detection accuracy is not good enough.
In addition, in the actual scene of the leakage, when the high-pressure gas and the liquid leak, the direction and the size of the jet are random, and along with the expansion of the leak, the direction of the high-pressure gas and the liquid jet may be changed continuously (for example, the jet beam is increased), the jet size may be expanded step by step, and the corresponding target to be detected for the leakage is changed dynamically in the image. The above-mentioned target detection method often cannot adapt to such dynamic changes, thereby causing the detection of the boundary box to be inaccurate, and the boundary box often has an area that does not belong to the leakage target to be detected, thereby causing the reduction of the detection precision of the leakage target to be detected.
Disclosure of Invention
The embodiment of the invention provides a method and a device for training a fully-connected classification network for leakage detection, which solve the problems of low detection speed and low detection precision in the prior art under the condition that a leakage target to be detected is dynamically changed in an image.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
in a first aspect of embodiments of the present invention, a method for training a fully-connected classification network for detecting a running fault and a leakage is provided, where the method includes:
determining network parameters through transfer learning by using the sample images;
initializing a convolutional neural network and a regional suggestion network according to network parameters;
training a convolutional neural network according to the regional suggestion network;
optimizing an area suggestion network according to the trained convolutional neural network; and
training a fully-connected classification network according to the sample characteristic graph of the sample image and the candidate bounding box of the sample image;
wherein the candidate bounding box of the sample image is extracted through the optimized region suggestion network.
In a second aspect of the embodiments of the present invention, there is provided a method for running leak detection, the method including:
identifying a bounding box of the image to be detected of the target by using the trained fully-connected classification network obtained by the method of the first aspect to determine a running rail leakage detection result. (ii) a
In a third aspect of embodiments of the present invention, there is provided a computer-readable storage medium storing a computer program for executing the method of the first aspect or the second aspect.
In a fourth aspect of the embodiments of the present invention, there is provided an apparatus for training a fully connected classification network for detecting a leakage, a running and a dropping, the apparatus comprising:
a determination module for determining network parameters by transfer learning using the sample images;
the initialization module is used for initializing the convolutional neural network and the area suggestion network according to the network parameters;
the first training module is used for training the convolutional neural network according to the regional suggestion network;
the optimization module is used for optimizing the area suggestion network according to the trained convolutional neural network; and
the second training module is used for training a fully-connected classification network according to the sample characteristic diagram of the sample image and the candidate bounding box of the sample image;
wherein the candidate bounding box of the sample image is extracted through the optimized region suggestion network.
In a fifth aspect of embodiments of the present invention, there is provided an apparatus for leakage detection, the apparatus comprising:
and the identification module is used for identifying the boundary frame of the to-be-detected image of the target through the trained fully-connected classification network obtained by using the device of the fourth aspect so as to determine the detection result of the leakage.
Compared with the prior art, the fully-connected classification network for training the running, the falling, the dropping and the leakage detection can automatically generate a training set according to a sample image and automatically train the training set to obtain network parameters by introducing the idea of transfer learning, then the convolutional neural network and the regional suggestion network are respectively initialized according to the network parameters obtained by training, and the network parameters are determined by adopting the transfer learning method, so that the time spent on training a network model is saved, and the detection speed can be improved when the target detection is finally carried out; secondly, the convolutional neural network is trained according to the preselected area of the area suggestion network, the area suggestion network is optimized according to the trained convolutional neural network, and the candidate bounding box of the sample image is extracted through the optimized area suggestion network, so that the extracted candidate bounding box of the sample image is reasonable, can adapt to the dynamic change in the sample image, and can improve the detection precision.
Drawings
The disclosure may be better understood by reference to the following description taken in conjunction with the accompanying drawings. It should be understood that the drawings are not necessarily drawn to scale. In the drawings:
FIG. 1 is a flow chart of a method for training a fully connected classification network for false drop detection according to an embodiment of the present invention;
FIG. 2 is a flow chart of a method for leak detection according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of an apparatus for training a fully connected classification network for detecting a leakage and a running leakage according to an embodiment of the present invention;
FIG. 4 is a schematic structural diagram of another apparatus for training a fully connected classification network for detecting a leakage and a running leakage according to an embodiment of the present invention; and
fig. 5 is a schematic structural diagram of another apparatus for detecting leakage and overflow according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It will of course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions may be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another.
For the convenience of clearly describing the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, the words "first", "second", and the like are used to distinguish the same items or similar items with basically the same functions or actions, and those skilled in the art can understand that the words "first", "second", and the like do not limit the quantity and execution order.
The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.
The term "comprises/comprising" when used herein refers to the presence of a feature, element or component, but does not preclude the presence or addition of one or more other features, elements or components.
The target detection scene realized by the embodiment of the invention mainly comprises liquid abnormity, gas abnormity, open fire abnormity, dust abnormity and the like. Wherein: fluid anomalies include, but are not limited to: an acid-base solution (a jet of water), water (a jet of water or shower head water spray), edible oil (yellowish oil), engine oil (blackish oil) or coal-water slurry (solid-liquid mixture); vapor anomalies include, but are not limited to: water vapor (humidifier-simulated steam) or smoke (smoke from paper combustion); abnormal open fire: open flame (simulation of a lighter) or electric spark (simulation of a short circuit); dust anomalies include, but are not limited to: bleed dust (white-off dust) or funnel coal ash (black-off/dark gray dust).
In the prior art, when identifying a running and leaking target, an R-CNN extracts a candidate frame from an image by using a selective search algorithm, inputs a normalized candidate region into a Convolutional Neural Network (CNN), extracts features, classifies the regions by using a Support Vector Machine (SVM), and finely adjusts the position and size of the candidate frame by using linear regression to obtain closer candidate frame coordinates. However, R-CNN is time consuming because these candidate boxes require CNN operations, which are computationally expensive, and many of them are duplicated.
Fast R-CNN selects corresponding regions in the characteristic map of R-CNN to obtain CNN abstract characteristics of each region; the primary abstract features in each region are then typically merged using a max pooling operation. Specifically, Fast R-CNN jointly trains a convolutional neural network, a classifier, and a bounding box regression model in a model, and in order to detect the position of an object in an image, the first step is to generate a series of random multi-scale bounding boxes or regions of interest to be tested. However, in Fast R-CNN, these regions are created by a selective search method, which is a rather slow process.
The fast R-CNN adds a full convolution Network (called as full convolution Network in English hereinafter: FCN) on the top of CNN characteristic, creates a Region suggestion Network (called as Region pro-social Network in English hereinafter: RPN), the Region suggestion Network slides a window on CNN characteristic map in turn, at each window position, the Network outputs a score and a bounding box on each anchor point, only transfers each bounding box which may be a target object to the fast R-CNN, so as to realize object classification and shrink bounding boxes.
The R-FCN, Fast R-CNN and Mask R-CNN all belong to target detection based on candidate areas, and are characterized by high detection precision, but the speed cannot meet the application real-time requirement.
In addition, in the actual scene of the leakage, when the high-pressure gas and the liquid leak, the direction and the size of the jet are random, and along with the expansion of the leak, the direction of the high-pressure gas and the liquid jet may be changed continuously (for example, the jet beam is increased), the jet size may be expanded step by step, and the corresponding target to be detected for the leakage is changed dynamically in the image. The target detection method related to the above cannot adapt to the dynamic change, so that the detection of the boundary frame is inaccurate, and the boundary frame often has an area which does not belong to the target to be detected, so that the detection precision of the target to be detected is reduced, the judgment of the severity of the leakage fault in the production process by a user is influenced, and the alarm is inaccurate.
In order to solve the above technical problem, an embodiment of the present invention provides a method for training a fully-connected classification network for detecting a leakage, overflow and leakage, as shown in fig. 1, the method includes:
101. network parameters are determined by migration learning using the sample images.
Among them, the above-mentioned migration learning is one of machine learning techniques, and a model trained on one task is reused on another related task. According to the scheme, the network parameters are trained on the basic data set, so that huge resources which need to be consumed by a deep learning model can be avoided, the training time is saved, and the efficiency of the training model is improved.
Illustratively, the step 101 may be specifically implemented by: and (3) pre-training the convolutional neural network and the regional suggestion network by using a data set, and determining network parameters by adopting a transfer learning method. Preferably, the data set is: ImageNet, but is not limited thereto. The convolutional neural network described above may be any CNN model, such as: ResNet, Inception, or VGG.
Optionally, before step 101, the method further includes:
101a, shooting a sample image containing an object to be detected through a camera.
Illustratively, the sample image described above is an image containing various types of leakage. Thus, when training is started, various types of leakage images are included, and the training results can identify various types of leakage targets, so that when two types of leakage exist, the two types of leakage can be identified through the content of the invention.
102. And initializing the convolutional neural network and the area suggestion network according to the network parameters.
Illustratively, the network parameters refer to weights trained on a data set, the convolutional neural network needs to be trained by data, information is obtained from the data, the data is further converted into corresponding weights, and then the weights are migrated into the convolutional neural network and the regional suggestion network, that is, feature learning is automatically performed by a method combining deep learning and migration learning. Therefore, the convolutional neural network and the area suggestion network are trained by means of a transfer learning mechanism, and a network model with generalization capability is obtained for identifying the jetting position and direction of high-pressure gas or liquid.
103. And training the convolutional neural network according to the regional suggestion network.
Preferably, the step 103 specifically includes the following steps:
103a1, generating a first training area using the area suggestion network.
103a2, training the convolutional neural network based on the first training region.
104. And optimizing the area suggestion network according to the trained convolutional neural network.
Illustratively, the step 104 specifically includes the following steps:
104a1, training the area proposal network according to the trained convolutional neural network.
104a2, generating a second training area using the trained area suggestion network.
104a3, optimizing the area proposal network according to the trained area proposal network and the second training area.
Preferably, the step 104a1 can be implemented by: initializing the area suggestion network by using the trained convolutional neural network, keeping the convolutional layer unchanged, and only adjusting the part of the trained convolutional neural network, which is different from the area suggestion network. The step 104a3 can be specifically realized by the following steps: and keeping the convolutional layer parameters of the trained area suggestion network unchanged, and adjusting the trained area suggestion network again by using the second training area to obtain the optimized area suggestion network. For example, the step 104a1 may refer to the following when implementing: a third training region is generated using the trained convolutional neural network, and then the region suggestion network is trained based on the third training region. In the above-mentioned steps 103 and 104, the convolutional neural network is trained by the regional suggestion network, and then the regional suggestion network is optimized by the trained convolutional neural network, because the candidate bounding box of the sample image is extracted by the optimized regional suggestion network, the extracted candidate bounding box of the sample image is more reasonable, and can adapt to the dynamic change in the sample image, thereby improving the detection accuracy.
105. And training the fully-connected classification network according to the sample characteristic graph of the sample image and the candidate bounding box of the sample image.
Wherein the candidate bounding box of the sample image is extracted through the optimized region suggestion network. Optionally, before the step 105, the method further includes:
and 105a, extracting candidate bounding boxes of the sample image according to the optimized area suggestion network.
Illustratively, the step 105a specifically includes:
105a1, learning the characteristics of the sample image by using the trained convolutional neural network to obtain a sample characteristic diagram.
105a2, convolving the convolution kernel of the optimized area suggestion network with the sample feature map, and determining the candidate bounding box of the sample image.
Assuming that the size of a given input image is M × N, the size of a sample feature map obtained through the convolution operation is M × N, and there are L feature maps in total, where M >0 and N > 0. Wherein the convolutional neural network comprises a plurality of convolutional layers.
For example, the optimized region proposed network RPN in step 104 is convolved on the sample profile of m × n using a convolution kernel of 3 × 3, i.e., a3 × 3 sliding window. The central point position of the convolution kernel of 3 x 3 is mapped into an original image, and 3 kinds of aspect ratio ratios (1:1, 1:2, 2:1), 3 kinds of scale scales (64, 128, 256) and 9 anchor points (English abbreviation: anchor) in total, namely a candidate boundary frame, are extracted from the central position.
Optionally, after the step 105a, the method further includes:
105b, inputting the candidate bounding box of the sample image processed by the region pooling layer into a full-connection classification network, and outputting the target classification and the bounding box position.
105c, adjusting the position of the boundary frame according to the coordinate correction amount.
The correction amount is a parameter for correcting the position of the bounding box.
And 105d, screening the candidate bounding boxes of the adjusted sample image according to a deviation sample filtering method, and determining a new candidate bounding box of the sample image.
The candidate bounding box that passes through the region of interest pooling layer ROI is input into the fully-connected layer, then the classified layer and the bounding box regression layer in step 105b described above. The classification layer performs final target classification, that is, determines whether the sample candidate frame is a target. The bounding box regression layer is used for target boundary regression correction, namely, the abscissa x and the ordinate y of the top left corner vertex of a candidate bounding box corresponding to the mapping of the central position of a prediction convolution kernel in an original image, and the width w and the height h of the candidate bounding box. Where there is a regression of 4 x 9 positions for each central position. In actual calculation, the bounding box regression used the smoothed L1-loss (norm).
In step 105d, a candidate bounding box with a sample deviation greater than a threshold is selected as a new candidate bounding box by a deviation sample filtering method. Non-maxima suppression is also employed to remove candidate bounding boxes with too large an overlap region (i.e., degree of overlap) before bias sample filtering methods are used.
The convolutional neural network is combined with a bias sample filtering method, so that the situation that the training result is deteriorated due to the fact that the number of positive and negative samples in the convolutional neural network is not uniform is effectively reduced.
Further optionally, the method further includes:
and 105e, updating the fully-connected classification network according to the sample feature map of the sample image and the new candidate bounding box of the sample image.
105f, circularly executing the following steps until the coordinate correction amount is a preset value: and inputting the candidate bounding box of the sample image processed by the regional pooling layer into a full-connection classification network, outputting the target classification, and outputting the position of the bounding box.
For example, the preset value is an empirical value, which can be set according to the actual picture condition, and when the correction amount is equal to the preset value, the corresponding network model of the fully-connected classification network is the most accurate.
Illustratively, the steps 105e and 105f are specifically realized by: and taking the sample characteristic graph of the sample image and a new candidate bounding box as a new training sample, performing ROI (region of interest) processing on the sample characteristic graph and the new candidate bounding box, inputting the sample characteristic graph and the new candidate bounding box into a classification network of a full-connection layer, performing target classification and target position adjustment again, and updating parameters of the full-connection classification network through repeated iterative training to gradually reduce the target position adjustment amount so as to obtain the trained full-connection classification network when the target position adjustment amount reaches a set value. Wherein, a training set and a testing set are distributed to the sample image according to the proportion of 5:1, and cross validation is carried out.
When the area suggestion network adopts a Feature Pyramid Network (FPN), candidate areas with various length-width ratios are generated in each layer of the feature pyramid network, then the candidate areas are selected by utilizing non-maximum value inhibition, and the multi-scale features are utilized for detection, so that the detection of tiny false-leakage targets is particularly effective by comprehensively utilizing the bottom layer features and the high layer features.
An embodiment of the present invention provides a method for detecting a running leakage, as shown in fig. 2, the method includes:
201. the trained full-connection classification network obtained by the method is used for identifying the boundary frame of the image to be detected of the target so as to determine the detection result of the leakage.
Preferably, after the step 201, the method further comprises:
202. and digging a boundary frame of the image to be detected according to a preset rule.
203. The clipped bounding box is displayed.
Optionally, the preset rule when the clipping is performed is to clip an adjacent region of the bounding box containing the image to be detected, where the shape of the adjacent region is not limited, and may be a circular region or a rectangular region.
Preferably, the preset rule is as follows:
1) selecting the size of the digging frame as L x L, L is (MIN (M, N))/P) if the size of the boundary frame is M x N; p ═ α × M/N, α is an adjustment coefficient, and takes a value of 10.
2) Z-scan is performed starting from one of the four corners of the bounding box, and edge detection is performed on the image in the cutout box.
3) If no edge is detected, removing the image in the digging frame, continuing to scan the next digging frame along the Z scanning direction, and repeating the step 2);
4) if an edge is detected, the scan is terminated and the corner of the next bounding box is selected and steps 2) -3) are repeated until the Z-scan from the four corners of the bounding box is completely completed.
Optionally, when the boundary frame of the image to be detected is excavated, if the selected excavation scale is too large, the true edge may be excavated, and at this time, the excavated boundary frame needs to be filled up by a plurality of minimum boundary frames, so that the complete boundary frame of the image to be detected is obtained.
When the display module performs display, the extracted result can be mapped back to the original image, and the boundary frame is drawn, so that the target detection of the leakage is completed. Through the processing of the steps, the speed of detecting the leakage target is improved, and the precision of detecting the leakage target is improved.
In order to further improve the detection precision of the leakage target, the mask of the leakage target can be directly obtained on the basis of obtaining the relative position of the leakage target and the boundary frame, and pixel-level division of the leakage target is realized. In addition, smoothing filtering may be performed according to the mask to remove blocking artifacts.
Compared with the prior art, the fully-connected classification network for training the running, the falling, the dropping and the leakage detection can automatically generate a training set according to a sample image and automatically train the training set to obtain network parameters by introducing the idea of transfer learning, then the convolutional neural network and the regional suggestion network are respectively initialized according to the network parameters obtained by training, and the network parameters are determined by adopting the transfer learning method, so that the time spent on training a network model is saved, and the detection speed can be improved when the target detection is finally carried out; secondly, the convolutional neural network is trained according to the preselected area of the area suggestion network, the area suggestion network is optimized according to the trained convolutional neural network, and the candidate bounding box of the sample image is extracted through the optimized area suggestion network, so that the extracted candidate bounding box of the sample image is reasonable, can adapt to the dynamic change in the sample image, and can improve the detection precision. An apparatus for training a fully-connected classification network for running leakage detection according to an embodiment of the present invention will be described based on the related description in the embodiment of the method for training a fully-connected classification network for running leakage detection according to fig. 1. Technical terms, concepts and the like related to the above embodiments in the following embodiments may refer to the above embodiments, and are not described in detail herein.
An embodiment of the present invention provides an apparatus for training a fully-connected classification network for detecting a running fault and a leakage, as shown in fig. 3, the apparatus includes: a first determination module 301, an initialization module 302, a first training module 303, an optimization module 304, and a second training module 305, wherein:
a first determining module 301, configured to determine network parameters through transfer learning using the sample images.
An initialization module 302, configured to initialize the convolutional neural network and the area recommendation network according to the network parameters.
A first training module 303, configured to train the convolutional neural network according to the region suggestion network.
And an optimization module 304, configured to optimize the area suggestion network according to the trained convolutional neural network.
A second training module 305, configured to train a fully-connected classification network according to the sample feature map of the sample image and the candidate bounding box of the sample image; wherein the candidate bounding box of the sample image is extracted through the optimized region suggestion network.
Among them, the above-mentioned migration learning is one of machine learning techniques, and a model trained on one task is reused on another related task. According to the scheme, the network parameters are trained on the basic data set, so that huge resources which need to be consumed by a deep learning model can be avoided, the training time is saved, and the efficiency of the training model is improved.
For example, the determining module may be specifically implemented by: and (3) pre-training the convolutional neural network and the regional suggestion network by using a data set, and determining network parameters by adopting a transfer learning method. Preferably, the data set is: ImageNet, but is not limited thereto. The convolutional neural network described above may be any CNN model, such as: ResNet, Inception, or VGG.
Optionally, the apparatus 4 further comprises: an acquisition module 306, wherein:
and the acquisition module 306 is used for shooting a sample image containing the object to be detected through the camera.
Illustratively, the sample image described above is an image containing various types of leakage. Thus, when training is started, various types of leakage images are included, and the training results can identify various types of leakage targets, so that when two types of leakage exist, the two types of leakage can be identified through the content of the invention.
The above network parameters refer to weights trained on a data set, the convolutional neural network needs to be trained by data, information is obtained from the data, the data is further converted into corresponding weights, and then the weights are migrated to the convolutional neural network and the regional suggestion network, that is, feature learning is automatically performed by a method combining deep learning and migration learning. Therefore, the convolutional neural network and the area suggestion network are trained by means of a transfer learning mechanism, and a network model with generalization capability is obtained for identifying the jetting position and direction of high-pressure gas or liquid.
Preferably, the first training module 303 is specifically configured to:
a first training area is generated using an area suggestion network.
The convolutional neural network is trained based on the first training region.
Preferably, the optimization module 304 is specifically configured to:
and training the area suggestion network according to the trained convolutional neural network.
Generating a second training area using the trained area suggestion network.
And optimizing the area suggestion network according to the trained area suggestion network and the second training area.
Preferably, when the optimization module proposes a network according to the trained convolutional neural network training region, the optimization module may specifically be implemented by: initializing the area suggestion network by using the trained convolutional neural network, keeping the convolutional layer unchanged, and only adjusting the part of the trained convolutional neural network, which is different from the area suggestion network. When the optimization module optimizes the area suggestion network according to the trained area suggestion network and the second training area suggestion network, the optimization module can be specifically realized by the following contents: and keeping the convolutional layer parameters of the trained area suggestion network unchanged, and adjusting the trained area suggestion network again by using the second training area to obtain the optimized area suggestion network.
Illustratively, the optimization module mentioned above can be implemented by the following steps when the proposed network is trained according to the trained convolutional neural network training area: a third training region is generated using the trained convolutional neural network, and then the region suggestion network is trained based on the third training region.
The contents of the first training module and the optimization module train the convolutional neural network through the regional suggestion network, and then optimize the regional suggestion network through the trained convolutional neural network.
Optionally, as shown in fig. 4, the apparatus 3 further includes: an extraction module 307, wherein:
and an extracting module 307, configured to extract candidate bounding boxes of the sample image according to the optimized area suggestion network.
Illustratively, the extracting module 307 is specifically configured to:
and learning the characteristics of the sample image by adopting the trained convolutional neural network to obtain a sample characteristic diagram.
And (4) convolving the convolution kernel of the optimized area suggestion network with the sample characteristic graph to determine a candidate bounding box of the sample image.
Assuming that the size of a given input image is M × N, the size of a sample feature map obtained through the convolution operation is M × N, and there are L feature maps in total, where M >0 and N > 0. Wherein the convolutional neural network comprises a plurality of convolutional layers.
For example, for the optimized region proposed network RPN, a convolution is performed on the sample profile of m × n using a3 × 3 convolution kernel, i.e., a3 × 3 sliding window. The central point position of the convolution kernel of 3 x 3 is mapped into an original image, and 3 kinds of aspect ratio ratios (1:1, 1:2, 2:1), 3 kinds of scale scales (64, 128, 256) and 9 anchor points (English abbreviation: anchor) in total, namely a candidate boundary frame, are extracted from the central position.
Optionally, as shown in fig. 4, the apparatus 3 further includes: an output module 308, an adjustment module 309, and a second determination module 310, wherein:
an output module 308, configured to input the candidate bounding box of the sample image subjected to the regionalization processing into a fully connected classification network, and output the target classification and the bounding box position.
And an adjusting module 309, configured to adjust the position of the bounding box according to the coordinate correction.
The second determining module 310 screens the candidate bounding boxes of the adjusted sample image according to the biased sample filtering method, and determines new candidate bounding boxes of the sample image.
The correction amount is a parameter for correcting the position of the bounding box.
In the above, the candidate bounding box passing through the ROI is input into the full-link layer, and then input into the classification layer and the bounding box regression layer. The classification layer performs final target classification, that is, determines whether the sample candidate frame is a target. The bounding box regression layer is used for target boundary regression correction, namely, the abscissa x and the ordinate y of the top left corner vertex of a candidate bounding box corresponding to the mapping of the central position of a prediction convolution kernel in an original image, and the width w and the height h of the candidate bounding box. Where there is a regression of 4 x 9 positions for each central position. In actual calculation, the bounding box regression used the smoothed L1-loss (norm).
The second determining module selects the candidate bounding box with the sample deviation larger than the threshold value as a new candidate bounding box by adopting a deviation sample filtering method. Non-maxima suppression is also employed to remove candidate bounding boxes with too large an overlap region (i.e., degree of overlap) before bias sample filtering methods are used. The convolutional neural network is combined with a bias sample filtering method, so that the situation that the training result is deteriorated due to the fact that the number of positive and negative samples in the convolutional neural network is not uniform is effectively reduced.
Further optionally, as shown in fig. 4, the apparatus 3 further includes: an update module 311 and a loop execution module 312, wherein:
the updating module 311 is further configured to update the fully-connected classification network according to the sample feature map of the sample image and the new candidate bounding box of the sample image.
A loop executing module 312, configured to execute the following steps in a loop until the coordinate correction amount is a preset value: and inputting the candidate bounding box of the sample image processed by the regional pooling layer into a full-connection classification network, outputting the target classification, and outputting the position of the bounding box.
For example, the preset value is an empirical value, which can be set according to the actual picture condition, and when the correction amount is equal to the preset value, the corresponding network model of the fully-connected classification network is the most accurate.
For example, the update module and the loop execution module may be further implemented by: and taking the sample characteristic graph of the sample image and a new candidate bounding box as a new training sample, performing ROI (region of interest) processing on the sample characteristic graph and the new candidate bounding box, inputting the sample characteristic graph and the new candidate bounding box into a classification network of a full-connection layer, performing target classification and target position adjustment again, and updating parameters of the full-connection classification network through repeated iterative training to gradually reduce the target position adjustment amount so as to obtain the trained full-connection classification network when the target position adjustment amount reaches a set value. Wherein, a training set and a testing set are distributed to the sample image according to the proportion of 5:1, and cross validation is carried out.
When the area suggestion network adopts the feature pyramid network, candidate areas with various length-width ratios are generated in each layer of the feature pyramid network, then the candidate areas are restrained and selected by using non-maximum values, the multi-scale features are used for detection, and the detection of tiny false-leakage targets is particularly effective by comprehensively using the bottom layer features and the high layer features.
An embodiment of the present invention provides a device for detecting leakage, and as shown in fig. 5, the device 5 includes:
the identification module 401 is configured to identify a bounding box of an image to be detected of the target through the trained fully-connected classification network obtained by using the apparatus described above, so as to determine a detection result of the running, the impersonation, and the leakage.
Preferably, the apparatus 4 further comprises: a digging module 402, and a display module 403, wherein:
and a clipping module 402, configured to clip a bounding box of the image to be detected according to a preset rule.
And a display module 403, configured to display the clipped bounding box.
Optionally, the preset rule when the clipping is performed is to clip an adjacent region of the bounding box containing the image to be detected, where the shape of the adjacent region is not limited, and may be a circular region or a rectangular region.
Preferably, the preset rule is as follows:
1) selecting the size of the digging frame as L x L, L is (MIN (M, N))/P) if the size of the boundary frame is M x N; p ═ α × M/N, α is an adjustment coefficient, and takes a value of 10.
2) Z-scan is performed starting from one of the four corners of the bounding box, and edge detection is performed on the image in the cutout box.
3) If no edge is detected, removing the image in the digging frame, continuing to scan the next digging frame along the Z scanning direction, and repeating the step 2);
4) if an edge is detected, the scan is terminated and the corner of the next bounding box is selected and steps 2) -3) are repeated until the Z-scan from the four corners of the bounding box is completely completed.
Optionally, when the boundary frame of the image to be detected is excavated, if the selected excavation scale is too large, the true edge may be excavated, and at this time, the excavated boundary frame needs to be filled up by a plurality of minimum boundary frames, so that the complete boundary frame of the image to be detected is obtained.
When the display module performs display, the extracted result can be mapped back to the original image, and the boundary frame is drawn, so that the target detection of the leakage is completed. Through the processing of the steps, the speed of detecting the leakage target is improved, and the precision of detecting the leakage target is improved.
Compared with the prior art, the fully-connected classification network for training the running, the falling, the dropping and the leakage detection can automatically generate a training set according to a sample image and automatically train the training set to obtain network parameters by introducing the idea of transfer learning, then the convolutional neural network and the regional suggestion network are respectively initialized according to the network parameters obtained by training, and the network parameters are determined by adopting the transfer learning method, so that the time spent on training a network model is saved, and the detection speed can be improved when the target detection is finally carried out; secondly, the convolutional neural network is trained according to the preselected area of the area suggestion network, the area suggestion network is optimized according to the trained convolutional neural network, and the candidate bounding box of the sample image is extracted through the optimized area suggestion network, so that the extracted candidate bounding box of the sample image is reasonable, can adapt to the dynamic change in the sample image, and can improve the detection precision.
An embodiment of the present invention provides a computer storage medium storing a computer program for executing the above-described method.
By way of example, computer-readable storage media can be any available media that can be accessed by a computer or a data storage device, such as a server, data center, etc., that includes one or more available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
Through the above description of the embodiments, it is clear to those skilled in the art that, for convenience and simplicity of description, only the division of the functional modules is illustrated, and in practical applications, the above function distribution may be completed by different functional modules according to needs, that is, the internal structure of the device may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.
In the several embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The above description is only for the specific embodiments of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present invention, and the changes or substitutions should be covered within the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (7)
1. A method of training a fully connected classification network for false drop detection, the method comprising:
determining network parameters through transfer learning by using the sample images;
initializing a convolutional neural network and a regional proposal network according to the network parameters;
training the convolutional neural network according to the area suggestion network;
optimizing the area suggestion network according to the trained convolutional neural network; and
training a fully-connected classification network according to the sample feature map of the sample image and the candidate bounding box of the sample image;
wherein the candidate bounding box of the sample image is extracted through the optimized regional suggestion network;
wherein training the convolutional neural network according to the region proposal network comprises: generating a first training area using the area suggestion network; and training the convolutional neural network based on the first training region;
wherein optimizing the area recommendation network according to the trained convolutional neural network comprises: training the area suggestion network according to the trained convolutional neural network; generating a second training area by using the trained area suggestion network; and optimizing the area proposal network according to the trained area proposal network and the second training area.
2. The method of claim 1, further comprising: before the step of training a fully-connected classification network based on the sample feature map of the sample image and the candidate box of the sample image,
inputting the candidate bounding box of the sample image processed by the regional pooling layer into a fully-connected classification network, and outputting a target classification and a bounding box position;
adjusting the position of the boundary frame according to the coordinate correction; and
and screening the candidate bounding boxes of the adjusted sample image according to a deviation sample filtering method, and determining a new candidate bounding box of the sample image.
3. The method of claim 2, further comprising:
updating a fully-connected classification network according to the sample feature map of the sample image and the new candidate bounding box of the sample image;
circularly executing the following steps until the coordinate correction amount is a preset value: and inputting the candidate bounding box of the sample image processed by the regional pooling layer into a full-connection classification network, outputting the target classification, and outputting the position of the bounding box.
4. A method for leak detection, the method comprising:
identifying a bounding box of an image to be detected of an object by using a trained fully connected classification network derived by the method of any one of claims 1-3 to determine a false drop and leak detection result.
5. The method of claim 4, further comprising:
digging a boundary frame of the image to be detected according to a preset rule; and
the clipped bounding box is displayed.
6. An apparatus for training a fully connected classification network for false drop detection, the apparatus comprising:
a determination module for determining network parameters by transfer learning using the sample images;
the initialization module is used for initializing the convolutional neural network and the area suggestion network according to the network parameters;
a first training module for training the convolutional neural network according to the regional suggestion network;
the optimization module is used for optimizing the area suggestion network according to the trained convolutional neural network; and
the second training module is used for training a fully-connected classification network according to the sample feature map of the sample image and the candidate bounding box of the sample image;
wherein the candidate bounding box of the sample image is extracted through the optimized regional suggestion network;
wherein training the convolutional neural network according to the region proposal network comprises: generating a first training area using the area suggestion network; and training the convolutional neural network based on the first training region;
wherein optimizing the area recommendation network according to the trained convolutional neural network comprises: training the area suggestion network according to the trained convolutional neural network; generating a second training area by using the trained area suggestion network; and optimizing the area proposal network according to the trained area proposal network and the second training area.
7. A device for the detection of a leakage, burst or drip, said device comprising: a recognition module for recognizing the bounding box of the image to be detected of the target by using the trained fully-connected classification network obtained by the apparatus of claim 6 to determine the detection result of the leakage.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002906.XA CN109712140B (en) | 2019-01-02 | 2019-01-02 | Method and device for training fully-connected classification network for leakage detection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910002906.XA CN109712140B (en) | 2019-01-02 | 2019-01-02 | Method and device for training fully-connected classification network for leakage detection |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109712140A CN109712140A (en) | 2019-05-03 |
| CN109712140B true CN109712140B (en) | 2021-01-26 |
Family
ID=66260740
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910002906.XA Active CN109712140B (en) | 2019-01-02 | 2019-01-02 | Method and device for training fully-connected classification network for leakage detection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109712140B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110443822B (en) * | 2019-07-16 | 2021-02-02 | 浙江工业大学 | Semantic edge-assisted high-resolution remote sensing target fine extraction method |
| CN111723720A (en) * | 2020-06-15 | 2020-09-29 | 中国石油大学(华东) | Intelligent visual real-time online monitoring system for organic gas leakage |
| CN111931826A (en) * | 2020-07-20 | 2020-11-13 | 华南理工大学 | Rolling bearing fault diagnosis method and system based on multi-scale convolution migration model |
| CN112465126B (en) * | 2020-07-27 | 2023-12-19 | 国电内蒙古东胜热电有限公司 | Loading pre-training convolutional network detection method and device for detecting running-off drips |
| CN112183185A (en) * | 2020-08-13 | 2021-01-05 | 天津大学 | Liquid leakage detection method based on optical flow method and CNN-SVM |
| CN112183187A (en) * | 2020-08-15 | 2021-01-05 | 天津大学 | Liquid leakage detection method based on Selective Search and LeNet-5 |
| CN112115941B (en) * | 2020-09-11 | 2023-12-05 | 北京锐安科技有限公司 | Fire detection method, device, equipment and storage medium |
| CN112934758B (en) * | 2020-12-14 | 2022-10-14 | 中科院计算所西部高等技术研究院 | Coal sorting hand-dialing control method based on image recognition |
| CN112686864A (en) * | 2020-12-30 | 2021-04-20 | 中山嘉明电力有限公司 | Liquid and gas leakage and overflow identification method and system based on image identification |
| CN114202725A (en) * | 2021-12-07 | 2022-03-18 | 牧原食品股份有限公司 | Biogas slurry overflow monitoring method and system based on intelligent vision |
| CN115457297B (en) * | 2022-08-23 | 2023-09-26 | 中国航空油料集团有限公司 | Oil leakage detection method and device for aviation oil depot and aviation oil safety operation and maintenance system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205909857U (en) * | 2016-05-11 | 2017-01-25 | 深圳钰湖电力有限公司 | Intelligent all -round real -time monitoring system of power plant |
| CN106203506B (en) * | 2016-07-11 | 2019-06-21 | 上海凌科智能科技有限公司 | A kind of pedestrian detection method based on depth learning technology |
| CN108280187B (en) * | 2018-01-24 | 2021-06-01 | 湖南省瞬渺通信技术有限公司 | Hierarchical image retrieval method based on depth features of convolutional neural network |
| CN108961229A (en) * | 2018-06-27 | 2018-12-07 | 东北大学 | Cardiovascular OCT image based on deep learning easily loses plaque detection method and system |
| CN109029641A (en) * | 2018-07-27 | 2018-12-18 | 华南理工大学 | A kind of water meter automatic testing method based on Faster-rcnn |
-
2019
- 2019-01-02 CN CN201910002906.XA patent/CN109712140B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109712140A (en) | 2019-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109712140B (en) | Method and device for training fully-connected classification network for leakage detection | |
| CN110033471B (en) | Frame line detection method based on connected domain analysis and morphological operation | |
| CN107543828B (en) | Workpiece surface defect detection method and system | |
| JP2024509411A (en) | Defect detection method, device and system | |
| CN107808161B (en) | Underwater target identification method based on optical vision | |
| CN109598287A (en) | The apparent flaws detection method that confrontation network sample generates is generated based on depth convolution | |
| CN107507170A (en) | A kind of airfield runway crack detection method based on multi-scale image information fusion | |
| CN109671071A (en) | A kind of underground piping defect location and grade determination method based on deep learning | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| CN108154159A (en) | A kind of method for tracking target with automatic recovery ability based on Multistage Detector | |
| US20180286074A1 (en) | Image processing device, image processing method and storage medium | |
| TW201512649A (en) | Method of chip detects inspecting, system therefor, and computer program product thereof | |
| CN109975307A (en) | Bearing surface defect detection system and detection method based on statistics projection training | |
| CN109977965B (en) | Method and device for determining detection target in remote sensing airport image | |
| CN114078106A (en) | Defect detection method based on improved Faster R-CNN | |
| Xu et al. | Based on improved edge detection algorithm for English text extraction and restoration from color images | |
| Kalpana et al. | An efficient edge detection algorithm for flame and fire image processing | |
| Huahong et al. | A new type method of adhesive handwritten digit recognition based on improved faster RCNN | |
| CN113792806A (en) | Anti-patch generation method | |
| Xie et al. | Defect detection of printed circuit board based on small target recognition network | |
| US20210233231A1 (en) | Image processing device, image processing method, and computer readable recording medium | |
| CN116584100A (en) | Image space detection suitable for overlay media content | |
| CN110188601A (en) | A kind of airport remote sensing images detection method based on study | |
| CN111553876A (en) | Pneumatic optical sight error image processing method and system | |
| CN110807456A (en) | Method and device for positioning bank card number |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20200527 Address after: 017000 Ordos the Inner Mongolia Autonomous Region District of Dongsheng City Fuxing Road No. 18 Applicant after: GUODIAN INNER MONGOLIA DONGSHENG COGENERATION Co.,Ltd. Applicant after: ZHONGYING QINGCHUANG TECHNOLOGY Co.,Ltd. Address before: 210000 Second Floor, Block B5, Kowloon Lake International Enterprise Headquarters Park, 19 Suyuan Avenue, Jiangning District, Nanjing City, Jiangsu Province Applicant before: ZHONGYING QINGCHUANG TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |