CN111222551A - Sewage pipeline defect image identification method and device, storage medium and electronic equipment - Google Patents
Sewage pipeline defect image identification method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN111222551A CN111222551A CN201911398732.XA CN201911398732A CN111222551A CN 111222551 A CN111222551 A CN 111222551A CN 201911398732 A CN201911398732 A CN 201911398732A CN 111222551 A CN111222551 A CN 111222551A
- Authority
- CN
- China
- Prior art keywords
- defect
- sewage pipeline
- image
- target
- characteristic information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 230
- 239000010865 sewage Substances 0.000 title claims abstract description 120
- 238000000034 method Methods 0.000 title claims abstract description 38
- 238000013145 classification model Methods 0.000 claims abstract description 34
- 239000013598 vector Substances 0.000 claims description 34
- 238000004590 computer program Methods 0.000 claims description 10
- 238000000605 extraction Methods 0.000 claims description 9
- 238000005259 measurement Methods 0.000 claims description 6
- 230000003993 interaction Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 abstract description 9
- 238000012549 training Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 7
- 239000013049 sediment Substances 0.000 description 5
- 238000004891 communication Methods 0.000 description 3
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a sewage pipeline defect image identification method, which belongs to the field of image detection and comprises the following steps: acquiring a target sewage pipeline defect image; extracting characteristic information from the target sewage pipeline defect image; calculating the class posterior probability that the target sewage pipeline defect image belongs to the classified sewage pipeline defect according to the characteristic information and the representative characteristic information of the classified sewage pipeline defect sample image based on a defect classification model; judging whether the class posterior probability is smaller than a set threshold value; if the class posterior probability is smaller than a set threshold value, outputting the target sewage pipeline defect image; and if the class posterior probability is not less than the set threshold value, outputting and determining the defect type of the target sewage pipeline defect image according to the class posterior probability. The method can learn the classification and identification of the defects of the sewage pipeline based on a small amount of sewage pipeline defect images and automatically learn.
Description
Technical Field
The invention belongs to the field of image processing, and particularly relates to a sewage pipeline defect image identification method, a sewage pipeline defect image identification device, a storage medium and electronic equipment.
Background
The existing sewage pipeline defect detection system is used for realizing automatic sewage pipeline defect detection by performing feature extraction and training a neural network based on computer vision. However, this method requires a large amount of label data, is relatively expensive to operate, and cannot identify images with other defects that are not included in the training set used by the inspection system.
Disclosure of Invention
In order to solve the above problems in the prior art, embodiments of the present application provide a method and an apparatus for identifying a sewage pipe defect image, a storage medium, and an electronic device. The method can classify the sewage pipeline defect images under the condition of a small amount of label data based on small sample learning and metric learning, and can identify new classes.
In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:
in a first aspect, an embodiment of the present application provides a sewage conduit defect image identification method, including:
acquiring a target sewage pipeline defect image;
extracting characteristic information from the target sewage pipeline defect image;
calculating the class posterior probability that the target sewage pipeline defect image belongs to the classified sewage pipeline defect according to the characteristic information and the representative characteristic information of the classified sewage pipeline defect sample image based on a defect classification model; wherein the number of sewer defect sample images of each category in the defect classification model is not more than 20;
judging whether the class posterior probability is smaller than a set threshold value;
if the class posterior probability is smaller than a set threshold value, outputting the target sewage pipeline defect image; and if the class posterior probability is not less than the set threshold value, determining and outputting the defect class of the defect image of the target sewage pipeline according to the class posterior probability.
Preferably, the feature information and the representative feature information are both vectors.
Preferably, each class of the sewer defect image has a plurality of modalities, and the representative feature information is a center vector of the sewer defect sample images of the plurality of modalities.
Preferably, the extracting of the feature information from the target sewer pipe defect image includes:
extracting a candidate region from the sewer defect image based on a region suggestion network;
extracting feature vectors from the candidate regions based on a feature extraction network.
Preferably, the calculating, based on the defect classification model, the class posterior probability that the target sewer line defect image belongs to the classified sewer line defect according to the characteristic information and the representative characteristic information of the classified sewer line defect sample image includes:
the measurement learning module inputs the representative characteristic information of the classified sewage pipeline defect sample image into a defect classification model;
inputting the characteristic information into the measurement learning module to obtain an embedded vector of the target sewage pipeline defect image;
and calculating the similarity according to the distance from the embedded vector to each representative characteristic information, and outputting the class posterior probability of the target sewage pipeline defect image.
Preferably, after outputting the target sewage pipe defect image when the class posterior probability is smaller than a set threshold, the method further includes:
acquiring a plurality of example images which belong to the same defect type with the target sewage pipeline defect image to obtain a defect sample set;
extracting example feature information from all example images of the defect sample set;
and adding the example characteristic information as representative characteristic information of the defect sample set into the defect classification model, and updating the defect classification model.
In a second aspect, an embodiment of the present application provides a sewage conduit defect image classification apparatus, including:
the image acquisition module is used for acquiring a defect image of the target sewage pipeline;
the characteristic extraction module is used for extracting characteristic information from the target sewage pipeline defect image;
the defect classification module is used for obtaining class posterior probability between the characteristic information and the classified representative characteristic information of the sewage pipeline defect image based on a defect classification model; determining the defect type of the target sewage pipeline defect image according to the type posterior probability;
the comparison module is used for judging whether the class posterior probability and the set threshold meet preset conditions or not;
the output interaction module is used for outputting the target sewage pipeline defect image if the class posterior probability is smaller than a set threshold value; and outputting the defect type of the target sewage pipeline defect image according to the class posterior probability if the class posterior probability is not less than the set threshold.
In a third aspect, embodiments of the present application provide a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the sewage conduit defect image identification method as described above when running.
In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the sewage conduit defect image identification method as described above through the computer program.
Compared with the prior art, the embodiment of the application realizes learning under a small amount of samples based on small sample learning and metric learning in a single end-to-end training process so as to realize detection of the defect types of the sewage pipelines and can identify and automatically learn the undetermined defect types of the sewage pipelines. Meanwhile, the model used by the method improves the generalization capability of the model by learning the representative vectors of multiple modes under each category.
Specific embodiments of the present invention are disclosed in detail with reference to the following description and the annexed drawings, which illustrate how the principles of the invention may be employed. It should be understood that the embodiments of the invention are not so limited in scope. The embodiments of the invention include many variations, modifications and equivalents within the spirit and scope of the appended claims.
Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments, in combination with or instead of the features of the other embodiments.
It should be emphasized that the term "comprises/comprising" when used herein, is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps or components.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a preferred embodiment of the present application;
FIG. 2 is a schematic flow chart of a preferred embodiment of the present application;
FIG. 3 is a schematic flow chart of a preferred embodiment of the present application;
FIG. 4 is a schematic flow chart of a preferred embodiment of the present application;
FIG. 5 is a schematic view of a sewage pipe defect image classification apparatus according to a preferred embodiment of the present application;
FIG. 6 is a schematic diagram of a computer device according to a preferred embodiment of the present application.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the drawings in the embodiment of the present invention, and it is obvious that the described embodiment is only a part of the embodiment of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It will be understood that when an element is referred to as being "disposed on" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. The terms "vertical," "horizontal," "left," "right," and the like as used herein are for illustrative purposes only and do not denote a unique embodiment.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
The embodiment of the invention provides a sewage pipeline image classification method and a corresponding image classification device, which can be arranged in any network equipment and are used for performing classification operation on a shot picture and an image of each frame of video capture, and are mainly used for identifying the class of a sewage pipeline defect in the picture or the image of each frame, or when the sewage pipeline defect does not belong to a known class, the sewage pipeline image classification method and the corresponding image classification device are used as a new defect class mark. The network devices may include terminals including, but not limited to, personal computers, hand-held or wearable devices, mobile terminals, multiprocessor systems, minicomputers, distributed computing environments that include any of the above systems or devices, and the like, or servers.
Referring to fig. 1, an embodiment of the present invention provides a sewage conduit defect image identification method, including:
step S1: and acquiring a defect image of the target sewage pipeline.
Step S2: and extracting characteristic information from the target sewage pipeline defect image.
Step S3: and calculating the class posterior probability of the classified sewage pipeline defect image according to the characteristic information and the representative characteristic information of the classified sewage pipeline defect image based on a defect classification model.
Step S4: judging whether the class posterior probability is smaller than a set threshold value; if the class posterior probability is smaller than a set threshold value, outputting the target sewage pipeline defect image; and if the class posterior probability is not less than the set threshold value, determining and outputting the defect class of the defect image of the target sewage pipeline according to the class posterior probability.
The detailed description is as follows:
in step S1, a target sewer pipe defect image is acquired.
In one embodiment of the invention, the target image to be processed may be an image that is visually processed, in particular an image that requires identification of a defect class of the sewage pipe in the image. The image can be a series of pictures shot by a high-frequency camera and capable of expressing the defect of the sewage pipeline, or can be each frame of continuous images shot by a video camera and capable of expressing the defect of the sewage pipeline.
In step S2, feature information is extracted from the target sewer piping defect image.
In some embodiments of the present invention, the defect classification model used in the method can simultaneously learn the embedding space, the backbone network parameters and the representative vectors of the training classes in a single end-to-end training process. The embedding space, the backbone network parameters and the representative vectors can be used as characteristic information to represent the defect image of the target sewage pipeline.
Preferably, the target sewer piping defect image is characterized using vectors. Specifically, referring to fig. 2, in some embodiments, the step S2 includes;
step S201: extracting a candidate region from the target sewage pipeline defect image based on a region suggestion network;
step S202: extracting feature vectors from the candidate regions based on a feature extraction network.
The Region suggestion Network (RPN) may take any one of the target sewer pipe defect images as an input, and then output a set of rectangular simple boxes, that is, regions of interest (RoIs), that is, the aforementioned candidate regions. The RoIs may be plural.
After the feature extraction network (inclusion) is trained, corresponding feature vectors X can be extracted from the RoIs, wherein X belongs to Rf. Where R represents ROIs and f is the number of ROIs generated in the first step.
In step S3, a class posterior probability that the target sewer line defect image belongs to the classified sewer line defect is calculated based on the defect classification model based on the feature information and the representative feature information of the classified sewer line defect sample image. Wherein the number of sewage pipe defect sample images of each category in the defect classification model is not more than 20.
In some embodiments of the present invention, the method uses a defect classification model in which classified sewer defect sample images, i.e., sample images having no more than 20 defect classification labels (labels), have been used as a training set during the training process. For example, in the defect classification model, the training set includes 20 sample images labeled "root intrusion", 20 sample images labeled "crack", and 20 sample images labeled "water seepage". After the defect classification model extracts 5 pieces of defect images from the 3 classes and trains the 5 pieces of defect images, the 3 classes of sewage pipeline defects can be identified from 15 sample images, and the classes of the sample images with the 3 classes of defects can be accurately judged.
And metric learning is used in this step to solve the problem of few sample classification. I.e. extracting features from stored classified sewer defect sample images of the defect classification model and characterizing them using representative feature information, i.e. representative vectors.
And in the embodiment of the invention, each category of the sewage pipeline defect image has a plurality of modals, and the model is generalized through sewage pipeline defect sample images of the plurality of modals. The representative vector is a center vector of the sewage pipe defect sample images of the plurality of modes.
During calculation, the characteristic vector X of each target sewage pipeline defect image is input into the metric learning module, and classification is carried out according to the distance between the characteristic vector X and the classified sewage pipeline defect sample image.
Specifically, referring to fig. 3, in some embodiments, the step S3 includes:
step S301: the measurement learning module inputs the representative characteristic information of the classified sewage pipeline defect sample image into a defect classification model;
step S302: inputting the characteristic information into the measurement learning module to obtain an embedded vector of the target sewage pipeline defect image;
step S303: and performing similarity calculation according to the distance from the embedded vector to each representative characteristic information, and outputting the class posterior probability of the target sewage pipeline defect image.
Specifically, the metric learning module consists of two Fully Connected (FC) layers with Batch Normalization (BN) and ReLU nonlinearities. The output of the metric learning module is an embedded vector E, and E ═ E (x) E ReAnd e is the number of RoIs randomly selected during the computation by the metric learning module, typically e ≦ f (the number of RoIs generated in step S201).
In particular, another FC layer of size NxKxe is used in this embodiment to learn a representative vector for each sewer defect class in the support set. Where N represents the number of total classes of sewer defects and K is the number of modes contained in the mixed distribution of each class. The output of the FC layer is reconstructed into an nxk × e tensor.
Wherein R is used for each representative vector in the sewage conduit defect class that has been determinedi.jIs represented by the formula (I), wherein Ri.j∈ReAnd R isi.jThe mixed distribution in the ith class is judged to be in the center of the j-th order mode of the embedding space by representing learning, and j is more than or equal to 1 and less than or equal to K.
In step S301, for the embedded vector E corresponding to the target sewage pipe defect image or RoIs, the defect classification model in this embodiment calculates the embedded vector E and R of the representative vector in the support seti.jN × K distance matrix in between. The elements of the NxK distance matrix are denoted by dij(E) To indicate. Wherein d isij(E)＝ d(E,Rij) For expression as embedding a vector E into each representative vector RijIs used to calculate a probability p of the target sewer pipe defect image or RoIs in the jth modality of the class i sewer pipe defect classij。
Wherein the content of the first and second substances,
in the above formula, σ2The variance is indicated.
Then, in the embodiment of the present invention, the following formula for calculating the posterior probability P of the discriminant class is:
where C ═ i denotes the ith class.
The maximum value of the probability in each determined sewage pipe defect class is taken as the maximum value of all modes included in the defect class and the probability P is calculated back as class a posteriori.
In step S4, it is determined whether the class posterior probability is smaller than a set threshold; if the class posterior probability is smaller than a set threshold value, outputting a defect image of the target sewage pipeline; and if the class posterior probability is not less than the set threshold value, outputting and determining the defect class of the target sewage pipeline according to the class posterior probability.
Specifically, in some embodiments of the present invention, the threshold is set to 0.5, and whether the class posterior probability P < 0.5 returned in the foregoing step is satisfied is determined.
When the inequality is judged to be satisfied, the defect type in the target sewage pipeline defect detection image is a new detection type. I.e. the defect class is not present in the training set. At this time, the acquired target sewer pipe defect image is returned.
When the inequality is judged to be satisfied, p is obtainedijThe maximum value of (3) determines the sewage pipeline defect type to which the target sewage pipeline defect image or the RoIs belongs.
For example, in some embodiments of the present invention, the posterior probability P of class is 0.4, which indicates that the target sewage pipe defect image is not any of the 3 known sewage pipe defects of "root intrusion", "crack", and "water seepage". For example, in some embodiments of the present invention, when class posterior probability P ═ P is calculatedi＝1When the value is 0.9, the target sewer pipe defect image belongs to the first type of tree root intrusion. However, the "root intrusion" herein may also have a plurality of modalities. For example, in some embodiments of the present invention, when class posterior probability P ═ P is calculatedi＝2,j＝20.5, indicating that the target sewer defect image is closest to the second modality in the second category of "cracks" and still belongs to the category of "crack" defects.
Referring to fig. 4, in some embodiments, after outputting the target sewer pipe defect image when the class posterior probability is less than a set threshold, the method further includes:
step S501: acquiring a plurality of example images which belong to the same defect type with the target sewage pipeline defect image to obtain a defect sample set;
step S502: extracting example feature information from all example images of the defect sample set;
step S503: and adding the example characteristic information as representative characteristic information of the defect sample set into the defect classification model, and updating the defect classification model.
In particular, the present embodiment is configured to add the detected defect type of the new sewer pipe to the defect classification model.
Step S501 is used for classifying the sewage pipeline defects of the type when the user receives the target sewage pipeline defect image returned by the defect classification model. For example, in some embodiments, a target sewer line defect is not any of the "root encroachment," "crack," and "weepage" training sets. The user manually identifies the target sewer pipe defect image as "sediment" based on the returned target sewer pipe defect image, and then the user is required to input a small number of example images of the category "sediment", for example, 2 sediment defect images, to create a sample set of the category "sediment".
The defect classification model is trained through the sample set, and common example feature vectors of all example images of the defect sample set are extracted, so that the example feature vectors are added to the representative vectors of classified sewage pipeline defect sample images in the defect classification model, and the defect classification model can detect 4 defect categories of 'root invasion', 'crack', 'water seepage' and 'sediment' in later detection.
Referring to fig. 5, in order to better implement the image target detection method provided by the embodiment of the present invention, the embodiment of the present invention further discloses a sewage conduit defect image classification device, which includes: the device comprises an image acquisition module, a feature extraction module, a defect classification module, a comparison module and an output interaction module.
The image acquisition module is used for acquiring a defect image of the target sewage pipeline.
And the characteristic extraction module is used for extracting characteristic information from the target sewage pipeline defect image.
And the defect classification module is used for obtaining class posterior probability between the characteristic information and the classified representative characteristic information of the sewage pipeline defect image based on a defect classification model. And determining the defect type of the target sewage pipeline defect image according to the type posterior probability.
And the comparison module is used for judging whether the class posterior probability and the set threshold meet preset conditions.
The output interaction module is used for outputting the target sewage pipeline defect image if the class posterior probability is smaller than a set threshold value; and outputting the defect type of the target sewage pipeline defect image according to the class posterior probability if the class posterior probability is not less than the set threshold.
Those skilled in the art will appreciate that all or some of the steps of the various methods of the above embodiments may be performed by instructions, or by instructions controlling associated hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.
In order to better implement the image target detection method provided by the embodiment of the invention, the embodiment of the invention also provides a computer readable medium, which stores a computer program, and the computer program is executed by a processor to realize the sewage pipeline defect image identification method. For example, the readable medium stores a plurality of instructions that can be loaded by the processor to perform the method for identifying a defective image of a sewage pipeline according to the embodiment of the present invention.
For example: step S1: acquiring a target sewage pipeline defect image;
step S2: extracting characteristic information from the target sewage pipeline defect image;
step S3: calculating the class posterior probability that the target sewage pipeline defect image belongs to the classified sewage pipeline defect according to the characteristic information and the representative characteristic information of the classified sewage pipeline defect image based on a defect classification model;
step S4: judging whether the class posterior probability is smaller than a set threshold value; if the class posterior probability is smaller than a set threshold value, outputting the target sewage pipeline defect image; and if the class posterior probability is not less than the set threshold value, determining and outputting the defect class of the defect image of the target sewage pipeline according to the class posterior probability.
The above operations can be implemented in the foregoing embodiments, and are not described in detail herein.
Wherein the readable medium may include: readable memory (ROM), random access memory (RAM, magnetic or optical disk, etc.).
Fig. 6 shows a schematic structural diagram of a computer device provided in an exemplary embodiment of the present application. The server is used for implementing the sewage pipeline defect image identification method provided in the above embodiment. Specifically, the method comprises the following steps: the server includes a Central Processing Unit (CPU)401, a system memory 404 including a Random Access Memory (RAM)402 and a Read Only Memory (ROM)403, and a system bus 405 connecting the system memory 404 and the central processing unit 401. The server 400 also includes a basic input/output system (I/O system) 406, which facilitates the transfer of information between devices within the computer, and a mass storage device 407 for storing an operating system 413, application programs 414, and other program modules 415.
The basic input/output system 406 includes a display 408 for displaying information and an input device 409 such as a mouse, keyboard, etc. for user input of information. Wherein the display 408 and the input device 409 are connected to the central processing unit 401 through an input output controller 410 connected to the system bus 405. The basic input/output system 406 may also include an input/output controller 410 for receiving and processing input from a number of other devices, such as a keyboard, mouse, or electronic stylus. Similarly, input/output controller 410 also provides output to a display screen, a printer, or other type of output device.
The mass storage device 407 is connected to the central processing unit 401 through a mass storage controller (not shown) connected to the system bus 405. The mass storage device 407 and its associated computer-readable media provide non-volatile storage for the server 400. That is, the mass storage device 407 may include a computer-readable medium (not shown) such as a hard disk or CD-ROM drive.
In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer-readable storage medium executable by a processor. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the exemplary embodiments of the present invention, and they should be construed as being included therein. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (9)
1. A sewage conduit defect image identification method, the method comprising:
acquiring a target sewage pipeline defect image;
extracting characteristic information from the target sewage pipeline defect image;
calculating the class posterior probability that the target sewage pipeline defect image belongs to the classified sewage pipeline defect according to the characteristic information and the representative characteristic information of the classified sewage pipeline defect sample image based on a defect classification model; wherein the number of sewer defect sample images of each category in the defect classification model is not more than 20;
judging whether the class posterior probability is smaller than a set threshold value;
if the class posterior probability is smaller than a set threshold value, outputting the target sewage pipeline defect image; and if the class posterior probability is not less than the set threshold value, determining and outputting the defect class of the defect image of the target sewage pipeline according to the class posterior probability.
2. The method of claim 1, wherein the characteristic information and the representative characteristic information are vectors.
3. The method of claim 2, wherein each class of the sewer pipe defect sample image has a plurality of modes, and the representative feature information is a center vector of the sewer pipe defect sample images of the plurality of modes.
4. The method of claim 2, wherein extracting the characteristic information from the target sewer pipe defect image comprises:
extracting candidate regions from the target sewer piping defect image based on a region suggestion network;
extracting feature vectors from the candidate regions based on a feature extraction network.
5. The method of claim 2, wherein calculating the class posterior probability that the target sewer line defect image belongs to the classified sewer line defect based on the defect classification model based on the characteristic information and representative characteristic information of the classified sewer line defect sample image comprises:
the measurement learning module inputs the representative characteristic information of the classified sewage pipeline defect sample image into a defect classification model;
inputting the characteristic information into the measurement learning module to obtain an embedded vector of the target sewage pipeline defect image;
and calculating the similarity according to the distance from the embedded vector to each representative characteristic information, and outputting the class posterior probability of the target sewage pipeline defect image.
6. The method of claim 1, wherein after outputting the target sewer pipe defect image if the class a posteriori probability is less than a set threshold, further comprising:
acquiring a plurality of example images which belong to the same defect type with the target sewage pipeline defect image to obtain a defect sample set;
extracting example feature information from all example images of the defect sample set;
and adding the example characteristic information as representative characteristic information of the defect sample set into the defect classification model, and updating the defect classification model.
7. A sewage conduit defect image classification apparatus, comprising:
the image acquisition module is used for acquiring a defect image of the target sewage pipeline;
the characteristic extraction module is used for extracting characteristic information from the target sewage pipeline defect image;
the defect classification module is used for obtaining class posterior probability between the characteristic information and the representative characteristic information of the classified sewage pipeline defect image based on a defect classification model and determining the defect type of the target sewage pipeline defect image according to the class posterior probability;
the comparison module is used for judging whether the class posterior probability and the set threshold meet preset conditions or not;
the output interaction module is used for outputting the target sewage pipeline defect image if the class posterior probability is smaller than a set threshold value; and outputting the defect type of the target sewage pipeline defect image according to the class posterior probability if the class posterior probability is not less than the set threshold.
8. A storage medium readable by a computer, in which a computer program is stored, wherein the computer program is configured to execute the sewage pipe defect image recognition method according to any one of claims 1-6 when executed.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and wherein the processor is configured to execute the sewer defect image recognition method of any of claims 1-6 by the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398732.XA CN111222551A (en) | 2019-12-30 | 2019-12-30 | Sewage pipeline defect image identification method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911398732.XA CN111222551A (en) | 2019-12-30 | 2019-12-30 | Sewage pipeline defect image identification method and device, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111222551A true CN111222551A (en) | 2020-06-02 |
Family
ID=70825954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911398732.XA Pending CN111222551A (en) | 2019-12-30 | 2019-12-30 | Sewage pipeline defect image identification method and device, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111222551A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0793545A (en) * | 1993-09-22 | 1995-04-07 | Mitsubishi Electric Corp | Picture processor |
| CN101976350A (en) * | 2010-10-20 | 2011-02-16 | 中国农业大学 | Grain storage pest detection and identification method based on video analytics and system thereof |
| CN106338520A (en) * | 2016-09-18 | 2017-01-18 | 南京林业大学 | Recognition method of surface defects of multilayer solid wood composite floor with surface board being jointed board |
| US20170357896A1 (en) * | 2016-06-09 | 2017-12-14 | Sentient Technologies (Barbados) Limited | Content embedding using deep metric learning algorithms |
| CN108346137A (en) * | 2017-01-22 | 2018-07-31 | 上海金艺检测技术有限公司 | Defect inspection method for industrial x-ray weld image |
| CN108615039A (en) * | 2016-12-09 | 2018-10-02 | 广东技术师范学院 | Cartridge case defect automatic testing method based on computer vision |
| CN109165673A (en) * | 2018-07-18 | 2019-01-08 | 广东工业大学 | Image classification method based on metric learning and more example support vector machines |
| US20190156155A1 (en) * | 2017-11-20 | 2019-05-23 | Fujitsu Limited | Information processing method and information processing apparatus |
| GB201912844D0 (en) * | 2019-09-06 | 2019-10-23 | Smiths Heimann Sas | Image retrieval system |
| CN112560876A (en) * | 2021-02-23 | 2021-03-26 | 中国科学院自动化研究所 | Single-stage small sample target detection method for decoupling measurement |
-
2019
- 2019-12-30 CN CN201911398732.XA patent/CN111222551A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0793545A (en) * | 1993-09-22 | 1995-04-07 | Mitsubishi Electric Corp | Picture processor |
| CN101976350A (en) * | 2010-10-20 | 2011-02-16 | 中国农业大学 | Grain storage pest detection and identification method based on video analytics and system thereof |
| US20170357896A1 (en) * | 2016-06-09 | 2017-12-14 | Sentient Technologies (Barbados) Limited | Content embedding using deep metric learning algorithms |
| CN106338520A (en) * | 2016-09-18 | 2017-01-18 | 南京林业大学 | Recognition method of surface defects of multilayer solid wood composite floor with surface board being jointed board |
| CN108615039A (en) * | 2016-12-09 | 2018-10-02 | 广东技术师范学院 | Cartridge case defect automatic testing method based on computer vision |
| CN108346137A (en) * | 2017-01-22 | 2018-07-31 | 上海金艺检测技术有限公司 | Defect inspection method for industrial x-ray weld image |
| US20190156155A1 (en) * | 2017-11-20 | 2019-05-23 | Fujitsu Limited | Information processing method and information processing apparatus |
| CN109165673A (en) * | 2018-07-18 | 2019-01-08 | 广东工业大学 | Image classification method based on metric learning and more example support vector machines |
| GB201912844D0 (en) * | 2019-09-06 | 2019-10-23 | Smiths Heimann Sas | Image retrieval system |
| CN112560876A (en) * | 2021-02-23 | 2021-03-26 | 中国科学院自动化研究所 | Single-stage small sample target detection method for decoupling measurement |
Non-Patent Citations (8)
| Title |
|---|
| LEONID KARLINSKY: "RepMet: Representative-based metric learning for classification and few-shot object detection", 《IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》, pages 4 * |
| 于重重;萨良兵;马先钦;陈秀新;赵霞;: "基于度量学习的小样本零器件表面缺陷检测", 仪器仪表学报, no. 07 * |
| 李凌燕: "基于机器视觉的焊点检测算法研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅰ辑》 * |
| 李凌燕: "基于机器视觉的焊点检测算法研究", 《中国优秀硕士学位论文全文数据库 工程科技Ⅰ辑》, 15 August 2019 (2019-08-15) * |
| 汪荣贵: "代表特征网络的小样本学习方法", 《中国图象图形学报》 * |
| 汪荣贵: "代表特征网络的小样本学习方法", 《中国图象图形学报》, vol. 24, no. 9, 16 September 2019 (2019-09-16), pages 1519 * |
| 郑岩: "基于深度特征度量的小样本学习方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
| 郑岩: "基于深度特征度量的小样本学习方法研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, 15 April 2020 (2020-04-15) * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108229322B (en) | Video-based face recognition method and device, electronic equipment and storage medium | |
| US10803398B2 (en) | Apparatus and method for information processing | |
| US11182899B2 (en) | Systems and methods for processing electronic images to detect contamination | |
| CN108985190B (en) | Target identification method and device, electronic equipment and storage medium | |
| CN109255767B (en) | Image processing method and device | |
| CN111369523B (en) | Method, system, equipment and medium for detecting cell stack in microscopic image | |
| CN112990318B (en) | Continuous learning method, device, terminal and storage medium | |
| CN112464803A (en) | Image comparison method and device | |
| CN108229289B (en) | Target retrieval method and device and electronic equipment | |
| CN112052840A (en) | Picture screening method, system, equipment and storage medium | |
| CN113643260A (en) | Method, apparatus, device, medium and product for detecting image quality | |
| CN114360053A (en) | Action recognition method, terminal and storage medium | |
| CN116415020A (en) | Image retrieval method, device, electronic equipment and storage medium | |
| CN111523399A (en) | Sensitive video detection and device | |
| CN116977260A (en) | Target defect detection method and device, electronic equipment and storage medium | |
| CN111222551A (en) | Sewage pipeline defect image identification method and device, storage medium and electronic equipment | |
| CN114639056A (en) | Live content identification method and device, computer equipment and storage medium | |
| CN110827261B (en) | Image quality detection method and device, storage medium and electronic equipment | |
| CN110647826B (en) | Method and device for acquiring commodity training picture, computer equipment and storage medium | |
| CN113313642A (en) | Image denoising method and device, storage medium and electronic equipment | |
| CN112184702A (en) | Picture cheating detection method and device, electronic equipment and storage medium | |
| CN112131418A (en) | Target labeling method, target labeling device and computer-readable storage medium | |
| US20200117444A1 (en) | Intelligent Visual Regression System | |
| CN117523345B (en) | Target detection data balancing method and device | |
| CN115861321B (en) | Production environment detection method and system applied to industrial Internet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200602 |