CN112308040A - River sewage outlet detection method and system based on high-definition images - Google Patents

River sewage outlet detection method and system based on high-definition images Download PDF

Info

Publication number
CN112308040A
CN112308040A CN202011344676.4A CN202011344676A CN112308040A CN 112308040 A CN112308040 A CN 112308040A CN 202011344676 A CN202011344676 A CN 202011344676A CN 112308040 A CN112308040 A CN 112308040A
Authority
CN
China
Prior art keywords
feature
target
image
model
layers
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011344676.4A
Other languages
Chinese (zh)
Inventor
谷永辉
刘昌军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shandong Jiexun Communication Technology Co ltd
Original Assignee
Shandong Jiexun Communication Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shandong Jiexun Communication Technology Co ltd filed Critical Shandong Jiexun Communication Technology Co ltd
Priority to CN202011344676.4A priority Critical patent/CN112308040A/en
Publication of CN112308040A publication Critical patent/CN112308040A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/52Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/23Clustering techniques
    • G06F18/232Non-hierarchical techniques
    • G06F18/2321Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/241Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • G06V10/44Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Multimedia (AREA)
  • Evolutionary Biology (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

The utility model provides a river sewage outlet detection method and system based on high-definition images, which comprises the following steps: constructing a model, extracting image features by using a multi-scale residual error feature extraction network in the model to fuse shallow features and deep features and generate a plurality of feature layers, and sending the plurality of feature layers into a feature pyramid structure for feature extraction; the high-definition image is extracted through a feature extraction network and a feature pyramid structure to generate a plurality of different output layers, a drain target is detected through a target classifier and a target regressor, and a recognition result is output. The YOLOV4 model for small target detection solves the problem of low detection precision of the aerial image river sewage outlet; supplementary artifical image drain that detects by plane improves the drain and detects the precision.

Description

River sewage outlet detection method and system based on high-definition images
Technical Field
The disclosure belongs to the technical field of target detection, and particularly relates to a river drain outlet detection method and system based on a high-definition image.
Background
The statements in this section merely provide background information related to the present disclosure and may not necessarily constitute prior art.
Illegal discharge of industrial wastewater and domestic sewage causes pollution of a large amount of rivers. The river drain outlet has: the device has the characteristics of large quantity, wide distribution, multiple forms, strong concealment and the like, and causes great difficulty in the inspection of the non-compliant sewage discharge outlets. The traditional ground investigation method has a plurality of defects: high cost, low efficiency, long period and easy omission. And adopt unmanned aerial vehicle to take photo by plane and detect then can overcome above-mentioned shortcoming, unmanned aerial vehicle takes photo by plane at present and detects and has been applied to river drain investigation.
Until now, the interpretation work of the river sewage outlet image of the unmanned aerial vehicle aerial photography mainly depends on expert experience, namely, image judging personnel need professional sewage outlet screening training and manually mark the position of the sewage outlet in the image. However, the aerial image of the unmanned aerial vehicle has the ultra-large resolution exceeding that of a common image, and the sewage draining port occupies less than 0.01% of the whole image in terms of pixels, and is various in form and serious in interference of background information. The sewage draining exit is difficult to distinguish manually, leakage detection and false detection are easy to occur, and the workload of field verification exploration personnel is greatly increased. Meanwhile, due to the fact that the number of the pictures returned by the unmanned aerial vehicle is large, the efficiency and the accuracy of distinguishing the drain are affected by factors such as visual fatigue caused by distinguishing the drain for a long time. The method of manually detecting the aerial images is time-consuming and labor-consuming.
Nowadays, target detection and identification based on deep learning become the mainstream method, which can be mainly expressed as: the depth feature extraction of the image- > target identification and positioning based on a depth neural network, wherein a Convolutional Neural Network (CNN) is mainly used as a depth neural network model.
Currently, the existing target detection and recognition algorithms based on deep learning can be roughly classified into the following three categories: target detection and identification algorithms based on regional recommendations, such as R-CNN, Fast-R-CNN, Faster-R-CNN; regression-based target detection and identification algorithms, such as YOLO, SSD; search-based object detection and recognition algorithms, such as AttentionNet based on visual attention.
The existing target detection methods are all used for detecting targets of general data sets such as Pascal VOC, COCO, Image Net and the like, and the targets are large in scale and relatively easy to detect. And the drain target in the aerial image has the characteristics of small size, multiple varieties, serious background information interference and the like, so that the detection difficulty is increased.
Disclosure of Invention
In order to overcome the defects of the prior art, the river drain outlet detection method based on the high-definition images is provided, the used model can be used for detecting small targets, the detection precision of the model is greatly improved, the missing rate is reduced, and the task of identifying the drain outlet target in the aerial images is realized.
In order to achieve the above object, one or more embodiments of the present disclosure provide the following technical solutions:
in a first aspect, a river sewage outlet detection method based on a high-definition image is disclosed, which comprises the following steps:
constructing a model, extracting image features by using a multi-scale residual error feature extraction network in the model to fuse shallow features and deep features and generate a plurality of feature layers, and sending the plurality of feature layers into a feature pyramid structure for feature extraction;
the high-definition image is extracted through a feature extraction network and a feature pyramid structure to generate a plurality of different output layers, a drain target is detected through a target classifier and a target regressor, and a recognition result is output.
According to the further technical scheme, before the model is built, high-definition river images are obtained, a data set required by a training model is built, the images in the data set are partitioned, rectangular frame labeling is carried out on a sewage outlet in the images, the model training data set and a test set are built, and the built model is trained and tested respectively.
According to the further technical scheme, the high-definition river image is an image obtained by aerial photography of the unmanned aerial vehicle, and a drain outlet in the aerial river image is marked by adopting a target detection marking tool;
the drain includes two types: in the labeling process, the tool frame is used for selecting the whole drain target, the coordinate of a rectangular frame of the drain target is recorded, and a category label of the drain is set;
storing the labeled information into a file according to a format protocol of a tool, which specifically comprises the following steps: target labeling category, target upper left corner X and Y coordinates, and target lower right corner X and Y coordinates.
According to the further technical scheme, after entering a feature extraction network, the image sequentially passes through corresponding convolution blocks from top to bottom, finally three feature layers with different scales and channels are formed, and the three feature layers are further transmitted into an SPP structure and a PaNet structure of a network feature pyramid.
According to the further technical scheme, three feature layers extracted from the network feature pyramid are processed by using an SPP structure and a PANet structure, wherein the SPP structure conducts three times of convolution on the minimum feature layer and then conducts processing on the minimum feature layer by using four maximum pools with different scales respectively, so that the sense field can be greatly increased, and the most remarkable context features can be separated.
According to the further technical scheme, the three output layers are placed into a PANET structure to be subjected to repeated feature extraction, the feature extraction from top to bottom is realized through the PANET, and the three output layers are sequentially divided into A, B, C by the number of channels;
and E, performing over-sampling and under-sampling to perform feature fusion with D to form a feature layer F, and fusing F with the feature layer C subjected to SPP to form a feature layer G, wherein the feature layer E, F, G is the final output layer.
In a second aspect, a river sewage outlet detection system based on high definition images is disclosed, comprising:
the model construction module is used for constructing a model, extracting image features by utilizing a multi-scale residual error feature extraction network in the model so as to fuse shallow features and deep features and generate a plurality of feature layers, and sending the feature layers into a feature pyramid structure for feature extraction continuously;
and the river drain outlet identification module generates a plurality of different output layers after the high-definition images are extracted through the feature extraction network and the feature pyramid structure, detects a drain outlet target through the target classifier and the target regressor and outputs an identification result.
The above one or more technical solutions have the following beneficial effects:
the model used by the technical scheme can be used for detecting the small target, so that the detection precision of the model is greatly improved, and the missing rate is reduced. The task of identifying the sewage outlet target in the aerial image is achieved.
The method and the device solve the problem of low detection precision of the river sewage outlet in the aerial image by using the YOLOV4 model for detecting the small target; supplementary artifical image drain that detects by plane improves the drain and detects the precision.
According to the technical scheme, the latest CSPDarkNet53 model and special feature pyramid structure SPP and PANet are adopted through the YOLOV4 network model to adapt to small target detection at the sewage discharge port of the aerial river image, and the target detection precision is improved.
According to the technical scheme, the CSPDarkNet53 feature extraction network is used for extracting image features, the CSPDarkNet53 feature extraction network has three feature output layers with features of different scales, feature extraction is effectively carried out on the problem that small targets are difficult to detect, and meanwhile the problem of neural network degradation is solved well by using a residual edge structure in each rolling block.
The technical scheme of the present disclosure uses a special characteristic pyramid structure: SPP and PANet structures. The SPP structure can effectively increase the receptive field and separate the most obvious context characteristics by further performing convolution and maximum pooling on the characteristic layer, thereby playing a good role in detecting small targets. The structure of the PANet is different from the characteristic fusion process of the traditional characteristic pyramid network from bottom to top, and the PANet also requires the characteristic extraction from top to bottom. Thereby enabling the model to better detect small target objects.
According to the technical scheme, a Mosaic data enhancement technology is used in the stage of training a data set, four pictures are firstly turned over, zoomed, changed in color gamut and the like, and the pictures are well arranged according to four directions. And then, mosaics use the processed four pictures to splice, and then send the pictures into a model for training. The data enhancement technology can greatly enrich the background of the detected object, and particularly solves the problem of serious interference of background information of small target detection.
The technical scheme disclosed uses the programming of the Pythrch deep learning framework, has excellent expansion capability, is convenient for later modification and tuning, and is convenient for users to operate. The method has certain practical application value in the inspection of the aerial photography river sewage discharge outlet.
Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The accompanying drawings, which are included to provide a further understanding of the disclosure, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure and are not to limit the disclosure.
FIG. 1 is a schematic view of a sewage outlet detection model of an aerial image of Yolov 4;
FIG. 2 is a schematic diagram of a CSPDarkNet53 feature extraction network;
FIG. 3 is a schematic diagram of a pyramid model with special features having SPP structure;
FIG. 4 is a schematic diagram of a pyramid model with special features of a PANet structure;
FIG. 5 shows a result diagram of detection of a target at a sewage drain outlet of an aerial photography.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
The embodiments and features of the embodiments in the present disclosure may be combined with each other without conflict.
The general idea proposed by the present disclosure:
the invention relates to an unmanned aerial vehicle aerial image river sewage outlet detection method based on YoloV4, which comprises the following steps: shooting river images by using an unmanned aerial vehicle; preprocessing the acquired image: separating, building materials, labeling targets, and dividing an image training set and a test set; in order to better aim at the target detection of small targets, a YOLOV4 model is constructed, a CSPDarkNet53 feature extraction network is used in the model, feature information of different scales is extracted by a multi-scale detector, meanwhile, a residual edge structure is adopted for spatial information compensation, the CSPDarkNet53 network improves the receptive field and strengthens the fusion of shallow semantic information and deep semantic information; while the data set is augmented using mosaic data enhancement techniques. The model Yolov4 is used for training, the performance of the model is evaluated on a test set, and the trained Yolov4 model is used for actual aerial image detection of a river sewage outlet. The aerial image drain outlet target detection method can be used for efficiently, accurately and quickly detecting the aerial image drain outlet target, overcomes the defects of easiness in false detection, easiness in missed detection and the like of the traditional method, and greatly improves the average precision of target detection.
Example one
The embodiment discloses that the embodiment provides an aerial image river sewage outlet detection method based on YOLOV 4. The method utilizes a multi-scale residual convolution network to obtain feature layers with different scales, effectively extracts feature information of small targets, and simultaneously, the residual edge structure ensures that the network is not easy to degenerate during training; the SPP and PANet structures are used for strengthening the fusion effect of the feature pyramid on the features, semantic differences of different scales of feature pyramid fusion money are effectively reduced, and the special feature fusion method from top to bottom and from bottom to top also reduces the feature loss phenomenon caused when the bottom-layer features are fused with the shallow-layer features. The model uses the technology of Mosaic data enhancement in the training process, so that the diversity of training samples is enhanced, and the robustness of the model is improved. The method comprises the following specific steps:
step 1: shooting river images by using an unmanned aerial vehicle, screening out images containing or suspected of containing a sewage outlet by an expert, and initially establishing a data set required by a training model;
step 2: preprocessing an aerial photography river drain image, wherein the preprocessing comprises high-resolution aerial photography image blocking, carrying out rectangular frame labeling on a drain in an aerial photography image by an expert, and establishing a model training data set and a test set, and the method comprises the following specific steps:
step 21: the aerial image related to the invention is a high-resolution image, the resolution is assumed to be W multiplied by H, in order to improve the drain labeling accuracy and the model training efficiency, the aerial image with large resolution needs to be segmented, in the invention, the image is segmented into 6 parts, the width W is trisected, the height H is trisected, and the image is equally divided into 6 parts.
Step 22: on the basis of the step 2.1, a marking tool LabelImg is used for marking a drain target in the aerial photography river image, and the drain related to the invention comprises two types: blowdown dam and blow off pipe, in the mark, the expert utilizes this toolbox to select whole drain target to record drain target rectangle frame coordinate, set up the classification label of drain simultaneously, then, according to the format protocol of LabelImg instrument with the information of mark deposit to XML format file in, specifically include: target labeling category, target upper left corner X and Y coordinates, and target lower right corner X and Y coordinates.
And step 3: construction of the YOLOV4 model as shown in fig. 1, in this model, image features were extracted using a multi-scale residual feature extraction network CSPDarkNet53 and three feature layers were generated (Conv1, Conv2, Conv 3). And sending the three characteristic layers into special characteristic pyramid structure SPP and PANet for continuously extracting the characteristics.
Step 31: the YOLOV4 model backbone feature network uses CSPDarkNet53, whose overall structure is shown in fig. 2 and is formed by stacking 53 convolutional blocks (reblock _ body), each of which contains a convolutional layer, a BN layer, and a MISHRelu layer, and contains downsampling and multiple residual structures. After entering the feature extraction network, the image sequentially passes through the corresponding convolution blocks from top to bottom, finally, three feature layers with different scales and channels are formed, and the three feature layers are transmitted into the SPP structure and the PANet structure which are unique to the YOLOV4 feature pyramid.
Step 32: on the basis of step 31, the three feature layers extracted by the CSPDarkNet53 are processed by using an SPP structure and a PANet structure, wherein the SPP structure performs three darknet Conv2D _ BN _ leak convolutions on the minimum feature layer (Conv3) as shown in fig. 3, and then performs processing on the minimum feature layer and the maximum pooling of four different scales respectively, so that the receptive field can be greatly increased, and the most significant context features can be separated. Finally, it is superimposed into a new feature layer Conv 4.
Step 33: based on step 32, the three output layers (Conv2, Conv3, Conv4) are put into a PANET structure for repeated feature extraction, and the PANET structure is shown in FIG. 4. Unlike the traditional feature pyramid bottom-up extraction structure, PANet also requires implementation of top-down feature extraction. The three output layers are divided into three output layers of Conv1, Conv2 and Conv4 in size order, wherein the smallest output layer Conv4 is subjected to feature fusion with the output layer Conv2 to form a new feature layer Conv5, then the up-sampling is continued to be subjected to feature fusion with the feature layer Conv1 to form a feature layer Conv6, the Conv6 is subjected to down-sampling and is subjected to feature fusion with the previous Conv5 to form a feature layer Conv7, and Conv7 is subjected to down-sampling and is subjected to feature fusion with the feature layer Conv4 subjected to SPP to form a feature layer Conv 8. The feature layers Conv6, Conv7 and Conv8 are final output layers and are called YoloHead.
And 4, step 4: and training a YoloOV 4 model by using a training set, extracting the image by a feature extraction network and a special feature pyramid structure, generating three different YoloHead output layers, and detecting a sewage outlet target according to a target classifier and a target regressor of the YoloHead. Wherein each layer YoloHead corresponds to the detected parameters of the target area: target center point coordinates, target width, target length, and target classification. The final prediction result can be obtained after the parameters are correspondingly processed.
And 5: the method comprises the following steps of testing the performance of a YOLOV4 model by using a test set, then using the model for detection and identification of the aerial image river sewage outlet, and outputting an identification result, wherein the method comprises the following specific steps:
step 51: traditional target detection is mostly used for detection: pedestrians, vehicles, buildings, etc., are generally large in target and small in original image size, so that the detection difficulty is relatively low. Because the target dimension detected by the invention is small and the original image size is large (7952 x 5304), the size of the anchor frame of the model needs to be adjusted. The method abandons the traditional mode of manually setting the size of the anchor frame, selects and uses the K-means algorithm to realize the calculation of the anchor frame, and firstly randomly selects K box as an initial anchor frame; using the IOU metric, assign each box to the anchor box that is closest to it (i.e., the IOU is largest); calculating the average value of the width and the height of all the marking frames in each cluster, and updating the anchor frame; the above two steps are repeated until the anchor frame is no longer changed or the maximum number of iterations is reached. The size of the anchor frame obtained by the method is more consistent with the size of a drain target in actual detection, the detection precision of the model is improved, and target missing detection or false detection caused by improper anchor frame size setting is avoided.
Step 52: model training of the target classifier and the regressor in the feature extraction network CSPDarkNet 53. The parameters y _ pre and y _ true required by loss need to be acquired in the training process:
obtaining y _ pre: the prediction frame and its category corresponding to each grid point are obtained from the three YoloHead layers obtained in step 33, that is, after the three feature layers are respectively corresponding to the grids of which the picture is divided into different sizes, the positions, confidence degrees and its categories corresponding to the three prior frames on each grid point. For the output y1, y2, and y3, the offset of each grid point, including width and height, the confidence of the box, and the prediction probability of each kind, which are prediction parameters of the model, are collectively referred to as y _ pre.
Obtaining y _ true: the offset position, length, width and type on the three-size grid corresponding to each real frame in the real image need to be acquired. We need to obtain y _ true data corresponding to the y _ pre format as follows for the annotation data of the expert. Firstly, the real value of the frame is taken, the center and the width and the height of the frame are obtained, and the mode that the size of the output image is changed into proportion is removed. And secondly, establishing a y _ true and y _ true list of all 0, wherein the y _ true list comprises three feature layers, and the data dimensions are (m,13,13,3,7), (m,26,26,3,7), (m,52,52,3,7), respectively. And thirdly, processing each picture, comparing the width and height of a real frame in each picture with the width and height of a prior frame, calculating an IOU value, selecting the highest IOU to obtain the position of the feature layer and the grid point of the feature layer, and storing the content in the corresponding y _ true.
Step 53: and calculating the loss according to the y _ pre and the y _ true to complete the training of the model.
Firstly, for each graph, calculating IOU of all real boxes and prediction boxes, taking out the prior box with the largest IOU in each network point, and if the largest IOU is less than 0.5, keeping the prior box, wherein the step aims to balance negative samples. And secondly, calculating the loss on xy and wh, wherein the calculated loss is that the target actually exists, and comparing the calculated loss with the result obtained after the real frame is coded and the unprocessed prediction result to obtain the loss. Calculating the loss of the confidence coefficient, wherein the loss consists of two parts, the first part is that a target actually exists, and the value of the confidence coefficient in the prediction result is compared with 1; the second part is that there is virtually no target, and its maximum IOU value is found in the first step as compared to 0. The formula is as follows:
Figure BDA0002799555600000091
where ρ is2(b,bgt) Representing the euclidean distances of the center points of the predicted frame and the real frame, respectively. c represents the diagonal distance of the minimum closure area that can contain both the prediction box and the real box.
And the equations for α and v are as follows:
Figure BDA0002799555600000101
Figure BDA0002799555600000102
the final LOSS is:
Figure BDA0002799555600000103
step 6: and testing the performance of the YOLOV4 model by using a test set, then using the model for aerial image river sewage outlet detection and identification, and outputting an identification result.
Drain aerial image training set and test set statistical table related in the embodiment
Table 1 river sewage outfall aerial photography image data set statistical table
Data set Sewage discharge dam Blow-off pipe
Training set 7650 3520
Test set 2434 1165
Total up to 10084 4685
In order to evaluate the target detection result of the sewage outlet, the invention adopts Precision (Precision) and recall rate (Rcall) indexes, and the calculation formula is as follows:
Figure BDA0002799555600000104
Figure BDA0002799555600000105
wherein TP (true Positive) represents the number of the sewage outlets which are correctly detected, FP (false positive) represents the number of the sewage outlets which are wrongly detected, and FN (false negative) represents the number of the sewage outlets which are not detected.
According to the precision and the recall rate, a P-R curve can be obtained, the lower area of the P-R curve is AP (Average precision), each type of sewage outlet correspondingly obtains an AP value, further, the Average precision mAP (mean Average precision) of target detection of the two types of sewage outlets can be calculated, and the precision of the detection algorithm is measured.
Fig. 5 is a diagram showing the detection results of the sewage outlets of rivers in different areas, and finally the method of the present invention is compared with the original fast RCNN method, and the detection results of the original fast RCNN backbone network using RestNet50 are shown in table 2.
TABLE 2 comparison of the results of the test by the method of the present invention and the conventional fast RCNN method
Method of producing a composite material AP (blowdown dam) AP (blow off pipe) mAP
Faster RCNN 68.3% 55.2% 61.8%
The method of the invention 82.8% 70.2% 76.5%
As can be seen from the table, compared with the traditional fast RCNN, the detection precision of the method provided by the invention on the sewage discharge dam and the sewage discharge pipe is respectively improved by 14.5 percent and 15 percent, the average precision is improved by 14.7 percent, and the method provided by the invention has certain advantages.
Example two
The object of this embodiment is to provide a computing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the processor implements the steps of the method in the first embodiment.
EXAMPLE III
An object of the present embodiment is to provide a computer-readable storage medium.
A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method of the first embodiment.
Example four
The purpose of this embodiment is to provide a river drain detecting system based on high definition image, includes:
the model construction module is used for constructing a model, extracting image features by utilizing a multi-scale residual error feature extraction network in the model so as to fuse shallow features and deep features and generate a plurality of feature layers, and sending the feature layers into a feature pyramid structure for feature extraction continuously;
and the river drain outlet identification module generates a plurality of different output layers after the high-definition images are extracted through the feature extraction network and the feature pyramid structure, detects a drain outlet target through the target classifier and the target regressor and outputs an identification result.
The steps involved in the apparatuses of the above second, third and fourth embodiments correspond to the first embodiment of the method, and the detailed description thereof can be found in the relevant description of the first embodiment. The term "computer-readable storage medium" should be taken to include a single medium or multiple media containing one or more sets of instructions; it should also be understood to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by a processor and that cause the processor to perform any of the methods of the present disclosure.
Those skilled in the art will appreciate that the modules or steps of the present disclosure described above can be implemented using general purpose computer means, or alternatively, they can be implemented using program code executable by computing means, whereby the modules or steps may be stored in memory means for execution by the computing means, or separately fabricated into individual integrated circuit modules, or multiple modules or steps thereof may be fabricated into a single integrated circuit module. The present disclosure is not limited to any specific combination of hardware and software.
The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure, and various modifications and changes may be made to the present disclosure by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Although the present disclosure has been described with reference to specific embodiments, it should be understood that the scope of the present disclosure is not limited thereto, and those skilled in the art will appreciate that various modifications and changes can be made without departing from the spirit and scope of the present disclosure.

Claims (10)

1. A river sewage outlet detection method based on high-definition images is characterized by comprising the following steps:
constructing a model, extracting image features by using a multi-scale residual error feature extraction network in the model to fuse shallow features and deep features and generate a plurality of feature layers, and sending the plurality of feature layers into a feature pyramid structure for feature extraction;
the high-definition image is extracted through a feature extraction network and a feature pyramid structure to generate a plurality of different output layers, a drain target is detected through a target classifier and a target regressor, and a recognition result is output.
2. The method as claimed in claim 1, wherein before the model is built, the high-definition river image is obtained, the data set required by the training model is built, the image blocks in the data set and the sewage outlet in the image are marked with rectangular frames, the model training data set and the model testing set are built, and the built model is trained and tested respectively.
3. The method for detecting the river sewage outlet based on the high-definition image as claimed in claim 1, wherein the high-definition river image is an image obtained by aerial photography by an unmanned aerial vehicle, and a target detection marking tool is adopted to mark the sewage outlet in the aerial river image;
the drain includes two types: in the labeling process, the tool frame is used for selecting the whole drain target, the coordinate of a rectangular frame of the drain target is recorded, and a category label of the drain is set;
storing the labeled information into a file according to a format protocol of a tool, which specifically comprises the following steps: target labeling category, target upper left corner X and Y coordinates, and target lower right corner X and Y coordinates.
4. The method for detecting the river sewage outlet based on the high-definition image as claimed in claim 1, wherein the image enters the feature extraction network and then sequentially passes through the corresponding convolution blocks from top to bottom, finally, three feature layers with different scales and channels are formed, and the three feature layers are further transmitted into an SPP structure and a PaNet structure of a network feature pyramid.
5. The method as claimed in claim 4, wherein the SPP structure and the PANet structure are used to process three feature layers extracted from the pyramid of network features, wherein the SPP structure performs three convolutions to the smallest feature layer, and then performs processing to the largest pools of four different scales, so that the receptive field can be greatly increased and the most significant contextual features can be separated.
6. The method for detecting the river sewage outlet based on the high-definition image as claimed in claim 5, wherein three output layers are placed into a PANET structure for repeated feature extraction, the PANET structure realizes feature extraction from top to bottom, and the three output layers are sequentially divided into A, B, C output layers according to the number of channels.
7. The method as claimed in claim 6, wherein the smallest output layer C is subjected to upsampling and fused with the output layer B to form a new feature layer D, then the upsampling and the feature layer A are fused to form a feature layer E, the upsampling and the feature layer A are further subjected to feature fusion to form a feature layer F, the feature layer F and the feature layer C subjected to SPP are fused to form a feature layer G, and the feature layer E, F, G is the final output layer.
8. The utility model provides a river drain detecting system based on high definition image which characterized by includes:
the model construction module is used for constructing a model, extracting image features by utilizing a multi-scale residual error feature extraction network in the model so as to fuse shallow features and deep features and generate a plurality of feature layers, and sending the feature layers into a feature pyramid structure for feature extraction continuously;
and the river drain outlet identification module generates a plurality of different output layers after the high-definition images are extracted through the feature extraction network and the feature pyramid structure, detects a drain outlet target through the target classifier and the target regressor and outputs an identification result.
9. A computing device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program performs the steps of the method of any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, is adapted to carry out the steps of the method according to any one of the preceding claims 1 to 7.
CN202011344676.4A 2020-11-26 2020-11-26 River sewage outlet detection method and system based on high-definition images Pending CN112308040A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011344676.4A CN112308040A (en) 2020-11-26 2020-11-26 River sewage outlet detection method and system based on high-definition images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011344676.4A CN112308040A (en) 2020-11-26 2020-11-26 River sewage outlet detection method and system based on high-definition images

Publications (1)

Publication Number Publication Date
CN112308040A true CN112308040A (en) 2021-02-02

Family

ID=74487780

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011344676.4A Pending CN112308040A (en) 2020-11-26 2020-11-26 River sewage outlet detection method and system based on high-definition images

Country Status (1)

Country Link
CN (1) CN112308040A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113076871A (en) * 2021-04-01 2021-07-06 华南理工大学 Fish shoal automatic detection method based on target shielding compensation
CN113297996A (en) * 2021-05-31 2021-08-24 贵州电网有限责任公司 Unmanned aerial vehicle aerial photographing insulator target detection method based on YoloV3
CN113343923A (en) * 2021-07-01 2021-09-03 江苏舆图信息科技有限公司 Real-time river drainage port drainage state identification method based on video images
CN114756425A (en) * 2022-03-08 2022-07-15 深圳集智数字科技有限公司 Intelligent monitoring method and device, electronic equipment and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109344821A (en) * 2018-08-30 2019-02-15 西安电子科技大学 Small target detecting method based on Fusion Features and deep learning
CN110766098A (en) * 2019-11-07 2020-02-07 中国石油大学(华东) Traffic scene small target detection method based on improved YOLOv3
CN111259827A (en) * 2020-01-19 2020-06-09 河海大学 Automatic detection method and device for water surface floating objects for urban river supervision

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109344821A (en) * 2018-08-30 2019-02-15 西安电子科技大学 Small target detecting method based on Fusion Features and deep learning
CN110766098A (en) * 2019-11-07 2020-02-07 中国石油大学(华东) Traffic scene small target detection method based on improved YOLOv3
CN111259827A (en) * 2020-01-19 2020-06-09 河海大学 Automatic detection method and device for water surface floating objects for urban river supervision

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
AI柠檬博客: ""YOLOV4论文详细解读"", 《微信公众号》 *
ALEXEY BOCHKOVSKIY等: ""YOLOv4: Optimal Speed and Accuracy of Object Detection"", 《ARXIV》 *

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113076871A (en) * 2021-04-01 2021-07-06 华南理工大学 Fish shoal automatic detection method based on target shielding compensation
CN113076871B (en) * 2021-04-01 2022-10-21 华南理工大学 Fish shoal automatic detection method based on target shielding compensation
CN113297996A (en) * 2021-05-31 2021-08-24 贵州电网有限责任公司 Unmanned aerial vehicle aerial photographing insulator target detection method based on YoloV3
CN113343923A (en) * 2021-07-01 2021-09-03 江苏舆图信息科技有限公司 Real-time river drainage port drainage state identification method based on video images
CN114756425A (en) * 2022-03-08 2022-07-15 深圳集智数字科技有限公司 Intelligent monitoring method and device, electronic equipment and computer readable storage medium

Similar Documents

Publication Publication Date Title
CN110188705B (en) Remote traffic sign detection and identification method suitable for vehicle-mounted system
CN112308040A (en) River sewage outlet detection method and system based on high-definition images
CN111179217A (en) Attention mechanism-based remote sensing image multi-scale target detection method
CN103530600B (en) Licence plate recognition method under complex illumination and system
CN110598609A (en) Weak supervision target detection method based on significance guidance
CN114841972A (en) Power transmission line defect identification method based on saliency map and semantic embedded feature pyramid
CN111797829A (en) License plate detection method and device, electronic equipment and storage medium
CN113920107A (en) Insulator damage detection method based on improved yolov5 algorithm
CN113487610B (en) Herpes image recognition method and device, computer equipment and storage medium
CN113496480A (en) Method for detecting weld image defects
CN111598854A (en) Complex texture small defect segmentation method based on rich robust convolution characteristic model
CN110751195A (en) Fine-grained image classification method based on improved YOLOv3
CN114049572A (en) Detection method for identifying small target
CN103353941A (en) Natural marker registration method based on viewpoint classification
CN114332921A (en) Pedestrian detection method based on improved clustering algorithm for Faster R-CNN network
CN116168240A (en) Arbitrary-direction dense ship target detection method based on attention enhancement
CN113158954B (en) Automatic detection method for zebra crossing region based on AI technology in traffic offsite
CN116912184B (en) Weak supervision depth restoration image tampering positioning method and system based on tampering area separation and area constraint loss
CN117593548A (en) Visual SLAM method for removing dynamic feature points based on weighted attention mechanism
CN112287895A (en) Model construction method, recognition method and system for river drain outlet detection
CN111179278B (en) Image detection method, device, equipment and storage medium
CN114882490B (en) Unlimited scene license plate detection and classification method based on point-guided positioning
CN114494893B (en) Remote sensing image feature extraction method based on semantic reuse context feature pyramid
CN116678893A (en) Hot rolled strip steel surface defect detection method based on deep learning
CN110889418A (en) Gas contour identification method

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination