CN113177929A - Sea surface oil spill detection method based on full convolution network, system and application thereof - Google Patents

Sea surface oil spill detection method based on full convolution network, system and application thereof Download PDF

Info

Publication number
CN113177929A
CN113177929A CN202110540801.7A CN202110540801A CN113177929A CN 113177929 A CN113177929 A CN 113177929A CN 202110540801 A CN202110540801 A CN 202110540801A CN 113177929 A CN113177929 A CN 113177929A
Authority
CN
China
Prior art keywords
sea surface
image
information
oil spill
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110540801.7A
Other languages
Chinese (zh)
Other versions
CN113177929B (en
Inventor
梁鸿
杨莹
巩亚明
魏学成
张千
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China University of Petroleum East China
Original Assignee
China University of Petroleum East China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China University of Petroleum East China filed Critical China University of Petroleum East China
Priority to CN202110540801.7A priority Critical patent/CN113177929B/en
Publication of CN113177929A publication Critical patent/CN113177929A/en
Application granted granted Critical
Publication of CN113177929B publication Critical patent/CN113177929B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20016—Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A20/00—Water conservation; Efficient water supply; Efficient water use
    • Y02A20/20—Controlling water pollution; Waste water treatment
    • Y02A20/204—Keeping clear the surface of open water from oil spills

Abstract

The invention discloses a sea surface oil spill detection method and a system, comprising the following steps: inputting the sea surface oil spill image serving as a data set into an initial sea surface oil spill detection network, extracting multi-scale characteristic information of the image, fusing the multi-scale characteristic information of the image through a Transformer module to obtain a Transformed characteristic pyramid, and predicting information of each pixel of a head network image of the characteristic pyramid; calculating the error between the predicted information and the label by using a loss function; and optimizing network parameters by adopting a back propagation algorithm until the error reaches an expected value, obtaining a sea surface oil spill detection network, and outputting detection information. And under the condition that the environmental picture has an oil film, judging that the oil spilling phenomenon exists on the sea surface site, and sending out an early warning signal. The method extracts the multi-scale features of the image to obtain the Transformed feature pyramid, solves the problem of scale change, and can quickly and accurately identify the oil film in the field photo.

Description

Sea surface oil spill detection method based on full convolution network, system and application thereof
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a sea surface oil spill detection method based on a full convolution network, a system and application thereof.
Background
The information in this background section is only for enhancement of understanding of the general background of the invention and is not necessarily to be construed as an admission or any form of suggestion that this information forms the prior art that is already known to a person of ordinary skill in the art.
Among the global sea-surface marine pollution, the most common is oil spill pollution. Sea surface spills destroy the marine ecosystem and affect human health and sea surface activities. The key research direction is to detect oil spillage timely and accurately and send out an alarm at the first time. Therefore, the fast and accurate detection of sea surface oil spill is the most important task at present, and the complex scene of the ocean brings great challenges to oil spill detection.
The oil film has different sizes, the oil spilling image has a great deal of noise, the contrast is low, the edge is fuzzy and the like, and the traditional common detection method mainly comprises the following steps: (1) segmenting based on a threshold value; (2) based on the edge information; (3) based on semantic segmentation. The method is simple and small in calculated amount, but features are difficult to extract automatically, the segmentation difficulty of oil spilling and the seawater background is high, the determination of an oil spilling region in the whole image is difficult, and the segmentation accuracy is low due to the influence of various factors.
In recent decades, two-stage and one-stage target detection algorithms based on Convolutional Neural Networks (CNN) have been greatly improved in terms of accuracy and speed, respectively.
The inventors have found that most detection algorithms based on convolutional neural networks are at the cost of loss of accuracy or speed. In practical application, due to the influence of environment and equipment, an appropriate detection model needs to be established from various aspects, so that the practical problem is solved. The Full Convolution Network (FCN) performs pixel-level prediction, is mainly used for semantic segmentation, can automatically extract feature information and retain spatial information in an original input image, but does not fully consider the relationship between pixels.
Disclosure of Invention
In order to solve the problems of high difficulty in segmenting the oil spill and seawater backgrounds, low segmentation accuracy and incapability of considering detection precision and speed in the prior art, the invention provides a sea surface oil spill detection method based on a full convolution network, which improves the detection precision of an oil film under the conditions of reducing memory and complicated calculation and simultaneously keeps real-time detection speed. The method is easy to carry out end-to-end training, and can quickly and accurately detect the oil film and give an alarm in time.
Specifically, the invention is realized by the following technical scheme:
the invention provides a sea surface oil spill detection method based on a full convolution network, which comprises the following steps:
inputting an image to be detected into a trained sea surface oil spill detection network, and outputting detection information of the image to be detected;
and the evaluation system compares the detection information with the characteristic value of the database, judges whether the sea surface has oil spill, judges that the oil spill occurs if an oil film is detected, and sends out a warning.
In a second aspect of the present invention, a system for detecting sea surface oil spill based on a full convolution network is provided, which includes: the backbone network is used for extracting the multi-scale characteristics of the oil spilling image;
the Transformer module uses the self-attention layer to replace the convolution layer, and dynamically changes the transformation mode and the feature fusion mode of the features through a Gate block so as to realize better feature fusion;
obtaining a Transformed characteristic pyramid by the multi-scale characteristic through a Transformer module;
the head network of the specific detection task obtains the final detection information.
The third aspect of the invention provides a sea surface oil spill detection method based on a full convolution network and/or an application of a sea surface oil spill detection system based on a full convolution network in the field of sea surface oil spill detection.
One or more embodiments of the present invention have the following advantageous effects:
1) the sea surface oil spill detection method and system based on the full convolution network have the advantages of being simple in structure, easy to train, small in calculated amount, convenient to optimize and the like.
2) Inputting the oil spilling image into the backbone network, and extracting the multi-scale features of the image;
3) and a Transformer module is inserted behind the backbone network, so that the characteristic global information is increased, and the detection of a large-area oil film is facilitated. The obtained transformated characteristic pyramid solves the problem of scale change, and simultaneously obtains multi-scale context information to establish the relation between pixels, thereby being beneficial to the detection of a small-area oil film;
4) the feature pyramid obtains the prediction information of each pixel of the image through a head network, so that the use of anchors is eliminated, and the memory and the complex calculation are reduced;
5) and removing redundant boundary boxes through non-maximum suppression (NMS) and outputting a final detection result.
6) And the evaluation system compares the detection information with the characteristic value of the database, judges whether the sea surface has oil spill, judges that the sea surface has oil spill and sends out a warning if the oil film is detected, and finally generates and stores a detection report.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. Embodiments of the invention are described in detail below with reference to the attached drawing figures, wherein:
fig. 1 is a flowchart of a method and a system for detecting sea surface oil spill based on a full convolution network according to embodiment 1 of the present invention;
fig. 2 is a flowchart of a method and a system for detecting sea surface oil spill based on a full convolution network according to embodiment 2 of the present invention;
FIG. 3 is a schematic diagram of an automatic identification and assessment system according to embodiment 2 of the present invention;
fig. 4 is a schematic network structure diagram of a detection network according to embodiment 3 of the present invention;
fig. 5 is a flowchart of a method and a system for detecting sea surface oil spill based on a full convolutional network according to embodiment 4 of the present invention.
Detailed Description
The invention will be further illustrated with reference to the following specific examples. It should be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. The experimental procedures, in which specific conditions are not noted in the following examples, are generally carried out according to conventional conditions or according to conditions recommended by the manufacturers.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present disclosure. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
In order to solve the problems of high difficulty in segmenting the oil spill and seawater backgrounds, low segmentation accuracy and incapability of considering detection precision and speed in the prior art, the invention provides a sea surface oil spill detection method based on a full convolution network, which improves the detection precision of an oil film under the conditions of reducing memory and complicated calculation and simultaneously keeps real-time detection speed. The method is easy to carry out end-to-end training, and can quickly and accurately detect the oil film and give an alarm in time.
Specifically, the invention is realized by the following technical scheme:
the invention provides a sea surface oil spill detection method based on a full convolution network, which comprises the following steps:
inputting an image to be detected into a trained sea surface oil spill detection network, and outputting detection information of the image to be detected;
and the evaluation system compares the detection information with the characteristic value of the database, judges whether the sea surface has oil spill, judges that the oil spill occurs if an oil film is detected, and sends out a warning.
The detection information of the image to be detected refers to an oil film characteristic value identified by a sea surface oil spill detection network, including but not limited to information such as size, color, thickness, refractive index, frequency spectrum and the like, and aims to compare with the characteristic value of the database and judge whether oil spill exists on the sea surface.
In some embodiments, an image of sea surface oil spill is captured with an unmanned aerial vehicle;
unmanned aerial vehicle controls can not receive the restriction of time, distance, marine weather condition etc. and realize quick, accurate, the real-time marine information that obtains, helps the very first time to discover marine oil spilling.
In some embodiments, the training method of the trained sea surface oil spill detection network comprises:
acquiring a sea surface oil spill image, preprocessing the sea surface oil spill image to obtain an image data set, and extracting a spectrogram of an oil film area;
combining an image data set and an oil film spectrogram to serve as a training sample set, and labeling the training sample set to obtain label information of the training sample set;
inputting the label information of the training sample set into a pre-constructed initial sea surface oil spill detection network, extracting a plurality of characteristic information, and combining the plurality of characteristic information to obtain fused characteristic information;
acquiring prediction information of each pixel of the oil spilling image according to the fusion characteristic information;
and optimizing to obtain a trained sea surface oil spill detection network, inputting an image to be detected into the sea surface oil spill detection network, and outputting detection information of the image to be detected.
Compared with the method that a single parameter or a characteristic value is input into the sea surface oil spill detection network, the method provided by the invention inputs the training sample set formed by combining the image data set and the oil film spectrogram into the sea surface oil spill detection network for training, and the obtained trained sea surface oil spill detection network can better identify the sea surface oil spill image at multiple angles, so that the accuracy is improved.
The plurality of feature information includes, but is not limited to, position information, size information, pixel information.
In order to accurately locate and identify the marine spill area, some embodiments of the invention further include a step of pre-processing the spill image, including:
(1) the method for limiting the contrast self-adaptive histogram equalization solves the problem of non-ideal contrast and simultaneously inhibits the enhancement of noise.
(2) The high-frequency reinforced filtering method carries out edge sharpening enhancement on the image and pulls open the gray distribution of the image to a certain extent. The method aims to make the oil spilling image clearer and enhance the contrast ratio of seawater and an oil film so as to improve the segmentation precision.
The inventor researches and discovers that if only the oil spill image is input into the trained sea surface oil spill detection network, the problems of high identification difficulty, long identification time, high noise and low accuracy can exist, and therefore, an oil film region spectrogram is selected in some embodiments. And the method for extracting the spectrogram of the oil film region comprises the steps of inputting the oil spill image into a full convolution network to carry out two-classification segmentation on the oil film region and a background image, and then extracting the spectrogram of the oil film region.
In the simulation calculation or prediction process, due to various objective or subjective reasons, errors are easy to occur, and the detection efficiency and precision are affected, and in order to reduce the adverse effect as much as possible, in one or more embodiments of the present invention, the process of calculating the errors is further included after obtaining the prediction information: and calculating the error of the prediction information and the label information by using a loss function.
The calculation error is only used for displaying the deviation degree of the measured information and the label information in a single direction, so that the system is optimized on the basis, and the error is reduced. The optimization method comprises the following steps: and optimizing the parameters of the initial sea surface oil spill detection network by adopting a back propagation method until the error reaches an expected value.
In some embodiments, the spectrogram of the oil film is obtained by performing two-dimensional fourier transform on the oil film region, and is mainly used for increasing the diversity of oil film characteristics to improve the detection accuracy.
In some embodiments, the prediction information for each pixel comprises category information and regression information.
In some embodiments, the class information of each pixel is a three-dimensional matrix of H × W × C; c is the category number of the data set; the regression information of each pixel is a three-dimensional matrix H multiplied by W multiplied by 4.
And predicting each pixel of the image to obtain the category and the regression bounding box of each pixel, so that the positioning of the target is improved, and the problem of unbalance of positive and negative samples is solved. The artificial preset anchor frame is removed, and the calculation complexity and the parameter number are reduced.
Further preferably, the loss function is:
Figure BDA0003071475100000051
px,y，tx,ya 2-dimensional vector of classification labels and a 4-dimensional vector of bounding box coordinates for the location (x, y) of each pixel of the sea surface oil spill detection network prediction image, respectively. N is a radical ofposIs the number of pixels of the whole image.
Figure BDA0003071475100000052
Is the sum of the classification losses for each pixel, where Lcls,tcx,yClass labels for the focus loss and true value box for each position (x, y), respectively. p { tcx,y> 0 is the probability of each pixel being a positive sample, Lreg,tx,yA regression target is trained for the IOU penalty and each pixel location (x, y). L isCIs a computational loss constraint. L iscls,LregThese two terms and LCThe balancing is performed by a balancing parameter lambda.
Further preferably, the p { tc }x,y> 0}, if tcx,yAnd > 0, p equals 1, otherwise p equals 0.
Further, the method can be used for preparing a novel materialPreferably, said LCIs defined as:
LC＝((CR-Ctarget)/(Cmax-Cmin))2 (2)
Ctarget＝Cmin+α·(Cmax-Cmin) (3)
wherein C ismax,CminRepresenting the calculated quantities of the highest and lowest configurations, respectively, CRRepresenting the actual calculated quantity, CtargetControlled by the hyper-parameter alpha.
Preferably, the step of generating and saving the detection report is further included after the alarm is given.
The sea surface oil spill detection method based on the full convolution network provided by the invention is characterized in that an oil spill image collected by an unmanned aerial vehicle is utilized, the oil spill image is labeled and preprocessed to serve as an image data set, the full convolution network is adopted to carry out two-class segmentation on an oil film region and a background image on the oil spill image, and then a two-dimensional Fourier transform is used for extracting a frequency spectrogram of the oil film region. And collecting the spectrogram and the image data of the oil film to obtain a training sample set. And inputting the training sample set into an initial sea surface oil spill detection network. Firstly, a backbone network extracts multi-scale features of an image, a transform module is inserted behind the backbone network to realize a better multi-scale feature fusion mode, local and global context information is captured to establish the relation between targets, features of different scales and spatial positions are obtained, and a Transformed feature pyramid is obtained. And then, the characteristic pyramid obtains the prediction information of each pixel of the oil spill image through a head network, calculates the error between the prediction information of each pixel of the oil spill image and the label of the oil spill image by using a loss function, and optimizes the parameters of the initial sea surface oil spill detection network by adopting a back propagation algorithm so that the error between the predicted value output by the network and the label value reaches the minimum expected value, thereby completing the training of the initial sea surface oil spill detection network and obtaining the sea surface oil spill detection network. And inputting the oil spilling image to be detected into the sea surface oil spilling detection network, and outputting the detection information of the oil spilling image to be detected.
The method provided by the invention provides a detection framework with self-adaptive fusion of pixel-level and multi-scale features, removes the use of anchors, reduces the complex calculation related to the memory and the anchors, and outputs the category information and regression information of each pixel in the oil spill image.
A Transformer module is innovatively introduced into a network architecture, the global information of the characteristics is increased, the generated Transformed characteristic pyramid is fused with the multi-scale characteristics to acquire the multi-scale context information to establish the relation between pixels, the oil film detection precision is greatly improved, meanwhile, the real-time detection is kept, the method can be applied to sea surface oil spill detection, the oil film is successfully detected, and therefore the response can be timely made to the sea surface oil spill.
The parameter type information is oil spilling or oil non-spilling;
the regression information is a four-dimensional vector bb*＝(l*,t*,r*,b*)，
Figure BDA0003071475100000061
Is the regression target for each pixel (x, y);
the global information is the relationship, such as similarity, established between each pixel in the feature map and all the other pixels;
the multi-scale features are feature maps with different scales generated by a backbone network, the features with large scales have less general semantic information, mainly comprise texture, edge and other information, but the spatial position information is richer, the features with small scales have more general semantic information, mainly comprise complete feature information, but have less spatial position information;
the multi-scale context information is the link (which may be considered as pixel-to-pixel herein) between the features of the multi-scale feature map.
Applying the concept of Transformer to the feature map, establishing the relation between pixels by using a self-attention mechanism (calculating the covariance between each pixel to obtain the similarity between the pixels);
the prediction information of each pixel of the image data is acquired by using a full convolution method to realize the classification and positioning of the target. The frequency spectrum information is used for increasing the diversity of the features and improving the detection precision.
The contrast ratio of the oil spilling region and the non-oil spilling region is weak, and the spectrogram of the oil spilling region is used for expanding the training data set to increase the diversity of characteristics, so that the oil film can be identified more accurately.
In a second aspect of the present invention, a system for detecting sea surface oil spill based on a full convolution network is provided, which includes: the backbone network is used for extracting the multi-scale characteristics of the oil spilling image;
the Transformer module uses the self-attention layer to replace the convolution layer, and dynamically changes the transformation mode and the feature fusion mode of the features through a Gate block so as to realize better feature fusion;
obtaining a Transformed characteristic pyramid by the multi-scale characteristic through a Transformer module;
the head network of the specific detection task obtains the final detection information.
In some embodiments, the backbone network is ResNet-50; the multi-scale feature information is obtained by performing 8-time, 16-time and 32-time down-sampling on an original image with 800 × 1024 pixels to obtain feature information of 100 × 128, 50 × 64 and 25 × 32 pixels respectively in three-layer scale.
In some embodiments, the Transformer module comprises a single-scale transform, a cross-scale transform, and a Gate block;
further preferably, the head network classification branch and regression branch are composed; the regression branch and the classification branch are parallel and share parameters;
further preferably, the detection information is related information of a class of an object in the oil spilling image and a detection bounding box;
the Transformer module is mainly used for establishing the relation between pixels, acquiring the global context information of the image, and combining the global context information with the local information acquired by the full convolution method to improve the detection precision of the oil films in the large and small areas.
The specific operations in the Transformer module are as follows: the Gate block is a Gate function, single-scale transformation or cross-scale transformation can be dynamically selected, multi-scale features are fused, an optimal feature fusion mode is obtained, and a feature pyramid with richer multi-scale features, context information and spatial position information is generated. And finally, predicting each pixel of the feature map of each layer of the feature pyramid. Prediction is achieved by classification and regression branches, and mainly comprises 4 convolution layers of 3 × 3.
The sea surface oil spill detection system based on the full convolution network provided by the invention has the advantages that a backbone network is ResNet-50; the Transformer module mainly comprises single-scale transformation, cross-scale transformation and Gate block; the head network consists of parallel classification branches and regression branches; non-maxima suppression is used primarily to remove redundant bounding boxes, thereby producing the final detected bounding box. The method can improve the detection precision of the oil film, simultaneously keep real-time detection and send out an alarm in time by removing the setting of the anchor and the parameters and complex calculation related to the anchor. The method has the characteristics of simple structure, easy training, less calculation amount and convenient optimization.
Further preferably, the detection report includes information on the size of the oil film, the severity of the oil spill, and the location.
In some embodiments, the system further comprises:
the device comprises a sea surface oil spilling image acquisition module, a preprocessing module, a frequency spectrum acquisition module and a sample label processing module;
the sea surface oil spill image acquisition module is used for acquiring a sea surface oil spill image;
the preprocessing module aims to acquire an image data set, limit contrast self-adaptive histogram equalization, suppress noise, high frequency and filter;
the frequency spectrum acquisition module is used for inputting the oil spill image into the full convolution network to carry out two-class segmentation on an oil film area and a background image and then extracting a frequency spectrum image of the oil film area;
the sample label processing module is used for labeling the image data set to obtain label information of the image data set.
The sea surface oil spill detection method and system based on the full convolution network have the advantages of being simple in structure, easy to train, small in calculated amount, convenient to optimize and the like.
The third aspect of the invention provides a sea surface oil spill detection method based on a full convolution network and/or an application of a sea surface oil spill detection system based on a full convolution network in the field of sea surface oil spill detection.
The present invention is described in further detail below with reference to specific examples, which are intended to be illustrative of the invention and not limiting.
Example 1
As shown in fig. 1, a flow chart of a method for detecting sea surface oil spill based on a full convolution network is disclosed, and the method specifically operates as follows:
1) firstly, acquiring a sea surface oil spill image, and carrying out preprocessing operation on the oil spill image to obtain an image data set. And inputting the oil spilling image into a full convolution network to carry out two-classification segmentation of an oil film region and a background image, and then extracting a frequency spectrogram of the oil film region.
2) Merging the image data set after being labeled and the spectrogram of the oil film to be used as a training sample set; labeling the training sample set to obtain label information of the training sample set;
inputting the label information of the training sample set into a pre-constructed initial sea surface oil spill detection network;
3) and optimizing the parameters of the initial sea surface oil spill detection network by adopting a back propagation method until the error reaches an expected value, so as to obtain the trained sea surface oil spill detection network.
4) And the evaluation system compares the detection information with the characteristic value of the database, judges whether the sea surface has oil spill, judges that the sea surface has oil spill and sends out a warning if the oil film is detected, and finally generates and stores a detection report.
5) And inputting the image to be detected into the sea surface oil spill detection network, and outputting the detection information of the image to be detected.
Example 2
As shown in fig. 2, it is a flowchart of a sea surface oil spill detection method based on a full convolution network:
step S01: simulating a sea surface oil spilling field in a laboratory, shooting an oil spilling image as a data training set, preprocessing the oil spilling image, then segmenting to obtain an oil film area, and using a frequency spectrogram obtained through two-dimensional Fourier transform and an image subjected to labeling as a training sample set.
The pretreatment operation mainly comprises the following steps: (1) the method for limiting the contrast self-adaptive histogram equalization solves the problem of non-ideal contrast and simultaneously inhibits the enhancement of noise. (2) The high-frequency reinforced filtering method carries out edge sharpening enhancement on the image and pulls open the gray distribution of the image to a certain extent. The method mainly makes the oil spilling image clearer, and enhances the contrast ratio of seawater and an oil film so as to improve the segmentation precision.
Step S02: inputting the training sample set into a pre-constructed initial sea surface oil spill detection network, extracting a plurality of characteristic information and outputting prediction information;
step S03: calculating an error of the prediction information and the data set label information by a loss function.
The errors mainly comprise the sum of classification errors, the sum of regression errors and the loss of calculated amount of each pixel of the oil spilling image.
Step S04: and optimizing parameters of the initial sea surface oil spill detection network by adopting a back propagation algorithm so that the error between a predicted value output by the network and a label value reaches a minimum expected value, finishing the training of the initial sea surface oil spill detection network and obtaining the sea surface oil spill detection network.
Step S05: and inputting the oil spilling image to be detected into the sea surface oil spilling detection network, and outputting the detection information of the oil spilling image to be detected.
The embodiment provides a sea surface oil spill detection method based on a full convolution network, a set of pixel-level detection network architecture is developed, the use of an anchor is eliminated, the quantity of parameters and complex calculation related to the anchor are reduced, and a Transformer module is innovatively introduced into the network architecture and used for increasing the global information of features; and (3) solving the problem of scale change by using a Transformed feature pyramid, and simultaneously acquiring multi-scale context information to establish the relation between pixels. The invention can better detect the oil film and is easy to deploy and implement. As shown in fig. 2, based on the above embodiment, in this embodiment, the image of oil spill to be detected is input to the sea surface oil spill detection network, and the sea surface oil spill detection information of the image of oil spill to be detected is output. As shown in fig. 3, an automatic identification and evaluation system is established, whether an oil film exists in the oil spill image can be judged according to the sea surface oil spill detection information, an early warning signal is generated according to the judgment result, and a detection report is generated and stored.
Example 3
As shown in fig. 4, the present embodiment provides a sea surface oil spill detection system based on a full convolution network, which is an end-to-end one-stage anchor-free detection network and includes a backbone network, a Transformer module, a Transformed feature pyramid, and a head network.
The detection network is based on full convolution for predicting the class information and regression information for each pixel in the image.
The backbone network adopts a ResNet-50 network and is used for extracting and storing the characteristic information of the oil spilling image.
The Transformer module mainly comprises single-scale transformation, cross-scale transformation and Gate block, and increases global information of features to facilitate detection of large-area oil films. (ii) a
The transformated characteristic pyramid solves the problem of scale change, and simultaneously obtains multi-scale context information to establish the relation between pixels, thereby being beneficial to the detection of a small-area oil film;
the head network consists of parallel classification branches and regression branches.
Example 4
Referring to fig. 5, fig. 5 is a flowchart of another sea surface oil spill detection method based on the full convolution network according to this embodiment; the specific operation steps are as follows:
step S11: training and optimizing the initial sea surface oil spill detection network by using a sea surface data set adopted by the unmanned aerial vehicle to obtain a sea surface oil spill detection network, and deploying the sea surface oil spill detection network into an automatic identification and evaluation system;
step S12: inputting an oil spill image to be detected into a sea surface oil spill detection network which completes training in advance, and outputting detection information of the oil spill image;
step S13: according to the detection information of the image, a judgment result is made whether an oil film exists on the sea surface site;
step S14: according to the judgment result, if the oil film exists on the sea surface site, judging that the oil spilling phenomenon exists in the sea surface site constructor, and obtaining the position of the oil film according to the position shot by the unmanned aerial vehicle;
step S15: and producing an early warning signal according to the position of the oil film, sending the early warning signal to a monitoring person, and converting detection information into a report for storage.
In this embodiment, the sea surface oil spill detection method based on the full convolution network is applied to a sea surface field oil spill detection system, and after the detection information of the oil spill image is acquired by using the sea surface oil spill detection network, the sea surface oil spill is judged and pre-warned according to the detection information.
The embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same or similar parts among the embodiments are referred to each other.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that changes may be made in the embodiments and/or equivalents thereof without departing from the spirit and scope of the invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A sea surface oil spill detection method based on a full convolution network is characterized by comprising the following steps:
inputting an image to be detected into a trained sea surface oil spill detection network, and outputting detection information of the image to be detected;
and the evaluation system compares the detection information with the characteristic value of the database, judges whether the sea surface has oil spill, judges that the oil spill occurs if an oil film is detected, and sends out a warning.
2. The sea surface oil spill detection method based on the full convolution network of claim 1, wherein the training method of the trained sea surface oil spill detection network comprises:
acquiring a sea surface oil spill image, preprocessing the sea surface oil spill image to obtain an image data set, and extracting a spectrogram of an oil film area;
combining an image data set and an oil film spectrogram to serve as a training sample set, and labeling the training sample set to obtain label information of the training sample set;
inputting the label information of the training sample set into a pre-constructed initial sea surface oil spill detection network, extracting a plurality of characteristic information, and combining the plurality of characteristic information to obtain fused characteristic information;
acquiring prediction information of each pixel of the oil spilling image according to the fusion characteristic information;
and optimizing to obtain a trained sea surface oil spill detection network, inputting an image to be detected into the sea surface oil spill detection network, and outputting detection information of the image to be detected.
3. The method for detecting sea surface oil spill based on the full convolution network of claim 1, wherein the step of preprocessing the oil spill image comprises: (1) the problem of unsatisfactory contrast is solved by a contrast-limiting self-adaptive histogram equalization method, and meanwhile, the enhancement of noise is inhibited; (2) the high-frequency enhanced filtering method is used for carrying out edge sharpening enhancement on the image and pulling back the gray distribution of the image to a certain extent;
preferably, the method for extracting the spectrogram of the oil film region is to input the oil spill image into a full convolution network to perform two-class segmentation of the oil film region and a background image, and then extract the spectrogram of the oil film region;
preferably, the process of calculating the error is further included after obtaining the prediction information: calculating an error between the prediction information and the tag information using a loss function;
preferably, the optimization method comprises: optimizing parameters of the initial sea surface oil spill detection network by adopting a back propagation method until the error reaches an expected value;
preferably, the step of generating and saving the detection report is further included after the alarm is given.
4. The sea surface oil spill detection method based on the full convolution network of claim 1, wherein the spectrogram of the oil film is obtained by performing two-dimensional Fourier transform on the oil film region, and is mainly used for increasing the diversity of oil film characteristics to improve detection accuracy;
preferably, the prediction information of each pixel includes category information and regression information.
5. The sea surface oil spill detection method based on the full convolution network of claim 1, wherein the category information of each pixel is a three-dimensional matrix H x W x C; c is the category number of the data set; the regression information of each pixel is a three-dimensional matrix H multiplied by W multiplied by 4.
6. A sea surface oil spill detection system based on a full convolution network is characterized by comprising:
the backbone network is used for extracting the multi-scale characteristics of the oil spilling image;
a Transformer module which replaces the convolution layer with a self-attention layer and dynamically changes a feature transformation mode and a feature fusion mode through a Gate block;
obtaining a Transformed characteristic pyramid by the multi-scale characteristic through a Transformer module;
the head network of the specific detection task obtains the final detection information.
7. The full convolution network-based sea surface oil spill detection system of claim 6 wherein the backbone network is ResNet-50; the multi-scale feature information is obtained by performing 8-time, 16-time and 32-time down-sampling on an original image with 800 × 1024 pixels to obtain feature information of 100 × 128, 50 × 64 and 25 × 32 pixels respectively in three-layer scale.
8. The full convolutional network based surface oil spill detection system of claim 6, wherein the fransformer module comprises a single-scale transform, a cross-scale transform, and a Gate block;
further preferably, the head network classification branch and regression branch are composed; the regression branch and the classification branch are parallel and share parameters;
further preferably, the detection information is related information of a class of an object in the oil spilling image and a detection bounding box;
further preferably, the detection report includes information on the size of the oil film, the severity of the oil spill, and the location.
9. The full convolutional network based sea surface oil spill detection system of claim 6, further comprising:
the device comprises a sea surface oil spilling image acquisition module, a preprocessing module, a frequency spectrum acquisition module and a sample label processing module;
preferably, the sea surface oil spill image acquisition module is used for acquiring a sea surface oil spill image;
preferably, the preprocessing module is aimed at acquiring image datasets, limiting contrast adaptive histogram equalization, suppressing noise, high frequency and filtering;
preferably, the spectrum acquiring module is used for inputting the oil spill image into the full convolution network to perform two-classification segmentation of an oil film region and a background image, and then extracting a spectrogram of the oil film region;
preferably, the sample label processing module is used for labeling the image data set to obtain label information of the image data set.
10. Use of the method of any one of claims 1 to 7 and/or the system of claim 8 or 9 in the field of sea surface spill detection.
CN202110540801.7A 2021-05-18 2021-05-18 Sea surface oil spill detection method based on full convolution network, system and application thereof Active CN113177929B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110540801.7A CN113177929B (en) 2021-05-18 2021-05-18 Sea surface oil spill detection method based on full convolution network, system and application thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110540801.7A CN113177929B (en) 2021-05-18 2021-05-18 Sea surface oil spill detection method based on full convolution network, system and application thereof

Publications (2)

Publication Number Publication Date
CN113177929A true CN113177929A (en) 2021-07-27
CN113177929B CN113177929B (en) 2022-08-09

Family

ID=76929330

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110540801.7A Active CN113177929B (en) 2021-05-18 2021-05-18 Sea surface oil spill detection method based on full convolution network, system and application thereof

Country Status (1)

Country Link
CN (1) CN113177929B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113570589A (en) * 2021-08-03 2021-10-29 河海大学 Deep learning SAR image oil spilling region identification method based on feature fusion
CN117115175A (en) * 2023-10-25 2023-11-24 天津东泉石油技术开发有限公司 Online monitoring and early warning method and system for oil spilling

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446631A (en) * 2018-03-20 2018-08-24 北京邮电大学 The smart frequency spectrum figure analysis method of deep learning based on convolutional neural networks
CN108957125A (en) * 2018-03-20 2018-12-07 北京邮电大学 Smart frequency spectrum figure analysis method based on machine learning
CN109919123A (en) * 2019-03-19 2019-06-21 自然资源部第一海洋研究所 Offshore spilled oil detection method based on Analysis On Multi-scale Features depth convolutional neural networks
CN111722199A (en) * 2020-08-10 2020-09-29 上海航天电子通讯设备研究所 Radar signal detection method based on convolutional neural network
CN111860615A (en) * 2020-06-30 2020-10-30 河海大学 Remote sensing water body target extraction method based on feature dictionary fusion
CN111914924A (en) * 2020-07-28 2020-11-10 西安电子科技大学 Rapid ship target detection method, storage medium and computing device
EP3816818A2 (en) * 2020-06-30 2021-05-05 Beijing Baidu Netcom Science and Technology Co., Ltd. Method and apparatus for visual question answering, computer device and medium

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108446631A (en) * 2018-03-20 2018-08-24 北京邮电大学 The smart frequency spectrum figure analysis method of deep learning based on convolutional neural networks
CN108957125A (en) * 2018-03-20 2018-12-07 北京邮电大学 Smart frequency spectrum figure analysis method based on machine learning
CN109919123A (en) * 2019-03-19 2019-06-21 自然资源部第一海洋研究所 Offshore spilled oil detection method based on Analysis On Multi-scale Features depth convolutional neural networks
CN111860615A (en) * 2020-06-30 2020-10-30 河海大学 Remote sensing water body target extraction method based on feature dictionary fusion
EP3816818A2 (en) * 2020-06-30 2021-05-05 Beijing Baidu Netcom Science and Technology Co., Ltd. Method and apparatus for visual question answering, computer device and medium
CN111914924A (en) * 2020-07-28 2020-11-10 西安电子科技大学 Rapid ship target detection method, storage medium and computing device
CN111722199A (en) * 2020-08-10 2020-09-29 上海航天电子通讯设备研究所 Radar signal detection method based on convolutional neural network

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
YAOHUA XIONG，HUI ZHOU: "Oil spills identification in SAR image based on Convolutional Neural Network", 《 2019 14TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE & EDUCATION (ICCSE)》 *
赵志宏，李乐豪，杨绍普，李晴: "一种频域特征提取自编码器及其在故障诊断中的应用研究", 《中国机械工程》 *
魏帼，郭浩: "基于全卷积神经网络的SAR海面溢油图像分割方法", 《计算机应用》 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113570589A (en) * 2021-08-03 2021-10-29 河海大学 Deep learning SAR image oil spilling region identification method based on feature fusion
CN117115175A (en) * 2023-10-25 2023-11-24 天津东泉石油技术开发有限公司 Online monitoring and early warning method and system for oil spilling
CN117115175B (en) * 2023-10-25 2024-01-09 天津东泉石油技术开发有限公司 Online monitoring and early warning method and system for oil spilling

Also Published As

Publication number Publication date
CN113177929B (en) 2022-08-09

Similar Documents

Publication Publication Date Title
Haurum et al. A survey on image-based automation of CCTV and SSET sewer inspections
CN109409263B (en) Method for detecting urban ground feature change of remote sensing image based on Siamese convolutional network
CN109598241B (en) Satellite image marine ship identification method based on Faster R-CNN
CN111626290A (en) Infrared ship target detection and identification method under complex sea surface environment
CN108647648A (en) A kind of Ship Recognition system and method under visible light conditions based on convolutional neural networks
CN113177929B (en) Sea surface oil spill detection method based on full convolution network, system and application thereof
KR102320985B1 (en) Learning method and learning device for improving segmentation performance to be used for detecting road user events using double embedding configuration in multi-camera system and testing method and testing device using the same
CN109460754A (en) A kind of water surface foreign matter detecting method, device, equipment and storage medium
CN110008900B (en) Method for extracting candidate target from visible light remote sensing image from region to target
CN112734761B (en) Industrial product image boundary contour extraction method
CN113076871A (en) Fish shoal automatic detection method based on target shielding compensation
Xing et al. Traffic sign recognition using guided image filtering
CN104318051A (en) Rule-based remote-sensing automatic extraction system and method of wide-range water body information
CN116152115B (en) Garbage image denoising processing method based on computer vision
CN112749654A (en) Deep neural network model construction method, system and device for video fog monitoring
CN114781514A (en) Floater target detection method and system integrating attention mechanism
CN111582004A (en) Target area segmentation method and device in ground image
CN110458019B (en) Water surface target detection method for eliminating reflection interference under scarce cognitive sample condition
Xing et al. The Improved Framework for Traffic Sign Recognition Using Guided Image Filtering
Bello-Salau et al. An examination of different vision based approaches for road anomaly detection
CN113901945A (en) Sea surface oil spill detection method and system based on fast RCNN
CN112686222B (en) Method and system for detecting ship target by satellite-borne visible light detector
CN115035429A (en) Aerial photography target detection method based on composite backbone network and multiple measuring heads
CN114463628A (en) Deep learning remote sensing image ship target identification method based on threshold value constraint
CN114049478A (en) Infrared ship image rapid identification method and system based on improved Cascade R-CNN

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant