CN114821358A - Optical remote sensing image marine ship target extraction and identification method - Google Patents

Abstract

The invention relates to a method for extracting and recognizing marine ship targets in optical remote sensing images, comprising: inputting visible light remote sensing images; using a visual saliency method based on covariance joint features to globally detect the sea surface in the remote sensing images to generate a single scale saliency map; downsample the generated single-scale saliency map, establish a multi-scale saliency map, and obtain the final saliency map; calculate the detection threshold of the surface ship target, binarize the saliency map, and realize the coarse segmentation of the saliency map; establish training set and test set; to obtain feature vectors that can be used for training and testing; according to the established framework and the positive and negative samples in the training set, train the model, and classify the feature vectors; according to the established framework and the positive and negative samples in the test set , test the candidate area, and complete the target extraction and identification of ships at sea from optical remote sensing images. The invention can efficiently search the target area on the sea surface, greatly reduces the false alarm rate, and improves the detection accuracy.

Description

A method of target extraction and recognition for ships at sea from optical remote sensing images

technical field

The invention relates to a method for extracting and identifying targets of ships at sea in optical remote sensing images.

Background technique

In recent years, marine remote sensing technology has always been one of the challenging research topics in the field of computer vision, among which ship detection technology is the most promising. With the rapid development of remote sensing information science, ship detection using remote sensing technology is not only used in military fields such as maritime reconnaissance, maritime strike analysis and evaluation, but also in civil fields such as island resource survey, marine exploration, and maritime rescue. important value.

With the increasing ability of remote sensing data acquisition of air and space platforms and the rapid development of high-resolution satellites, more and more remote sensing data can be used for research. From the perspective of data acquisition, the current ship detection can be roughly divided into three categories: synthetic aperture radar (SAR) image ship detection, infrared (IR) image ship detection and visible light remote sensing (VRS) image ship detection. Since the synthetic aperture radar (SAR) method has the ability of day and night imaging that is not affected by complex weather conditions such as light, clouds and fog, and has a certain penetrability, it is mostly used to monitor oil spills and ocean currents on the ocean surface. Countries around the world have been working on the development of new SAR payloads, continuously improving spatial resolution, and achieving impressive performance. However, the coherent imaging mechanism of SAR images results in a large amount of speckle noise in the image, which seriously interferes with edge and texture features, and cannot utilize color information, so it is not suitable for ship target recognition. In addition, infrared images are used to enhance visual effects in low-light conditions, but they also suffer from low signal-to-noise ratios and insufficient structural information. However, there are more features available in visible light images, such as color, texture, edge, direction, and frequency domain features, so more details and complex structures can be captured.

At present, the extraction and detection methods of the target area of ships at sea include: the traditional optical remote sensing image ship detection is based on the grayscale information segmentation image, this kind of method is only suitable for the situation of calm sea surface and no cloud and fog interference, and the robustness is poor; there are also The method based on template matching, this type of method mainly eliminates the offshore island area according to the shape of the ship target, but for different types of ships in different scenes, the template is not easy to select; there are also traditional machine learning methods, mainly to separate the target. and background regions, highly dependent on ideal training samples; based on deep learning methods, it can effectively classify targets and backgrounds, but it has high hardware requirements, complex training steps, and poor interpretability; methods based on sparse representation are currently not systematic, and there are It is partly applied to ship detection in infrared images; there are also segmentation methods based on visual saliency, which can generate fewer candidate regions than the number of candidate regions generated by the sliding window method and the grayscale information segmentation method in the machine learning method. Suspected target area.

All in all, the current ship target detection task in optical remote sensing images has at least the following shortcomings:

1. Islands, dense clouds, waves and various uncertain sea conditions lead to a high false alarm rate.

2. Visible light imaging sensor parameter limitation, sea clutter interference, and ship wake interference reduce the homogeneity of ship targets.

3. The color, texture, size, type and other factors of the ship cause the low correlation of the target grayscale;

4. Based on the rapidity of large-scale remote sensing data, reducing the computational burden has become a key issue;

5. The problem of low detection efficiency caused by different target directions and insignificant features during the detection process.

Therefore, how to perform fast, stable and robust extraction and detection under complex conditions such as harsh sea conditions, low target homogeneity and gray correlation, target geometric distortion, and target rotation has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

In view of this, it is necessary to provide an optical remote sensing image marine ship target extraction and identification method.

The present invention provides a method for extracting and recognizing a marine vessel target from an optical remote sensing image. The method includes the following steps: a. inputting a visible light remote sensing image; b. introducing a covariance statistic and a homologous similarity measure, and using a covariance-based joint feature The visual saliency method in the remote sensing image performs global detection on the sea surface in the remote sensing image, and generates a single-scale saliency map; c. Down-sampling the generated single-scale saliency map, establishes a multi-scale saliency map, and merges it through the stacking fusion mechanism The final saliency map is obtained by integrating the saliency map; d. According to the grayscale statistical characteristics of the obtained final saliency map, the detection threshold of the ship target on the sea surface is calculated, and the saliency map is binarized to realize the rough segmentation of the saliency map and mark it back to the original remote sensing image, find out the area of each target, separate the suspected target and the sea surface background; e. establish a training set and a test set, the training set includes positive samples and negative samples, and the test set includes positive samples and negative samples; f. Design CF-Fourier features, embed aggregated channel features (ACF) and pyramid features (FGPM), and establish a framework to obtain feature vectors that can be used for training and testing; g. According to the established framework and positive and negative samples in the training set, train the model , using the Boosting decision tree to classify the feature vector in step f; h. According to the established framework and the positive and negative samples in the test set, test the candidate area, and complete the extraction and identification of the marine ship target of the optical remote sensing image.

Preferably, the step a includes:

Input the optical remote sensing image f(x, y) with a spatial resolution of H×W. The remote sensing image includes ships, sea fog, thick clouds, islands, etc., among which the size and color polarity of the ships are different. , the positions on the sea surface are randomly distributed.

Preferably, the step b includes:

Step S21: according to the described remote sensing image of input, calculate the brightness of pixel m, extract the gradient features of the horizontal direction and the vertical direction, the second-order derivative feature of brightness, and the brightness L of the Lab color space close to human vision, the opposite color dimension a and b feature, and the position coordinates (x, y) form a nine-dimensional feature vector f _m :

Step S22: Divide the remote sensing image into square regions R of the same size, calculate the feature mean, and construct a 9×9 covariance feature matrix symmetrically with f _m as the region descriptor S:

Step S23: Cholesky decomposition is adopted for the region descriptor to obtain each row vector Li in the upper triangular matrix, then the region descriptor is equivalent to a set of point sets S in the _Euclidean space:

Combining the feature mean μ and the point set S obtains the eigenvector ψ _μ (C _R ) with Euclidean space computing power for the _CR encoding:

ψ _μ (C _R )=(μ,s ₁ ,s ₂ ,...,s _k ,,s _k+1 ,.s _k+2 ..,s _2k );

Step S24: Through the context similarity measure, find the most similar T measures to represent the saliency of the region, and the formula is as follows:

Step S25: On the basis of the above-mentioned saliency, a homologous similarity weight function w _j is designed to enhance the contrast and obtain a sparse map of the salient region:

Among them, the weight function is measured by the inverse feature distance function, which is defined as a Gaussian function:

Preferably, the step d includes:

Using the OTSU method, an adaptive segmentation threshold T is obtained to establish a connected region to extract the target:

Preferably, the step f specifically includes:

Step S61: The gradient of the plane image I(x,y) at the pixel (x,y) is expressed as (D(x,y), θ(D(x,y))), and the continuous gradient direction pulse curve is calculated as :

h(ζ)=||D(x,y)||δ(ζ-θ(D(x,y)));

Step S62: Use Fourier analysis on the gradient direction pulse curve:

coefficient

Step S63: Rotate the image in the vector field and find the condition of rotation invariance and the designed self-directed kernel function P _j (r):

Step S64: Using the above kernel function convolution modeling, according to the above conditions of rotation invariance, the Fourier HOG rotation invariance descriptor is expressed as follows:

Step S65: Introduce the circular frequency filter CF, use the brightness difference between the ship and the surrounding background, design the gray value change mode of the ship target, and calculate the discrete Fourier transform DFT of the gray value at the pixel (i, j):

Step _S66 : The extracted rotation invariant gradient features and circumferential frequency features are sent to the classifier to distinguish whether it is a real ship or a false alarm: for the direct collection of image pyramid features, the efficiency is slow. Estimate the scale factor λ to achieve fast pyramid feature estimation for different scales d ₁ :

F _d1 =F _d0 ·(d ₀ /d ₁ ) ^-λ .

Preferably, described step g specifically comprises the following steps:

By using the positive and negative samples as the input of the model in a ratio of 1:3, the model is trained, and the confidence scores of the candidate regions are generated by classification, and the intersection ratio is used as the criterion for determining whether the target is true.

Preferably, the step h specifically includes:

The suspected ship target area fragments extracted in step d are tested to determine whether it is a real target or a false alarm. If it is a real target, it will be retained, and if it is a false alarm, it will be eliminated, and finally marked in the input image.

This application does not have many complex parameter settings, nor does it rely on the prior knowledge of the sea surface background and target distribution characteristics. Aiming at the characteristics of ship targets in the sea surface background, a method for combined visual saliency detection based on covariance joint statistical features is proposed. Integrating regional covariance estimation with homologous similarity weighted fusion corrects the deficiency and enhances the overall advantage, thereby suppressing sea surface background interference. The multi-scale stacking fusion enhances the overall continuity of the detected targets and the distinguishability between targets, and efficiently searches the sea surface target area. For false alarms such as thick clouds and islands that may appear in the image, the Aggregated Channel Feature-Feature Pyramid Acceleration Framework (ACF-FPGM) embedded in the CF-Fourier spatial frequency domain joint feature is used to further identify the detected targets. Whether the target is a ship or not, the false alarm rate is greatly reduced and the detection accuracy is improved.

In addition, the detection and identification time of this application is in the second level, the real-time performance is good, and the degree of automation has been significantly improved, which can realize the rapid discovery, positioning and quantity determination of ship targets in a large area of sea area and multiple background interference, and the detection is robust. it is good. It lays a foundation for further calculating the position, heading and other intelligence information of each ship in combination with the UAV platform or satellite attitude data, as well as the classification and identification of ship targets.

Description of drawings

Fig. 1 is the flow chart of the method for extracting and recognizing marine vessel targets in optical remote sensing images according to the present invention;

FIG. 2 is a flowchart of a method for extracting and identifying targets of ships at sea from optical remote sensing images according to an embodiment of the present invention;

3 is a schematic diagram of a process for extracting a visually saliency target provided by an embodiment of the present invention;

FIG. 4 is a schematic diagram of a multi-scale fusion effect legend provided by an embodiment of the present invention: wherein, FIG. 4(a) the original image; FIG. 4(b) σ=2-4; FIG. ⁴ (c) σ= ^2-5 ; (d) σ= ^2-6 ; Fig. 4 (e) fusion saliency map;

FIG. 5(a) is an original image I provided by an embodiment of the present invention;

FIG. 5(b) is a directional gradient map Dx/Dy provided by an embodiment of the present invention;

Figure 5 (c) is a schematic diagram of the Fourier analysis coefficient of the gradient provided by the embodiment of the present invention:

FIG. 6 is a schematic diagram of a CF feature gray value change mode and a feature diagram provided by an embodiment of the present invention: wherein, FIG. 6(a) Ships; FIG. 6(b) grayscale statistics map; FIG. 6(c) pseudo-color map;

FIG. 7 is a schematic diagram of a fine discrimination flow according to an embodiment of the present invention.

Detailed ways

The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Referring to FIG. 1 and FIG. 2 , it is a working flow chart of a preferred embodiment of the method for extracting and recognizing a marine vessel target from an optical remote sensing image of the present invention.

Step S1, input a visible light remote sensing image f(x, y) with a spatial resolution of H*W. in particular:

Input the optical remote sensing image f(x, y) with a spatial resolution of H×W. The remote sensing image includes ships, sea fog, thick clouds, islands, etc., among which the size and color polarity of the ships are different. , the location distribution on the sea surface is also very random.

Step S2, introducing covariance statistics and homology similarity measures, and using a visual saliency method based on covariance joint features to globally detect the sea surface in the remote sensing image, and generate a single-scale saliency map. in particular:

Step S21: according to the described remote sensing image of input, calculate the brightness of pixel m, extract the gradient features of the horizontal direction and the vertical direction, the second-order derivative feature of brightness, and the brightness L of the Lab color space close to human vision, the opposite color dimension a and b feature, the above three types of features respectively correspond to the three-line image sequence of the second column of Figure 3, and the position coordinates (x, y) form a nine-dimensional feature vector f _m :

Step S22: Divide the remote sensing image into square regions R of the same size, calculate the feature mean, and construct a 9×9 covariance feature matrix symmetrically with f _m as the region descriptor S:

Step S23: Cholesky decomposition is adopted for the region descriptor to obtain each row vector Li in the upper triangular matrix, then the region descriptor is equivalent to a set of point sets S in the _Euclidean space:

Combining the feature mean μ and the point set S obtains the eigenvector ψ _μ (C _R ) with Euclidean space computing power for the _CR encoding:

ψ _μ (C _R )=(μ,s ₁ ,s ₂ ,...,s _k ,,s _k+1 ,.s _k+2 ..,s _2k )

Step S24: Through the context similarity measure, the similarity is represented by the Euclidean distance, and the most similar T measures are found to represent the saliency of the region. Wherein, the context includes: the radius is three times the length of the 3-division unit, that is, the numbering range of R _i is 1-9 and does not include the number 5 of R. In this embodiment, T=5 is set, and the formula is as follows:

Step S25: On the basis of the above-mentioned saliency, a homologous similarity weight function w _j is designed to enhance the contrast and obtain a sparse map of the salient region:

Among them, the weight function is measured by the inverse feature distance function, which is defined as a Gaussian function:

Step S3, down-sampling the generated single-scale saliency map, establishes a multi-scale saliency map, and obtains the final saliency map through a stacking and multiplying fusion mechanism and normalized fusion. in particular:

According to the above single-scale saliency map generation process, expand to multi-scale, balance the confrontation relationship between the regional representation ability and the spatial resolution of the saliency map, adopt the multi-scale product fusion strategy and normalize, please refer to Figure 4, this embodiment uses 3 scales, Γ={σ|2 ^k }(k=-4,-5,-6), as shown in Figure 4(b)-Figure 4(d), the final saliency map (Figure 4(e)) is in The fine-scale cloud removal effect is better, and the coarse-scale is conducive to highlighting the ship target:

Step S4, according to the gray-scale statistical features of the obtained final saliency map, calculate the detection threshold of the surface ship target, binarize the saliency map, realize the rough segmentation of the saliency map, and mark it back to the original remote sensing image to find out each target. , separate the suspected target and the sea background. in particular:

This embodiment adopts the OTSU method to obtain an adaptive segmentation threshold T to establish a connected region to extract the target:

Step S5, please refer to FIG. 7 together to establish a training set and a test set. The training set includes positive samples and negative samples; the test set includes positive samples and negative samples.

In this example, a training set and a test set are established, with a total of 630 data sets, and the size of each image is 56 pixels*56 pixels. The positive samples include various ships in different backgrounds, and the size of the ships varies from 6 to 20 pixels; the negative samples come from possible background interference at sea, such as ocean waves, wake waves, thin clouds, dense Clouds, islands, etc.

Step S6, design CF-Fourier features, embed aggregated channel features (ACF) and pyramid features (FGPM), and establish a framework to obtain feature vectors that can be used for training and testing. in particular:

Step S61: As shown in Figure 5(a) and Figure 5(b), the gradient of the plane image I(x,y) at the pixel (x,y) is expressed as (D(x,y), θ(D( x, y))), then the continuous gradient direction pulse curve is calculated as:

h(ζ)=||D(x,y)||δ(ζ-θ(D(x,y)))

Step S62: Use Fourier analysis on the gradient direction pulse curve:

对应的Fourier域系数图像如图5(c)所示；coefficient

The corresponding Fourier domain coefficient image is shown in Figure 5(c);

Step S63: Rotate the image in the vector field and find the condition of rotation invariance and the designed self-directed kernel function P _j (r):

Step S64: Using the above kernel function convolution modeling, according to the above conditions of rotation invariance, the Fourier HOG rotation invariance descriptor is expressed as follows:

Step S65: Introduce a circular frequency filter (CF), and use the brightness difference between the ship and the surrounding background to design the gray value change mode of the ship target, please refer to Figure 6, and the gray value at the pixel (i, j) needs to be calculated The discrete Fourier transform (DFT) of :

Step S66: The extracted rotation invariant gradient features and circular frequency features will be sent to the classifier to distinguish whether it is a real ship or a false alarm. Feature refinement is performed structurally using Aggregated Channel Features (ACF). The fast detection rate and low computational requirements are crucial for inputting the acquired ACF into a boosting decision tree. In view of the slow efficiency of directly collecting image pyramid features, on the basis of the basic scale d ₀ , the fast pyramid feature estimation of different scales d ₁ is achieved by estimating the scale factor λ:

F _d1 =F _d0 ·(d ₀ /d ₁ ) ^-λ

Step S7, according to the established framework and the positive and negative samples in the training set, train the model, and use the Boosting decision tree to classify the feature vector in step S6.

By using the positive and negative samples as the input of the model in a ratio of 1:3, the model is trained, and the confidence score of the candidate region is generated by classification, and the intersection ratio (iou) is used as the criterion for determining whether it is a true target.

Step S8, according to the established framework and the positive and negative samples in the test set, test the candidate area, and complete the extraction and identification of the marine ship target of the optical remote sensing image.

In this embodiment, the suspected ship target area fragments extracted in step S4 are set to 56*56, and are sent to the model for testing to determine whether it is a real target or a false alarm. marked in.

This application includes visual saliency segmentation extraction and fine-grained discrimination for supervised systems. In the stage of visual saliency segmentation and extraction, by constructing covariance features belonging to surface ships, and using homologous similarity to design second-order statistic weights, the single saliency map obtained after the optimal similarity measurement can well weaken the background. Highlight the target; finally design a multi-scale fusion strategy and an adaptive threshold segmentation module, which can achieve efficient extraction of unsupervised surface ship areas in multiple environments such as different target scales, cluttered cloud and fog backgrounds, sea clutter and wake wave interference. In the fine discrimination stage of the supervision system, the CF-Fourier HOG feature in the space frequency domain of the ship is designed, which is rotationally invariant. Under the framework of aggregated channel features and fast feature pyramid acceleration, the identification of ship targets is completed, and the virtual Police culling.

Although the present invention has been described with reference to the current preferred embodiments, those skilled in the art should understand that the above preferred embodiments are only used to illustrate the present invention, not to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the scope of the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (7)

1. A marine ship target extraction and identification method based on optical remote sensing images is characterized by comprising the following steps:

a. inputting a visible light remote sensing image;

b. introducing covariance statistics and homologous similarity measurement, and using a visual saliency method based on covariance combined characteristics to carry out global detection on the sea surface in the remote sensing image to generate a single-scale saliency map;

c. sampling the generated single-scale saliency map in a downlink manner, establishing a multi-scale saliency map, and obtaining a final saliency map through a superposition fusion mechanism and normalized fusion;

d. calculating a detection threshold value of a sea surface ship target according to the obtained gray statistical characteristics of the final saliency map, binarizing the saliency map, realizing rough segmentation of the saliency map, marking back an original remote sensing image, finding out the area of each target, and separating a suspected target from a sea surface background;

e. establishing a training set and a test set, wherein the training set comprises positive samples and negative samples, and the test set comprises positive samples and negative samples;

f. designing a CF-Fourier characteristic, embedding a polymerization channel characteristic ACF and a pyramid characteristic FGPM, and establishing a framework to obtain a characteristic vector for training and testing;

g. training a model according to the built frame and positive and negative samples in the training set, and classifying the feature vectors in the step f by using a Boosting decision tree;

h. and testing the candidate area according to the established frame and the positive and negative samples in the test set, and finishing the extraction and identification of the marine ship target of the optical remote sensing image.

2. The method of claim 1, wherein step a comprises:

the method comprises the steps of inputting an optical remote sensing image f (x, y) with the spatial resolution of H multiplied by W, wherein the remote sensing image comprises ships, sea fog, thick cloud layers, islands and the like, the sizes and the color polarities of the ships are different, and the ships are randomly distributed on the sea surface.

3. The method of claim 2, wherein step b comprises:

step S21: calculating the brightness of a pixel m according to the input remote sensing image, extracting gradient characteristics in the horizontal direction and the vertical direction, brightness second derivative characteristics, brightness L of Lab color space close to human vision, opposite color dimension a and b characteristics, and forming a nine-dimensional characteristic vector f with position coordinates (x, y) _m ：

Step S22: dividing the remote sensing image into square regions R with the same size, calculating a characteristic mean value, and using f _m A 9 × 9 covariance feature matrix is constructed symmetrically as a region descriptor S:

step S23, Cholesky decomposition is carried out on the region descriptor to obtain each row vector L in the upper triangular matrix _i Then the region descriptor is equivalent to a set of points S in euclidean space:

combining the characteristic mean value mu and the point set S to obtain a pair C _R Encoding a feature vector psi with Euclidean spatial computation capability _μ (C _R )：

ψ _μ (C _R )＝(μ,s ₁ ,s ₂ ,...,s _k ,,s _k+1 ,.s _k+2 ..,s _2k )；

Step S24: by the context similarity measure, the most similar T measures are found to represent the significance of the region, and the formula is as follows:

step S25: designing a homologous similarity weight function w on the basis of the significance _j And enhancing contrast to obtain a sparse graph of the salient region:

wherein the weight function is measured using an inverse function of the feature distance, defined as a gaussian function:

4. the method of claim 3, wherein step d comprises:

adopting an OTSU method to acquire a self-adaptive segmentation threshold T to establish a connected region for extracting a target:

5. the method according to claim 4, wherein said step f specifically comprises:

step S61: the gradient of the planar image I (x, y) at the pixel (x, y) is represented as (D (x, y), θ (D (x, y))), and the continuous gradient direction pulse curve is calculated as:

h(ζ)＝||D(x,y)||δ(ζ-θ(D(x,y)))；

step S62: fourier analysis was used for gradient direction pulse curves:

coefficient of performance

Step S63: self-guiding kernel function P for rotating image in vector field and searching condition of rotation invariance _j (r)：

Step S64: by adopting the kernel function convolution modeling, according to the condition of the rotation invariance, the Fourier HOG rotation invariance descriptor is expressed as follows:

step S65: introducing a circumferential frequency filter CF, designing a gray value change mode of a ship target by using the brightness difference between a ship and a surrounding background, and calculating Discrete Fourier Transform (DFT) of the gray value at a pixel (i, j):

step S66: the extracted rotation invariance gradient characteristic and the extracted circumference frequency characteristic are sent to a classifier to distinguish whether the ship is a real ship or a false alarm: the efficiency is slow for directly collecting image pyramid features at the basic scale d ₀ Based on the different scales d realized by estimating the scale factor lambda ₁ Fast pyramid feature estimation of (1):

F _d1 ＝F _d0 ·(d ₀ /d ₁ ) ^-λ 。

6. the method according to claim 5, wherein said step g comprises the steps of:

by dividing the positive and negative samples by 1: and 3, as an input end of the model, training the model, classifying to generate confidence scores of the candidate regions, and using the cross-over ratio as a judgment standard for judging whether the candidate regions are true targets.

7. The method according to claim 6, wherein said step h specifically comprises:

and d, testing the fragments of the suspected ship target area extracted in the step d, judging whether the fragments are real targets or false alarms, if the fragments are real targets, retaining the fragments, if the fragments are false alarms, removing the fragments, and finally marking the fragments in the input image.

Images