CN113255513A - Remote sensing ship target tracking method based on background self-selection - Google Patents
Remote sensing ship target tracking method based on background self-selection Download PDFInfo
- Publication number
- CN113255513A CN113255513A CN202110560777.3A CN202110560777A CN113255513A CN 113255513 A CN113255513 A CN 113255513A CN 202110560777 A CN202110560777 A CN 202110560777A CN 113255513 A CN113255513 A CN 113255513A
- Authority
- CN
- China
- Prior art keywords
- tracking
- background
- image
- target
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 54
- 238000001514 detection method Methods 0.000 claims abstract description 53
- 230000007246 mechanism Effects 0.000 claims abstract description 19
- 238000001914 filtration Methods 0.000 claims abstract description 14
- 238000013528 artificial neural network Methods 0.000 claims abstract description 7
- 241000195940 Bryophyta Species 0.000 claims description 12
- 230000011218 segmentation Effects 0.000 claims description 12
- 238000007781 pre-processing Methods 0.000 claims description 5
- 230000008859 change Effects 0.000 claims description 3
- 239000006185 dispersion Substances 0.000 claims description 3
- 238000012958 reprocessing Methods 0.000 claims description 3
- 230000001360 synchronised effect Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 abstract description 8
- 230000004044 response Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000005070 sampling Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000012935 Averaging Methods 0.000 description 2
- 102100021709 Rho guanine nucleotide exchange factor 4 Human genes 0.000 description 2
- 101710128386 Rho guanine nucleotide exchange factor 4 Proteins 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000009795 derivation Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/30—Noise filtering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a remote sensing ship multi-target tracking method based on background self-selection, which is characterized in that a front module is used for carrying out high-precision target detection and background classification on a specific frame, and the sequence frame is divided into a pure ocean background and a sea-land background by the front module. For the frame image of the pure ocean background, tracking multiple targets by adopting a high-speed related filtering algorithm; and for the frame image containing the sea-land background, performing multi-target tracking by adopting a high-precision neural network. In addition, a tracking loss returning mechanism is arranged at each module, and for the tracking loss and scene switching phenomena of the tracking module, the tracking loss returning mechanism returns to the front-end module to detect and classify again. The method fully explores the characteristic information of the remote sensing video of the ship, greatly improves the tracking speed while ensuring the detection precision, has a complete network structure and a clear process, and has high field application value.
Description
Technical Field
The invention relates to a remote sensing ship multi-target tracking method based on background self-selection, and belongs to the technical field of multi-target tracking.
Background
Recently, with the frequent offshore military activities and the development of marine transportation trade, the offshore ship target is used as a key target for safety monitoring and wartime striking, and the rapid and accurate identification and tracking of the ship target by using a technical means has great practical significance and application value. The remote sensing video has the following characteristics: the quality of some frames in the long video is very poor, and the long video is represented by motion blur, cloud and fog occlusion and the like in the ship video. The video target detection and tracking mainly considers how to fuse more features above space and time, thereby making up the defects of the features obtained by a single frame in training or detection. In addition, from the shooting scene, the picture content of the remote sensing video of the ship is mainly divided into a sea-land background and a pure sea background. For frame images containing sea and land backgrounds, a port is provided with a large number of buildings, and due to the limitation of a remote sensing satellite shooting angle, a ship is difficult to detect well, and a high-precision high-accuracy detection network is preferred; for the frame image of the pure ocean background, because the single ocean background and the ship have better discrimination, the people prefer to pursue a high-speed tracking network. This feature is unique to remote video of vessels.
At present, the application of scholars at home and abroad in the aspect is still in the exploration stage, and the research is mainly aimed at the detection of vehicle and airplane targets. The algorithm of mixed Gaussian background modeling and multi-frame difference updated by combining the background model can better solve the problems of low detection rate and false alarm rate easily generated in satellite video micro-target detection. Aiming at the problem that the CN algorithm cannot be found again once the target is lost, the full-automatic tracking can be realized in a circular detection mode, and the realized method has stronger robustness. However, the following two problems still exist in the field at present: firstly, most of the existing researches are based on target detection of a ship remote sensing image, and deep research on a ship remote sensing video target tracking model is lacked; and secondly, the remote sensing video of the ship has strong field characteristics, and the traditional method is difficult to have generalization.
Disclosure of Invention
The invention provides a remote sensing ship multi-target tracking method based on background self-selection, aiming at the problems in the prior art, and a preposed module is introduced to divide a sequence frame into a pure sea background and a sea-land background. For the frame image of the pure ocean background, tracking multiple targets by adopting a high-speed related filtering algorithm; and for the frame image containing the sea-land background, performing multi-target tracking by adopting a high-precision neural network. In addition, a tracking loss returning mechanism is arranged at each module, and for the tracking loss and scene switching phenomena of the tracking module, the tracking loss returning mechanism returns to the front-end module to detect and classify again.
In order to achieve the purpose, the technical scheme of the invention is that a remote sensing ship multi-target tracking method based on background self-selection comprises the following steps:
step 1) preprocessing, namely performing operations such as denoising and feature enhancement on an input video sequence, removing interference of weather factors and shooting angles on the sequence, improving image definition and highlighting feature information;
step 2) target detection and sea-land background classification, wherein each frame preprocessed in the step 1) is subjected to target detection, a target detection result is used as initial target information of a subsequent tracking module, meanwhile, semantic segmentation is carried out on the image, and the image is divided into a pure sea background image and an image containing a sea-land background according to a segmentation result of the background;
step 3) tracking targets under different backgrounds, performing target tracking of different methods on the sequence according to the classification result of the step 2), and improving the tracking speed by adopting a related filtering tracking method for the pure ocean background sequence; the mainstream neural network method is adopted for the sequence containing the sea-land background, so that the tracking precision is improved as much as possible;
and 4) a loss return mechanism, wherein a tracking loss detection and return mechanism is arranged on the target tracking module, and a certain frame with tracking loss is returned to the step 2) for re-detection and classification.
In the preferred scheme of the method, in the step 1), a remote sensing image defogging method based on dark channel prior is adopted, and the minimum value of the three color channel intensity values of each pixel point is obtained to obtain a dark channel image of the image. The interference of the fog on the remote sensing image is rapidly and effectively removed while the real-time processing requirement is ensured, the image definition is improved, and the real color of the scenery is restored. The sequence image is then multiplied by a hanning window, which gradually reduces the pixel values near the edges to zero, so that more emphasis is placed near the center of the object.
In a preferred embodiment of the method of the present invention, the method for target detection and sea-land background classification in step 2) comprises: and carrying out ship target detection and background segmentation on the image by using YOLACT, judging whether the image contains land features according to the result of the background segmentation, if a large number of land features exist, considering that the image belongs to the image containing ocean and land backgrounds, and if not, considering that the image belongs to the image containing pure ocean backgrounds.
In the preferred scheme of the method of the invention, the method for tracking the target under different backgrounds in the step 3) comprises the following steps:
a) and for the sequence of the pure ocean background, a MOSSE tracker based on correlation filtering is adopted for tracking multiple targets, and the result of target detection in the step 2) is used as the initial target characteristic of the MOSSE. In the actual tracking process, the influence of factors such as appearance transformation of the target needs to be considered, so that a plurality of images of the target need to be considered at the same time as a reference, and the robustness of the filter template is improved. Meanwhile, a plurality of ship targets appear in one frame of image, and the targets may belong to different ship types but show similar characteristics in a frequency domain. The idea of ASEF is adopted to carry out synthesis averaging, then error accumulation is carried out on m frames of images, and the optimization problem is converted into:
h is defined as the optimal general filtering template to be found, niDefined as the number of ships in the ith frame image, H* ijDefining an optimal filtering template required by the jth ship in the ith frame image under Fourier transform, FiDefined as the Fourier transformation of the ith frame imageChanged input image, GijDefined as the corresponding output of the jth ship in the ith frame image under the Fourier transform.
A closed-form solution of H may be obtained by derivation,
b) and for sequences containing sea-land backgrounds, performing multi-target tracking by adopting a Siamese-RPN network, respectively performing corresponding multi-thread synchronous tracking on a classification branch and a regression branch, and taking the result of target detection in the step 2) as a template frame of the Siamese-RPN.
In the preferred embodiment of the method of the present invention, the specific process of the loss return mechanism in step 4) is as follows:
a) in Siamese-RPN, fractional values score for various positions around the center pointiWhen the video environment is very close to the original video environment, the video environment is considered to be obviously changed or the tracking is lost, at the moment, an alarm of the tracking loss needs to be sent out, a sea-land detection module is started to update a template frame, the dispersion degree of all bundingboxes is reflected by a standard deviation sigma, and N represents the number of the bounding boxes counted and sorted near the central point;
b) in the MOSSE, when the distance between the detection frame after the inverse fast fourier transform and the previous frame has a large position change, we consider that there is a tracking loss phenomenon, and at this time, the detection frame will return to the front-end module for reprocessing.
The method provided by the invention fully explores the characteristic information of the remote sensing video of the ship, provides a classification tracking mechanism, well adapts to application scenes, and greatly improves the tracking speed while ensuring the detection precision. Compared with the prior art, the invention has the following advantages:
(1) the method has strong adaptability to the remote sensing video of the ship. The remote sensing video of the ship is different from the target tracking of a general scene, the remote sensing near the land has the interference of very complex ground information, and the foreground and the background in the sea are very easy to segment. The invention respectively processes the two conditions by utilizing a mechanism of background selection, fully utilizes the characteristic information of the scene and improves the adaptability of the algorithm to the application field.
(2) High precision and high speed. Algorithms based on correlation filtering have extremely fast tracking speed but are not very accurate. The opposite is true for target tracking with mainstream deep neural networks. The algorithm provided by the invention integrates the advantages of the two methods, and obtains very good performance on the evaluation of speed and precision.
(3) Adaptability of scene switching. In the real-time transmission process, the problems of transmission interruption and scene switching exist, and a general target tracking algorithm depends on the labeled first frame information and cannot process the scene switching. In the algorithm, the initial information is provided by utilizing the preposed target detection module, and a plurality of tracking loss return mechanisms are arranged in the follow-up modules, so that the problem of tracking loss caused by scene switching is avoided to the greatest extent.
(4) The advantages of each network are fully utilized, and the defects of the network are made up. The YOLACT has high precision but relatively large time overhead, and the method of detecting the N frames is adopted to ensure that the time overhead of the detection can be ignored in the video stream. One of the disadvantages of siemese-RPN is that using the first frame as template matching is not robust to large variations in the target. Whereas the template frame is updated in time using a trace-back mechanism in the network of the present invention. The MOSSE has extremely fast tracking speed, but the adopted characteristics are single-channel gray characteristics, and the capability of representing the target is limited.
Drawings
FIG. 1 is a flow chart of the method of the present invention;
FIG. 2 is a simplified schematic diagram;
fig. 3 is a flow diagram of a MOSSE detector with tracking loss detection.
The specific implementation mode is as follows:
for the purpose of enhancing an understanding of the present invention, the present embodiment will be described in detail below with reference to the accompanying drawings.
A remote sensing ship multi-target tracking method based on background self-selection is characterized in that a front module is used for carrying out high-precision target detection and background classification on a specific frame, and the sequence frame is divided into a pure sea background and a sea-land background by the front module. For the frame image of the pure ocean background, tracking multiple targets by adopting a high-speed related filtering algorithm; and for the frame image containing the sea-land background, performing multi-target tracking by adopting a high-precision neural network. In addition, a tracking loss returning mechanism is arranged at each module, and for the tracking loss and scene switching phenomena of the tracking module, the tracking loss returning mechanism returns to the front-end module to detect and classify again.
Firstly, the architecture:
the architecture of the remote sensing vessel multi-target tracking method based on background self-selection is shown in fig. 1, and specific descriptions of several main parts are given below.
A remote sensing ship multi-target tracking method based on background self-selection comprises the following steps:
step 1, pretreatment:
the method comprises the steps of firstly, solving the minimum value of three color channel intensity values of each pixel point by adopting a remote sensing image defogging method based on dark primary color prior to obtain a dark primary color image of the image. The interference of the fog on the remote sensing image is rapidly and effectively removed while the real-time processing requirement is ensured, the image definition is improved, and the real color of the scenery is restored. The sequence image is then multiplied by a hanning window, which gradually reduces the pixel values near the edges to zero, so that more emphasis is placed near the center of the object.
Step 2, target detection and sea-land background classification:
and carrying out ship target detection and background segmentation on the image by using YOLACT, judging whether the image contains land features according to the result of the background segmentation, if a large number of land features exist, considering that the image belongs to the image containing ocean and land backgrounds, and if not, considering that the image belongs to the image containing pure ocean backgrounds.
The target tracking method under different backgrounds in the step 3 comprises the following steps:
a) and for the sequence of the pure ocean background, a MOSSE tracker based on correlation filtering is adopted for tracking multiple targets, and the result of target detection in the step 2) is used as the initial target characteristic of the MOSSE. In the actual tracking process, the influence of factors such as appearance transformation of the target needs to be considered, so that m images of the target need to be considered as reference at the same time, and the robustness of the filter template is improved. Meanwhile, a plurality of ship targets appear in one frame of image, and the targets may belong to different ship types but show similar characteristics in a frequency domain. The idea of ASEF is adopted to carry out synthesis averaging, then error accumulation is carried out on m frames of images, and the optimization problem is converted into:
h is defined as the optimal general filtering template to be found, niDefined as the number of ships in the ith frame image, H* ijDefining an optimal filtering template required by the jth ship in the ith frame image under Fourier transform, FiDefined as the input image of the i-th frame image under Fourier transform, GijDefined as the corresponding output of the jth ship in the ith frame image under the Fourier transform.
A closed-form solution of H may be obtained by derivation,
b) and for sequences containing sea-land backgrounds, performing multi-target tracking by adopting a Siamese-RPN network, respectively performing corresponding multi-thread synchronous tracking on a classification branch and a regression branch, and taking the result of target detection in the step 2) as a template frame of the Siamese-RPN.
Step 4, losing a return mechanism;
in Siamese-RPN, fractional values score for various positions around the center pointiWhen the video environment is very close to each other, the video environment is considered to be obviously changed or the tracking is lost, and then a tracking loss alarm needs to be sent out and a sea and land detection module needs to be startedUpdating the template frame by blocks, reflecting the dispersion degree of all bundling boxes by using a standard deviation sigma, and expressing the number of bounding boxes counted and sorted near the central point by N;
in the MOSSE, when the distance between the detection frame after the inverse fast fourier transform and the previous frame has a large position change, we consider that there is a tracking loss phenomenon, and at this time, the detection frame will return to the front-end module for reprocessing.
II, a specific process:
referring to fig. 1, a code search method based on structured information specifically includes the following steps:
step 1) preprocessing, performing operations such as denoising and feature enhancement on an input video sequence, removing interference of weather factors and shooting angles on the sequence, improving image definition and highlighting feature information.
And 2) carrying out target detection and sea-land background classification, carrying out target detection on each frame preprocessed in the step 1), and taking a target detection result as initial target information of a subsequent tracking module. And meanwhile, performing semantic segmentation on the image, and dividing the image into a pure ocean background image and an image containing a sea-land background according to a segmentation result of the background.
And 3) tracking targets under different backgrounds, and tracking the targets of the sequence by different methods according to the classification result of the step 2). A correlation filtering tracking method is adopted for the pure ocean background sequence, so that the tracking speed is improved; the mainstream neural network method is adopted for the sequence containing the sea-land background, so that the tracking precision is improved as much as possible.
And 4) a loss return mechanism, wherein a tracking loss detection and return mechanism is arranged on the target tracking module, and a certain frame with tracking loss is returned to the step 2) for re-detection and classification.
Third, the concrete application embodiment:
for convenience of description, we assume a simplified application example as shown in fig. 2: orange represents the vessel target to be tracked, gray represents the land background, and blue represents the sea background.
According to the aforementioned steps, the following steps are carried out in sequence:
firstly, preprocessing a video sequence, and solving the minimum value of three color channel intensity values of each pixel point to obtain a dark primary color image of the image by adopting a remote sensing image defogging method based on dark primary color prior. The sequence image is then multiplied by a hanning window, which gradually reduces the pixel values near the edges to zero, so that more emphasis is placed near the center of the object.
And secondly, entering a target detection and sea-land background classification module, classifying the first frame into an image containing a sea-land background due to the existence of a land background in the first frame, entering a multi-target tracking module containing the sea-land background in the subsequent sequence, and detecting two ship targets in the first frame to serve as template information of a Simese-RPN network.
Third, the score values around the center point at the fourth frame are [0.1,0.12,0.09,0.15,0.11,0.13,0.13,0.07, respectively]And if the standard deviation is smaller than the set threshold, the score values are relatively close, and the tracking loss phenomenon is considered to exist. And when the tracking loss phenomenon caused by scene switching occurs, returning to the target detection and sea-land background classification module. And (4) re-detecting and classifying the frame, classifying the frame into an image of a pure ocean background because the land background does not exist in the frame, and entering a multi-target tracking module under the pure ocean background by the subsequent sequence. As shown in fig. 3, in the tracking process, we only need to perform a correlation operation on the image of the frame and the current frame, and use the corresponding coordinate of the largest point in the obtained response result as the position of the target in the current frame. In the filter initialization module, feature extraction f of the first frame0And the ideal confidence map g of the Gaussian distribution0Given that, naturally, an initialized filter template h can be obtained0. In the filter update module, the MOSSE algorithm is advantageous in that the filter template can utilize a customized confidence map giTraining and updating on line reduce the possibility of target loss. One training includes two samples, the first sample is to locate the previous one in the current frame i +1The frame i target position is sampled and the response is obtained by a filter, the response is called as an over response, and the Fourier transform of the over response is G in a regression modeli. Obtaining the pixel location with the maximum response in the current frame by using the transition response, carrying out secondary sampling on the current frame, wherein the target is located at the center of the sampling frame, and the Fourier transform of the sampling result is F in the modeli. From this, H can be obtained. And updating the filter template and entering the next frame.
And fourthly, completing tracking on all sequence frames and outputting a sequence video after visualization processing.
It should be noted that the above-mentioned embodiments are not intended to limit the scope of the present invention, and all equivalent modifications and substitutions based on the above-mentioned technical solutions are within the scope of the present invention as defined in the claims.
Claims (5)
1. A remote sensing ship multi-target tracking method based on background self-selection is characterized by comprising the following steps:
step 1) preprocessing, namely performing operations such as denoising and feature enhancement on an input video sequence, removing interference of weather factors and shooting angles on the sequence, improving image definition and highlighting feature information;
step 2) target detection and sea-land background classification, wherein each frame preprocessed in the step 1) is subjected to target detection, a target detection result is used as initial target information of a subsequent tracking module, meanwhile, semantic segmentation is carried out on the image, and the image is divided into a pure sea background image and an image containing a sea-land background according to a segmentation result of the background;
step 3) tracking targets under different backgrounds, performing target tracking of different methods on the sequence according to the classification result of the step 2), and improving the tracking speed by adopting a related filtering tracking method for the pure ocean background sequence; the mainstream neural network method is adopted for the sequence containing the sea-land background, so that the tracking precision is improved as much as possible;
and 4) a loss return mechanism, wherein a tracking loss detection and return mechanism is arranged on the target tracking module, and a certain frame with tracking loss is returned to the step 2) for re-detection and classification.
2. The remote sensing vessel target tracking method based on background self-selection according to claim 1, wherein in the step 1), a specific flow of video preprocessing is as follows:
a) firstly, solving the minimum value of three color channel intensity values of each pixel point to obtain a dark primary color image of the image by adopting a remote sensing image defogging method based on dark primary color prior;
b) the sequence image is then multiplied by a hanning window, which gradually reduces the pixel values near the edges to zero, so that more emphasis is placed near the center of the object.
3. The remote sensing vessel target tracking method based on background self-selection according to claim 1, wherein in the step 2), the target detection and sea-land background classification method comprises the following steps:
and carrying out target detection and background segmentation on the image by using YOLACT, judging whether the image contains land features according to the result of the background segmentation, if a large number of land features exist, considering that the image belongs to the image containing ocean and land backgrounds, and if not, considering that the frame belongs to the image containing pure ocean backgrounds.
4. The remote sensing vessel target tracking method based on background self-selection according to claim 1, wherein the target tracking method under different backgrounds in the step 3) is as follows:
a) and for the sequence of the pure ocean background, a MOSSE tracker based on correlation filtering is adopted for tracking multiple targets, and the result of target detection in the step 2) is used as the initial target characteristic of the MOSSE.
b) And for sequences containing sea-land backgrounds, performing multi-target tracking by adopting a Siamese-RPN network, and performing corresponding multi-thread synchronous tracking on the classification branch and the regression branch respectively. And taking the result of the target detection in the step 2) as a template frame of the Siamese-RPN.
5. The remote sensing vessel target tracking method based on background self-selection according to claim 1, wherein the specific flow of the loss return mechanism in the step 4) is as follows:
a) in Siamese-RPN, fractional values score for various positions around the center pointiWhen the video environment is very close to the original video environment, the video environment is considered to be obviously changed or the tracking is lost, at the moment, an alarm of the tracking loss needs to be sent out, a sea-land detection module is started to update a template frame, the dispersion degree of all bundingboxes is reflected by a standard deviation sigma, and N represents the number of the bounding boxes counted and sorted near the central point;
b) in the MOSSE, when the distance between the detection frame after the inverse fast fourier transform and the previous frame has a large position change, we consider that there is a tracking loss phenomenon, and at this time, the detection frame will return to the front-end module for reprocessing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110560777.3A CN113255513A (en) | 2021-05-21 | 2021-05-21 | Remote sensing ship target tracking method based on background self-selection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110560777.3A CN113255513A (en) | 2021-05-21 | 2021-05-21 | Remote sensing ship target tracking method based on background self-selection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113255513A true CN113255513A (en) | 2021-08-13 |
Family
ID=77183762
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110560777.3A Pending CN113255513A (en) | 2021-05-21 | 2021-05-21 | Remote sensing ship target tracking method based on background self-selection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113255513A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830879A (en) * | 2018-05-29 | 2018-11-16 | 上海大学 | A kind of unmanned boat sea correlation filtering method for tracking target suitable for blocking scene |
| CN111986233A (en) * | 2020-08-20 | 2020-11-24 | 西安电子科技大学 | Large-scene minimum target remote sensing video tracking method based on feature self-learning |
| CN112053385A (en) * | 2020-08-28 | 2020-12-08 | 西安电子科技大学 | Remote sensing video shielding target tracking method based on deep reinforcement learning |
| CN112308881A (en) * | 2020-11-02 | 2021-02-02 | 西安电子科技大学 | Ship multi-target tracking method based on remote sensing image |
| CN112700476A (en) * | 2021-01-08 | 2021-04-23 | 北京工业大学 | Infrared ship video tracking method based on convolutional neural network |
-
2021
- 2021-05-21 CN CN202110560777.3A patent/CN113255513A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108830879A (en) * | 2018-05-29 | 2018-11-16 | 上海大学 | A kind of unmanned boat sea correlation filtering method for tracking target suitable for blocking scene |
| CN111986233A (en) * | 2020-08-20 | 2020-11-24 | 西安电子科技大学 | Large-scene minimum target remote sensing video tracking method based on feature self-learning |
| CN112053385A (en) * | 2020-08-28 | 2020-12-08 | 西安电子科技大学 | Remote sensing video shielding target tracking method based on deep reinforcement learning |
| CN112308881A (en) * | 2020-11-02 | 2021-02-02 | 西安电子科技大学 | Ship multi-target tracking method based on remote sensing image |
| CN112700476A (en) * | 2021-01-08 | 2021-04-23 | 北京工业大学 | Infrared ship video tracking method based on convolutional neural network |
Non-Patent Citations (3)
| Title |
|---|
| LI BO 等: "High Performance Visual Tracking with Siamese Region Proposal Network", 《IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》, 16 December 2018 (2018-12-16), pages 8971 - 8980 * |
| YANG XUE 等: "Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network", 《IEEE ACCESS》, vol. 6, 13 September 2018 (2018-09-13), pages 50839 - 50849 * |
| 冼允廷 等: "基于深度学习的多船舶目标跟踪与流量统计", 《微型电脑应用》, vol. 36, no. 3, 20 March 2020 (2020-03-20), pages 11 - 14 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111626290B (en) | Infrared ship target detection and identification method under complex sea surface environment | |
| Lipschutz et al. | New methods for horizon line detection in infrared and visible sea images | |
| CN110458160A (en) | A kind of unmanned boat waterborne target recognizer based on depth-compression neural network | |
| CN114418953B (en) | Sonar target detection method and system based on time history accumulated image | |
| Yu et al. | Object detection-tracking algorithm for unmanned surface vehicles based on a radar-photoelectric system | |
| Wang et al. | Ship detection in foggy remote sensing image via scene classification R-CNN | |
| CN114764801A (en) | Weak and small ship target fusion detection method and device based on multi-vision significant features | |
| Bai et al. | The generalized detection method for the dim small targets by faster R-CNN integrated with GAN | |
| CN115393734A (en) | SAR image ship contour extraction method based on fast R-CNN and CV model combined method | |
| Huang et al. | Identity recognition on waterways: A novel ship information tracking method based on multimodal data | |
| CN115273154A (en) | Thermal infrared pedestrian detection method and system based on edge reconstruction and storage medium | |
| Sun et al. | IRDCLNet: Instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes | |
| Huang et al. | Survey of target detection algorithms in SAR images | |
| CN106991682B (en) | Automatic port cargo ship extraction method and device | |
| Fu et al. | Real-time infrared horizon detection in maritime and land environments based on hyper-laplace filter and convolutional neural network | |
| Zhang et al. | Saliency-based centernet for ship detection in sar images | |
| CN113255513A (en) | Remote sensing ship target tracking method based on background self-selection | |
| Yang et al. | Foreground enhancement network for object detection in sonar images | |
| Wei et al. | Remote sensing image aircraft detection technology based on deep learning | |
| CN113850783B (en) | Sea surface ship detection method and system | |
| CN116188944A (en) | Infrared dim target detection method based on Swin-transducer and multi-scale feature fusion | |
| CN111008555B (en) | Unmanned aerial vehicle image small and weak target enhancement extraction method | |
| Zhang et al. | Ship detection based on improved YOLO algorithm | |
| CN114332682A (en) | Marine panoramic defogging target identification method | |
| Brosch et al. | Automatic target recognition on high resolution sar images with deep learning domain adaptation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |