CN114782487A - Sea surface ship detection tracking method and system - Google Patents
Sea surface ship detection tracking method and system Download PDFInfo
- Publication number
- CN114782487A CN114782487A CN202210303693.6A CN202210303693A CN114782487A CN 114782487 A CN114782487 A CN 114782487A CN 202210303693 A CN202210303693 A CN 202210303693A CN 114782487 A CN114782487 A CN 114782487A
- Authority
- CN
- China
- Prior art keywords
- detection
- sea
- tracking
- sea surface
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 340
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 56
- 238000012805 post-processing Methods 0.000 claims abstract description 14
- 239000013598 vector Substances 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 14
- 238000007689 inspection Methods 0.000 claims description 7
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 238000000605 extraction Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 3
- 238000012512 characterization method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a sea surface ship detection tracking method and a system, comprising the following steps: acquiring a navigation video frame of a sea surface ship; inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set; and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result. The invention improves the anchor-free paradigm detection model, improves the detection performance, obtains the appearance characteristic extraction capability, appoints the detection model to complete sea level line detection, utilizes the double matching strategy of large and small targets, fully utilizes the appearance characteristic of the target to carry out detection tracking, and improves the precision of the sea surface ship detection tracking algorithm.
Description
Technical Field
The invention relates to the technical field of target detection, in particular to a sea surface ship detection tracking method and system.
Background
With the development of artificial intelligence technology and computer vision technology, the application in sea area monitoring scenes is more and more, and the corresponding detection effect requirement is higher and higher.
The ship multi-target video tracking means that the navigation track of each ship is calculated in a given ship video so as to assist in completing a maritime supervision task. Generally, in an important water source protection area, a ship multi-target video tracking technology is adopted, so that the track of a ship can be effectively tracked and predicted, and the ship can be reminded of driving away from a protection area in time. At present, a multi-target tracking algorithm is mainly based on a mode of combining detection and tracking, the method often has the problem of switching of target IDs, and even if an appearance characteristic description model is introduced, due to the fact that one more model is introduced, the real-time performance of the algorithm is reduced.
Therefore, how to overcome the above problems and improve the accuracy of ship multi-target video tracking becomes a problem to be solved urgently.
Disclosure of Invention
The invention provides a sea surface ship detection tracking method and a system, which are used for solving the defects of low precision and high false detection rate in the prior art when a traditional target detection method is adopted for sea surface ship detection.
In a first aspect, the present invention provides a method for detecting and tracking a sea-surface vessel, comprising:
acquiring a navigation video frame of a sea surface ship;
inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set;
and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
According to the sea surface ship detection and tracking method provided by the invention, the sea surface ship target detection model is obtained through the following steps:
acquiring the marked video frame training set;
determining an anchor-free normal form detection model without anchor nodes, and performing multi-task decomposition on the anchor-free normal form detection model based on a decoupling head to obtain a plurality of target detection branch models;
and training the plurality of target detection branch models based on the labeled video frame training set to obtain the sea surface ship target detection model.
According to the sea surface ship detection and tracking method provided by the invention, the acquiring of the labeled video frame training set comprises the following steps:
acquiring a plurality of sea surface navigation video frames in different time periods and/or different positions;
and determining sea level line labels and ship labels of the plurality of sea surface navigation video frames, and determining that the same ship between different videos in the same video is the same ship label.
According to the sea surface ship detection tracking method provided by the invention, the anchorless node anchor-free normal form detection model is determined, and the anchor-free normal form detection model is subjected to multi-task decomposition based on a decoupling head to obtain a plurality of target detection branch models, and the method comprises the following steps:
acquiring a feature map of the anchor-free normal form detection model;
and adding a plurality of preset convolution kernels to the feature map to obtain a sea level line position branch model, a target position branch model and a target appearance characteristic branch model.
According to the sea surface vessel detection and tracking method provided by the present invention, the training of the multiple target detection branch models based on the sea level line position information, the sea level line extraction threshold and the labeled video frame training set to obtain the sea surface vessel target detection model comprises:
determining coordinates of the upper left corner of a sea horizon line detection frame in the sea horizon line position information, the width of the sea horizon line detection frame and the height of the sea horizon line detection frame;
acquiring a detection result of each ship target in the marked video frame training set, and determining a coordinate of the lower right corner of a ship detection frame of each ship target detection result;
determining a preset protection threshold, and if the sum of the ordinate of the coordinate at the lower right corner of the ship detection frame and the preset protection threshold is judged to be smaller than the ordinate of the coordinate at the upper left corner of the sea horizon detection frame, deleting the current ship target detection result;
and traversing all ship target detection results in the marked video frame training set to obtain the sea surface ship target detection model.
According to the sea surface ship detection and tracking method provided by the invention, the sea surface ship detection result is post-processed based on the sea level line detection result to obtain the sea surface ship detection and tracking result, and the method comprises the following steps:
determining a detection frame area threshold, and dividing the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold;
tracking and matching the preset large target detection set based on the appearance characteristic vector, tracking and matching the preset small target detection set based on the template matching characteristic to obtain a current detection frame matching and tracking object, putting the current detection matching and tracking object into a matching success queue, and putting an unmatched object into a matching failure queue;
performing secondary matching on the detection frame in the matching failure queue and the tracking object which is not successfully matched based on the cross-over ratio IOU, updating the tracking object if the matching is successful, and otherwise, putting the current tracking result into a matching candidate queue;
if the number of the lost frames of the detection result in the matching failure queue is judged to continuously exceed a preset frame threshold value, deleting the corresponding tracking object;
if the confidence coefficient of the detection result in the matching candidate queue exceeds a preset confidence coefficient threshold value and the corresponding frame number continuously exceeds a preset frame number, determining the corresponding tracking object as a new tracking object;
and storing the subgraph corresponding to each current video frame in all the tracked objects to obtain the position information of the next frame.
According to the sea surface ship detection tracking method provided by the invention, the tracking matching is carried out on the preset large target detection set based on the appearance characteristic vector, and the tracking matching is carried out on the preset small target detection set based on the template matching characteristic, so as to obtain the current detection frame matching tracking object, and the method comprises the following steps:
comparing the detection result of the current frame in the preset large target detection set with all the tracking objects stored in the previous frame one by one based on the appearance feature vector to obtain a feature vector comparison result, and if the Euclidean distance of the feature vector comparison result is smaller than a preset distance threshold and is the minimum distance, obtaining that the current detection frame matches with the tracking object;
and performing template matching on the image of the area corresponding to the detection result of the current frame in the preset small target detection set and the images of the areas corresponding to all the tracking objects stored in the previous frame based on the template matching characteristics to obtain the similarity between the images, and if the similarity between the images is greater than a preset similarity threshold and is optimal matching, obtaining the tracking object matched with the current detection frame.
In a second aspect, the present invention also provides a sea-surface vessel detection and tracking system, comprising:
the acquisition module is used for acquiring a navigation video frame of a sea surface ship;
the detection module is used for inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set;
and the tracking module is used for post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
In a third aspect, the present invention also provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods for vessel detection and tracking as described above.
In a fourth aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for sea surface vessel detection and tracking as described in any one of the above.
In a fifth aspect, the present invention also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the method for sea vessel detection and tracking as described in any one of the above.
According to the sea surface ship detection tracking method and system, the anchor-free paradigm detection model is improved, the detection performance is improved, the appearance feature extraction capability is obtained, the detection model is appointed to complete sea level line detection, the double matching strategy of large and small targets is utilized, the target appearance features are fully utilized to carry out detection tracking, and the precision of a sea surface ship detection tracking algorithm is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow diagram of a sea-surface vessel inspection and tracking method provided by the present invention;
FIG. 2 is a second schematic flow chart of the method for detecting and tracking a sea surface vessel according to the present invention;
FIG. 3 is a schematic structural diagram of a decoupling head of a sea surface ship detection model provided by the invention;
FIG. 4 is a schematic structural diagram of a sea surface vessel detection and tracking system provided by the present invention;
fig. 5 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Fig. 1 is a schematic flow chart of a method for detecting and tracking a sea vessel according to the present invention, as shown in fig. 1, including:
step S1, acquiring a navigation video frame of a sea surface ship;
step S2, inputting the navigation video frame of the sea surface ship into a pre-trained sea surface ship target detection model to obtain a sea surface ship detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on a labeled video frame training set;
and step S3, post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
Specifically, as shown in fig. 2, a sea vessel navigation video frame to be detected is determined from a sea vessel navigation video, and the sea vessel navigation video frame is input into a trained sea vessel target detection model, that is, into a target detection module, to obtain a sea vessel detection result.
It should be noted that the sea surface ship target detection model is characterized in that a plurality of labels are carried out on a plurality of video frame training sets, a decoupling head design is adopted, multitask branch improvement is carried out on the target detection model to obtain a plurality of branch detection models, the performance of the detection models is improved, model training is completed by adopting a data set for collecting the labels, then post-processing is carried out on a ship target detection result according to a sea horizon detection result, namely the detection result is input into a tracking module in fig. 2, the state updating of a tracked object and the addition of a new tracked object are completed, an N frame image returned by the tracking module and an image obtained after the initialization of the tracked object are input into the target detection module again, a detection frame is introduced for carrying out large and small target classification detection, a double-target matching strategy is adopted, and the appearance characteristics of the target are used for detection and tracking.
The invention uses an improved anchor-free paradigm detection model and introduces a decoupling head to carry out multi-task regression, thereby improving the performance of the model, trains the model through collected data with labels, the trained model has the ship target detection, the appearance characteristics of the ship target and the sea level line detection capability, then carries out post-processing on the ship detection result through sea level line position information, eliminates the targets above the sea level line, detects the false alarm rate, improves the detection performance, matches the detected ship target according to a designed tracking strategy, and finally completes the detection and tracking of the sea surface ship target.
Based on the above embodiment, the sea surface vessel target detection model is obtained by the following steps:
acquiring the marked video frame training set;
determining an anchor-free normal form detection model without anchor nodes, and performing multi-task decomposition on the anchor-free normal form detection model based on a decoupling head to obtain a plurality of target detection branch models;
and training the plurality of target detection branch models based on the labeled video frame training set to obtain the sea surface ship target detection model.
Specifically, the sea surface ship target detection model constructed by the invention adopts a decoupling head to carry out multi-branch task prediction appearance characteristics on the basis of an anchor-free normal mode detection model to obtain a plurality of prediction branches, so that the decoupling of the prediction branches is realized, and each branch model is responsible for predicting different characteristics;
and simultaneously, setting inference tasks of the model as ship target position detection, ship target appearance characteristic description and sea horizon detection, and training the improved anchor-free mode detection model to obtain the sea surface ship target detection model by taking sea horizon position information and a sea horizon extraction threshold as the basis and taking a marked video frame training set as the input of the model in the inference training process.
According to the invention, through the improved anchor-free paradigm detection model, the detection performance is improved, the appearance feature extraction capability is obtained, and the sea horizon detection is completed while the detection model is set, so that the false detection rate of the model is reduced.
Based on any of the above embodiments, the obtaining the labeled video frame training set includes:
acquiring a plurality of sea surface navigation video frames in different time periods and/or different positions;
and determining the sea level line labels and the ship labels of the plurality of sea surface navigation video frames, and determining the same ship between different videos in the same video as the same ship label.
It should be noted that before the model is trained, a certain amount of training data needs to be collected and acquired, and the sea level line and the ship in each video frame image in the video are respectively marked by collecting a plurality of videos at different time periods and different positions in an actual application scene.
In particular, assigning the same ID number to the same ship target between different video frames in the same video segment can be used for training of appearance characterization.
According to the invention, through labeling different targets on the video frame training set, the appearance characteristic description can be conveniently identified in the training process.
Based on any one of the embodiments, the determining of the anchor-free normal-form detection model without the anchor nodes, and performing multi-task decomposition on the anchor-free normal-form detection model based on the decoupling heads to obtain a plurality of target detection branch models comprise:
acquiring a feature map of the anchor-free paradigm detection model;
and adding a plurality of preset convolution kernels to the feature map to obtain a sea level line position branch model, a target position branch model and a target appearance characteristic branch model.
Specifically, an anchor-free paradigm detection model is adopted, 3x3 convolution kernels are respectively carried out on the featuremap, and 3 prediction branches are obtained, so that prediction branch decoupling is achieved, and the three branches are respectively responsible for predicting sea level line positions, target positions and appearance feature description. The improved detection model and the added appearance characterization based on the decoupled head are shown in fig. 3.
Here, the appearance feature description branch is responsible for generating an Nx 1-dimensional feature vector to represent the target appearance, where N is a hyper-parameter and can be set according to actual requirements.
According to the invention, through improving the traditional target detection model, the detection performance is improved, the appearance feature extraction capability is obtained, and the mutual influence among the multi-task branches is eliminated through the decoupling head.
Based on any of the above embodiments, the training the multiple target detection branch models based on the labeled video frame training set to obtain the sea surface vessel target detection model includes:
determining coordinates of the upper left corner of a sea horizon line detection frame in the sea horizon line position information, the width of the sea horizon line detection frame and the height of the sea horizon line detection frame;
acquiring a detection result of each ship target in the marked video frame training set, and determining a coordinate of the lower right corner of a ship detection frame of each ship target detection result;
determining a preset protection threshold, and if the sum of the ordinate of the coordinate at the lower right corner of the ship detection frame and the preset protection threshold is judged to be smaller than the ordinate of the coordinate at the upper left corner of the sea horizon detection frame, deleting the current ship target detection result;
and traversing all ship target detection results in the marked video frame training set to obtain the sea surface ship target detection model.
Specifically, when the detection model is trained, a sea horizon extraction threshold thresh _ sea is set, and sea horizon position information (x, y, w, h) is obtained, wherein x and y are coordinates of the upper left corner of a sea horizon detection frame, and w and h are the width and the height of the sea horizon detection frame respectively.
And (3) setting the coordinates of the lower right corner of a ship detection frame of each ship target detection result in the marked video frame training set as (xr, yr), setting a preset protection threshold value thresh _ p, and if yr + thresh _ p is less than y, indicating that the current detection result is positioned on the sea horizon, and rejecting the current detection result if the current detection result is false detection.
And traversing each ship target detection result, repeating the operation on all the video frame training sets, and completing the training to obtain the sea surface ship target detection model.
The invention trains the model through the marked labeled data, the trained model has the ship target detection, the ship target appearance characteristic and the sea horizon detection capability, and the ship detection result is post-processed through the sea horizon position information to remove targets above the sea horizon, thereby reducing the detection false alarm rate and improving the detection performance.
Based on any of the above embodiments, the performing post-processing on the sea surface vessel detection result based on the sea horizon detection result to obtain a sea surface vessel detection tracking result includes:
determining a detection frame area threshold, and dividing the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold;
tracking and matching the preset large target detection set based on the appearance characteristic vector, tracking and matching the preset small target detection set based on the template matching characteristic to obtain a current detection frame matching and tracking object, putting the current detection matching and tracking object into a matching success queue, and putting an unmatched object into a matching failure queue;
performing secondary matching on the detection frame in the matching failure queue and the tracking object which is not successfully matched based on the cross-over ratio IOU, updating the tracking object if the matching is successful, and otherwise, putting the current tracking result into a matching candidate queue;
if the number of the lost frames of the detection result in the matching failure queue continuously exceeds a preset frame threshold value, deleting the corresponding tracking object;
if the confidence coefficient of the detection result in the matching candidate queue exceeds a preset confidence coefficient threshold value and the corresponding frame number continuously exceeds a preset frame number, determining the corresponding tracking object as a new tracking object;
and storing the subgraph corresponding to each current video frame in all the tracked objects to obtain the position information of the next frame.
The tracking and matching of the preset large target detection set based on the appearance feature vector and the tracking and matching of the preset small target detection set based on the template matching feature to obtain a current detection frame matching tracking object comprise:
comparing the detection result of the current frame in the preset large target detection set with all the tracking objects stored in the previous frame one by one based on the appearance characteristic vector to obtain a characteristic vector comparison result, and if the Euclidean distance of the characteristic vector comparison result is smaller than a preset distance threshold and is the minimum distance, obtaining that the current detection frame matches the tracking object;
and matching the image of the corresponding area of the detection result of the current frame in the preset small target detection set with the images of the corresponding areas of all the tracking objects stored in the previous frame based on the template matching characteristics to obtain the similarity between the images, and if the similarity between the images is greater than a preset similarity threshold and is optimal matching, obtaining the matching tracking object of the current detection frame.
Specifically, after the detection result is obtained, the detection result is input into the tracking module to complete the tracking object state and add a new tracking object, and the method specifically includes the following steps:
setting a threshold value thresh _ area of a detection frame, dividing a detection result into a preset large target detection set and a preset small target detection set according to the area size of the detection frame, wherein the preset large target detection set with the area larger than thresh _ area is called detect _ app, and the preset small target detection set with the area smaller than thresh _ area is called detect _ tem.
The detection frames in the preset large target detection set detect _ app have large areas and contain rich characteristic information, the detection result and all tracking objects stored in the previous needle are compared one by one according to the appearance characteristic vectors, if the Euclidean distance between the appearance characteristic vectors is smaller than a set threshold and is the minimum distance, the appearance characteristic vectors are successfully matched, the states of the tracking objects are updated, otherwise, the tracking objects matched with the current detection frame are not found, the current detection result is placed in a matching failure queue detect _ lost, and the matching failure queue detect _ lost is used for matching in the subsequent steps.
And the detection frames in the preset small target detection set detect _ tem are lack of feature information because of small area, and are easy to cause mismatching, so the invention adopts template matching features for matching, carries out template feature matching on the image in the area corresponding to the detection result and the images corresponding to all the tracking objects stored in the previous frame, if the similarity between the two images is greater than a set threshold and is optimal matching, the two images are successfully matched, updates the states of the tracking objects, otherwise, the tracking objects matched with the current detection frame are not found, and puts the current detection result into a matching failure queue detect _ lost for matching use in subsequent steps.
Further, performing secondary matching on the detection frame which is not successfully matched in the match failure queue detect _ lost and the tracking object which is not successfully matched based on an IOU (Intersection over Unit), updating the tracking object if the matching is successful, and putting the current detection result into a match candidate queue detect _ candidate if the matching is failed; and defining the state of the tracking object which is not successfully matched as lost, and deleting the tracking object if the number of the lost frames continuously exceeds a preset frame threshold value thresh _ buffer, wherein the thresh _ buffer is a super parameter and can be set by self.
If the confidence of the detection result of the matching candidate queue detect _ candidate exceeds the preset confidence threshold thresh _ new, the target may be a newly appeared target, a new tracking object is constructed based on the detection result, and if the object continuously appears for more than a preset number of frames, if the number of frames is set to be 3, the new tracking object is determined.
And finally, storing all the tracked objects in the subgraph corresponding to each subframe, predicting the position information of the next frame, and using the position information for the tracking and matching work of the next frame.
According to the invention, through designing a double matching strategy of large and small targets, the appearance characteristics of the targets are fully utilized for detection and tracking, and the detected ship targets are subjected to matching iteration according to the designed tracking strategy, so that the detection and tracking of the ship targets on the sea surface are finally completed.
The following describes the system for detecting and tracking a marine vessel on the sea according to the present invention, and the system for detecting and tracking a marine vessel on the sea described below and the method for detecting and tracking a marine vessel on the sea described above can be referred to correspondingly.
Fig. 4 is a schematic structural diagram of a sea vessel detection and tracking system provided by the present invention, as shown in fig. 4, including: an acquisition module 41, a detection module 42, and a tracking module 43, wherein:
the obtaining module 41 is configured to obtain a navigation video frame of a sea surface ship; the detection module 42 is configured to input the sea vessel sailing video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on a labeled video frame training set; the tracking module 43 is configured to perform post-processing on the sea surface ship detection result based on the sea horizon detection result to obtain a sea surface ship detection tracking result.
The method improves the anchor-free paradigm detection model, improves the detection performance, obtains the appearance feature extraction capability, appoints the detection model to complete sea level line detection, fully utilizes the appearance features of the targets to carry out detection and tracking by utilizing a double matching strategy of large and small targets, and improves the precision of a sea surface ship detection and tracking algorithm.
Fig. 5 illustrates a physical structure diagram of an electronic device, which may include, as shown in fig. 5: a processor (processor)510, a communication Interface (Communications Interface)520, a memory (memory)530 and a communication bus 540, wherein the processor 510, the communication Interface 520 and the memory 530 communicate with each other via the communication bus 540. Processor 510 may invoke logic instructions in memory 530 to perform a method for sea vessel detection tracking, the method comprising: acquiring a navigation video frame of a sea surface ship; inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set; and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
Furthermore, the logic instructions in the memory 530 may be implemented in the form of software functional units and stored in a computer readable storage medium when the software functional units are sold or used as independent products. Based on such understanding, the technical solution of the present invention or a part thereof which substantially contributes to the prior art may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, and various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product comprising a computer program, the computer program being stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is capable of executing the method for sea surface vessel detection and tracking provided by the above methods, the method comprising: acquiring a navigation video frame of a sea surface ship; inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set; and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program, when executed by a processor, being implemented to perform the method for sea vessel detection and tracking provided by the above methods, the method comprising: acquiring a navigation video frame of a sea surface ship; inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set; and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A method for sea surface vessel inspection tracking, comprising:
acquiring a navigation video frame of a sea surface ship;
inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on a labeled video frame training set;
and post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
2. The method for sea surface vessel inspection and tracking of claim 1, wherein the sea surface vessel object inspection model is obtained by:
acquiring the marked video frame training set;
determining an anchor-free normal form detection model without anchor nodes, and performing multi-task decomposition on the anchor-free normal form detection model based on a decoupling head to obtain a plurality of target detection branch models;
and training the plurality of target detection branch models based on the labeled video frame training set to obtain the sea surface ship target detection model.
3. The method for sea-surface vessel inspection and tracking of claim 2, wherein the obtaining the annotated training set of video frames comprises:
acquiring a plurality of sea surface navigation video frames in different time periods and/or different positions;
and determining sea level line labels and ship labels of the plurality of sea surface navigation video frames, and determining that the same ship between different videos in the same video is the same ship label.
4. The sea surface vessel detection and tracking method of claim 2, wherein the determining of the anchorless node anchor-free normal form detection model is performed on the anchor-free normal form detection model through multitask decomposition based on a decoupling head to obtain a plurality of target detection branch models, and the method comprises the following steps of:
acquiring a feature map of the anchor-free normal form detection model;
and adding a plurality of preset convolution kernels to the feature map to obtain a sea level line position prediction branch model, a target position prediction branch model and a target appearance characteristic branch model.
5. The method for detecting and tracking a sea surface vessel of claim 2, wherein training the plurality of target detection branch models based on the annotated video frame training set to obtain the sea surface vessel target detection model comprises:
determining the coordinate of the upper left corner of a sea horizon detection frame in the sea horizon position information, the width of the sea horizon detection frame and the height of the sea horizon detection frame;
acquiring a detection result of each ship target in the marked video frame training set, and determining a coordinate of the lower right corner of a ship detection frame of each ship target detection result;
determining a preset protection threshold, and if the sum of the ordinate of the coordinate at the lower right corner of the ship detection frame and the preset protection threshold is judged to be smaller than the ordinate of the coordinate at the upper left corner of the sea horizon detection frame, deleting the current ship target detection result;
and traversing all ship target detection results in the marked video frame training set to obtain the sea surface ship target detection model.
6. The sea surface vessel detection and tracking method of claim 1, wherein the performing post-processing on the sea surface vessel detection result based on the sea horizon detection result to obtain a sea surface vessel detection and tracking result comprises:
determining a detection frame area threshold, and dividing the sea surface ship detection result into a preset large target detection set and a preset small target detection set based on the detection frame area threshold;
tracking and matching the preset large target detection set based on the appearance characteristic vector, tracking and matching the preset small target detection set based on the template matching characteristic to obtain a current detection frame matching tracking object, putting the current detection matching tracking object into a matching success queue, and putting an unmatched object into a matching failure queue;
performing secondary matching on the detection frame in the matching failure queue and the tracking object which is not successfully matched based on the cross-over ratio IOU, updating the tracking object if the matching is successful, and otherwise, putting the current tracking result into a matching candidate queue;
if the number of the lost frames of the detection result in the matching failure queue continuously exceeds a preset frame threshold value, deleting the corresponding tracking object;
if the confidence coefficient of the detection result in the matching candidate queue exceeds a preset confidence coefficient threshold value and the corresponding frame number continuously exceeds a preset frame number, determining the corresponding tracked object as a new tracked object;
and storing the subgraph corresponding to each current video frame in all the tracked objects to obtain the position information of the next frame.
7. The sea surface vessel detection and tracking method according to claim 6, wherein the tracking and matching the preset large target detection set based on the appearance feature vector and the tracking and matching the preset small target detection set based on the template matching feature to obtain the current detection frame matching tracking object comprises:
comparing the detection result of the current frame in the preset large target detection set with all the tracking objects stored in the previous frame one by one based on the appearance characteristic vector to obtain a characteristic vector comparison result, and if the Euclidean distance of the characteristic vector comparison result is smaller than a preset distance threshold and is the minimum distance, obtaining that the current detection frame matches the tracking object;
and performing template matching on the image of the area corresponding to the detection result of the current frame in the preset small target detection set and the images of the areas corresponding to all the tracking objects stored in the previous frame based on the template matching characteristics to obtain the similarity between the images, and if the similarity between the images is greater than a preset similarity threshold and is optimal matching, obtaining the tracking object matched with the current detection frame.
8. A sea surface vessel inspection and tracking system, comprising:
the acquisition module is used for acquiring a navigation video frame of a sea surface ship;
the detection module is used for inputting the sea vessel navigation video frame into a pre-trained sea vessel target detection model to obtain a sea vessel detection result; the sea surface ship target detection model is obtained by training a target detection model based on multi-task branch improvement based on an annotated video frame training set;
and the tracking module is used for post-processing the sea surface ship detection result based on the sea level line detection result to obtain a sea surface ship detection tracking result.
9. An electronic device comprising a memory, a processor and a computer program stored on said memory and executable on said processor, wherein said processor when executing said program performs the steps of the method of sea vessel detection and tracking according to any of claims 1 to 7.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the steps of the method for sea surface vessel inspection and tracking according to any of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210303693.6A CN114782487A (en) | 2022-03-24 | 2022-03-24 | Sea surface ship detection tracking method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210303693.6A CN114782487A (en) | 2022-03-24 | 2022-03-24 | Sea surface ship detection tracking method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114782487A true CN114782487A (en) | 2022-07-22 |
Family
ID=82425224
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210303693.6A Pending CN114782487A (en) | 2022-03-24 | 2022-03-24 | Sea surface ship detection tracking method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114782487A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116091552A (en) * | 2023-04-04 | 2023-05-09 | 上海鉴智其迹科技有限公司 | Target tracking method, device, equipment and storage medium based on deep SORT |
| CN116246205A (en) * | 2023-02-27 | 2023-06-09 | 杭州数尔安防科技股份有限公司 | Optimization method and system for ship tracking and character recognition |
| CN116823838A (en) * | 2023-08-31 | 2023-09-29 | 武汉理工大学三亚科教创新园 | Ocean ship detection method and system with Gaussian prior label distribution and characteristic decoupling |
-
2022
- 2022-03-24 CN CN202210303693.6A patent/CN114782487A/en active Pending
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116246205A (en) * | 2023-02-27 | 2023-06-09 | 杭州数尔安防科技股份有限公司 | Optimization method and system for ship tracking and character recognition |
| CN116246205B (en) * | 2023-02-27 | 2024-04-19 | 杭州数尔安防科技股份有限公司 | Optimization method and system for ship tracking and character recognition |
| CN116091552A (en) * | 2023-04-04 | 2023-05-09 | 上海鉴智其迹科技有限公司 | Target tracking method, device, equipment and storage medium based on deep SORT |
| CN116823838A (en) * | 2023-08-31 | 2023-09-29 | 武汉理工大学三亚科教创新园 | Ocean ship detection method and system with Gaussian prior label distribution and characteristic decoupling |
| CN116823838B (en) * | 2023-08-31 | 2023-11-14 | 武汉理工大学三亚科教创新园 | Ocean ship detection method and system with Gaussian prior label distribution and characteristic decoupling |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114782487A (en) | Sea surface ship detection tracking method and system | |
| US11878433B2 (en) | Method for detecting grasping position of robot in grasping object | |
| Li et al. | Contour knowledge transfer for salient object detection | |
| US10936911B2 (en) | Logo detection | |
| Liang et al. | Interpretable structure-evolving LSTM | |
| CN111161311A (en) | Visual multi-target tracking method and device based on deep learning | |
| CN108805871B (en) | Blood vessel image processing method and device, computer equipment and storage medium | |
| CN110349147B (en) | Model training method, fundus macular region lesion recognition method, device and equipment | |
| JP2019517701A (en) | Method and system for detecting an object in an image | |
| CN110555901A (en) | Method, device, equipment and storage medium for positioning and mapping dynamic and static scenes | |
| CN113255611B (en) | Twin network target tracking method based on dynamic label distribution and mobile equipment | |
| CN113628245B (en) | Multi-target tracking method, device, electronic equipment and storage medium | |
| Wang et al. | Point linking network for object detection | |
| CN111354022B (en) | Target Tracking Method and System Based on Kernel Correlation Filtering | |
| CN110738688B (en) | Infrared ultra-weak moving target detection method | |
| CN114926726B (en) | Unmanned ship sensing method based on multitask network and related equipment | |
| CN108345821A (en) | Face tracking method and apparatus | |
| CN111091101A (en) | High-precision pedestrian detection method, system and device based on one-step method | |
| CN112037146A (en) | Medical image artifact automatic correction method and device and computer equipment | |
| CN114118303B (en) | Face key point detection method and device based on prior constraint | |
| CN113689348B (en) | Method, system, electronic device and storage medium for restoring multi-task image | |
| CN109063567B (en) | Human body recognition method, human body recognition device and storage medium | |
| CN116704264B (en) | Animal classification method, classification model training method, storage medium, and electronic device | |
| CN110799984A (en) | Tracking control method, device and computer readable storage medium | |
| CN116778486A (en) | Point cloud segmentation method, device, equipment and medium of angiography image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |