CN117011768B - Multi-ship target detection and tracking method based on robust data association - Google Patents
Multi-ship target detection and tracking method based on robust data association Download PDFInfo
- Publication number
- CN117011768B CN117011768B CN202310983360.7A CN202310983360A CN117011768B CN 117011768 B CN117011768 B CN 117011768B CN 202310983360 A CN202310983360 A CN 202310983360A CN 117011768 B CN117011768 B CN 117011768B
- Authority
- CN
- China
- Prior art keywords
- target
- ship
- tracking
- result
- detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 88
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000005259 measurement Methods 0.000 claims abstract description 20
- 239000011159 matrix material Substances 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims abstract description 12
- 230000008569 process Effects 0.000 claims description 16
- 238000001914 filtration Methods 0.000 claims description 5
- 230000007704 transition Effects 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 2
- 238000010276 construction Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 5
- 230000001788 irregular Effects 0.000 description 5
- 238000013461 design Methods 0.000 description 3
- 238000012360 testing method Methods 0.000 description 3
- 238000000691 measurement method Methods 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 241001239379 Calophysus macropterus Species 0.000 description 1
- 101000642315 Homo sapiens Spermatogenesis-associated protein 17 Proteins 0.000 description 1
- 102100036408 Spermatogenesis-associated protein 17 Human genes 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000007123 defense Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000012163 sequencing technique Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/70—Labelling scene content, e.g. deriving syntactic or semantic representations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Software Systems (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Mathematical Physics (AREA)
- Mathematical Optimization (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Pure & Applied Mathematics (AREA)
- Computational Linguistics (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- Algebra (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multi-ship target detection and tracking method based on robust data association, which comprises the following steps: constructing a target detection model based on a loss function of the robustness data association measurement and training; acquiring a marine video, and detecting the marine video based on the trained target detection model to acquire a detection result of a ship target; designing a state prediction algorithm, and obtaining a prediction result of a ship target based on the state prediction algorithm; constructing a tracking matching strategy, acquiring robustness data association measurement of the detection result and the prediction result, and further acquiring a similarity matrix; and inputting the similarity matrix into a Hungary algorithm to obtain a ship target tracking result. The multi-ship target detection and tracking method provided by the invention is more stable, simpler and higher in precision, and is suitable for the problem of multi-ship tracking.
Description
Technical Field
The invention belongs to the technical field of image target tracking, and particularly relates to a multi-ship target detection and tracking method based on robust data association.
Background
Multi-target tracking technology based on computer vision and image processing has proven to play an important role in the field of marine monitoring, and the motion trail of a ship can be analyzed to avoid collision accidents and become an integral part. However, the multi-vessel target tracking problem has its specificity, unlike common multi-target tracking contexts (e.g., pedestrians and vehicles). Due to objective limitations such as irregular motion interference of ocean currents and low-frame-rate images shot by ship-borne cameras, the current excellent multi-target tracking method cannot be directly suitable for multi-ship target tracking. Therefore, there is an urgent need to propose a data correlation measurement method suitable for ship target tracking without subverting the existing multi-target tracking results.
Disclosure of Invention
The invention aims to provide a multi-ship target detection and tracking method based on robust data association, which not only realizes faster convergence and higher precision in ship target detection based on the measurement of robust data association, but also compensates the problem of inaccurate prediction of a Kalman filter caused by irregular motion interference of ocean currents and low-frame-rate images shot by a ship-borne camera in a tracking stage, and can solve the problem of frequent target ID loss of ship tracking caused by objective limitations such as irregular motion interference of ocean currents and low-frame-rate images shot by the ship-borne camera in the current multi-ship target tracking.
In order to achieve the above purpose, the invention provides a multi-ship target detection and tracking method based on robust data association, comprising the following steps:
constructing a target detection model based on a loss function of the robustness data association measurement and training;
Acquiring a marine video, and detecting the marine video based on the trained target detection model to acquire a detection result of a ship target;
designing a state prediction algorithm, and obtaining a prediction result of a ship target based on the state prediction algorithm;
constructing a tracking matching strategy, acquiring robustness data association measurement of the detection result and the prediction result, and further acquiring a similarity matrix;
And inputting the similarity matrix into a Hungary algorithm to obtain a ship target tracking result.
Optionally, the constructing of the loss function of the robustness data association metric includes: based on a detection result and a prediction result of a ship target, acquiring a minimum convex shape comprising the detection result and the prediction result; and obtaining the ratio of the detection result and the prediction result to the minimum convex respectively, sequencing, selecting the minimum value of the ratio, assigning the minimum value of the ratio to the robustness data association measurement, and further obtaining a loss function based on the robustness data association measurement.
Optionally, the training the target detection model includes: marking the acquired maritime videos as a detection data set, and performing data enhancement processing on the detection data set; training the target detection model based on the processed detection data set.
Optionally, the process of designing the state prediction algorithm includes: acquiring a boundary frame of a ship target in each frame of image in a maritime video, acquiring process noise at a preset moment and a state transition matrix of a system based on a Kalman filtering algorithm, and constructing a motion state equation of the boundary frame; based on the motion state equation of the bounding box, measuring noise and an observation matrix at the next moment of the preset moment are obtained, an observation equation of the system is obtained, and then a state prediction algorithm is obtained.
Optionally, the process of obtaining a predicted result of the ship target includes: acquiring initial error covariance estimation and posterior estimation of an initial target state space, and respectively acquiring prior estimation of error covariance and prior estimation of a target state at preset time based on a state prediction algorithm, namely a ship target prediction result; based on the prior estimation of the error covariance, obtaining an optimal Kalman gain, carrying out data fusion on a predicted result and an actual detection result of a ship target based on the optimal Kalman gain, and obtaining posterior estimation of a target state space at the next moment of a preset moment through iterative updating; and inputting the posterior estimation of the target state space at the next moment into the motion state equation of the boundary box, and continuously obtaining the prior estimation of the target state space at the next moment, so that the prior estimation is iteratively updated until a final ship target prediction result is obtained.
Optionally, the process of constructing the tracking matching policy includes: and presetting a ship tracking frame, detecting an image of the marine video at a preset moment based on a trained target detection model, initializing a Kalman filter based on a detection result, and designing a state prediction algorithm based on the initialized Kalman filter to further obtain a prediction result of a ship target at the next moment of the preset moment.
Optionally, the process of obtaining the ship target tracking result further includes: and designing a target disappearing stopping tracking strategy, and judging that the target disappears when the continuous 60-frame images of the running track of the ship cannot be successfully associated, and stopping updating the running track.
The invention has the technical effects that:
(1) The invention improves and innovates on the basis of the traditional data association mode, and provides a multi-ship target detection and tracking method based on robust data association, wherein the robust data association measurement method is a target detection weight network with high training speed and higher precision in a target detection stage, and can solve the problem of accuracy of a Kalman filter prediction result caused by objective difficulties such as irregular disturbance of ocean currents, low frame rate of images shot by a ship-based camera and the like, and the problem of frequent switching of tracking IDs. According to the invention, the tracking result is optimized and compensated by considering the distance and the shape between the detection result and the prediction result based on the data association measurement of the tracking design, so that the stability of the tracker is improved.
(2) The robustness data association measurement provided by the invention is popular and clear in structure, is simple and feasible in specific deployment and implementation, and can improve the tracking precision of a related algorithm after being deployed to a target advanced tracker. The multi-ship target detection and tracking method provided by the invention is more stable, simpler and higher in precision, and is suitable for the problem of multi-ship tracking.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
FIG. 1 is an overall frame flow chart of a multi-vessel target detection and tracking method based on robust data correlation in an embodiment of the invention;
FIG. 2 is a detailed flowchart of a tracking matching strategy in a multi-ship target detection and tracking method based on robust data correlation in an embodiment of the present invention;
FIG. 3 is a diagram showing a comparison of a loss function using a robust data correlation metric and an original loss function to a YOLO-v3 convergence rate in an embodiment of the present invention;
fig. 4 is a schematic diagram comparing a robust data correlation metric and a conventional number correlation metric in an embodiment of the present invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
Example 1
As shown in fig. 1-2, the method for detecting and tracking the multi-ship targets based on the robustness data association is provided in the embodiment, and can be applied to the fields of ocean monitoring systems, sea defense, ship situation estimation and the like. Fig. 1 is a block diagram of a multi-ship target tracking method based on robust data association according to this embodiment, and in general, this embodiment includes 2 steps, step S1: inputting the marine image data set into a target detection model for training; step S2: detecting a ship target in the video by using the trained model and designing a robustness matching strategy for tracking;
step S1: based on the existing target detection model, data enhancement operation is firstly carried out on the sea test ship data set, so that the robustness of the data set is improved, and the network detection capability is improved;
The step S1 further comprises the steps of:
Step S11: marking the maritime video sequence collected by Jiangsu automation as a detection data set, wherein the marking format of the detection data set is MOT17 format;
Step S12: performing data enhancement operation on the detection data set marked in the step S11, and improving diversity and robustness of the detection data set;
The data enhancement method is used for randomly cutting four random images of the detection data set and then splicing the random images to one image, so that the detection data set is enriched.
Step S13: correcting and optimizing an original YOLO-v3 target detection network to enable the original YOLO-v3 target detection network to be suitable for multi-ship target detection;
the YOLO-v3 is a high-performance target detection network commonly used in industry.
The step S13 further includes the steps of:
Step S131: replacing the regression loss function of the boundary box in the original YOLO-v3 target detection network with a loss function based on a robustness data association metric TIoU (THE TRACKING version of Intersection of Union), so that faster convergence speed and stronger detection performance are realized in the training process;
Specifically, the loss function based on the robust data correlation metric TIoU is defined as follows:
LossTIoU＝1-TIoU
Wherein B gt＝(xgt,ygt,wgt,hgt) is the ship detection target real value marked in the step S11, B KF＝(xKF,yKF,wKF,hKF) is the prediction result of the ship detection target detection network in the step S13 YOLO-v3, and C is the minimum convex shape containing B gt and B KF. The ratio of B dt to C and B KF to C is obtained; assigning TIoU a smaller value to the ratio; calculating a loss function based on the robust data correlation metric TIoU according to TIoU;
Fig. 3 is a schematic diagram showing a comparison between a loss function based on a robust data correlation metric TIoU and an original loss function to a YOLO-v3 convergence speed in the multi-ship target detection and tracking method based on robust data correlation according to the present embodiment.
Step S14: training a corrected target detection model (YOLO-v 3 target detection network) by using the detection data set to obtain a network weight model suitable for detecting marine vessels;
Fig. 2 is a detailed flowchart of a tracking matching strategy in step S2 in the multi-ship target detection and tracking method based on robust data correlation according to the present embodiment.
Step S2: detecting a ship target in the video by using the trained model and designing a robustness matching strategy for tracking;
the step S2 further comprises the steps of:
step S21: detecting a ship target by using the ship target detection network obtained in the step S14, and obtaining target state prediction quantity of the detected ship target by using a Kalman filtering algorithm;
the step S21 further includes:
Step S211: obtaining a boundary frame of a ship target in each frame of image of the marine video by the step S14; the bounding box of a specific ship object is defined as:
[xc,yc,w,h]
where [ x c,yc ] is the center point coordinates of the bounding box, w is the width of the bounding box, and h is the height of the bounding box. Considering the state space of position and velocity at a certain time k, a specific state space is represented by an 8-dimensional column vector as follows:
step S212: designing a state prediction algorithm of a ship target according to a Kalman filtering algorithm;
the state prediction algorithm is as follows, and the motion state equation of the bounding box under the Cartesian coordinate system is defined as follows:
xk＝Fxk-1+nk-1
where x k is the target state space, n k-1 is the process noise at time k-1, and F is the state transition matrix of the system. The observation equation of the system is:
zk＝Hxk+vk
wherein the observed value z k＝[xc(k),yc(k),w(k),h(k)],vk is measurement noise at the k moment, and H is an observation matrix.
Preferably, the random variables n k and v k represent process noise and measurement noise, respectively, subject to a normal distribution of zero-mean Gaussian white noise, n k～N(0,Qk),vk～N(0,Rk), the process noise covariance matrix Q k and the measurement noise covariance matrix R k artificially set up hyper-parameters from the marine dataset, the system state transition matrixObservation matrix
The specific prediction process based on the state prediction algorithm comprises the following steps: posterior estimation of initial target state spaceTaking x 0, the initial filter error covariance estimate P 0 is known; according to the posterior estimation of the target state space, the posterior estimation is input into a state prediction algorithm x k＝Fxk-1+nk-1 to obtain prior estimation of the target state space, namely a ship target prediction result:
Wherein the method comprises the steps of Is a posterior estimate of the target state at time k-1. A priori estimation of error covariance:
Pk∣k-1＝FPk-1∣k-1F+Qk
Where P k-1∣k-1 is a posterior estimate of the k-1 time error covariance. Optimal kalman gain:
Where P k∣k-1 is an a priori estimate of the k-time error covariance. Posterior estimation of target state space:
Wherein the method comprises the steps of And (3) iteratively updating posterior estimation of the target state space at the k moment by combining the prior estimation of the target state space at the k-1 moment with the calculated optimal Kalman gain at the k moment. And solving the prior estimation of the target state space at the next moment based on the posterior estimation of the target state space, and carrying out iterative updating according to the prior estimation until a final ship target prediction result is obtained.
Posterior estimation of error covariance:
Pk∣k＝(I-KkH)Pk∣k-1
wherein P k∣k-1 is the prior estimate of the error covariance at the time of k-1 calculated above, and is combined with the optimal Kalman gain at the time of k calculated above to iteratively update the posterior estimate of the error covariance at the time of k.
Step S213, according to the state prediction algorithm of the ship target in the step S212, predicting the motion state of the detected ship boundary frame to obtain the prior estimation of the state space of the target ship;
step S22: designing a tracking matching strategy of a ship tracking frame to realize online ship target tracking;
The step S22 further includes:
step S221: detecting a 1 st frame image of a video through the ship target detection network trained in the step S14, wherein the obtained detection result is used for initializing a tracking frame;
Step S222: initializing a Kalman filter at the moment k by using the detection result at the moment k-1, and obtaining a predicted value of a position area where a target at the moment k possibly appears through the step S21; detecting a target position at the moment k through the ship target detection network trained in the step S14, and calculating a robustness data association metric TIoU of a detection result and a prediction result;
The robust data association metric TIoU in step S222 is calculated as follows:
Wherein B dt＝(xdt,ydt,wdt,hdt) is the ship detection result obtained in the step S14, B KF＝(xKF,yKF,wKF,hKF) is the prediction result obtained in the step S21 kalman filter algorithm, and C is the minimum convexity including B dt and B KF. The ratio of B dt to C and B KF to C is obtained; assigning TIoU a smaller value to the ratio;
Fig. 4 is a schematic diagram showing a comparison between a robust data correlation metric and a conventional number correlation metric in a multi-ship target detection and tracking method based on robust data correlation according to the present embodiment.
Compared with the traditional data correlation metric IoU, the data correlation metric TIoU based on tracking design has the following characteristics that objective limitations of multi-ship target tracking due to ocean current irregular motion interference, low-frame-rate image shooting by a carrier-based camera and the like are achieved, so that the intersection between a detection result B dt and a Kalman filtering prediction result B KF is small and even zero. Therefore IoU cannot reflect the similarity between the predicted and detected results, which leads to serious tracking loss. Whereas the calculation of the data correlation metric TIoU based on the tracking design takes into account not only the size of the overlap region between the detection result and the prediction result, but also the similarity of their shapes. More importantly, TIoU still performs very well in multi-vessel target tracking, even though there is little or no overlap between the test results and the predicted results.
Preferably, the conventional data association metric IoU (Intersection of Union) is calculated as follows:
wherein all symbol meanings in the formulae have already been given above.
Step S223: calculating a robustness data association metric TIoU through the step 222 to obtain a similarity matrix M i×j;
The similarity matrix M i×j defined in step S233 is:
Mi×j＝TIoU{Bi dt,Bj KF}
Wherein B i dt represents the state space of the i-th detection result, B j KF represents the state space of the j-th prediction result, and M i×j represents the similarity between the detection result and the prediction result;
Step S224: and (3) inputting the similarity matrix M i×j obtained in the step S223 into a Hungary algorithm to obtain a ship target tracking result.
Preferably, the nature of the hungarian algorithm is based on the idea that the number of matches succeeds is as large as possible.
Step S225: and designing a target disappearing stopping tracking strategy, setting each track, judging target disappearing when continuous 60-frame images cannot be successfully associated, and stopping updating the track.
Next, the validity of the present embodiment was verified by means of experimental results. Experiments were performed on maritime video sequences acquired by Jiangsu automation. The sea-borne video sequences of the Jiangsu automation station comprise 4 real sea condition scenes of harbor areas, harbor departure, harbor entering and open sea, sea surface targets are divided into 7 types of sailing ships, fishing ships, passenger ships, cargo ships, speedboats, floating targets, other special ships and the like by the data set, 45 video sequences of 29758 images are selected as training sets, 35 video sequences of 27812 images are selected as test sets, and the robust data association measurement and the traditional data association measurement based on tracking are respectively implemented into a main flow tracking frame, and experimental results are shown in table 1.
TABLE 1
Wherein IDF1 represents model identity retention capability and concerns correlation performance; RECALL represents detection capability; MOTA represents the accuracy of target tracking; MOTP represents predictive power. Table 1 shows that the multi-ship tracking algorithm based on the robust data correlation is superior to the multi-target tracking algorithm based on the traditional data correlation in terms of various indexes representing tracking performance under different tracking models. The effectiveness and superiority of the robust data correlation metric are demonstrated.
Meanwhile, the ship tracking result measured by the traditional data association is compared with the ship tracking result measured by the robust data association. It can be observed from the results that the IDs of the ship targets are frequently switched by adopting the tracking algorithm before the improvement, but the IDs of the ship tracking results adopting the robustness data association measurement are not switched all the time, which benefits from the association compensation capability of the robustness data association measurement with high intensity.
The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims (3)
1. The multi-ship target detection and tracking method based on the robust data association is characterized by comprising the following steps of:
constructing a target detection model based on a loss function of the robustness data association measurement and training;
Acquiring a marine video, and detecting the marine video based on the trained target detection model to acquire a detection result of a ship target;
designing a state prediction algorithm, and obtaining a prediction result of a ship target based on the state prediction algorithm;
constructing a tracking matching strategy, acquiring robustness data association measurement of the detection result and the prediction result, and further acquiring a similarity matrix;
inputting the similarity matrix into a Hungary algorithm to obtain a ship target tracking result;
The construction of the loss function of the robustness data association metric comprises the following steps: based on a detection result and a prediction result of a ship target, acquiring a minimum convex shape comprising the detection result and the prediction result; the ratio of the detection result and the prediction result to the minimum convex is obtained and sequenced, and the minimum value of the ratio is selected and assigned to the robustness data association measurement, so that a loss function based on the robustness data association measurement is obtained;
The process of designing the state prediction algorithm includes: acquiring a boundary frame of a ship target in each frame of image in a maritime video, acquiring process noise at a preset moment and a state transition matrix of a system based on a Kalman filtering algorithm, and constructing a motion state equation of the boundary frame; based on the motion state equation of the boundary box, measuring noise and an observation matrix at the next moment of the preset moment are obtained, and then a state prediction algorithm is obtained;
The process of obtaining the predicted result of the ship target comprises the following steps: acquiring initial error covariance estimation and posterior estimation of an initial target state space, and respectively acquiring prior estimation of error covariance and prior estimation of a target state at preset time based on a state prediction algorithm, namely a ship target prediction result; based on the prior estimation of the error covariance, obtaining an optimal Kalman gain, carrying out data fusion on a predicted result and an actual detection result of a ship target based on the optimal Kalman gain, and obtaining posterior estimation of a target state space at the next moment of a preset moment through iterative updating; inputting posterior estimation of the target state space at the next moment into a motion state equation of the boundary box, and continuously obtaining prior estimation of the target state space at the next moment, and iteratively updating the prior estimation until a final ship target prediction result is obtained;
the process for constructing the tracking matching strategy comprises the following steps: and presetting a ship tracking frame, detecting an image of the marine video at a preset moment based on a trained target detection model, initializing a Kalman filter based on a detection result, and designing a state prediction algorithm based on the initialized Kalman filter to further obtain a prediction result of a ship target at the next moment of the preset moment.
2. The method for multi-vessel target detection and tracking based on robust data correlation of claim 1,
The process of training the target detection model comprises the following steps: marking the acquired maritime videos as a detection data set, and performing data enhancement processing on the detection data set; training the target detection model based on the processed detection data set.
3. The method for multi-vessel target detection and tracking based on robust data correlation of claim 1,
The process of obtaining the ship target tracking result further comprises the following steps: and designing a target disappearing stopping tracking strategy, and judging that the target disappears when the continuous 60-frame images of the running track of the ship cannot be successfully associated, and stopping updating the running track.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310983360.7A CN117011768B (en) | 2023-08-07 | 2023-08-07 | Multi-ship target detection and tracking method based on robust data association |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310983360.7A CN117011768B (en) | 2023-08-07 | 2023-08-07 | Multi-ship target detection and tracking method based on robust data association |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117011768A CN117011768A (en) | 2023-11-07 |
| CN117011768B true CN117011768B (en) | 2024-07-23 |
Family
ID=88561519
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310983360.7A Active CN117011768B (en) | 2023-08-07 | 2023-08-07 | Multi-ship target detection and tracking method based on robust data association |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117011768B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113269073A (en) * | 2021-05-19 | 2021-08-17 | 青岛科技大学 | Ship multi-target tracking method based on YOLO V5 algorithm |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107818571B (en) * | 2017-12-11 | 2018-07-20 | 珠海大横琴科技发展有限公司 | Ship automatic tracking method and system based on deep learning network and average drifting |
| CN109509214B (en) * | 2018-10-15 | 2021-07-16 | 杭州电子科技大学 | Ship target tracking method based on deep learning |
| CN110517292A (en) * | 2019-08-29 | 2019-11-29 | 京东方科技集团股份有限公司 | Method for tracking target, device, system and computer readable storage medium |
| CN111368938A (en) * | 2020-03-19 | 2020-07-03 | 南京因果人工智能研究院有限公司 | Multi-target vehicle tracking method based on MDP |
| KR102378887B1 (en) * | 2021-02-15 | 2022-03-25 | 인하대학교 산학협력단 | Method and Apparatus of Bounding Box Regression by a Perimeter-based IoU Loss Function in Object Detection |
| CN114842559B (en) * | 2022-06-29 | 2022-10-14 | 山东省人工智能研究院 | Video interaction action detection method based on multi-mode time perception and attention |
| CN115546502A (en) * | 2022-10-14 | 2022-12-30 | 西安镭映光电科技有限公司 | Infrared small target detection method based on YOLOv4 multi-scale feature fusion |
-
2023
- 2023-08-07 CN CN202310983360.7A patent/CN117011768B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113269073A (en) * | 2021-05-19 | 2021-08-17 | 青岛科技大学 | Ship multi-target tracking method based on YOLO V5 algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117011768A (en) | 2023-11-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110232350B (en) | Real-time water surface multi-moving-object detection and tracking method based on online learning | |
| CN111239766B (en) | Water surface multi-target rapid identification and tracking method based on laser radar | |
| Lee et al. | Image-based ship detection and classification for unmanned surface vehicle using real-time object detection neural networks | |
| CN108830879A (en) | A kind of unmanned boat sea correlation filtering method for tracking target suitable for blocking scene | |
| CN108694724A (en) | A kind of long-time method for tracking target | |
| CN108664930A (en) | A kind of intelligent multi-target detection tracking | |
| CN111723632B (en) | Ship tracking method and system based on twin network | |
| CN110287837A (en) | Sea obstacle detection method based on prior estimate network and space constraint mixed model | |
| CN110458160A (en) | A kind of unmanned boat waterborne target recognizer based on depth-compression neural network | |
| Yu et al. | Object detection-tracking algorithm for unmanned surface vehicles based on a radar-photoelectric system | |
| CN109919026A (en) | A kind of unmanned surface vehicle local paths planning method | |
| CN109558877B (en) | KCF-based offshore target tracking algorithm | |
| CN116246139A (en) | Target identification method based on multi-sensor fusion for unmanned ship navigation environment | |
| CN114627156A (en) | Consumption-level unmanned aerial vehicle video moving target accurate tracking method | |
| Praczyk et al. | Concept and first results of optical navigational system | |
| Ross et al. | Bev-slam: Building a globally-consistent world map using monocular vision | |
| CN117011768B (en) | Multi-ship target detection and tracking method based on robust data association | |
| Chantler et al. | Detection and tracking of returns in sector-scan sonar image sequences | |
| CN117115193A (en) | Infrared ship positioning method based on line inhibition | |
| Shao et al. | A robust method for multi object tracking in autonomous ship navigation systems | |
| Wang et al. | Fast target tracking based on improved deep sort and YOLOv3 fusion algorithm | |
| CN114690772A (en) | Meeting time-space prediction method and system for sea area non-tracking navigation ship | |
| Chen et al. | AdapTrack: An adaptive FairMOT tracking method applicable to marine ship targets | |
| Sadhu et al. | Obstacle detection for image-guided surface water navigation | |
| JP4482623B2 (en) | Extraction method of ships from time series voyage images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |