CN114926500A - Twin network target tracking method and system based on sorting - Google Patents

Twin network target tracking method and system based on sorting Download PDF

Info

Publication number
CN114926500A
CN114926500A CN202210549797.5A CN202210549797A CN114926500A CN 114926500 A CN114926500 A CN 114926500A CN 202210549797 A CN202210549797 A CN 202210549797A CN 114926500 A CN114926500 A CN 114926500A
Authority
CN
China
Prior art keywords
loss function
classification
twin
network
rpn
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210549797.5A
Other languages
Chinese (zh)
Inventor
凌强
汤峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology of China USTC
Original Assignee
University of Science and Technology of China USTC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology of China USTC filed Critical University of Science and Technology of China USTC
Priority to CN202210549797.5A priority Critical patent/CN114926500A/en
Publication of CN114926500A publication Critical patent/CN114926500A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to a twin network target tracking method and a twin network target tracking system based on sequencing, wherein the method comprises the following steps: s1: constructing a classification and sequencing loss function, and training classification branches in the twin RPN target tracking network; s2: constructing a sequencing loss function based on IoU, and aligning classification branches and regression branches in the twin RPN target tracking network; s3: and combining the classification sorting loss function, the sorting loss function based on IoU and the original loss function in the RPN network to construct a total loss function and guide the training of the twin RPN target tracking network. According to the method provided by the invention, the target tracking precision of the existing twin RPN network can be effectively improved by constructing the sorting loss and the sorting loss function based on IoU.

Description

Twin network target tracking method and system based on sorting
Technical Field
The invention relates to the field of pattern recognition and computer vision, in particular to a twin network target tracking method and system based on sequencing.
Background
Recently, target tracking algorithms SiamRPN and SiamRPN + + based on twin networks have attracted great attention. Current twin network based trackers mainly describe visual tracking as two independent subtasks, classification and regression. When learning to classify the sub-network, these methods process each sample separately, for example, a certain sample label is a positive sample (1) or a negative sample (0), and the network outputs a classification score of 1 or 0 for the sample as much as possible under the guidance of the classification loss function, but the relationship between the positive and negative samples is not considered. This can result in the twin network tracing having difficulty in discriminating difficult negative examples (objects similar to the tracked object). In the training phase, the classification sub-network is responsible for classifying the training samples, i.e. the simple negative samples containing a large amount of semantic-free information, while some difficult negative samples, which are extremely rare, are easily swamped by a large amount of simple negative samples in the training phase. Although most non-target samples (falling on the background area) can be identified as the background by the classifier when tested, as long as an interfering target has a high foreground classification score, it can interfere with the tracker, and once its score exceeds the classification score of the true target in a certain frame, the tracker will bias towards the interfering target, resulting in tracking failure, which frequently occurs in the past twin network trackers.
Furthermore, there is a mismatch problem between classification and regression, since the classification and regression tasks are handled independently. In particular, the classification loss function causes the model to distinguish between foreground and background without taking regression branches into account. The purpose of the regression branch is to regress the bounding box of the target for all positive samples, regardless of the classification of the samples. Thus, samples with higher target box regression accuracy may have relatively lower target classification scores, while samples with higher target classification scores may yield lower regression accuracy. Therefore, how to improve the target tracking accuracy of the twin RPN network becomes a problem to be solved urgently.
Disclosure of Invention
In order to solve the technical problem, the invention provides a twin network target tracking method and system based on sorting.
The technical solution of the invention is as follows: a twin network target tracking method based on sorting comprises the following steps:
step S1: constructing a classification and sequencing loss function, and training classification branches in the twin RPN target tracking network;
step S2: constructing an IoU-based ordering loss function, and aligning classification branches and regression branches in the twin RPN target tracking network;
step S3: and combining the classification sorting loss function, the sorting loss function based on IoU and the original loss function in the RPN network to construct a total loss function and guide the training of the twin RPN target tracking network.
Compared with the prior art, the invention has the following advantages:
1. the invention discloses a twin network target tracking method based on sorting, which utilizes a sorting and sorting loss function to restrict positive sample sorting scores to be larger than difficult negative sample scores, so that the difficult negative samples can be classified as foreground targets, and the tracking failure caused by mistakenly selecting negative samples by a tracker is avoided.
2. The classification and regression branches in the twin RPN network are connected by using the IoU-based sequencing loss function, so that the classification precision and the regression prediction precision can be reflected by the classification branch prediction score.
Drawings
FIG. 1 is a flowchart of a twin network target tracking method based on sorting according to an embodiment of the present invention;
FIG. 2 is a system architecture diagram of a twin RPN target tracking network in an embodiment of the present invention;
fig. 3 is a block diagram of a twin network target tracking system based on sorting according to an embodiment of the present invention.
Detailed Description
The invention provides a twin network target tracking method based on sorting, which can effectively improve the target tracking precision of the existing twin network by constructing sorting loss and a sorting loss function based on IoU.
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings.
Example one
As shown in fig. 1, a twin network target tracking method based on sorting provided by an embodiment of the present invention includes the following steps:
step S1: constructing a classification and sequencing loss function, and training classification branches in the twin RPN target tracking network;
step S2: constructing a sequencing loss function based on IoU, and aligning classification branches and regression branches in the twin RPN target tracking network;
step S3: and combining the classification sorting loss function, the sorting loss function based on IoU and the original loss function in the RPN network to construct a total loss function and guide the training of the twin RPN target tracking network.
As shown in fig. 2, a system architecture of a twin RPN target tracking network is shown, which has two inputs, one is a template image containing a first frame tracking target, and the other is a search image containing a subsequent frame tracking target and a background, and a backbone network is used for extracting features of two paths of images. Inputting the two characteristic graphs into an RPN module, firstly fusing the two characteristics, and respectively inputting the fused characteristics into a classification branch for predicting the category of each sample (candidate frame), namely whether the sample belongs to a foreground target or a background; and a regression branch for predicting the bounding box of the tracked target.
In one embodiment, step S1: constructing a classification and sequencing loss function, and training classification branches in a twin RPN target tracking network, wherein the method specifically comprises the following steps:
step S11: respectively calculating the foreground classification score mean value of positive and negative samples output by the classification branch in the twin RPN according to the following formula (1):
Figure BDA0003654374210000031
wherein, A pos Is a positive sample set, A neg Is a difficult negative sample set; negative sample j - A weight coefficient of
Figure BDA0003654374210000032
exp () is an exponential function; positive sample j + Weight w of j+ Is composed of
Figure BDA0003654374210000033
Wherein N is pos Is a positive sample set A pos The number of the middle samples; p is a radical of j+ And p j- Respectively positive samples j predicted by the branch of the classification + And difficult negative example j - (ii) a foreground classification score of;
according to the embodiment of the invention, all negative samples are sorted according to the foreground classification scores output by the classification branches in the twin RPN network, the negative samples with the foreground classification scores lower than 0.5 are filtered, and the remaining negative samples form a difficult negative sample set A neg ；
Step S12: p obtained according to step S11 + And P - And constructing a classification and sorting loss function as shown in formula (2):
Figure BDA0003654374210000034
wherein exp () and log () are an exponential function and a logarithmic function, respectively; beta is a parameter for controlling the size of the loss value, and alpha is a parameter for controlling the sorting distance, in the embodiment of the invention, the value of beta can be 4, and the value of alpha is 0.5. Equation (2) can constrain the mean value P of the foreground classification scores of the positive samples + Greater than the mean value P of the classification of the foreground of the difficult negative sample - In particular, if P - Ratio P + If the value is large, then L rank_cls The loss value will be large; on the contrary, if P - Ratio P + If the value is small, then L rank_cls The loss value will be smaller. Therefore, the neural network is in reverse propagation for L rank_cls The value is as small as possible, and P is constrained - As much as possible of P + Therefore, the classification score of the difficult negative sample is effectively reduced, and the purpose of suppressing the difficult background is achieved.
The invention converts the classification problem in the twin RPN network into a sorting problem, and restricts the foreground classification score of the positive sample to be larger than the foreground classification score of the difficult negative sample. Compared with the existing classification problem, the sequencing mode provided by the invention serves as a loose constraint term, and the difficult negative sample foreground classification mean value P in the formula (2) - May be large, fall on the foreground object class, but only require a guaranteeProve their foreground classification score mean P - Lower than positive sample foreground classification score mean P + Therefore, the tracker can be prevented from selecting negative samples by mistake, and the tracking failure can be avoided.
In one embodiment, the step S2: constructing an IoU-based sequencing loss function, and aligning classification branches and regression branches in the twin RPN target tracking network, wherein the method specifically comprises the following steps:
step S21: for positive samples i + ,j + ∈A pos In other words, the following constraint is agreed, as shown in equation (3):
Figure BDA0003654374210000041
wherein the content of the first and second substances,
Figure BDA0003654374210000042
and
Figure BDA0003654374210000043
are respectively positive samples i + And j + (ii) a foreground classification score of;
Figure BDA0003654374210000044
and
Figure BDA0003654374210000045
are respectively positive samples i + And j + The regression score obtained by the regression branch prediction is represented by iou (intersection over union);
according to the formula (3), the current sample i + Regression score of (2)
Figure BDA0003654374210000046
Greater than positive sample j + Regression score of
Figure BDA0003654374210000047
Then the sample i can be constrained + Foreground classification score of
Figure BDA0003654374210000048
Greater than positive sample j + Foreground classification score of
Figure BDA0003654374210000049
Similarly, when the sample i is positive + Foreground classification score of
Figure BDA00036543742100000410
Greater than positive sample j + Foreground classification score of
Figure BDA00036543742100000411
Then the positive sample i may be constrained + Regression score of
Figure BDA00036543742100000412
Greater than positive sample j + Regression score of
Figure BDA00036543742100000413
Through the constraint condition of the formula (3), the score of the classification branch not only reflects the foreground classification precision of the target, but also reflects the regression precision of the target;
step S22: construction of IoU-based ordering loss function L rank-iou As shown in the following formula (4):
Figure BDA00036543742100000414
wherein xp () is an exponential function; gamma is a parameter for controlling the magnitude of the loss value, and in the embodiment of the invention, the value of gamma can be 3.
The first term of equation (4) reflects the first constraint in equation (3), i.e., when
Figure BDA00036543742100000415
Is greater than
Figure BDA00036543742100000416
When the temperature of the water is higher than the set temperature,
Figure BDA00036543742100000417
is greater than
Figure BDA00036543742100000418
The smaller the value of the first term of equation (4); similarly, the second term of equation (4) reflects the second constraint in equation (3), i.e., when
Figure BDA00036543742100000419
Is greater than
Figure BDA00036543742100000420
When the temperature of the water is higher than the set temperature,
Figure BDA00036543742100000421
is greater than
Figure BDA00036543742100000422
The smaller the value of the second term of equation (4) is. Therefore, under the action of the formula (4), the higher the regression accuracy, the higher the sample classification score is, and the classification score of the classification branch can reflect the accuracy of target frame prediction to a certain extent, so that the formula (4) can connect the classification branch with the regression branch, and the classification branch can reflect the classification accuracy and the regression accuracy at the same time.
The invention constructs the sequencing loss function based on IoU, and the loss function can connect the classification branch and the regression branch on the premise of not adding additional branches, so that the classification branch can reflect the classification precision and the regression precision (namely the target frame prediction precision) at the same time.
In one embodiment, the step S3: combining the classification sorting loss function, the sorting loss function based on IoU and the original loss function in the RPN network to construct a total loss function and guide the training of a twin RPN target tracking network, specifically comprising:
sorting order loss function L rank_cls IoU-based ordering loss function L rank_iou And the loss function L existing in the RPN network RPN Adding to construct the total loss function L total Expressed by the following formula (5):
L total ＝L RPN +L rank-cls +L rank-iou (5)
through L total The training of the twin RPN target tracking network can be guided, and the training is used for optimizing and updating network parameters.
Example two
As shown in fig. 3, an embodiment of the present invention provides a twin network target tracking system based on sorting, which includes the following modules:
a classification loss function building module 41, configured to build a classification loss function and train classification branches in the twin RPN target tracking network;
an IoU-based ranking loss function building module 42 configured to build a IoU-based ranking loss function to align classification branches and regression branches in the twin RPN target tracking network;
and a total loss function constructing module 43, configured to combine the sorted ranking loss function, the ranking loss function based on IoU, and the original loss function in the RPN network, to construct a total loss function, and guide training of the twin RPN target tracking network.
The above examples are provided for the purpose of describing the present invention only and are not intended to limit the scope of the present invention. The scope of the invention is defined by the appended claims. Various equivalent substitutions and modifications can be made without departing from the spirit and principles of the invention, and are intended to be within the scope of the invention.

Claims (5)

1. A twin network target tracking method based on sequencing is characterized by comprising the following steps:
step S1: constructing a classification sequencing loss function, and training classification branches in the twin RPN target tracking network;
step S2: constructing an IoU-based ordering loss function, and aligning classification branches and regression branches in the twin RPN target tracking network;
step S3: and combining the classification sorting loss function, the sorting loss function based on IoU and the original loss function in the RPN network to construct a total loss function and guide the training of the twin RPN target tracking network.
2. The twin network target tracking method based on sorting as claimed in claim 1, wherein the step S1: constructing a classification and sequencing loss function, and training classification branches in a twin RPN target tracking network, wherein the method specifically comprises the following steps:
step S11: respectively calculating the foreground classification score mean values of positive and negative samples output by the classification branches in the twin RPN according to the following formula (1):
Figure FDA0003654374200000011
wherein A is pos Is a positive sample set, A neg Is a difficult negative sample set; negative example j - A weight coefficient of
Figure FDA0003654374200000012
exp () is an exponential function; positive sample j + Weight of (2)
Figure FDA0003654374200000013
Is composed of
Figure FDA0003654374200000014
Wherein N is pos Is a positive sample set A pos The number of the middle samples; p is a radical of j+ And p j- Respectively positive samples j predicted by the classification branch + And difficult negative example j - (ii) a foreground classification score of;
step S12: p obtained according to step S11 + And P - And constructing a classification and sorting loss function as shown in formula (2):
Figure FDA0003654374200000015
wherein exp () and log () are exponential function and logarithmic function, respectively; beta is a parameter for controlling the size of the loss value, and alpha is a parameter for controlling the sorting distance.
3. The twin network target tracking method based on sorting as claimed in claim 1, wherein the step S2: constructing an IoU-based ordering loss function, and aligning classification branches and regression branches in the twin RPN target tracking network, wherein the method specifically comprises the following steps:
step S21: for positive samples i + ,j + ∈A pos In other words, the following constraint is agreed, as shown in equation (3):
Figure FDA0003654374200000021
wherein the content of the first and second substances,
Figure FDA0003654374200000022
and
Figure FDA0003654374200000023
are respectively positive samples i + And j + (ii) a foreground classification score of;
Figure FDA0003654374200000024
and
Figure FDA0003654374200000025
are respectively positive samples i + And j + (ii) a regression score derived from the regression branch prediction;
step S22: construction of IoU-based ordering loss function L rank-iou As shown in the following formula (4):
Figure FDA0003654374200000026
where γ is a parameter controlling the magnitude of the loss value, and exp () is an exponential function.
4. The twin network target tracking method based on sorting as claimed in claim 1, wherein the step S3: combining the classification ranking loss function, the IoU-based ranking loss function and the original loss function in the RPN network to construct a total loss function and guide the training of the twin RPN target tracking network, specifically comprising:
sorting the classification order loss function L rank_cls The IoU-based ordering loss function L rank_iou And loss function L existing in RPN network RPN Adding to construct the total loss function L total Expressed by the following formula (5):
L total ＝L RPN +L rank-cls +L rank-iou (5)
5. a twin network target tracking system based on sequencing is characterized by comprising the following modules:
a classification loss function building module for building a classification loss function and training classification branches in the twin RPN target tracking network;
an IoU-based ranking loss function building module for building a IoU-based ranking loss function to align classification branches and regression branches in the twin RPN target tracking network;
and a total loss function constructing module, configured to combine the classification sorting loss function, the sorting loss function based on IoU, and an original loss function in the RPN network to construct a total loss function, and guide training of the twin RPN target tracking network.
CN202210549797.5A 2022-05-20 2022-05-20 Twin network target tracking method and system based on sorting Pending CN114926500A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210549797.5A CN114926500A (en) 2022-05-20 2022-05-20 Twin network target tracking method and system based on sorting

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210549797.5A CN114926500A (en) 2022-05-20 2022-05-20 Twin network target tracking method and system based on sorting

Publications (1)

Publication Number Publication Date
CN114926500A true CN114926500A (en) 2022-08-19

Family

ID=82809487

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210549797.5A Pending CN114926500A (en) 2022-05-20 2022-05-20 Twin network target tracking method and system based on sorting

Country Status (1)

Country Link
CN (1) CN114926500A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630794A (en) * 2023-04-25 2023-08-22 北京卫星信息工程研究所 Remote sensing image target detection method based on sorting sample selection and electronic equipment

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116630794A (en) * 2023-04-25 2023-08-22 北京卫星信息工程研究所 Remote sensing image target detection method based on sorting sample selection and electronic equipment
CN116630794B (en) * 2023-04-25 2024-02-06 北京卫星信息工程研究所 Remote sensing image target detection method based on sorting sample selection and electronic equipment

Similar Documents

Publication Publication Date Title
Luo et al. Progressive graph learning for open-set domain adaptation
CN109919934B (en) Liquid crystal panel defect detection method based on multi-source domain deep transfer learning
WO2018137357A1 (en) Target detection performance optimization method
CN114743020B (en) Food identification method combining label semantic embedding and attention fusion
NL2025689B1 (en) Crop pest detection method based on f-ssd-iv3
Tian et al. Striking the right balance: Recall loss for semantic segmentation
CN110689091A (en) Weak supervision fine-grained object classification method
CN115049884B (en) Broad-sense few-sample target detection method and system based on fast RCNN
CN115811440B (en) Real-time flow detection method based on network situation awareness
CN114926500A (en) Twin network target tracking method and system based on sorting
CN113837308A (en) Knowledge distillation-based model training method and device and electronic equipment
CN116385773A (en) Small target detection method, storage medium and electronic equipment
Chen et al. Multi-level attentive adversarial learning with temporal dilation for unsupervised video domain adaptation
CN114741517A (en) Training method, device, equipment and medium of text classification model and text classification method, device and equipment
CN115050022A (en) Crop pest and disease identification method based on multi-level self-adaptive attention
CN111626291A (en) Image visual relationship detection method, system and terminal
CN114724156A (en) Form identification method and device and electronic equipment
CN117371511A (en) Training method, device, equipment and storage medium for image classification model
CN111598000A (en) Face recognition method, device, server and readable storage medium based on multiple tasks
Li et al. GADet: A Geometry-Aware X-ray Prohibited Items Detector
CN113076490B (en) Case-related microblog object-level emotion classification method based on mixed node graph
CN115294405A (en) Method, device, equipment and medium for constructing crop disease classification model
CN114387483A (en) Target detection method, model training method, device, equipment and storage medium
Soujanya et al. A CNN based approach for handwritten character identification of Telugu guninthalu using various optimizers
CN115082762A (en) Target detection unsupervised domain adaptation system based on regional recommendation network center alignment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination