CN109829934A - A kind of novel image tracking algorithm based on twin convolutional network - Google Patents
A kind of novel image tracking algorithm based on twin convolutional network Download PDFInfo
- Publication number
- CN109829934A CN109829934A CN201811591372.0A CN201811591372A CN109829934A CN 109829934 A CN109829934 A CN 109829934A CN 201811591372 A CN201811591372 A CN 201811591372A CN 109829934 A CN109829934 A CN 109829934A
- Authority
- CN
- China
- Prior art keywords
- branch
- network
- algorithm
- convolutional network
- layers
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a kind of novel image tracking algorithms based on twin convolutional network, the following steps are included: step S1, chooses the full convolutional network of twin convolutional network, first five layer of AlexNet network is chosen, it carries out building neural network framework, network inputs is carried out to the target tracked；Step S2 adds RPN layers on the basis of the full convolutional coding structure；Step S3, is specifically trained according to algorithm；Step S4, the responsible characteristic pattern to target and tracing area of the filtering part are filtered operation acquisition output response figure according to the specific algorithm of the step S3 and obtain a result.The present invention is added RPN layers on the basis of track algorithm of the tradition based on twin convolutional network, enhances track algorithm to the separating capacity of target and background；Conventional target tracing task is converted into single sample (one-shot) Detection task using meta-learning principle, improves the operational efficiency of track algorithm.
Description
Technical field
The invention belongs to deep learning and computer vision field, particularly relate to a kind of novel based on twin convolutional network
Image tracking algorithm.
Background technique
Target following technology is an important branch in Computer Vision Task, is widely used in automatic Pilot, depending on
The fields such as frequency monitoring and robot.Conventional target track algorithm depend on by hand mark feature and filtering algorithm (such as KCF,
TLD etc.), rate is very fast but accuracy rate and robustness are lower, it is difficult to meet application request.Recently as artificial intelligence
With the rise of deep learning, convolutional neural networks algorithm steps into target tracking domain, and achieve original performance with
Achievement, wherein algorithm frame based on twin convolutional network is by its good performance and succinct network structure,
Very big concern is received in international computer vision top meeting in recent years and tracking race.
RPN (regionproposalnetwork) layer is proposed in algorithm of target detection faster-rcnn earliest,
It is instead of traditional candidate region generating algorithm (such as sliding window), for generating the candidate region in detection algorithm.Its
Feature is that operation efficiency is very high without redundant computation, can generate the candidate region of high quality, be greatly improved related two-
The performance of stage detection algorithm.
To further increase track algorithm to the separating capacity of target and background, while traditional tracing task is converted into list
Sample learning (one-shot) Detection task improves the operation efficiency in algorithm reasoning stage, proposes a kind of novel based on twin volume
The image tracking algorithm of product network.
Summary of the invention
In view of above-mentioned technical background, the present invention is added RPN layers on the basis of the algorithm of the twin convolutional network of tradition, further
Track algorithm is improved to the separating capacity of target and background, while traditional tracing task is converted into single sample learning (one-
Shot) Detection task improves the operation efficiency in algorithm reasoning stage.
To achieve the goals above, present invention employs following technical solutions:
Design a kind of novel image tracking algorithm based on twin convolutional network, comprising the following steps:
Step S1 chooses the full convolutional network of twin convolutional network, chooses first five layer of AlexNet network, is built
Neural network framework carries out network inputs to the target tracked；The network inputs by tracking target image and with
Track area image two parts are constituted；
Step S2 adds RPN layers on the basis of the full convolutional coding structure, and described RPN layers by supervised learning part and filter
Wave part is constituted, and the supervised learning part is made of recurrence branch and classification branch, and the recurrence branch is reg branch, institute
Stating classification branch is cls branch；By the output of the full convolutional networkWithDescribed RPN layers of cls branch is inputted respectively
Road and reg branch are simultaneously filtered the output of two branches；With mathematic(al) representation are as follows:
Wherein, the A indicates that RPN exports characteristic pattern, and the w indicates that the width of output characteristic pattern, the h indicate output feature
The height of figure, the k indicate anchor number, and the * indicates filtering operation；
Due to joined RPN layers, so using multitask loss function as the loss function of this algorithm, the cls branch
Road uses cross entropy as loss function, and reg branch uses smoothL1loss as loss function；
Step S3 is specifically trained according to following algorithm:
1): defining Ax, Ay, Aw, Ah indicates that the centre coordinate of anchor and wide high, Tx, Ty, Tw, Th expression ground are real
The centre coordinate of condition and wide height.Therefore normalized cumulant can be indicated by following mathematic(al) representations:
2): the mathematic(al) representation of smoothL1loss are as follows:
By the loss function that branch 1) must be returned are as follows:
3) it defines to obtain final loss function by above-mentioned steps S1, S2 and multitask loss function are as follows:
Loss=Lcls+ λ Lreg
Wherein, Lcls is the cross entropy loss function of classification branch, and Lreg is the loss of smoothL1 described in step S2
Function, parameter lambda are hyper parameter, weight of the λ for two branches of balanced sort and recurrence.
Step S4, the filtering part are responsible for the specific calculation to the characteristic pattern of target and tracing area according to the step S3
Method is filtered operation acquisition output response figure and obtains a result.The algorithm output is target response value highest point on characteristic pattern
Coordinate.
Preferably, the output feature of the cls branch is 2k layers total, and two layers is a group, every layer of spy in each group
Levying each pixel on figure indicates the classification information of an anchor；
The output feature of the reg branch is 4k layers shared, and four layers are a group, each of the characteristic pattern of each group
Pixel respectively indicates dx, dy, dw, the bounding box coordinates of k anchor on dh, the 4k layers of each pixel of expression.
Preferably, described image track algorithm is converted to the algorithm of single pattern detection are as follows:
(1) the twin convolutional network part is divided into template branch and detection branch by, described
Classification information is embedded into convolution kernel by template branch for the training parameter in Detection task；The detection branch
The information being embedded into convolution kernel is detected on road；
Use average loss to find weight matrix W as the loss function of network, set up following mathematic(al) representation:
Wherein, xi is i-th of input sample, ψ (xi；W) indicate that anticipation function, li indicate the actual value of i-th of sample；
(2) defined function ω is feedforward equation, by (z；W' it) is mapped as W, therefore the mathematic(al) representation in (1) indicates are as follows:
Wherein, zi is i-th of template sample, i.e. target area image；
(3) defined functionCharacteristic operation is extracted for twin network, ζ is RPN layers of operation, therefore mathematical table in S step 2
It is converted up to formula are as follows:
In the algorithm forward inference stage, the target area in first frame be admitted to template branch generate cls, reg volumes
Template branch is removed after product core, and it is (related in detectionbranch that residue frame passes sequentially through detection branch
Deconvolution parameter is calculated in advance and is completed) and cls, reg convolution kernel calculate and generate final characteristic pattern.Because of mesh required for tracking
Information is marked, only by providing by the first frame of template branch, so this algorithm can be considered as primary single pattern detection task.
The novel image tracking algorithm based on twin convolutional network of one kind proposed by the present invention, beneficial effect are: this hair
It is bright to increase RPN layers on the basis of twin convolutional network structure, increasing track algorithm to the separating capacity of target and background
While, traditional tracing task is converted into single sample (one-shot) Detection task, improves the operational efficiency of algorithm, it is quasi-
True property and speed have great promotion compared to traditional track algorithm.
Detailed description of the invention
Fig. 1 is flow diagram of the invention.
Fig. 2 is the principle of the present invention structural schematic diagram.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation description, it is clear that described embodiments are only a part of the embodiments of the present invention, instead of all the embodiments.
Embodiment 1
A kind of novel image tracking algorithm based on twin convolutional network, comprising the following steps:
Step S1 chooses the full convolutional network of twin convolutional network, chooses first five layer of AlexNet network, is built
Neural network framework carries out network inputs to the target tracked；Network inputs are by tracking target image and tracking area
Area image two parts are constituted.
Step S2 adds RPN layers on the basis of full convolutional coding structure, and RPN layers by supervised learning part and filtering part structure
At supervised learning part is made of recurrence branch and classification branch, and recurrence branch is reg branch, and classification branch is cls branch；
By the output of full convolutional networkWithRespectively input RPN layer cls branch and reg branch and to two branches output progress
Filtering processing；With mathematic(al) representation are as follows:
Wherein, A indicates that RPN exports characteristic pattern, and w indicates that the width of output characteristic pattern, h indicate that the height of output characteristic pattern, k indicate
Anchor number, * indicate filtering operation；
Wherein, cls branch uses cross entropy as loss function, and reg branch uses smoothL1loss as loss letter
Number；
Step S3 is specifically trained according to following algorithm:
1): defining Ax, Ay, Aw, Ah indicates that the centre coordinate of anchor and wide high, Tx, Ty, Tw, Th expression ground are real
The centre coordinate of condition and wide height.Therefore normalized cumulant is indicated by following mathematic(al) representations:
2): the mathematic(al) representation of smoothL1loss are as follows:
By the loss function that branch 1) must be returned are as follows:
3) it defines to obtain final loss function by above-mentioned steps S1, S2 and multitask loss function are as follows:
Loss=Lcls+ λ Lreg
Wherein, Lcls is the cross entropy loss function of classification branch, and Lreg is the loss of smoothL1 described in step S2
Function, parameter lambda are hyper parameter, weight of the λ for two branches of balanced sort and recurrence.
Step S4, filtering part are responsible for filtering the characteristic pattern of target and tracing area according to the specific algorithm of step S3
Wave operation obtains output response figure and obtains a result, and algorithm output is coordinate of the target response value highest point on characteristic pattern.
The output feature of cls branch is 2k layers total, and two layers is a group, in each group each of on every layer of characteristic pattern
Pixel indicates the classification information of an anchor；
The output feature of reg branch is 4k layers shared, and four layers are a group, each pixel of the characteristic pattern of each group
Point respectively indicates dx, dy, dw, the bounding box coordinates of k anchor on dh, the 4k layers of each pixel of expression.
Convert image tracking algorithm to the algorithm of single pattern detection are as follows:
(1) twin convolutional network part is divided into template branch and detection branch by, and template branch is used
Training parameter in Detection task, classification information is embedded into convolution kernel；Detection branch will be embedded into convolution kernel
Information detected；
Use average loss to find weight matrix W as the loss function of network, set up following mathematic(al) representation:
Wherein, xi is i-th of input sample, ψ (xi；W) indicate that anticipation function, li indicate the actual value of i-th of sample；
(2) defined function ω is feedforward equation, by (z；W' it) is mapped as W, therefore the mathematic(al) representation in (1) indicates are as follows:
Wherein, zi is i-th of template sample, i.e. target area image；
(3) defined functionCharacteristic operation is extracted for twin network, ζ is RPN layers of operation, therefore mathematical table in S step 2
It is converted up to formula are as follows:
Embodiment 2
Implement this algorithm on VOT2015 data set, and (including tradition is based on the track algorithm of other state-of-the-art technologies
The track algorithm of twin network) compare performance indexes, as a result as shown in the table:
Embodiment 3
Implement this algorithm on VOT2016 data set, and (including tradition is based on the track algorithm of other state-of-the-art technologies
The track algorithm of twin network) compare performance indexes, as a result as shown in the table:
The foregoing is only a preferred embodiment of the present invention, but scope of protection of the present invention is not limited thereto,
Anyone skilled in the art in the technical scope disclosed by the present invention, according to the technique and scheme of the present invention and its
Inventive concept is subject to equivalent substitution or change, should be covered by the protection scope of the present invention.
Claims (4)
1. a kind of novel image tracking algorithm based on twin convolutional network, it is characterised in that: the following steps are included:
Step S1 chooses the full convolutional network of twin convolutional network, chooses first five layer of AlexNet network, carries out building nerve
Network frame carries out network inputs to the target tracked；
Step S2 adds RPN layers on the basis of the full convolutional coding structure, and described RPN layers by supervised learning part and filtering part
Point constitute, the supervised learning part by recurrence branch and classification branch form, the recurrences branch be reg branch, described point
Class branch is cls branch；By the output of the full convolutional networkWithInput respectively described RPN layers cls branch and
Reg branch is simultaneously filtered the output of two branches；With mathematic(al) representation are as follows:
Wherein, the A indicates that RPN exports characteristic pattern, and the w indicates that the width of output characteristic pattern, the h indicate output characteristic pattern
Height, the k indicate anchor number, and the * indicates filtering operation；
Wherein, the cls branch uses cross entropy as loss function, and reg branch uses smoothL1loss as loss letter
Number；
Step S3 is specifically trained according to following algorithm:
1): defining Ax, Ay, Aw, Ah indicates the centre coordinate and wide high, Tx, Ty of anchor, Tw, Th expression ground truth
Centre coordinate and wide height；Normalized cumulant is indicated by following mathematic(al) representations:
2): the mathematic(al) representation of smoothL1loss are as follows:
By the loss function that branch 1) must be returned are as follows:
3) it defines to obtain final loss function by above-mentioned steps S1, S2 and multitask loss function are as follows:
Loss=Lcls+ λ Lreg
Wherein, Lcls is the cross entropy loss function of classification branch, and Lreg is smoothL1 loss function described in step S2,
Parameter lambda is hyper parameter, weight of the λ for two branches of balanced sort and recurrence.
Step S4, the filtering part be responsible for the characteristic pattern of target and tracing area according to the specific algorithm of the step S3 into
Row filtering operation obtains output response figure and obtains a result.
2. a kind of novel image tracking algorithm based on twin convolutional network according to claim 1, it is characterised in that: described
The output feature of cls branch is 2k layers total, and two layers is a group, each pixel table in each group on every layer of characteristic pattern
Show the classification information of an anchor；
The output feature of the reg branch is 4k layers shared, and four layers are a group, each pixel of the characteristic pattern of each group
Point respectively indicates dx, dy, dw, the bounding box coordinates of k anchor on dh, the 4k layers of each pixel of expression.
3. the novel image tracking algorithm based on twin convolutional network of one kind according to claim 2, it is characterised in that: will
Described image track algorithm is converted into the algorithm of single pattern detection are as follows:
(1) the twin convolutional network part is divided into template branch and detection branch, the template branch by
Classification information is embedded into convolution kernel by road for the training parameter in Detection task；The detection branch will be embedded into
Information in convolution kernel is detected；
Use average loss to find weight matrix W as the loss function of network, set up following mathematic(al) representation:
Wherein, xi is i-th of input sample, ψ (xi；W) indicate that anticipation function, li indicate the actual value of i-th of sample；
(2) defined function ω is feedforward equation, by (z；W' it) is mapped as W, therefore the mathematic(al) representation in (1) indicates are as follows:
Wherein, zi is i-th of template sample, i.e. target area image；
(3) defined functionCharacteristic operation is extracted for twin network, ζ is RPN layers of operation, therefore mathematic(al) representation turns in S step 2
It turns to:
4. the novel image tracking algorithm based on twin convolutional network of one kind according to claim 1, it is characterised in that: institute
It states network inputs to be made of tracking target image and tracing area image two parts, the algorithm output is target response value highest
Coordinate of the point on characteristic pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811591372.0A CN109829934A (en) | 2018-12-20 | 2018-12-20 | A kind of novel image tracking algorithm based on twin convolutional network |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811591372.0A CN109829934A (en) | 2018-12-20 | 2018-12-20 | A kind of novel image tracking algorithm based on twin convolutional network |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109829934A true CN109829934A (en) | 2019-05-31 |
Family
ID=66861176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811591372.0A Pending CN109829934A (en) | 2018-12-20 | 2018-12-20 | A kind of novel image tracking algorithm based on twin convolutional network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109829934A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287874A (en) * | 2019-06-25 | 2019-09-27 | 北京市商汤科技开发有限公司 | Target tracking method and device, electronic equipment and storage medium |
| CN111724409A (en) * | 2020-05-18 | 2020-09-29 | 浙江工业大学 | Target tracking method based on densely connected twin neural network |
| CN111723632A (en) * | 2019-11-08 | 2020-09-29 | 珠海达伽马科技有限公司 | Ship tracking method and system based on twin network |
| CN111915644A (en) * | 2020-07-09 | 2020-11-10 | 苏州科技大学 | Real-time target tracking method of twin guiding anchor frame RPN network |
| CN112613558A (en) * | 2020-12-23 | 2021-04-06 | 武汉工程大学 | High-accuracy intelligent target identification tracking system and method for security camera |
| CN112749710A (en) * | 2019-10-31 | 2021-05-04 | 北京市商汤科技开发有限公司 | Target detection and intelligent driving method, device, equipment and storage medium |
| CN112802056A (en) * | 2020-11-03 | 2021-05-14 | 南京理工大学 | Heterogenous matching method based on twin RPN |
| CN113536912A (en) * | 2021-06-09 | 2021-10-22 | 中国铁塔股份有限公司黑龙江省分公司 | Twin comparison same-class tower type early warning algorithm based on standard model |
| CN117115167A (en) * | 2023-10-24 | 2023-11-24 | 诺比侃人工智能科技(成都)股份有限公司 | Coiled steel displacement judging method and system based on feature detection |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909082A (en) * | 2017-10-30 | 2018-04-13 | 东南大学 | Sonar image target identification method based on depth learning technology |
-
2018
- 2018-12-20 CN CN201811591372.0A patent/CN109829934A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107909082A (en) * | 2017-10-30 | 2018-04-13 | 东南大学 | Sonar image target identification method based on depth learning technology |
Non-Patent Citations (1)
| Title |
|---|
| BO LI 等: "High Performance Visual Tracking with Siamese Region Proposal Network", 《2018 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110287874A (en) * | 2019-06-25 | 2019-09-27 | 北京市商汤科技开发有限公司 | Target tracking method and device, electronic equipment and storage medium |
| CN112749710A (en) * | 2019-10-31 | 2021-05-04 | 北京市商汤科技开发有限公司 | Target detection and intelligent driving method, device, equipment and storage medium |
| CN111723632A (en) * | 2019-11-08 | 2020-09-29 | 珠海达伽马科技有限公司 | Ship tracking method and system based on twin network |
| CN111723632B (en) * | 2019-11-08 | 2023-09-15 | 珠海达伽马科技有限公司 | Ship tracking method and system based on twin network |
| CN111724409A (en) * | 2020-05-18 | 2020-09-29 | 浙江工业大学 | Target tracking method based on densely connected twin neural network |
| CN111915644A (en) * | 2020-07-09 | 2020-11-10 | 苏州科技大学 | Real-time target tracking method of twin guiding anchor frame RPN network |
| CN111915644B (en) * | 2020-07-09 | 2023-07-04 | 苏州科技大学 | Real-time target tracking method of twin guide anchor frame RPN network |
| CN112802056A (en) * | 2020-11-03 | 2021-05-14 | 南京理工大学 | Heterogenous matching method based on twin RPN |
| CN112613558A (en) * | 2020-12-23 | 2021-04-06 | 武汉工程大学 | High-accuracy intelligent target identification tracking system and method for security camera |
| CN113536912A (en) * | 2021-06-09 | 2021-10-22 | 中国铁塔股份有限公司黑龙江省分公司 | Twin comparison same-class tower type early warning algorithm based on standard model |
| CN117115167A (en) * | 2023-10-24 | 2023-11-24 | 诺比侃人工智能科技(成都)股份有限公司 | Coiled steel displacement judging method and system based on feature detection |
| CN117115167B (en) * | 2023-10-24 | 2023-12-29 | 诺比侃人工智能科技(成都)股份有限公司 | Coiled steel displacement judging method and system based on feature detection |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109829934A (en) | A kind of novel image tracking algorithm based on twin convolutional network | |
| Zhang et al. | A convolutional neural network for leaves recognition using data augmentation | |
| Li et al. | Fast and accurate green pepper detection in complex backgrounds via an improved Yolov4-tiny model | |
| CN107766850A (en) | Based on the face identification method for combining face character information | |
| CN104933417A (en) | Behavior recognition method based on sparse spatial-temporal characteristics | |
| CN106991408A (en) | The generation method and method for detecting human face of a kind of candidate frame generation network | |
| Meng et al. | Hyperspectral image classification with mixed link networks | |
| CN103729467B (en) | Community structure discovery method in social network | |
| CN109886267A (en) | A kind of soft image conspicuousness detection method based on optimal feature selection | |
| CN107833239A (en) | A kind of searching of optimal matching method for tracking target based on weighted model constraint | |
| CN114419413A (en) | Method for constructing sensing field self-adaptive transformer substation insulator defect detection neural network | |
| Sui et al. | BIBED-Seg: Block-in-block edge detection network for guiding semantic segmentation task of high-resolution remote sensing images | |
| CN116051957A (en) | Personal protection item detection network based on attention mechanism and multi-scale fusion | |
| Jia et al. | Collaborative contrastive learning for hyperspectral and LiDAR classification | |
| Yu et al. | Disease Recognition of Apple Leaf Using Lightweight Multi-Scale Network with ECANet. | |
| Zu et al. | Disease identification of Lentinus edodes sticks based on deep learning model | |
| Mei et al. | Cascade residual capsule network for hyperspectral image classification | |
| CN114399018A (en) | EfficientNet ceramic fragment classification method based on rotation control strategy sparrow optimization | |
| Liu et al. | Color image edge detection method based on the improved whale optimization algorithm | |
| CN107610136A (en) | Well-marked target detection method based on the sequence of convex closure structure center query point | |
| Xiao et al. | Road extraction from point clouds of open-pit mine using LPFE-Net | |
| Yin et al. | An improved algorithm based on convolutional neural network for smoke detection | |
| Li et al. | Siamese visual tracking with deep features and robust feature fusion | |
| Li et al. | Edge and node graph convolutional neural network for human action recognition | |
| Lai et al. | A LPSO-SGD algorithm for the Optimization of Convolutional Neural Network |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190531 |