CN111598925B - Visual target tracking method and device based on ECO algorithm and region growing segmentation - Google Patents
Visual target tracking method and device based on ECO algorithm and region growing segmentation Download PDFInfo
- Publication number
- CN111598925B CN111598925B CN202010410576.0A CN202010410576A CN111598925B CN 111598925 B CN111598925 B CN 111598925B CN 202010410576 A CN202010410576 A CN 202010410576A CN 111598925 B CN111598925 B CN 111598925B
- Authority
- CN
- China
- Prior art keywords
- sample
- target
- algorithm
- image
- eco
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20092—Interactive image processing based on input by user
- G06T2207/20101—Interactive definition of point of interest, landmark or seed
Abstract
The invention provides a visual target tracking method and a visual target tracking device based on an ECO algorithm and region growing segmentation, wherein the method comprises the following steps: reading a first frame image of a video or image sequence, and determining a tracking target; taking the current frame tracking target as a sample, and putting the sample into an ECO algorithm for training a correlation filter; reading the next frame of image, and obtaining a target position by utilizing a trained relevant filter convolution image candidate region; judging whether a target in the image is shielded or deformed; if shielding or deformation occurs, dividing the candidate region, and taking the divided target as a sample to replace a sample generated by an ECO algorithm; updating a sample set of the ECO algorithm by using a sample fusion updating strategy; training an ECO algorithm by using the acquired sample set, and updating a correlation filter; the above steps are circularly performed. According to the invention, when the target is shielded or deformed, the ECO algorithm sample set is updated by combining region growing segmentation, so that the training effect is improved.
Description
Technical Field
The invention relates to the field of computer vision application target tracking, in particular to a visual target tracking device based on an ECO algorithm and region growth segmentation.
Background
The visual target tracking is an important research direction in computer vision, has extremely wide application in the fields of military unmanned aerial vehicle, accurate guidance, air early warning and civil video monitoring, man-machine interaction, unmanned and the like, but the target tracking faces the challenges of target scale transformation, heavy shielding, rapid movement, beyond-view and illumination change and the like, so that the reliable real-time visual target tracking method is provided and has important practical significance.
The traditional related filtering visual target tracking algorithm mainly uses a given sample to train a judging classifier to judge whether the target is tracked or surrounding background information. The method mainly uses a rotation matrix to collect samples, uses fast Fourier transform to accelerate calculation of an algorithm, and an ECO target tracking algorithm focuses on solving the problem of oversized model on the basis, and reduces the model updating frequency to accelerate the tracking speed and resist model drift by reducing parameters, simplifying a training set. The method has good tracking effect on slight shielding, posture change and illumination change of the target, and is easy to cause tracking failure under the conditions of heavy shielding, rapid movement, out of view and the like of the target in the long-term tracking process of the target.
Disclosure of Invention
The invention aims to provide a visual target tracking method and device based on an ECO algorithm and region growing segmentation, which are used for solving the problem that training effect is affected due to incorrect sample acquisition of the ECO target tracking algorithm under the condition of target shielding or deformation.
The invention is realized in the following way:
in one aspect, the invention provides a visual target tracking method based on an ECO algorithm and region growing segmentation, comprising the following steps:
s1, reading a first frame image of a video or image sequence, and determining a tracking target;
s2, taking a current frame tracking target as a sample, and putting the sample into an ECO algorithm to train a correlation filter;
s3, reading a next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the position with the maximum probability of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the target position;
s4, judging whether a target in the image is shielded or deformed;
s5, dividing the candidate region if shielding or deformation occurs, taking the divided target as a sample to replace a sample generated by an ECO algorithm, and if shielding or deformation does not occur, still using the sample generated by the ECO algorithm;
s6, updating a sample set of the ECO algorithm by using a sample fusion updating strategy;
s7, training an ECO algorithm by using the acquired sample set, and updating a correlation filter;
and S8, circularly executing the steps S3-S7 until the current video or image sequence is processed.
Further, the step S2 specifically includes:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein the method comprises the steps ofN, M corresponds to the number of rows and columns of the two-dimensional discrete features;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the layer d characteristic channel corresponding to the correlation filter,continuous domain feature map for layer d feature channel,/for layer d feature channel>For all characteristic channels>Confidence score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j The corresponding weights, w, are regularization factors.
Further, the step S4 specifically includes:
and converting the target area identified by the current frame into a binary image, dividing the binary image into a plurality of small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated.
Further, in the step S5, the candidate region is segmented using a region growing segmentation algorithm, where the region growing segmentation algorithm is as follows:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold T, if the condition is met, merging (x, y) with (x 0, y 0), and pushing (x, y) into a stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
Further, the sample fusion update policy used in step S6 specifically includes:
setting the maximum sample number in the sample set, if:
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
On the other hand, the invention also provides a visual target tracking device based on the ECO algorithm and the region growing segmentation, which comprises the following components:
the tracking target determining module is used for reading a first frame image of the video or image sequence and determining a tracking target;
the filter training module is used for taking the current frame tracking target as a sample and putting the sample into an ECO algorithm to train a relevant filter;
the target position determining module is used for reading the next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the maximum probability position of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the current frame of image;
the shielding deformation judging module is used for judging whether a target in the image is shielded or deformed;
the sample determining module is used for dividing the candidate region if shielding or deformation occurs, taking the divided target as a sample to replace the sample generated by the ECO algorithm, and still using the sample generated by the ECO algorithm if shielding or deformation does not occur;
the sample set updating module is used for updating the sample set of the ECO algorithm by using a sample fusion updating strategy;
the filter updating module is used for training an ECO algorithm by using the acquired sample set and updating the related filter;
the circulation control module is used for controlling the target position determining module, the shielding deformation judging module, the sample determining module, the sample set updating module and the filter updating module to be executed in a circulation mode until the current video or image sequence is processed.
Further, the filter training module is specifically configured to:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein the method comprises the steps ofN, M corresponds to the number of rows and columns of the two-dimensional discrete features;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the layer d characteristic channel corresponding to the correlation filter,continuous domain feature map for layer d feature channel,/for layer d feature channel>For all characteristic channels>Confidence score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j The corresponding weights, w, are regularization factors.
Further, the shielding deformation judging module is specifically configured to:
and converting the target area identified by the current frame into a binary image, dividing the binary image into a plurality of small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated.
Further, the sample determination module specifically uses a region growing segmentation algorithm to segment the candidate region, and the region growing segmentation algorithm comprises the following steps:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold T, if the condition is met, merging (x, y) with (x 0, y 0), and pushing (x, y) into a stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
Further, the sample fusion updating strategy used by the sample set updating module is specifically:
setting the maximum sample number in the sample set, if:
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
Compared with the prior art, the invention has the following beneficial effects:
according to the visual target tracking method and device based on the ECO algorithm and the region growing segmentation, the ECO target tracking algorithm is adopted by the algorithm main body, the region growing segmentation algorithm is added at the same time, the situation that when the target is shielded or deformed, the target in an ECO algorithm frame is easy to deviate, the target tracking is affected when the target is not shielded or deformed, and then when the target is shielded or deformed, a target sample with a more complete shape contour is obtained by combining local region growing segmentation, the sample set of the ECO algorithm is updated, and the training effect is improved.
Drawings
FIG. 1 is a flowchart of a visual target tracking method based on an ECO algorithm and region growing segmentation provided by an embodiment of the invention;
fig. 2 is a block diagram of a visual target tracking device based on an ECO algorithm and region growing segmentation according to an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
As shown in fig. 1, an embodiment of the present invention provides a visual target tracking method based on an ECO algorithm and region growing segmentation, including the following steps:
s1, reading a first frame image of a video or image sequence, and manually setting or determining a tracking target through a target detection algorithm.
S2, taking a current frame tracking target as a sample, and putting the sample into an ECO algorithm to train a correlation filter; the method specifically comprises the following steps:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein the method comprises the steps ofN, M corresponds to the number of rows and columns of the two-dimensional discrete features;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the layer d characteristic channel corresponding to the correlation filter,continuous domain feature map for layer d feature channel,/for layer d feature channel>For all characteristic channels>Confidence score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j Corresponding weights, w is regularAnd (5) a chemokines.
S3, reading a next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the position with the maximum probability of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the target position.
S4, judging whether a target in the image is shielded or deformed; the method specifically comprises the following steps:
and converting the target area identified by the current frame into a binary image, dividing the binary image into 20 small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated. In other embodiments, the number of the blocks is not limited to 20, and can be reasonably selected according to practical situations.
S5, dividing the candidate region if shielding or deformation occurs, taking the divided target as a sample to replace a sample generated by an ECO algorithm, and if shielding or deformation does not occur, still using the sample generated by the ECO algorithm; the candidate regions are preferably segmented using a region growing segmentation algorithm which proceeds as follows:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold value T, if the condition is met, (x, y) and (x 0, y 0) are combined (in the same area), and (x, y) is pushed onto the stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
S6, updating a sample set of the ECO algorithm by using a sample fusion updating strategy; the sample fusion updating strategy specifically comprises the following steps:
setting the maximum sample number in the sample set, if:
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
S7, training an ECO algorithm by using the acquired sample set, and updating the correlation filter.
And S8, circularly executing the steps S3-S7 until the current video or image sequence is processed.
Based on the same inventive concept, the embodiment of the invention also provides a visual target tracking device based on the ECO algorithm and the region growing segmentation, and because the principle of the device for solving the technical problem is similar to that of the embodiment of the method, the implementation of the device can refer to the implementation of the method, and the repetition is omitted.
As shown in fig. 2, a visual target tracking device based on an ECO algorithm and region growing segmentation according to an embodiment of the present invention includes:
the tracking target determining module is used for reading a first frame image of the video or image sequence and determining a tracking target;
the filter training module is used for taking the current frame tracking target as a sample and putting the sample into an ECO algorithm to train a relevant filter;
the target position determining module is used for reading the next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the maximum probability position of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the current frame of image;
the shielding deformation judging module is used for judging whether a target in the image is shielded or deformed;
the sample determining module is used for dividing the candidate region if shielding or deformation occurs, taking the divided target as a sample to replace the sample generated by the ECO algorithm, and still using the sample generated by the ECO algorithm if shielding or deformation does not occur;
the sample set updating module is used for updating the sample set of the ECO algorithm by using a sample fusion updating strategy;
the filter updating module is used for training an ECO algorithm by using the acquired sample set and updating the related filter;
the circulation control module is used for controlling the target position determining module, the shielding deformation judging module, the sample determining module, the sample set updating module and the filter updating module to be executed in a circulation mode until the current video or image sequence is processed.
In one embodiment, the filter training module is specifically configured to:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein the method comprises the steps ofN, M corresponds to the number of rows and columns of the two-dimensional discrete features;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the layer d characteristic channel corresponding to the correlation filter,continuous domain feature map for layer d feature channel,/for layer d feature channel>For all characteristic channels>Confidence score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j The corresponding weights, w, are regularization factors.
In one embodiment, the shielding deformation determining module is specifically configured to:
and converting the target area identified by the current frame into a binary image, dividing the binary image into a plurality of small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated.
In one embodiment, the sample determination module segments the candidate region specifically using a region growing segmentation algorithm that proceeds as follows:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold T, if the condition is met, merging (x, y) with (x 0, y 0), and pushing (x, y) into a stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
In one embodiment, the sample fusion update policy used by the sample set update module is specifically:
setting the maximum sample number in the sample set, if:
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
In summary, according to the visual target tracking method and device based on the ECO algorithm and the region growing segmentation provided by the embodiment of the invention, the ECO target tracking algorithm is adopted by the algorithm main body, the region growing segmentation algorithm is added at the same time, the target in the ECO algorithm frame is observed to be easy to deviate when the target is shielded or deformed, so that the target tracking is affected when the target is not shielded or deformed, and then a target sample with a more complete shape contour is obtained by combining with the local region growing segmentation when the target is shielded or deformed, the sample set of the ECO algorithm is updated, and the training effect is improved.
Those of ordinary skill in the art will appreciate that all or a portion of the steps in the various methods of the embodiments may be implemented by a program that instructs associated hardware, the program may be stored on a computer readable storage medium, the storage medium may include: read Only Memory (ROM), random access Memory (RAM, random Access Memory), magnetic or optical disk, and the like.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.
Claims (10)
1. The visual target tracking method based on the ECO algorithm and the region growing segmentation is characterized by comprising the following steps of:
s1, reading a first frame image of a video or image sequence, and determining a tracking target;
s2, taking a current frame tracking target as a sample, and putting the sample into an ECO algorithm to train a correlation filter;
s3, reading a next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the position with the maximum probability of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the target position;
s4, judging whether a target in the image is shielded or deformed;
s5, dividing the candidate region based on a region growing dividing algorithm if shielding or deformation occurs, taking the divided target as a sample to replace a sample generated by an ECO algorithm, and still using the sample generated by the ECO algorithm if shielding or deformation does not occur;
s6, updating a sample set of the ECO algorithm by using a sample fusion updating strategy;
s7, training an ECO algorithm by using the acquired sample set, and updating a correlation filter;
s8, executing the steps S3-S7 in a circulating way until the current video or image sequence is processed.
2. The method for visual target tracking based on the ECO algorithm and the region growing partition as defined in claim 1, wherein said step S2 comprises the following steps:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein X (n, m) is the discrete feature after dimension reduction, and N, M corresponds to the number of rows and columns of the two-dimensional discrete feature;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the d layer characteristic channel corresponds to the relevant filter, J d {x d The d-th layer characteristic channel continuous domain characteristic diagram, x is all characteristic channels, S f { x } is the confidence of the score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j The corresponding weights, w, are regularization factors.
3. The method for visual target tracking based on the ECO algorithm and the region growing partition as defined in claim 1, wherein said step S4 comprises the following steps:
and converting the target area identified by the current frame into a binary image, dividing the binary image into a plurality of small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated.
4. The method for visual target tracking based on the ECO algorithm and the region-growing segmentation according to claim 1, wherein the candidate region is segmented in step S5 using a region-growing segmentation algorithm, the region-growing segmentation algorithm comprises the following steps:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold T, if the condition is met, merging (x, y) with (x 0, y 0), and pushing (x, y) into a stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
5. The method for tracking a visual target based on the ECO algorithm and the region growing segmentation as set forth in claim 1, wherein the sample fusion update strategy used in step S6 is specifically:
setting the maximum sample number in the sample set, if:
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
6. A visual target tracking apparatus based on an ECO algorithm and region growing segmentation, comprising:
the tracking target determining module is used for reading a first frame image of the video or image sequence and determining a tracking target;
the filter training module is used for taking the current frame tracking target as a sample and putting the sample into an ECO algorithm to train a relevant filter;
the target position determining module is used for reading the next frame of image, and acquiring a confidence coefficient map of the image candidate region by utilizing the features corresponding to the convolved image candidate region of the trained correlation filter, wherein the maximum probability position of the confidence coefficient map corresponds to the target position of the current frame of image and the confidence coefficient probability of the current frame of image;
the shielding deformation judging module is used for judging whether a target in the image is shielded or deformed;
the sample determining module is used for dividing the candidate region based on the region growing dividing algorithm if shielding or deformation occurs, taking the divided target as a sample to replace the sample generated by the ECO algorithm, and still using the sample generated by the ECO algorithm if shielding or deformation does not occur;
the sample set updating module is used for updating the sample set of the ECO algorithm by using a sample fusion updating strategy;
the filter updating module is used for training an ECO algorithm by using the acquired sample set and updating the related filter;
the circulation control module is used for controlling the target position determining module, the shielding deformation judging module, the sample determining module, the sample set updating module and the filter updating module to be executed in a circulation mode until the current video or image sequence is processed.
7. The ECO algorithm and region growing segmentation based visual target tracking device of claim 6, wherein said filter training module is specifically configured to:
firstly extracting multi-scale features x including hog features and color cn features from a candidate region z where a tracking target is located, performing pca dimension reduction on the extracted features, and then converting a time domain into a frequency domain through Fourier transformation, wherein a Fourier transformation formula is as follows:
wherein X (n, m) is the discrete feature after dimension reduction, and N, M corresponds to the number of rows and columns of the two-dimensional discrete feature;
the eigenvalues after fourier transform are further inserted into the continuous domain as follows:
wherein X is d Is a layer d feature, N d Line number for each layer of features, b d For the weight calculated in advance, T is the right side of the continuous interval, and then the convolution operation is performed on each feature map and the correlation filter, and the formula is as follows:
wherein f d Corresponding to the d layer characteristic channel corresponds to the relevant filter, J d {x d The d-th layer characteristic channel continuous domain characteristic diagram, x is all characteristic channels, S f { x } is the confidence of the score for sample x;
solving the correlation filter is accomplished by minimizing the loss function as follows:
wherein y is j For label, i.e. generated correspondingly from the current training sample and subject to gaussian distribution, alpha j For training sample x j The corresponding weights, w, are regularization factors.
8. The ECO algorithm and region growing segmentation based visual target tracking device of claim 6, wherein said occlusion distortion determination module is specifically configured to:
and converting the target area identified by the current frame into a binary image, dividing the binary image into a plurality of small blocks, finding a non-shielding target judged last time, equally dividing the blocks and subtracting the corresponding blocks, and if the difference value of each small block is larger than a set threshold value, considering that shielding or deformation is generated.
9. The ECO algorithm and region growing segmentation based visual target tracking device of claim 6, wherein the sample determination module segments candidate regions specifically using a region growing segmentation algorithm, the region growing segmentation algorithm process being as follows:
(1) Randomly selecting a pixel point (x 0, y 0) from four sides of a target area identified by an ECO algorithm;
(2) Taking (x 0, y 0) as a center, considering that the absolute value of the difference between the gray values of the (x, y) and seed pixels of the 4 neighborhood or 8 neighborhood pixels of (x 0, y 0) is smaller than a certain threshold T, if the condition is met, merging (x, y) with (x 0, y 0), and pushing (x, y) into a stack;
(3) Fetching a pixel from the stack and returning it to step (2) as (x 0, y 0);
(4) Returning to step (1) when the stack is empty;
(5) Repeating the steps (1) - (4) until each point in the image is attributed, and ending the growth.
10. The ECO algorithm and region growth segmentation based visual target tracking device of claim 6, wherein the sample set update module uses a sample fusion update strategy of:
setting the maximum sample number in the sample set, if
(1) The sample set is not full, and a new sample is placed at a vacant place;
(2) The sample set is full, and samples with a weight lower than a threshold value are in the sample set, and at the moment, the old samples are replaced by the new samples;
(3) The sample set is full, and the minimum value in the distances between the new sample and all the existing samples is smaller than the minimum value between the distances of the existing samples, and the new sample and the latest sample are fused at the moment;
(4) The sample set is full, the minimum value of the distances between the new sample and all the existing samples is larger than the minimum value of the distances between the existing samples, at the moment, the two nearest existing samples are fused, and the new sample is placed at the blank of the fused sample set.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010410576.0A CN111598925B (en) | 2020-05-15 | 2020-05-15 | Visual target tracking method and device based on ECO algorithm and region growing segmentation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010410576.0A CN111598925B (en) | 2020-05-15 | 2020-05-15 | Visual target tracking method and device based on ECO algorithm and region growing segmentation |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111598925A CN111598925A (en) | 2020-08-28 |
| CN111598925B true CN111598925B (en) | 2023-10-17 |
Family
ID=72187164
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010410576.0A Active CN111598925B (en) | 2020-05-15 | 2020-05-15 | Visual target tracking method and device based on ECO algorithm and region growing segmentation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111598925B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111986236A (en) * | 2020-09-01 | 2020-11-24 | 安徽炬视科技有限公司 | Anti-occlusion target tracking algorithm based on online learning |
| CN112330719B (en) * | 2020-12-02 | 2024-02-27 | 东北大学 | Deep learning target tracking method based on feature map segmentation and self-adaptive fusion |
| CN113030943B (en) * | 2021-03-05 | 2023-08-18 | 中国人民解放军空军工程大学航空机务士官学校 | Multi-target tracking algorithm based on monopulse radar signal acquisition azimuth range profile |
| CN113066108A (en) * | 2021-04-14 | 2021-07-02 | 武汉卓目科技有限公司 | Anti-occlusion visual target tracking method and device based on ECO algorithm |
| CN113223054B (en) * | 2021-05-28 | 2023-03-24 | 武汉卓目科技有限公司 | Target tracking method and device for improving jitter property of ECO (equal cost offset) tracking frame |
| CN113313738B (en) * | 2021-07-15 | 2021-10-01 | 武汉卓目科技有限公司 | Unmanned aerial vehicle target tracking method and device based on ECO and servo linkage |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596951A (en) * | 2018-03-30 | 2018-09-28 | 西安电子科技大学 | A kind of method for tracking target of fusion feature |
| CN109887001A (en) * | 2019-01-31 | 2019-06-14 | 中国科学院深圳先进技术研究院 | Method for tracking target, device, computer equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109448022B (en) * | 2018-10-16 | 2021-08-06 | 天津津航技术物理研究所 | Infrared anti-occlusion tracking method for guidance system |
| AU2020100044A4 (en) * | 2020-01-10 | 2020-02-13 | Lin, Xichan Miss | Method of tracking of Surgical Target and Tool |
-
2020
- 2020-05-15 CN CN202010410576.0A patent/CN111598925B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108596951A (en) * | 2018-03-30 | 2018-09-28 | 西安电子科技大学 | A kind of method for tracking target of fusion feature |
| CN109887001A (en) * | 2019-01-31 | 2019-06-14 | 中国科学院深圳先进技术研究院 | Method for tracking target, device, computer equipment and storage medium |
Non-Patent Citations (5)
| Title |
|---|
| Martin Danelljan等.Beyond correlation filters: Learning continuous convolution operators for visual tracking.Computer vision - ECCV 2016.2016,第472-488页. * |
| Martin Danelljan等.ECO Efficient Convolution Operators for Tracking.2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2017,第6931-6939页. * |
| 杨露菁等.智能图像处理及应用.中国铁道出版社有限公司,2019,第178页. * |
| 赵涓涓.基于PET-CT的肺癌早期计算机辅助诊断技术.国防工业出版社,2015,第24-26页. * |
| 郭洪伟.基于ECO的视频目标跟踪.中国优秀博硕士学位论文全文数据库（硕士） 信息科技辑.2020,(第2期),第20-21页. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111598925A (en) | 2020-08-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111598925B (en) | Visual target tracking method and device based on ECO algorithm and region growing segmentation | |
| CN108710865B (en) | Driver abnormal behavior detection method based on neural network | |
| CN111666921B (en) | Vehicle control method, apparatus, computer device, and computer-readable storage medium | |
| CN108053419B (en) | Multi-scale target tracking method based on background suppression and foreground anti-interference | |
| CN111445478B (en) | Automatic intracranial aneurysm region detection system and detection method for CTA image | |
| CN107633226B (en) | Human body motion tracking feature processing method | |
| CN109658442B (en) | Multi-target tracking method, device, equipment and computer readable storage medium | |
| CN112036231B (en) | Vehicle-mounted video-based lane line and pavement indication mark detection and identification method | |
| CN110008900B (en) | Method for extracting candidate target from visible light remote sensing image from region to target | |
| CN111612817A (en) | Target tracking method based on depth feature adaptive fusion and context information | |
| CN111192294B (en) | Target tracking method and system based on target detection | |
| CN111415364B (en) | Conversion method, system and storage medium for image segmentation sample in computer vision | |
| CN112766170B (en) | Self-adaptive segmentation detection method and device based on cluster unmanned aerial vehicle image | |
| CN109448024B (en) | Visual tracking method and system for constructing constraint correlation filter by using depth data | |
| CN112700469A (en) | Visual target tracking method and device based on ECO algorithm and target detection | |
| CN112686247A (en) | Identification card number detection method and device, readable storage medium and terminal | |
| CN110751671B (en) | Target tracking method based on kernel correlation filtering and motion estimation | |
| CN110751163A (en) | Target positioning method and device, computer readable storage medium and electronic equipment | |
| CN113066108A (en) | Anti-occlusion visual target tracking method and device based on ECO algorithm | |
| CN111598926B (en) | Target tracking method and device for optimizing ECO feature extraction performance | |
| CN112861652A (en) | Method and system for tracking and segmenting video target based on convolutional neural network | |
| CN113223054B (en) | Target tracking method and device for improving jitter property of ECO (equal cost offset) tracking frame | |
| CN113313738B (en) | Unmanned aerial vehicle target tracking method and device based on ECO and servo linkage | |
| CN115984316B (en) | Industrial image edge extraction method and device for complex environment | |
| CN113192106B (en) | Livestock tracking method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |