CN112614150A - Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning - Google Patents
Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning Download PDFInfo
- Publication number
- CN112614150A CN112614150A CN202011511434.XA CN202011511434A CN112614150A CN 112614150 A CN112614150 A CN 112614150A CN 202011511434 A CN202011511434 A CN 202011511434A CN 112614150 A CN112614150 A CN 112614150A
- Authority
- CN
- China
- Prior art keywords
- pedestrian
- model
- training
- label
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2155—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning, wherein the method comprises the following steps: selecting two pedestrian tracking models based on a neural network, and performing supervised training on the pedestrian tracking models by using labeled training data; predicting the pseudo label and performing offline interpolation optimization; learning by adopting an interactive semi-supervised method; final prediction and output. The invention adopts an off-line interpolation optimization method, utilizes complete video information, and carries out interpolation correction on the disconnected part of the pedestrian track, so that the pedestrian track disconnection condition is less, and the influence of pedestrian shielding is less. The invention also provides a method for performing semi-supervised self-learning on test data by using the dual models, so that the models can be gradually familiar with the test data, the performance is greatly improved after multiple rounds of iteration, and good performance can be kept in the face of scenes which are not seen by the models.
Description
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to an offline pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning.
Background
The pedestrian tracking technology is a core part of a human-centered video analysis technology, and is a technical premise of many important downstream applications, such as pedestrian search, behavior recognition, event analysis and the like. Generally, pedestrian tracking mainly comprises two parts, namely, the first part is to detect pedestrians in a single-frame image, namely, to locate all pedestrians in a picture and output a boundary frame of each pedestrian; the second part is to perform data correlation on all bounding boxes of the same pedestrian in consecutive adjacent frames, i.e. to form a trajectory belonging to the particular pedestrian using pedestrian Re-identification (Person Re-ID).
Based on the different implementation methods of the two parts, the existing pedestrian tracking technology is mainly divided into two groups, one is a 'two-step walking' method, namely, pedestrian detection is firstly carried out on a picture, then local image features of pedestrians are extracted, and Re-ID is carried out, and the other is a 'one-time' method, namely, a multi-task learning structure is used, so that a network simultaneously completes image pedestrian detection and Re-ID feature extraction tasks to obtain a faster reasoning speed.
The pedestrian track detected by the existing pedestrian tracking technology is often broken, the condition that middle frames are not detected often occurs, because the environment in a real scene is very complex, the pedestrian shielding phenomenon is serious, pedestrians are often only visible on the half body, the head or the legs, the sensitivity of the pedestrian detection technology to illumination and posture change is added, most of the existing pedestrian tracking technologies are not optimized for the pedestrian shielding phenomenon in the real and complex scene, the existing pedestrian tracking technology is almost an online tracking technology by depending on the boundary frame result provided by a pedestrian detection part, namely, the current frame result is output by only using the information of the current frame or the past frame, and the subsequent video information is not used for correcting the result.
Meanwhile, almost all pedestrian tracking technologies use a basic transfer learning method, a large amount of data sets are used for pre-training a model, then simple transfer learning is performed on part of marked test data or real industrial data, and then actual business prediction is performed on unmarked real data, but a large difference (such as illumination, background, angle, pedestrian characteristics and the like) often exists between data used for training a pedestrian tracking model and business data of a real scene which is not seen by the model, and if the simple transfer learning is used, the performance is often poor. Almost all existing pedestrian tracking technologies do not take into account the "data set adaptation" problem described above, with "perform well on the training set, but poorly on the test set".
Disclosure of Invention
The invention mainly aims to overcome the defects in the prior art and provide an offline pedestrian tracking method, an offline pedestrian tracking system and a storage medium based on dual-model interactive semi-supervised learning.
In order to achieve the purpose, the invention adopts the following technical scheme:
the invention provides an off-line pedestrian tracking method based on dual-model interactive semi-supervised learning, which comprises the following steps:
s1, selecting two pedestrian tracking models based on the neural network, and performing supervised training on the pedestrian tracking models by using the labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1And FT2；
S2 order iteration model FS1＝FT1，FS2＝FT2；
S3, Using the iterative model FS1And FS2Directly predicting the label-free test data, and simultaneously optimizing the result by using an offline interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2; the off-line interpolation optimization method is used for splicing disconnected pedestrian tracks and recovering pedestrian track boundary frames with lost intermediate frames;
s4, learning by adopting an interactive semi-supervised method: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, it goes againstTo model the iteration FS2Combining the output pseudo label 2 with label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with label training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
s5, using the iterated model FS1And FS2And predicting the label-free test data to obtain a final output result.
As a preferred technical solution, in step S3, the offline interpolation optimization method includes the steps of:
(1) judging an effective track, reserving the effective pedestrian track and participating in an interpolation optimization process;
(2) and judging whether the interval length of the front and rear frame numbers of the disconnected part of each effective pedestrian track meets the interpolation optimization condition or not, and performing the interpolation optimization process.
As a preferred technical solution, in step S3, in the offline interpolation optimization method, the specific judgment of the effective trajectory is as follows:
for the pedestrian track, when the total track frame number N is larger than the set track minimum frame number threshold NminAnd "confidence level above minimum confidence threshold thr" is satisfied in the traceconf"number of frames
Nt＝|{conft|conft＞thrconf,t＝1,2,3,...,N}|
Greater than a threshold N of a minimum number of qualified framesvalThen it is considered as a valid track, otherwise it is directly ignored.
As a preferred technical solution, in step S3, in the offline interpolation optimization method, the interpolation optimization conditions specifically include:
for the disconnected position of each effective pedestrian track, defining the frame number of the last frame of a continuous frame before the disconnected position as ftThe frame number of the first frame of a section of continuous frames after the disconnection position is ft+1Judging the number of lost frames (f)t+1-ft) Whether the following equation is satisfied:
1＜(ft+1-ft)＜δmax，
wherein deltamaxIf the preset maximum interval length meets the formula, the disconnected part is considered to meet the interpolation optimization condition, and interpolation can be carried out; if the formula is not satisfied, the disconnected part is considered to not satisfy the interpolation optimization condition, and interpolation is not performed.
As a preferred technical solution, in step S3, in the offline interpolation optimization method, an interpolation optimization process is used to supplement the pedestrian frame coordinates lost at the disconnection point, and f, f is a frame for each framet＜f＜ft+1The corresponding calculation formula of the pedestrian boundary frame b is as follows:
wherein, b ═ xmin,ymin,xmax,ymax]，xmin,ymin,xmax,ymaxRespectively representing the minimum value and the maximum value of the pedestrian boundary frame on x and y coordinate axes; bt+1A pedestrian boundary frame corresponding to a first frame of a section of continuous frames after the disconnection; btA pedestrian boundary frame corresponding to the last frame of a section of continuous frames before the disconnection;
the offline interpolation optimization process is mainly used to improve FS1And FS2The output quality of the semi-supervised learning process and the robustness of the semi-supervised learning process.
Preferably, in step S4, the pair FS1And FS2Performing cycle iteration training refers to repeatedly using FS1And FS2The output results mutually guide the learning and training process of the other side, so that the models can respectively take the advantages and mutually improve the advantages.
As a preferred technical solution, in step S5, the final output result may adopt one of the results in the dual model as the final prediction.
The invention also provides an off-line pedestrian tracking system based on dual-model interactive semi-supervised learning, which comprises a model pre-training module, a pseudo label prediction module, an interactive semi-supervised learning module and a final prediction and output module;
the model pre-training module performs supervised training on two selected pedestrian tracking models based on the neural network by using labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1、FT2And an iterative model FS1、FS2；
The pseudo tag prediction module utilizes an iterative model FS1And FS2Predicting the label-free test data and optimizing by using an off-line interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2;
the interactive semi-supervised learning module is used for executing an interactive semi-supervised learning method, and specifically comprises the following steps: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with the label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with the labeled training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
a final prediction and output module: using the iterated model FS1And FS2And predicting the label-free test data to obtain a final output result.
The invention also provides a storage medium which stores a program, and the program is characterized in that when being executed by a processor, the off-line pedestrian tracking method based on the dual-model interactive semi-supervised learning is realized.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) the invention adopts an off-line interpolation optimization method to perform interpolation supplement optimization processing on the frame losing the boundary frame, and solves the problem that the prior art is generally in an on-line type, only depends on the information of the current frame and the past frame, and cannot correct the track by utilizing the subsequent information, thereby achieving the technical effects of less pedestrian track disconnection and less influence of pedestrian shielding.
(2) The invention adopts a semi-supervised learning method, performs multi-round semi-supervised learning on unseen scene data, partially masters scene modes, and solves the problems that the prior art uses simple transfer learning, always performs prediction on strange scenes during final prediction, the effect is generally not stable, and the performance is greatly reduced during testing, thereby achieving the technical effects of keeping good performance and outputting better results on the unseen scenes of the model.
Drawings
FIG. 1 is a flowchart of an offline pedestrian tracking method based on dual-model interactive semi-supervised learning according to an embodiment of the present invention;
FIG. 2 is a schematic structural diagram of an offline pedestrian tracking system based on dual-model interactive semi-supervised learning according to an embodiment of the present invention;
fig. 3 is a schematic structural diagram of a storage medium according to an embodiment of the present invention.
Detailed Description
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
Examples
As shown in fig. 1, the offline pedestrian tracking method based on dual-model interactive semi-supervised learning provided by the present invention includes the following training processes:
s1, selecting two pedestrian tracking models with better performance based on the neural network, and carrying out supervised training on the pedestrian tracking models by using the labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1And FT2；
S2 order iteration model FS1＝FT1，FS2＝FT2；
S3, Using the iterative model FS1And FS2Directly predicting the label-free test data, and simultaneously optimizing the result by using an offline interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2; the off-line interpolation optimization method is used for splicing disconnected pedestrian tracks and recovering pedestrian track boundary frames with lost intermediate frames;
s4, interactive semi-supervised learning method: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with label training data to be used as an iterative model FS1Training the training data; repeating the step F for 3-4 timesS1And FS2The output results mutually guide the learning and training process of the other side to carry out the loop iteration training so as to improve FS1And FS2The output quality of the system and the robustness of the semi-supervised learning process;
s5, using the iterated model FS1And FS2And predicting the unlabeled test data, wherein the final output result can adopt one result in the dual models as the final prediction, or adopt a result obtained by continuously fusing the results of the two models as the final prediction.
The offline interpolation optimization method in step S3 can splice the broken pedestrian tracks, and the boundary frame with the lost intermediate frame can be recovered, which can improve the track breaking phenomenon caused by pedestrian occlusion or unstable detection algorithm, and at the same time, will play an important role in the interactive semi-supervised learning training method, and effectively improve the robustness of the semi-supervised learning method. The off-line interpolation optimization method comprises the following specific steps:
(1) and judging the effective track. For the pedestrian track, when the total track frame number N is larger than the set track minimum frame number threshold NminAnd "confidence level above minimum confidence threshold thr" is satisfied in the traceconf"number of frames
Nt＝|{conft|conft＞thrconf,t＝1,2,3,...,N}|
Greater than a threshold N of a minimum number of qualified framesvalIf the track is valid, the track is considered to be a valid track, otherwise, the track is directly ignored; wherein, the track minimum frame number threshold NminThe frame rate of the video is generally set, and if the video is 30 frames per second, N is the valuemin30; minimum qualified frame number threshold NvalIs generally set to 5;
(2) reserving an effective pedestrian track and participating in an interpolation optimization process;
(3) and judging whether the interval length of the front and rear frame numbers of the disconnected part of each effective pedestrian track meets the interpolation optimization condition. For the disconnected position of each effective pedestrian track, defining the frame number of the last frame of a continuous frame before the disconnected position as ftThe frame number of the first frame of a section of continuous frames after the disconnection position is ft+1Judging the number of lost frames (f)t+1-ft) Whether the following equation is satisfied:
1＜(ft+1-ft)＜δmax，
wherein deltamaxFor a set maximum interval length, it is generally set to Nmin/3, i.e. delta, when the video frame rate is 30max＝10；
If the above formula is satisfied, the disconnected connection is considered to satisfy the interpolation optimization condition, and interpolation can be performed; if the formula is not satisfied, the disconnected connection is considered to not satisfy the interpolation optimization condition, and interpolation is not performed;
(4) and (5) an interpolation optimization process. For the disconnected part meeting the interpolation optimization condition, the interpolation optimization process supplements the missing pedestrian frame coordinates, and f (f) is carried out for each framet＜f＜ft+1) The corresponding calculation formula of the pedestrian boundary frame b is as follows:
wherein, b ═ xmin,ymin,xmax,ymax]，xmin,ymin,xmax,ymaxRespectively representing the minimum value and the maximum value of the pedestrian boundary frame on x and y coordinate axes; bt+1A pedestrian boundary frame corresponding to a first frame of a section of continuous frames after the disconnection; btThe pedestrian boundary frame corresponding to the last frame of the previous continuous frame at the disconnected position.
Through the step S1, two slightly reliable original models are obtained, the pseudo labels output by the unlabeled data for the first time have certain reference value, meanwhile, the two models are bound to show the advantages and disadvantages on different videos, the models can learn the advantages of the other side better through a mutual guidance mode, the models cannot be easily 'bad' due to the addition of training data, and finally, due to the fact that the output results of the two iterative models both use an interpolation frame optimization technology, the performance of the models can be guaranteed to develop towards better and better directions, and the robustness of semi-supervised learning is improved. After several iterations, both models will have a larger lift than the original model.
Furthermore, the double-model interactive semi-supervised learning method can be further expanded into a multi-model semi-supervised learning method: when training an initial model in the first step of the method, three or even more pedestrian tracking models with good or bad can be adopted for initialization; the subsequent interactive semi-supervised learning guide training process can randomly distribute the pseudo labels output by the plurality of models.
As shown in fig. 2, the present embodiment further provides an offline pedestrian tracking system based on dual-model interactive semi-supervised learning, which includes a model pre-training module, a pseudo label prediction module, an interactive semi-supervised learning module, and a final prediction and output module;
the model pre-training module performs supervised training on two selected pedestrian tracking models based on the neural network by using labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1、FT2And an iterative model FS1、FS2；
The pseudo tag prediction module utilizes an iterative model FS1And FS2Predicting the label-free test data and optimizing by using an off-line interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2;
the interactive semi-supervised learning module is used for executing an interactive semi-supervised learning method, and specifically comprises the following steps: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with the label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with the labeled training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
a final prediction and output module: using the iterated model FS1And FS2And predicting the label-free test data to obtain a final output result.
It should be noted that the system provided in the above embodiment is only illustrated by the division of the above functional modules, and in practical applications, the above function allocation may be completed by different functional modules according to needs, that is, the internal structure is divided into different functional modules to complete all or part of the above described functions.
As shown in fig. 3, this embodiment further provides a storage medium storing a program, where the program, when executed by a processor, implements an offline pedestrian tracking method based on dual-model interactive semi-supervised learning, specifically:
selecting two pedestrian tracking models with better performance based on the neural network, and carrying out supervised training on the pedestrian tracking models by using labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1And FT2；
Order iteration model FSI＝FT1，FS2＝FT2；
Using the iterative model FS1And FS2Directly predicting the label-free test data, and simultaneously optimizing the result by using an offline interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2; the off-line interpolation optimization method is used for splicing disconnected pedestrian tracks and recovering pedestrian track boundary frames with lost intermediate frames;
learning by adopting an interactive semi-supervised method: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with label training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
using the iterated model FS1And FS2For no markAnd predicting the test data to obtain the final output result.
It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.
Claims (9)
1. An off-line pedestrian tracking method based on dual-model interactive semi-supervised learning is characterized by comprising the following steps:
selecting two pedestrian tracking models based on a neural network, and carrying out supervised training on the pedestrian tracking models by using labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1And FT2；
Order iteration model FS1＝FT1，FS2＝FT2；
Using the iterative model FS1And FS2Directly predicting the label-free test data, and simultaneously optimizing the result by using an offline interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2; the off-line interpolation optimization method is used for splicing disconnected pedestrian tracks and enabling the pedestrian tracks with lost intermediate framesRestoring the trace bounding box;
learning by adopting an interactive semi-supervised method: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with label training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
using the iterated model FS1And FS2And predicting the label-free test data to obtain a final output result.
2. The offline pedestrian tracking method based on the dual-model interactive semi-supervised learning as claimed in claim 1, wherein the offline interpolation optimization method comprises the following steps:
judging an effective track, reserving the effective pedestrian track and participating in an interpolation optimization process;
and judging whether the interval length of the front and rear frame numbers of the disconnected part of each effective pedestrian track meets the interpolation optimization condition or not, and performing the interpolation optimization process.
3. The offline pedestrian tracking method based on the dual-model interactive semi-supervised learning as recited in claim 1 or 2, wherein the offline interpolation optimization method specifically determines the effective trajectory as follows:
for the pedestrian track, when the total track frame number N is larger than the set track minimum frame number threshold NminAnd "confidence level above minimum confidence threshold thr" is satisfied in the traceconf"number of frames
Nt＝|{conft|conft＞thrrconf，t＝1，2，3，...，N}|
Greater than a threshold N of a minimum number of qualified framesvalThen it is considered as a valid track, otherwise it is directly ignored.
4. The offline pedestrian tracking method based on dual-model interactive semi-supervised learning according to claim 1 or 2, wherein in the offline interpolation optimization method, the interpolation optimization conditions are specifically as follows:
for the disconnected position of each effective pedestrian track, defining the frame number of the last frame of a continuous frame before the disconnected position as ftThe frame number of the first frame of a section of continuous frames after the disconnection position is ft+1Judging the number of lost frames (f)t+1-ft) Whether the following equation is satisfied:
1＜(ft+1-ft)＜δmax，
wherein deltamaxIf the preset maximum interval length meets the formula, the disconnected part is considered to meet the interpolation optimization condition, and interpolation can be carried out; if the formula is not satisfied, the disconnected part is considered to not satisfy the interpolation optimization condition, and interpolation is not performed.
5. The offline pedestrian tracking method based on dual-model interactive semi-supervised learning as claimed in claim 1 or 2, wherein in the offline interpolation optimization method, an interpolation optimization process is used to supplement the pedestrian frame coordinates lost at the disconnected part, and for each frame f, ft＜f＜ft+1The corresponding calculation formula of the pedestrian boundary frame b is as follows:
wherein, b ═ xmin，ymin，xmax，ymax]，xmin，ymin，xmax，ymaxRespectively representing the minimum value and the maximum value of the pedestrian boundary frame on x and y coordinate axes; bt+1After the connection is brokenA pedestrian bounding box corresponding to a first frame of a segment of continuous frames; btA pedestrian boundary frame corresponding to the last frame of a section of continuous frames before the disconnection;
the offline interpolation optimization procedure is used to increase FS1And FS2The output quality of the semi-supervised learning process and the robustness of the semi-supervised learning process.
6. The method of claim 1, wherein the pair F is used for off-line pedestrian tracking based on dual-model interactive semi-supervised learningS1And FS2Performing cycle iteration training refers to repeatedly using FS1And FS2The output results mutually guide the learning and training process of the other side, so that the models can respectively take the advantages and mutually improve the advantages.
7. The method of claim 1, wherein the final output result adopts one of the results in the dual model as the final prediction.
8. An off-line pedestrian tracking system based on double-model interactive semi-supervised learning, which is characterized in that the off-line pedestrian tracking system is applied to the off-line pedestrian tracking method based on double-model interactive semi-supervised learning of any one of claims 1 to 7, and comprises a model pre-training module, a pseudo label prediction module, an interactive semi-supervised learning module and a final prediction and output module;
the model pre-training module performs supervised training on two selected pedestrian tracking models based on the neural network by using labeled training data until the models can basically fit the training data to obtain initial model weight F with good prediction performance on a training data setT1、FT2And an iterative model FS1、FS2；
The pseudo tag prediction module utilizes an iterative model FS1And FS2Predicting the label-free test data and optimizing by using an off-line interpolation optimization method to obtain the iterative model FS1And FS2The output results of (1), namely pseudo tag 1 and pseudo tag 2;
the interactive semi-supervised learning module is used for executing an interactive semi-supervised learning method, and specifically comprises the following steps: using the iterative model FS1The output pseudo label 1 is used as a training label of the label-free test data to obtain pseudo label data 1, and the pseudo label data 1 and the labeled training data are mixed together to be used as the iterative model FS2Training the training data again; in the same way, the iterative model F is reversely usedS2Combining the output pseudo label 2 with the label-free test data to obtain pseudo label data 2, and mixing the pseudo label data 2 with the labeled training data to be used as an iterative model FS1Training the training data; for the FS1And FS2Performing loop iteration training;
a final prediction and output module: using the iterated model FS1And FS2And predicting the label-free test data to obtain a final output result.
9. A storage medium storing a program, wherein the program, when executed by a processor, implements the offline pedestrian tracking method based on dual-model interactive semi-supervised learning of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011511434.XA CN112614150A (en) | 2020-12-18 | 2020-12-18 | Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011511434.XA CN112614150A (en) | 2020-12-18 | 2020-12-18 | Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112614150A true CN112614150A (en) | 2021-04-06 |
Family
ID=75243630
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011511434.XA Pending CN112614150A (en) | 2020-12-18 | 2020-12-18 | Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112614150A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630367A (en) * | 2023-07-25 | 2023-08-22 | 苏州浪潮智能科技有限公司 | Target tracking method, device, electronic equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027421A (en) * | 2019-11-26 | 2020-04-17 | 西安宏规电子科技有限公司 | Graph-based direct-push type semi-supervised pedestrian re-identification method |
| WO2020232909A1 (en) * | 2019-05-20 | 2020-11-26 | 平安科技（深圳）有限公司 | Pedestrian visual tracking method, model training method and device, apparatus and storage medium |
-
2020
- 2020-12-18 CN CN202011511434.XA patent/CN112614150A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020232909A1 (en) * | 2019-05-20 | 2020-11-26 | 平安科技（深圳）有限公司 | Pedestrian visual tracking method, model training method and device, apparatus and storage medium |
| CN111027421A (en) * | 2019-11-26 | 2020-04-17 | 西安宏规电子科技有限公司 | Graph-based direct-push type semi-supervised pedestrian re-identification method |
Non-Patent Citations (1)
| Title |
|---|
| ANCONG WU ET AL.: "Transductive Multi-Object Tracking in Complex Events by Interactive Self-Training", 《PROCEEDINGS OR THE 28TH ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116630367A (en) * | 2023-07-25 | 2023-08-22 | 苏州浪潮智能科技有限公司 | Target tracking method, device, electronic equipment and storage medium |
| CN116630367B (en) * | 2023-07-25 | 2023-11-03 | 苏州浪潮智能科技有限公司 | Target tracking method, device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109961034B (en) | Video target detection method based on convolution gating cyclic neural unit | |
| Nam et al. | Modeling and propagating cnns in a tree structure for visual tracking | |
| WO2023056889A1 (en) | Model training and scene recognition method and apparatus, device, and medium | |
| CN109426805B (en) | Method, apparatus and computer program product for object detection | |
| Yeo et al. | Superpixel-based tracking-by-segmentation using markov chains | |
| US9165199B2 (en) | Controlled human pose estimation from depth image streams | |
| Badrinarayanan et al. | Semi-supervised video segmentation using tree structured graphical models | |
| Zhao et al. | Unsupervised monocular depth estimation in highly complex environments | |
| CN112801019B (en) | Method and system for eliminating re-identification deviation of unsupervised vehicle based on synthetic data | |
| KR20190056792A (en) | System and method for face detection and emotion recognition based deep-learning | |
| CN111898566B (en) | Attitude estimation method, attitude estimation device, electronic equipment and storage medium | |
| Yoon et al. | A novel online action detection framework from untrimmed video streams | |
| CN113393496A (en) | Target tracking method based on space-time attention mechanism | |
| CN112614150A (en) | Off-line pedestrian tracking method, system and storage medium based on dual-model interactive semi-supervised learning | |
| Liu et al. | A deep Q-learning network based active object detection model with a novel training algorithm for service robots | |
| Lu et al. | Label-efficient video object segmentation with motion clues | |
| CN112199994B (en) | Method and device for detecting interaction of3D hand and unknown object in RGB video in real time | |
| CN116188785A (en) | Polar mask old man contour segmentation method using weak labels | |
| He et al. | CPSPNet: Crowd counting via semantic segmentation framework | |
| CN114694261A (en) | Video three-dimensional human body posture estimation method and system based on multi-level supervision graph convolution | |
| CN113920170A (en) | Pedestrian trajectory prediction method and system combining scene context and pedestrian social relationship and storage medium | |
| CN113989920A (en) | Athlete behavior quality assessment method based on deep learning | |
| Yang et al. | Adaptive fusion of RGBD data for two-stream FCN-based level set tracking | |
| Hardy et al. | LInKs" Lifting Independent Keypoints"-Partial Pose Lifting for Occlusion Handling With Improved Accuracy in 2D-3D Human Pose Estimation | |
| Qiu et al. | Deepblindness: Fast blindness map estimation and blindness type classification for outdoor scene from single color image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20210406 |
|
| WD01 | Invention patent application deemed withdrawn after publication |