CN117455958B - Track tracking method and device for sperm identification - Google Patents
Track tracking method and device for sperm identification Download PDFInfo
- Publication number
- CN117455958B CN117455958B CN202311790237.XA CN202311790237A CN117455958B CN 117455958 B CN117455958 B CN 117455958B CN 202311790237 A CN202311790237 A CN 202311790237A CN 117455958 B CN117455958 B CN 117455958B
- Authority
- CN
- China
- Prior art keywords
- sperm
- frame
- track
- frames
- matching
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 70
- 238000001514 detection method Methods 0.000 claims abstract description 67
- 230000033001 locomotion Effects 0.000 claims abstract description 47
- 239000013598 vector Substances 0.000 claims abstract description 38
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 29
- 238000012549 training Methods 0.000 claims abstract description 29
- 230000003287 optical effect Effects 0.000 claims abstract description 17
- 238000004364 calculation method Methods 0.000 claims abstract description 16
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 230000001502 supplementing effect Effects 0.000 claims abstract description 10
- 238000009499 grossing Methods 0.000 claims description 19
- 239000011159 matrix material Substances 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 7
- 230000006870 function Effects 0.000 claims description 6
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 239000000284 extract Substances 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000013135 deep learning Methods 0.000 claims description 3
- 230000002950 deficient Effects 0.000 claims description 3
- 238000002372 labelling Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 239000013589 supplement Substances 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 16
- 238000004458 analytical method Methods 0.000 description 7
- 230000019100 sperm motility Effects 0.000 description 6
- 238000006243 chemical reaction Methods 0.000 description 5
- 230000000875 corresponding effect Effects 0.000 description 5
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000000877 morphologic effect Effects 0.000 description 3
- 238000012216 screening Methods 0.000 description 3
- 238000009825 accumulation Methods 0.000 description 2
- 239000012535 impurity Substances 0.000 description 2
- 230000001965 increasing effect Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 239000000047 product Substances 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 208000007466 Male Infertility Diseases 0.000 description 1
- 238000012300 Sequence Analysis Methods 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000012217 deletion Methods 0.000 description 1
- 230000037430 deletion Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 208000000509 infertility Diseases 0.000 description 1
- 230000036512 infertility Effects 0.000 description 1
- 231100000535 infertility Toxicity 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 210000000582 semen Anatomy 0.000 description 1
- 238000003860 storage Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Abstract
The invention requests protection of a track tracking method and a track tracking device oriented to sperm identification, which are used for carrying out frame segmentation on a sperm video to be detected to obtain a plurality of sperm detection frames to form a sperm detection frame set; training a sperm detection model, and carrying out feature recognition on a plurality of sperm detection frames to obtain a feature vector set of each frame; performing similarity matching on a characteristic vector set of a target sperm between two frames in a plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm; and carrying out chain combining operation and chain supplementing operation on the first motion trail to obtain a second motion trail of the target sperm. The invention improves the registration rate, reduces the loss rate caused by detector errors and shielding overlapping by adopting a method for keeping track of frame skip matching, optimizes the track chain by adopting a chain combining operation, predicts and supplements the chain for sperms by utilizing optical flow calculation and historical track information, improves the robustness of the track chain and reduces the dependence on hardware.
Description
Technical Field
The invention belongs to the technical field of computer-aided sperm sample analysis, and particularly relates to a sperm identification-oriented track tracking method and device.
Background
The global population is affected by infertility, and 30% -50% of cases in this population originate from men, sperm analysis is an important means of diagnosing male infertility. The method can reduce subjective errors in manual quality analysis, improve timeliness, easily obtain quantized analysis indexes and can stably detect motion characteristics and morphological characteristics. Sperm motility is an important indicator of the quality of sperm and is characterized by the state of motion of sperm in semen. Therefore, the automatic sperm activity analysis technology mainly obtains the movement state of sperms in images through continuous sperm image sequences so as to derive an activity index. In the continuous image sequence analysis process, firstly, the sperms of each image are required to be automatically identified, then, the sperms identified in the sequence are correlated, so that the sperm movement characteristics are acquired, and further, the sperm movement characteristics are analyzed and the sperm activity is judged. However, the recognition, association and movement characteristic analysis of the sperm directly affect the accuracy of the final sperm cell activity determination. The prior art has the following defects:
The recognition algorithm based on the deep learning model lacks support for the data set.
The morphological principle commonly used in association is to consider the area, eccentricity, direction angle, etc. of the sperm head by weight or singleness, but most of the head shapes are similar for sperm, and when associating, different sperm under the same field of view cannot be well distinguished and additional calculation needs to be introduced, which increases the time complexity.
When dealing with occlusion overlap, the predicted position will typically be the position of the sperm at the current frame, which results in the accumulation of errors affecting the matching of subsequent frames.
Disclosure of Invention
In view of the above-mentioned drawbacks, the present invention aims to provide a trajectory tracking method and apparatus for sperm identification, and aims to propose a prediction method of a non-linear model to reduce the cost of data set fabrication, improve matching accuracy, reduce the influence of occlusion and overlapping on target tracking, and reduce errors caused by missed detection and false detection of a detector, and reduce the dependence of an algorithm on hardware.
According to a first aspect of the present invention, the present invention claims a trajectory tracking method for sperm identification, comprising:
frame segmentation is carried out on the sperm video to be detected to obtain a plurality of sperm detection frames, and a sperm detection frame set is formed;
Training a sperm detection model, and carrying out feature recognition on the sperm detection frames to obtain a feature vector set of each frame;
performing similarity matching on a characteristic vector set of a target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
carrying out chain combining operation and chain supplementing operation on the first motion trail to obtain a second motion trail of the target sperm;
and carrying out chain combining operation on the first motion trail, and further comprising:
selecting a segment track of the track head in the middle of the visual field of the non-first frame from the first motion track to obtain a segment track set；
JudgingIf the track->If the frame is missing, the new track adding set is obtained by breaking the frame missing>；
At the position ofSearching track of missing part frame and track of track termination in the neighborhood of head node of (a) to get set +.>For the trace of the missing frame in the set, the missing part is compared with +.>Whether the head nodes coincide or not, the number of missing frames and +.>Whether the lengths are identical or not, remain in +.>Otherwise, deleting;
then, the tracks in the set are found to be the optimal combined track according to the principle that the speed direction is the most similar and the head-tail distance is the nearest Merging onto the optimal merged track, and marking the obtained new track as a merged track +.>；
JudgingThe tail of the track, if the tail is near the edge or the frame where the tail is located is the last frame of the video, will +.>Track from the collection->Delete in (a) otherwise->Searching whether the track of the head node at the adjacent position of the tail node exists in the next frame of the frame where the tail node is positioned, if so, finding the optimal merging track according to the principle that the speed direction is the most similar and the head-tail distance is the nearest, and adding ∈>Combining the segment track with the optimal combined track, and marking the combined track as a combined track +.>；
Repeating untilDelete +.>Is provided.
Further, before the sperm detection model is trained, the method further comprises:
the sample picture is enhanced by mixup data according to different weights, and then the enhanced data set is enhanced by Mosaic data;
and the mixup data enhancement synthesizes two images into one image according to different weights, reduces the contrast of the images, increases the number of difficult samples, and reduces and segments the sample images into a new image.
Further, the training sperm detection model further comprises:
Constructing a training data set by adopting a data engine, wherein the data engine firstly marks sperm targets of partial frames in a sample video manually and trains an initial model;
and detecting sperms of other frames by the initial model to obtain defective labeling data, manually correcting, taking all data as a training set to train the model again, and repeating the training process until the model is trained to meet the preset generalization condition.
Further, the sperm detection model further comprises:
the method comprises the steps of adopting a fast-RCNN model in a target detection algorithm based on deep learning, wherein the fast-RCNN model comprises an input layer, a background layer, a region candidate layer and an output layer;
the input layer inputs a picture, the background layer extracts the characteristics of the whole picture, the region candidate layer provides a potential region containing a target object for a subsequent layer, and the output layer outputs the position of a detection frame, the size of the detection frame, the confidence level of whether the target object is in a calibration candidate frame and the confidence level of each category of the target object;
during training, using Mosaic data enhancement and mixup data enhancement, wherein the region candidate layer completes a detection frame regression task and is trained by using a GIoU loss function;
And the backup layer and the output layer complete classification tasks and are trained by using a cross entropy loss function.
Further, the matching of the similarity of the feature vector set of the target sperm between two frames in the plurality of sperm detection frames by using the association matching algorithm to obtain a first motion track of the target sperm, further includes:
the association matching algorithm adopts a multi-principle matching method, and comprises an individual characteristic most similar principle and an average speed smoothing principle;
the principle of the most similarity of individual features is that the feature similarity obtained by moving sperms from the i-1 frame to the i frame through RoIPooling in a sperm identification model is the highest;
the average speed smoothing principle refers to that the characteristic similarity between the displacement of the sperm moving from the i-1 frame to the i frame and the average speed and the direction is highest;
and the weights of the individual characteristic most similar principle and the average speed smoothing principle are different, wherein the weight of the individual characteristic most similar principle is larger than that of the average speed smoothing principle.
Further, the method further comprises the following steps:
calculation of sperm of the i-1 st frame+.>Characteristic similarity of sperm->Obtain the corresponding matching degree->Obtaining corresponding matching degree according to a speed smoothing principle >Adding according to preset weight to obtain final matching degree +.>；
Combining the frame skip matching scene when the reservation tracking exists, and calculating all sperms in the i-1 th frame and sperms in the i-th frameObtaining a matching degree matrix;
adopting the association matching algorithm to carry out association matching on sperms in the i-1 th frame and the i-th frame;
and repeating the association matching until all adjacent frames are matched, and obtaining a first motion trail of the target sperm.
Further, calculating the characteristic similarity of the sperm further comprises:
distance ofThe Euclidean distance is adopted for calculation, and the calculation formula is as follows:
；
wherein,sperm +.>Coordinate positions of (2);
the feature similarity uses the dot product operation with the formula:
；
wherein the method comprises the steps ofIs sperm +.>Vector description of the features, compressing the value range to obtain；
The average velocity smoothing principle is to infer sperm of i-1 frameOccurs in the ith frameAt the point of->) Distance from each otherAt the same time consider->Direction of->The following frame is not changed in large angle, and the formula is adopted:
；
wherein the method comprises the steps ofIs super-parameter (herba Cinchi Oleracei)>The speed of sperm l representing frame i-1,/->Represents the included angle between the i-th frame sperm l and the i-1 th frame sperm k, +.>A direction of the velocity of sperm l in the i-1 th frame;
according to the weightAdding to obtain the total matching degree- >. Further, performing a chain supplementing operation on the first motion trail, further includes:
acquiring a track with a missing in the middle of the combined track, and if the ith frame is missing, calculating spermsTo obtain the light flow of the lightFlow direction->And optical flow size->Sperm velocity direction +.>And speed size->；
Obtaining a predictive vector direction as a preset weight sum optical flow and speed+/>The predictive vector size is +.>+/>；
And adding the predicted vector into the combined track as a track, and repeating until the whole track chain is complete, so as to obtain a second motion track of the target sperm.
According to a second aspect of the present invention, the present invention claims a trajectory tracking device for sperm identification, comprising:
the sperm frame segmentation module is used for carrying out frame segmentation on the sperm video to be detected to obtain a plurality of sperm detection frames to form a sperm detection frame set;
the sperm characteristic detection module trains a sperm detection model, performs characteristic recognition on the sperm detection frames and obtains a characteristic vector set of each frame;
the characteristic similarity matching module is used for carrying out similarity matching on a characteristic vector set of the target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
The track chain optimization module performs chain combining operation and chain supplementing operation on the first motion track to obtain a second motion track of the target sperm;
the track tracking device facing the sperm identification is used for executing the track tracking method facing the sperm identification.
According to the scheme, a sperm target recognition model is trained through a data engine, the sperm target in each frame in a picture sequence is automatically detected through the model, meanwhile, the characteristic description vector of each sperm target is stored for association matching, multi-principle matching is used during association matching, the registration rate is improved, the loss rate caused by error of a detector and shielding overlapping is reduced by adopting a method for keeping track of frame jump matching, a track chain is optimized by adopting a chain combining operation, prediction chain compensation is carried out on sperms through optical flow calculation and historical track information, and the robustness of the track chain is improved and the dependence on hardware is reduced.
Drawings
FIG. 1 is a workflow diagram of a sperm identification oriented trajectory tracking method according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a data enhancement method of a tracking method for sperm identification according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of a data engine of a tracking method for sperm identification according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of a sperm target detection model of a sperm identification-oriented trajectory tracking method according to an embodiment of the present invention;
fig. 5 is a schematic diagram of a backhaul layer structure of a sperm identification-oriented trajectory tracking method according to an embodiment of the present invention;
FIG. 6 is a schematic view of a candidate layer structure of a track tracking method for sperm identification according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of an output layer structure of a tracking method for sperm identification according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a first calculation example of a tracking method for sperm identification according to an embodiment of the present invention in response to occlusion;
FIG. 9 is a schematic diagram of a second calculation example of a tracking method for sperm identification according to an embodiment of the present invention in response to occlusion;
FIG. 10 is a schematic flow chart of a correlation matching algorithm of a tracking method for sperm identification according to an embodiment of the present invention
FIG. 11 is a schematic diagram of a chain combining operation example of a trajectory tracking method for sperm identification according to an embodiment of the present invention;
Fig. 12 is a schematic diagram of a chain-supplementing operation example of a trajectory tracking method for sperm identification according to an embodiment of the present invention;
fig. 13 is a block diagram of a track following device for sperm identification according to an embodiment of the present invention.
Detailed Description
Examples of the present invention will be described in detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
According to a first embodiment of the present invention, referring to fig. 1, the present invention claims a track tracking method for sperm identification, including:
frame segmentation is carried out on the sperm video to be detected to obtain a plurality of sperm detection frames, and a sperm detection frame set is formed;
training a sperm detection model, and carrying out feature recognition on a plurality of sperm detection frames to obtain a feature vector set of each frame;
performing similarity matching on a characteristic vector set of a target sperm between two frames in a plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
And carrying out chain combining operation and chain supplementing operation on the first motion trail to obtain a second motion trail of the target sperm.
Further, before training the sperm detection model, the method further comprises:
the sample picture is enhanced by mixup data according to different weights, and then the enhanced data set is enhanced by Mosaic data;
the mixup data enhancement synthesizes two images into one image with different weights, reduces the contrast of the images, increases the number of difficult samples, and the mosaics data enhancement reduces and pieces together the sample images into a new image.
To increase model generalization ability before each training, data enhancement as shown in fig. 2 is performed on the data, and taking the first training as an example, 100 images of the data set are randomly and uniformly divided into two setsThen will be from the collection->Picking up an image +.>And (2) with collection->Random one image->According to->Is synthesized into a graph to be added into the collection +.>And +.>From the collection->Deletion of (C) up to->；
And then fromAnd 1, 1 and 2 images are respectively selected from the original 100 images and scaled to the original 1/4 size, and then are spliced into one image to be added into the set +.>Stopping until the number of the sets reaches 50% of the number of the data sets, namely, 50 is pieced together;
Will beCombining the image data of 200 images into a dataset according to 6: and 4, dividing 120 training sets into 80 test sets. Attention to->Satisfy->Which is randomly generated. The data enhancement can increase the number of data samples from different dimensions, and for a sperm image, the mosaic stage enables the sperm concentration of the image seed to change so as to supplement data with different concentrations in a data set, and the mixup stage enables data with different contrast and definition to be increased, so that the situation of poor video quality caused by improper modulation of acquisition equipment is simulated, and the robustness and generalization of a model are enhanced.
Further, training the sperm detection model further comprises:
constructing a training data set by adopting a data engine, firstly marking sperm targets of partial frames in a sample video by manpower, and training an initial model by the data engine;
and detecting sperms of other frames by the initial model to obtain defective labeling data, manually correcting, taking all data as a training set to train the model again, and repeating the training process until the model is trained to meet the preset generalization condition.
In this embodiment, to train the model, a data engine as shown in fig. 3 is used, and first, 100 pictures are manually labeled as a dataset, according to 6:4, the data set is divided into 60 data sets, 40 data sets are training sets, a preliminary Faster-RCNN model is trained, then the preliminary model is used for reasoning the unlabeled 50 pictures to obtain pictures marked by the model, the pictures are corrected manually and then added into the data set, the training set is 90 picture test sets, 60 pictures are obtained by training the model again, and the steps are repeated until the performance is not improved.
Further, the sperm detection model further comprises:
the method comprises the steps of adopting a fast-RCNN model in a target detection algorithm based on deep learning, wherein the fast-RCNN model comprises an input layer, a background layer, a region candidate layer and an output layer;
the input layer inputs a picture, the background layer extracts the characteristics of the whole picture, the region candidate layer provides a potential region containing a target object for the subsequent layer, and the output layer outputs the position of a detection frame, the size of the detection frame, the confidence degree of whether the target object is in a calibration candidate frame and the confidence degree of each category of the target object;
during training, using Mosaic data enhancement and mixup data enhancement, completing a detection frame regression task by the region candidate layer, and training by using a GIoU loss function;
the backup layer and the output layer complete classification tasks and are trained using cross entropy loss functions.
Wherein in this embodiment, the trained Faster-RCNN model trains out sperm with good recognition and extraction of each sperm characteristic, the model essentially comprises 4 parts as shown in FIG. 4, and the input image Z is passed through the input layer to obtain a fixed size MxN imageThe method comprises the steps of carrying out a first treatment on the surface of the Image +.>Extracting features, and adding herba Vernoniae Cinerariifolii>After 13 convolutions in the module, a characteristic diagram F with a larger receptive field is finally obtained; then through a region candidate layer structure shown in fig. 6, in the structure, the feature map F is subjected to semantic conversion through a 3×3 convolution layer, then the channel conversion is performed through the convolution operation of branches 1×1 on the feature map F, then the classification is performed through softmax to obtain whether the score is the score of a foreground object, meanwhile, the frame regression prediction is performed on the lower branch after the channel conversion by adding predefined anchor frame information to obtain predicted candidate frames, and the predicted candidate frames are combined to obtain n candidate region RoI sets to be output; and then the output layer as shown in fig. 7 is accessed, the RoI is converted from the original image level scale to the same scale as the feature image F through RoIPooling pooling by the feature images F and RoI, n RoI pools to a fixed number m, and then the two prediction heads are accessed after the full connection layer to obtain more accurate prediction frames and class scores of the prediction frames respectively.
Specifically, the trained Faster-RCNN model is trained to better identify sperm and extract characteristics of each sperm, the model mainly comprises 4 parts as shown in figure 4, an input image Z is obtained by downsampling 1920×1080×3 sperm images through an input layer to obtain 640×640×3 images；
Image pairs by a backhaul layer as shown in fig. 5Extracting features, and adding herba Vernoniae Cinerariifolii>After 13 convolutions and 4 pulling layers in the module, a characteristic diagram F with the size of 40 multiplied by 512 is finally obtained, each point in the characteristic diagram is a description of the characteristic of the sperm image obtained by sensing the whole image signal from different scales and different positions in the original image, so that the characteristic of each sperm is not a global characteristic;
then through the area candidate layer structure shown in fig. 6, the global sperm characteristic diagram F is subjected to semantic decoding through a 3×3 convolution layer, then a channel conversion is performed through a convolution operation of which the branches are 1×1 to obtain a matrix of 40×40×18, then data in the matrix is subjected to screening activation through softmax, meanwhile, the characteristic diagram F is subjected to a lower branch channel conversion to obtain a matrix of 40×40×36, and a predefined anchor frame information im_info is added to perform frame regression prediction, classification prediction and screening to obtain P (number of non-fixed) sperm candidate frames;
The candidate frames are input into an output layer module as shown in fig. 7, the candidate frames with different sizes and the feature map F are input into the roitool to obtain 7×7 feature maps with P fixed sizes, and at this time, the feature maps are essentially the features of the images contained in each candidate frame, namely the features of sperms, and then the two prediction heads are connected after passing through the full connection layer to obtain more accurate prediction frames and the class scores of the prediction frames respectively. In this embodiment, 7×7 feature maps obtained by RoI are straightened into feature vectors of 1×49 for calculating individual feature similarity。
Further, performing similarity matching on the feature vector set of the target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion trail of the target sperm, and further comprising:
the association matching algorithm adopts a multi-principle matching method, and comprises an individual characteristic most similar principle and an average speed smoothing principle;
the principle of the most similarity of individual features is that the feature similarity obtained by moving sperms from the i-1 frame to the i frame through RoIPooling in a sperm identification model is the highest;
the average speed smoothing principle refers to that the similarity between the displacement of the sperm from the i-1 frame to the i frame and the average speed and the direction characteristics is highest;
The weights of the individual characteristic most similar principle and the average speed smoothing principle are different, wherein the weight of the individual characteristic most similar principle is larger than that of the average speed smoothing principle.
Further, the method further comprises the following steps:
calculation of sperm of the i-1 st frame+.>Characteristic similarity of sperm->Obtain the corresponding matching degree->Obtaining corresponding matching degree according to a speed smoothing principle>Adding according to preset weight to obtain final matching degree +.>；
Combining the frame skip matching scene when the reservation tracking exists, and calculating all sperms in the i-1 th frame and sperms in the i-th frameObtaining a matching degree matrix;
adopting an association matching algorithm to carry out association matching on the sperms in the i-1 th frame and the i th frame;
and repeating the association matching until all adjacent frames are matched, and obtaining a first motion trail of the target sperm.
Further, calculating the characteristic similarity of the sperm further comprises:
distance ofThe Euclidean distance is adopted for calculation, and the calculation formula is as follows:
；
wherein,sperm +.>Coordinate positions of (2);
the feature similarity uses the dot product operation with the formula:
；
wherein the method comprises the steps ofIs sperm +.>Vector description of the features, compressing the value range to obtain；
The average velocity smoothing principle is to infer sperm of i-1 frame At the ith frame appear at the and point (+)>) Distance from each otherAt the same time consider->Direction of->The following frame is not changed in large angle, and the formula is adopted:
；
wherein the method comprises the steps ofIs super-parameter；
According to the weightAdding to obtain the total matching degree->。
In this embodiment, when the association matching algorithm is used to perform association matching on the sperm in the i-1 th frame and the i-th frame, the method includes:
according to the matching degree matrix, adopting Hungary algorithm to match optimal matching objects for each sperm, and using the spermObject optimally matched with it->Is compared with a threshold value T, and when the matching degree is greater than the threshold value T, the matching degree is +.>Added to->If the track is smaller than the threshold T, the track cannot be matched.
If the situation of no matching occurs:
in case one, for sperm in the i-1 frameThe fact that the sperm of the ith frame is not matched with the field of view can be divided into two cases due to the fact that the situation of the actual sperm is separated from the field of view, so that the sperm cannot be matched, and whether the position of the sperm is at the edge of the field of view is judged, if so, the sperm is not tracked any more; and in the second case, the error detection is omitted, and the position of the error detection is fixed at the position of the i-1 frame to keep track.
In case two, regarding the unmatched sperm which are added in the i frame, the unmatched sperm is set as a new tracking object if the next tracking object is the next one Sperm in the frame that do not find a match are not tracking the object.
The steps are repeated until the matching step is completed for all sperm in the i-1 frame.
Assuming that the video shares F frames, each frame is identified by the Faster-RCNN to obtain a sperm set of each frame
；
Wherein the method comprises the steps ofTracking is to +.>The process of matching the sperm. />It is possible that the number of detected sperm may vary from frame to frame.
Let the reserved trace set be
；
Representation of progressAnd->To date, no sperm sets of the matching object have been found. Wherein c represents the current about to be performed->And->Is matched with->Indicating that the a-th sperm in the x-th frame has not found any matching object so far, and +.>，/>. When doing->And->Will be +.>All sperm and +.>Matching is performed. At this time, as the frame skip matching is carried out, the matching degree is about how many frames the sperm actually moves are considered during the matching>The method comprises the following steps:
；
for example: assume thatNot and->If the sperm of any one of the collection is successfully matched, the sperm is added into the retention tracking collection +.>In->And->Will also be +.>And->Matching is performed. Thus, when similarity is calculated at the time of matching +.>Consideration of->In practice it moves 6 frame intervals so that it . The high degree of match region generated when it matches is shown in fig. 8 as region C in the figure.
In this embodiment, the total matching degree matrix needs to be calculated first, and the distance principle is not explicitly used in the present invention, and is included in the speed smoothing principle, and corresponds to the distance minimum principle when the average speed is small. This can avoid interference of the distance principle with the matching degree when sperm meet in the field of view. And the principle does not form hard decision boundaries, but rather a soft boundary and search area formed by analyzing the speed magnitude and direction. And features of the recognition stage are introduced, which have more enhanced expressive and distinguishing capabilities than conventional geometric features, since they are complex features that are trained in that they take into account not only geometric features but also domain features.
Examples are 10 th and 11 th frames,for and +.>Associated sperm, at this point, match +.>And (3) withTheir positions are +.>The feature vector is +.>Calculate its distance +.>By->Compressing its value to between 0 and 1, usually +.>Taking 0.2, calculating the similarity +.>By compression of->Usually +.>Taking 0.1, calculating the displacement of sperm from 9 th frame to 10 th frame as its speed according to the distance formula Calculate its velocity direction +.>Calculate->And->Direction of->Calculate->Calculate the total matching degreeContinue to repeat the calculation +.>Until the total match between all the tracked sperm and the sperm in the 11 th frame is calculated to obtain a match matrix. When calculating the +.>In the case of the path shown in fig. 9, the occlusion is taken as an example, the target sperm should appear in the region a in the 10 th frame, but no corresponding sperm is found when the target sperm is matched due to the occlusion by the bubble impurity, at this time, the target sperm is fixed at the position of the 9 th frame, tracking is kept and frame skip matching is performed, and when the target sperm is matched with the 11 th frame, the target sperm is predicted to appear in the region B according to the algorithm. At this time calculate +.>And->Between->Is->. The frame-skipping matching method for keeping tracking and fixing the sperm in situ can reduce the problem of error accumulation caused by a prediction algorithm and improve the registration rate.
After the matching degree matrix is calculated, through the associated matching flow shown in fig. 10, firstly, screening out the most relevant sperm in the 11 th frame of the tracked sperm by using the Hungary algorithm, wherein the sperm is the most relevant sperm in the 11 th frame of the tracked spermIs selected from semen->The best matching object willA value smaller than the threshold value T indicates +. >Not->Sperm, which also means +.>There are no matching sperm in frame 11, and the threshold T is typically 0.6. Due to the speed direction->In the first quadrant, which falls with the upper left corner of the image as origin, so the right and lower sides of the image are edges where sperm may swim out of view, calculate +.>And->And is in charge of>Comparison, when less than speed, indicates that at the edge, the sperm is no longer tracked, otherwise it is fixed at the 10 th frame position and tracking is preserved. And repeatedly judging all the objects to be tracked. In the matching algorithm, the Hungary algorithm can take account of the overall situation, two most similar sperms in the whole matching process are found, and the situation that the sperms which are judged first in a non-maximum value inhibition method are more preferred does not occur, so that the situation that the matched objects are more easily obtained by the sperms which are judged first is avoided. And the reduced number of objects to keep track of reduces the computational complexity, considering whether the sperm are near the edge of the field of view.
Further, the chain combining operation is performed on the first motion trail, and the method further comprises the following steps:
selecting a segment track with a track head appearing in the middle of the visual field of the non-first frame from the first motion track to obtainTo a segment track set；
JudgingIf the track->If the frame is missing, the new track adding set is obtained by breaking the frame missing >；
At the position ofSearching track of missing part frame and track of track termination in the neighborhood of head node of (a) to get set +.>For the trace of the missing frame in the set, the missing part is compared with +.>Whether the head nodes coincide or not, the number of missing frames and +.>Whether the lengths are identical or not, remain in +.>Otherwise, deleting;
then, the tracks in the set are found to be the optimal combined track according to the principle that the speed direction is the most similar and the head-tail distance is the nearestMerging onto the optimal merged track, and marking the obtained new track as a merged track +.>；
JudgingThe tail of the track, if the tail is near the edge or the frame where the tail is located is the last frame of the video, will +.>Track from the collection->Delete in (a) otherwise->Searching whether the track of the head node at the adjacent position of the tail node exists in the next frame of the frame where the tail node is positioned, if so, finding the optimal merging track according to the principle that the speed direction is the most similar and the head-tail distance is the nearest, and adding ∈>Combining the segment track with the optimal combined track, and marking the combined track as a combined track +.>；
Repeating untilDelete +.>Is provided.
In this embodiment, when all adjacent frames are matched to obtain a track chain, a chain combining operation is performed, as shown in fig. 11, since the head node of the track chain 2 exists in the middle section of the field of view of the 22 nd frame, a square area with the head node as the center and 2 times of the average speed of the track chain 2 as the side length is searched, and since the 21 frame node of the track 1 is in the area and starts to be missing at 22 frames to coincide with the track 2, the missing length is the same as the track 2 length, and the missing length is added into the set. Calculate the distance of the head node of track 2 from the 21 frame node of track 1, and the average speed of the track, while according to 1 of track 1 The 9 frame node and the 20 frame node and the 22 frame node and the 23 frame node of the track 2 respectively calculate directions, calculate the similarity, and the similarity is larger than the threshold valueCombining results in a new track 1 as in fig. 11. And since the tail node of the new track 1 is at the image edge, the new track 1 can be deleted from the segment track set. Especially when the segment track is also a missing frame track, for example, the combination of track 4 and track 3 in fig. 11 needs to break track 4 to obtain new 3 segment tracks, and the same steps are adopted for combining each segment track, wherein the average speed and direction of the segment track are calculated by adopting an optical flow calculation method. This operation is essentially a repair operation because the resulting trajectory when the matching is completed is likely to result in a frame skip of the trajectory due to the threshold, and the detector erroneously detects some impurities as sperm resulting in many trajectories with only one point. The chain combining operation optimizes all track chains, merges the segment tracks into the missing frame track and deletes some unnecessary segment tracks, thereby enhancing the quality of the tracks and reducing the error introduced by the detector.
Further, the chain supplementing operation is performed on the first motion trail, and the method further comprises the following steps:
Acquiring a track with a missing in the middle of the combined track, and if the ith frame is missing, calculating spermsTo obtain the optical flow direction +.>And optical flow size->Sperm velocity direction +.>And speed size->；
Obtaining a predictive vector direction as a preset weight sum optical flow and speed+/>The predictive vector size is +.>+/>；
And adding the predicted vector into the combined track as a track, and repeating until the whole track chain is complete, so as to obtain a second motion track of the target sperm.
In this embodiment, after the chain combining operation is optimized for all the tracks, there may still exist more track chains with frame missing mainly due to overlapping and shielding of sperm, and the tracks need to be completed in a predictive manner. The supplemental chain operation of fig. 12, where the traces are missing portions of 22 and 23 frames. Supplementing the track of 22 frames, wherein the nodes in the track of 20 and 21 frames respectively correspond to sperms detected by the frames,/>By->Calculating to obtain optical flow direction->And optical flow size->And calculates the speed direction +.>And speed size->. The weighted sum yields a prediction result of +.>+/>The predictive vector size is +.>+/>Usually +.>. And obtaining the track of 22 frames according to the obtained prediction vector. Note that when there is no way to solve for optical flow, the velocity vector is used as the prediction at this time +. >When the velocity vector is not obtained, the optical flow is taken as the prediction at the moment +.>. The velocity vector is used as a prediction vector when the node in the 23 rd frame is obtained. The complete trace is obtained through the chain-supplementing operation, as in the lower half of fig. 12. During the entire chain of the chain, the position of the sperm is predicted by the speed of the previous frame and the optical flow of the current frame without using a linear motion model, because although the motion of the sperm is nearly linear between frames when photographed by a high frame rate camera, there is a fast forward motion sperm in one sperm sample with high motion, requiring higher frame rates for these sperm, and possibly increasing frame rate requirements due to the presence of magnification, which adds to the cost of the device. It is therefore necessary to consider video at low frame rate devices, where the straight line model is no longer applicable.
According to the obtained sperm track chain, the method can be used for sperm analysis, firstly, the sperm obtained through a sperm identification model can obtain the average number of the sperm in the visual field in video time to assist a detector to count, and secondly, the method can calculate the sperm movement parameters according to the track chain, and can judge the sperm movement category and the sperm movement speed. Even the detected sperms can be segmented out and then morphological discrimination can be carried out on the sperms, so that artificial subjective errors are reduced.
According to a second embodiment of the present invention, referring to fig. 13, the present invention claims a trajectory tracking device for sperm identification, comprising:
the sperm frame segmentation module is used for carrying out frame segmentation on the sperm video to be detected to obtain a plurality of sperm detection frames to form a sperm detection frame set;
the sperm characteristic detection module trains a sperm detection model, performs characteristic recognition on a plurality of sperm detection frames, and obtains a characteristic vector set of each frame;
the characteristic similarity matching module is used for carrying out similarity matching on a characteristic vector set of the target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
the track chain optimization module performs chain combining operation and chain supplementing operation on the first movement track to obtain a second movement track of the target sperm;
a sperm identification-oriented track tracking device is used for executing a sperm identification-oriented track tracking method.
Those skilled in the art will appreciate that various modifications and improvements can be made to the disclosure. For example, the various devices or components described above may be implemented in hardware, or may be implemented in software, firmware, or a combination of some or all of the three.
A flowchart is used in this disclosure to describe the steps of a method according to an embodiment of the present disclosure. It should be understood that the steps that follow or before do not have to be performed in exact order. Rather, the various steps may be processed in reverse order or simultaneously. Also, other operations may be added to these processes.
Those of ordinary skill in the art will appreciate that all or a portion of the steps of the methods described above may be implemented by a computer program to instruct related hardware, and the program may be stored in a computer readable storage medium, such as a read only memory, a magnetic disk, or an optical disk. Alternatively, all or part of the steps of the above embodiments may be implemented using one or more integrated circuits. Accordingly, each module/unit in the above embodiment may be implemented in the form of hardware, or may be implemented in the form of a software functional module. The present disclosure is not limited to any specific form of combination of hardware and software.
Unless defined otherwise, all terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
The foregoing is illustrative of the present disclosure and is not to be construed as limiting thereof. Although a few exemplary embodiments of this disclosure have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this disclosure. Accordingly, all such modifications are intended to be included within the scope of this disclosure as defined in the claims. It is to be understood that the foregoing is illustrative of the present disclosure and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The disclosure is defined by the claims and their equivalents.
In the description of the present specification, reference to the terms "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
While embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that: many changes, modifications, substitutions and variations may be made to the embodiments without departing from the spirit and principles of the invention, the scope of which is defined by the claims and their equivalents.
Claims (9)
1. A sperm identification-oriented trajectory tracking method, comprising:
frame segmentation is carried out on the sperm video to be detected to obtain a plurality of sperm detection frames, and a sperm detection frame set is formed;
training a sperm detection model, and carrying out feature recognition on the sperm detection frames to obtain a feature vector set of each frame;
performing similarity matching on a characteristic vector set of a target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
carrying out chain combining operation and chain supplementing operation on the first motion trail to obtain a second motion trail of the target sperm;
and carrying out chain combining operation on the first motion trail, and further comprising:
selecting a segment track of the track head in the middle of the visual field of the non-first frame from the first motion track to obtain a segment track set ；
JudgingIf the track->If the frame is missing, the new track adding set is obtained by breaking the frame missing>；
At the position ofSearching track of missing part frame and track of track termination in the neighborhood of head node of (a) to get set +.>For the trace of the missing frame in the set, the missing part is compared with +.>Whether the head nodes coincide or not, the number of missing frames and +.>Whether the lengths are identical or not, remain in +.>Otherwise, deleting;
then, the tracks in the set are found to be the optimal combined track according to the principle that the speed direction is the most similar and the head-tail distance is the nearestMerging onto the optimal merged track, and marking the obtained new track as a merged track +.>；
JudgingThe tail of the track, if the tail is near the edge or the frame where the tail is located is the last frame of the video, will +.>Track from the collection->Delete in (a) otherwise->Searching whether the track of the head node at the adjacent position of the tail node exists in the next frame of the frame where the tail node is positioned, if so, finding the optimal merging track according to the principle that the speed direction is the most similar and the head-tail distance is the nearest, and adding ∈>Combining the segment track with the optimal combined track, and marking the combined track as a combined track +.>；
Repeating untilDelete +. >Is provided.
2. The sperm-oriented trajectory tracking method of claim 1, further comprising, prior to said training of the sperm detection model:
the sample picture is enhanced by mixup data according to different weights, and then the enhanced data set is enhanced by Mosaic data;
and the mixup data enhancement synthesizes two images into one image according to different weights, reduces the contrast of the images, increases the number of difficult samples, and reduces and segments the sample images into a new image.
3. The sperm-identification-oriented trajectory tracking method of claim 2, wherein said training sperm detection model further comprises:
constructing a training data set by adopting a data engine, wherein the data engine firstly marks sperm targets of partial frames in a sample video manually and trains an initial model;
and detecting sperms of other frames by the initial model to obtain defective labeling data, manually correcting, taking all data as a training set to train the model again, and repeating the training process until the model is trained to meet the preset generalization condition.
4. The sperm-identification-oriented trajectory tracking method of claim 2, wherein said sperm detection model further comprises:
The method comprises the steps of adopting a fast-RCNN model in a target detection algorithm based on deep learning, wherein the fast-RCNN model comprises an input layer, a background layer, a region candidate layer and an output layer;
the input layer inputs a picture, the background layer extracts the characteristics of the whole picture, the region candidate layer provides a potential region containing a target object for a subsequent layer, and the output layer outputs the position of a detection frame, the size of the detection frame, the confidence level of whether the target object is in a calibration candidate frame and the confidence level of each category of the target object;
during training, using Mosaic data enhancement and mixup data enhancement, wherein the region candidate layer completes a detection frame regression task and is trained by using a GIoU loss function;
and the backup layer and the output layer complete classification tasks and are trained by using a cross entropy loss function.
5. The sperm identification-oriented trajectory tracking method of claim 1, wherein the matching of the similarity of the feature vector set of the target sperm between two frames of the plurality of sperm detection frames using the correlation matching algorithm, to obtain a first motion trajectory of the target sperm, further comprises:
the association matching algorithm adopts a multi-principle matching method, and comprises an individual characteristic most similar principle and an average speed smoothing principle;
The principle of the most similarity of individual features is that the feature similarity obtained by moving sperms from the i-1 frame to the i frame through RoIPooling in a sperm identification model is the highest;
the average speed smoothing principle refers to that the average speed and the directional characteristic similarity of the displacement of the sperm from the i-1 frame to the i frame are highest;
and the weights of the individual characteristic most similar principle and the average speed smoothing principle are different, wherein the weight of the individual characteristic most similar principle is larger than that of the average speed smoothing principle.
6. The sperm-oriented trajectory tracking method of claim 5, further comprising:
calculation of sperm of the i-1 st frame+.>Characteristic similarity of sperm->Obtain the corresponding matching degree->Obtaining corresponding matching degree according to a speed smoothing principle>Adding according to preset weight to obtain final matching degree +.>；
Combining the frame skip matching scene when the reservation tracking exists, and calculating all sperms in the i-1 th frame and sperms in the i-th frameObtaining a matching degree matrix;
adopting the association matching algorithm to carry out association matching on sperms in the i-1 th frame and the i-th frame;
and repeating the association matching until all adjacent frames are matched, and obtaining a first motion trail of the target sperm.
7. The sperm-oriented trajectory tracking method of claim 6, wherein calculating characteristic similarity of sperm further comprises:
distance ofThe Euclidean distance is adopted for calculation, and the calculation formula is as follows:
；
wherein,sperm +.>Coordinate positions of (2);
the feature similarity uses the dot product operation with the formula:
；
wherein the method comprises the steps ofIs sperm +.>Vector description of the feature, compressing the value range to obtain +.>；
The average velocity smoothing principle is to infer sperm of i-1 frameAt the ith frame appear at the and point (+)>) Distance of->At the same time consider->Direction of->The following frame is not changed in large angle, and the formula is adopted:
；
wherein the method comprises the steps ofIs super-parameter (herba Cinchi Oleracei)>The speed of sperm l representing frame i-1,/->Represents the included angle between the i-th frame sperm l and the i-1 th frame sperm k, +.>A direction of the velocity of sperm l in the i-1 th frame;
according to the weightAdding to obtain the total matching degree->。
8. The sperm-identification-oriented trajectory tracking method of claim 7, wherein the first motion trajectory is subjected to a chain-filling operation, further comprising
Acquiring a track with a missing in the middle of the combined track, and if the ith frame is missing, calculating spermsTo obtain the optical flow direction +. >And optical flow size->Sperm velocity direction +.>And speed size->；
Obtaining a predictive vector direction as a preset weight sum optical flow and speed+/>The predictive vector size is +.>+/>；
And adding the predicted vector into the combined track as a track, and repeating until the whole track chain is complete, so as to obtain a second motion track of the target sperm.
9. A sperm identification-oriented trajectory tracking device, comprising:
the sperm frame segmentation module is used for carrying out frame segmentation on the sperm video to be detected to obtain a plurality of sperm detection frames to form a sperm detection frame set;
the sperm characteristic detection module trains a sperm detection model, performs characteristic recognition on the sperm detection frames and obtains a characteristic vector set of each frame;
the characteristic similarity matching module is used for carrying out similarity matching on a characteristic vector set of the target sperm between two frames in the plurality of sperm detection frames by adopting a correlation matching algorithm to obtain a first motion track of the target sperm;
the track chain optimization module performs chain combining operation and chain supplementing operation on the first motion track to obtain a second motion track of the target sperm;
the sperm identification-oriented trajectory tracking device is configured to perform a sperm identification-oriented trajectory tracking method according to any one of claims 1-8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311790237.XA CN117455958B (en) | 2023-12-22 | 2023-12-22 | Track tracking method and device for sperm identification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311790237.XA CN117455958B (en) | 2023-12-22 | 2023-12-22 | Track tracking method and device for sperm identification |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117455958A CN117455958A (en) | 2024-01-26 |
| CN117455958B true CN117455958B (en) | 2024-03-08 |
Family
ID=89585953
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311790237.XA Active CN117455958B (en) | 2023-12-22 | 2023-12-22 | Track tracking method and device for sperm identification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117455958B (en) |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008250999A (en) * | 2007-03-08 | 2008-10-16 | Omron Corp | Object tracing method, object tracing device and object tracing program |
| CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
| CN104237121A (en) * | 2014-09-18 | 2014-12-24 | 山东大学 | Sperm motility performance detection method |
| CN106574975A (en) * | 2014-04-25 | 2017-04-19 | 三星电子株式会社 | Trajectory matching using peripheral signal |
| KR20180077865A (en) * | 2016-12-29 | 2018-07-09 | 단국대학교 산학협력단 | Online apparatus and method for Multiple Camera Multiple Target Tracking Based on Multiple Hypothesis Tracking |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112580476A (en) * | 2020-12-14 | 2021-03-30 | 深圳创怀医疗科技有限公司 | Sperm identification and multi-target track tracking method |
| CN113591527A (en) * | 2021-01-14 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Object track identification method and device, electronic equipment and storage medium |
| CN113724294A (en) * | 2021-08-31 | 2021-11-30 | 天津大学 | Sperm motility characteristic extraction method and detection device |
| CN114882065A (en) * | 2022-07-12 | 2022-08-09 | 深圳市瑞图生物技术有限公司 | Method and device for judging fluidity of detection object, analyzer and storage medium |
| CN115184243A (en) * | 2022-08-09 | 2022-10-14 | 广州健芯生物科技合伙企业(有限合伙) | Automatic semen detection method and system |
| CN115240130A (en) * | 2022-06-15 | 2022-10-25 | 首都师范大学 | Pedestrian multi-target tracking method and device and computer readable storage medium |
| CN116523972A (en) * | 2023-05-09 | 2023-08-01 | 西安交通大学 | Two-stage multi-target tracking method and product based on sparse optical flow motion compensation |
| CN116883457A (en) * | 2023-08-09 | 2023-10-13 | 北京航空航天大学 | Light multi-target tracking method based on detection tracking joint network and mixed density network |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014137750A (en) * | 2013-01-17 | 2014-07-28 | Hitachi Solutions Ltd | Information processing device, information processing system, and information processing program |
| US9784961B2 (en) * | 2013-03-08 | 2017-10-10 | Church & Dwight Co., Inc. | Sperm motility test device and method |
| US20170109879A1 (en) * | 2014-04-03 | 2017-04-20 | Drexel University | Computer-implemented methods, computer-readable media, and systems for tracking a plurality of spermatozoa |
-
2023
- 2023-12-22 CN CN202311790237.XA patent/CN117455958B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008250999A (en) * | 2007-03-08 | 2008-10-16 | Omron Corp | Object tracing method, object tracing device and object tracing program |
| CN106574975A (en) * | 2014-04-25 | 2017-04-19 | 三星电子株式会社 | Trajectory matching using peripheral signal |
| CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
| CN104237121A (en) * | 2014-09-18 | 2014-12-24 | 山东大学 | Sperm motility performance detection method |
| KR20180077865A (en) * | 2016-12-29 | 2018-07-09 | 단국대학교 산학협력단 | Online apparatus and method for Multiple Camera Multiple Target Tracking Based on Multiple Hypothesis Tracking |
| CN111914664A (en) * | 2020-07-06 | 2020-11-10 | 同济大学 | Vehicle multi-target detection and track tracking method based on re-identification |
| CN112580476A (en) * | 2020-12-14 | 2021-03-30 | 深圳创怀医疗科技有限公司 | Sperm identification and multi-target track tracking method |
| CN113591527A (en) * | 2021-01-14 | 2021-11-02 | 腾讯科技（深圳）有限公司 | Object track identification method and device, electronic equipment and storage medium |
| CN113724294A (en) * | 2021-08-31 | 2021-11-30 | 天津大学 | Sperm motility characteristic extraction method and detection device |
| CN115240130A (en) * | 2022-06-15 | 2022-10-25 | 首都师范大学 | Pedestrian multi-target tracking method and device and computer readable storage medium |
| CN114882065A (en) * | 2022-07-12 | 2022-08-09 | 深圳市瑞图生物技术有限公司 | Method and device for judging fluidity of detection object, analyzer and storage medium |
| CN115184243A (en) * | 2022-08-09 | 2022-10-14 | 广州健芯生物科技合伙企业(有限合伙) | Automatic semen detection method and system |
| CN116523972A (en) * | 2023-05-09 | 2023-08-01 | 西安交通大学 | Two-stage multi-target tracking method and product based on sparse optical flow motion compensation |
| CN116883457A (en) * | 2023-08-09 | 2023-10-13 | 北京航空航天大学 | Light multi-target tracking method based on detection tracking joint network and mixed density network |
Non-Patent Citations (3)
| Title |
|---|
| Calibrating trajectory data for similarity-based analysis;Su H, et al;Proceedings of the 2013 ACM SIGMOD international conference on management of data;20130627;第833-844页 * |
| Distributed trajectory similarity search;Dong Xie， et al;Proceedings of the VLDB Endowment;20171231;第1478-1489页 * |
| 基于位置约束的时空轨迹ID修复问题研究;崔星灿;山东大学;20181115;第1-114页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117455958A (en) | 2024-01-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108810620B (en) | Method, device, equipment and storage medium for identifying key time points in video | |
| CN108171196B (en) | Face detection method and device | |
| Zhou et al. | Target-aware object discovery and association for unsupervised video multi-object segmentation | |
| Liu et al. | SRAF-Net: A scene-relevant anchor-free object detection network in remote sensing images | |
| CN111582126B (en) | Pedestrian re-recognition method based on multi-scale pedestrian contour segmentation fusion | |
| CN110533661A (en) | Adaptive real-time closed-loop detection method based on characteristics of image cascade | |
| CN111931571B (en) | Video character target tracking method based on online enhanced detection and electronic equipment | |
| Han et al. | Class-aware feature aggregation network for video object detection | |
| Yeh et al. | Online selection of tracking features using AdaBoost | |
| CN111401308A (en) | Fish behavior video identification method based on optical flow effect | |
| CN113269038B (en) | Multi-scale-based pedestrian detection method | |
| CN113989604A (en) | Tire DOT information identification method based on end-to-end deep learning | |
| CN112613565B (en) | Anti-occlusion tracking method based on multi-feature fusion and adaptive learning rate updating | |
| CN109583361A (en) | The scene video text tracking method minimized based on energy | |
| CN117455958B (en) | Track tracking method and device for sperm identification | |
| Zhao et al. | Cooperative use of recurrent neural network and siamese region proposal network for robust visual tracking | |
| Lokkondra et al. | DEFUSE: deep fused end-to-end video text detection and recognition | |
| CN112380970B (en) | Video target detection method based on local area search | |
| CN116385915A (en) | Water surface floater target detection and tracking method based on space-time information fusion | |
| CN111401286B (en) | Pedestrian retrieval method based on component weight generation network | |
| CN115359091A (en) | Armor plate detection tracking method for mobile robot | |
| CN111583341B (en) | Cloud deck camera shift detection method | |
| Baabou et al. | A comparative study and state-of-the-art evaluation for pedestrian detection | |
| JP4133246B2 (en) | Image deformation information generation apparatus, image deformation information generation method, and image deformation information generation program | |
| CN113408356A (en) | Pedestrian re-identification method, device and equipment based on deep learning and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |