CN113470073A - Animal center tracking method based on deep learning - Google Patents
Animal center tracking method based on deep learning Download PDFInfo
- Publication number
- CN113470073A CN113470073A CN202110765039.2A CN202110765039A CN113470073A CN 113470073 A CN113470073 A CN 113470073A CN 202110765039 A CN202110765039 A CN 202110765039A CN 113470073 A CN113470073 A CN 113470073A
- Authority
- CN
- China
- Prior art keywords
- animal
- tracking
- video
- training
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000013135 deep learning Methods 0.000 title claims abstract description 12
- 241001465754 Metazoa Species 0.000 claims abstract description 68
- 238000012549 training Methods 0.000 claims abstract description 39
- 238000001514 detection method Methods 0.000 claims abstract description 23
- 238000013528 artificial neural network Methods 0.000 claims abstract description 18
- 230000033001 locomotion Effects 0.000 claims abstract description 14
- 238000012360 testing method Methods 0.000 claims description 17
- 230000000694 effects Effects 0.000 claims description 10
- 230000009467 reduction Effects 0.000 claims description 10
- 238000010586 diagram Methods 0.000 claims description 8
- 238000002790 cross-validation Methods 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 5
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000012804 iterative process Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims 1
- 238000010171 animal model Methods 0.000 abstract description 9
- 238000000605 extraction Methods 0.000 abstract description 5
- 238000001727 in vivo Methods 0.000 abstract description 4
- 241000699666 Mus <mouse, genus> Species 0.000 description 17
- 241000699670 Mus sp. Species 0.000 description 13
- 238000002474 experimental method Methods 0.000 description 12
- 230000008569 process Effects 0.000 description 3
- 208000019901 Anxiety disease Diseases 0.000 description 2
- OYPRJOBELJOOCE-UHFFFAOYSA-N Calcium Chemical compound [Ca] OYPRJOBELJOOCE-UHFFFAOYSA-N 0.000 description 2
- 241000282693 Cercopithecidae Species 0.000 description 2
- 101000577696 Homo sapiens Proline-rich transmembrane protein 2 Proteins 0.000 description 2
- 102100028840 Proline-rich transmembrane protein 2 Human genes 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000036506 anxiety Effects 0.000 description 2
- 229910052791 calcium Inorganic materials 0.000 description 2
- 239000011575 calcium Substances 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000003137 locomotive effect Effects 0.000 description 2
- 241001515942 marmosets Species 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000011740 C57BL/6 mouse Methods 0.000 description 1
- 241000283984 Rodentia Species 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000003542 behavioural effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000010835 comparative analysis Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 239000002689 soil Substances 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
- 238000011816 wild-type C57Bl6 mouse Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30241—Trajectory
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Multimedia (AREA)
- Geometry (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an animal center tracking method based on deep learning, which comprises model training and animal tracking, wherein during the model training, a pre-trained neural network is adopted for carrying out the characteristic extraction of a picture, a target detection layer of a YOLO algorithm is adopted for recognizing a boundary frame of a target animal, and a trained model is finally obtained; when the animal is tracked, all the frame pictures in the video are tracked by using the trained model, the boundary frame of the animal is calculated, and finally the trajectory graph of the animal motion is obtained. The animal center tracking method based on deep learning provides a new experimental animal behavioristics tracking and positioning method, the method can accurately position the animal center point in various experimental paradigms and complex experimental environments, overcomes the interference of in-vivo recording wires and the entrance of experimenters into scenes, and can be suitable for different animal models.
Description
Technical Field
The invention relates to the field of image processing methods, in particular to an animal center tracking method based on deep learning.
Background
In behavioral research in the fields of biology, medicine, neuroscience and the like, the method has an important role in accurately obtaining the position of an animal in an experimental scene through video data. With the development of in vivo recording technology and the diversification of experimental paradigms, researchers often encounter the problem that experimental animals are difficult to accurately position.
Conventional methods (e.g., Limelight, ANY-maze, Lima, and Lima, respectively,
XT, TopScan, etc.) extracts the outline of an experimental animal by using a background noise reduction method, and then determines the position of the animal by calculating the center of the outline, although the method has high calculation speed, the method is only suitable for simple scenes with clean background, high image signal-to-noise ratio and small interference, and is easily influenced by the wires of in-vivo recording equipment and the necessary operation of experimenters entering the scenes in the experimental process, and cannot adapt to the diversification of experimental paradigms.
The development of computer vision technology in recent years provides a large number of image data processing algorithms and provides a new solution for tracking experimental animals. However, most of these algorithms are currently based on the evaluation of animal pose by characteristic point detection, such as DeepLabCut (1. Mathis, A., et al., DeepLabCut: market testing of user-defined body parts with depth testing. Nat. Neurosci, 2018.21 (9): p.1281-1289.[2] Nath, T., et al., use DeepLabCut for 3D market testing of space protocols and devices, 2019.14 (7): p.2152-2176., LEAP.1. Wang, Z.R., et al., leap.1. basic vision testing of animal pose, repair, soil, repair, 2019.8.), Trex (Walter, T.and I.D.Couzin, TRex, a fast multi-animal tracking system with Markesses identification, and 2D evaluation of location and visual fields, Elife, 2021.10.), etc. Although the method can track a plurality of characteristic points of the animal, the behavior analysis of the animal is richer. But because the exact animal center point cannot be objectively evaluated when creating training data, it takes longer and less effective to track the animal center position.
Disclosure of Invention
In order to overcome the defect that the existing method is difficult to accurately track the experimental animal in a diversified experimental paradigm, the invention provides an animal center tracking method based on deep learning based on a deep learning algorithm.
An animal center tracking method based on deep learning comprises the following steps:
(1) model training
(1.1) randomly extracting pictures from a scene video to be tested, then manually creating a data set of a bounding box for extracting the pictures for model training,
(1.2) dividing the data set into a training set, a cross validation set and a test set, wherein the data of the training set is used for training the parameters of the model, the cross validation set and the test set are used for validating and testing the effect of the model,
during training, the pre-trained neural network is adopted to extract the characteristics of the picture, the target detection layer of the YOLO algorithm is adopted to identify the boundary frame of the target animal, and finally the trained model is obtained,
(2) animal tracking
(2.1) manually defining the tracked area before each tracking, and setting the part of each frame image in the video outside the area as a color similar to the background,
(2.2) tracking all the frame pictures in the video by using the trained model, calculating the boundary frame of the animal,
(2.3) converting the format of each frame in the video into a gray scale format, calculating the average value of all the frames of the video as the background noise of the video,
(2.4) subtracting the background noise corresponding to the bounding box area from each frame of picture in the video by using a background noise reduction method to obtain the outline of the animal in the bounding box,
and (2.5) finally, tracking the animal by calculating the centroid of the animal outline in all the frames of the video to obtain a trajectory diagram of the animal movement.
Preferably, in step (1.1), the picture is in RGB format.
Preferably, in step (1.2), the pre-trained neural network is resnet18, mobilenetv2 or resnet 50.
Preferably, in step (1.2), the target detection layer of the YOLO-specific recognition bounding box is connected after the pre-trained neural network.
Preferably, in step (1.2), a mini-batch gradient descent method is adopted during training, and parameters in the network are adjusted through back propagation in an iterative process.
Preferably, in step (2.2), if the YOLO calculates a plurality of possible bounding boxes, the bounding box with the largest predicted p-value is selected for subsequent processing.
Preferably, in step (2.2), the predicted bounding box is enlarged in a ratio of 1: 1.5.
Preferably, when the animal is tracked, the animal tracking of a single video file is realized by dbt _ singleTracking; animal tracking of multiple video files is achieved by dbt _ batch tracking; the tracking completed video is created by dbt _ createlabedvideo.
Preferably, the animal center tracking method corrects the failed tracking frame manually through dbt _ manual tracking, or derives the failed tracking frame through dbt _ optimum, and combines the failed tracking frame with the data set in the step (1.1), and retrains a new model through the step (1.2), so as to serve as a final trained model for the animal tracking in the step (2).
The animal center tracking method based on deep learning provides a new experimental animal behavioristics tracking and positioning method, the method can accurately position the animal center point in various experimental paradigms and complex experimental environments, overcomes the interference of in-vivo recording wires and the entrance of experimenters into scenes, and can be suitable for different animal models.
Drawings
Fig. 1 is a technical flowchart of the present invention, wherein a: model training and optimizing processes; b: animal tracking process; c: schematic diagram of the procedure for tracking black mice in open field experiment.
Fig. 2 is a graph comparing the performance of three pre-trained neural networks, wherein a: comparing training time of the YOLO models of the three neural networks under different picture sizes; b: comparing the detection speed and the detection accuracy of the YOLO model of the three neural networks under different picture sizes, wherein the dots represent the picture sizes of 224, 320, 416 and 512 from small to large in sequence.
FIG. 3 is a diagram showing the comparison result between DeepBhvTracking and three other methods, wherein a-d are a schematic tracking and positioning diagram and an animal track diagram of an L-shaped labyrinth by a background noise reduction method, a YOLO method, a DeepLabcut method and a DeepBhvTracking method respectively; e-h are respectively the animal track schematic diagrams of a background noise reduction method, a YOLO method, a DeepLabCut method and a DeepBhvTracking method in a three-box maze; i: comparing the training time of the DeepBhvTracking model and the DeepLabCut model; j: comparing detection time of DeepBhvTracking with three other methods; k: comparing the pixel variation of each frame of four tracking methods under different experimental paradigms; l: and comparing the error of the four tracking methods in different experimental paradigms with the error of the real value. P < 0.05, p < 0.01, p < 0.001, and significant results are the Bonferroni corrected rank sum test.
Fig. 4 is a graph of the effect of deep bhvtracking in various experimental paradigms of various animals, wherein a: black mouse treadmill experiment; b: the black mouse inverted V-shaped maze; c: black mouse elevated plus maze; d: three-box experiment of white mice; e: marmoset monkey cage.
Fig. 5 is a graph of an application example result of deep bhvtracking in the medical field, wherein a: schematic trace of wild type C57BL/6 under open field; b: indicating the residence time of the mouse at each position of the open field; c: indicating the movement speed of the mouse at each position of the open field; d: average speed comparison of three mice; e: comparing the residence time of the three mice in the center and corner areas of the open field; f: the average speed of movement of the three mice in the center and corner areas of the open field was compared. P < 0.05, p < 0.01, p < 0.001, and significant results are the Bonferroni corrected rank sum test.
Detailed Description
The method is a tool kit developed based on MATLAB software, and is named as DeepBhvTracking, the detailed code list of the DeepBhvTracking is shown in Table 1, and when the method is used, a deep learning tool kit, a computer vision tool kit and a pre-trained neural network (such as a net18, a mobilenetv2, a net50 or other pre-trained neural networks) need to be installed in advance. The testing part related to the application is completed on dell computer (CPU Intel (R) core (TM) i7-10700 CPU @2.90GHz, RAM 64GB, GPU Inter (R) UHD Graphics 6308 GB).
TABLE 1 DeepBhvTracking detailed code List
Example 1
The method mainly comprises the steps of carrying out transfer learning through a pre-trained neural network and calculating a boundary box of an animal through a You Only Look One (YOLO) algorithm, then extracting the outline of the animal in the boundary box by using a background noise reduction method, and finally calculating the central locus of the animal according to the outline of the animal. The method is divided into two parts of model training (figure 1a) and animal tracking (figures 1b and c), and comprises the following steps:
model training
Firstly, randomly extracting pictures (RGB format) from a scene video to be detected as a data set, and then manually creating a bounding box of the data set pictures through an Image Label. The acquisition of the data set is realized by dbt _ dataset.
The data set is divided into a training set (70%), a cross validation set (10%) and a test set (20%) according to the number of pictures, data of the training set is used for training parameters of a YOLO model, and the cross validation set and the test set are used for verifying and testing the effect of the model. In order to make the generalization capability of the model stronger, data enhancement (including flipping, rotating and adding noise) is carried out on the data of the training set. The pre-trained neural network is adopted to extract the features of the pictures of the training set, and the parameters on the network train a large number of images in the ImageNet large database, so that the capability of extracting rich information features from the images is achieved. And connecting a target detection layer of a YOLO specific recognition bounding box after the pre-trained neural network. During training, a mini-batch gradient descent method is adopted, and parameters in the network are adjusted through back propagation in the iterative process. Finally, the trained YOLO model is saved for later animal tracking. The training and verification of the model is realized by dbt _ training.
(II) animal tracking
To avoid the effect of factors outside the experimental field on tracking animals, a tracking area is manually defined before each tracking, and the part outside the area of each frame of image in the video is set to be similar color of the background. Then, tracking all the frame pictures in the video by using the trained YOLO model, and calculating the boundary frame of the animal. Since YOLO accounts work out a number of possible bounding boxes, we select the box with the largest predicted p-value for later processing. To avoid that the predicted bounding box only covers the influence of the animal part body on the tracking effect, we enlarge the bounding box in a ratio of 1: 1.5. Then the video is converted from a color format to a gray scale format, and then the average value of each pixel in the video in time is calculated to be used as the background noise of the video. At the moment, a background noise reduction method is used, and the corresponding background noise is subtracted from each frame of the boundary frame area picture in the video, so that the outline of the animal in the boundary frame can be obtained. And finally, tracking the animal by calculating the centroid of the contour of the animal in the boundary box in all frames of the video to obtain a trajectory diagram of the animal movement. And then analyzing the motion related parameters of the animal, such as the motion speed, the acceleration, the motion time and the like through the trajectory graph. Animal tracking of a single video file can be realized through dbt _ singleTracking, and batch tracking of animals in a plurality of video files can also be realized through dbt _ batchTracking. Finally, the tracking effect can also be detected by dbt _ createlabedvideo creating a tracking completed video.
To avoid the impact of the tracking failed frames (including frames where YOLO failed to detect animals and detected errors) on the experimental analysis, dbt _ manual tracking was provided to correct the tracking failed frames manually. At the same time, dbt _ optimum is provided to derive images of failed tracking, which are merged with the previous training data set and the new model is trained to optimize the model previously trained.
(III) Pre-trained neural network comparison
Fig. 2 shows a comparison of the performance of 3 common pre-trained neural networks (resnet18, mobilenetv2, and resnet50) at different picture sizes. In the resnet18 network, the part before the 'res 5b _ branch2a _ relu' layer is selected as a feature extraction layer, and the layer after the feature extraction layer is replaced by a YOLO target detection layer; in the mobilenetv2 network, selecting a part before a 'block _16_ expanded _ relu' layer as a feature extraction layer, and replacing the layer after the layer with a YOLO target detection layer; in the 'rescenet 50' network, the part before the 'activation _40_ relu' layer is selected as a feature extraction layer, and the layer after the layer is replaced with a YOLO target detection layer.
The results show that the training time and detection accuracy of the YOLO model increase with increasing picture size, but the detection speed decreases with increasing picture size (fig. 2a, b). Under the same picture size, the training time of the resnet18 network is shortest, the detection speed is fastest, and the detection accuracy is minimum; the training time of the resnet50 network is longest, the detection speed is slowest, but the detection accuracy is highest (fig. 2a and b). For the trade-off of detection speed and accuracy, the resnet50 network was chosen as the pre-trained neural network for training the YOLO model.
Example 2
DeepBhvTracking was compared in performance with three commonly used tracking and localization methods in three typical experimental paradigms. The three experimental paradigms are respectively: (1) black mouse open field experiment (n ═ 6): the paradigm scene is simple, no interference of wires and experimenters exists, and the signal-to-noise ratio is high; (2) black mouse L-maze experiment (n ═ 6): the model mouse head is provided with a calcium imaging recording device, and an experimenter needs to enter a scene to perform necessary operation due to the interference of electric wires; (3) white mouse three-box experiment (n ═ 6): the model mouse head is provided with a calcium imaging recording device, the interference of electric wires exists, the color of the mouse is similar to that of the background, and the signal-to-noise ratio is low. The three tracking methods are as follows: the method comprises a background noise reduction method, a YOLO algorithm and a DeepLabCut algorithm, wherein the YOLO, the DeepLabCut and the DeepBhvTracking are all methods developed based on deep learning, and a data set is required for model training. The three methods adopt the same data set pictures for training, and the data set is formed by extracting 300 pictures from each of the three normal forms (6 videos in each normal form and 50 pictures in each video). When comparing DeepBhvTracking with other methods, the significance level was corrected by Bonferroni method using the rank sum test. The results of the four methods of tracking and locating are not manually corrected for later comparative analysis.
The results show that the background noise reduction method cannot accurately track animals in complex scenes such as an L-shaped maze and a three-box experiment (fig. 3a and e), and the variation per frame and the error from the true value calculated in the two paradigms are both significantly larger than those of the DeepBhvtracking method (fig. 3k and 1). But the background noise reduction method has the advantages of high detection speed and no need of training a model (fig. 3j), and has better tracking effect in a simple scene such as an open field experiment (fig. 3k and 1). Although the YOLO algorithm effectively solves the interference of wires and human factors, since YOLO can only locate the animal through the bounding box, the center position of the animal cannot be accurately obtained, so that the tracked track jumps, resulting in track abnormality (fig. 3b, f, k, l). The DeepLabCut is used as an attitude estimation method based on feature point detection, and can also effectively solve the interference of wires and human factors (figures 3c and g). However, the central position of the animal cannot be accurately marked manually when the training data is created, and partial frame identification fails, so that the tracking effect in the L-shaped maze is poor, and the calculated variation of each frame and the error from the true value are both significantly larger than those of the deep bvtracking method (fig. 3k and L). Meanwhile, the detection speed of the deep LabCut is also slower (FIG. 2 j). In summary, deep bhvtracking can guarantee relatively good tracking effect at a faster processing speed in various paradigms (fig. 2).
Example 3
To verify the availability and generalization ability of DeepBhvTracking, it was applied in tracking animals of different paradigms: black mouse treadmill experiment (fig. 4a), black mouse inverted V-maze (fig. 4b), black mouse elevated plus maze (fig. 4c), white mouse triple box experiment (fig. 4d) and marmoset movement in monkey cages (fig. 4 e). The animal relatively smooth motion track can be obtained in different paradigms, which shows that the DeepBhvTracking can be widely applied to different animal models and different motion scenes.
Example 4
Fig. 5 is an application example of deep bhvtracking in the field of neuromedical research, and 3 mice were used for open field experiments, which are respectively: wild type C57BL/6 mice (n ═ 6), mutant PRRT2 mice (n ═ 6), and mutant FMRl mice (n ═ 6). Open field experiments are one of the common paradigms for assessing rodent locomotor status and anxiety levels. First, we calculated the trajectory of each mouse moving within 8 minutes of open field using DeepBhvTracking (figure 5 a). Then, the moving time and the moving speed of the mouse at each position of the open field can be calculated through the moving track (fig. 5b and c). All mice tend to stay more in the corners and the speed of movement at the corners is less than the speed of movement at the center. Also, the mean locomotor speed was greater for both mutant mice than for wild-type mice, but PRRT2 mice stayed more centrally than for wild-type mice, indicating a lower level of anxiety.
Claims (9)
1. An animal center tracking method based on deep learning is characterized by comprising the following steps:
(1) model training
(1.1) randomly extracting pictures from a scene video to be tested, then manually creating a data set of a bounding box for extracting the pictures for model training,
(1.2) dividing the data set into a training set, a cross validation set and a test set, wherein the data of the training set is used for training the parameters of the model, the cross validation set and the test set are used for validating and testing the effect of the model,
during training, the pre-trained neural network is adopted to extract the characteristics of the picture, the target detection layer of the YOLO algorithm is adopted to identify the boundary frame of the target animal, and finally the trained model is obtained,
(2) animal tracking
(2.1) manually defining the tracked area before each tracking, and setting the part of each frame image in the video outside the area as a color similar to the background,
(2.2) tracking all frame pictures in the video by using a trained model, calculating a boundary frame of an animal, (2.3) converting the format of each frame in the video into a gray format, calculating the average value of all frames of the video as the background noise of the video,
(2.4) subtracting the background noise corresponding to the bounding box area from each frame of picture in the video by using a background noise reduction method to obtain the outline of the animal in the bounding box,
and (2.5) finally, tracking the animal by calculating the centroid of the animal outline in all the frames of the video to obtain a trajectory diagram of the animal movement.
2. The animal centric tracking method according to claim 1, characterized in that in step (1.1), the picture is in RGB format.
3. The animal centric tracking method according to claim 1, characterized in that in step (1.2), the pre-trained neural network is resnet18, mobilenetv2 or resnet 50.
4. The animal centric tracking method according to claim 1, characterized in that in step (1.2) an object detection layer of a YOLO-specific recognition bounding box is connected after the pre-trained neural network.
5. The animal centric tracking method according to claim 1, characterized in that in step (1.2), the mini-batch gradient descent method is used during training, and parameters in the network are adjusted by back propagation in an iterative process.
6. The animal centric tracking method according to claim 1, characterized in that in step (2.2), if YOLO calculates a plurality of possible bounding boxes, the bounding box with the largest predicted p-value is selected for later processing.
7. The method for animal center tracking according to claim 1, wherein in step (2.2), the predicted bounding box is enlarged in a ratio of 1: 1.5.
8. The animal centric tracking method according to claim 1, characterized in that in animal tracking, animal tracking of a single video file is realized by dbt _ singleTracking; animal tracking of multiple video files is achieved by dbt _ batch tracking; the tracking completed video is created by dbt _ createlabedvideo.
9. The animal centric tracking method according to claim 1, characterized in that the failed tracking frame is corrected manually by dbt _ manual tracking or derived by dbt _ optimization, and is merged with the data set in step (1.1), and then retrained by step (1.2) again to be used as the final trained model for animal tracking in step (2).
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110765039.2A CN113470073A (en) | 2021-07-06 | 2021-07-06 | Animal center tracking method based on deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110765039.2A CN113470073A (en) | 2021-07-06 | 2021-07-06 | Animal center tracking method based on deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113470073A true CN113470073A (en) | 2021-10-01 |
Family
ID=77878699
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110765039.2A Pending CN113470073A (en) | 2021-07-06 | 2021-07-06 | Animal center tracking method based on deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113470073A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359821A (en) * | 2021-12-01 | 2022-04-15 | 北京昭衍新药研究中心股份有限公司 | Group primate motion quantity estimation method based on target detection |
| CN116309727A (en) * | 2023-05-26 | 2023-06-23 | 中南大学 | Unmanned aerial vehicle target tracking method and simulation system based on deep learning algorithm |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108550147A (en) * | 2018-04-13 | 2018-09-18 | 华中科技大学苏州脑空间信息研究院 | Full cranial nerve beam projected path method for automatic tracking and system |
| CN109584266A (en) * | 2018-11-15 | 2019-04-05 | 腾讯科技（深圳）有限公司 | A kind of object detection method and device |
| CN110781806A (en) * | 2019-10-23 | 2020-02-11 | 浙江工业大学 | Pedestrian detection tracking method based on YOLO |
| CN111612002A (en) * | 2020-06-04 | 2020-09-01 | 广州市锲致智能技术有限公司 | Multi-target object motion tracking method based on neural network |
-
2021
- 2021-07-06 CN CN202110765039.2A patent/CN113470073A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108550147A (en) * | 2018-04-13 | 2018-09-18 | 华中科技大学苏州脑空间信息研究院 | Full cranial nerve beam projected path method for automatic tracking and system |
| CN109584266A (en) * | 2018-11-15 | 2019-04-05 | 腾讯科技（深圳）有限公司 | A kind of object detection method and device |
| CN110781806A (en) * | 2019-10-23 | 2020-02-11 | 浙江工业大学 | Pedestrian detection tracking method based on YOLO |
| CN111612002A (en) * | 2020-06-04 | 2020-09-01 | 广州市锲致智能技术有限公司 | Multi-target object motion tracking method based on neural network |
Non-Patent Citations (2)
| Title |
|---|
| 于红: "水产动物目标探测与追踪技术及应用研究进展", 《大连海洋大学学报》, vol. 35, no. 6, pages 793 - 804 * |
| 王春林 等: "《人工智能》", 30 September 2020, 西安电子科技大学出版社, pages: 181 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114359821A (en) * | 2021-12-01 | 2022-04-15 | 北京昭衍新药研究中心股份有限公司 | Group primate motion quantity estimation method based on target detection |
| CN116309727A (en) * | 2023-05-26 | 2023-06-23 | 中南大学 | Unmanned aerial vehicle target tracking method and simulation system based on deep learning algorithm |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11908244B2 (en) | Human posture detection utilizing posture reference maps | |
| CN110070074B (en) | Method for constructing pedestrian detection model | |
| CN105550678B (en) | Human action feature extracting method based on global prominent edge region | |
| CN102789568B (en) | Gesture identification method based on depth information | |
| CN107424171B (en) | Block-based anti-occlusion target tracking method | |
| CN107516316B (en) | Method for segmenting static human body image by introducing focusing mechanism into FCN | |
| CN104392223B (en) | Human posture recognition method in two-dimensional video image | |
| CN105160310A (en) | 3D (three-dimensional) convolutional neural network based human body behavior recognition method | |
| CN106648078B (en) | Multi-mode interaction method and system applied to intelligent robot | |
| CN105426929B (en) | Object shapes alignment device, object handles devices and methods therefor | |
| KR20130013122A (en) | Apparatus and method for detecting object pose | |
| CN106682641A (en) | Pedestrian identification method based on image with FHOG- LBPH feature | |
| CN101986348A (en) | Visual target identification and tracking method | |
| CN113470073A (en) | Animal center tracking method based on deep learning | |
| CN110991278A (en) | Human body action recognition method and device in video of computer vision system | |
| CN103902989A (en) | Human body motion video recognition method based on non-negative matrix factorization | |
| Hu et al. | Depth sensor based human detection for indoor surveillance | |
| CN112329656A (en) | Feature extraction method for human action key frame in video stream | |
| KR20200101521A (en) | Semantic matchaing apparatus and method | |
| Guo et al. | Gesture recognition of traffic police based on static and dynamic descriptor fusion | |
| CN117541994A (en) | Abnormal behavior detection model and detection method in dense multi-person scene | |
| CN117854155B (en) | Human skeleton action recognition method and system | |
| TWI845797B (en) | Object detection device and object detection method | |
| CN116994049A (en) | Full-automatic flat knitting machine and method thereof | |
| US20230005162A1 (en) | Image processing system, image processing method, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |