CN116563895A - Video-based animal individual identification method - Google Patents
Video-based animal individual identification method Download PDFInfo
- Publication number
- CN116563895A CN116563895A CN202310840442.6A CN202310840442A CN116563895A CN 116563895 A CN116563895 A CN 116563895A CN 202310840442 A CN202310840442 A CN 202310840442A CN 116563895 A CN116563895 A CN 116563895A
- Authority
- CN
- China
- Prior art keywords
- features
- node
- local
- feature
- nodes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 241001465754 Metazoa Species 0.000 title claims abstract description 66
- 238000000034 method Methods 0.000 title claims abstract description 39
- 238000011176 pooling Methods 0.000 claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims abstract description 14
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 14
- 230000004931 aggregating effect Effects 0.000 claims abstract description 9
- 238000000605 extraction Methods 0.000 claims abstract description 9
- 230000004913 activation Effects 0.000 claims description 10
- 238000012935 Averaging Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 208000025174 PANDAS Diseases 0.000 description 15
- 208000021155 Paediatric autoimmune neuropsychiatric disorders associated with streptococcal infection Diseases 0.000 description 15
- 240000004718 Panda Species 0.000 description 15
- 235000016496 Panda oleosa Nutrition 0.000 description 15
- 239000004973 liquid crystal related substance Substances 0.000 description 6
- 230000036544 posture Effects 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 4
- 230000033001 locomotion Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 241000894007 species Species 0.000 description 2
- 230000001629 suppression Effects 0.000 description 2
- 241000283153 Cetacea Species 0.000 description 1
- 241000282376 Panthera tigris Species 0.000 description 1
- 241000283216 Phocidae Species 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- RFHAOTPXVQNOHP-UHFFFAOYSA-N fluconazole Chemical compound C1=NC=NN1CC(C=1C(=CC(F)=CC=1)F)(O)CN1C=NC=N1 RFHAOTPXVQNOHP-UHFFFAOYSA-N 0.000 description 1
- 238000005065 mining Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000011179 visual inspection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/042—Knowledge-based neural networks; Logical representations of neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/70—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in livestock or poultry
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Human Computer Interaction (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a video-based animal individual identification method, which comprises the following steps: s1, extracting characteristics of each frame in animal tracking video based on a convolutional neural network; s2, acquiring local discriminant features in each frame based on a local discriminant feature extraction module; step S3, the local discriminant features are aggregated based on the hypergraph neural network, and final local features are obtained; step S4, aggregating the cross-frame features in the features of each frame extracted in the step S1 by adopting time average pooling to obtain global features; and S5, connecting the local features and the global features to obtain the mixed features of the animals so as to realize individual identification of the animals. When the method is applied, the individual identification accuracy of animals can be improved under the unconstrained shooting environment.
Description
Technical Field
The invention relates to the technical field of computer vision, in particular to a video-based animal individual identification method.
Background
With the development of computer technology, the application of computer vision and image processing technology to animal individual identification research is becoming more and more widespread, and picture-based animal individual identification methods are also rapidly developing. In picture-based animal individual identification methods, it is often based on certain conditions, such as facial visibility, side pattern visibility, etc. Among them, pandas, northeast tigers, seal, whales, etc. are more common. However, the animal face or the whole body in the real scene is clear and free from the picture which is difficult to capture, a large amount of unavailable data is often required to be discarded, and the performance of the recognition model can be greatly reduced under the conditions of shielding, darkness or blurring.
At present, the video-based re-recognition task is generally applied to pedestrian recognition, compared with pedestrians, animal motions are more random, a large number of external shielding, self-shielding or motion blurring and the like are easy to occur, the differences among different animal individuals of the same species are small, no obvious difference exists in appearance, and the same animal individual can generate larger intra-class differences under different postures. Thus, there are major challenges in applying video-based re-identification to individual animal identification, no relevant reports are reported today, and no relevant introduction is available in the prior art.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a video-based animal individual identification method which can improve the individual identification accuracy of animals under an unconstrained shooting environment when applied.
The aim of the invention is mainly realized by the following technical scheme:
a video-based animal individual identification method comprising the steps of:
s1, extracting characteristics of each frame in animal tracking video based on a convolutional neural network;
s2, acquiring local discriminant features in each frame based on a local discriminant feature extraction module;
step S3, the local discriminant features are aggregated based on the hypergraph neural network, and final local features are obtained;
step S4, aggregating the cross-frame features in the features of each frame extracted in the step S1 by adopting time average pooling to obtain global features;
and S5, connecting the local features and the global features to obtain the mixed features of the animals so as to realize individual identification of the animals.
Further, the local discriminant feature obtained in step S2 is a feature map on each frame, where the feature map is obtained by searching information of each frame through a sliding window.
Further, step S2 further includes sliding the obtained feature images with windows of different sizes, outputting feature images of different sizes, calculating an activation mean value of each window through global averaging pooling, sorting the activation mean values, selecting a set number of windows according to the sorting result, clipping a local area selected by the windows on the input original image, and then sending the local area to a convolutional neural network again for learning, so as to obtain a new feature image as a local discriminant feature.
Further, the step S3 includes the following steps:
step S31, constructing hypergraph g= (V, E); wherein G represents a hypergraph, V represents a node set, E represents a hyperedge set, the nodes of the hypergraph are local discriminant features obtained in the step S2, each hyperedge is connected with a plurality of nodes, and the nodes connected on each hyperedge are connected according to a set fixed time threshold valueT h To select;
and step S32, updating the node characteristics by adopting a hypergraph neural network to obtain hypergraph characteristics.
Further, in step S31, the nodes connected to each superside are determined according to a set fixed time thresholdT h To select the method comprises the following steps: for any node, a fixed time threshold is found out from the time interval between the node and the nodeT h Then K nearest connection points of the node are calculated by adopting a K nearest neighbor algorithm, and then K+1 characteristic nodes are connected by adopting a superside; and executing the operations to establish the supersides for all the nodes to obtain a plurality of supersides.
Further, the step S32 includes constructing an over-edge feature according to the feature of the node and finishing feature update of the node by converging the over-edge feature;
the construction of the superside feature according to the feature of the node comprises the following steps: for each node, acquiring all superedges containing the node, and for each superedge, carrying out average operation on other node characteristics except the node, thereby constructing the characteristic of the superedge;
the feature updating of the convergent superside feature completion node comprises the following steps: aggregating the superside characteristics to obtain all superside information related to the node, performing connection operation on the information of the node and the superside information, and updating the node characteristics by using a full connection layer; and executing the same operation on the nodes of all the graphs, and executing the average pooling operation on the characteristics of all the nodes to obtain the final local characteristics.
The invention adopts a double-branch network structure, comprising local branches and global branches, extracts the local features and global features of animals respectively, fully digs fine-grained features and simultaneously retains global information, and combines the fine-grained features to obtain more differentiated feature expression. Because animals are more flexible and have more various postures compared with pedestrians, the invention designs a fine-granularity characteristic extraction module which is used for adaptively searching the distinguishing characteristics in each frame on a local branch so as to adapt to complex posture changes of the animals, and the difference between different individuals with similar appearance is found out as much as possible by further mining the distinguishing local characteristics. The invention establishes the time dependency relationship among the local features of different frames by using the hypergraph neural network, and aggregates the local features so as to better solve the problem that animal individuals in certain frames are blocked.
In summary, compared with the prior art, the invention has the following beneficial effects: the invention is inspired by a pedestrian re-identification technology based on video, and provides an animal individual identification method based on video to solve the problem of the animal individual identification method based on pictures.
Drawings
The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention. In the drawings:
FIG. 1 is a flow chart of an embodiment of the present invention;
fig. 2 is a schematic diagram of a network architecture corresponding to the application of an embodiment of the present invention in panda individual identification.
Detailed Description
For the purpose of making apparent the objects, technical solutions and advantages of the present invention, the present invention will be further described in detail with reference to the following examples and the accompanying drawings, wherein the exemplary embodiments of the present invention and the descriptions thereof are for illustrating the present invention only and are not to be construed as limiting the present invention.
Examples:
as shown in fig. 1, the video-based animal individual identification method comprises the following steps: s1, extracting characteristics of each frame in animal tracking video based on a convolutional neural network; s2, acquiring local discriminant features in each frame based on a local discriminant feature extraction module; step S3, the local discriminant features are aggregated based on the hypergraph neural network, and final local features are obtained; step S4, aggregating the cross-frame features in the features of each frame extracted in the step S1 by adopting time average pooling to obtain global features; and S5, connecting the local features and the global features to obtain the mixed features of the animals so as to realize individual identification of the animals. In the embodiment, the time-averaged pooling in step S4, that is, global-averaged pooling in the time dimension, aggregates the features of the cross-frame to obtain the global features of the animal. In the embodiment, steps S1 to S3 are sequentially executed, step S4 is executed synchronously with steps S2 and S3, and step S5 is executed after steps S3 and S4 are executed. In the embodiment, the convolutional neural network uses the resnet50 network as a backbone network to extract the features of each frame.
To alleviate the pressure of manual identification, many deep learning-based methods have been studied for automated individual animal identification. Often, different individuals of the same species differ only slightly in appearance, and therefore most work is distinguished based on the fine-grained local characteristics of animals. At present, the work of animal re-identification is almost based on pictures, a method which is more suitable for animal video individual identification is not designed according to the characteristics of animals, and the performance of the existing method is obviously reduced in an unconstrained environment. Therefore, the embodiment proposes to use video to perform individual recognition so as to enhance the robustness of the individual recognition model and expand the application scene. Currently, in video-based re-recognition work, the subject is typically a pedestrian. Video re-recognition based on deep learning mainly explores how to aggregate a series of image-level features into video-level features, i.e. how to perform temporal modeling. The prior time modeling method of pedestrian re-identification technology based on video mainly comprises five main categories: time pooling, time attention, optical flow, recurrent Neural Networks (RNNs), and three-dimensional convolutional neural networks. However, the existing method does not consider time modeling of parts with finer granularity, is difficult to distinguish individuals with similar appearance, and is difficult to deal with the problems of shielding, posture change and the like. In order to further capture the identification characteristics among different individuals, some time modeling methods combining local information and global information are proposed, the time modeling methods often adopt the form of horizontal segmentation of a characteristic diagram when acquiring the local characteristics of pedestrians, animals are different from pedestrians, the pedestrians are often stable up-down structures, and many animals can have any postures. Therefore, the embodiment adopts a more flexible local feature acquisition mode, and the identifying features are searched through model self-adaptive learning.
As shown in fig. 2, when the present embodiment is applied to panda individual recognition, a panda video clip with T frames is provided, and the present embodiment extracts the features of each frame through a convolutional neural network. Then, in the local branches, the embodiment obtains a discriminating local part of each frame by using a local discriminating feature extracting module (DPSM, discriminative Patch Search Module), and obtains a local feature representation of the panda by aggregating the local features by a hypergraph neural network module (PHM, patch Hypergraph Module) based on the local key features. Meanwhile, in the global branch, the embodiment uses time-averaged pooling to aggregate the cross-frame features to obtain the global feature representation of the pandas. Finally, by linking together the local and global features of the panda, a hybrid feature representation is obtained to achieve individual identification of the panda. In FIG. 2TFor the number of video level clip frames,Hfor the height of the video level clip frame,Wfor the width of the video-level clip frame,F 1 to extract features of the first frame via the convolutional neural network,F T to extract the first via convolutional neural networkTThe characteristics of the frame are such that,BNthe batch standardization is indicated and the batch is standardized,FCindicating that the full-link layer is to be formed,L tri representing the loss of a triplet,L ce representing the cross-entropy loss,f local the local features are represented by a graph of the local features,f global representing global features.
The local discriminant features obtained in step S2 of the present embodiment are feature graphs on each frame, where the feature graphs are obtained by searching information of each frame by means of a sliding window. Step S2 of this embodiment further includes sliding the obtained feature graphs by using windows of different sizes, outputting feature graphs of different sizes, calculating an activation mean value of each window by global averaging pooling, sorting the activation mean values, selecting a set number of windows from large to small according to the sorting result, clipping a local area selected by the windows on the original graph, and then sending the local area to a convolutional neural network again for learning, so as to obtain a new feature graph as a local discriminant feature.
The different animal individuals of the same special type are quite similar, and when the embodiment is applied to the identification of panda individuals, some local features may not be visible in one frame but are visible in another frame due to shielding. This example better distinguishes between different animal individuals by learning fine-grained features. In pedestrian re-recognition, the local features are often obtained by horizontally slicing the feature map, but animals are greatly different from pedestrians. The animal can generate larger deformation and self-shielding under the condition of larger movement amplitude. The inventors believe that the use of horizontal dicing does not match well to the distinguishing characteristics of the animal. Therefore, the present embodiment proposes a local discriminant feature extraction module (DPSM, discriminative Patch Search Module) that searches for key information on each frame through network adaptive learning.
Specifically, the embodiment obtains the feature map on each input frame through the convolutional neural network, searches the most information-rich area in each frame by using a sliding window mode in target detection in the local discriminant feature extraction module, uses the most information-rich area as a local key feature, and uses the full convolutional network to realize the process. In this embodiment, the obtained feature images are slid by using windows with different sizes, and feature images with different sizes are output. This is due to the fact that animals are more mobile, such as standing, eating, sleeping, etc., which can exhibit rich shape characteristics, it is difficult to match the characteristics of animals in all shapes if a single size sliding window is used. Then, the embodiment calculates the activation mean value of each window through global average pooling, and the larger the activation mean value is, the more abundant the information contained under the window is. Thus, the present embodiment sorts the activation averages. In addition, by visual inspection, the inventors found that the distinguishing characteristics of the animals are distributed mostly in the head, tail and neck, and thus the window that the present embodiment is expected to select when applied is not concentrated in only one area. In this embodiment, non-maximum suppression (NMS) is used to select N windows according to the sorting result, and these local areas are cut on the original image input, and then sent to the convolutional neural network again for learning, so as to obtain a new feature map. In this way, the present embodiment can pay attention to the detail difference between different individuals of the same specific type as much as possible.
The local discriminant feature extraction module of this embodiment is implemented with a local pooling layer that slides on the feature map using sliding windows of different sizes (i.e., pooling cores of different sizes) to calculate the activation mean for each window. The means are then sorted and a fixed number of windows are selected using non-maximum suppression (NMS), each window corresponding to a local discriminant feature.
After searching the discriminant feature of each frame, the present embodiment does not reflect the frame-to-frame association. During panda motion, a portion that is occluded on one frame may appear in other frames. The present embodiment further achieves this by establishing a time-dependent relationship between different frames to obtain a more comprehensive characterization, particularly using hypergraph neural networks. Each edge of the hypergraph neural network can be connected with a plurality of nodes compared with the standard graph neural network, which is helpful for modeling complex data relations and obtaining more powerful feature expression.
Step S3 of the present embodiment includes the steps of: step S31, constructing hypergraph g= (V, E); wherein G represents a hypergraph, V represents a node set, E represents a hyperedge set, the nodes of the hypergraph are local discriminant features obtained in the step S2, each hyperedge is connected with a plurality of nodes, and the nodes connected on each hyperedge are connected according to a set fixed time threshold valueT h To select; step S32. And updating the node characteristics by adopting a hypergraph neural network to obtain hypergraph characteristics. The nodes are represented as follows:
wherein, the liquid crystal display device comprises a liquid crystal display device,Va set of nodes is represented and,the node is represented by a set of nodes,Tfor the number of video level clip frames,Nfor the number of windows selected.
In step S31 of this embodiment, the nodes connected to each superside are set according to the set fixed time thresholdT h To select the method comprises the following steps: for any node, a fixed time threshold is found out from the time interval between the node and the nodeT h Then K nearest connection points of the node are calculated by adopting a K nearest neighbor algorithm, and then K+1 characteristic nodes are connected by adopting a superside; and executing the operations to establish the supersides for all the nodes to obtain a plurality of supersides. Wherein each node does not have more than one superside, as it may be included in K-nearest neighbor nodes of other nodes.
The hyperedge is expressed as:
wherein, the liquid crystal display device comprises a liquid crystal display device,Erepresenting a set of hyperedges that are to be processed,representing the union of the sets->Representation->Is a set of K neighboring nodes. Thus, the present embodiment completes the construction of the hypergraph.
Step S32 of the present embodiment includes constructing a superside feature according to a feature of a node and completing feature update of the node by aggregating the superside feature, where constructing the superside feature according to the feature of the node includes: and for each node, acquiring all the superedges comprising the node, and for each superedge, carrying out average operation on the characteristics of other nodes except the node, thereby constructing the characteristics of the superedge.
Wherein, the liquid crystal display device comprises a liquid crystal display device,representing the division of +/in each superside>Features of nodes other than the node.
The feature updating of the converging superside feature completion node in this embodiment includes: and aggregating the superside characteristics to obtain all the superside information related to the nodes, and then performing splicing operation on the information of the nodes and the superside information, so that the characteristics of each node comprise other characteristics adjacent in time, the characteristics of the nodes are enhanced or supplemented, and the learning of more complete animal characteristic expression is facilitated. Then updating the node characteristics by using a full connection layer; and executing the same operation on the nodes of all the graphs, and executing the average pooling operation on the characteristics of all the nodes to obtain the final local characteristics.
According to the embodiment, the local features and the global features are connected to obtain the mixed features of the animals to realize individual identification of the animals, so that the animal features with more representativeness and discrimination are obtained.
When the embodiment is applied, the cross entropy loss is adoptedL ce And triplet lossL tri To co-monitor training. Triple lossL tri Can make the degree of distinction between different classes higher and cross entropy lossL ce The model can be aided in learning the inter-class differences from global, both in combination to help learn more discriminative animal feature representations.
The embodiment provides a video-based automatic animal individual identification network framework combining local features and global features. In the embodiment, the animal characteristic representation with finer granularity is obtained through the local discriminant characteristic extraction module so as to improve the discriminant capability of the model. The hypergraph neural network is further used for modeling the time dependency relationship between local features so as to obtain more comprehensive video-level feature representation and alleviate the problem of model performance degradation caused by shielding.
To evaluate the effectiveness of the method of this example, the inventors constructed an animal video database, with video data from a adult panda breeding study base, containing 171 videos of 60 small pandas. The performance on the present dataset shows that the present embodiment can adaptively learn the local discriminant features of the pandas in each frame when applied, although the pandas exhibit large deformation or posture changes in different frames. In the embodiment, whether video clips in the query set and the gallery set are from the same source video is evaluated respectively, and the evaluation proves that the accuracy of the method in the aspect of animal individual identification is high, and the effectiveness of the method is proved. The method is also compared with a picture baseline method when the method is applied to the identification of panda individuals, the probability of incorrect matching of the picture baseline method is high, and the method can be correctly matched when the method is applied, so that the effectiveness of the method in the identification of animal individuals is proved.
The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the scope of the invention, but to limit the invention to the particular embodiments, and any modifications, equivalents, improvements, etc. that fall within the spirit and principles of the invention are intended to be included within the scope of the invention.
Claims (6)
1. A video-based method for identifying an individual animal comprising the steps of:
s1, extracting characteristics of each frame in animal tracking video based on a convolutional neural network;
s2, acquiring local discriminant features in each frame based on a local discriminant feature extraction module;
step S3, the local discriminant features are aggregated based on the hypergraph neural network, and final local features are obtained;
step S4, aggregating the cross-frame features in the features of each frame extracted in the step S1 by adopting time average pooling to obtain global features;
and S5, connecting the local features and the global features to obtain the mixed features of the animals so as to realize individual identification of the animals.
2. The method according to claim 1, wherein the local discriminant features obtained in step S2 are feature maps on each frame, and the feature maps are obtained by searching for information of each frame through a sliding window.
3. The method for identifying animal individuals based on video according to claim 2, wherein step S2 further comprises sliding windows of different sizes after obtaining the feature map, outputting the feature map of different sizes, calculating an activation mean value of each window through global averaging pooling, sorting the activation mean values, selecting a set number of windows according to the sorting result, clipping a local area selected by the windows on the input original map, and then sending the local area to a convolutional neural network again for learning, so as to obtain a new feature map as a local discriminant feature.
4. The video-based animal individual identification method according to claim 1, wherein the step S3 comprises the steps of:
step S31, constructing hypergraph g= (V, E); wherein G represents a hypergraph, V represents a node set, E represents a hyperedge set, the nodes of the hypergraph are local discriminant features obtained in the step S2, each hyperedge is connected with a plurality of nodes, and the nodes connected on each hyperedge are connected according to a set fixed time threshold valueT h To select;
and step S32, updating the node characteristics by adopting a hypergraph neural network to obtain hypergraph characteristics.
5. The video-based animal individual identification method according to claim 4, wherein the node connected to each superside in the step S31 is determined according to a set fixed time thresholdT h To select the method comprises the following steps: for any node, a fixed time threshold is found out from the time interval between the node and the nodeT h Then K nearest connection points of the node are calculated by adopting a K nearest neighbor algorithm, and then K+1 characteristic nodes are connected by adopting a superside; and executing the operations to establish the supersides for all the nodes to obtain a plurality of supersides.
6. The video-based animal individual identification method according to claim 4, wherein the step S32 includes constructing superside features from the features of the nodes and completing feature update of the nodes by converging the superside features;
the construction of the superside feature according to the feature of the node comprises the following steps: for each node, acquiring all superedges containing the node, and for each superedge, carrying out average operation on other node characteristics except the node, thereby constructing the characteristic of the superedge;
the feature updating of the convergent superside feature completion node comprises the following steps: aggregating the superside characteristics to obtain all superside information related to the node, performing connection operation on the information of the node and the superside information, and updating the node characteristics by using a full connection layer; and executing the same operation on the nodes of all the graphs, and executing the average pooling operation on the characteristics of all the nodes to obtain the final local characteristics.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310840442.6A CN116563895A (en) | 2023-07-11 | 2023-07-11 | Video-based animal individual identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310840442.6A CN116563895A (en) | 2023-07-11 | 2023-07-11 | Video-based animal individual identification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116563895A true CN116563895A (en) | 2023-08-08 |
Family
ID=87503897
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310840442.6A Pending CN116563895A (en) | 2023-07-11 | 2023-07-11 | Video-based animal individual identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116563895A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418127A (en) * | 2020-11-30 | 2021-02-26 | 浙江大学 | Video sequence coding and decoding method for video pedestrian re-identification |
| CN115410222A (en) * | 2022-07-26 | 2022-11-29 | 安徽朗巴智能科技有限公司 | Video pedestrian re-recognition network with posture sensing function |
| CN116229511A (en) * | 2023-02-23 | 2023-06-06 | 西北大学 | Identification re-recognition method based on golden monkey trunk feature extraction |
-
2023
- 2023-07-11 CN CN202310840442.6A patent/CN116563895A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112418127A (en) * | 2020-11-30 | 2021-02-26 | 浙江大学 | Video sequence coding and decoding method for video pedestrian re-identification |
| CN115410222A (en) * | 2022-07-26 | 2022-11-29 | 安徽朗巴智能科技有限公司 | Video pedestrian re-recognition network with posture sensing function |
| CN116229511A (en) * | 2023-02-23 | 2023-06-06 | 西北大学 | Identification re-recognition method based on golden monkey trunk feature extraction |
Non-Patent Citations (2)
| Title |
|---|
| LEI LI等: "Video-based Red Panda Individual Identification by Adaptively Aggregating Discriminative Features", 《IJCNN》, pages 1 - 7 * |
| YICHAO YAN等: "Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification", 《CVPR》, pages 2899 - 2908 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2019169816A1 (en) | Deep neural network for fine recognition of vehicle attributes, and training method thereof | |
| CN110263712B (en) | Coarse and fine pedestrian detection method based on region candidates | |
| CN111639564B (en) | Video pedestrian re-identification method based on multi-attention heterogeneous network | |
| CN103093198B (en) | A kind of crowd density monitoring method and device | |
| CN110929593A (en) | Real-time significance pedestrian detection method based on detail distinguishing and distinguishing | |
| CN101470809A (en) | Moving object detection method based on expansion mixed gauss model | |
| CN113255616B (en) | Video behavior identification method based on deep learning | |
| CN109165658B (en) | Strong negative sample underwater target detection method based on fast-RCNN | |
| CN110717863B (en) | Single image snow removing method based on generation countermeasure network | |
| CN107194948B (en) | Video significance detection method based on integrated prediction and time-space domain propagation | |
| CN103810473A (en) | Hidden Markov model based human body object target identification method | |
| CN111652035B (en) | Pedestrian re-identification method and system based on ST-SSCA-Net | |
| CN111723773A (en) | Remnant detection method, device, electronic equipment and readable storage medium | |
| CN114677323A (en) | Semantic vision SLAM positioning method based on target detection in indoor dynamic scene | |
| JP6448212B2 (en) | Recognition device and recognition method | |
| CN111461076A (en) | Smoke detection method and smoke detection system combining frame difference method and neural network | |
| CN114782997A (en) | Pedestrian re-identification method and system based on multi-loss attention adaptive network | |
| CN116704490B (en) | License plate recognition method, license plate recognition device and computer equipment | |
| CN111582057B (en) | Face verification method based on local receptive field | |
| CN113191352A (en) | Water meter pointer reading identification method based on target detection and binary image detection | |
| CN116563895A (en) | Video-based animal individual identification method | |
| CN110929632A (en) | Complex scene-oriented vehicle target detection method and device | |
| CN108256444B (en) | Target detection method for vehicle-mounted vision system | |
| CN117197687A (en) | Unmanned aerial vehicle aerial photography-oriented detection method for dense small targets | |
| CN113763418B (en) | Multi-target tracking method based on head and shoulder detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20230808 |
|
| RJ01 | Rejection of invention patent application after publication |