CN117456620A - Multi-mode milk cow individual identification method and system integrating action information - Google Patents
Multi-mode milk cow individual identification method and system integrating action information Download PDFInfo
- Publication number
- CN117456620A CN117456620A CN202311397300.3A CN202311397300A CN117456620A CN 117456620 A CN117456620 A CN 117456620A CN 202311397300 A CN202311397300 A CN 202311397300A CN 117456620 A CN117456620 A CN 117456620A
- Authority
- CN
- China
- Prior art keywords
- cow
- image
- features
- dairy
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000009471 action Effects 0.000 title claims abstract description 69
- 238000000034 method Methods 0.000 title claims abstract description 68
- 239000008267 milk Substances 0.000 title claims abstract description 23
- 235000013336 milk Nutrition 0.000 title claims abstract description 23
- 210000004080 milk Anatomy 0.000 title claims abstract description 23
- 241000283690 Bos taurus Species 0.000 claims abstract description 242
- 235000013365 dairy product Nutrition 0.000 claims abstract description 132
- 230000004927 fusion Effects 0.000 claims abstract description 57
- 238000012549 training Methods 0.000 claims abstract description 46
- 238000012706 support-vector machine Methods 0.000 claims abstract description 44
- 238000013135 deep learning Methods 0.000 claims abstract description 43
- 238000007781 pre-processing Methods 0.000 claims abstract description 27
- 238000012360 testing method Methods 0.000 claims abstract description 27
- 230000000694 effects Effects 0.000 claims abstract description 14
- 238000000605 extraction Methods 0.000 claims description 18
- 239000013598 vector Substances 0.000 claims description 17
- 238000010606 normalization Methods 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 12
- 238000005520 cutting process Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 11
- 238000010008 shearing Methods 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 230000005021 gait Effects 0.000 claims description 6
- 239000011159 matrix material Substances 0.000 claims description 6
- 238000012545 processing Methods 0.000 claims description 6
- 238000013527 convolutional neural network Methods 0.000 claims description 5
- 238000007500 overflow downdraw method Methods 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004891 communication Methods 0.000 description 7
- 238000005516 engineering process Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 3
- 238000011176 pooling Methods 0.000 description 3
- 238000003708 edge detection Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000007774 longterm Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 241000283070 Equus zebra Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 238000009395 breeding Methods 0.000 description 1
- 230000001488 breeding effect Effects 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000003938 response to stress Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/70—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in livestock or poultry
Abstract
The invention provides a multi-mode milk cow individual identification method and system integrating action information, comprising the following steps: collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows; converting the complete cow body image extracted from the video data into a clip image and an RGB image; preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set; constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model; feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features; and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model. The invention solves the problem of low individual identification accuracy of the existing dairy cows.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to a multi-mode milk cow individual identification method and system integrating action information.
Background
The individual identification of the dairy cows is the premise and the basis of the refined breeding of the dairy cows, the tracing of products, the analysis of behaviors, the monitoring of illness state and the grading of body conditions, and the accurate and reliable identification of each dairy cow has important significance.
In the traditional dairy cow individual identification method by means of manpower, an ear tag is the most common one. However, the ear tag is frequently lost and damaged in actual production and life, and is not suitable for long-term use. Electronic tags based on RFID technology are also common. However, the technology has limited recognition distance, high cost and large limitation. Machine vision is a popular research direction in the field of individual identification of cows, and the method is characterized in that under the condition that human intervention is not needed, the cows are recorded by using a camera, and the positioning, identification and tracking of targets in a scene are realized by analyzing videos. The identification method improves the real-time performance and the automation degree of identification, and can reduce the management cost of farms and reduce the stress response of cattle. In the aspect of machine vision identification of cows, as the pattern information of each cow is different, the cow has certain uniqueness, and the deep learning technology based on the cow body back or side pattern has a higher application prospect in individual identification. However, there are often many difficulties in applying these methods in a practical environment. For example, the appearance of cows can be greatly affected in different light and even dark environments. In addition, for a variety (e.g., zebra cows) having a pattern similar to or even a pure color of a non-pattern, a variety having a non-pattern characteristic, it is difficult to perform individual identification of cows using a pattern-based method. Therefore, the method can be used for individual identification by integrating the shape change (namely action information) of the dairy cows during walking by referring to the field of human research, and the algorithm has robustness to the identification of the dairy cows with illumination change and similar patterns. However, a single passing action feature can only be identified from a part of the moving object in a limited way, and the change of road conditions and environment and the like can have negative effects on the moving object. To improve the recognition performance of the algorithm, it is necessary to attempt to use multi-feature fusion to improve the accuracy of the single-modality recognition technology. Therefore, it is needed to propose a multi-mode milk cow individual identification method integrating action information.
Disclosure of Invention
The invention provides a multi-mode individual dairy cow identification method and system integrating action information, which are used for solving the problem of low individual dairy cow identification accuracy in the prior art.
The invention provides a multi-mode milk cow individual identification method integrating action information, which comprises the following steps:
collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
converting the complete cow body image extracted from the video data into a clip image and an RGB image;
preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features;
and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
According to the multi-mode individual identification method of dairy cows integrating action information, which is provided by the invention, the video data of the dairy cows during the activity is collected, and the complete cow body image of the dairy cows is extracted, which comprises the following steps:
selecting to collect video data when the milking of the cows is completed to leave the milking parlor;
the camera is fixedly arranged on the inner side of the aisle and is set at a distance from the dairy cow, the focal length of the camera is adjusted to enable the whole camera to have a visual field which is about 3 times of the length of the body of the dairy cow, and the shooting range at least comprises 2 gait cycles;
and extracting the whole cow body of the dairy cow through the acquired video data of the walking of the dairy cow.
According to the multi-mode milk cow individual identification method integrating action information provided by the invention, the complete cow body image extracted from the video data is converted into a clip image and an RGB image, and the method specifically comprises the following steps:
preprocessing video data of walking cows into video frames under a side view angle;
dividing a complete cow body contour side view through a preset semantic dividing network model based on the video frame;
and converting the segmented cattle body into a shearing image and an RGB image by modifying the semantic segmentation network model parameters.
According to the multi-mode milk cow individual identification method integrating action information provided by the invention, preprocessing is carried out on the clip map and the RGB picture to generate a data set, and the method specifically comprises the following steps:
cutting the clipping image and the RGB image, automatically detecting and cutting edge regions in the image through an image boundary detection algorithm, and removing irrelevant or unnecessary parts in the image to obtain an interested region;
aligning based on the cut image data, finding key feature points in the image through a feature point detection and matching algorithm, and aligning through a transformation matrix to enable objects or features in the image to be in the same position;
normalization is performed based on the aligned image data, and a linear normalization method is used to scale the pixel values to between 0 and 1 for normalization processing.
According to the multi-mode dairy cow individual identification method integrating action information provided by the invention, a deep learning network model is constructed based on the training set and the testing set, and action feature extraction and pattern feature extraction are carried out through the deep learning network model, and the method specifically comprises the following steps:
inputting the silhouette into a deep learning network model, carrying out feature extraction on a video frame sequence of the silhouette through a 3D convolution network and a two-way long-short-term memory network, and outputting action features in the form of probability vectors;
and inputting the RGB image into a deep learning network model, extracting deep features through a convolutional neural network, and outputting pattern features in the form of probability vectors through an output layer.
According to the multi-mode dairy cow individual identification method integrating the action information, which is provided by the invention, the action characteristics and the pattern characteristics are subjected to characteristic fusion, and a preset support vector machine is trained based on the fusion characteristics, and the method specifically comprises the following steps:
feature fusion is carried out on the features in the form of the two probability vectors of the action feature and the pattern feature through three fusion methods of feature weighted summation, feature value summation and feature maximum value;
selecting an optimal fusion strategy as characteristic weighted summation, wherein the weight ratio of the action characteristic to the pattern characteristic is 4:6;
and training a preset support vector machine based on the fusion characteristics.
According to the multi-mode milk cow individual identification method integrating action information provided by the invention, the trained support vector machine model is used for carrying out milk cow individual identification on milk cow images acquired in real time, and the method specifically comprises the following steps:
acquiring cow video data in real time, and inputting the video data into a trained support vector machine model;
and performing individual identification of the dairy cows through the support vector machine model.
The invention also provides a multi-mode milk cow individual identification system integrating action information, which comprises:
the data acquisition module is used for acquiring video data of dairy cows during activity and extracting complete cow body images of the dairy cows;
the image conversion module is used for converting the complete cow body image extracted from the video data into a clip image and an RGB image;
the preprocessing module is used for preprocessing the clip images and the RGB images to generate a data set, and dividing the data set into a training set and a testing set;
the feature extraction module is used for constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
the feature fusion module is used for carrying out feature fusion on the action features and the pattern features, and training a preset support vector machine based on the fusion features;
and the identification module is used for carrying out individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the multi-mode milk cow individual identification method integrated with the action information according to any one of the above when executing the program.
The invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a multi-modal cow individual identification method incorporating motion information as any one of the above.
According to the multi-mode individual dairy cow identification method and system integrating action information, complete dairy cow bodies are extracted by collecting video data of dairy cow walking, and the video data are converted into two data forms of a shearing image and an RGB image; preprocessing such as cutting, aligning, normalizing and the like is carried out on cow data; dividing the two parts of data into two parts of a training set and a testing set respectively; constructing a deep learning network model to extract action features of the body of the dairy cow during walking and pattern features of the dairy cow body; and carrying out feature fusion on the two features, classifying the dairy cows by using a support vector machine model based on the fused features, and completing individual identification of the dairy cows. The accurate identification of the dairy cows is realized, and the individual dairy cows cannot be influenced.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the invention, and other drawings can be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic flow chart of a multi-modal milk cow individual identification method incorporating motion information provided by the invention;
FIG. 2 is a second flow chart of a method for identifying individual multi-modal cows incorporating motion information provided by the invention;
FIG. 3 is a third flow chart of a method for identifying individual multi-modal cows incorporating motion information according to the present invention;
FIG. 4 is a flow chart of a method for identifying individual multi-modal cows incorporating motion information according to the present invention;
FIG. 5 is a schematic flow chart of a method for identifying individual multi-modal cows incorporating motion information provided by the invention;
FIG. 6 is a flowchart of a method for identifying individual multi-modal cows incorporating motion information according to the present invention;
FIG. 7 is a schematic diagram of a multi-modal milk cow individual identification method incorporating motion information according to the present invention;
FIG. 8 is a schematic diagram of module connection of a multi-modal individual identification system for cows incorporating motion information provided by the invention;
fig. 9 is a schematic diagram of the extraction of cow whole bovine body and conversion into a cut-off and RGB diagram provided by the invention;
fig. 10 is a schematic diagram of preprocessing cow data provided by the present invention;
FIG. 11 is a schematic diagram of the deep learning network model constructed to extract body motion features and body pattern features of dairy cows during walking;
fig. 12 is a schematic diagram of feature fusion of motion features and body pattern features of a cow during walking and classification of the cow based on the fused features, provided by the invention;
fig. 13 is a schematic structural diagram of an electronic device provided by the present invention.
Reference numerals:
110: a data acquisition module; 120: an image conversion module; 130: a preprocessing module; 140: a feature extraction module; 150: a feature fusion module; 160: an identification module;
1310: a processor; 1320: a communication interface; 1330: a memory; 1340: a communication bus.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The following describes a multi-modal milk cow individual recognition method integrating action information according to the present invention with reference to fig. 1 to 7, including:
s100, acquiring video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
s200, converting the complete cow body image extracted from the video data into a clip image and an RGB image;
s300, preprocessing the clip diagram and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
s400, constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
s500, carrying out feature fusion on the action features and the pattern features, and training a preset support vector machine based on the fusion features;
s600, performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
According to the invention, the action features and the pattern features can be accurately extracted after the deep learning network model is constructed through training, the training of the support vector machine is completed after feature fusion, and the individual dairy cows are accurately identified through the support vector machine model, so that the identification accuracy is improved, and adverse effects on the individual dairy cows are avoided.
The method for extracting the complete cow body image of the dairy cow comprises the following steps of:
s101, selecting to acquire video data when the milking of the dairy cows is completed to leave a milking parlor;
s102, a camera is fixedly arranged on the inner side of a passageway and is set to be away from a dairy cow, the focal length of the camera is adjusted to enable the whole camera to be approximately 3 times of the length of the body of the dairy cow, and the shooting range at least comprises 2 gait cycles;
s103, extracting the whole cow body of the dairy cow through the acquired video data of the dairy cow walking.
In the invention, video data acquisition is selected when the milking of the cows leaves the milking parlor, the camera is fixedly arranged at the inner side of the passageway and is about 3 meters away from the cows, and the focal length of the camera is adjusted to ensure that the whole camera vision is about 3 times of the body length of the cows, so that the shooting range at least comprises 2 gait cycles. The video data of the complete dairy cow after passing is ensured to be acquired in 2 gait cycles, and a clear image is provided for the follow-up.
Converting the complete cow body image extracted from the video data into a clip image and an RGB image, wherein the method specifically comprises the following steps:
s201, preprocessing video data of walking cows into video frames under a side view angle;
s202, dividing a complete cow body contour side view through a preset semantic division network model based on a video frame;
s203, converting the segmented cattle bodies into a shearing image and an RGB image by modifying semantic segmentation network model parameters.
In the invention, the whole cow body of the dairy cow is extracted through the acquired video data of the dairy cow walking and converted into two data forms of a scissoring image and an RGB image, and the method comprises the following steps: under the side view angle, preprocessing the video of the walking dairy cow into a video frame, and then using a deep learning method to divide the whole dairy cow body outline side view by using a deep LabV3+ semantic division network model. And finally, converting the segmented cattle body into data in two forms of a shearing image and an RGB image by modifying network parameters.
Referring to fig. 9, a complete bovine body is extracted according to the present invention and converted into two data forms of a scissors map and an RGB map. In the side view, the deep LabV3+ semantic segmentation network model is used for segmenting the whole cow body outline as shown in fig. 9 (a), and then network parameters are modified to convert the segmented cow body into data in two forms of a shearing map and an RGB map as shown in fig. 9 (b) and 9 (c).
Preprocessing the clip map and the RGB picture to generate a data set, which specifically comprises the following steps:
s301, cutting the cut-off map and the RGB picture, automatically detecting and cutting out an edge area in the image through an image boundary detection algorithm, and removing irrelevant or unnecessary parts in the image to obtain an interested area;
s302, aligning based on the cut image data, finding key feature points in the image through a feature point detection and matching algorithm, and aligning through a transformation matrix to enable objects or features in the image to be in the same position;
s303, normalizing based on the aligned image data, and scaling the pixel value to be between 0 and 1 by using a linear normalization method to perform normalization processing.
In the invention, the cow data in two forms are preprocessed, and the method comprises the following steps:
the clipping refers to automatically detecting and clipping edge regions in an image by using an image boundary detection algorithm (Canny edge detection), removing irrelevant or unnecessary parts in the image, and obtaining an interested region;
alignment means that a feature point detection and matching algorithm ORB is used to find key feature points in an image, and the key feature points are aligned through a transformation matrix, so that objects or features in the image are ensured to be in the same position, and the objects or features are kept consistent for subsequent processing steps;
normalization refers to the process of scaling pixel values between 0 and 1 for normalization using a linear normalization method.
By preprocessing the two forms of cow image data, the related image data can be more in line with the subsequent training requirement. The data after clipping, feature point detection and matching algorithm ORB alignment and linear normalization are processed by a Canny edge detection algorithm, as shown in fig. 10 (a) and 10 (b).
Constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model, wherein the method specifically comprises the following steps:
s401, inputting the silhouette graph into a deep learning network model, carrying out feature extraction on a video frame sequence of the silhouette graph through a 3D convolution network and a two-way long-short-term memory network, and outputting action features in the form of probability vectors;
s402, inputting the RGB image into a deep learning network model, extracting deep features through a convolutional neural network, and outputting pattern features in the form of probability vectors through an output layer.
In the invention, when the data set is divided, the data set is divided into a training set and a testing set according to the proportion of 7:3.
The method for constructing the deep learning network model to extract the action characteristics and the body pattern characteristics of the body of the dairy cow during walking comprises the following steps: in terms of motion feature extraction, the input silhouette data is a sequence of video frames that first pass through a 3D convolution network (r3d_18) that replaces all of the 2D convolution kernels, convolution layers, and pooling layers in the res net18 with 3D convolution kernels, convolution layers, and pooling layers, which are 3D versions of the res net18 responsible for extracting features from the video. The 3D convolution operation is performed in the time dimension and in the space dimension, capturing spatio-temporal feature information in the video. The convolved feature map is fed to a bi-directional LSTM (BiLSTM) layer that captures long-term and short-term dependencies in the input sequence, with 3200 hidden units. Finally, through an output layer, the device consists of two full-connection layers, two ReLU linear rectification function layers and a SoftMax layer, and finally, the depth characteristic with Softmax normalized exponential function activation is finally output in the form of a probability vector; in the aspect of pattern feature extraction, input RGB image data is an image, deep features are extracted through operations such as convolution, pooling and the like of an AlexNet convolutional neural network, and the deep features are finally output in the form of probability vectors through an output layer, wherein the deep features consist of three full-connection layers, two ReLU layers and a softMax layer. According to the invention, a deep learning network model is constructed to extract the action characteristics and body pattern characteristics of the body of the dairy cow during walking. The 3D CNN-BiLSTM with R3D_18 as the main trunk is used for extracting the body motion characteristics of the dairy cows, alexNet is used for extracting the body pattern characteristics of the dairy cows, and the body pattern characteristics are output in the form of probability vectors, as shown in fig. 11 (a) and 11 (b).
Feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features, and the method specifically comprises the following steps:
s501, feature fusion is carried out on the features in the form of the motion feature and the pattern feature in the form of two probability vectors through three fusion methods of feature weighted summation, feature value summation and feature maximum value;
s502, selecting an optimal fusion strategy as characteristic weighted summation, wherein the weight ratio of action characteristics to pattern characteristics is 4:6;
s503, training a preset support vector machine based on the fusion characteristics.
The method for classifying the dairy cows comprises the steps of carrying out feature fusion on action features and body pattern features of the dairy cows during walking, and classifying the dairy cows based on the fused features, wherein the method comprises the following steps: and carrying out feature fusion on the features in the two probability vector forms through three fusion methods of feature weighted summation, feature value summation and feature maximum value, selecting an optimal fusion strategy as the feature weighted summation, enabling the weight ratio of action features to pattern features to be 4:6, training an SVM support vector machine model by using fusion features, and carrying out individual identification on dairy cows.
According to the invention, the action features and the body pattern features of the dairy cows are fused, and the dairy cows are classified and identified based on the fused features. Fig. 12 (a) is a CMC curve for individual identification of cows using three feature fusion strategies, namely, feature value summation, feature maximum value and feature weighted summation, for motion feature, pattern feature and feature fusion, wherein the optimal fusion strategy is feature weighted summation, and the motion feature and pattern feature weight ratio is 4:6. The SVM model is trained using the fused probability vectors (i.e., fusion features) and individual identification is performed on the cows, the confusion matrix of which is shown in FIG. 12 (b).
Based on the trained support vector machine model, the method for identifying the dairy cow individuals in the dairy cow image acquired in real time specifically comprises the following steps:
s601, acquiring cow video data in real time, and inputting the video data into a trained support vector machine model;
s602, performing individual identification of the dairy cows through the support vector machine model.
By the multi-mode individual dairy cow identification method integrating the motion information, provided by the invention, video data of dairy cow walking are collected, a complete cow body is extracted, and the data are converted into two data forms of a shearing image and an RGB image; preprocessing such as cutting, aligning, normalizing and the like is carried out on cow data; dividing the two parts of data into two parts of a training set and a testing set respectively; constructing a deep learning network model to extract action features of the body of the dairy cow during walking and pattern features of the dairy cow body; and carrying out feature fusion on the two features, classifying the dairy cows by using a support vector machine model based on the fused features, and completing individual identification of the dairy cows. The accurate identification of the dairy cows is realized, and the individual dairy cows cannot be influenced.
Referring to fig. 8, the invention also discloses a multi-mode milk cow individual identification system integrating action information, which comprises:
the data acquisition module 110 is used for acquiring video data of dairy cows during activity and extracting complete cow body images of the dairy cows;
the image conversion module 120 is configured to convert the complete bovine body image extracted from the video data into a clip image and an RGB image;
the preprocessing module 130 is configured to preprocess the clip map and the RGB image to generate a data set, and divide the data set into a training set and a testing set;
the feature extraction module 140 is configured to construct a deep learning network model based on the training set and the test set, and perform action feature extraction and pattern feature extraction through the deep learning network model;
the feature fusion module 150 is configured to perform feature fusion on the motion feature and the pattern feature, and train a preset support vector machine based on the fusion feature;
and the identification module 160 is used for carrying out individual dairy cow identification on the dairy cow images acquired in real time based on the trained support vector machine model.
The data acquisition module is used for selecting video data acquisition when the dairy cows are milked to leave the milking parlor;
the camera is fixedly arranged on the inner side of the aisle and is set at a distance from the dairy cow, the focal length of the camera is adjusted to enable the whole camera to have a visual field which is about 3 times of the length of the body of the dairy cow, and the shooting range at least comprises 2 gait cycles;
and extracting the whole cow body of the dairy cow through the acquired video data of the walking of the dairy cow.
The image conversion module is used for preprocessing the video data of the walking dairy cows into video frames under the view angle of the side view;
dividing a complete cow body contour side view through a preset semantic dividing network model based on the video frame;
and converting the segmented cattle body into a shearing image and an RGB image by modifying the semantic segmentation network model parameters.
The preprocessing module is used for cutting the cut-off images and the RGB images, automatically detecting and cutting edge areas in the images through an image boundary detection algorithm, removing irrelevant or unnecessary parts in the images, and obtaining an interested area;
aligning based on the cut image data, finding key feature points in the image through a feature point detection and matching algorithm, and aligning through a transformation matrix to enable objects or features in the image to be in the same position;
normalization is performed based on the aligned image data, and a linear normalization method is used to scale the pixel values to between 0 and 1 for normalization processing.
The feature extraction module inputs the silhouette graph into a deep learning network model, performs feature extraction on a video frame sequence of the silhouette graph through a 3D convolution network and a two-way long-short-term memory network, and outputs action features in the form of probability vectors;
and inputting the RGB image into a deep learning network model, extracting deep features through a convolutional neural network, and outputting pattern features in the form of probability vectors through an output layer.
The feature fusion module is used for carrying out feature fusion on the features in the form of the two probability vectors of the action feature and the pattern feature through three fusion methods of feature weighted summation, feature value summation and feature maximum value;
selecting an optimal fusion strategy as characteristic weighted summation, wherein the weight ratio of the action characteristic to the pattern characteristic is 4:6;
and training a preset support vector machine based on the fusion characteristics.
The recognition module is used for acquiring the video data of the dairy cows in real time and inputting the video data into the trained support vector machine model;
and performing individual identification of the dairy cows through the support vector machine model.
The multi-mode individual dairy cow identification system integrating the action information provided by the invention is used for collecting video data of dairy cow walking, extracting a complete cow body and converting the video data into two data forms of a shearing image and an RGB image; preprocessing such as cutting, aligning, normalizing and the like is carried out on cow data; dividing the two parts of data into two parts of a training set and a testing set respectively; constructing a deep learning network model to extract action features of the body of the dairy cow during walking and pattern features of the dairy cow body; and carrying out feature fusion on the two features, classifying the dairy cows by using a support vector machine model based on the fused features, and completing individual identification of the dairy cows. The accurate identification of the dairy cows is realized, and the individual dairy cows cannot be influenced.
Fig. 13 illustrates a physical structure diagram of an electronic device, as shown in fig. 13, which may include: processor 1310, communication interface (Communications Interface) 1320, memory 1330 and communication bus 1340, wherein processor 1310, communication interface 1320, memory 1330 communicate with each other via communication bus 1340. Processor 1310 may invoke logic instructions in memory 1330 to perform a multi-modal individual identification method for dairy cows incorporating motion information, the method comprising: collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
converting the complete cow body image extracted from the video data into a clip image and an RGB image;
preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features;
and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
Further, the logic instructions in the memory 1330 can be implemented in the form of software functional units and can be stored in a computer readable storage medium when sold or used as a stand alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being storable on a non-transitory computer readable storage medium, the computer program, when executed by a processor, being capable of executing a multi-modal milk cow individual identification method incorporating motion information provided by the above methods, the method comprising: collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
converting the complete cow body image extracted from the video data into a clip image and an RGB image;
preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features;
and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform a multi-modal milk cow individual identification method incorporating motion information provided by the above methods, the method comprising: collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
converting the complete cow body image extracted from the video data into a clip image and an RGB image;
preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features;
and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (10)
1. The multi-mode milk cow individual identification method integrating action information is characterized by comprising the following steps of:
collecting video data of dairy cows during activity, and extracting complete cow body images of the dairy cows;
converting the complete cow body image extracted from the video data into a clip image and an RGB image;
preprocessing the cut-off map and the RGB picture to generate a data set, and dividing the data set into a training set and a testing set;
constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
feature fusion is carried out on the action features and the pattern features, and training is carried out on a preset support vector machine based on the fusion features;
and performing individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
2. The method for identifying multi-modal milk cow individuals incorporating motion information according to claim 1, wherein the capturing video data of milk cow activity, extracting complete cow body images of milk cows, comprises:
selecting to collect video data when the milking of the cows is completed to leave the milking parlor;
the camera is fixedly arranged on the inner side of the aisle and is set at a distance from the dairy cow, the focal length of the camera is adjusted to enable the whole camera to have a visual field which is about 3 times of the length of the body of the dairy cow, and the shooting range at least comprises 2 gait cycles;
and extracting the whole cow body of the dairy cow through the acquired video data of the walking of the dairy cow.
3. The method for identifying multi-modal dairy cow individuals incorporating motion information according to claim 1, wherein converting the complete cow body image extracted from the video data into a clip map and an RGB picture comprises:
preprocessing video data of walking cows into video frames under a side view angle;
dividing a complete cow body contour side view through a preset semantic dividing network model based on the video frame;
and converting the segmented cattle body into a shearing image and an RGB image by modifying the semantic segmentation network model parameters.
4. The method for identifying multi-modal dairy cow individuals incorporating motion information according to claim 1, wherein preprocessing the clip map and RGB pictures to generate a dataset specifically comprises:
cutting the clipping image and the RGB image, automatically detecting and cutting edge regions in the image through an image boundary detection algorithm, and removing irrelevant or unnecessary parts in the image to obtain an interested region;
aligning based on the cut image data, finding key feature points in the image through a feature point detection and matching algorithm, and aligning through a transformation matrix to enable objects or features in the image to be in the same position;
normalization is performed based on the aligned image data, and a linear normalization method is used to scale the pixel values to between 0 and 1 for normalization processing.
5. The multi-modal individual dairy cow identification method integrating action information according to claim 1, wherein a deep learning network model is constructed based on the training set and the test set, and action feature extraction and pattern feature extraction are performed through the deep learning network model, specifically comprising:
inputting the silhouette into a deep learning network model, carrying out feature extraction on a video frame sequence of the silhouette through a 3D convolution network and a two-way long-short-term memory network, and outputting action features in the form of probability vectors;
and inputting the RGB image into a deep learning network model, extracting deep features through a convolutional neural network, and outputting pattern features in the form of probability vectors through an output layer.
6. The method for identifying the individual multi-modal dairy cows with the integrated action information according to claim 1, wherein the action features and the pattern features are subjected to feature fusion, and a preset support vector machine is trained based on the fusion features, and the method specifically comprises the following steps:
feature fusion is carried out on the features in the form of the two probability vectors of the action feature and the pattern feature through three fusion methods of feature weighted summation, feature value summation and feature maximum value;
selecting an optimal fusion strategy as characteristic weighted summation, wherein the weight ratio of the action characteristic to the pattern characteristic is 4:6;
and training a preset support vector machine based on the fusion characteristics.
7. The multi-modal individual identification method for dairy cows incorporating motion information according to claim 1, wherein the training-based support vector machine model performs individual identification for dairy cows on real-time acquired dairy cow images, and specifically comprises:
acquiring cow video data in real time, and inputting the video data into a trained support vector machine model;
and performing individual identification of the dairy cows through the support vector machine model.
8. A multi-modal individual cow identification system incorporating motion information, the system comprising:
the data acquisition module is used for acquiring video data of dairy cows during activity and extracting complete cow body images of the dairy cows;
the image conversion module is used for converting the complete cow body image extracted from the video data into a clip image and an RGB image;
the preprocessing module is used for preprocessing the clip images and the RGB images to generate a data set, and dividing the data set into a training set and a testing set;
the feature extraction module is used for constructing a deep learning network model based on the training set and the testing set, and extracting action features and pattern features through the deep learning network model;
the feature fusion module is used for carrying out feature fusion on the action features and the pattern features, and training a preset support vector machine based on the fusion features;
and the identification module is used for carrying out individual dairy cow identification on the dairy cow image acquired in real time based on the trained support vector machine model.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor, when executing the computer program, implements the multi-modal individual cow identification method incorporating motion information as claimed in any one of claims 1 to 6.
10. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the multi-modal individual cow identification method incorporating motion information according to any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311397300.3A CN117456620A (en) | 2023-10-25 | 2023-10-25 | Multi-mode milk cow individual identification method and system integrating action information |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311397300.3A CN117456620A (en) | 2023-10-25 | 2023-10-25 | Multi-mode milk cow individual identification method and system integrating action information |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117456620A true CN117456620A (en) | 2024-01-26 |
Family
ID=89579264
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311397300.3A Pending CN117456620A (en) | 2023-10-25 | 2023-10-25 | Multi-mode milk cow individual identification method and system integrating action information |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117456620A (en) |
-
2023
- 2023-10-25 CN CN202311397300.3A patent/CN117456620A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10282589B2 (en) | Method and system for detection and classification of cells using convolutional neural networks | |
| CN110298291B (en) | Mask-RCNN-based cow face and cow face key point detection method | |
| Zin et al. | Image technology based cow identification system using deep learning | |
| EP3499414B1 (en) | Lightweight 3d vision camera with intelligent segmentation engine for machine vision and auto identification | |
| CN105260750B (en) | A kind of milk cow recognition methods and system | |
| CN112633297B (en) | Target object identification method and device, storage medium and electronic device | |
| CN111639629B (en) | Pig weight measurement method and device based on image processing and storage medium | |
| CN107766864B (en) | Method and device for extracting features and method and device for object recognition | |
| CN108491807B (en) | Real-time monitoring method and system for oestrus of dairy cows | |
| CN107767358B (en) | Method and device for determining ambiguity of object in image | |
| CN111539320B (en) | Multi-view gait recognition method and system based on mutual learning network strategy | |
| Jenifa et al. | Classification of cotton leaf disease using multi-support vector machine | |
| CN116543386A (en) | Agricultural pest image identification method based on convolutional neural network | |
| JP2023115104A (en) | Image processing apparatus, image processing method, and program | |
| CN111353429A (en) | Interest degree method and system based on eyeball turning | |
| Xiang et al. | Measuring stem diameter of sorghum plants in the field using a high-throughput stereo vision system | |
| Farhood et al. | Recent advances of image processing techniques in agriculture | |
| CN112257730A (en) | Plant pest image identification method, device, equipment and storage medium | |
| CN117456620A (en) | Multi-mode milk cow individual identification method and system integrating action information | |
| CN111079617B (en) | Poultry identification method and device, readable storage medium and electronic equipment | |
| CN110738225B (en) | Image recognition method and device | |
| CN109934045B (en) | Pedestrian detection method and device | |
| Murthi et al. | A semi-automated system for smart harvesting of tea leaves | |
| CN114255203B (en) | Fry quantity estimation method and system | |
| CN116758589B (en) | Cattle face recognition method for processing gesture and visual angle correction |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |