CN112734731B - Livestock temperature detection method, device, equipment and storage medium - Google Patents
Livestock temperature detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN112734731B CN112734731B CN202110033020.9A CN202110033020A CN112734731B CN 112734731 B CN112734731 B CN 112734731B CN 202110033020 A CN202110033020 A CN 202110033020A CN 112734731 B CN112734731 B CN 112734731B
- Authority
- CN
- China
- Prior art keywords
- livestock
- temperature
- infrared
- image
- super
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 244000144972 livestock Species 0.000 title claims abstract description 177
- 238000001514 detection method Methods 0.000 title claims abstract description 96
- 238000003860 storage Methods 0.000 title claims abstract description 13
- 238000000034 method Methods 0.000 claims abstract description 38
- 238000012545 processing Methods 0.000 claims abstract description 25
- 238000012549 training Methods 0.000 claims description 35
- 238000013135 deep learning Methods 0.000 claims description 17
- 239000011159 matrix material Substances 0.000 claims description 12
- 238000013527 convolutional neural network Methods 0.000 claims description 11
- 210000004013 groin Anatomy 0.000 claims description 11
- 238000013528 artificial neural network Methods 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004140 cleaning Methods 0.000 claims description 8
- 238000001914 filtration Methods 0.000 claims description 7
- 125000004122 cyclic group Chemical group 0.000 claims description 6
- 230000004927 fusion Effects 0.000 claims description 6
- 230000002452 interceptive effect Effects 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000005065 mining Methods 0.000 claims description 5
- 238000000605 extraction Methods 0.000 claims description 4
- 238000012544 monitoring process Methods 0.000 abstract description 9
- 230000036760 body temperature Effects 0.000 abstract description 8
- 201000010099 disease Diseases 0.000 abstract description 6
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 abstract description 6
- 241001465754 Metazoa Species 0.000 description 11
- 238000003062 neural network model Methods 0.000 description 11
- 230000011218 segmentation Effects 0.000 description 11
- 230000006870 function Effects 0.000 description 8
- 230000000306 recurrent effect Effects 0.000 description 8
- 206010037660 Pyrexia Diseases 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 4
- 230000001537 neural effect Effects 0.000 description 4
- 238000001931 thermography Methods 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000002159 abnormal effect Effects 0.000 description 3
- 238000005520 cutting process Methods 0.000 description 3
- 238000013136 deep learning model Methods 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 230000002776 aggregation Effects 0.000 description 2
- 238000004220 aggregation Methods 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000009529 body temperature measurement Methods 0.000 description 2
- 238000009395 breeding Methods 0.000 description 2
- 230000001488 breeding effect Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 239000002184 metal Substances 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 241000283690 Bos taurus Species 0.000 description 1
- 241001494479 Pecora Species 0.000 description 1
- 241000282887 Suidae Species 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000003745 diagnosis Methods 0.000 description 1
- 238000003708 edge detection Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4053—Scaling of whole images or parts thereof, e.g. expanding or contracting based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10048—Infrared image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/70—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in livestock or poultry
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Radiation Pyrometers (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
Abstract
The application discloses a livestock temperature detection method, which comprises the steps of acquiring an infrared photographing instruction, and acquiring infrared images of livestock by using an infrared photographing device; processing the infrared image by using a super-resolution model to obtain a super-resolution image; detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock; and detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value. The method can realize all-weather automatic patrol on the premise of ensuring the precision, provides important guarantee for monitoring the body temperature of livestock, reduces the calculated amount, reduces the hardware pressure, saves the cost and prevents the spread of livestock diseases. The invention also discloses a livestock temperature detection device, equipment and a storage medium.
Description
Technical Field
The invention belongs to the technical field of livestock breeding, and particularly relates to a method, a device, equipment and a storage medium for detecting temperature of livestock.
Background
The livestock breeding industry is labor intensive industry, in the prior art, when the temperature of livestock is required to be detected, one is in a manual mode, is time-consuming and labor-consuming, depends on personal quality of operators, is not easy to find out all livestock with abnormal temperature due to the influence of various factors, and the other is to detect the temperature of the livestock by an automatic technology, wherein the adopted target detection method comprises two types, namely the method is to acquire the target outline through the edge detection of the traditional opencv, but the method is difficult to adapt to the requirement of complex scene target detection, and the other is a deep learning method, mainly aims at pictures such as pedestrians, vehicles and the like, has lower accuracy and is not suitable for being applied to the field of livestock temperature detection.
Disclosure of Invention
In order to solve the problems, the invention provides a method, a device, equipment and a storage medium for detecting the temperature of livestock, which can realize all-weather automatic patrol on the premise of ensuring the precision, provide important guarantee for monitoring the body temperature of the livestock, reduce the calculated amount, lower the hardware pressure, save the cost and prevent the spread of livestock diseases.
The invention provides a livestock temperature detection method, which comprises the following steps:
acquiring an infrared photographing instruction, and acquiring an infrared image of livestock by using an infrared photographing device;
processing the infrared image by using a super-resolution model to obtain a super-resolution image;
detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock;
and detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value.
Preferably, in the above method for detecting temperature of livestock, before the detecting the target livestock in the super-resolution image, training a detection model is further included, including:
marking the infrared image by using a data marking tool to obtain an initial training set;
obtaining a diversified data set by utilizing a data enhancement mode;
training by using a deep learning target detection model to obtain an initial model;
reasoning the initial model, cleaning the data of the diversified data sets by utilizing a temperature matrix, judging Shan Zhangsuo whether an object in a temperature interval to be detected exists in the infrared image, and if so, reasoning by using the deep learning target detection model to obtain a prediction result BBox in the infrared image;
and filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result.
Preferably, in the method for detecting temperature of livestock, the acquiring an infrared image of the livestock by using the infrared photographing device is as follows:
infrared images of livestock are acquired by using a thermal infrared imager mounted on a car of a rail above a farm aisle.
Preferably, in the livestock temperature detection method, the data enhancement mode includes a mixup mode, a turnover mode, a translation mode, a random cutting mode, a random noise adding mode and a GAN mode.
Preferably, in the livestock temperature detection method, the prediction result BBox in the infrared image is obtained through a yolov4 model.
The invention provides a livestock temperature detection device, which comprises:
the infrared photographing component is used for acquiring an infrared photographing instruction and acquiring an infrared image of livestock;
the image processing component is used for processing the infrared image by utilizing the super-resolution model to obtain a super-resolution image;
a temperature detection key point coordinate determining unit for detecting a target livestock in the super-resolution image and detecting coordinates of a temperature detection key point of the target livestock;
and the alarm component is used for detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value.
Preferably, the livestock temperature detection device further comprises a detection model training component for marking the infrared image by using a data marking tool to obtain an initial training set; obtaining a diversified data set by utilizing a data enhancement mode; training by using a deep learning target detection model to obtain an initial model; reasoning the initial model, cleaning the data of the diversified data sets by utilizing a temperature matrix, judging Shan Zhangsuo whether an object in a temperature interval to be detected exists in the infrared image, and if so, reasoning by using the deep learning target detection model to obtain a prediction result BBox in the infrared image; and filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result.
Preferably, in the livestock temperature detection device, the infrared photographing component is specifically configured to collect infrared images of the livestock by using a thermal infrared imager installed on a car of a rail above a farm aisle.
The present invention provides a computer device comprising:
a memory for storing a computer program;
a processor for implementing the steps of any of the livestock temperature detection methods as described above when executing said computer program.
The present invention provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of any of the livestock temperature detection methods described above.
As can be seen from the above description, the method for detecting temperature of livestock provided by the present invention includes acquiring an infrared photographing instruction, and acquiring an infrared image of livestock by using an infrared photographing device; then processing the infrared image by using a super-resolution model to obtain a super-resolution image; detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock; and finally, detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value, wherein the method can realize all-weather automatic inspection on the premise of ensuring the accuracy without manual participation, thereby providing important guarantee for monitoring the body temperature of livestock, reducing the calculated amount, lowering the hardware pressure, saving the cost and preventing the spread of livestock diseases. The livestock temperature detection device, the equipment and the storage medium provided by the invention have the same advantages as the method.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of an embodiment of a method for detecting temperature of livestock according to the present invention;
fig. 2 is a schematic diagram of an embodiment of a livestock temperature detecting device provided by the invention;
fig. 3 is a schematic diagram of an embodiment of a computer device according to the present invention.
Detailed Description
The core of the invention is to provide a method, a device, equipment and a storage medium for detecting the temperature of livestock, which can realize all-weather automatic patrol on the premise of ensuring the precision, provide important guarantee for the temperature monitoring of livestock, reduce the calculated amount, lower the hardware pressure, save the cost and prevent the spread of livestock diseases.
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
An embodiment of a method for detecting temperature of livestock provided by the present invention is shown in fig. 1, and fig. 1 is a schematic diagram of an embodiment of a method for detecting temperature of livestock provided by the present invention, where the method includes the following steps:
s1: acquiring an infrared photographing instruction, and acquiring an infrared image of livestock by using an infrared photographing device;
specifically, the infrared photographing device can be, but not limited to, a thermal infrared imager, which converts invisible infrared energy emitted by an object into a visible thermal image, so that any object with a temperature higher than 0 DEG can be imaged, and the temperature of livestock is obviously higher than the ambient temperature, so that the state of the livestock can be observed in all weather without being influenced by light, and body temperature monitoring can be conveniently performed after the livestock target is captured by combining a target detection technology. Moreover, the infrared image of livestock can be acquired by using the infrared thermal imager arranged on the rail car above the farm passageway, and the existing mode of holding a temperature measuring gun and observing the temperature of human eyes through the pictures of the thermal imager is not adopted. In the method, an RFID card can be attached to the right center of each column for realizing positioning, the inspection trolley is utilized for fixed-point parking, infrared thermal imaging devices on the trolley are combined, infrared thermal imaging data of livestock are collected, primary cleaning is carried out on the collected data, data with normal angles and livestock in the columns are selected as an initial sample set, whether the maximum temperature value in a temperature matrix exceeds a set threshold value, for example, the current threshold value is 45 ℃, the data exceeding 45 ℃ is judged to be abnormal data, and the data do not participate in reasoning of a subsequent model.
S2: processing the infrared image by using a super-resolution model to obtain a super-resolution image;
it should be noted that, the single-picture super-resolution (SISR) algorithm used in the step is integrated into a cyclic neural network through a cross-scale non-local (CS-NL) attention module, and a final super-resolution image is obtained by combining local CS-NL prior and non-local prior to a powerful cyclic aggregation unit, and imaging details are captured by using a super-resolution enhancement technology, so that the accuracy of model reasoning can be greatly improved, and meanwhile, the cost of equipment is reduced. Specifically, the infrared livestock picture is to extract image features through a convolutional neural network, then enter a series of recurrent neural units, the recurrent neural units form a recurrent network, and finally, the information of the recurrent units is fused through the convolutional network to generate a high-resolution picture. For each recursion unit, the features of the previous layer pass through three branches, namely a cross-scale non-local attention branch, a full-scale non-local attention branch and a local branch, and all features are extracted by the last three branches through interactive mapping fusion and used as the input of the next layer, and the method specifically comprises the following operations: extracting characteristics of the infrared livestock pictures through a convolutional neural network; taking the features of the last step as input, carrying out feature mining through an SEM unit (Self-samples Minig) and processing the fused information through a Stride convolution layer, a Deconv layer network and the like, wherein the feature mining comprises the steps of extracting features through a cross-scale non-local attention network, extracting features through an inline scale non-local attention network, extracting features through a local information extraction network, carrying out interactive mapping fusion on the features obtained in the three steps; repeating a plurality of SEM units to form a circulating neural network; combining the information mined by the plurality of SEM units through a network Concat, and obtaining a final high-resolution infrared livestock picture through a convolutional neural network. Taking an example of an infrared camera with 256 x 192 of resolution of a certain farm, the image ambiguity is too high, particularly, temperature measurement key parts such as the edge part and the head part of livestock are difficult to accurately distinguish and clear, through the super-resolution model processing, an infrared livestock image with high definition and 512 x 384 of resolution is obtained, the step can accurately detect the target based on the infrared livestock image, and the example segmentation can provide great help, so that the detection of the livestock dying of illness is facilitated.
S3: detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock;
in the steps, data acquired by using the thermal infrared imager are required to be processed, gray image data directly corresponding to the temperature matrix is encoded into an intuitive infrared heat map, then enhancement processing is carried out on the infrared heat map, a large number of infrared images can be obtained, the data are used as a training data set, a neural network model for livestock example segmentation is obtained through training, finally livestock in the infrared images can be segmented through the trained model, and all the livestock are monitored by combining the temperature. Specifically, gray map data corresponding to a temperature matrix in the collected infrared data are selected, gamma transformation and local enhancement of a histogram are carried out on the gray map data according to the contrast of the gray map data, the gamma transformation can stretch the contrast of a highlight region, so that the texture of livestock with higher temperature is stronger, the local enhancement of the histogram can press and darken the highlight region with lower contrast, so that the highlight region of an image is softer, details are clearer, and then a heat metal code is used for coding the processed gray map into an infrared heat map conforming to visual habit; constructing a training data set; collecting the collected infrared data, performing data processing in the last step, selecting enough images with different livestock distribution, marking the images, marking the outline of each livestock in the infrared images, connecting the outlines, and marking all pixels corresponding to the same livestock area in the images as the same ID, wherein the IDs corresponding to different livestock are different; training a neural network model of livestock instance segmentation, building a neural network model suitable for instance segmentation, setting a proper loss function, inputting a training data set, and continuously and iteratively updating weights in the network model through counter propagation to finally obtain the neural network model of livestock instance segmentation. The method directly detects the key points of the auricles and the groins on the infrared heat map of the livestock, reduces the requirement on hardware resources, reduces the complexity of the processing flow and improves the efficiency.
S4: and detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value.
It should be noted that the preset threshold may be, but not limited to, 40 degrees celsius, when 40 degrees celsius is taken as a basis for determining fever, and when the temperature value at the detection key point exceeds 40 degrees celsius, it is determined that the livestock is fever, or in hot summer, whether the temperature at the detection key point of one livestock is higher than the temperature of the detection key points of other livestock by one value, for example, 1.8 degrees celsius, and when the livestock meeting the above requirement is determined to be fever, the situation that the absolute temperature of the key points is inaccurate due to the influence of environmental temperature factors is relieved, and the livestock with the temperature value lower than the set threshold is determined to be dead, and an alarm is required when the situation occurs, and the livestock with fever obtained by prediction can be pushed to each unit, and the breeder can diagnose and process in time in combination with the pushing result.
It can be seen that the above embodiment obtains a more intuitive infrared image by encoding and enhancing image data directly related to temperature, and then performs example segmentation on livestock to obtain a temperature region of each livestock, thereby performing body temperature monitoring on multiple livestock. The livestock in the above scheme may be, but is not limited to, pigs, cattle and sheep, and any similar livestock may be treated by the above method.
As can be seen from the above description, in the embodiment of the method for detecting the temperature of livestock provided by the present invention, the method includes acquiring an infrared photographing instruction, and acquiring an infrared image of livestock by using an infrared photographing device; then processing the infrared image by using a super-resolution model to obtain a super-resolution image; detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock; finally, the temperature of the temperature detection key point is detected, and an alarm is given when the temperature is higher than a preset threshold value, so that the method does not need manual participation, all-weather automatic inspection can be realized on the premise of ensuring the precision, important guarantee is provided for monitoring the body temperature of livestock, the calculated amount can be reduced, the hardware pressure is reduced, the cost is saved, and the spread of livestock diseases is prevented.
It should be further noted that, for the infrared image of the livestock acquired by the thermal infrared imager, the model training is performed by using a convolutional neural network deep learning method to obtain a deep learning model capable of directly extracting key points such as groin and auricle of the livestock from the infrared image of the livestock, so in a specific embodiment of the above livestock temperature detection method, before detecting the target livestock in the super-resolution image, the method further comprises training the detection model, and further includes:
the method comprises the steps of marking an infrared image by using a data marking tool to obtain an initial training set, specifically, manually marking, wherein a certain amount of diversified samples are needed for training a key point detection model based on deep learning, and the frame of livestock, key points such as groin and auricle and the like are marked on the infrared image manually to be used as 'knowledge' for deep neural network model learning;
obtaining a diverse data set using a data enhancement approach, including but not limited to augmenting the data set using a mixup approach;
training by using a deep learning target detection model to obtain an initial model;
reasoning the initial model, carrying out data cleaning on the diversified data sets by utilizing a temperature matrix, judging whether an object in a temperature interval to be detected exists in the image outside Shan Zhanggong, and if so, carrying out reasoning by using a deep learning target detection model to obtain a prediction result BBox in the infrared image;
and filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result, namely, deducing and obtaining coordinate values of key points of the auricles and the groins of the livestock.
Specifically, a data set trained by a deep learning model needs to be established in advance, a diversified sample is provided for the deep neural network, the deep neural network model is trained, and according to a file marked manually, all livestock in a picture are respectively extracted through operation steps of cutting, adjusting the size, standardizing and the like, and the size is set to be fixed; converting the coordinates of the auricle and inguinal key points of the marked livestock into a 64x64 real hematmap through a Gaussian kernel; extracting features through a simple baseline neural network to obtain a predicted hetmap; calculating a value of the L2 loss function using the real and predicted hetmap; continuously updating the model weight in an error back propagation mode until the model converges or the iteration termination condition is met; the model application effect evaluation and optimization show that the phenomenon that the back edge is misjudged to be the groin when livestock stands can be found through practical application, the loss function can be improved, the loss value generated by detecting the invisible key points is newly added in the original loss function, the model is retrained, the model detection effect is improved, and the detection precision is enhanced. After training of the infrared heat map livestock key point detection model based on deep learning is completed, 1 new infrared heat map of livestock is provided as input of the model, and coordinate information of key points such as auricles, groins and the like of all livestock in the picture can be obtained after target detection and key point detection model processing.
In another specific embodiment of the above method for detecting temperature of livestock, the data enhancement mode may include a mix up mode, a flip mode, a shift mode, a random clipping mode, a random noise adding mode, and a GAN mode, that is, at least one mode may be used to expand the data set to obtain a sample set for model training.
Further, the prediction result BBox in the infrared image can be obtained through a yolov4 model. Specifically, during model training, pictures resize in a training set are set to the same size, real BBox position information of the pictures is obtained, predicted BBox information is obtained through a yolov4 model, the real BBox position information is compared with the predicted BBox information, the sum of classification loss, confidence loss, location loss and iou loss is used as final loss, a back propagation algorithm is used for continuously updating weights until the model converges or iteration termination conditions are met, then model reasoning and post-processing are carried out, specifically, in an on-line reasoning stage, a result with the maximum confidence coefficient is firstly obtained through multiple reasoning by combining with a TTA method to serve as an output result of target detection, then a threshold value with the minimum BBox is set, the reasoning result is filtered when the threshold value of the BBox is smaller than a given threshold value, and when the threshold value of the BBox is larger than the given threshold value, the infrared heat map livestock target detection result is output.
An embodiment of an apparatus for detecting temperature of livestock provided by the present invention is shown in fig. 2, and fig. 2 is a schematic diagram of an embodiment of an apparatus for detecting temperature of livestock provided by the present invention, the apparatus includes:
the infrared photographing component 201 is configured to acquire an infrared photographing instruction, collect an infrared image of an animal, and specifically, the infrared photographing component may be, but not limited to, an infrared thermal imager, where the infrared thermal imager converts invisible infrared energy emitted by an object into a visible thermal image, so that any object with a temperature higher than 0 ° can be imaged, and the temperature of the animal is significantly higher than the ambient temperature, so that the state of the animal can be observed in all weather without being affected by light, and body temperature monitoring can be conveniently performed after capturing an animal target in combination with a target detection technology. Moreover, the infrared image of livestock can be acquired by using the infrared thermal imager arranged on the rail car above the farm passageway, and the existing mode of holding a temperature measuring gun and observing the temperature of human eyes through the pictures of the thermal imager is not adopted. In the method, an RFID card can be stuck at the right center of each column for realizing positioning, a patrol trolley is utilized for fixed-point parking, infrared thermal imaging devices on the trolley are combined, infrared thermal imaging data of livestock are collected, primary cleaning is carried out on the collected data, data with normal angles and livestock in the columns are selected as an initial sample set, whether the maximum temperature value in a temperature matrix exceeds a set threshold value, for example, the current threshold value is 45 ℃, and the data exceeding 45 ℃ are judged to be abnormal data and do not participate in reasoning of a subsequent model;
the image processing unit 202 is configured to process the infrared image by using a super-resolution model to obtain a super-resolution image, integrate the super-resolution (SISR) algorithm of a single picture into a cyclic neural network through a cross-scale non-local (CS-NL) attention module, and obtain a final super-resolution image by combining local CS-NL prior and non-local prior to a powerful cyclic aggregation unit, and capture imaging details by using a super-resolution enhancement technology, so that the accuracy of model reasoning can be greatly improved, and the cost of equipment is reduced. Specifically, the infrared livestock picture is to extract image features through a convolutional neural network, then enter a series of recurrent neural units, the recurrent neural units form a recurrent network, and finally, the information of the recurrent units is fused through the convolutional network to generate a high-resolution picture. For each recursion unit, the features of the previous layer pass through three branches, namely a cross-scale non-local attention branch, a full-scale non-local attention branch and a local branch, and all features are extracted by the last three branches through interactive mapping fusion and used as the input of the next layer, and the method specifically comprises the following operations: extracting characteristics of the infrared livestock pictures through a convolutional neural network; taking the features of the last step as input, carrying out feature mining through an SEM unit (Self-samples Minig) and processing the fused information through a Stride convolution layer, a Deconv layer network and the like, wherein the feature mining comprises the steps of extracting features through a cross-scale non-local attention network, extracting features through an inline scale non-local attention network, extracting features through a local information extraction network, carrying out interactive mapping fusion on the features obtained in the three steps; repeating a plurality of SEM units to form a circulating neural network; combining the information mined by the plurality of SEM units through a network Concat, and obtaining a final high-resolution infrared livestock picture through a convolutional neural network. Taking an infrared camera with 256 x 192 resolution of a certain farm as an example, the image has high ambiguity, particularly, the edge part, the head part and other temperature measurement key parts of the livestock are difficult to accurately distinguish and clear, the infrared livestock image with high definition and 512 x 384 resolution is obtained through the super-resolution model processing, the accurate target detection based on the infrared livestock image can be realized, and the example segmentation can provide great help, so that the detection of the livestock dying of illness is facilitated;
the temperature detection key point coordinate determining unit 203 is configured to detect a target livestock in the super-resolution image, detect coordinates of a temperature detection key point of the target livestock, use data acquired by the thermal infrared imager, encode gray image data directly corresponding to the temperature matrix into an intuitive infrared heat map, and then perform enhancement processing on the infrared heat map to obtain a large number of infrared images, train the data as a training dataset to obtain a neural network model for livestock instance segmentation, and finally segment livestock in the infrared image through the trained model, and monitor all the livestock in combination with temperature. Specifically, gray map data corresponding to a temperature matrix in the collected infrared data are selected, gamma transformation and local enhancement of a histogram are carried out on the gray map data according to the contrast of the gray map data, the gamma transformation can stretch the contrast of a highlight region, so that the texture of livestock with higher temperature is stronger, the local enhancement of the histogram can press and darken the highlight region with lower contrast, so that the highlight region of an image is softer, details are clearer, and then a heat metal code is used for coding the processed gray map into an infrared heat map conforming to visual habit; constructing a training data set; collecting the collected infrared data, performing data processing in the last step, selecting enough images with different livestock distribution, marking the images, marking the outline of each livestock in the infrared images, connecting the outlines, and marking all pixels corresponding to the same livestock area in the images as the same ID, wherein the IDs corresponding to different livestock are different; training a neural network model of livestock instance segmentation, building a neural network model suitable for instance segmentation, setting a proper loss function, inputting a training data set, and continuously and iteratively updating weights in the network model through counter propagation to finally obtain the neural network model of livestock instance segmentation. The method directly detects the key points of the auricles and the groins on the infrared heat map of the livestock, reduces the requirement on hardware resources, reduces the complexity of the processing flow and improves the efficiency;
the alarm component 204 is configured to detect the temperature of the temperature detection key point, and alarm when the temperature is higher than a preset threshold, where the preset threshold may be, but not limited to, 40 degrees celsius, when the temperature value at the detection key point exceeds 40 degrees celsius when 40 degrees celsius is taken as a basis for determining fever, it may be determined whether the temperature at the detection key point of one of the animals is higher than the temperature at the detection key point of the other animal by a value, for example, 1.8 degrees celsius in hot summer, when it is determined that the animal meeting the above requirement is a fever animal, the situation that the absolute temperature of the key point is inaccurate due to the influence of environmental temperature factors is alleviated, and when the animal with the temperature value lower than the preset threshold is determined to be a dead animal, the alarm needs to be performed, and the predicted fever animal may be pushed to each unit, and the breeder may perform timely diagnosis and treatment in combination with the pushing result.
In a specific embodiment of the above livestock temperature detection device, the device may further include a detection model training component, configured to label the infrared image with a data labeling tool, so as to obtain an initial training set; obtaining a diversified data set by utilizing a data enhancement mode; training by using a deep learning target detection model to obtain an initial model; reasoning the initial model, carrying out data cleaning on the diversified data sets by utilizing a temperature matrix, judging whether an object in a temperature interval to be detected exists in the image outside Shan Zhanggong, and if so, carrying out reasoning by using a deep learning target detection model to obtain a prediction result BBox in the infrared image; and filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result.
Specifically, a data set trained by a deep learning model needs to be established in advance, a diversified sample is provided for the deep neural network, the deep neural network model is trained, and according to a file marked manually, all livestock in a picture are respectively extracted through operation steps of cutting, adjusting the size, standardizing and the like, and the size is set to be fixed; converting the coordinates of the auricle and inguinal key points of the marked livestock into a 64x64 real hematmap through a Gaussian kernel; extracting features through a simple baseline neural network to obtain a predicted hetmap; calculating a value of the L2 loss function using the real and predicted hetmap; continuously updating the model weight in an error back propagation mode until the model converges or the iteration termination condition is met; the model application effect evaluation and optimization show that the phenomenon that the back edge is misjudged to be the groin when livestock stands can be found through practical application, the loss function can be improved, the loss value generated by detecting the invisible key points is newly added in the original loss function, the model is retrained, the model detection effect is improved, and the detection precision is enhanced. After training of the infrared heat map livestock key point detection model based on deep learning is completed, 1 new infrared heat map of livestock is provided as input of the model, and coordinate information of key points such as auricles, groins and the like of all livestock in the picture can be obtained after target detection and key point detection model processing.
Fig. 3 is a schematic diagram of an embodiment of a computer device according to the present invention, where the computer device includes:
a memory 301 for storing a computer program;
a processor 302 for implementing the steps of any of the livestock temperature detection methods described above when executing a computer program.
In an embodiment of the computer readable storage medium provided by the invention, a computer program is stored on the computer readable storage medium, and the computer program realizes the steps of any livestock temperature detection method when being executed by a processor.
The livestock temperature detection device, the equipment and the storage medium provided by the invention can realize all-weather automatic patrol under the premise of ensuring the precision, provide important guarantee for monitoring the livestock body temperature, reduce the calculated amount, lower the hardware pressure, save the cost and prevent the spread of livestock diseases.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (8)
1. A method for detecting temperature of livestock, comprising:
acquiring an infrared photographing instruction, and acquiring an infrared image of livestock by using an infrared photographing device;
processing the infrared image by using a super-resolution model to obtain a super-resolution image;
detecting target livestock in the super-resolution image, and detecting coordinates of temperature detection key points of the target livestock;
detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value;
the infrared image is processed by utilizing a super-resolution model, the super-resolution image is obtained, the characteristics of the infrared image of livestock are extracted through a convolutional neural network, the characteristics are taken as input, the characteristics are mined through SEM units, the characteristics are extracted through a cross-scale non-local attention network, the characteristics are extracted through an inline scale non-local attention network, the characteristics are extracted through a local information extraction network, the characteristics obtained in the three steps are subjected to interactive mapping fusion, the fused information is processed through a Stride convolutional layer and a Dev layer network, and a plurality of SEM units are repeated to form a cyclic neural network; combining the information mined by the plurality of SEM units through a network Concat, and obtaining a final super-resolution image through a convolutional neural network;
before the target livestock in the super-resolution image is detected, training a detection model is further included, and the method comprises the following steps:
marking the infrared image by using a data marking tool to obtain an initial training set;
obtaining a diversified data set by utilizing a data enhancement mode;
training by using a deep learning target detection model to obtain an initial model;
reasoning the initial model, cleaning the data of the diversified data sets by utilizing a temperature matrix, judging Shan Zhangsuo whether an object in a temperature interval to be detected exists in the infrared image, and if so, reasoning by using the deep learning target detection model to obtain a prediction result BBox in the infrared image;
filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result;
and filtering out a part of targets by setting a threshold value of BBox, wherein the final livestock target detection result is obtained by the following steps: and obtaining the coordinate information of the auricles and groin of each livestock in the picture.
2. The method for detecting the temperature of livestock according to claim 1, wherein the step of acquiring the infrared image of the livestock by using the infrared photographing device is:
infrared images of livestock are acquired by using a thermal infrared imager mounted on a car of a rail above a farm aisle.
3. The method of claim 1, wherein the data enhancement mode includes a mix up, flip, pan, random cut, add random noise, GAN mode.
4. The livestock temperature detection method of claim 1, wherein the prediction result BBox in the infrared image is obtained by a yolov4 model.
5. A livestock temperature detection device, comprising:
the infrared photographing component is used for acquiring an infrared photographing instruction and acquiring an infrared image of livestock;
the image processing component is used for processing the infrared image by utilizing the super-resolution model to obtain a super-resolution image;
a temperature detection key point coordinate determining unit for detecting a target livestock in the super-resolution image and detecting coordinates of a temperature detection key point of the target livestock;
the alarm component is used for detecting the temperature of the temperature detection key point, and alarming when the temperature is higher than a preset threshold value;
the image processing component is specifically used for extracting the characteristics of an infrared image of livestock through a convolutional neural network, taking the characteristics as input, carrying out characteristic mining through SEM units, and comprises the steps of extracting the characteristics through a cross-scale non-local attention network, extracting the characteristics through an inline scale non-local attention network, extracting the characteristics through a local information extraction network, carrying out interactive mapping fusion on the characteristics obtained in the three steps, processing the fused information through a Stride convolutional layer and a Deconv layer network, and repeating a plurality of SEM units to form the cyclic neural network; combining the information mined by the plurality of SEM units through a network Concat, and obtaining a final super-resolution image through a convolutional neural network;
the detection model training component is used for marking the infrared image by using a data marking tool to obtain an initial training set; obtaining a diversified data set by utilizing a data enhancement mode; training by using a deep learning target detection model to obtain an initial model; reasoning the initial model, cleaning the data of the diversified data sets by utilizing a temperature matrix, judging Shan Zhangsuo whether an object in a temperature interval to be detected exists in the infrared image, and if so, reasoning by using the deep learning target detection model to obtain a prediction result BBox in the infrared image; filtering out a part of targets by setting a threshold value of BBox to obtain a final livestock target detection result;
the detection model training component is used for obtaining the coordinate information of the auricles and the groins of the livestock in the pictures.
6. The livestock temperature detection device of claim 5, wherein the infrared photographing means is specifically configured to collect infrared images of the livestock using a thermal infrared imager mounted on a car of a rail above the farm aisle.
7. A computer device, comprising:
a memory for storing a computer program;
a processor for carrying out the steps of the method for detecting the temperature of livestock as claimed in any of claims 1 to 4 when executing said computer program.
8. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, implements the steps of the livestock temperature detection method according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110033020.9A CN112734731B (en) | 2021-01-11 | 2021-01-11 | Livestock temperature detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110033020.9A CN112734731B (en) | 2021-01-11 | 2021-01-11 | Livestock temperature detection method, device, equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112734731A CN112734731A (en) | 2021-04-30 |
| CN112734731B true CN112734731B (en) | 2023-07-28 |
Family
ID=75590249
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110033020.9A Active CN112734731B (en) | 2021-01-11 | 2021-01-11 | Livestock temperature detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112734731B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113487486A (en) * | 2021-07-23 | 2021-10-08 | 河南牧原智能科技有限公司 | Image resolution enhancement processing method, device and equipment and readable storage medium |
| CN113610871A (en) * | 2021-08-11 | 2021-11-05 | 河南牧原智能科技有限公司 | Individual segmentation method and system based on infrared imaging |
| CN113678751B (en) * | 2021-08-19 | 2022-08-02 | 安徽大学 | Intelligent passageway device for beef cattle body shape parameter acquisition and exercise health recognition |
| CN114200967B (en) * | 2021-11-29 | 2024-09-20 | 河南牧原智能科技有限公司 | Method for determining preset angle, method for detecting livestock body temperature and product |
| CN114264378A (en) * | 2021-12-20 | 2022-04-01 | 红原县雪月天佑农牧科技有限公司 | Intelligent livestock management system and method |
| CN114885853B (en) * | 2022-05-10 | 2023-11-24 | 河南讯飞智元信息科技有限公司 | Livestock body temperature monitoring method, device, electronic equipment and storage medium |
| CN114743224B (en) * | 2022-06-13 | 2022-09-09 | 金乡县康华乳业有限公司 | Animal husbandry livestock body temperature monitoring method and system based on computer vision |
| CN115342937B (en) * | 2022-10-20 | 2023-05-05 | 正大农业科学研究有限公司 | Temperature anomaly detection method and device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180051785A (en) * | 2016-11-09 | 2018-05-17 | 건국대학교 산학협력단 | Livestock monitoring system |
| CN110200598A (en) * | 2019-06-12 | 2019-09-06 | 天津大学 | A kind of large-scale plant that raises sign exception birds detection system and detection method |
| WO2019200735A1 (en) * | 2018-04-17 | 2019-10-24 | 平安科技（深圳）有限公司 | Livestock feature vector acquisition method, apparatus, computer device and storage medium |
| CN112085089A (en) * | 2020-09-03 | 2020-12-15 | 国网浙江省电力有限公司电力科学研究院 | Intelligent temperature measurement method for transformer substation equipment based on deep learning algorithm |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111192200A (en) * | 2020-01-02 | 2020-05-22 | 南京邮电大学 | Image super-resolution reconstruction method based on fusion attention mechanism residual error network |
-
2021
- 2021-01-11 CN CN202110033020.9A patent/CN112734731B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20180051785A (en) * | 2016-11-09 | 2018-05-17 | 건국대학교 산학협력단 | Livestock monitoring system |
| WO2019200735A1 (en) * | 2018-04-17 | 2019-10-24 | 平安科技（深圳）有限公司 | Livestock feature vector acquisition method, apparatus, computer device and storage medium |
| CN110200598A (en) * | 2019-06-12 | 2019-09-06 | 天津大学 | A kind of large-scale plant that raises sign exception birds detection system and detection method |
| CN112085089A (en) * | 2020-09-03 | 2020-12-15 | 国网浙江省电力有限公司电力科学研究院 | Intelligent temperature measurement method for transformer substation equipment based on deep learning algorithm |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112734731A (en) | 2021-04-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112734731B (en) | Livestock temperature detection method, device, equipment and storage medium | |
| CN107895367B (en) | Bone age identification method and system and electronic equipment | |
| WO2021047232A1 (en) | Interaction behavior recognition method, apparatus, computer device, and storage medium | |
| CN111046880B (en) | Infrared target image segmentation method, system, electronic equipment and storage medium | |
| CN108492319B (en) | Moving target detection method based on deep full convolution neural network | |
| Yu et al. | Railway obstacle detection algorithm using neural network | |
| CN109472191B (en) | Pedestrian re-identification and tracking method based on space-time context | |
| CN111723773B (en) | Method and device for detecting carryover, electronic equipment and readable storage medium | |
| CN108318773B (en) | Transmission conductor strand breakage detection method and system | |
| Noe et al. | Automatic detection and tracking of mounting behavior in cattle using a deep learning-based instance segmentation model | |
| CN113139489A (en) | Crowd counting method and system based on background extraction and multi-scale fusion network | |
| Li et al. | Temporal aggregation network using micromotion features for early lameness recognition in dairy cows | |
| CN117893870B (en) | Animal husbandry and veterinary animal body temperature detection system based on IR thermal imaging | |
| Yang et al. | A defencing algorithm based on deep learning improves the detection accuracy of caged chickens | |
| Shao et al. | An end-to-end food portion estimation framework based on shape reconstruction from monocular image | |
| Skladchykov et al. | Application of YOLOX deep learning model for automated object detection on thermograms | |
| CN115661717A (en) | Livestock crawling behavior marking method and device, electronic equipment and storage medium | |
| CN114255203B (en) | Fry quantity estimation method and system | |
| CN108764311A (en) | A kind of shelter target detection method, electronic equipment, storage medium and system | |
| CN114022831A (en) | Binocular vision-based livestock body condition monitoring method and system | |
| Tian et al. | Real-time vehicle detection under complex road conditions | |
| CN113111847A (en) | Automatic monitoring method, device and system for process circulation | |
| CN112307916A (en) | Alarm monitoring method based on visible light camera | |
| Wang et al. | Deep learning method for rapidly estimating pig body size | |
| CN113239945B (en) | Infrared weak and small target detection method utilizing trajectory constraint |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |