CN113920138A - Cow body size detection device based on RGB-D camera and detection method thereof - Google Patents

Cow body size detection device based on RGB-D camera and detection method thereof Download PDF

Info

Publication number
CN113920138A
CN113920138A CN202111211000.2A CN202111211000A CN113920138A CN 113920138 A CN113920138 A CN 113920138A CN 202111211000 A CN202111211000 A CN 202111211000A CN 113920138 A CN113920138 A CN 113920138A
Authority
CN
China
Prior art keywords
cow
rgb
camera
image
point cloud
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111211000.2A
Other languages
Chinese (zh)
Inventor
张红涛
谭联
王宇
张震
闫跃飞
李忠洋
邴丕彬
顾波
刘新宇
翟浩然
张雨鑫
罗一铭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan Cow Performance Test Co ltd
North China University of Water Resources and Electric Power
Original Assignee
Henan Cow Performance Test Co ltd
North China University of Water Resources and Electric Power
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan Cow Performance Test Co ltd, North China University of Water Resources and Electric Power filed Critical Henan Cow Performance Test Co ltd
Priority to CN202111211000.2A priority Critical patent/CN113920138A/en
Publication of CN113920138A publication Critical patent/CN113920138A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a cow body ruler detection device based on an RGB-D camera and a detection method thereof, belonging to the technical field of livestock informatization, and comprising the following steps: a cow image acquisition platform is built in an experimental field to obtain a target image; processing the RGB image to realize the identification of the dairy cow; processing the RGB image and the point cloud image to realize registration fusion of the RGB and the point cloud; and processing the fused point cloud image in the steps to realize the automatic detection of the milk cow body size. According to the invention, a milk cow image acquisition platform of the RGB-D camera is constructed, and non-contact automatic detection of the milk cow body size is realized based on image processing and mode identification. The degree of automation of detection is improved, the workload of manual detection is greatly relieved, and the problem that the body size of the dairy cow corresponds to the identity accurately is solved.

Description

Cow body size detection device based on RGB-D camera and detection method thereof
Technical Field
The invention belongs to the technical field of livestock informatization, and particularly relates to a cow body size detection device and method based on an RGB-D camera.
Background
China is a developing agricultural kingdom, and animal husbandry is an important component of modern agriculture and plays an important role in promoting the adjustment of agricultural industry structure and improving the quality of life of urban and rural residents. The dairy cow breeding industry is rapidly developed in recent years, and the traditional free-ranging mode is gradually transited to a large-scale intensive breeding mode. The method mainly depends on manpower to obtain various body size data of the dairy cows in the traditional breeding mode, needs a large amount of time and manpower input, can not meet the development requirement of the modern dairy cow breeding industry far away, and needs to depend on accurate breeding supported by an information technology, so that the automation of dairy cow breeding is the development trend of modern large-scale dairy cow breeding.
The automatic detection of the body size of the dairy cattle is the inevitable development direction of the modern technology of dairy cattle breeding. In the process of breeding the dairy cows, the body size parameters of the dairy cows can reflect the growth and development conditions of the dairy cows and can also reflect various performance indexes of the reproductive capacity, the milk production capacity and the like of the dairy cows.
At present, the method for measuring the body size parameters of the dairy cows comprises contact measurement and non-contact measurement. In the traditional contact measurement, professionals measure the body size characters of the dairy cows by using tools such as tape rulers, measuring sticks, circular touch detectors and the like, and the method has the defects of high workload, time and labor waste and even the situation that the dairy cows attack people; in addition, the measurement result is greatly deviated due to the difference of the measurement experience of workers. The non-contact measurement usually mainly uses a common monocular camera to detect the body size of the cow, the measurement method is limited by external illumination, environment and cow pose, the curved surface data of the body shape of the cow are not easy to obtain, and the measurable body size indexes are few.
In recent years, with the rapid development of computers, electronic industries and stereoscopic vision technologies, the body size of the dairy cows is detected based on the stereoscopic vision technology in non-contact measurement, and compared with manual detection, the measuring method has higher efficiency, can avoid the influence of subjective factors in the measuring process, and can avoid the stress reaction of the dairy cows. In the non-contact body size measurement of the milk cow, the situations of the dropping of an ear tag of the milk cow, the swinging of the head of the milk cow and the like often occur, so that the defects of failure in identity identification, inaccurate identification and the like of the milk cow are caused. With the rapid development of computer vision technology, the related scholars propose an automatic cow identification mode based on an image algorithm, but the identification accuracy is still at a low level.
Disclosure of Invention
The invention aims to provide a cow body size detection device and a detection method based on an RGB-D camera.
In order to achieve the purpose, the invention provides the following technical scheme:
a milk cow body chi detection device based on RGB-D camera:
the system comprises a cow channel, an RGB-D camera, an infrared sensor and a computer;
the milk cow channel is composed of a left cross rod fence and a right cross rod fence, and only one milk cow is allowed to pass through the milk cow channel once;
the RGB-D camera is arranged on one side of the cow channel and is formed by combining a 3D camera and an RGB camera, and is connected with a computer to upload the acquired point cloud image and the RGB image to the computer;
the infrared sensor is arranged on an upper cross bar fence of the cow channel, is connected with computer equipment and is used for judging whether the cow is in the best shooting position or not and sending an image acquisition signal;
the computer identifies the identity of the cow by using the black-white pattern characteristics of the cow for the received RGB image, and performs fusion processing analysis on the point cloud image and the RGB image according to the body type and color characteristics of the cow for the received point cloud image so as to realize non-contact cow body size detection.
Preferably, the positive direction of the x axis of the coordinate system imaged by the 3D camera is horizontally towards the right, the positive direction of the y axis is vertically downwards, and the positive direction of the z axis is vertically backwards.
A cow body size detection method based on an RGB-D camera comprises the following steps:
A. the method comprises the following steps of constructing a cow image acquisition platform in an experimental field to acquire a target image, and specifically comprising the following steps:
a 1: constructing a cow collecting channel: the cow collecting channel only allows one cow to pass through once; installing an infrared sensor on a cow channel to judge whether the cow is in the best shooting position, and sending an image acquisition instruction when the infrared sensor detects that the cow passes through the channel;
a 2: constructing an image acquisition device: installing an RGB-D camera on the bracket and arranged on one side of the cow channel, and connecting the RGB-D camera to a computer; receiving an image acquisition instruction, acquiring RGB images by using an RGB camera, transmitting the RGB images to computer equipment, acquiring point cloud images by using a 3D camera, and transmitting the point cloud images to the computer equipment;
B. processing the RGB image of the step a2 to realize the identification of the dairy cow;
C. processing the RGB image and the point cloud image of the step a2 to realize registration fusion of the RGB and the point cloud;
D. and C, processing the fused point cloud image obtained in the step C to realize automatic detection of the milk cow body size.
Wherein, the step B specifically comprises the following steps:
b 1: image preprocessing: performing image normalization and filtering preprocessing on the RGB image of the step a 2;
b 2: image segmentation: performing operations such as edge segmentation and region segmentation on the RGB image of the step a2 to extract the abdominal region of the cow;
b 3: image feature extraction: extracting feature data of the RGB image in the step b2 through an intelligent algorithm, performing dimensionality reduction optimization on the feature data, and constructing a feature space for classification and identification of the dairy cow individuals;
b 4: and constructing a relationship model of the extracted cow black-white pattern feature space and the cow individuals through a neural network classifier, a support vector machine classifier or a deep learning technology, and ensuring that the model identification precision is over 95 percent.
Wherein, the characteristic data in the step b3 comprises texture characteristics and color characteristics of black and white patterns of the dairy cows.
Wherein, the step C specifically comprises the following steps:
c 1: calibrating a camera: calibrating the RGB camera and the 3D camera based on OpenCV, and solving parameters and relative positions of the two cameras;
c 2: and (3) registration fusion of images: and c1, converting the coordinate systems of the RGB camera and the 3D camera by using the parameters of the camera in the step c1, and directly projecting the 3D camera point cloud onto the RGB camera.
6. The cow body size detection method based on the RGB-D camera as claimed in claim 3, wherein the step D specifically comprises the following steps:
d 1: image preprocessing: c, removing ground noise, milk cow channels and preprocessing of background environment discrete points from the fused point cloud image in the step C, and extracting parameters of a ground model;
d 2: extracting a point cloud of the dairy cow: extracting color features, depth features and three-dimensional surface normal features of the fused point cloud image obtained in the step d1, removing interferences such as complex background, discrete points and the like in the step d1, and segmenting and extracting the point cloud of the dairy cow;
d 3: extracting the characteristic points of the dairy cows: and d, extracting the characteristic points of the cow body from the point cloud image of the step d2 according to the geometric shape and the color characteristics of the cow body.
d 4: detecting a milk cow body size: through the calculation of the space distance, the automatic detection of the body size of the dairy cow can be realized.
Wherein, the feature points of the cow body extracted in the step d3 are as follows:
2.1: extracting a target area by using a straight-through filtering method: through three-dimensional visualization discovery of the whole cow point cloud, the characteristic points of the cow body are basically located in the shoulder, abdomen, tail and hind limb areas of the cow, and the point cloud data of the four parts are extracted by using a straight-through filtering method;
2.2: extracting characteristic points such as milk cow milkvetch points, body depth upper points, body depth lower points and the like in shoulder, abdomen, tail and hind limb areas of the milk cow point cloud based on color characteristics, depth characteristics and three-dimensional surface normal characteristics; extracting feature points including shoulder end points, abdomen width points, waist corner points, ischial node nodes and hind limb flying nodes of the dairy cows by utilizing a method for searching extreme points based on depth features and morphological features;
2.3: extracting the astragalus membranaceus points, the upper points of body depth and the lower points of body depth in the step 2.2, and constructing a plane equation of the symmetrical plane of the cow body;
2.4: calculating the body ruler parameters of the cow by using a distance equation in space: and D, performing corresponding cow body size detection by calculating the distance in space by using the ground plane equation of the step D, the plane equation of the cow body symmetry plane of the step 2.3 and the characteristic points of the cow body of the step 2.2.
Advantageous effects
The invention provides a cow body ruler detection device and a detection method based on an RGB-D camera, and compared with the prior art, the cow body ruler detection device has the following beneficial effects:
(1) according to the method, the RGB image of the milk cow is acquired by using the visible light camera, the identity of the milk cow is identified through the black-white pattern characteristics of the milk cow, the model identification precision is over 95%, and identity information is provided for the body size detection of the individual milk cow; meanwhile, the problem of failure in milk cow identity identification caused by ear tag falling off, milk cow head swinging and the like is solved.
(2) According to the invention, the high-resolution pixel information acquired by the RGB camera and the point cloud information acquired by the 3D camera are subjected to image fusion, and the individual point cloud data of the dairy cow can be more accurately extracted based on the color characteristics, the depth characteristics and the three-dimensional curved surface normal characteristics of the dairy cow, so that the defects that the RGB camera is easily illuminated, and a moving target and a background color are close to each other, and cannot be detected, and the like, are overcome, and the defects that the 3D camera is low in resolution, high in target edge noise and the like are overcome.
(3) The method utilizes the 3D imaging technology to automatically measure the body size parameters of the dairy cow, solves the problems of large workload, dairy cow aggressors and the like of the traditional method for measuring the body size of the dairy cow by utilizing tools such as a tape, a measuring stick, a circular touch sensor and the like, also protects the normal life habit of the dairy cow, and also solves the defect that the body size index is less in measurement because only single-side image data is obtained by measuring the body size of the dairy cow by a monocular camera.
(4) Compared with the existing milk cow body ruler detection based on the stereoscopic vision technology, the milk cow body ruler automatic detection method based on the three-dimensional vision technology has the advantages that the milk cow body ruler parameters are automatically measured by utilizing the 3D imaging technology, the automatic calculation of a plurality of index parameters of the milk cow body height, the body depth, the waist strength and the like is realized through the fusion of the milk cow RGB and the point cloud images based on the color characteristics, the depth characteristics and the three-dimensional curved surface normal characteristics, and the automatic calculation of a plurality of index parameters of the milk cow body length, the body diagonal length, the shoulder width, the abdomen width, the nojiri angle, the hind limb side view and the like is realized based on the depth characteristics and the morphological characteristics. The invention has more measurement parameters, and a plurality of indexes are measured by adopting fused data of RGB and point cloud images, so that the average relative error of the body size parameters is controlled within 3 percent.
(5) The device is simple and convenient to operate, complete in function, stable and reliable. The method can realize the identification and the body size detection of the milk cow under the non-contact condition, and enable the milk cow identification and the body size information to correspond one to one, thereby improving the automation degree of detection, greatly relieving the workload of manual detection and solving the problem of accurate correspondence of the milk cow body size and the identity.
Drawings
FIG. 1 is a schematic structural diagram of a detecting device according to the present invention.
FIG. 2 is a block diagram of the detection method of the present invention.
FIG. 3 is a schematic diagram of the measuring points of the cow body measuring method.
In fig. 1, 1-infrared sensor, 2-cow tunnel, 3-RGB camera, 4-3D camera, 5-camera support, 6-computer.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, the present embodiment provides a cow body size detection device based on RGB-D camera:
the system comprises a cow channel 2, an RGB-D camera, an infrared sensor 1 and a computer 6;
the milk cow channel 2 is composed of left and right cross rod fences, and only one milk cow is allowed to pass through the milk cow channel 2 once; the cow channel 2 is composed of left and right cross bar fences, the heights of the upper and lower cross bar fences on one side are 1.2 m and 0.6 m in sequence, and the widths of the cross bar fences on the two sides are 1 m.
The RGB-D camera is fixed on an machine support 5 and is installed on one side of the cow channel 2, the RGB-D camera is formed by combining a 3D camera 4 and an RGB camera 3 and is connected with a computer 6, and the collected point cloud image and the collected RGB image are uploaded to the computer 6; the RGB-D camera is 1 meter higher than the ground and 21.9 meters away from the cow channel; the positive direction of the x axis of the coordinate system imaged by the 3D camera 4 is horizontally rightward, the positive direction of the y axis is vertically downward, the positive direction of the z axis is vertically backward, and the resolution of the 3D camera 4 is 640 multiplied by 480; the resolution size of the RGB camera 3 is 2048 × 1536.
The infrared sensor 1 is arranged on an upper cross bar fence of the cow channel 2, is connected with a computer 6 and is used for judging whether the cow is in the best shooting position or not and sending an image acquisition signal; the infrared sensor 1 is installed at a position 0.75 m to the right from the horizontal direction of the RGB-D camera.
The computer 6 identifies the identity of the cow by using the black and white pattern characteristics of the cow for the received RGB image, and performs fusion processing analysis on the point cloud image and the RGB image according to the body shape and color characteristics of the cow for the received point cloud image so as to realize non-contact cow body size detection.
Corresponding to the cow body size detection device based on the RGB-D camera provided by the embodiment, the invention also provides an embodiment of the cow body size detection method based on the RGB-D camera. FIG. 2 is a block diagram of the detection method of the present invention, including the following steps:
A. the method comprises the following steps of constructing a cow image acquisition platform in an experimental field to acquire a target image, and specifically comprising the following steps:
a 1: constructing a cow collecting channel: the cow collecting channel only allows one cow to pass through once; installing an infrared sensor on a cow channel to judge whether the cow is in the best shooting position, and sending an image acquisition instruction when the infrared sensor detects that the cow passes through the channel;
a 2: constructing an image acquisition device: installing an RGB-D camera on the bracket and arranged on one side of the cow channel, and connecting the RGB-D camera to a computer; receiving an image acquisition instruction, acquiring RGB images by using an RGB camera, transmitting the RGB images to computer equipment, acquiring point cloud images by using a 3D camera, and transmitting the point cloud images to the computer equipment;
B. the RGB image of the step a2 is processed to realize the identification of the dairy cow, and the method comprises the following steps:
b 1: image preprocessing: performing image normalization and filtering preprocessing on the RGB image of the step a 2;
b 2: image segmentation: performing operations such as edge segmentation and region segmentation on the RGB image of the step a2 to extract the abdominal region of the cow;
b 3: image feature extraction: extracting feature data of the RGB image in the step b2 through an intelligent algorithm, performing dimensionality reduction optimization on the feature data, and constructing a feature space for classification and identification of the dairy cow individuals; the characteristic data in the step b3 comprises texture characteristics and color characteristics of black and white patterns of the dairy cows;
b 4: and constructing a relation model of the extracted cow black-white pattern feature space and the cow individual through technologies such as a neural network classifier, a support vector machine classifier or deep learning, and ensuring that the model identification accuracy is over 95 percent.
C. Processing the RGB image and the point cloud image of the step a2 to realize registration fusion of RGB and point cloud, comprising the following steps:
c 1: calibrating a camera: calibrating the RGB camera and the 3D camera based on OpenCV, and solving parameters and relative positions of the two cameras;
c 2: and (3) registration fusion of images: and c1, converting the coordinate systems of the RGB camera and the 3D camera by using the parameters of the camera in the step c1, and directly projecting the 3D camera point cloud onto the RGB camera.
D. And C, processing the fused point cloud image obtained in the step C to realize automatic detection of the milk cow body size. The method specifically comprises the following steps:
d 1: image preprocessing: c, removing ground noise, milk cow channels and preprocessing of background environment discrete points from the fused point cloud image in the step C, and extracting parameters of a ground model;
d 2: extracting a point cloud of the dairy cow: extracting color features, depth features and three-dimensional surface normal features of the fused point cloud image obtained in the step d1, removing interferences such as complex background, discrete points and the like in the step d1, and segmenting and extracting the point cloud of the dairy cow;
d 3: extracting the characteristic points of the dairy cows: and d, extracting the characteristic points of the cow body from the point cloud image of the step d2 according to the geometric shape and the color characteristics of the cow body.
d 4: detecting a milk cow body size: through the calculation of the space distance, the automatic detection of the body size of the dairy cow can be realized.
The feature points of the cow body extracted in the step d3 are as follows:
2.1: extracting a target area by using a straight-through filtering method: through three-dimensional visualization discovery of the whole cow point cloud, the characteristic points of the cow body are basically located in the shoulder, abdomen, tail and hind limb areas of the cow, and the point cloud data of the four parts are extracted by using a straight-through filtering method.
2.1, extracting the target area by adopting a direct filtering method: on the premise of defining the target area, a threshold range is set on a specified space dimension, and data on the dimension is divided into the threshold range and the range which is not in the threshold range, so that whether filtering is performed or not is selected, and the point cloud of the target area can be quickly acquired.
2.2: extracting characteristic points such as milk cow milkvetch points, body depth upper points, body depth lower points and the like in shoulder, abdomen, tail and hind limb areas of the milk cow point cloud based on color characteristics, depth characteristics and three-dimensional surface normal characteristics; and extracting the characteristic points of the shoulder end point, the abdomen width point, the waist angular point, the ischial node, the hind limb flying node and the like of the cow by using a method for searching the extreme point based on the depth characteristic and the morphological characteristic.
Step 2.2, extracting the points of astragalus membranaceus and shoulder: in the step 2.1, the RGB color features of the cow are extracted from the shoulder area of the cow, and the color difference value between adjacent or similar pixel points is calculated. Performing a loop N1 times, and continuously searching the minimum value and color difference point of the point cloud coordinate y (i.e. the point M of Astragalus)1) And N1 is the number of the point clouds in the shoulder area of the cow. On the basis of extracting the astragalus membranaceus point, 5 point clouds are expanded left and right by taking the x coordinate of the astragalus membranaceus point as a reference to perform straight-through filtering to reduce the shoulder area of the cow, and circulation is performed for N2 times to continuously search the minimum value of the point cloud coordinate z (namely the shoulder end point M)2) And N2 is the number of the point clouds in the shoulder area of the reduced cow.
2.2, extracting body depth upper points, body depth lower points and abdomen width points: in the step 2.1, the RGB color features of the cow are extracted from the belly region of the cow, and the color difference between adjacent or similar pixel points is calculated. The circulation is performed for N3 times, and the minimum value, the maximum value and the color difference value point of the point cloud coordinate y are continuously searched ((namely the point M on the body depth)3Body depth point M4) While continuously finding the minimum value of the point cloud coordinate z (i.e. the abdominal end point M)5) And N3 is the number of the point clouds in the abdominal region of the cow.
Step 2.2, extraction of ischial node: in the step 2.1, in the cow tail area extraction, loop N4 times is performed, the highest point column of the tail area is extracted, and two extreme points (namely waist angle point M) of the y coordinate of the highest point column are calculated6Ischial node M7) Wherein N4 is the number of the point clouds in the tail area of the cow.
Step 2.2, extracting hind limb flying nodes: in the area of cow hind limbs extracted in the step 2.1, the circulation is performed for N5 times, and the maximum point cloud coordinate x is continuously searchedValue (i.e. hind limb flying node M)8) And N5 is the number of the point clouds in the hind limb area of the dairy cow.
2.3: and (3) extracting the astragalus membranaceus points, the upper body depth points and the lower body depth points in the step 2.2, and constructing a plane equation of the symmetrical plane of the cow body.
Step 2.3, coordinates of the milkvetch point, the body depth upper point and the body depth lower point are selected to establish a plane equation of the symmetrical plane of the cow body: three point cloud coordinates P1(x1,y1,z1)、P2(x2,y2,z2)、P3(x3,y3,z3) The plane equation of the symmetrical plane of the cow body can be set as follows:
A1(x-x1)+B1(y-y1)+C1(z-z1)＝0。
wherein: a. the1＝(y3-y1)×(z3-z1)-(z2-z1)×(y3-y1)B1＝(x3-x1)×(z2-z1)-(x2-x1)×(z3-z1)
C1＝(y3-y1)×(x2-x1)-(x3-x1)×(y2-y1)
The plane equation of the symmetrical plane of the cow body is as follows: a. the1x+B1y+C1z+D10. Wherein D is1＝-(A1x1+B1y1+C1z1)。
2.4: calculating the body ruler parameters of the cow by using a distance equation in space: by using the ground plane equation A of said step D1x+B1y+C1z+D1The plane equation A of the symmetrical plane of the cow body in the step 2.3 is 02x+B2y+C2z+D20, the characteristic point of the cow body of step 2.2 is the milkvetch point M1(x1，y1，z1) Shoulder end point M2(x2，y2，z2) Upper point of body depthM3(x3，y3，z3) Body depth point M4(x4，y4，z4) Point of abdominal fate M5(x5，y5，z5) Waist corner point M6(x6，y6，z6) Ischial node M7(x7，y7，z7) Hind limb flying node M8(x8，y8，z8) And carrying out corresponding cow body size detection by calculating the distance in the space.
Cow height W1: the vertical distance from the milkvetch point to the ground,
Figure BDA0003308925520000091
cow body length W2: horizontal distance, W, from the milkvetch point to the ischial node2＝|x6-x1|
Oblique length W of milk cow body3: the distance between the shoulder end point of the cow and the ischial node,
Figure BDA0003308925520000092
depth of cow body W4: vertical distance, W, from the top to the bottom of the cow's belly4＝|y3-y4|
Cow shoulder width W5: twice the distance from the shoulder point of the cow to the symmetrical plane of the body,
Figure BDA0003308925520000093
cow abdomen width W6: twice the distance from the abdominal end point of the cow to the symmetrical plane of the body,
Figure BDA0003308925520000101
nojirima W of dairy cattle7: dairy cowThe ischial node is twice the distance from the body plane of symmetry,
Figure BDA0003308925520000102
nojiri angle W of milk cow8: the relative height difference between the ischial node and the waist angle of the cow,
Figure BDA0003308925520000103
cow waist strength W9: the degree of curvature of the transverse process of the lumbar vertebrae of the cow is determined by the point M of the lumbar angle of the cow6(x6，y6，z6) And point M on the depth of body3(x3，y3，z3) Ischial node M7(x7，y7，z7) Form a vector
Figure BDA0003308925520000104
Computing vectors
Figure BDA0003308925520000105
Figure BDA0003308925520000106
Angle theta therebetween1。
Figure BDA0003308925520000107
Figure BDA0003308925520000108
Figure RE-GDA0003370281700000109
Cow hind limb side view W10: the bending degree of the hind limb flying node of the milk cow is determined by the hind limb flying node M of the milk cow8(x8，y8，z8) For reference, searching 5 field points in the positive direction of the z-axis of the coordinate to form a vector
Figure RE-GDA00033702817000001010
Searching 5 field points in the negative direction of the z-axis to form a vector
Figure RE-GDA00033702817000001011
Calculating the bending degree theta of the hind limb flying node of the cow2。
Figure RE-GDA00033702817000001012
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (8)

1. The utility model provides a milk cow physique chi detection device based on RGB-D camera which characterized in that:
the system comprises a cow channel, an RGB-D camera, an infrared sensor and a computer;
the milk cow channel is composed of a left cross rod fence and a right cross rod fence, and only one milk cow is allowed to pass through the milk cow channel once;
the RGB-D camera is arranged on one side of the cow channel, is formed by combining a 3D camera and an RGB camera, is connected with a computer, and uploads a collected point cloud image and an RGB image to the computer;
the infrared sensor is arranged on an upper cross bar fence of the cow channel, is connected with computer equipment and is used for judging whether the cow is in the best shooting position or not and sending an image acquisition signal;
the computer identifies the identity of the cow by using the black-white pattern characteristics of the cow for the received RGB image, and performs fusion processing analysis on the point cloud image and the RGB image according to the body type and color characteristics of the cow for the received point cloud image so as to realize non-contact cow body size detection.
2. The cow body size detection device based on the RGB-D camera as claimed in claim 1, wherein: the positive direction of the x axis of the coordinate system for imaging by the 3D camera is horizontally towards the right, the positive direction of the y axis is vertically downwards, and the positive direction of the z axis is vertically backwards.
3. A cow body size detection method based on an RGB-D camera is characterized by comprising the following steps:
A. the method comprises the following steps of constructing a cow image acquisition platform in an experimental field to acquire a target image, and specifically comprising the following steps:
a 1: constructing a cow collecting channel: the cow collecting channel only allows one cow to pass through once; installing an infrared sensor on a cow channel to judge whether the cow is in the best shooting position, and sending an image acquisition instruction when the infrared sensor detects that the cow passes through the channel;
a 2: constructing an image acquisition device: installing an RGB-D camera on the bracket and arranged on one side of the cow channel, and connecting the RGB-D camera to a computer; receiving an image acquisition instruction, acquiring an RGB image by using an RGB camera, transmitting the RGB image to computer equipment, acquiring a point cloud image by using a 3D camera, and transmitting the point cloud image to the computer equipment;
B. processing the RGB image of the step a2 to realize the identification of the dairy cow;
C. processing the RGB image and the point cloud image of the step a2 to realize registration and fusion of the RGB and the point cloud;
D. and C, processing the fused point cloud image obtained in the step C to realize automatic detection of the milk cow body size.
4. The cow body size detection method based on the RGB-D camera as claimed in claim 3, wherein the step B specifically comprises the following steps:
b 1: image preprocessing: performing image normalization and filtering pretreatment on the RGB image of the step a 2;
b 2: image segmentation: performing operations such as edge segmentation and region segmentation on the RGB image of the step a2 to extract the abdominal region of the cow;
b 3: image feature extraction: extracting feature data of the RGB image in the step b2 through an intelligent algorithm, performing dimensionality reduction optimization on the feature data, and constructing a feature space for classification and identification of the dairy cow individuals;
b 4: and constructing a relationship model of the extracted cow black-white pattern feature space and the cow individuals through a neural network classifier, a support vector machine classifier or a deep learning technology, and ensuring that the model identification precision is over 95 percent.
5. The cow body size detection method based on the RGB-D camera as claimed in claim 4, wherein: the characteristic data in the step b3 comprises texture characteristics and color characteristics of black and white patterns of the dairy cows.
6. The cow body size detection method based on the RGB-D camera as claimed in claim 3, wherein the step C comprises the following steps:
c 1: calibrating a camera: calibrating an RGB camera and a 3D camera based on OpenCV, and solving parameters and relative positions of the two cameras;
c 2: and (3) registration fusion of images: and c1, converting the coordinate systems of the RGB camera and the 3D camera by using the parameters of the camera in the step c1, and directly projecting the 3D camera point cloud onto the RGB camera.
7. The cow body size detection method based on the RGB-D camera as claimed in claim 3, wherein the step D specifically comprises the following steps:
d 1: image preprocessing: c, removing ground noise, milk cow channels and preprocessing of background environment discrete points from the fused point cloud image in the step C, and extracting parameters of a ground model;
d 2: extracting a point cloud of the dairy cow: extracting color features, depth features and three-dimensional surface normal features of the fused point cloud image obtained in the step d1, removing interferences such as complex background, discrete points and the like in the step d1, and segmenting and extracting the point cloud of the dairy cow;
d 3: extracting the characteristic points of the dairy cows: and d, extracting the characteristic points of the cow body from the point cloud image of the step d2 according to the geometric shape and the color characteristics of the cow body.
d 4: detecting a milk cow body size: through the calculation of the space distance, the automatic detection of the body size of the dairy cow can be realized.
8. The cow body size detection method based on the RGB-D camera according to claim 7, wherein: the feature points of the cow body extracted in the step d3 are as follows:
2.1: extracting a target area by using a straight-through filtering method: the method comprises the steps of discovering that the whole cow point cloud is three-dimensionally visualized, wherein the characteristic points of the cow body are basically located in the shoulder, abdomen, tail and hind limb areas of the cow, and extracting the point cloud data of the four parts by using a direct filtering method;
2.2: extracting characteristic points such as milk cow milkvetch points, body depth upper points, body depth lower points and the like in shoulder, abdomen, tail and hind limb areas of the milk cow point cloud based on color characteristics, depth characteristics and three-dimensional surface normal characteristics; extracting feature points including a shoulder end point, an abdomen width point, a waist corner point, an ischial node and a hind limb flying node of the milk cow by using a method for searching for extreme points based on depth features and morphological features;
2.3: extracting the astragalus membranaceus points, the upper points of body depth and the lower points of body depth in the step 2.2, and constructing a plane equation of a symmetrical plane of the cow body;
2.4: calculating the body ruler parameters of the cow by using a distance equation in space: and D, performing corresponding cow body size detection by calculating the distance in space by using the ground plane equation of the step D, the plane equation of the cow body symmetry plane of the step 2.3 and the characteristic points of the cow body of the step 2.2.
CN202111211000.2A 2021-10-18 2021-10-18 Cow body size detection device based on RGB-D camera and detection method thereof Pending CN113920138A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111211000.2A CN113920138A (en) 2021-10-18 2021-10-18 Cow body size detection device based on RGB-D camera and detection method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111211000.2A CN113920138A (en) 2021-10-18 2021-10-18 Cow body size detection device based on RGB-D camera and detection method thereof

Publications (1)

Publication Number Publication Date
CN113920138A true CN113920138A (en) 2022-01-11

Family

ID=79241707

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111211000.2A Pending CN113920138A (en) 2021-10-18 2021-10-18 Cow body size detection device based on RGB-D camera and detection method thereof

Country Status (1)

Country Link
CN (1) CN113920138A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114972472A (en) * 2022-04-21 2022-08-30 北京福通互联科技集团有限公司 Beef cattle three-dimensional depth image acquisition method based on laser array and monocular camera
CN114966733A (en) * 2022-04-21 2022-08-30 北京福通互联科技集团有限公司 Beef cattle three-dimensional depth image acquisition system based on laser array and monocular camera
CN115294181A (en) * 2022-08-18 2022-11-04 东北农业大学 Cow body shape assessment index measuring method based on two-stage key point positioning

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114972472A (en) * 2022-04-21 2022-08-30 北京福通互联科技集团有限公司 Beef cattle three-dimensional depth image acquisition method based on laser array and monocular camera
CN114966733A (en) * 2022-04-21 2022-08-30 北京福通互联科技集团有限公司 Beef cattle three-dimensional depth image acquisition system based on laser array and monocular camera
CN115294181A (en) * 2022-08-18 2022-11-04 东北农业大学 Cow body shape assessment index measuring method based on two-stage key point positioning

Similar Documents

Publication Publication Date Title
CN113920138A (en) Cow body size detection device based on RGB-D camera and detection method thereof
Yu et al. Segmentation and measurement scheme for fish morphological features based on Mask R-CNN
CN103411553B (en) The quick calibrating method of multi-linear structured light vision sensors
CN109141248A (en) Pig weight measuring method and system based on image
CN108764257A (en) A kind of pointer instrument recognition methods of various visual angles
CN109145808B (en) Tuna identification method based on self-adaptive fish body bending model
CN107657639A (en) A kind of method and apparatus of quickly positioning target
CN101512551A (en) A method and a system for measuring an animal's height
CN111612850B (en) Point cloud-based pig body ruler parameter measurement method
CN111243005A (en) Livestock weight estimation method, device, equipment and computer readable storage medium
CN105913013A (en) Binocular vision face recognition algorithm
CN108961330A (en) The long measuring method of pig body and system based on image
CN107016353B (en) A kind of integrated method and system of variable resolution target detection and identification
CN102704215A (en) Automatic cutting method of embroidery cloth based on combination of DST file parsing and machine vision
CN108209926A (en) Human Height measuring system based on depth image
CN110569735A (en) Analysis method and device based on back body condition of dairy cow
WO2023060632A1 (en) Street view ground object multi-dimensional extraction method and system based on point cloud data
CN111767794A (en) Cage-rearing poultry abnormal behavior detection method and detection system based on machine vision
Hao et al. An improved PointNet++ point cloud segmentation model applied to automatic measurement method of pig body size
CN112861872A (en) Penaeus vannamei phenotype data determination method, device, computer equipment and storage medium
CN102855485A (en) Automatic wheat earing detection method
CN113470106A (en) Non-contact cow body size information acquisition method
CN110851978B (en) Camera position optimization method based on visibility
CN112288793A (en) Livestock individual backfat detection method and device, electronic equipment and storage medium
CN116152697A (en) Three-dimensional model measuring method and related device for concrete structure cracks

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination