Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Further, the drawings are merely schematic illustrations of the present disclosure, in which the same reference numerals denote the same or similar parts, and thus, a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The following detailed description of exemplary embodiments of the disclosure refers to the accompanying drawings.
Fig. 1 schematically shows a flow chart of a method of detecting a body scale of an animal according to an exemplary embodiment of the present disclosure. Referring to fig. 1, the animal body ruler detection method 100 may include:
step S1, acquiring an overhead view image of the livestock;
step S2, inputting the overhead view image into a trained body size detection model, wherein the body size detection model comprises a plurality of neural networks connected in series, and the sizes of input pictures of the neural networks are the same;
and step S3, outputting the body size data of the livestock.
In the disclosed embodiment, the livestock can be pigs, cattle and sheep, and the body size data can be shoulder width, hip width and body length of the livestock. It is understood that the method 100 for detecting body size of livestock provided by the present disclosure can be used for detecting body sizes of various types of livestock or animals, and the detected body size data can also include more types, and those skilled in the art can set the method according to actual needs.
Fig. 2 is a schematic structural diagram of the body ruler detection model.
Referring to FIG. 2, in an embodiment of the present disclosure, the body ruler detection model 11 is implemented based on an MTCNN network, and may include a first network P-Net, a second network R-Net, and a third network O-Net connected in series. P-Net is used for analyzing the input overlook image 12 to obtain a candidate frame, R-Net is used for deleting the livestock-free candidate frame, O-Net is used for outputting the position of the livestock frame and the key point, and finally body size data 13 of the livestock is output.
The first network P-Net to the third network O-Net output three groups of characteristic values: the classification data, the frame regression data and the key point regression data, and the second network R-Net and the third network O-Net respectively take the characteristic values output by the first network P-Net and the second network R-Net as input data and adjust the input image according to the input data and the preset input image size.
Fig. 3A to 3C are schematic structural diagrams of the first network to the third network, respectively.
Referring to fig. 3A to 3C, the sizes of targets faced by the livestock body scale detection method provided by the present disclosure are not greatly different, and therefore, in the embodiment of the present disclosure, the sizes of the input pictures of the first network to the third network are all set to be the same, so as to reduce the amount of calculation, increase the calculation speed, and improve the search efficiency and accuracy.
In addition, different from the traditional MTCNN network mainly applied to the field of face recognition, for animal body size detection, if the small input picture size of the traditional MTCNN network is applied, more false alarm frames appear, and the calculation capacity is wasted, so for example, taking the pig body size detection as an example, the input picture sizes of the first network to the third network can be set to 160 × 160, so as to ensure that the position of the pig can be quickly positioned.
Referring to fig. 3A, for the MTCNN network, the input data source is a picture pyramid. The forming process of the picture pyramid comprises the following steps: the pictures are divided into different sizes according to different scales, and the pictures are cut and stacked like a pyramid, so that one picture is equivalent to a plurality of pictures. After the top view image is processed into an image pyramid, each picture in the pyramid is processed into an input matrix of 160 x 3 according to three eigen channels (R, G, B), which are input into a first network P-Net.
According to the MTCNN network setting, through five groups of operations, P-Net outputs three groups of characteristic values of the overlook image: pig classification, box regression, and key point regression. In order to improve the calculation efficiency for the livestock body size detection, five groups of operation parameters of P-Net are adjusted in the embodiment of the present disclosure, specifically, the size of one or more convolution kernels is adjusted from conventional 3 × 3 to 5 × 5, the pooling parameter is adjusted from conventional 3 × 3 to 4 × 4, and the pooling step size is set to 4, so as to improve the detection speed as much as possible while ensuring the detection accuracy. The adjusted operation parameters can be any one or more of five groups of operation parameters, and the technical personnel in the field can set the operation parameters according to the types of the detected livestock.
Of the three eigenvalues output by P-Net, pigs were sorted into a matrix set of 1 x 2, where 1 x 1 corresponds to the input image size 160 x 160, and at an input image size of 320 x 320, the pig sorting matrix set was adaptively switched to 4 x 2. Where 2 represents the identification category, including, for example, "background" and "foreground", in a binarized representation. The frame regression feature value is a matrix group of 1 × 4, and 1 × 1 also corresponds to the input image size 160 × 160, which is not described again; and 4 represents the abscissa and ordinate of the upper left corner point and the abscissa and ordinate of the lower right corner point of the detection frame.
The key point regression feature values are in a 1 x 12 matrix set. Unlike the setting of detecting five key points when the conventional MTCNN network is applied to the field of face recognition, in one embodiment of the present disclosure, body ruler data is set to shoulder width, hip width, body length of livestock, and thus, the number of regression key points is set to 6, i.e., the key points include six points of left shoulder, right shoulder, left hip, right hip, head, and tail root. Thus, "12" in the regression feature values of the key points indicates that the abscissa and ordinate of the six points total 12 data.
Since one picture is divided into many small pyramid-type pictures, it is necessary to perform Non-Maximum Suppression (NMS) processing on the entire processed picture once and finally generate a corresponding processed result. Specifically, the characteristic value output by the P-Net is subjected to bounding box regression (bounding box regression) processing and non-maximum suppression processing to form an input parameter of the R-Net.
In fig. 3B, the second network R-Net performs four sets of operations on the input image to output three sets of feature values. The input image size of R-Net is also 160 × 160, and the input image is obtained by performing Resize (size transformation) processing on the frame region to be selected, which is determined according to the input parameters and the original input image. Each set of operation parameters in R-Net can be adjusted to a convolution kernel size of 5 x 5, a pooling parameter of 4 x 4, and a pooling step of 4. Like P-Net, the number of adjusted operational parameters may differ from animal to animal.
In the same way, the characteristic value output by the R-Net is subjected to bounding box regression (bounding box regression) processing and non-maximum suppression processing to form an input parameter of the O-Net.
In FIG. 3C, the third network O-Net performs five sets of operations on the Resize processed input image to output three sets of feature values. The input image is formed in the same way as R-Net. And extracting final body size data after the characteristic value output by the O-Net is subjected to bounding box regression (bounding box regression) processing and non-maximum suppression processing.
The manner of training the body size detection model 11 may be step-by-step training. Firstly training P-Net, then training R-Net by using the output data of the trained P-Net, and finally training O-Net by using the output data of the trained R-Net, so as to improve the correction capability of the R-Net and the O-Net on the input data. When generating the R-NET and O-NET network training data, the search box can be initialized to be rectangular (the length-width ratio is different according to different detected types of livestock) so as to adapt to the proportion of the body types of the livestock.
Fig. 4A and 4B are a schematic diagram of a search box and a schematic diagram of a keypoint, respectively, in an embodiment of the present disclosure.
Referring to fig. 4A, when the tested livestock is a pig, the search box is set to be rectangular with an aspect ratio of 2: 1.
Referring to fig. 4B, six point locations of the tail root a, the head B, the left shoulder C, the right shoulder D, the left hip E, and the right hip F are set as key points, and are used as body ruler data.
After the body size data are obtained, the estimated body weight of the livestock can be further output according to the relation between the body size data and the unit volume body weight of the livestock.
It is worth mentioning that, because the camera has a fish-eye effect, the method provided by the present disclosure can be used for accurately detecting the body size data of one or more livestock located in the center of the overlooking image. With regard to the prior art in through detecting the object profile in the image and discern the object different, the method that this disclosure provided can be owing to used not be restricted to the body chi detection model of colour, light and gesture and carry out livestock discernment, can also guarantee accurate body chi and measure when the livestock is crowded, the gesture is different, once only detects the body chi of bull livestock promptly, helps improving detection efficiency, reduces the time of driving the livestock.
In conclusion, the livestock body scale detection method provided by the invention can effectively detect the body scale information of the livestock in the image under various illumination conditions and livestock postures, and improve the accuracy of visual weight estimation.
Corresponding to the method embodiment, the present disclosure also provides a livestock body ruler detection apparatus, which may be used to execute the method embodiment.
Fig. 5 schematically shows a block diagram of an animal body size detection arrangement in an exemplary embodiment of the disclosure.
Referring to fig. 5, the animal body ruler detecting apparatus 500 may include:
a camera 51 for acquiring overhead images of the livestock;
a processor 52, coupled to the camera, for acquiring the overhead image; inputting the overlook images into a trained body scale detection model, wherein the body scale detection model comprises a plurality of neural networks which are connected in series, and the sizes of input pictures of the neural networks are the same; and outputting the body ruler data of the livestock.
In an exemplary embodiment of the present disclosure, the body-size data comprises a shoulder width, a hip width, a body length of the animal.
In an exemplary embodiment of the present disclosure, the plurality of serially connected neural networks includes a first network, a second network, and a third network connected in series, and the number of regression key points of the first network, the second network, and the third network is 6.
In an exemplary embodiment of the present disclosure, the processor 52 is further configured to output the body weight of the animal based on the body size data.
In an exemplary embodiment of the disclosure, the animal is one or more animals located in the center of the overhead image, and the body ruler data comprises body ruler data for each of the animals.
In an exemplary embodiment of the present disclosure, the livestock includes pigs, cattle, sheep.
In an exemplary embodiment of the present disclosure, the input picture sizes of the first, second, and third nets are all 160 × 160, the convolution kernel sizes of the first and second nets include 5 × 5, the pooling parameters include 4 × 4, and the pooling step size includes 4; the convolution kernel size and pooling parameter of the third network are both 3 x 3, and the pooling step size is 2.
Since the functions of the apparatus 500 have been described in detail in the corresponding method embodiments, the disclosure is not repeated herein.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 600 according to this embodiment of the invention is described below with reference to fig. 6. The electronic device 600 shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 6, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: the at least one processing unit 610, the at least one memory unit 620, and a bus 630 that couples the various system components including the memory unit 620 and the processing unit 610.
Wherein the storage unit stores program code that is executable by the processing unit 610 to cause the processing unit 610 to perform steps according to various exemplary embodiments of the present invention as described in the above section "exemplary methods" of the present specification. For example, the processing unit 610 may perform a method as shown in fig. 1.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)6201 and/or a cache memory unit 6202, and may further include a read-only memory unit (ROM) 6203.
The memory unit 620 may also include a program/utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
Bus 630 may be one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. As shown, the network adapter 660 communicates with the other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above section "exemplary methods" of the present description, when said program product is run on the terminal device.
Referring to fig. 7, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.