CN108388830B - Animal body shape detection method and device based on convolutional neural network - Google Patents

Animal body shape detection method and device based on convolutional neural network Download PDF

Info

Publication number
CN108388830B
CN108388830B CN201810019554.4A CN201810019554A CN108388830B CN 108388830 B CN108388830 B CN 108388830B CN 201810019554 A CN201810019554 A CN 201810019554A CN 108388830 B CN108388830 B CN 108388830B
Authority
CN
China
Prior art keywords
image
feature map
value
animal
image feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201810019554.4A
Other languages
Chinese (zh)
Other versions
CN108388830A (en
Inventor
高万林
仲贞
王敏娟
于丽娜
陈治昌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China Agricultural University
Original Assignee
China Agricultural University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China Agricultural University filed Critical China Agricultural University
Priority to CN201810019554.4A priority Critical patent/CN108388830B/en
Publication of CN108388830A publication Critical patent/CN108388830A/en
Application granted granted Critical
Publication of CN108388830B publication Critical patent/CN108388830B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/446—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering using Haar-like filters, e.g. using integral image techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/255—Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

The invention provides an animal body shape detection method and device based on a convolutional neural network. The method comprises the following steps: carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map; registering the depth image of the animal with the first image feature map to obtain a second image feature map; superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map; and inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape. The invention can improve the accuracy and real-time property of animal body shape detection.

Description

Animal body shape detection method and device based on convolutional neural network
Technical Field
The invention relates to the technical field of image detection, in particular to a method and a device for detecting animal body shapes based on a convolutional neural network.
Background
Some animals on earth have many similarities with human beings in physiological and pathological processes of life activities and can be referred to each other, and particularly, pigs, which are large animals, have close relationship with human beings and have many similarities. Therefore, studying the health status of animals, especially pigs, is of great value to the understanding of human life.
In recent years, artificial intelligence dominated by machine learning has been rapidly developed, and a major breakthrough has been made in many research fields. The image detection technology is an important research hotspot and difficulty in the field of artificial intelligence. The body shape of the animal is one of the phenotypic characteristics of the animal, which can reflect the health condition of the animal, so the body shape detection of the animal is an important research direction.
Due to the fact that the postures of animals are different, visible light images are easily influenced by illumination and complex breeding environments, the animals are mutually shielded, and the like, animal body shape detection is always a difficult point in the field of image detection, and therefore a robustness algorithm for animal body shape detection is very important to research.
Disclosure of Invention
The present invention provides a convolutional neural network based animal body shape detection method and apparatus that overcomes, or at least partially solves, the above-mentioned problems.
According to one aspect of the present invention, there is provided a method for detecting the shape of an animal, comprising:
carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
registering the depth image of the animal with the first image feature map to obtain a second image feature map;
superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map;
and inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape.
Further, the extracting the features of the original image of the animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map specifically includes:
converting the original image into a first gray image, and carrying out Gabor filtering on the first gray image by adopting a Gabor filter with a scale parameter sigma of 5 to obtain amplitude characteristic images of animal figures in multiple directions;
utilizing a sequencing filter to encode the amplitude characteristic images of the animal body shapes in the multiple directions to obtain GOM characteristic encoded images of the animal body shapes in the multiple directions;
and obtaining the GOM feature coded image of the animal body shape according to the GOM feature coded images of the animal body shape in the multiple directions, wherein the GOM feature coded image of the animal body shape is used as the first image feature map.
Further, the expression of the Gabor filter is:
Figure GDA0002385912860000021
where σ represents the scale parameter of the Gabor filter, θkAn angle value representing a k-th direction;
Figure GDA0002385912860000022
as a scale parameter in the x-direction of the Gabor filter,
Figure GDA0002385912860000023
as a scale parameter in the y-direction of the Gabor filter,
Figure GDA0002385912860000024
for the value of x for the k-th angular direction,
Figure GDA0002385912860000025
is the value of y, f, in the k-th angular directionkIs the center frequency in the k direction;
the expression of the sequencing filter is:
Figure GDA0002385912860000031
where ω and represents the center position and scale of the sequencing filter, N, respectivelypNumber of positive lobes, NnNumber of negative lobes, coefficient CpAnd CnIs a coefficient that keeps the number of positive and negative lobes balanced, and NpCp＝NnCn；piIs a scale parameter of the positive lobe,njscale parameter of the negative lobe, ωpiIs the central position of the positive lobe, ωnjIs the central position of the negative valve
The plurality of directions, specifically 8 directions, includes 0 °,22.5 °,45 °,67.5 °,90 °,112.5 °,135 ° and 157.5 °.
Further, registering the depth image of the animal with the first image feature map to obtain a second image feature map, specifically including:
registering the depth image and the first image feature map in an affine transformation mode to obtain an image feature map of the depth image, wherein the affine transformation formula is as follows:
Figure GDA0002385912860000032
wherein x is1Is the value of the first characteristic image in the x direction, y1Is a value in the y direction of the first feature image, x2Is the value of the second characteristic image in the x direction, y2Is the value of the y direction of the second feature image, txIs the translation value, t, in the x directionyThe value of the translation in the y direction, s the scaling, and θ the counterclockwise rotation angle with (x, y) as the axis.
Further, the superimposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superimposed feature fusion map specifically include:
compressing the gray value of the pixel point in the second image characteristic image to a gray value range consistent with the gray value range of the first image characteristic image;
based on the compressed second image characteristic map and the corresponding pixel points in the first image characteristic map, selecting the gray value of the pixel point with a larger gray value as the gray value of the pixel point at the corresponding position of the fused image;
and obtaining the characteristic fusion image according to the gray values of the pixel points at the corresponding positions of all the fusion images, thereby obtaining and obtaining the candidate target area.
Further, compressing the gray value of the pixel point in the second image feature map to a gray value range consistent with the second image feature map further includes: if the gray value of the compressed second image feature map is non-integer, performing approximate calculation through the following formula to obtain an approximate gray value Ia(x,y)：
Figure GDA0002385912860000041
And I (x, y) represents the gray value of the pixel point of the second image characteristic diagram.
Further, the convolutional neural network comprises three convolutional layers, three pooling layers and three full-connection layers; inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape, which specifically comprises:
the first layer of convolutional layer is filtered by using 96 convolutional filters with the convolutional kernel size of 11 multiplied by 11 and the step size of 4; the second convolution layer uses 256 convolution filters with convolution kernel size of 5 multiplied by 5 and step length of 1 to filter; the third layer of convolution layer uses 384 convolution filters with convolution kernel size of 3 multiplied by 3 and step length of 1 to filter;
sending the filtering results of the first layer of convolutional layer, the second layer of convolutional layer and the third layer of convolutional layer into a maximum pooling layer, wherein the maximum pooling layer is set with a pooling window of 3 x 3 and a step length of 2;
and obtaining the detection result of the animal body shape by passing the output result of the maximum pooling layer through three full-connection layers.
According to another aspect of the present invention, there is also provided an animal body shape detecting apparatus including:
the first image feature map module is used for extracting features of an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
the second image feature map module is used for registering the depth image of the animal with the first image feature map to obtain a second image feature map;
the candidate target area module is used for superposing the first image characteristic diagram and the second image characteristic diagram and obtaining a candidate target area according to the superposed characteristic fusion diagram; and
and the body shape detection result module is used for inputting the candidate target area into the convolutional neural network to obtain the detection result of the animal body shape.
According to another aspect of the present invention, there is provided a non-transitory computer readable storage medium, characterized in that the non-transitory computer readable storage medium stores computer instructions for causing the computer to execute the method of the convolutional neural network-based animal body shape detection method of the present invention and any optional embodiment thereof.
The invention provides an animal body shape detection method based on a convolutional neural network, which comprises the steps of extracting features of an original image of an animal, registering the extracted features with a depth image of the animal to obtain a feature map of the depth image, fusing the two feature maps to obtain a candidate region, and inputting the obtained candidate region into a discrimination model of the convolutional neural network so that the discrimination model of the convolutional network outputs an animal body shape detection result, and the accuracy and the real-time performance of animal body shape detection can be improved.
Drawings
FIG. 1 is a schematic flow chart of a method for detecting an animal body shape based on a convolutional neural network according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of an animal body shape detection device based on a convolutional neural network according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a frame of an electronic device according to an embodiment of the invention.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
Fig. 1 is a schematic flow chart of an animal body shape detection method based on a convolutional neural network according to an embodiment of the present invention, where the animal body shape detection method based on the convolutional neural network shown in fig. 1 includes:
s100, extracting the features of an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
the Gabor sequencing feature of the embodiment of the invention is a feature which can be used for describing image texture information, and the frequency and the direction of a Gabor filter are similar to the human visual system, so that the Gabor sequencing feature is particularly suitable for texture representation and discrimination.
S200, registering the depth image of the animal with the first image feature map to obtain a second image feature map;
the depth image (depth image) according to the embodiment of the present invention is also referred to as a range image (range image), and refers to an image in which the distance (depth) from an image collector to each point in a scene is used as a pixel value, and directly reflects the geometric shape of a visible surface of an object in the image.
In the embodiment of the present invention, the original image and the depth image in step S100 and step S200 are images from the same animal. And the second image feature map is obtained by registering the depth image of the animal and the image feature map of the original image through affine transformation.
S300, overlapping the first image feature map and the second image feature map, and obtaining a candidate target area according to the overlapped feature fusion map;
and S400, inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape.
The invention provides an animal body shape detection method based on a convolutional neural network, which comprises the steps of extracting features of an original image of an animal, registering the extracted features with a depth image of the animal to obtain a feature map of the depth image, fusing the two feature maps to obtain a candidate region, and inputting the obtained candidate region into a discrimination model of the convolutional neural network so that the discrimination model of the convolutional network outputs an animal body shape detection result, and the accuracy and the real-time performance of animal body shape detection can be improved.
In an optional embodiment, in step S100, the extracting features of the original image of the animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map specifically includes:
s100.1, converting the original image into a first gray image, and carrying out Gabor filtering on the first gray image by adopting a Gabor filter with a scale parameter sigma of 5 to obtain amplitude characteristic images of animal body shapes in multiple directions;
s100.2, coding the amplitude characteristic images of the animal body shapes in the multiple directions by utilizing a sequencing filter to obtain GOM characteristic coded images of the animal body shapes in the multiple directions;
and S100.3, obtaining the GOM feature coded image of the animal body shape according to the GOM feature coded images of the animal body shape in the multiple directions, and using the GOM feature coded image of the animal body shape as the first image feature map.
Specifically, the expression of the Gabor filter is as follows:
Figure GDA0002385912860000071
where σ represents the scale parameter of the Gabor filter, θkAn angle value representing a k-th direction;
Figure GDA0002385912860000072
as a scale parameter in the x-direction of the Gabor filter,
Figure GDA0002385912860000073
as a scale parameter in the y-direction of the Gabor filter,
Figure GDA0002385912860000074
for the value of x for the k-th angular direction,
Figure GDA0002385912860000075
is the value of y, f, in the k-th angular directionkIs the center frequency in the k direction;
specifically, the expression of the sequence filter is:
Figure GDA0002385912860000081
where ω and represents the center position and scale of the sequencing filter, N, respectivelypNumber of positive lobes, NnNumber of negative lobes, coefficient CpAnd CnIs a coefficient that keeps the number of positive and negative lobes balanced, and NpCp＝NnCn；piIs a scale parameter of the positive lobe,njscale parameter of the negative lobe, ωpiIs the central position of the positive lobe, ωnjIs the central position of the negative valve
Specifically, since the three-lobe sequencing filter is robust, let C be the same as the three-lobe sequencing filter in the present embodimentp＝1,Np＝2；Cn＝2,Nn＝1。
Specifically, the plurality of directions are 8 directions, including 0 °,22.5 °,45 °,67.5 °,90 °,112.5 °,135 °, and 157.5 °.
Specifically, in this embodiment, amplitude characteristic images of animal figures in 8 directions are obtained by using a Gabor filter shown in formula (1); then, coding the amplitude characteristic images of the animal body shapes in the 8 directions by using a sequencing filter shown in a formula (2) to obtain GOM characteristic coded images of the animal body shapes in the 8 directions; finally, the GOM feature coded images with the animal body shapes in the 8 directions are merged into a GOM feature coded image with the animal body shape as the first image feature map, and the method comprises the following steps:
Figure GDA0002385912860000082
where k represents the kth direction, where oritations is 8 in this embodiment, and Ordinal (k) represents the result of filtering the animal body shape amplitude feature image in the kth direction by the sequencing filter.
In an optional embodiment, in step S200, the registering the depth image of the animal with the first image feature map to obtain a second image feature map specifically includes:
registering the depth image and the first image feature map in an affine transformation mode to obtain an image feature map of the depth image, wherein the affine transformation formula is as follows:
Figure GDA0002385912860000091
wherein x is1Is the value of the first characteristic image in the x direction, y1Is a value in the y direction of the first feature image, x2Is the value of the second characteristic image in the x direction, y2Is the value of the y direction of the second feature image, txIs the translation value, t, in the x directionyThe value of the translation in the y direction, s the scaling, and θ the counterclockwise rotation angle with (x, y) as the axis.
In an optional embodiment, in step S300, the superimposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superimposed feature fusion map specifically includes:
compressing the gray value of the pixel point in the second image characteristic image to a gray value range consistent with the gray value range of the first image characteristic image;
based on the compressed second image characteristic map and the corresponding pixel points in the first image characteristic map, selecting the gray value of the pixel point with a larger gray value as the gray value of the pixel point at the corresponding position of the fused image;
and obtaining the characteristic fusion image according to the gray values of the pixel points at the corresponding positions of all the fusion images, thereby obtaining and obtaining the candidate target area.
In this embodiment, the gray value of the pixel point of the compressed second image feature map is compared with the gray value of the corresponding pixel point in the first image feature map, that is, two corresponding pixel points are compared, where the two pixel points include a pixel point of the first image feature map and a corresponding pixel point on the second image feature map; and selecting the gray value of the pixel point with the larger gray value in the two pixel points as the gray value of the pixel point at the corresponding position of the fused image. And after all pixel points of the compressed second image characteristic diagram and the first image characteristic diagram are compared in this way, all the compared pixel points with larger gray values are obtained, so that a fusion image is constructed, and a candidate target region is obtained.
Further, compressing the gray value of the pixel point in the second image feature map to a gray value range consistent with the gray value range of the first image feature map further comprises: if the gray value of the compressed second image feature map is non-integer, performing approximate calculation through the following formula to obtain an approximate gray value Ia(x,y)：
Figure GDA0002385912860000101
And I (x, y) represents the gray value of the pixel point of the second image characteristic diagram.
In this embodiment, the gray scale value of the compressed second image feature map is likely not an integer, and for the gray scale value that is not an integer, the approximate calculation is performed by a rounding method; after the approximate calculation, the gray values of the compressed second image feature map can be guaranteed to be integers, so that the subsequent fusion calculation is facilitated.
In an alternative embodiment, the convolutional neural network comprises three convolutional layers, three pooling layers and three fully-connected layers; step S400, inputting the candidate target region into a convolutional neural network to obtain a detection result of the animal body shape, which specifically includes:
the first layer of convolutional layer is filtered by using 96 convolutional filters with the convolutional kernel size of 11 multiplied by 11 and the step size of 4; the second convolution layer uses 256 convolution filters with convolution kernel size of 5 multiplied by 5 and step length of 1 to filter; the third layer of convolution layer uses 384 convolution filters with convolution kernel size of 3 multiplied by 3 and step length of 1 to filter;
sending the filtering results of the first layer of convolutional layer, the second layer of convolutional layer and the third layer of convolutional layer into a maximum pooling layer, wherein the maximum pooling layer is set with a pooling window of 3 x 3 and a step length of 2;
and obtaining the detection result of the animal body shape by passing the output result of the maximum pooling layer through three full-connection layers.
The embodiment of the invention obtains the image characteristic diagram of the original image by processing the original image; simultaneously registering the depth image with the image feature map of the original image to obtain an image feature map of the depth image; superposing the image characteristic diagram of the original image and the image characteristic diagram of the depth image to obtain a multi-source image characteristic fusion diagram to obtain a candidate target area; and inputting the candidate target area into a discrimination model of the convolutional neural network so that the convolutional network discrimination model outputs an animal body shape detection result. The invention improves the accuracy and the real-time property of the animal body shape detection.
Fig. 2 is a schematic diagram of an animal body shape detection device based on a convolutional neural network according to an embodiment of the present invention, and the animal body shape detection device based on the convolutional neural network shown in fig. 2 includes:
the first image feature map module is used for extracting features of an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
the second image feature map module is used for registering the depth image of the animal with the first image feature map to obtain a second image feature map;
the candidate target area module is used for superposing the first image characteristic diagram and the second image characteristic diagram and obtaining a candidate target area according to the superposed characteristic fusion diagram; and
and the temperature detection result module is used for inputting the candidate target area into the convolutional neural network to obtain the detection result of the animal body shape.
The device of the embodiment of the invention can be used for executing the technical scheme of the embodiment of the animal body shape detection method based on the convolutional neural network shown in fig. 1, and the implementation principle and the technical effect are similar, and are not repeated here.
Fig. 3 shows a schematic diagram of a framework of an electronic device according to an embodiment of the invention.
Referring to fig. 3, the apparatus includes: a processor (processor)601, a memory (memory)602, and a bus 603; wherein, the processor 601 and the memory 602 complete the communication with each other through the bus 603;
the processor 601 is configured to call program instructions in the memory 602 to perform the methods provided by the above-mentioned method embodiments, for example, including: carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map; registering the depth image of the animal with the first image feature map to obtain a second image feature map; superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map; and inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape.
Another embodiment of the present invention discloses a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the methods provided by the above-mentioned method embodiments, for example, including: carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map; registering the depth image of the animal with the first image feature map to obtain a second image feature map; superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map; and inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape.
Another embodiment of the invention provides a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform a method provided by the above method embodiments, for example, comprising: carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map; registering the depth image of the animal with the first image feature map to obtain a second image feature map; superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map; and inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape.
Those of ordinary skill in the art will understand that: the implementation of the above-described apparatus embodiments or method embodiments is merely illustrative, wherein the processor and the memory may or may not be physically separate components, i.e. may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (8)

1. A method for detecting the shape of an animal, comprising:
carrying out feature extraction on an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
registering the depth image of the animal with the first image feature map to obtain a second image feature map;
superposing the first image feature map and the second image feature map, and obtaining a candidate target region according to the superposed feature fusion map;
inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape;
registering the depth image of the animal with the first image feature map to obtain a second image feature map, specifically comprising:
registering the depth image and the first image feature map in an affine transformation mode to obtain an image feature map of the depth image, wherein the affine transformation formula is as follows:
Figure FDA0002385912850000011
wherein x is1Is the value of the first characteristic image in the x direction, y1Is a value in the y direction of the first feature image, x2Is the value of the second characteristic image in the x direction, y2Is the value of the y direction of the second feature image, txIs the translation value, t, in the x directionyThe value of the translation in the y direction, s the scaling, and θ the counterclockwise rotation angle with (x, y) as the axis.
2. The method according to claim 1, wherein the extracting features of the original image of the animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map specifically comprises:
converting the original image into a first gray image, and carrying out Gabor filtering on the first gray image by adopting a Gabor filter with a scale parameter sigma of 5 to obtain amplitude characteristic images of animal figures in multiple directions;
utilizing a sequencing filter to encode the amplitude characteristic images of the animal body shapes in the multiple directions to obtain GOM characteristic encoded images of the animal body shapes in the multiple directions;
and obtaining the GOM feature coded image of the animal body shape according to the GOM feature coded images of the animal body shape in the multiple directions, wherein the GOM feature coded image of the animal body shape is used as the first image feature map.
3. The method of claim 2, wherein the Gabor filter is expressed by:
Figure FDA0002385912850000021
where σ represents the scale parameter of the Gabor filter, θkAn angle value representing a k-th direction;
Figure FDA0002385912850000022
as a scale parameter in the x-direction of the Gabor filter,
Figure FDA0002385912850000023
as a scale parameter in the y-direction of the Gabor filter,
Figure FDA0002385912850000024
for the value of x for the k-th angular direction,
Figure FDA0002385912850000025
is the value of y, f, in the k-th angular directionkIs the center frequency in the k direction;
the expression of the sequencing filter is:
Figure FDA0002385912850000026
where ω and represents the center position and scale of the sequencing filter, N, respectivelypNumber of positive lobes, NnNumber of negative lobes, coefficient CpAnd CnIs a coefficient that keeps the number of positive and negative lobes balanced, and NpCp＝NnCn；piIs a scale parameter of the positive lobe,njscale parameter of the negative lobe, ωpiIs the central position of the positive lobe, ωnjIs the central position of the negative lobe;
the plurality of directions, specifically 8 directions, includes 0 °,22.5 °,45 °,67.5 °,90 °,112.5 °,135 ° and 157.5 °.
4. The method according to claim 1, wherein the superimposing the first image feature map and the second image feature map and obtaining the candidate target region according to the superimposed feature fusion map specifically include:
compressing the gray value of the pixel point in the second image characteristic image to a gray value range consistent with the gray value range of the first image characteristic image;
based on the compressed second image characteristic map and the corresponding pixel points in the first image characteristic map, selecting the gray value of the pixel point with a larger gray value as the gray value of the pixel point at the corresponding position of the fused image;
and obtaining the characteristic fusion image according to the gray values of the pixel points at the corresponding positions of all the fusion images, thereby obtaining and obtaining the candidate target area.
5. The method of claim 4, wherein the compressing the gray scale values of the pixels in the second image feature map to a gray scale value range consistent with the first image feature map further comprises: if the gray value of the compressed second image feature map is non-integer, performing approximate calculation through the following formula to obtain an approximate gray value Ia(x,y)：
Figure FDA0002385912850000031
And I (x, y) represents the gray value of the pixel point of the second image characteristic diagram.
6. The method of claim 1, wherein the convolutional neural network comprises three convolutional layers, three pooling layers, three fully-connected layers; inputting the candidate target area into a convolutional neural network to obtain a detection result of the animal body shape, which specifically comprises:
the first layer of convolutional layer is filtered by using 96 convolutional filters with the convolutional kernel size of 11 multiplied by 11 and the step size of 4; the second convolution layer uses 256 convolution filters with convolution kernel size of 5 multiplied by 5 and step length of 1 to filter; the third layer of convolution layer uses 384 convolution filters with convolution kernel size of 3 multiplied by 3 and step length of 1 to filter;
sending the filtering results of the first layer of convolutional layer, the second layer of convolutional layer and the third layer of convolutional layer into a maximum pooling layer, wherein the maximum pooling layer is set with a pooling window of 3 x 3 and a step length of 2;
and obtaining the detection result of the animal body shape by passing the output result of the maximum pooling layer through three full-connection layers.
7. An animal body shape detection device, comprising:
the first image feature map module is used for extracting features of an original image of an animal by using a Gabor sequencing feature descriptor with illumination invariance to obtain a first image feature map;
the second image feature map module is used for registering the depth image of the animal with the first image feature map to obtain a second image feature map; registering the depth image of the animal with the first image feature map to obtain a second image feature map, specifically comprising:
registering the depth image and the first image feature map in an affine transformation mode to obtain an image feature map of the depth image, wherein the affine transformation formula is as follows:
Figure FDA0002385912850000041
wherein x is1Is the value of the first characteristic image in the x direction, y1Is a value in the y direction of the first feature image, x2Is the value of the second characteristic image in the x direction, y2Is the value of the y direction of the second feature image, txIs the translation value, t, in the x directionyThe value is a translation value in the y direction, s is a scaling scale, and theta is a counterclockwise rotation angle taking (x, y) as an axis;
the candidate target area module is used for superposing the first image characteristic diagram and the second image characteristic diagram and obtaining a candidate target area according to the superposed characteristic fusion diagram; and
and the body shape detection result module is used for inputting the candidate target area into the convolutional neural network to obtain the detection result of the animal body shape.
8. A non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the method of any one of claims 1 to 6.
CN201810019554.4A 2018-01-09 2018-01-09 Animal body shape detection method and device based on convolutional neural network Expired - Fee Related CN108388830B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810019554.4A CN108388830B (en) 2018-01-09 2018-01-09 Animal body shape detection method and device based on convolutional neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810019554.4A CN108388830B (en) 2018-01-09 2018-01-09 Animal body shape detection method and device based on convolutional neural network

Publications (2)

Publication Number Publication Date
CN108388830A CN108388830A (en) 2018-08-10
CN108388830B true CN108388830B (en) 2020-08-14

Family

ID=63076883

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810019554.4A Expired - Fee Related CN108388830B (en) 2018-01-09 2018-01-09 Animal body shape detection method and device based on convolutional neural network

Country Status (1)

Country Link
CN (1) CN108388830B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245621B (en) * 2019-06-17 2023-10-17 深圳Tcl新技术有限公司 Face recognition device, image processing method, feature extraction model, and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102629320A (en) * 2012-03-27 2012-08-08 中国科学院自动化研究所 Ordinal measurement statistical description face recognition method based on feature level
CN103248703A (en) * 2013-05-16 2013-08-14 中国农业大学 Automatic monitoring system and method for live pig action
CN103745474A (en) * 2014-01-21 2014-04-23 南京理工大学 Image registration method based on inertial sensor and camera
CN105160305A (en) * 2015-08-10 2015-12-16 中国民航大学 Finger multi-mode characteristic fusion method
CN105354555A (en) * 2015-11-17 2016-02-24 南京航空航天大学 Probabilistic graphical model-based three-dimensional face recognition method
CN106408597A (en) * 2016-09-08 2017-02-15 西安电子科技大学 Neighborhood entropy and consistency detection-based SAR (synthetic aperture radar) image registration method
CN107180241A (en) * 2017-04-20 2017-09-19 华南理工大学 A kind of animal classification method of the profound neutral net based on Gabor characteristic with fractal structure

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8798374B2 (en) * 2008-08-26 2014-08-05 The Regents Of The University Of California Automated facial action coding system
JP2016134803A (en) * 2015-01-20 2016-07-25 キヤノン株式会社 Image processor and image processing method
US10121245B2 (en) * 2015-09-14 2018-11-06 University Of Notre Dame Identification of inflammation in tissue images
US10275641B2 (en) * 2015-10-01 2019-04-30 Intellivision Technologies Corp Methods and systems for extracting feature descriptors for an image

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102629320A (en) * 2012-03-27 2012-08-08 中国科学院自动化研究所 Ordinal measurement statistical description face recognition method based on feature level
CN103248703A (en) * 2013-05-16 2013-08-14 中国农业大学 Automatic monitoring system and method for live pig action
CN103745474A (en) * 2014-01-21 2014-04-23 南京理工大学 Image registration method based on inertial sensor and camera
CN105160305A (en) * 2015-08-10 2015-12-16 中国民航大学 Finger multi-mode characteristic fusion method
CN105354555A (en) * 2015-11-17 2016-02-24 南京航空航天大学 Probabilistic graphical model-based three-dimensional face recognition method
CN106408597A (en) * 2016-09-08 2017-02-15 西安电子科技大学 Neighborhood entropy and consistency detection-based SAR (synthetic aperture radar) image registration method
CN107180241A (en) * 2017-04-20 2017-09-19 华南理工大学 A kind of animal classification method of the profound neutral net based on Gabor characteristic with fractal structure

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Gabor Ordinal Measures for Face Recognition;Zhenhua Chai等;《IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY》;20140131;第14-26页 *
手指特征粒的鲁棒性内涵表达方法研究;仲贞;《中国优秀硕士学位论文全文数据库》;20170315;第I138-5337页 *

Also Published As

Publication number Publication date
CN108388830A (en) 2018-08-10

Similar Documents

Publication Publication Date Title
CN110135455B (en) Image matching method, device and computer readable storage medium
Tao et al. Spatial information inference net: Road extraction using road-specific contextual information
Cai et al. Exploiting spatial-temporal relationships for 3d pose estimation via graph convolutional networks
CN108428224B (en) Animal body surface temperature detection method and device based on convolutional neural network
Wang et al. 360sd-net: 360 stereo depth estimation with learnable cost volume
CN109522874B (en) Human body action recognition method and device, terminal equipment and storage medium
Li et al. Deep learning based imaging data completion for improved brain disease diagnosis
CN109416727B (en) Method and device for removing glasses in face image
Sigal Human pose estimation
KR20210028226A (en) Automatic determination of the normal posture of 3D objects and the superposition of 3D objects using deep learning
Zhou et al. Image classification using biomimetic pattern recognition with convolutional neural networks features
Shen et al. Accurate point cloud registration with robust optimal transport
Chen et al. Cross parallax attention network for stereo image super-resolution
CN111274916A (en) Face recognition method and face recognition device
CN111507908B (en) Image correction processing method, device, storage medium and computer equipment
CN113043267A (en) Robot control method, device, robot and computer readable storage medium
CN105335952B (en) Matching power flow computational methods and device and parallax value calculating method and equipment
CN113409384A (en) Pose estimation method and system of target object and robot
CN110070610B (en) Feature point matching method, and feature point matching method and device in three-dimensional reconstruction process
CN114219855A (en) Point cloud normal vector estimation method and device, computer equipment and storage medium
CN114219890A (en) Three-dimensional reconstruction method, device and equipment and computer storage medium
CN111582220B (en) Bone point behavior recognition system based on shift map convolution neural network and recognition method thereof
CN110942512A (en) Indoor scene reconstruction method based on meta-learning
CN111524232A (en) Three-dimensional modeling method and device and server
Fuentes-Jimenez et al. Texture-generic deep shape-from-template

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200814

Termination date: 20210109

CF01 Termination of patent right due to non-payment of annual fee