CN109670440A - The recognition methods of giant panda face and device - Google Patents

The recognition methods of giant panda face and device Download PDF

Info

Publication number
CN109670440A
CN109670440A CN201811533790.4A CN201811533790A CN109670440A CN 109670440 A CN109670440 A CN 109670440A CN 201811533790 A CN201811533790 A CN 201811533790A CN 109670440 A CN109670440 A CN 109670440A
Authority
CN
China
Prior art keywords
region
key
face
panda
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201811533790.4A
Other languages
Chinese (zh)
Other versions
CN109670440B (en
Inventor
董培祥
朱立松
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
CCTV INTERNATIONAL NETWORKS WUXI Co Ltd
Original Assignee
CCTV INTERNATIONAL NETWORKS WUXI Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by CCTV INTERNATIONAL NETWORKS WUXI Co Ltd filed Critical CCTV INTERNATIONAL NETWORKS WUXI Co Ltd
Priority to CN201811533790.4A priority Critical patent/CN109670440B/en
Publication of CN109670440A publication Critical patent/CN109670440A/en
Application granted granted Critical
Publication of CN109670440B publication Critical patent/CN109670440B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G06V40/171—Local features and components; Facial parts ; Occluding parts, e.g. glasses; Geometrical relationships
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of recognition methods of giant panda face and devices, wherein includes: the key area that S10 identifies the facial area of giant panda face from images to be recognized and identifies default key position from facial area in the recognition methods of giant panda face；S20 extracts global characteristics from facial area；S30 successively extracts local feature from the corresponding key area of each key position；S40 extracts geometrical characteristic according in facial area and the corresponding key area of each key position；S50 completes the identification to giant panda face in image according to the global characteristics, local feature and geometrical characteristic of extraction.While it is comprehensively described from image of the different dimensions to input, according to the exclusive feature of giant panda face, geometrical characteristic this description is proposed, except calculating speed is fast, recognition efficiency is high, substantially increases the accuracy rate of identification.

Description

Panda face identification method and device
Technical Field
The invention relates to the technical field of computer vision, in particular to a panda face identification method and device.
Background
Giant pandas are national treasures in China, survive for at least three million years on the earth, most ancient organisms in the same period as the giant pandas are killed, successfully adapt to the climate change of the earth, survive to the present, are called as 'activated stones', and are first-class protective animals in China. According to the investigation result of the fourth panda nationwide published by the national forestry administration in 2015, the number of wild panda nationwide is 1864 and the number of captive panda is 375 by 2013.
At present, in the process of image processing, the identification of pandas still depends on human eyes, but because a human visual system is insensitive to animal faces, the identification of the human eyes is difficult, the efficiency is low, the cost is high, and the identification difficulty of wild pandas is larger. In recent years, with the rapid development of artificial intelligence technology, the face recognition technology has substantially progressed, the test accuracy of the current advanced face recognition algorithm on the public data set is over 99%, and the face recognition algorithm is widely applied in practice. Animal face recognition is relatively much less studied than face recognition. Chinese patent CN103065129A discloses a panda identification method, which relates to panda identification, but is not panda face identification but is panda identification of the whole panda.
Disclosure of Invention
Aiming at the defects of the prior art, the invention provides a panda face identification method and device, which effectively solve the technical problem that the panda face cannot be accurately identified in the prior art.
In order to achieve the purpose, the invention is realized by the following technical scheme:
a panda face recognition method comprises the following steps:
s10 identifying the face area of the panda face from the image to be identified and identifying the key area of the preset key part from the face area;
s20 extracting global features from the face region;
s30, extracting local features from the key areas corresponding to the key parts in sequence;
s40, extracting geometric features according to the face area and key areas corresponding to the key parts;
s50, according to the extracted global features, local features and geometric features, the panda face in the image is identified.
Further preferably, the key parts comprise eyes, a nose and a mouth;
before step S10, training a convolutional neural network separately for each key site;
in step S10, an eye region, a nose region, and a mouth region are identified from the face region using the convolutional neural network corresponding to each key part.
Further preferably, in step S30, the method further includes:
s31, determining a preset number of key points according to the key area corresponding to each key part;
s32, extracting the feature of the corresponding position in the image for each key point, and obtaining the local feature of the key area after fusion.
Further preferably, in step S40, the method further includes:
s41 obtaining an area ratio feature r according to the facial region and the eye regions：
Wherein S isfIs the area of the facial region, SreIs the area of the right eye region, SleIs the area of the left eye region;
s42 obtaining angle characteristics according to the eye region, the nose region and the mouth regionAnd
wherein,the angle between the nose region and the two eye regions,is the angle between the mouth region and the two eyes, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region.
Further preferably, in step S50, the method further includes:
s51, obtaining a feature vector of the panda face according to the extracted global features, local features and geometric features;
s52 sequentially calculating Euclidean distances between the feature vectors and all panda face feature vectors in the database to complete the identification of panda faces;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is represented by i ═ 0, 1.., M-1}, and M is the total number of panda faces in the database.
The invention also provides a panda face recognition device, which comprises:
the area identification module is used for identifying a face area of a panda face from an image to be identified and identifying a key area of a preset key part from the face area;
the feature extraction module is used for extracting global features from the face area, sequentially extracting local features from key areas corresponding to the key parts and extracting geometric features according to the face area and the key areas corresponding to the key parts;
and the panda recognition module is used for finishing recognition of panda faces in the images according to the extracted global features, local features and geometric features.
Further preferably, the key parts comprise eyes, a nose and a mouth; in the region identification module, eye regions, nose regions, and mouth regions are identified from the face region using a convolutional neural network corresponding to each key part.
Further preferably, in the extracting local features from the key regions corresponding to the key parts by the feature extracting module, the extracting local features specifically include: after a preset number of key points are determined in a key area corresponding to each key part, the features of corresponding positions in the image are extracted for each key point, and the local features of the key areas are obtained after fusion.
Further preferably, the extracting geometric features in the key region corresponding to the face region and each key part by the feature extracting module includes:
obtaining an area ratio feature r according to the facial region and the eye regions：
Wherein S isfBeing a facial regionArea, SreIs the area of the right eye region, SleIs the area of the left eye region;
obtaining angular characteristics from the eye region, nose region and mouth regionAnd
wherein,the angle between the nose region and the two eye regions,is the angle between the mouth region and the two eyes, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region.
Further preferably, the panda recognition module includes:
the calculation unit is used for obtaining the feature vectors of the panda faces according to the extracted global features, local features and geometric features, and sequentially calculating Euclidean distances between the feature vectors and all the panda face feature vectors in the database;
the identification unit is used for finishing the identification of the panda faces according to the calculation result of the calculation unit;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is represented by i ═ 0, 1.., M-1}, and M is the total number of panda faces in the database.
In the method and the device for the panda face, the global features, the local features and the geometric features are extracted from the face area of the identified panda and the key area corresponding to the key part, then the extracted features are fused, and the identity of the panda in the image is matched according to the fused features.
Drawings
A more complete understanding of the present invention, and the attendant advantages and features thereof, will be more readily understood by reference to the following detailed description when considered in conjunction with the accompanying drawings wherein:
FIG. 1 is a schematic flow chart of a panda face recognition method according to the present invention;
FIG. 2 is a diagram of a recognition network for detecting and locating panda facial regions in an image using the fast RCNN algorithm in accordance with the present invention;
FIG. 3 is a schematic diagram of image coordinates in the present invention;
FIG. 4 is a schematic diagram of key point extraction in a key region according to the present invention;
FIG. 5 is a schematic diagram of area ratio feature extraction in the present invention;
FIG. 6 is a schematic diagram of angular feature extraction according to the present invention;
fig. 7 is a schematic structural view of a panda face recognition device according to the present invention.
100-an identification device, 110-an area identification module, 120-a feature extraction module and 130-a panda identification module.
Detailed Description
In order to make the contents of the present invention more comprehensible, the present invention is further described below with reference to the accompanying drawings. The invention is of course not limited to this particular embodiment, and general alternatives known to those skilled in the art are also covered by the scope of the invention.
As shown in fig. 1, a schematic flow chart of a panda face recognition method provided by the present invention is shown, and as can be seen from the diagram, the recognition method includes:
s10 identifying the face area of the panda face from the image to be identified and identifying the key area of the preset key part from the face area;
s20 extracting global features from the face region;
s30, extracting local features from the key areas corresponding to the key parts in sequence;
s40, extracting geometric features according to the face area and the key areas corresponding to the key parts;
s50, according to the extracted global features, local features and geometric features, the panda face in the image is identified.
For an input image to be recognized, firstly, the position of the face of a panda is detected and positioned, and then whether the positioned position is the face of the panda is further confirmed. In the recognition method provided by the present invention, the fast RCNN algorithm is used to detect and locate the position of the face region of pandas in the image, as shown in fig. 2, the process of training the recognition network includes:
step 1: initializing a feature extraction network (ConvNet) by using an ImageNet model, and independently training a panda face region nomination network (RPN network);
step 2: initializing by using an ImageNet model, generating a candidate region of a panda face region by using an RPN (resilient packet network) obtained by step1, and training a panda face recognition network (Fast RCNN), wherein parameters of ConvNet and the RPN are not shared;
step 3: initializing a new RPN network by using the network of step2, setting the learning rate of ConvNets to 0 (no longer updating the parameters of ConvNets, only updating the RPN network layer), and retraining so that the ConvNets and the RPN network share all the common convolutional layers;
step 4: on the basis of step3, ConvNet is still fixed, Fast RCNN is also added, and a network layer specific to FastRCCNN is trained, wherein at the moment, ConvNet, RPN network and Fast RCNN share all the common convolutional layers, and the training of the network is completed.
In the process of identifying the face area of the panda by using a trained identification network, inputting an image to be identified, and outputting a face area boundary bbox _ pred and a face judgment confidence probability cls _ score, wherein the face area boundary bbox _ pred is a rectangular frame coordinate and comprises (x) coordinates1,y1) And (x)2,y2) Two sets of coordinates, (x)1,y1) Is the coordinate of the top left vertex of the rectangular box, (x)2,y2) Determining the position of the face area according to the coordinates of the right lower vertex of the rectangular frame; cls _ score represents the possibility that the currently selected face area automatically judged by the recognition network is a panda face, the value range is 0-1, in practice, a plurality of images are counted to obtain an experience threshold delta of the cls _ score and are set, and the confidence probability of the cls _ score is judged for the face obtained when one image is recognized>And delta, considering the currently detected face area as the face of the panda, and then performing subsequent processing, or not performing the subsequent processing.
After the face area of the panda is identified from the image to be identified, a pre-trained Convolutional Neural Network (CNN) is used for identifying a key area of a preset key part, wherein the key part comprises eyes (a left eye and a right eye), a nose and a mouth. Before detection, a convolutional neural network is trained for each key part individually, so that the convolutional neural networks corresponding to the key parts are used for identifying an eye region, a nose region and a mouth region from a face region. The eye region, nose region and mouth region are respectively represented by a point set, specifically, the point set corresponding to the left eye region is represented by gammaleIndicating that the point set corresponding to the right eye region is represented by gammareIndicating that the points of the nose region are set by gammanIndicating that the points of the mouth region are set by gammamThe set of coordinates of all points in the area corresponding to the element in each point set, i.e., γ { (u)i,vi) And the coordinates of each point in the set are determined according to a preset origin of coordinates, for example, as shown in fig. 3, the upper left corner of the image is the origin of coordinates.
And then, extracting global features from the face area by adopting a pre-trained CNN network, inputting an image corresponding to the panda face area in the extraction process, and obtaining the global features with preset quantity after the features are extracted by the CNN network. For example, in an example, a 4096-dimensional global feature is preset to be output, a CNN network is constructed according to the quantity, a panda face in a database is used to train the CNN network, and then an image corresponding to a panda face area in an image to be recognized is input, so that a 4096-dimensional global feature f is obtainedc＝(x0,x1,...,x4095) In other examples, the dimensions of the global features may be set according to actual conditions, such as 1024, 2048, 8192, and the like.
And then, extracting local features from the key regions corresponding to the key parts, wherein in the process, firstly, a preset number of key points are determined according to the key regions corresponding to the key parts, then, for each key point, the features of the corresponding position in the image are extracted, and the local features of the key regions are obtained after fusion. For the extraction of the key points, a CNN network is established for each key area (eye area, mouth area and nose area) (the CNN network is established according to the dimension of the key points to be extracted in each preset key area and trained) for detection and extraction. Inputting an image corresponding to the corresponding key area aiming at the extraction of the key point in each key area, for example, when the key point of the right eye area is extracted, inputting an image corresponding to the right eye area; when the key points of the nose region are extracted, the image corresponding to the nose region is input. In an example, as shown in fig. 4, the key points of the eye region, the nose region and the mouth region are respectively detected according to three pre-established CNN networks, and 41 key points are extracted in total, wherein 26 key points are extracted from the eye region, the right eye region corresponds to the key points with numbers 1 to 13, the left eye region corresponds to the key points with numbers 14 to 26, the nose region corresponds to the 8 key points with numbers 27 to 34, and the mouth region corresponds to the 7 key points with numbers 35 to 41. In other examples, the number of extracted key points in each key area is set according to actual conditions, and then a corresponding CNN network is established according to the number of set key points and training is performed.
After extracting the key points, extracting a scale-invariant feature transform (SIFT) feature descriptor of the image for each key point, wherein the SIFT feature extracts a local feature of the image, has invariance of scale and rotation, and each SIFT descriptor is 128-dimensional. For all SIFT feature descriptors, using BoW (Bag of Words) model to represent and generate local features, such as generating 4096-dimensional local feature fl＝(x0,x1,...,x4095). It should be noted that the dimensions of the extracted global features and the dimensions of the extracted local features may be set according to the purpose of expression as required, and may be the same or different, for example, in an example, the dimensions of the global features and the dimensions of the local features are both set to 4096 dimensions; for another example, the dimension of the global feature is set to 2048, the dimensions of the local features are all set to 4096, and the like.
Because the panda face has uniqueness different from other animal faces, the face has very definite black and white area distribution, and the distribution of the areas has certain difference among different individuals, the geometric features of the characteristic face are extracted by utilizing the area distribution so as to improve the identification capability of the system.
Specifically, an area ratio feature r is obtained from the face region and the eye regionsNamely, the proportion of the area of the eye region to the whole face region is calculated:
wherein S isfIs the area of the facial region, SreIs the area of the right eye region, SleThe area of each region is determined according to the number of pixels included in the image, as shown in fig. 5.
The angular characteristics includeAndas shown in figure 6 of the drawings,the angle between the nose region and the two eye regions,is the angle between the mouth region and the two eyes, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region. For a region γ { (u)i,vi) }, geometric centerSeat ofMark ucAnd vcThe calculation method of (a) is that,
integrating the extracted area bit and angle features to obtain geometric features
Extracting global feature f of panda facecLocal feature flAnd geometrical characteristics fgThen, the feature vector of the panda face is obtainedCalculating the feature vectors F in turntF is between the facial feature vectors of all pandas in the databaseiComparing the Euclidean distances between the large cats in the image, wherein the large cat in the image is the panda with the minimum Euclidean distance in the database, and finishing the identification of the panda face;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is i ═ 0, 1., M-1}, M is the total number of panda faces in the database, | | | Ft-Fi||2As feature vector FtAnd between feature vectors FiThe euclidean distance between them.
As shown in fig. 7, which is a schematic structural diagram of a panda face recognition device according to the present invention, it can be seen that the recognition device 100 includes: the panda recognition method comprises an area recognition module 110, a feature extraction module 120 and a panda recognition module 130, wherein the feature extraction module 120 is respectively connected with the area recognition module 110 and the panda recognition module 130, and the area recognition module 110 is used for recognizing a face area of a panda face from an image to be recognized and recognizing a key area of a preset key part from the face area; the feature extraction module 120 is configured to extract global features from the face region, sequentially extract local features from the key regions corresponding to the key portions, and extract geometric features according to the face region and the key regions corresponding to the key portions; the panda recognition module 130 is configured to complete recognition of a panda face in the image according to the extracted global features, local features, and geometric features.
For an input image to be recognized, the area recognition module 110 first detects and locates the position of the face of a panda. In the recognition apparatus 100 provided by the present invention, the fast RCNN algorithm is used to detect and locate the position of the face region of panda in the image, as shown in fig. 2, the process of training the recognition network includes:
step 1: initializing a feature extraction network (ConvNet) by using an ImageNet model, and independently training a panda face region nomination network (RPN network);
step 2: initializing by using an ImageNet model, generating a candidate region of a panda face region by using an RPN (resilient packet network) obtained by step1, and training a panda face recognition network (Fast R-CNN), wherein parameters of ConvNet and the RPN are not shared;
step 3: initializing a new RPN network by using the network of step2, setting the learning rate of ConvNets to 0 (no longer updating the parameters of ConvNets, only updating the RPN network layer), and retraining so that the ConvNets and the RPN network share all the common convolutional layers;
step 4: on the basis of step3, ConvNet is still fixed, Fast RCNN is also added, and a network layer specific to FastRCCNN is trained, wherein at the moment, ConvNet, RPN network and Fast RCNN share all the common convolutional layers, and the training of the network is completed.
The region identification module 110 inputs an image to be identified and outputs a face region boundary bbox _ pred and a face judgment confidence probability cls _ score in the process of identifying the face region of the panda by using a trained identification network, wherein the face region boundary bbox _ pred is a rectangular box coordinate including (x)1,y1) And (x)2,y2) Two sets of coordinates, (x)1,y1) Is the coordinate of the top left vertex of the rectangular box, (x)2,y2) Determining the position of the face area according to the coordinates of the right lower vertex of the rectangular frame; cls _ score represents the possibility that the currently selected face area automatically judged by the recognition network is a panda face, the value range is 0-1, in practice, a plurality of images are counted to obtain an experience threshold delta of the cls _ score and are set, and the confidence probability of the cls _ score is judged for the face obtained when one image is recognized>And delta, considering the currently detected face area as the face of the panda, and then performing subsequent processing, or not performing the subsequent processing.
After the face area of the panda is identified from the image to be identified, the area identification module 110 further identifies a key area of a preset key part by using a pre-trained Convolutional Neural Network (CNN), specifically, the key part includes an eye (left eye and right eye), a nose and a mouth. Before detection, a convolutional neural network is trained for each key part individually, so that the convolutional neural networks corresponding to the key parts are used for identifying an eye region, a nose region and a mouth region from a face region. The eye region, nose region and mouth region are respectively represented by a point set, specifically, the point set corresponding to the left eye region is represented by gammaleIndicating that the point set corresponding to the right eye region is represented by gammareIndicating that the points of the nose region are set by gammanIndicating that the points of the mouth region are set by gammamThe set of coordinates of all points in the area corresponding to the element in each point set, i.e., γ { (u)i,vi) And the coordinates of each point in the set are determined according to a preset origin of coordinates, for example, as shown in fig. 3, the upper left corner of the image is the origin of coordinates.
Then, the feature extraction module 120 extracts global features from the face area by using a pre-trained CNN network, and in the extraction process, inputs an image corresponding to the panda face area, and obtains a preset number of global features after extracting features through the CNN network. Such as, in one exampleIf 4096-dimensional global features are preset to be output, a CNN network is constructed according to the quantity, and after a panda face in a database is used for training the CNN network, an image corresponding to a panda face area in an image to be recognized is input to obtain 4096-dimensional global features fc＝(x0,x1,...,x4095) In other examples, the dimensions of the global features may be set according to actual conditions, such as 1024, 2048, 8192, and the like.
Then, the feature extraction module 120 extracts local features from the key regions corresponding to the key portions, and in this process, first, a preset number of key points are determined according to the key regions corresponding to each key portion, and then, for each key point, features at corresponding positions in the image are extracted, and the local features of the key regions are obtained after fusion. For the extraction of the key points, a CNN network is established for each key area (eye area, mouth area and nose area) (the CNN network is established according to the dimension of the key points to be extracted in each preset key area and trained) for detection and extraction. Inputting an image corresponding to the corresponding key area aiming at the extraction of the key point in each key area, for example, when the key point of the right eye area is extracted, inputting an image corresponding to the right eye area; when the key points of the nose region are extracted, the image corresponding to the nose region is input. In an example, as shown in fig. 4, the key points of the eye region, the nose region and the mouth region are respectively detected according to three pre-established CNN networks, and 41 key points are extracted in total, wherein 26 key points are extracted from the eye region, the right eye region corresponds to the key points with numbers 1 to 13, the left eye region corresponds to the key points with numbers 14 to 26, the nose region corresponds to the 8 key points with numbers 27 to 34, and the mouth region corresponds to the 7 key points with numbers 35 to 41. In other examples, the number of extracted key points in each key area is set according to actual conditions, and then a corresponding CNN network is established according to the number of set key points and training is performed.
After extracting the key points, SIFT (scale-invariant feature transform) of the image is extracted for each key pointFeature transformation) feature descriptors, wherein SIFT features extract local features of the image, the SIFT features have invariance of scale and rotation, and each SIFT descriptor is 128-dimensional. For all SIFT feature descriptors, using BoW (Bag of Words) model to represent and generate local features, such as generating 4096-dimensional local feature fl＝(x0,x1,...,x4095). It should be noted that the dimensions of the extracted global features and the dimensions of the extracted local features may be set according to the purpose of expression as required, and may be the same or different, for example, in an example, the dimensions of the global features and the dimensions of the local features are both set to 4096 dimensions; for another example, the dimension of the global feature is set to 2048, the dimensions of the local features are all set to 4096, and the like.
Because the panda face has uniqueness different from other animal faces, the face has very definite black and white area distribution, and the distribution of the areas has certain difference among different individuals, the feature extraction module 120 further extracts the geometric features of the feature face by using the area distribution so as to improve the recognition capability of the system.
Specifically, an area ratio feature r is obtained from the face region and the eye regionsNamely, the proportion of the area of the eye region to the whole face region is calculated:
wherein S isfIs the area of the facial region, SreIs the area of the right eye region, SleThe area of each region is determined according to the number of pixels included in the image, as shown in fig. 5.
The angular characteristics includeAndas shown in figure 6 of the drawings,the angle between the nose region and the two eye regions,is the angle between the mouth region and the two eyes, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region. For a region γ { (u)i,vi) }, geometric centerCoordinate u ofcAnd vcThe calculation method of (a) is that,
integrating the extracted area bit and angle features to obtain geometric features
Extracting global feature f of panda facecLocal feature flAnd geometrical characteristics fgThen, the feature vector of the panda face is obtainedThe calculation unit in the panda recognition module calculates the characteristic vector F in turntF is between the facial feature vectors of all pandas in the databaseiComparing Euclidean distances between the pandas, finishing the identification of the panda faces by the identification unit according to the calculation result of the calculation unit, wherein the large cat in the image is the panda with the minimum Euclidean distance in the database;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is i ═ 0, 1., M-1}, M is the total number of panda faces in the database, | | | Ft-Fi||2As feature vector FtAnd between feature vectors FiThe euclidean distance between them.

Claims (10)

1. A panda face recognition method is characterized by comprising the following steps:
s10 identifying the face area of the panda face from the image to be identified and identifying the key area of the preset key part from the face area;
s20 extracting global features from the face region;
s30, extracting local features from the key areas corresponding to the key parts in sequence;
s40, extracting geometric features according to the face area and key areas corresponding to the key parts;
s50, according to the extracted global features, local features and geometric features, the panda face in the image is identified.
2. The identification method of claim 1,
the key parts comprise eyes, a nose and a mouth;
before step S10, training a convolutional neural network separately for each key site;
in step S10, an eye region, a nose region, and a mouth region are identified from the face region using the convolutional neural network corresponding to each key part.
3. The identification method according to claim 1 or 2, characterized in that in step S30, further comprising:
s31, determining a preset number of key points according to the key area corresponding to each key part;
s32, extracting the feature of the corresponding position in the image for each key point, and obtaining the local feature of the key area after fusion.
4. The identification method according to claim 2, wherein in step S40, further comprising:
s41 obtaining an area ratio feature r according to the facial region and the eye regions：
Wherein S isfIs the area of the facial region, SreIs the area of the right eye region, SleIs the area of the left eye region;
s42 obtaining an angular characteristic theta from the eye region, the nose region and the mouth regionmAnd thetan：
θm＝∠crecmcle
θn＝∠crecncle
Wherein, thetamIs the angle between the nose region and the two eye regions, θnIs the angle between the mouth region and the two eyes, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region.
5. The identification method according to claim 1, 2 or 4, characterized in that in step S50, further comprising:
s51, obtaining a feature vector of the panda face according to the extracted global features, local features and geometric features;
s52 sequentially calculating Euclidean distances between the feature vectors and all panda face feature vectors in the database to complete the identification of panda faces;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is represented by i ═ 0, 1.., M-1}, and M is the total number of panda faces in the database.
6. A panda face recognition device, comprising:
the area identification module is used for identifying a face area of a panda face from an image to be identified and identifying a key area of a preset key part from the face area;
the feature extraction module is used for extracting global features from the face area, sequentially extracting local features from key areas corresponding to the key parts and extracting geometric features according to the face area and the key areas corresponding to the key parts;
and the panda recognition module is used for finishing recognition of panda faces in the images according to the extracted global features, local features and geometric features.
7. The identification device of claim 6 wherein the critical portions include eyes, nose and mouth; in the region identification module, eye regions, nose regions, and mouth regions are identified from the face region using a convolutional neural network corresponding to each key part.
8. The identification device according to claim 6 or 7, wherein, in the extracting local features from the key region corresponding to each key part by the feature extracting module, the extracting local features specifically include: after a preset number of key points are determined in a key area corresponding to each key part, the features of corresponding positions in the image are extracted for each key point, and the local features of the key areas are obtained after fusion.
9. The recognition apparatus according to claim 7, wherein the feature extraction module, in extracting geometric features from the face region and the key regions corresponding to the key parts, comprises:
obtaining an area ratio feature r according to the facial region and the eye regions：
Wherein S isfIs the area of the facial region, SreIs the area of the right eye region, SleIs the area of the left eye region;
obtaining an angular characteristic theta from the eye region, the nose region and the mouth regionmAnd thetan：
θm＝∠crecmcle
θn＝∠crecncle
Wherein, thetamIs the angle between the nose region and the two eye regions, θnIs the mouth region and two eyesAngle between the parts, creIs the geometric center of the right eye region, cleIs the geometric center of the left eye region, cmIs the geometric center of the mouth region, cnThe geometric center of the nose region.
10. The identification device according to claim 6, 7 or 9, wherein in the panda identification module, comprising:
the calculation unit is used for obtaining the feature vectors of the panda faces according to the extracted global features, local features and geometric features, and sequentially calculating Euclidean distances between the feature vectors and all the panda face feature vectors in the database;
the identification unit is used for finishing the identification of the panda faces according to the calculation result of the calculation unit;
p＝arg mini||Ft-Fi||2
wherein p is the serial number of the panda face in the image in the database, FtIs a feature vector of the panda face identified in the image, FiThe ith panda face feature vector in the database is represented by i ═ 0, 1.., M-1}, and M is the total number of panda faces in the database.
CN201811533790.4A 2018-12-14 2018-12-14 Identification method and device for big bear cat face Active CN109670440B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811533790.4A CN109670440B (en) 2018-12-14 2018-12-14 Identification method and device for big bear cat face

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811533790.4A CN109670440B (en) 2018-12-14 2018-12-14 Identification method and device for big bear cat face

Publications (2)

Publication Number Publication Date
CN109670440A true CN109670440A (en) 2019-04-23
CN109670440B CN109670440B (en) 2023-08-08

Family

ID=66143982

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811533790.4A Active CN109670440B (en) 2018-12-14 2018-12-14 Identification method and device for big bear cat face

Country Status (1)

Country Link
CN (1) CN109670440B (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245625A (en) * 2019-06-19 2019-09-17 山东浪潮人工智能研究院有限公司 A kind of field giant panda recognition methods and system based on twin neural network
CN111666881A (en) * 2020-06-08 2020-09-15 成都大熊猫繁育研究基地 Giant panda pacing, bamboo eating and oestrus behavior tracking analysis method
CN112016527A (en) * 2020-10-19 2020-12-01 成都大熊猫繁育研究基地 Panda behavior recognition method, system, terminal and medium based on deep learning
CN112926479A (en) * 2021-03-08 2021-06-08 新疆爱华盈通信息技术有限公司 Cat face identification method and system, electronic device and storage medium
CN115393904A (en) * 2022-10-20 2022-11-25 星宠王国(北京)科技有限公司 Dog nose print identification method and system
CN115909403A (en) * 2022-11-25 2023-04-04 天津大学四川创新研究院 Low-cost high-precision pig face identification method based on deep learning

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360421A (en) * 2011-10-19 2012-02-22 苏州大学 Face identification method and system based on video streaming
CN104573644A (en) * 2014-12-29 2015-04-29 天津瑞为拓新科技发展有限公司 Multi-mode face identification method
CN107066983A (en) * 2017-04-20 2017-08-18 腾讯科技（上海）有限公司 A kind of auth method and device

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102360421A (en) * 2011-10-19 2012-02-22 苏州大学 Face identification method and system based on video streaming
CN104573644A (en) * 2014-12-29 2015-04-29 天津瑞为拓新科技发展有限公司 Multi-mode face identification method
CN107066983A (en) * 2017-04-20 2017-08-18 腾讯科技（上海）有限公司 A kind of auth method and device

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110245625A (en) * 2019-06-19 2019-09-17 山东浪潮人工智能研究院有限公司 A kind of field giant panda recognition methods and system based on twin neural network
CN111666881A (en) * 2020-06-08 2020-09-15 成都大熊猫繁育研究基地 Giant panda pacing, bamboo eating and oestrus behavior tracking analysis method
CN112016527A (en) * 2020-10-19 2020-12-01 成都大熊猫繁育研究基地 Panda behavior recognition method, system, terminal and medium based on deep learning
CN112016527B (en) * 2020-10-19 2022-02-01 成都大熊猫繁育研究基地 Panda behavior recognition method, system, terminal and medium based on deep learning
CN112926479A (en) * 2021-03-08 2021-06-08 新疆爱华盈通信息技术有限公司 Cat face identification method and system, electronic device and storage medium
CN115393904A (en) * 2022-10-20 2022-11-25 星宠王国(北京)科技有限公司 Dog nose print identification method and system
CN115909403A (en) * 2022-11-25 2023-04-04 天津大学四川创新研究院 Low-cost high-precision pig face identification method based on deep learning
CN115909403B (en) * 2022-11-25 2023-08-22 天津大学四川创新研究院 Low-cost high-precision pig face recognition method based on deep learning

Also Published As

Publication number Publication date
CN109670440B (en) 2023-08-08

Similar Documents

Publication Publication Date Title
CN109670440B (en) Identification method and device for big bear cat face
CN107644204B (en) Human body identification and tracking method for security system
JP7113657B2 (en) Information processing device, information processing method, and program
CN109409297B (en) Identity recognition method based on dual-channel convolutional neural network
WO2015165365A1 (en) Facial recognition method and system
WO2019169532A1 (en) License plate recognition method and cloud system
US9489561B2 (en) Method and system for estimating fingerprint pose
CN112232117A (en) Face recognition method, face recognition device and storage medium
CN106407958B (en) Face feature detection method based on double-layer cascade
CN105426870A (en) Face key point positioning method and device
Bailo et al. Robust road marking detection and recognition using density-based grouping and machine learning techniques
CN111914642B (en) Pedestrian re-identification method, device, equipment and medium
CN108805016B (en) Head and shoulder area detection method and device
CN103577815A (en) Face alignment method and system
CN107506693A (en) Distort face image correcting method, device, computer equipment and storage medium
WO2021000832A1 (en) Face matching method and apparatus, computer device, and storage medium
CN112699834B (en) Traffic identification detection method, device, computer equipment and storage medium
CN114120188B (en) Multi-row person tracking method based on joint global and local features
CN110969052A (en) Operation correction method and equipment
CN106874877A (en) A kind of combination is local and global characteristics without constraint face verification method
CN106909879A (en) A kind of method for detecting fatigue driving and system
CN106599918A (en) Vehicle tracking method and system
CN103544478A (en) All-dimensional face detection method and system
CN111104846B (en) Data detection method and device, computer equipment and storage medium
CN106980845B (en) Face key point positioning method based on structured modeling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant