CN106250866A - Neural network-based image feature extraction modeling and image recognition method and device - Google Patents
Neural network-based image feature extraction modeling and image recognition method and device Download PDFInfo
- Publication number
- CN106250866A CN106250866A CN201610665948.8A CN201610665948A CN106250866A CN 106250866 A CN106250866 A CN 106250866A CN 201610665948 A CN201610665948 A CN 201610665948A CN 106250866 A CN106250866 A CN 106250866A
- Authority
- CN
- China
- Prior art keywords
- picture
- image
- verification
- neural network
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000013528 artificial neural network Methods 0.000 title claims abstract description 141
- 238000000605 extraction Methods 0.000 title claims abstract description 107
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000012795 verification Methods 0.000 claims abstract description 206
- 238000012549 training Methods 0.000 claims abstract description 110
- 238000012360 testing method Methods 0.000 claims abstract description 52
- 238000010200 validation analysis Methods 0.000 claims description 21
- 238000012545 processing Methods 0.000 claims description 9
- 238000001514 detection method Methods 0.000 claims description 7
- 230000009286 beneficial effect Effects 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 53
- 238000010586 diagram Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000002790 cross-validation Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000011478 gradient descent method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000012886 linear function Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
- G06V40/169—Holistic features and representations, i.e. based on the facial image taken as a whole
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biomedical Technology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image feature extraction modeling method and device based on a neural network, wherein a first picture, a second picture, a first classification of the first picture and a second classification of the second picture are obtained from a training set of a preset application scene; determining a global loss cost function value according to the first picture, the first classification, the second picture and the second classification; training an image object verification neural network on a training set according to the global loss cost function value and the training parameters; and testing the image object verification neural network through a test set of a preset application scene, determining the test precision according to the test result, and determining a target image object verification feature extraction model according to the test precision and the image object verification neural network. The method and the device can achieve the beneficial effect of improving the picture identification precision when the image characteristic model obtained by modeling is applied to image identification in an image identification application scene for image identification. The invention also provides an image identification method and device.
Description
Technical Field
The invention relates to the technical field of image recognition, in particular to an image feature extraction modeling method and device based on a neural network and an image recognition method and device.
Background
Image recognition is a technique in which a computer processes, analyzes, and understands images to recognize various different patterns of objects and objects. The image recognition aiming at the human face is the human face recognition, which is a biological recognition technology for carrying out identity recognition based on the facial feature information of a person, generally, after an image or a video stream containing the human face is collected by a camera or a camera, the human face is automatically detected and tracked in the image, and then the face recognition is carried out on the detected human face, which is generally called as the portrait recognition and the face recognition.
At present, the face recognition algorithm is based on a face photo and identity information corresponding to the face photo, model training is performed by using a neural network, and finally face recognition is performed by using a classifier. In the training of the model in the face recognition neural network, only the identity information of the face image is considered, and the recognition accuracy of the face recognition by using the model is yet to be further improved.
Disclosure of Invention
In view of the foregoing, it is desirable to provide an image feature extraction modeling method and apparatus based on a neural network, which can improve the recognition accuracy in an image recognition application scenario, and an image recognition method and apparatus using an image feature model established by the image feature extraction modeling method and apparatus based on the neural network.
An image feature extraction modeling method based on a neural network comprises the following steps:
acquiring a first picture, a second picture, a first classification of the first picture and a second classification of the second picture from a training set of a preset application scene;
taking the first picture, the first classification, the second picture and the second classification as input of an image object verification neural network, and determining a global loss cost function value;
training the image object validation neural network on the training set according to the global loss cost function value and the training parameters;
and testing the image object verification neural network through the test set of the preset application scene, determining the test precision according to the test result, and determining a target image object verification feature extraction model according to the test precision and the image object verification neural network.
An image recognition method, comprising:
acquiring a picture to be identified, and determining the verification feature to be identified by taking the picture to be identified as the input of the target image object verification feature extraction model determined by the image feature extraction modeling method based on the neural network;
and comparing the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determining the class of the picture corresponding to the picture verification feature closest to the verification features to be identified as the class of the picture to be identified.
An image feature extraction modeling device based on a neural network comprises:
the image classification acquisition module is used for acquiring a first image, a second image, a first classification of the first image and a second classification of the second image from a training set of a preset application scene;
a loss cost determination module, configured to determine a global loss cost function value by using the first picture, the first class, the second picture, and the second class as inputs of an image object verification neural network;
the neural network training module is used for training the image object verification neural network on the training set according to the global loss cost function value and the training parameters;
and the characteristic model determining module is used for testing the image object verification neural network through the test set of the preset application scene, determining the test precision according to the test result, and determining a target image object verification characteristic extraction model according to the test precision and the image object verification neural network.
An image recognition apparatus comprising:
the image feature extraction modeling device comprises a to-be-identified feature determination module, a to-be-identified feature determination module and a to-be-identified feature extraction module, wherein the to-be-identified feature determination module is used for acquiring a to-be-identified picture, and determining to-be-identified verification features by taking the to-be-identified picture as the input of a target image object verification feature extraction model determined by the image feature extraction modeling device based;
and the comparison and classification determining module is used for comparing the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determining the classification of the picture corresponding to the picture verification feature with the closest distance to the verification features to be identified as the classification of the picture to be identified.
According to the image feature extraction modeling method and device based on the neural network, due to the fact that the global loss cost function value adopted in model training is related to the first image and the second image, and also related to the first classification of the first image and the second classification of the second image. Therefore, the image object verification feature model obtained through modeling is related to the first classification of the first picture and the second classification of the second picture. Therefore, the beneficial effect of improving the image identification precision when the image object verification characteristic model obtained by modeling is applied to the preset application scene for image identification can be achieved.
According to the image identification method and the image identification device, the verification features to be identified are determined through the target image object verification feature extraction model determined by the neural network-based image feature extraction modeling method or device, and the classification of the images to be identified is finally determined through comparing the verification features to be identified with the image verification features in the training set, so that the image identification method and the image identification device are high in identification precision.
Drawings
FIG. 1 is a flowchart of an image feature extraction modeling method based on a neural network according to an embodiment;
FIG. 2 is a detailed flow chart of one step of the neural network-based image feature extraction modeling method of FIG. 1;
FIG. 3 is a flow chart of another embodiment of a neural network-based image feature extraction modeling method;
FIG. 4 is an exemplary diagram of a neural network-based image feature extraction modeling method before face alignment processing according to an embodiment;
FIG. 5 is a diagram of the exemplary image of FIG. 4 after face alignment processing;
FIG. 6 is a detailed flow chart of another step of the neural network-based image feature extraction modeling method of FIG. 1;
FIG. 7 is a flow diagram of an image recognition method according to an embodiment;
FIG. 8 is a block diagram of an image feature extraction modeling apparatus based on a neural network according to an embodiment;
FIG. 9 is a block diagram of an image feature extraction modeling apparatus based on a neural network according to another embodiment;
fig. 10 is a block diagram of an image recognition apparatus according to an embodiment.
Detailed Description
To facilitate an understanding of the invention, the invention will now be described more fully with reference to the accompanying drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "or/and" includes any and all combinations of one or more of the associated listed items.
As shown in fig. 1, a neural network-based image feature extraction modeling method according to an embodiment of the present invention includes:
s140: the method comprises the steps of obtaining a first picture, a second picture, a first classification of the first picture and a second classification of the second picture from a training set of a preset application scene.
The preset application scene may be a scene with a high requirement on image recognition accuracy, and particularly a scene with a high requirement on face recognition accuracy, such as a bank VTM (Virtual Teller Machine) verification scene, a jewelry store VIP (Virtual inportant Person, honored guest) recognition scene, and the like.
Each picture includes an object to be identified, which may be, for example, an item or person to be identified. The same category represents the same object, such as the same person or the same item.
S160: and taking the first picture, the first classification, the second picture and the second classification as the input of an image object verification neural network, and determining a global loss cost function value.
Through the first picture and the second picture, the object characteristics and the verification characteristics of the first picture and the second picture can be respectively determined according to the image object verification characteristic extraction model in the image object verification neural network, and then the value of the global loss cost function is determined according to the first classification, the second classification, the object characteristics and the verification characteristics.
The image object verification neural network is based on the image object recognition neural network in the prior art and comprises an image object verification feature extraction model determined based on an image object recognition feature extraction model in the image object recognition neural network.
Specifically, the image object recognition neural network is a trained image object recognition neural network, and is a trained neural network for performing image recognition by using the prior art. In this way, subsequent training is performed on the basis of the trained image object recognition neural network without starting training from scratch. Therefore, training time can be saved, and the optimal neural network can be found more quickly. Further, the trained image object recognition neural network is a deep neural network, i.e., the image object recognition deep neural network.
An image object verification neural network may be constructed from the image object verification feature extraction model. Preferably, the image object verification neural network is a deep neural network, i.e. an image object verification deep neural network.
The image object verification feature extraction model obtains the image object verification features according to the image object identification features on the basis of obtaining the image object identification features through the image object identification feature extraction model. Specifically, the image object identification features are subjected to two-norm normalization processing to obtain image object verification features. The second norm normalization processing specifically includes that a result obtained by squaring and then squaring each characteristic element of the image object identification characteristic is used as each characteristic element denominator of the image object verification characteristic; and then, the characteristic value of each characteristic element in the image object identification characteristic related to whether the image object is verified to be the same object is used as a molecule of one characteristic element of the image object verification characteristic.
S170: and training the image object validation neural network on the training set according to the global loss cost function value and the training parameters.
In one embodiment, a gradient value of each parameter in an image object verification feature extraction model of an image object verification neural network can be determined according to the global loss cost function value and a chain derivative rule; and training the image object verification neural network on the training set by adopting a random gradient descent method according to the global loss cost function value and the training parameters.
The training parameters include a feature distance threshold and a learning rate. In a preferred embodiment, the value of the feature distance threshold may be set to 0.2 or 0.25 by default; the value of the learning rate may be set to 0.0001 by default.
S180: and testing the image object verification neural network through the test set of the preset application scene, determining the test precision according to the test result, and determining a target image object verification feature extraction model according to the test precision and the image object verification neural network.
Training to validate the neural network for the image object may continue as long as the capacity of the test set is large enough. In this embodiment, after each training for a preset time, the image object verification neural network is tested through the test set of the preset application scenario, and the test precision is determined according to the test result. The test accuracy can be determined from the test results in the existing manner.
And when the test precision reaches the preset precision, the training is not continued, and the image object verification neural network at the moment is the target image object verification neural network. The target image object verification feature extraction model may be determined by a target image object verification neural network. The preset precision is the precision requirement which needs to be met by presetting the test.
In a preferred embodiment, cross-validation is used. The test set is a set of images without intersection with the training set, preferably, the images are face pictures.
In a specific embodiment, the test set is made in the following way: and (4) except K classes used for making the training set in the N classes, the rest N-K classes of face photos are used for making the test set. The test set consists of randomly extracted face picture verification pairs, and the extraction rule is as follows:
the a-th face picture of the nth classification, the b-th face picture of the nth classification (positive sample pair)
...
The c & ltth & gt face picture of the ith classification and the d & ltth & gt face picture of the jth classification (negative sample pair)
...
According to the rules of the international standard face verification test set, 3000 pairs of positive and negative sample pairs are respectively adopted, and 6000 pairs are obtained. The test rule is as follows: judging the two photos in the positive sample pair as the same person, namely xi1 is ═ 1; judging that the two photos in the negative sample pair are not the same person, namely judging correctly, namely xi1 is ═ 1; otherwise, it determines an error, i.e. xi0. The test accuracy is defined as:
in one embodiment, the preset test precision is not preset, the change process of the preset precision is gradually improved, and when a certain precision is reached, large fluctuation is generated; this accuracy is referred to herein as the maximum stable accuracy. Therefore, when the test precision is not stably improved any more, namely the maximum stable precision is reached, the image object verification neural network is not trained any more, and the current image object verification neural network is the optimal image object verification neural network. And taking an image object verification feature extraction model in the optimal image object verification neural network, and only taking one picture as input, taking the image object verification feature as output and omitting the output of the image object identification feature so as to obtain a final target image object verification feature extraction model.
According to the image feature extraction modeling method based on the neural network, due to the fact that the global loss cost function value adopted in the process of training the model is related to the first image and the second image, and also related to the first classification of the first image and the second classification of the second image. Therefore, the image object verification feature model obtained through modeling is related to the first classification of the first picture and the second classification of the second picture. Therefore, the beneficial effect of improving the image identification precision when the image object verification characteristic model obtained by modeling is applied to the preset application scene for image identification can be achieved.
Referring to fig. 1, in one embodiment, after step S140 and before step S160, the method further includes the steps of:
acquiring an image object recognition neural network, and determining an image object verification neural network according to the image object recognition neural network.
The method comprises the steps of obtaining an image object recognition neural network for carrying out image object recognition in the prior art, and determining an image object verification neural network according to the image object recognition neural network.
In an embodiment where one of the images is a face image, the image object recognition neural network is a face identity recognition neural network, preferably, a face identity recognition deep neural network; the image object authentication neural network is a face identity authentication neural network, preferably, a face identity authentication deep neural network. The relationship between the face identity recognition neural network and the face identity verification neural network is consistent with the relationship between the image object recognition neural network and the image object verification neural network, and therefore, the details are not repeated herein.
Referring to fig. 2, in one embodiment, the image object identifying neural network includes an image object identifying feature extraction model determined based on an image object identifying feature extraction model of the image object identifying neural network.
The step of determining a global loss cost function value using the first picture, the first class, the second picture, and the second class as inputs to the image object verification neural network, that is, S160, includes:
s261: taking the first picture and the first classification as model input of the image object verification feature extraction model, determining a first object feature and a first verification feature, taking the second picture and the second classification as another model input of the image object verification feature extraction model, and determining a second object feature and a second verification feature; or, the image object verification feature extraction models comprise two same models, the first picture and the first classification are used as model inputs of one image object verification feature extraction model to determine a first object feature and a first verification feature, the second picture and the second classification are used as model inputs of the other image object verification feature extraction model to determine a second object feature and a second verification feature; .
Specifically, the first picture and the first classification may be used as one model input, the second picture and the second classification may be used as another model input, and the first picture and the second classification may be used as model inputs of the image object verification feature extraction model of the image object verification neural network, and the first object feature and the first verification feature, and the second object feature and the second verification feature may be determined. The image object verification feature extraction model can also comprise two same models; and taking the first picture and the first classification as model input of one image object verification feature extraction model, taking the second picture and the second classification as model input of the other image object verification feature extraction model, executing the two image object verification feature extraction models in parallel, and finally respectively determining a first object feature and a first verification feature and a second object feature and a second verification feature.
S263: determining a first object information loss function value based on the first object characteristic and the first classification.
The loss function value can be determined from an object feature and a classification in a manner commonly used in existing neural networks. Specifically, an obtained classification information is determined according to the first object feature, and an identification result is determined according to whether the obtained classification information and the obtained first classification belong to the same class or not, so that a loss function value is reflected.
S265: and determining a second object information loss function value according to the second object characteristics and the second classification.
The second object information loss function value is identical to the first object information loss function value, and therefore, the detailed description thereof is omitted.
S267: determining a validation loss function value based on the first classification, the second classification, the first validation feature, and the second validation feature.
Specifically, the formula for the validation loss function is:
VerifyLoss＝yd+(1-y)max(α-d，0)
wherein,α is a characteristic distance threshold value in the training parameter, VerifyLoss represents a verification loss function value, y represents belonging to the same classification, N1 represents a first classification, N2 represents a second classification, d is a characteristic distance;a first verification feature is represented that is,representing a second authentication feature;representing a two-norm operation.
S269: determining a global loss cost function value according to the first object information loss function value, the second object information loss function value and the verification loss function value.
In this embodiment, the global loss cost function value is a linear function value with respect to the first object information loss function value, the second object information loss function value, and the verification loss function. Specifically, the formula of the global penalty cost function is:
Loss＝Soft max Loss_1+Soft max Loss_2+VerifyLoss
wherein, Loss is a global Loss cost function value, and SoftmaxLoss _1 is a first object information Loss function value; SoftmaxLoss _2 is the second object information loss function value; VerifyLoss is the validation loss function value.
In the process of training the model, the global loss cost function value is not only related to the first object information loss function value and the second object information loss function value, but also related to the verification loss function value, so that the identification precision of the image can be further improved when the image object verification feature model obtained through modeling is applied to a preset application scene.
Referring to fig. 3, in one embodiment, steps S340 to S380 correspond to steps S140 to S180 in sequence. The image is a face picture; before the step of obtaining the first picture, the second picture, the first classification of the first picture, and the second classification of the second picture from the training set of the preset application scenario (i.e. step S340), the method further includes:
s310: and acquiring a video picture in the preset application scene, and carrying out face detection on the video picture to obtain a face picture.
And a camera is used for acquiring video pictures in a preset application scene, and the video pictures are stored in a computer through network transmission and a data line. The method comprises the steps of carrying out face detection on collected video pictures in the existing mode, and extracting and storing the face pictures in a computer hard disk.
S320: and obtaining classification information for classifying the face pictures, classifying the face pictures according to the classification information, and performing face alignment processing on the classified face pictures to form a training set.
The detected and extracted face pictures are manually classified, so that the computer acquires manually input classification information and classifies the face pictures according to the classification information. The face photos belonging to the same category are put together and marked by the category information.
Because the face angle and the face position in the face picture are inconsistent, in order to ensure that stable features are extracted and a good face recognition effect is obtained, the key point alignment operation needs to be performed on the face picture through the existing mode so as to perform face alignment processing, and influence of the face angle on face recognition is removed. Wherein the key points include the positions of the eyes, the nose tip, the mouth corners and the like. Fig. 4 shows an example of an acquired face picture, that is, before face alignment processing, and fig. 5 shows an aligned face picture.
Referring to fig. 6, in one embodiment, the step of training the image object validation neural network on the training set according to the loss cost function value and the training parameters, i.e. step S170, includes:
s671: and acquiring initial training parameters, and training the image object verification neural network on the training set according to the global loss cost function value and the initial training parameters.
S673: and updating a training parameter, and training the image object verification neural network on the training set according to the global loss cost function value and the updated training parameter.
Thus, training parameters of the image object verification neural network are continuously adjusted to train, and optimal training parameters are determined. Through extensive debugging and experimentation, it was found that the best algorithm accuracy improvement is obtained for the method described herein when the feature distance threshold α is 0.2 and the learning rate lr is 0.001.
As shown in fig. 7, the present invention further provides an image recognition method using the neural network-based image feature extraction modeling method, including:
s740: and acquiring a picture to be identified, and determining the verification feature to be identified by taking the picture to be identified as the input of the target image object verification feature extraction model determined by the image feature extraction modeling method based on the neural network.
The target image object verification feature extraction model is established by the image feature extraction modeling method based on the neural network.
Specifically, a picture to be identified is collected through a camera, and the picture to be identified is transmitted to a computer; and the computer acquires the picture to be recognized, inputs the picture to be recognized into the established target image object verification feature extraction model for operation, and thus can determine the picture features of the picture to be recognized, namely the features to be recognized.
S760: and comparing the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determining the class of the picture corresponding to the picture verification feature closest to the verification features to be identified as the class of the picture to be identified.
In this embodiment, the picture verification features of all pictures in the training set are determined in advance through the target image object verification feature extraction model. After the verification features to be identified are determined, calculating the distance between the verification features to be identified and the picture verification features of each picture in the training set; and the classification of the picture corresponding to the picture verification feature closest to the verification feature to be identified is the classification of the picture to be identified.
According to the image identification method, the target image object verification feature extraction model determined by the neural network-based image feature extraction modeling method is used for determining the verification features to be identified, and the classification of the pictures to be identified is finally determined by comparing the verification features to be identified with the picture verification features in the training set, so that the image identification method is high in identification precision.
With continued reference to fig. 7, in one embodiment, the method further includes:
s720: and acquiring a target image object verification feature extraction model.
The invention also provides a virtual device corresponding to the image feature extraction modeling method based on the neural network. As shown in fig. 8, an image feature extraction modeling apparatus based on a neural network according to an embodiment includes:
the image classification acquisition module 840 is used for acquiring a first image, a second image, a first classification of the first image and a second classification of the second image from a training set of a preset application scene;
a loss cost determination module 860, configured to determine a global loss cost function value by using the first picture, the first class, the second picture, and the second class as inputs of an image object verification neural network;
a neural network training module 870 for training the image object validation neural network on the training set according to the global loss cost function value and the training parameters;
a feature model determining module 880, configured to test the image object validation neural network through the test set of the preset application scenario, determine a test precision according to a test result, and determine a target image object validation feature extraction model according to the test precision and the image object validation neural network.
According to the image feature extraction modeling device based on the neural network, due to the fact that the global loss cost function value adopted in model training is related to the first picture and the second picture, and also related to the first classification of the first picture and the second classification of the second picture. Therefore, the image object verification feature model obtained through modeling is related to the first classification of the first picture and the second classification of the second picture. Therefore, the beneficial effect of improving the image identification precision when the image object verification characteristic model obtained by modeling is applied to the preset application scene for image identification can be achieved.
Referring to fig. 9, in one embodiment, the method further includes:
the network obtaining and determining module 950 is configured to obtain an image object recognition neural network, and determine an image object verification neural network according to the image object recognition neural network.
With continued reference to FIG. 9, in one embodiment, the image object verification neural network includes an image object verification feature extraction model determined based on an image object identification feature extraction model of the image object identification neural network. The loss cost determination module 960 includes:
a picture feature determination unit 961 (not shown) configured to determine a first object feature and a first verification feature by inputting the first picture and the first classification as a model of the image object verification feature extraction model, determine a second object feature and a second verification feature by inputting the second picture and the second classification as another model of the image object verification feature extraction model; or the image object verification feature extraction models comprise two same models; a picture feature determination unit 961, configured to determine a first object feature and a first verification feature by using the first picture and the first classification as model inputs of one of the image object verification feature extraction models, determine a second object feature and a second verification feature by using the second picture and the second classification as model inputs of the other image object verification feature extraction model;
a first loss determining unit (not shown) for determining a first object information loss function value according to the first object feature and the first classification;
a second loss determining unit (not shown) for determining a second object information loss function value according to the second object characteristic and the second classification;
a verification loss determining unit (not shown) for determining a verification loss function value according to the first classification, the second classification, the first verification feature and the second verification feature;
a global loss determining unit (not shown) for determining a global loss cost function value according to the first object information loss function value, the second object information loss function value, and the verification loss function value.
With continued reference to fig. 9, in one embodiment, the image is a face picture; the device, still include:
the image acquisition detection module 910 is configured to acquire a video image in the preset application scene, and perform face detection on the video image to obtain a face image;
the image classifying and aligning module 920 is configured to obtain classification information for classifying the face images, classify the face images according to the classification information, and perform face aligning processing on each classified face image to form a training set.
In one embodiment, the neural network training module 970 includes:
an initial parameter training unit 971, configured to obtain an initial training parameter, train the image object validation neural network on the training set according to the global loss cost function value and the initial training parameter;
and an update parameter training unit 973, configured to update a training parameter, and train the image object validation neural network on the training set according to the global loss cost function value and the updated training parameter.
The invention also provides a virtual device corresponding to the image identification method. As shown in fig. 10, an image recognition apparatus of an embodiment includes:
the to-be-identified feature determining module 1040 is configured to obtain a to-be-identified picture, and determine a to-be-identified verification feature by using the to-be-identified picture as an input of the target image object verification feature extraction model determined by the image feature extraction modeling apparatus based on the neural network;
the comparison and classification determining module 1060 is configured to compare the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determine the classification to which the picture corresponding to the picture verification feature closest to the verification features to be identified belongs as the classification of the picture to be identified.
The image recognition device determines the verification features to be recognized according to the target image object verification feature extraction model determined by the image feature extraction modeling device based on the neural network, compares the verification features to be recognized with the picture verification features in the training set, and finally determines the classification of the pictures to be recognized, so that the image recognition device is high in recognition accuracy.
In one embodiment, the method further comprises the following steps:
and a feature model obtaining module 1020, configured to obtain a target image object verification feature extraction model.
The above examples only show some embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the present invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. An image feature extraction modeling method based on a neural network is characterized by comprising the following steps:
acquiring a first picture, a second picture, a first classification of the first picture and a second classification of the second picture from a training set of a preset application scene;
taking the first picture, the first classification, the second picture and the second classification as input of an image object verification neural network, and determining a global loss cost function value;
training the image object validation neural network on the training set according to the global loss cost function value and the training parameters;
and testing the image object verification neural network through the test set of the preset application scene, determining the test precision according to the test result, and determining a target image object verification feature extraction model according to the test precision and the image object verification neural network.
2. The neural network-based image feature extraction modeling method of claim 1, wherein the image object verification neural network includes an image object verification feature extraction model determined based on an image object recognition feature extraction model of the image object recognition neural network;
the step of determining a global loss cost function value using the first picture, the first class, the second picture, and the second class as inputs to the image object validation neural network comprises:
taking the first picture and the first classification as model input of the image object verification feature extraction model, determining a first object feature and a first verification feature, taking the second picture and the second classification as another model input of the image object verification feature extraction model, and determining a second object feature and a second verification feature; or, the image object verification feature extraction models comprise two same models, the first picture and the first classification are used as model inputs of one image object verification feature extraction model to determine a first object feature and a first verification feature, the second picture and the second classification are used as model inputs of the other image object verification feature extraction model to determine a second object feature and a second verification feature;
determining a first object information loss function value according to the first object characteristics and the first classification;
determining a second object information loss function value according to the second object characteristics and the second classification;
determining a validation loss function value based on the first classification, the second classification, the first validation feature, and the second validation feature;
determining a global loss cost function value according to the first object information loss function value, the second object information loss function value and the verification loss function value.
3. The neural network-based image feature extraction modeling method of claim 1, wherein the image is a face picture;
before the step of obtaining a first picture, a second picture, a first classification of the first picture and a second classification of the second picture from a training set of a preset application scene, the method further includes:
acquiring a video picture in the preset application scene, and carrying out face detection on the video picture to obtain a face picture;
and obtaining classification information for classifying the face pictures, classifying the face pictures according to the classification information, and performing face alignment processing on the classified face pictures to form a training set.
4. The method according to claim 1, wherein the step of training the image object validation neural network on the training set according to the global loss cost function value and the training parameters comprises:
acquiring an initial training parameter, and training the image object verification neural network on the training set according to the global loss cost function value and the initial training parameter;
and updating a training parameter, and training the image object verification neural network on the training set according to the global loss cost function value and the updated training parameter.
5. An image recognition method, comprising:
acquiring a picture to be identified, and determining a verification feature to be identified by taking the picture to be identified as the input of a target image object verification feature extraction model determined by the neural network-based image feature extraction modeling method of any one of claims 1 to 4;
and comparing the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determining the class of the picture corresponding to the picture verification feature closest to the verification features to be identified as the class of the picture to be identified.
6. An image feature extraction modeling device based on a neural network is characterized by comprising:
the image classification acquisition module is used for acquiring a first image, a second image, a first classification of the first image and a second classification of the second image from a training set of a preset application scene;
a loss cost determination module, configured to determine a global loss cost function value by using the first picture, the first class, the second picture, and the second class as inputs of an image object verification neural network;
the neural network training module is used for training the image object verification neural network on the training set according to the global loss cost function value and the training parameters;
and the characteristic model determining module is used for testing the image object verification neural network through the test set of the preset application scene, determining the test precision according to the test result, and determining a target image object verification characteristic extraction model according to the test precision and the image object verification neural network.
7. The neural network-based image feature extraction modeling apparatus of claim 6, wherein the image object verification neural network includes an image object verification feature extraction model determined based on an image object recognition feature extraction model of the image object recognition neural network;
the loss cost determination module includes:
a picture feature determination unit, configured to use the first picture and the first classification as model inputs of the image object verification feature extraction model, determine a first object feature and a first verification feature, use the second picture and the second classification as another model input of the image object verification feature extraction model, and determine a second object feature and a second verification feature; or the image object verification feature extraction models comprise two same models; a picture feature determination unit, configured to determine a first object feature and a first verification feature by using the first picture and the first classification as model inputs of one of the image object verification feature extraction models, determine a second object feature and a second verification feature by using the second picture and the second classification as model inputs of the other image object verification feature extraction model;
a first loss determining unit for determining a first object information loss function value according to the first object feature and the first classification;
a second loss determining unit, configured to determine a second object information loss function value according to the second object feature and the second classification;
a verification loss determining unit configured to determine a verification loss function value according to the first classification, the second classification, the first verification feature, and the second verification feature;
a global loss determining unit configured to determine a global loss cost function value according to the first object information loss function value, the second object information loss function value, and the verification loss function value.
8. The neural network-based image feature extraction modeling apparatus of claim 6, wherein the image is a face picture; the device, still include:
the image acquisition detection module is used for acquiring a video image in the preset application scene and carrying out face detection on the video image to obtain a face image;
and the image classification and alignment module is used for acquiring classification information for classifying the face images, classifying the face images according to the classification information, and performing face alignment treatment on the classified face images to form a training set.
9. The neural network-based image feature extraction modeling apparatus of claim 6, wherein the neural network training module comprises:
an initial parameter training unit, configured to obtain an initial training parameter, train the image object validation neural network on the training set according to the global loss cost function value and the initial training parameter;
and the updating parameter training unit is used for updating a training parameter and training the image object verification neural network on the training set according to the global loss cost function value and the updated training parameter.
10. An image recognition apparatus, comprising:
the image feature extraction modeling device comprises a to-be-identified feature determination module, a to-be-identified feature determination module and a feature recognition module, wherein the to-be-identified feature determination module is used for acquiring a to-be-identified picture, and determining to-be-identified verification features by taking the to-be-identified picture as the input of a target image object verification feature extraction model determined by the image feature extraction modeling device based on the neural network according to any one of claims 6 to 9;
and the comparison and classification determining module is used for comparing the verification features to be identified with the picture verification features corresponding to the pictures in the training set, and determining the classification of the picture corresponding to the picture verification feature with the closest distance to the verification features to be identified as the classification of the picture to be identified.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610665948.8A CN106250866A (en) | 2016-08-12 | 2016-08-12 | Neural network-based image feature extraction modeling and image recognition method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610665948.8A CN106250866A (en) | 2016-08-12 | 2016-08-12 | Neural network-based image feature extraction modeling and image recognition method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106250866A true CN106250866A (en) | 2016-12-21 |
Family
ID=57592678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610665948.8A Pending CN106250866A (en) | 2016-08-12 | 2016-08-12 | Neural network-based image feature extraction modeling and image recognition method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106250866A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778785A (en) * | 2016-12-23 | 2017-05-31 | 东软集团股份有限公司 | Build the method for image characteristics extraction model and method, the device of image recognition |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
| CN108509961A (en) * | 2017-02-27 | 2018-09-07 | 北京旷视科技有限公司 | Image processing method and device |
| CN108629377A (en) * | 2018-05-10 | 2018-10-09 | 北京达佳互联信息技术有限公司 | A kind of the loss value-acquiring method and device of disaggregated model |
| CN109034040A (en) * | 2018-07-19 | 2018-12-18 | 北京影谱科技股份有限公司 | A kind of character recognition method based on cast, device, equipment and medium |
| WO2019072057A1 (en) * | 2017-10-13 | 2019-04-18 | 华为技术有限公司 | Image signal processing method, apparatus and device |
| WO2019114523A1 (en) * | 2017-12-12 | 2019-06-20 | 腾讯科技（深圳）有限公司 | Classification training method, server and storage medium |
| CN110569737A (en) * | 2019-08-15 | 2019-12-13 | 深圳华北工控软件技术有限公司 | Face recognition deep learning method and face recognition acceleration camera |
| CN111191655A (en) * | 2018-11-14 | 2020-05-22 | 佳能株式会社 | Object identification method and device |
| CN111341459A (en) * | 2020-02-28 | 2020-06-26 | 上海交通大学医学院附属上海儿童医学中心 | Training method of classified deep neural network model and genetic disease detection method |
| CN111950728A (en) * | 2020-08-17 | 2020-11-17 | 珠海格力电器股份有限公司 | Image feature extraction model construction method, image retrieval method and storage medium |
| CN112689763A (en) * | 2018-09-20 | 2021-04-20 | 美国西门子医学诊断股份有限公司 | Hypothesis and verification network and method for sample classification |
| CN113033424A (en) * | 2021-03-29 | 2021-06-25 | 广东众聚人工智能科技有限公司 | Multi-branch video anomaly detection method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824054A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascaded depth neural network-based face attribute recognition method |
| CN104866810A (en) * | 2015-04-10 | 2015-08-26 | 北京工业大学 | Face recognition method of deep convolutional neural network |
-
2016
- 2016-08-12 CN CN201610665948.8A patent/CN106250866A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103824054A (en) * | 2014-02-17 | 2014-05-28 | 北京旷视科技有限公司 | Cascaded depth neural network-based face attribute recognition method |
| CN104866810A (en) * | 2015-04-10 | 2015-08-26 | 北京工业大学 | Face recognition method of deep convolutional neural network |
Non-Patent Citations (2)
| Title |
|---|
| FLORIAN SCHROFF: "FaceNet: A unified embedding for face recognition and clustering", 《THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION》 * |
| YI SUN 等: "Deep Learning Face Representation by Joint Identification-Verification", 《HTTP://ARXIV.ORG/PDF/1406.4773》 * |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106778785A (en) * | 2016-12-23 | 2017-05-31 | 东软集团股份有限公司 | Build the method for image characteristics extraction model and method, the device of image recognition |
| CN106778785B (en) * | 2016-12-23 | 2019-09-17 | 东软集团股份有限公司 | Construct the method for image Feature Selection Model and the method, apparatus of image recognition |
| CN108509961A (en) * | 2017-02-27 | 2018-09-07 | 北京旷视科技有限公司 | Image processing method and device |
| WO2019072057A1 (en) * | 2017-10-13 | 2019-04-18 | 华为技术有限公司 | Image signal processing method, apparatus and device |
| CN109688351A (en) * | 2017-10-13 | 2019-04-26 | 华为技术有限公司 | A kind of image-signal processing method, device and equipment |
| US11430209B2 (en) | 2017-10-13 | 2022-08-30 | Huawei Technologies Co., Ltd. | Image signal processing method, apparatus, and device |
| US11017220B2 (en) | 2017-12-12 | 2021-05-25 | Tencent Technology (Shenzhen) Company Limited | Classification model training method, server, and storage medium |
| WO2019114523A1 (en) * | 2017-12-12 | 2019-06-20 | 腾讯科技（深圳）有限公司 | Classification training method, server and storage medium |
| CN108416295A (en) * | 2018-03-08 | 2018-08-17 | 天津师范大学 | A kind of recognition methods again of the pedestrian based on locally embedding depth characteristic |
| CN108416295B (en) * | 2018-03-08 | 2021-10-15 | 天津师范大学 | Pedestrian re-identification method based on local embedding depth features |
| CN108629377A (en) * | 2018-05-10 | 2018-10-09 | 北京达佳互联信息技术有限公司 | A kind of the loss value-acquiring method and device of disaggregated model |
| CN109034040A (en) * | 2018-07-19 | 2018-12-18 | 北京影谱科技股份有限公司 | A kind of character recognition method based on cast, device, equipment and medium |
| CN112689763A (en) * | 2018-09-20 | 2021-04-20 | 美国西门子医学诊断股份有限公司 | Hypothesis and verification network and method for sample classification |
| CN111191655A (en) * | 2018-11-14 | 2020-05-22 | 佳能株式会社 | Object identification method and device |
| CN111191655B (en) * | 2018-11-14 | 2024-04-16 | 佳能株式会社 | Object identification method and device |
| CN110569737A (en) * | 2019-08-15 | 2019-12-13 | 深圳华北工控软件技术有限公司 | Face recognition deep learning method and face recognition acceleration camera |
| CN111341459A (en) * | 2020-02-28 | 2020-06-26 | 上海交通大学医学院附属上海儿童医学中心 | Training method of classified deep neural network model and genetic disease detection method |
| CN111950728A (en) * | 2020-08-17 | 2020-11-17 | 珠海格力电器股份有限公司 | Image feature extraction model construction method, image retrieval method and storage medium |
| CN113033424A (en) * | 2021-03-29 | 2021-06-25 | 广东众聚人工智能科技有限公司 | Multi-branch video anomaly detection method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106529571B (en) | Multilayer image feature extraction modeling and image recognition method and device based on neural network | |
| CN105975959B (en) | Face feature extraction modeling and face recognition method and device based on neural network | |
| CN106250866A (en) | Neural network-based image feature extraction modeling and image recognition method and device | |
| CN106897658B (en) | Method and device for identifying human face living body | |
| Yang et al. | Sieving regression forest votes for facial feature detection in the wild | |
| US9514356B2 (en) | Method and apparatus for generating facial feature verification model | |
| CN112232117A (en) | Face recognition method, face recognition device and storage medium | |
| US8620036B2 (en) | System and method for controlling image quality | |
| CN111133433A (en) | Automatic authentication for access control using facial recognition | |
| CN109858375B (en) | Living body face detection method, terminal and computer readable storage medium | |
| CN109800643A (en) | A kind of personal identification method of living body faces multi-angle | |
| KR102078249B1 (en) | Method for face authentication, apparatus for face authentication and computer program for the same | |
| WO2009041963A1 (en) | Iris recognition using consistency information | |
| CN105303150A (en) | Method and system for implementing image processing | |
| US20220157078A1 (en) | Adaptive learning and matching of face modalities | |
| WO2023124869A1 (en) | Liveness detection method, device and apparatus, and storage medium | |
| CN106295574A (en) | Face feature extraction modeling and face recognition method and device based on neural network | |
| US10755074B2 (en) | Latent fingerprint pattern estimation | |
| WO2015131710A1 (en) | Method and device for positioning human eyes | |
| CN110956098B (en) | Image processing method and related equipment | |
| US20230230277A1 (en) | Object position estimation device, object position estimation method, and recording medium | |
| Suganthy et al. | Effective iris recognition for security enhancement | |
| CN112182537A (en) | Monitoring method, device, server, system and storage medium | |
| Jang et al. | Biometrics System Using Deep Convolution GAN | |
| Basak et al. | Face recognition using fuzzy logic |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20161221 |