WO2018068416A1 - 基于神经网络的多层图像特征提取建模、图像识别方法及装置 - Google Patents

基于神经网络的多层图像特征提取建模、图像识别方法及装置 Download PDF

Info

Publication number
WO2018068416A1
WO2018068416A1 PCT/CN2016/113147 CN2016113147W WO2018068416A1 WO 2018068416 A1 WO2018068416 A1 WO 2018068416A1 CN 2016113147 W CN2016113147 W CN 2016113147W WO 2018068416 A1 WO2018068416 A1 WO 2018068416A1
Authority
WO
WIPO (PCT)
Prior art keywords
picture
verification
neural network
layer
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/113147
Other languages
English (en)
French (fr)
Inventor
张玉兵
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Original Assignee
Guangzhou Shiyuan Electronics Thecnology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Shiyuan Electronics Thecnology Co Ltd filed Critical Guangzhou Shiyuan Electronics Thecnology Co Ltd
Publication of WO2018068416A1 publication Critical patent/WO2018068416A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]

Definitions

  • the invention relates to the field of image recognition technology, in particular to a multi-layer image feature extraction modeling method and device based on neural network, and an image recognition method and device.
  • Image recognition is a technique in which a computer processes, analyzes, and understands images to identify targets and objects in various modes.
  • the image recognition for the face is the face recognition, which is a biometric technology based on the facial feature information of the person for identification.
  • the image or video stream containing the face is captured by the camera or the camera, and the image is automatically imaged.
  • the detection and tracking of the face, and then the face recognition of the detected face usually called portrait recognition, face recognition.
  • the face recognition algorithm is based on the face photo and the corresponding identity information, using the neural network for model training, and finally using the classifier for face recognition.
  • the model in the face recognition neural network only the identity information of the face image is considered, and the recognition accuracy of the face recognition using the model needs to be further improved.
  • a multi-layer image feature extraction modeling method based on neural network comprising:
  • Testing the multi-layer image object verification neural network by using a test set of the preset application scenario, and according to The test result determines the test accuracy, and the target multi-layer image object verification feature extraction model is determined according to the test accuracy and the multi-layer image object verification neural network.
  • An image recognition method comprising:
  • a multi-layer image feature extraction modeling device based on neural network comprising:
  • a picture classification obtaining module configured to acquire, from a training set of the preset application scenario, a first picture, a second picture, a first category of the first picture, and a second category of the second picture;
  • a loss cost determining module configured to determine, as the input of the multi-layer image object verification neural network, the first picture, the first category, the second picture, and the second category to determine a global loss cost function value
  • a neural network training module configured to train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the training parameter;
  • a feature model determining module configured to test the multi-layer image object verification neural network by using a test set of the preset application scenario, and determine a test accuracy according to the test result, according to the test accuracy and the multi-layer image object
  • the verification neural network determines the target multi-layer image object verification feature extraction model.
  • An image recognition device comprising:
  • the feature determining module is configured to obtain a picture to be identified, and determine the input of the target multi-layer image object verification feature extraction model determined by the neural network-based multi-layer image feature extraction modeling device.
  • the verification feature to be identified is configured to obtain a picture to be identified, and determine the input of the target multi-layer image object verification feature extraction model determined by the neural network-based multi-layer image feature extraction modeling device. The verification feature to be identified;
  • a comparison classification determining module configured to compare the verification feature to be identified with a picture verification feature corresponding to the picture in the training set, and determine a classification of the picture corresponding to the picture verification feature that is closest to the verification feature to be identified as The classification of the picture to be identified.
  • the above neural network-based multi-layer image feature extraction modeling method and device are adopted when training the model
  • the global loss cost function value is related not only to the first picture and the second picture but also to the first classification of the first picture and the second classification of the second picture. Therefore, the modeled image object verification feature model is related to the first category of the first picture and the second category of the second picture. Therefore, the beneficial effect of improving the image recognition accuracy when the imaged object verification feature model obtained by the modeling is applied to the preset application scene for image recognition can be achieved.
  • the image object verification neural network is used to verify the multi-layer multi-layer image object verification neural network, and the determined image object verification feature extraction model is a multi-level multi-layer image object verification feature extraction model.
  • the multi-layer image object verification feature extraction model not only outputs the highest layer image object verification feature, but also outputs the image object verification feature of the middle layer. Therefore, the beneficial effect of improving the image recognition accuracy when the imaged object verification feature model obtained by the modeling is applied to the preset application scene for image recognition can be further achieved.
  • the above image recognition method and device determine the feature to be identified by the target multi-layer image object verification feature extraction model determined by the above-described neural network-based multi-layer image feature extraction modeling method or device, and pass the identification feature and training to be identified
  • the concentrated picture verification features are compared to finally determine the classification of the picture to be identified. Therefore, the image recognition method and apparatus have high recognition accuracy.
  • FIG. 1 is a flow chart of a neural network based multi-layer image feature extraction modeling method according to an embodiment
  • FIG. 2 is a specific flowchart of a step of the neural network-based multi-layer image feature extraction modeling method of FIG. 1;
  • FIG. 3 is a flow chart of a neural network based multi-layer image feature extraction modeling method according to another embodiment
  • FIG. 4 is a diagram showing an example of a face-to-face alignment process in a multi-layer image feature extraction modeling method based on a neural network according to an embodiment
  • FIG. 5 is a diagram showing a result of performing face alignment processing on the example diagram of FIG. 4;
  • FIG. 6 is a specific flowchart of another step of the neural network-based multi-layer image feature extraction modeling method of FIG. 1;
  • FIG. 7 is a flow chart of an image recognition method according to an embodiment
  • FIG. 8 is a structural diagram of a neural network based multi-layer image feature extraction modeling apparatus according to an embodiment
  • FIG. 9 is a structural diagram of a neural network based multi-layer image feature extraction modeling apparatus according to another embodiment.
  • Fig. 10 is a view showing the configuration of an image recognition apparatus of an embodiment.
  • a neural network based multi-layer image feature extraction modeling method includes:
  • S140 Acquire a first picture, a second picture, a first category of the first picture, and a second category of the second picture from a training set of the preset application scenario.
  • the preset application scenario may be a scene with high accuracy for image recognition, especially for scenes with high accuracy of face recognition, such as bank VTM (Virtual Teller Machine) verification, and jewelry store VIP (Very Important Person, VIP) Identify and other scenes.
  • bank VTM Virtual Teller Machine
  • VIP Very Important Person, VIP
  • Each picture includes an object to be identified, for example, an item or a person to be identified.
  • the same category represents the same object, such as the same person or the same item.
  • S160 The first picture, the first category, the second picture, and the second category are used as input of the multi-layer image object verification neural network, and the global loss cost function value is determined.
  • the first image and the second image are used to verify the multi-layer image object verification feature extraction model in the neural network according to the multi-layer image object, and the object features and the verification features of the first image and the second image are respectively determined, and then according to the first classification, The second classification and these object features, validation features determine the value of the global loss cost function.
  • the multi-layer image object verification neural network is based on the prior art image object recognition neural network, and includes a multi-layer image object verification feature extraction model determined based on the image object recognition feature extraction model in the image object recognition neural network.
  • the image object recognition neural network identifies the neural network for the trained image object, and is a well-trained neural network that uses the prior art for image recognition.
  • the neural network is identified in the trained image object Subsequent training on the basis of the network, without the need to start training from scratch. Therefore, training time can be saved and the optimal neural network can be found faster.
  • the trained image object recognition neural network is a deep neural network, that is, the image object identifies a deep neural network.
  • a multi-layer image object verification neural network can be constructed according to the multi-layer image object verification feature extraction model.
  • the multi-layer image object verifies the neural network as a deep neural network, ie the image object verifies the deep neural network.
  • the multi-layer image object verification feature extraction model obtains the image object identification feature according to the image object recognition feature, and obtains the image object verification feature according to the image object recognition feature.
  • the image object recognition feature is subjected to a two-norm normalization process to obtain an image object verification feature.
  • the second norm normalization process is specifically: the result obtained by squaring each feature element of the image object recognition feature and then squared is used as the denominator of each feature element of the image object verification feature; and then the image object recognition feature is Each feature value of a feature element associated with verifying whether it is the same object is a numerator of a feature element of the image object verification feature.
  • S170 Train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the training parameter.
  • the gradient value of each parameter in the multi-layer image object verification feature extraction model of the multi-layer image object verification neural network may be determined according to the global loss cost function value and the chain derivation rule;
  • the loss cost function value and the training parameter train the multi-layer image object verification neural network on the training set using a stochastic gradient descent method.
  • Training parameters include feature distance thresholds and learning rates.
  • the value of the feature distance threshold may be set to 0.2 or 0.25 by default; the value of the learning rate may be set to 0.0001 by default.
  • the multi-layer image object verification neural network is tested by using the test set of the preset application scenario, and the test accuracy is determined according to the test result, and the target is determined according to the test accuracy and the multi-layer image object verification neural network.
  • Multi-layer image object verification feature extraction model
  • the training of the multi-layer image object verification neural network can continue.
  • the multi-layer image object verification neural network is tested by using the test set of the preset application scenario, and the test accuracy is determined according to the test result.
  • the test accuracy can be determined from the test results in an existing manner.
  • the multi-layer image object verification neural network verifies the neural network for the target multi-layer image object.
  • Target multi-layer map can be determined by verifying the neural network with the target multi-layer image object
  • the preset accuracy is the precision requirement that the test needs to be set in advance.
  • test set is a collection of images that do not intersect the training set, preferably the image is a face picture.
  • the test set is created by dividing the N categories into K categories for making the training set, and the remaining N-K categories of face photos are used to create a test set.
  • the test set consists of randomly extracted face image verification pairs. The extraction rules are as follows:
  • the positive and negative sample pairs here take 3000 pairs, a total of 6000 pairs.
  • the test accuracy is defined as:
  • the preset test accuracy is not set in advance, and the change process of the preset accuracy is gradually increased first, and a large fluctuation is generated when a certain precision is reached; here, the accuracy is recorded as the maximum stable precision. Therefore, when the test accuracy is no longer steadily improved, that is, when the maximum stable accuracy is reached, the multi-layer image object verification neural network is no longer trained.
  • the current multi-layer image object verification neural network is the optimal multi-layer image object verification neural network. .
  • the above-mentioned neural network-based multi-layer image feature extraction modeling method because of the global loss cost function value adopted when training the model, is not only related to the first picture and the second picture, but also to the first classification and the second of the first picture.
  • the second category of the picture is related. Therefore, the modeled image object verification feature model is related to the first category of the first picture and the second category of the second picture. Therefore, the imaged object verification feature model obtained by modeling can be applied to the preset application.
  • the effect of image recognition accuracy is improved when image recognition is performed in the scene.
  • the image object verification neural network is used to verify the multi-layer multi-layer image object verification neural network, and the determined image object verification feature extraction model is a multi-level multi-layer image object verification feature extraction model.
  • the multi-layer image object verification feature extraction model not only outputs the highest layer image object verification feature, but also outputs the image object verification feature of the middle layer. Therefore, the beneficial effect of improving the image recognition accuracy when the imaged object verification feature model obtained by the modeling is applied to the preset application scene for image recognition can be further achieved.
  • step S140 before step S160, the method further includes the steps of:
  • the image object recognition neural network is a face recognition neural network, preferably, a face recognition deep neural network; and the multi-layer image object verification neural network is a face authentication.
  • a neural network preferably a face authentication deep neural network.
  • the multi-layer image object verification neural network includes a multi-layer image object verification feature extraction model determined based on an image object recognition feature extraction model of the image object recognition neural network.
  • the step of determining the global loss cost function value by using the first picture, the first classification, the second picture, and the second classification as an input of the multi-layer image object verification neural network, that is, S160 include:
  • S261 input the first picture and the first classification as a model input of the multi-layer image object verification feature extraction model, determine a multi-layer first object feature and a multi-layer first verification feature, and use the second image And the second classification as another model input of the multi-layer image object verification feature extraction model, determining a multi-layer second object feature and a multi-layer second verification feature; or the multi-layer image object verification feature extraction model Including the same two, the first picture and the first classification are used as model inputs of one of the multi-layer image object verification feature extraction models, and are determined a multi-layer first object feature and a multi-layer first verification feature, the second picture and the second classification being used as a model input of another multi-layer image object verification feature extraction model, and determining a multi-layer second object feature With a multi-layer second verification feature.
  • the first picture and the first category may be input as one model
  • the second picture and the second category are input as another model
  • the multi-layer image object verification is performed twice in succession.
  • the multi-layer image object of the neural network validates the model input of the feature extraction model, determines the multi-layer first object feature and the multi-layer first verification feature, and the multi-layer second object feature and the multi-layer second verification feature.
  • the multi-layer image object verification feature extraction model includes the same two; inputting the first picture and the first classification as a model input of the multi-layer image object verification feature extraction model, The second picture and the second classification are used as model inputs of another multi-layer image object verification feature extraction model, and the two multi-layer image object verification feature extraction models are executed in parallel, and finally the multi-layer first object features are determined respectively.
  • S263 Determine a first object information loss function value according to the multi-layer first object feature and the first category.
  • the loss function value can be determined from an object feature and a classification using methods commonly used in existing neural networks. Specifically, a piece of the obtained classification information is determined according to the multi-layer first object feature, and according to the obtained classification information and whether the acquired first classification belongs to the same category, the recognition result is determined, and then the response value is reflected.
  • S265 Determine a second object information loss function value according to the multi-layer second object feature and the second classification.
  • the second object information loss function value is consistent with the determination of the first object information loss function value, and thus is not described herein.
  • S267 Determine a verification loss function value according to the first classification, the second classification, the multi-layer first verification feature, and the multi-layer second verification feature.
  • VerifyLoss k yd k +(1-y)max( ⁇ -d k ,0)
  • ⁇ is the feature distance threshold in the training parameters.
  • VerifyLoss k represents the verification loss function value of the k-th layer verification feature; y represents the same classification; N1 represents the first classification, N2 represents the second classification; d k is the characteristic distance of the k-th layer verification feature; Representing a first verification feature of the kth layer of the multi-layer first verification feature, Representing a second verification feature of the kth layer of the multi-layer second verification feature; Represents a two-norm operation.
  • S269 Determine a global loss cost function value according to the first object information loss function value, the second object information loss function value, and the verification loss function value.
  • the global loss cost function value is a linear function value with respect to the first object information loss function value, the second object information loss function value, and the verification loss function.
  • the formula for the global loss cost function is:
  • Loss is the global loss cost function value
  • SoftmaxLoss_1 is the first object information loss function value
  • SoftmaxLoss_2 is the second object information loss function value
  • VerifyLoss k is the verification loss function value of the k-th layer verification feature
  • m is the total of the verification features The number of layers
  • the modeling result can be further improved.
  • steps S340-S380 are sequentially corresponding to steps S140-S180.
  • the image is a face image; the step of acquiring a first picture, a second picture, a first classification of the first picture, and a second classification of the second picture from the training set of the preset application scenario (ie, the step Before S340), it also includes:
  • S310 Collect a video picture in the preset application scenario, and perform face detection on the video picture to obtain a face picture.
  • the camera is used to capture video images in a preset application scenario and stored in a computer via network transmission and data lines. Face detection is performed on the captured video image by the existing method, and the face image is extracted and stored in the computer hard disk.
  • S320 Acquire classification information for classifying the face image, classify each face image according to the classification information, and perform face alignment processing on each of the classified face images to form a training set. .
  • the face images detected and extracted are manually classified. Therefore, the computer obtains the manually input classification information and classifies them according to the classification information. Face photos belonging to the same category are put together and marked by classification information.
  • the key point alignment operation of the face image needs to be performed in the existing way to perform face alignment processing, and the influence of the face angle on the face recognition is removed.
  • the key points include the position of the eyes, nose and mouth.
  • a captured face image is an example image before face alignment processing
  • the aligned face image is as shown in FIG. 5.
  • the step of training the multi-layer image object verification neural network on the training set according to the loss cost function value and the training parameter, that is, step S170, includes:
  • S671 Acquire initial training parameters, and train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the initial training parameter.
  • S673 Update training parameters, and train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the updated training parameter.
  • the training parameters of the multi-layer image object verification neural network are continuously adjusted for training, and the optimal training parameters are determined.
  • the best algorithm accuracy can be improved.
  • the present invention further provides an image recognition method using the above neural network-based multi-layer image feature extraction modeling method, including:
  • S740 Acquire an image to be identified, and use the image to be recognized as an input of a target multi-layer image object verification feature extraction model determined by a neural network-based multi-layer image feature extraction modeling method, and determine a verification feature to be identified.
  • the target multi-layer image object verification feature extraction model is a target multi-layer image object verification feature extraction model established by the above-described neural network-based multi-layer image feature extraction modeling method.
  • the image to be recognized is collected by the camera, and the image to be recognized is transmitted to the computer; the computer acquires the image to be recognized, and inputs the target multi-layer image object verification feature extraction model established by the image to be recognized, thereby Determining the picture feature of the picture to be identified, that is, the feature to be identified.
  • S760 Compare the to-be-identified verification feature with the picture verification feature corresponding to the picture in the training set, and determine the classification of the picture corresponding to the picture verification feature that is closest to the verification feature to be identified as the picture to be recognized. Classification.
  • the image verification feature of all the pictures in the training set is determined in advance by the target multi-layer image object verification feature extraction model. After determining the verification feature to be identified, calculating the verification feature to be identified and each picture in the training set The picture verification feature distance; the picture corresponding to the picture verification feature closest to the verification feature to be identified belongs to the classification of the picture to be identified.
  • the target multi-layer image object verification feature extraction model determined by the above-described neural network-based multi-layer image feature extraction modeling method determines the verification feature to be identified, and the image is verified by the verification feature to be identified and the training set. The features are compared to finally determine the classification of the image to be recognized. Therefore, the above image recognition method has high recognition accuracy.
  • the method further includes:
  • the present invention also provides a virtual device corresponding to a neural network based multi-layer image feature extraction modeling method.
  • a neural network-based multi-layer image feature extraction modeling apparatus of an embodiment includes:
  • a picture classification obtaining module 840 configured to acquire, from a training set of the preset application scenario, a first picture, a second picture, a first category of the first picture, and a second category of the second picture;
  • a loss cost determination module 860 configured to determine, as the input of the multi-layer image object verification neural network, the first picture, the first category, the second picture, and the second category to determine a global loss cost function value
  • the neural network training module 870 is configured to train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the training parameter;
  • a feature model determining module 880 configured to test the multi-layer image object verification neural network by using a test set of the preset application scenario, and determine a test accuracy according to the test result, according to the test accuracy and the multi-layer image
  • the object verification neural network determines the target multi-layer image object verification feature extraction model.
  • the above-mentioned neural network-based multi-layer image feature extraction modeling device because of the global loss cost function value used when training the model, is not only related to the first picture and the second picture, but also to the first classification and the second of the first picture.
  • the second category of the picture is related. Therefore, the modeled image object verification feature model is related to the first category of the first picture and the second category of the second picture. Therefore, the beneficial effect of improving the image recognition accuracy when the imaged object verification feature model obtained by the modeling is applied to the preset application scene for image recognition can be achieved.
  • the image object verification neural network is used to verify the multi-layer multi-layer image object verification neural network, and the determined image object verification feature extraction model is a multi-level multi-layer image object verification feature extraction model.
  • the multi-layer image object verification feature extraction model not only outputs the highest layer image object verification feature, but also outputs the image object verification feature of the middle layer. Therefore, the beneficial effect of improving the image recognition accuracy when the imaged object verification feature model obtained by the modeling is applied to the preset application scene for image recognition can be further achieved.
  • the method further includes:
  • the network acquisition determining module 950 is configured to acquire an image object recognition neural network, and determine a multi-layer image object verification neural network according to the image object recognition neural network.
  • the multi-layer image object verification neural network includes a multi-layer image object verification feature extraction model determined based on an image object recognition feature extraction model of the image object recognition neural network.
  • the loss cost determination module 960 includes:
  • a picture feature determining unit 961 (not shown) for the picture feature determining unit, configured to input the first picture and the first category as a model input of the multi-layer image object verification feature extraction model, and determine multiple layers a first object feature and a plurality of first verification features, the second picture and the second classification being input as another model of the multi-layer image object verification feature extraction model, and determining a plurality of second object features and a layer second verification feature; or the multi-layer image object verification feature extraction model includes the same two; a picture feature determining unit 961, configured to use the first picture and the first category as one of the plurality
  • the layer image object validates the model input of the feature extraction model, determines the multi-layer first object feature and the multi-layer first verification feature, and extracts the second image and the second classification as another multi-layer image object verification feature Model input of the model, determining a multi-layer second object feature and a multi-layer second verification feature;
  • a first loss determining unit (not shown), configured to determine a first object information loss function value according to the multi-layer first object feature and the first category;
  • a second loss determining unit (not shown), configured to determine a second object information loss function value according to the multi-layer second object feature and the second category;
  • a verification loss determining unit (not shown) for determining a verification loss function value according to the first classification, the second classification, the multi-layer first verification feature, and the multi-layer second verification feature;
  • a global loss determining unit (not shown) is configured to determine a global loss cost function value according to the first object information loss function value, the second object information loss function value, and the verification loss function value.
  • the image is a face image; the device further includes:
  • the picture collection and detection module 910 is configured to collect a video picture in the preset application scenario, and perform face detection on the video picture to obtain a face picture;
  • the image classification alignment module 920 is configured to acquire classification information for classifying the facial image, classify the facial image according to each of the classification information, and perform a facial expression on each of the classified facial images. Align processing to form a training set.
  • the neural network training module 970 includes:
  • the initial parameter training unit 971 is configured to acquire initial training parameters, and train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the initial training parameter;
  • the update parameter training unit 973 is configured to update the training parameter, and train the multi-layer image object verification neural network on the training set according to the global loss cost function value and the updated training parameter.
  • an image recognition apparatus of an embodiment includes:
  • the to-be-identified feature determining module 1040 is configured to acquire a picture to be recognized, and use the image to be recognized as an input of a target multi-layer image object verification feature extraction model determined by the neural network-based multi-layer image feature extraction modeling device. Determining the verification feature to be identified;
  • the comparison classification determining module 1060 is configured to compare the verification feature to be identified with the image verification feature corresponding to the picture in the training set, and determine the classification of the picture corresponding to the picture verification feature that is closest to the verification feature to be identified. The classification of the picture to be identified.
  • the image recognition apparatus described above determines the verification feature to be identified based on the target multi-layer image object verification feature extraction model determined by the above-described neural network-based multi-layer image feature extraction modeling device, and passes the image to be recognized and the image in the training set The verification features are compared to finally determine the classification of the image to be recognized, and therefore, the recognition accuracy of the image recognition apparatus described above is high.
  • the method further includes:
  • the feature model obtaining module 1020 is configured to acquire a target multi-layer image object verification feature extraction model.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Image Analysis (AREA)

Abstract

一种基于神经网络的多层图像特征提取建模方法及装置，从预设应用场景的训练集中获取第一图片、第二图片、第一图片的第一分类及第二图片的第二分类(S140)；根据第一图片、第一分类、第二图片及第二分类确定全局损失代价函数值(S160)；根据全局损失代价函数值及训练参数在训练集上训练多层图像对象验证神经网络(S170)；通过预设应用场景的测试集对多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据测试精度及多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型(S180)。该方法及装置可以达到将建模得到的图像特征模型应用图像识别应用场景中进行图像识别时，提高图片识别精度的有益效果。还提供一种图像识别方法及装置。

Description

基于神经网络的多层图像特征提取建模、图像识别方法及装置 技术领域
本发明涉及图像识别技术领域，尤其涉及一种基于神经网络的多层图像特征提取建模方法及装置以及一种图像识别方法及装置。
背景技术
图像识别是计算机对图像进行处理、分析和理解，以识别各种不同模式的目标和对像的技术。针对人脸的图像识别即为人脸识别，它是一种基于人的脸部特征信息进行身份识别的生物识别技术，一般是用摄像机或摄像头采集含有人脸的图像或视频流后，自动在图像中检测和跟踪人脸，进而对检测到的人脸进行脸部识别，通常也叫做人像识别、面部识别。
目前，人脸识别算法均是基于人脸照片和与其对应的身份信息，使用神经网络进行模型训练，并最终使用分类器进行人脸识别。在人脸识别神经网络中对模型的训练中只考虑人脸图片的身份信息，对于利用该模型的人脸识别的识别精度还有待进一步地提升。
发明内容
基于此，有必要提供一种能够在图像识别应用场景中提高识别精度的基于神经网络的多层图像特征提取建模方法及装置，以及一种应用通过该基于神经网络的多层图像特征提取建模方法及装置建立的图像特征模型的图像识别方法及装置。
一种基于神经网络的多层图像特征提取建模方法，包括：
从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类；
将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值；
根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络；
通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据 测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
一种图像识别方法，包括：
获取待识别图片，并将所述待识别图片作为上述的基于神经网络的多层图像特征提取建模方法确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征；
将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
一种基于神经网络的多层图像特征提取建模装置，包括：
图片分类获取模块，用于从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类；
损失代价确定模块，用于将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值；
神经网络训练模块，用于根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络；
特征模型确定模块，用于通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
一种图像识别装置，包括：
待识特征确定模块，用于获取待识别图片，并将所述待识别图片作为上述的基于神经网络的多层图像特征提取建模装置确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征；
对比分类确定模块，用于将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
上述基于神经网络的多层图像特征提取建模方法及装置，由于训练模型的时候采用的 全局损失代价函数值，不仅与第一图片、第二图片相关，还与第一图片的第一分类、第二图片的第二分类相关。因此，建模得到的图像对象验证特征模型与第一图片的第一分类、第二图片的第二分类相关。从而，可以达到将建模得到的图像对象验证特征模型应用在预设应用场景中进行图像识别时，提高图像识别精度的有益效果。同时，采用的图像对象验证神经网络为多层次的多层图像对象验证神经网络，确定的图像对象验证特征提取模型为多层次的多层图像对象验证特征提取模型。相较于图像对象验证特征提取模型，多层图像对象验证特征提取模型不仅输出最高层图像对象验证特征，还输出中间层的图像对象验证特征。从而，可以进一步达到将建模得到的图像对象验证特征模型应用在预设应用场景中进行图像识别时，提高图像识别精度的有益效果。
上述图像识别方法及装置，由于通过上述基于神经网络的多层图像特征提取建模方法或装置确定的目标多层图像对象验证特征提取模型确定待识别验证特征，并通过该待识别验证特征与训练集中的图片验证特征进行对比，最终确定待识别图片的分类，因此，上述图像识别方法及装置的识别精度高。
附图说明
图1为一实施例的基于神经网络的多层图像特征提取建模方法的流程图；
图2为图1的基于神经网络的多层图像特征提取建模方法的一个步骤的具体流程图；
图3为另一实施例的基于神经网络的多层图像特征提取建模方法的流程图；
图4为一实施例的基于神经网络的多层图像特征提取建模方法中进行人脸对齐处理前的示例图；
图5为图4的示例图进行人脸对齐处理后的结果图；
图6为图1的基于神经网络的多层图像特征提取建模方法的另一个步骤的具体流程图；
图7为一实施例的图像识别方法的流程图；
图8为一实施例的基于神经网络的多层图像特征提取建模装置的结构图；
图9为另一实施例的基于神经网络的多层图像特征提取建模装置的结构图；
图10为一实施例的图像识别装置的结构图。
具体实施方式
为了便于理解本发明，下面将参照相关附图对本发明进行更全面的描述。附图中给出了本发明的较佳的实施例。但是，本发明可以以许多不同的形式来实现，并不限于本文所描述的实施例。相反地，提供这些实施例的目的是使对本发明的公开内容的理解更加透彻全面。
除非另有定义，本文所使用的所有的技术和科学术语与属于本发明的技术领域的技术人员通常理解的含义相同。本文中在本发明的说明书中所使用的术语只是为了描述具体的实施例的目的，不是旨在于限制本发明。本文所使用的术语“或/和”包括一个或多个相关的所列项目的任意的和所有的组合。
如图1所示，为本发明一个实施例的基于神经网络的多层图像特征提取建模方法，包括：
S140：从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类。
预设应用场景可以为对图像识别精度要求较高的场景，尤其是对人脸识别精度要求较高的场景，如银行VTM(Virtual Teller Machine，远程柜员机)验证、珠宝店VIP(Very Important Person，贵宾)识别等场景。
每一张图片上均包括待识别的对象，如，可以为待识别的物品或人。同一分类表示同一个对象，如可以为同一个人或同一个物品。
S160：将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值。
通过第一图片及第二图片，根据多层图像对象验证神经网络中的多层图像对象验证特征提取模型可以分别确定第一图片、第二图片的对象特征及验证特征，进而根据第一分类、第二分类以及这些对象特征、验证特征确定全局损失代价函数的值。
多层图像对象验证神经网络以现有技术的图像对象识别神经网络为基础，包括以图像对象识别神经网络中的图像对象识别特征提取模型为基础确定的多层图像对象验证特征提取模型。
具体地，该图像对象识别神经网络为经过训练的图像对象识别神经网络，是采用现有技术进行图像识别的已经训练好了的神经网络。如此，在经过训练的图像对象识别神经网 络的基础上进行后续训练，而无需再从零开始训练。因此，可以节约训练时间，更快地找到最优神经网络。进一步地，经过训练的图像对象识别神经网络为深度神经网络，即图像对象识别深度神经网络。
根据多层图像对象验证特征提取模型可以构造多层图像对象验证神经网络。优选地，多层图像对象验证神经网络为深度神经网络，即图像对象验证深度神经网络。
需要说明的是，多层图像对象验证特征提取模型在通过图像对象识别特征提取模型得到图像对象识别特征的基础上，根据图像对象识别特征得到图像对象验证特征。具体地，对图像对象识别特征进行二范数归一化处理，得到图像对象验证特征。二范数归一化处理具体为，将图像对象识别特征的每一个特征元素求平方和之后再开平方得到的结果作为图像对象验证特征的每个特征元素分母；再将图像对象识别特征中的每个与验证是否为同一对象相关的特征元素的特征值作为图像对象验证特征的一个特征元素的分子。
S170：根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络。
在一个实施例中，可以根据全局损失代价函数值和链式求导法则可以确定多层图像对象验证神经网络的多层图像对象验证特征提取模型中的每一个参数的梯度值；根据所述全局损失代价函数值及训练参数采用随机梯度下降法在所述训练集上训练所述多层图像对象验证神经网络。
训练参数包括特征距离阈值及学习速率。在一个较优实施例中，特征距离阈值的值可以默认设置为0.2或0.25；学习速率的值可以默认设置为0.0001。
S180：通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
只要测试集的容量足够大，对多层图像对象验证神经网络的训练可以一直持续下去。在本实施例中，每次训练预设时间之后，通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度。可以采用现有方式根据测试结果确定测试精度。
当测试精度到达预设精度时，不再继续训练，此时的多层图像对象验证神经网络为目标多层图像对象验证神经网络。通过目标多层图像对象验证神经网络可以确定目标多层图 像对象验证特征提取模型。其中，预设精度为预先设置测试需要达到的精度要求。
在一个较优的实施例中，采用交叉验证的方式进行验证。测试集为与训练集没有交集的图像的集合，优选地，图像为人脸图片。
在一个具体的实施例中，测试集制作方式为：将N个分类中除了用于制作训练集的K个分类，剩下的N-K个分类的人脸照片用于制作测试集。测试集由随机抽取的人脸图片验证对组成，抽取规则如下：
第n个分类的第a张人脸图片，第n个分类的第b张人脸图片(正样本对)
                    ...
第i个分类的第c张人脸图片，第j个分类的第d张人脸图片(负样本对)
                    ...
按照国际标准人脸验证测试集的规则，此处正、负样本对各取3000对，共6000对。测试规则为：将正样本对中的两张照片判断成同一个人，则判断正确，即xi＝1；将负样本对中的两张照片判断成不是同一个人，则判断正确，即xi＝1；其它则判断错误，即xi＝0。则测试精度定义为：
Figure PCTCN2016113147-appb-000001
在其中一个实施例中，并不预先设置预设测试精度，预设精度的变化过程为先逐渐提升，当到达某个精度之后将产生较大波动；这里把这个精度记为最大稳定精度。因此，当测试精度不再稳定提升时，即到达最大稳定精度时，不再继续训练多层图像对象验证神经网络，当前多层图像对象验证神经网络即为最优的多层图像对象验证神经网络。取最优的多层图像对象验证神经网络中的一个多层图像对象验证特征提取模型，并仅以一张图片为输入、以图像对象验证特征为输出、而省略图像对象识别特征的输出，从而得到最终的目标多层图像对象验证特征提取模型。
上述基于神经网络的多层图像特征提取建模方法，由于训练模型的时候采用的全局损失代价函数值，不仅与第一图片、第二图片相关，还与第一图片的第一分类、第二图片的第二分类相关。因此，建模得到的图像对象验证特征模型与第一图片的第一分类、第二图片的第二分类相关。从而，可以达到将建模得到的图像对象验证特征模型应用在预设应用 场景中进行图像识别时，提高图像识别精度的有益效果。同时，采用的图像对象验证神经网络为多层次的多层图像对象验证神经网络，确定的图像对象验证特征提取模型为多层次的多层图像对象验证特征提取模型。相较于图像对象验证特征提取模型，多层图像对象验证特征提取模型不仅输出最高层图像对象验证特征，还输出中间层的图像对象验证特征。从而，可以进一步达到将建模得到的图像对象验证特征模型应用在预设应用场景中进行图像识别时，提高图像识别精度的有益效果。
请继续参阅图1，在其中一个实施例中，步骤S140之后，步骤S160之前，还包括步骤：
获取图像对象识别神经网络，并根据所述图像对象识别神经网络确定多层图像对象验证神经网络。
获取现有技术的进行图像对象识别的图像对象识别神经网络，并根据所述图像对象识别神经网络确定多层图像对象验证神经网络。
可以理解地，在其中一个图像为人脸图像的实施例中，图像对象识别神经网络为人脸身份识别神经网络，优选为，人脸身份识别深度神经网络；多层图像对象验证神经网络为人脸身份验证神经网络，优选为，人脸身份验证深度神经网络。其中，人脸身份识别神经网络和人脸身份验证神经网络的关系与图像对象识别神经网络和多层图像对象验证神经网络的关系一致，故在此不作赘述。
请参阅图2，在其中一个实施例中，所述多层图像对象验证神经网络包括以图像对象识别神经网络的图像对象识别特征提取模型为基础确定的多层图像对象验证特征提取模型。
所述将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为所述多层图像对象验证神经网络的输入，确定全局损失代价函数值的步骤，即S160，包括：
S261：将所述第一图片与所述第一分类作为所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为所述多层图像对象验证特征提取模型的另一个模型输入，确定多层第二对象特征与多层第二验证特征；或者，所述多层图像对象验证特征提取模型包括相同的两个，将所述第一图片与所述第一分类作为其中一个所述多层图像对象验证特征提取模型的模型输入，确定 多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为另一个所述多层图像对象验证特征提取模型的模型输入，确定多层第二对象特征与多层第二验证特征。
具体地，可以将所述第一图片与所述第一分类作为一个模型输入、所述第二图片与所述第二分类作为另一个模型输入，分两次先后作为所述多层图像对象验证神经网络的所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，以及多层第二对象特征与多层第二验证特征。也可以是所述多层图像对象验证特征提取模型包括相同的两个；将所述第一图片与所述第一分类作为一个所述多层图像对象验证特征提取模型的模型输入、将所述第二图片与所述第二分类作为另一个所述多层图像对象验证特征提取模型的模型输入，两个多层图像对象验证特征提取模型并行执行，最终分别确定多层第一对象特征与多层第一验证特征，以及多层第二对象特征与多层第二验证特征。
S263：根据所述多层第一对象特征及所述第一分类确定第一对象信息损失函数值。
可以采用现有神经网络中常用的方式根据一个对象特征及一个分类确定损失函数值。具体地，根据多层第一对象特征确定一个得到的分类信息，根据得到的分类信息与获取的第一分类是否属于同一类别，确定识别结果，进而反应到损失函数值中。
S265：根据所述多层第二对象特征及所述第二分类确定第二对象信息损失函数值。
第二对象信息损失函数值与第一对象信息损失函数值的确定是一致的，故在此不作赘述。
S267：根据所述第一分类、所述第二分类、所述多层第一验证特征及所述多层第二验证特征确定验证损失函数值。
具体地，验证损失函数的公式为：
VerifyLossk＝ydk+(1-y)max(α-dk，0)
其中，
Figure PCTCN2016113147-appb-000002
α为训练参数中的特征距离阈值。VerifyLossk表示第k层验证特征的验证损失函数值；y表示属于同一分类；N1表示第一分类，N2表示第二分类；dk为第k层验证特征的特征距离；
Figure PCTCN2016113147-appb-000003
表示多层第一验证特征中的第k层的第一验证特征，
Figure PCTCN2016113147-appb-000004
表示多层第二验证特征中的第k层的第二验证特 征；
Figure PCTCN2016113147-appb-000005
表示二范数运算。
S269：根据所述第一对象信息损失函数值、所述第二对象信息损失函数值及所述验证损失函数值确定全局损失代价函数值。
在本实施例中，全局损失代价函数值是关于第一对象信息损失函数值、第二对象信息损失函数值及验证损失函数的线性函数值。具体地，全局损失代价函数的公式为：
Figure PCTCN2016113147-appb-000006
其中，Loss为全局损失代价函数值，SoftmaxLoss_1为第一对象信息损失函数值；SoftmaxLoss_2为第二对象信息损失函数值；VerifyLossk为第k层验证特征的验证损失函数值；m为验证特征的总层数；λk为各层验证特征的权重系数。经过实验验证，λm＝1，λk＝0.2，k＝1，2，3，...，m-1时，能取得最好的训练效果和识别准确度。
由于在训练模型的过程中，全局损失代价函数值不仅与第一对象信息损失函数值及第二对象信息损失函数值相关，还与验证损失函数值相关，因此，可以进一步提高将建模得到的图像对象验证特征模型应用在预设应用场景中时，图像的识别精度。
请参阅图3，在其中一个实施例中，步骤S340～S380依次对应为步骤S140～S180。所述图像为人脸图片；所述从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类的步骤(即步骤S340)之前，还包括：
S310：在所述预设应用场景中采集视频图片，并对所述视频图片进行人脸检测得到人脸图片。
使用摄像头在预设应用场景中采集视频图片，并通过网络传输和数据线存放在计算机中。通过现有方式对采集到的视频图片进行人脸检测，将人脸图片提取出来存储在计算机硬盘中。
S320：获取对所述人脸图片进行分类的分类信息，根据所述分类信息对各所述人脸图片进行分类，并对分类后的各所述人脸图片进行人脸对齐处理，形成训练集。
人工对检测并提取出的人脸图片进行分类，因此，计算机获取人工输入的分类信息，并根据分类信息分类。属于相同分类的人脸照片放在一起并通过分类信息予以标记。
由于人脸图片中的人脸角度和人脸位置是不一致的，为了保证提取到稳定的特征并取 得较好的人脸识别效果，需要通过现有方式对人脸图片进行关键点对齐操作，以进行人脸对齐处理，去除人脸角度对人脸识别带来的影响。其中，关键点包括眼睛、鼻尖和嘴角等的位置。如图4所示为一个采集到的人脸图片的，即进行人脸对齐处理前的示例图，对齐后的人脸图片如图5所示。
请参阅图6，在其中一个实施例中，所述根据所述损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络的步骤，即步骤S170，包括：
S671：获取初始训练参数，根据所述全局损失代价函数值及所述初始训练参数在所述训练集上训练所述多层图像对象验证神经网络。
S673：更新训练参数，根据所述全局损失代价函数值及更新后的所述训练参数在所述训练集上训练所述多层图像对象验证神经网络。
如此，不断调整多层图像对象验证神经网络的训练参数进行训练，确定最优地训练参数。通过大量调试和试验，发现针对本文描述的方法，特征距离阈值α＝0.2且学习速率lr＝0.001时，能得到最好的算法精度提升。
如图7所示，本发明还提供一种应用上述基于神经网络的多层图像特征提取建模方法的图像识别方法，包括：
S740：获取待识别图片，并将所述待识别图片作为基于神经网络的多层图像特征提取建模方法确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征。
目标多层图像对象验证特征提取模型为通过上述基于神经网络的多层图像特征提取建模方法建立的目标多层图像对象验证特征提取模型。
具体地，通过摄像头采集待识别图片，并将该待识别图片传输至计算机；计算机获取该待识别图片，将该待识别图片输入所建立的目标多层图像对象验证特征提取模型进行运算，从而可以确定待识别图片的图片特征，即待识别特征。
S760：将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
在本实施例中，预先通过目标多层图像对象验证特征提取模型确定训练集中所有图片的图片验证特征。确定待识别验证特征之后，计算待识别验证特征与训练集中的各个图片 的图片验证特征的距离；与待识别验证特征距离最近的图片验证特征对应的图片所属的分类即为待识别图片的分类。
上述图像识别方法，由于通过上述基于神经网络的多层图像特征提取建模方法确定的目标多层图像对象验证特征提取模型确定待识别验证特征，并通过该待识别验证特征与训练集中的图片验证特征进行对比，最终确定待识别图片的分类，因此，上述图像识别方法的识别精度高。
请继续参阅图7，在其中一个实施例中，还包括：
S720：获取目标多层图像对象验证特征提取模型。
本发明还提供一种与基于神经网络的多层图像特征提取建模方法对应的虚拟装置。如图8所示，一个实施例的基于神经网络的多层图像特征提取建模装置，包括：
图片分类获取模块840，用于从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类；
损失代价确定模块860，用于将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值；
神经网络训练模块870，用于根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络；
特征模型确定模块880，用于通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
上述基于神经网络的多层图像特征提取建模装置，由于训练模型的时候采用的全局损失代价函数值，不仅与第一图片、第二图片相关，还与第一图片的第一分类、第二图片的第二分类相关。因此，建模得到的图像对象验证特征模型与第一图片的第一分类、第二图片的第二分类相关。从而，可以达到将建模得到的图像对象验证特征模型应用在预设应用场景中进行图像识别时，提高图像识别精度的有益效果。同时，采用的图像对象验证神经网络为多层次的多层图像对象验证神经网络，确定的图像对象验证特征提取模型为多层次的多层图像对象验证特征提取模型。相较于图像对象验证特征提取模型，多层图像对象验证特征提取模型不仅输出最高层图像对象验证特征，还输出中间层的图像对象验证特征。 从而，可以进一步达到将建模得到的图像对象验证特征模型应用在预设应用场景中进行图像识别时，提高图像识别精度的有益效果。
请参阅图9，在其中一个实施例中，还包括：
网络获取确定模块950，用于获取图像对象识别神经网络，并根据所述图像对象识别神经网络确定多层图像对象验证神经网络。
请继续参阅图9，在其中一个实施例中，所述多层图像对象验证神经网络包括以图像对象识别神经网络的图像对象识别特征提取模型为基础确定的多层图像对象验证特征提取模型。所述损失代价确定模块960，包括：
图片特征确定单元961(图未示)，用于图片特征确定单元，用于将所述第一图片与所述第一分类作为所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为所述多层图像对象验证特征提取模型的另一个模型输入，确定多层第二对象特征与多层第二验证特征；或者，所述多层图像对象验证特征提取模型包括相同的两个；图片特征确定单元961，用于将所述第一图片与所述第一分类作为其中一个所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为另一个所述多层图像对象验证特征提取模型的模型输入，确定多层第二对象特征与多层第二验证特征；
第一损失确定单元(图未示)，用于根据所述多层第一对象特征及所述第一分类确定第一对象信息损失函数值；
第二损失确定单元(图未示)，用于根据所述多层第二对象特征及所述第二分类确定第二对象信息损失函数值；
验证损失确定单元(图未示)，用于根据所述第一分类、所述第二分类、所述多层第一验证特征及所述多层第二验证特征确定验证损失函数值；
全局损失确定单元(图未示)，用于根据所述第一对象信息损失函数值、所述第二对象信息损失函数值及所述验证损失函数值确定全局损失代价函数值。
请继续参阅图9，在其中一个实施例中，所述图像为人脸图片；所述装置，还包括：
图片采集检测模块910，用于在所述预设应用场景中采集视频图片，并对所述视频图片进行人脸检测得到人脸图片；
图片分类对齐模块920，用于获取对所述人脸图片进行分类的分类信息，根据各所述分类信息对所述人脸图片进行分类，并对分类后的各所述人脸图片进行人脸对齐处理，形成训练集。
在其中一个实施例中，所述神经网络训练模块970，包括：
初始参数训练单元971，用于获取初始训练参数，根据所述全局损失代价函数值及所述初始训练参数在所述训练集上训练所述多层图像对象验证神经网络；
更新参数训练单元973，用于更新训练参数，根据所述全局损失代价函数值及更新后的所述训练参数在所述训练集上训练所述多层图像对象验证神经网络。
本发明还提供一种与图像识别方法对应的虚拟装置。如图10所示，一个实施例的图像识别装置，包括：
待识特征确定模块1040，用于获取待识别图片，并将所述待识别图片作为上述的基于神经网络的多层图像特征提取建模装置确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征；
对比分类确定模块1060，用于将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
上述图像识别装置，由于根据通过上述基于神经网络的多层图像特征提取建模装置确定的目标多层图像对象验证特征提取模型确定待识别验证特征，并通过该待识别验证特征与训练集中的图片验证特征进行对比，最终确定待识别图片的分类，因此，上述图像识别装置的识别精度高。
在其中一个实施例中，还包括：
特征模型获取模块1020，用于获取目标多层图像对象验证特征提取模型。
以上实施例仅表达了本发明的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出多个变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。

Claims (10)

  1. 一种基于神经网络的多层图像特征提取建模方法，其特征在于，包括：
    从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类；
    将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值；
    根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络；
    通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
  2. 根据权利要求1所述的基于神经网络的多层图像特征提取建模方法，其特征在于，所述多层图像对象验证神经网络包括以图像对象识别神经网络的图像对象识别特征提取模型为基础确定的多层图像对象验证特征提取模型；
    所述将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为所述多层图像对象验证神经网络的输入，确定全局损失代价函数值的步骤包括：
    将所述第一图片与所述第一分类作为所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为所述多层图像对象验证特征提取模型的另一个模型输入，确定多层第二对象特征与多层第二验证特征；或者，所述多层图像对象验证特征提取模型包括相同的两个，将所述第一图片与所述第一分类作为其中一个所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为另一个所述多层图像对象验证特征提取模型的模型输入，确定多层第二对象特征与多层第二验证特征；
    根据所述多层第一对象特征及所述第一分类确定第一对象信息损失函数值；
    根据所述多层第二对象特征及所述第二分类确定第二对象信息损失函数值；
    根据所述第一分类、所述第二分类、所述多层第一验证特征及所述多层第二验证特征确定验证损失函数值；
    根据所述第一对象信息损失函数值、所述第二对象信息损失函数值及所述验证损失函 数值确定全局损失代价函数值。
  3. 根据权利要求1所述的基于神经网络的多层图像特征提取建模方法，其特征在于，所述图像为人脸图片；
    所述从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类的步骤之前，还包括：
    在所述预设应用场景中采集视频图片，并对所述视频图片进行人脸检测得到人脸图片；
    获取对所述人脸图片进行分类的分类信息，根据所述分类信息对各所述人脸图片进行分类，并对分类后的各所述人脸图片进行人脸对齐处理，形成训练集。
  4. 根据权利要求1所述的基于神经网络的多层图像特征提取建模方法，其特征在于，所述根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络的步骤，包括：
    获取初始训练参数，根据所述全局损失代价函数值及所述初始训练参数在所述训练集上训练所述多层图像对象验证神经网络；
    更新训练参数，根据所述全局损失代价函数值及更新后的所述训练参数在所述训练集上训练所述多层图像对象验证神经网络。
  5. 一种图像识别方法，其特征在于，包括：
    获取待识别图片，并将所述待识别图片作为权利要求1-4任意一项所述的基于神经网络的多层图像特征提取建模方法确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征；
    将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
  6. 一种基于神经网络的多层图像特征提取建模装置，其特征在于，包括：
    图片分类获取模块，用于从预设应用场景的训练集中获取第一图片、第二图片、所述第一图片的第一分类及所述第二图片的第二分类；
    损失代价确定模块，用于将所述第一图片、所述第一分类、所述第二图片及所述第二分类作为多层图像对象验证神经网络的输入，确定全局损失代价函数值；
    神经网络训练模块，用于根据所述全局损失代价函数值及训练参数在所述训练集上训练所述多层图像对象验证神经网络；
    特征模型确定模块，用于通过所述预设应用场景的测试集对所述多层图像对象验证神经网络进行测试，并根据测试结果确定测试精度，根据所述测试精度及所述多层图像对象验证神经网络确定目标多层图像对象验证特征提取模型。
  7. 根据权利要求6所述的基于神经网络的多层图像特征提取建模装置，其特征在于，所述多层图像对象验证神经网络包括以图像对象识别神经网络的图像对象识别特征提取模型为基础确定的多层图像对象验证特征提取模型；
    所述损失代价确定模块，包括：
    图片特征确定单元，用于将所述第一图片与所述第一分类作为所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为所述多层图像对象验证特征提取模型的另一个模型输入，确定多层第二对象特征与多层第二验证特征；或者，所述多层图像对象验证特征提取模型包括相同的两个；图片特征确定单元，用于将所述第一图片与所述第一分类作为其中一个所述多层图像对象验证特征提取模型的模型输入，确定多层第一对象特征与多层第一验证特征，将所述第二图片与所述第二分类作为另一个所述多层图像对象验证特征提取模型的模型输入，确定多层第二对象特征与多层第二验证特征；
    第一损失确定单元，用于根据所述多层第一对象特征及所述第一分类确定第一对象信息损失函数值；
    第二损失确定单元，用于根据所述多层第二对象特征及所述第二分类确定第二对象信息损失函数值；
    验证损失确定单元，用于根据所述第一分类、所述第二分类、所述多层第一验证特征及所述多层第二验证特征确定验证损失函数值；
    全局损失确定单元，用于根据所述第一对象信息损失函数值、所述第二对象信息损失函数值及所述验证损失函数值确定全局损失代价函数值。
  8. 根据权利要求6所述的基于神经网络的多层图像特征提取建模装置，其特征在于，所述图像为人脸图片；所述装置，还包括：
    图片采集检测模块，用于在所述预设应用场景中采集视频图片，并对所述视频图片进 行人脸检测得到人脸图片；
    图片分类对齐模块，用于获取对所述人脸图片进行分类的分类信息，根据各所述分类信息对所述人脸图片进行分类，并对分类后的各所述人脸图片进行人脸对齐处理，形成训练集。
  9. 根据权利要求6所述的基于神经网络的多层图像特征提取建模装置，其特征在于，所述神经网络训练模块，包括：
    初始参数训练单元，用于获取初始训练参数，根据所述全局损失代价函数值及所述初始训练参数在所述训练集上训练所述多层图像对象验证神经网络；
    更新参数训练单元，用于更新训练参数，根据所述全局损失代价函数值及更新后的所述训练参数在所述训练集上训练所述多层图像对象验证神经网络。
  10. 一种图像识别装置，其特征在于，包括：
    待识特征确定模块，用于获取待识别图片，并将所述待识别图片作为权利要求6-9任意一项所述的基于神经网络的多层图像特征提取建模装置确定的目标多层图像对象验证特征提取模型的输入，确定待识别验证特征；
    对比分类确定模块，用于将所述待识别验证特征与训练集中的图片对应的图片验证特征进行对比，并将与所述待识别验证特征距离最近的图片验证特征对应的图片所属的分类确定为所述待识别图片的分类。
PCT/CN2016/113147 2016-10-14 2016-12-29 基于神经网络的多层图像特征提取建模、图像识别方法及装置 Ceased WO2018068416A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610900080.5A CN106529571B (zh) 2016-10-14 2016-10-14 基于神经网络的多层图像特征提取建模、图像识别方法及装置
CN201610900080.5 2016-10-14

Publications (1)

Publication Number Publication Date
WO2018068416A1 true WO2018068416A1 (zh) 2018-04-19

Family

ID=58331848

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/113147 Ceased WO2018068416A1 (zh) 2016-10-14 2016-12-29 基于神经网络的多层图像特征提取建模、图像识别方法及装置

Country Status (2)

Country Link
CN (1) CN106529571B (zh)
WO (1) WO2018068416A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543526A (zh) * 2018-10-19 2019-03-29 谢飞 基于深度差异性特征的真假面瘫识别系统
CN110569737A (zh) * 2019-08-15 2019-12-13 深圳华北工控软件技术有限公司 一种人脸识别深度学习方法及加速人脸识别摄像机
CN111128348A (zh) * 2019-12-27 2020-05-08 上海联影智能医疗科技有限公司 医学图像处理方法、装置、存储介质及计算机设备
CN111382619A (zh) * 2018-12-28 2020-07-07 广州市百果园信息技术有限公司 图片推荐模型的生成、图片推荐方法、装置、设备及介质
CN111382601A (zh) * 2018-12-28 2020-07-07 河南中原大数据研究院有限公司 生成对抗网络模型的光照人脸图像识别预处理系统及方法
CN111461052A (zh) * 2020-04-13 2020-07-28 安徽大学 基于迁移学习的多个生育期小麦倒伏区域识别方法
CN111767826A (zh) * 2020-06-24 2020-10-13 浙江大学 一种定时定点场景异常检测方法
CN113255695A (zh) * 2021-05-21 2021-08-13 广州广电运通金融电子股份有限公司 一种目标重识别的特征提取方法及系统
CN113887532A (zh) * 2021-11-17 2022-01-04 安徽省公共气象服务中心 一种基于场景分类的高速公路积雪图像识别与订正方法
CN114120420A (zh) * 2021-12-01 2022-03-01 北京百度网讯科技有限公司 图像检测方法和装置
CN115909442A (zh) * 2022-11-09 2023-04-04 联想(北京)有限公司 一种识别模型的获得方法、装置及电子设备

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110069648A (zh) * 2017-09-25 2019-07-30 杭州海康威视数字技术股份有限公司 一种图像检索方法及装置
KR102374747B1 (ko) * 2017-12-15 2022-03-15 삼성전자주식회사 객체를 인식하는 장치 및 방법
CN114879702B (zh) * 2022-07-06 2022-09-30 季华实验室 多智能体巡检控制方法、装置、系统、设备及介质
CN118379600A (zh) * 2024-06-19 2024-07-23 深圳市超像素智能科技有限公司 图像处理方法、神经网络的训练方法及相关装置

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150347820A1 (en) * 2014-05-27 2015-12-03 Beijing Kuangshi Technology Co., Ltd. Learning Deep Face Representation
CN105512273A (zh) * 2015-12-03 2016-04-20 中山大学 一种基于可变长深度哈希学习的图像检索方法
CN105608450A (zh) * 2016-03-01 2016-05-25 天津中科智能识别产业技术研究院有限公司 基于深度卷积神经网络的异质人脸识别方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150347820A1 (en) * 2014-05-27 2015-12-03 Beijing Kuangshi Technology Co., Ltd. Learning Deep Face Representation
CN105512273A (zh) * 2015-12-03 2016-04-20 中山大学 一种基于可变长深度哈希学习的图像检索方法
CN105608450A (zh) * 2016-03-01 2016-05-25 天津中科智能识别产业技术研究院有限公司 基于深度卷积神经网络的异质人脸识别方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
SCHROFF, FLORIAN ET AL.: "FaceNet: A Unified Embedding for Face Recognition and Clustering", THE IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR, 31 December 2015 (2015-12-31), pages 815 - 822, XP032793492 *

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109543526A (zh) * 2018-10-19 2019-03-29 谢飞 基于深度差异性特征的真假面瘫识别系统
CN111382619B (zh) * 2018-12-28 2023-04-18 广州市百果园信息技术有限公司 图片推荐模型的生成、图片推荐方法、装置、设备及介质
CN111382619A (zh) * 2018-12-28 2020-07-07 广州市百果园信息技术有限公司 图片推荐模型的生成、图片推荐方法、装置、设备及介质
CN111382601A (zh) * 2018-12-28 2020-07-07 河南中原大数据研究院有限公司 生成对抗网络模型的光照人脸图像识别预处理系统及方法
CN110569737A (zh) * 2019-08-15 2019-12-13 深圳华北工控软件技术有限公司 一种人脸识别深度学习方法及加速人脸识别摄像机
CN111128348A (zh) * 2019-12-27 2020-05-08 上海联影智能医疗科技有限公司 医学图像处理方法、装置、存储介质及计算机设备
CN111128348B (zh) * 2019-12-27 2024-03-26 上海联影智能医疗科技有限公司 医学图像处理方法、装置、存储介质及计算机设备
CN111461052B (zh) * 2020-04-13 2023-05-23 安徽大学 基于迁移学习的多个生育期小麦倒伏区域识别方法
CN111461052A (zh) * 2020-04-13 2020-07-28 安徽大学 基于迁移学习的多个生育期小麦倒伏区域识别方法
CN111767826A (zh) * 2020-06-24 2020-10-13 浙江大学 一种定时定点场景异常检测方法
CN113255695A (zh) * 2021-05-21 2021-08-13 广州广电运通金融电子股份有限公司 一种目标重识别的特征提取方法及系统
CN113887532A (zh) * 2021-11-17 2022-01-04 安徽省公共气象服务中心 一种基于场景分类的高速公路积雪图像识别与订正方法
CN114120420A (zh) * 2021-12-01 2022-03-01 北京百度网讯科技有限公司 图像检测方法和装置
CN114120420B (zh) * 2021-12-01 2024-02-13 北京百度网讯科技有限公司 图像检测方法和装置
CN115909442A (zh) * 2022-11-09 2023-04-04 联想(北京)有限公司 一种识别模型的获得方法、装置及电子设备
CN115909442B (zh) * 2022-11-09 2026-02-27 联想(北京)有限公司 一种识别模型的获得方法、装置及电子设备

Also Published As

Publication number Publication date
CN106529571A (zh) 2017-03-22
CN106529571B (zh) 2018-04-20

Similar Documents

Publication Publication Date Title
CN106529571B (zh) 基于神经网络的多层图像特征提取建模、图像识别方法及装置
CN105975959B (zh) 基于神经网络的人脸特征提取建模、人脸识别方法及装置
CN106250866A (zh) 基于神经网络的图像特征提取建模、图像识别方法及装置
US10565433B2 (en) Age invariant face recognition using convolutional neural networks and set distances
US9747494B2 (en) Facial matching system
CN107122744B (zh) 一种基于人脸识别的活体检测系统及方法
CN105608446B (zh) 一种视频流异常事件的检测方法及装置
CN104036236B (zh) 一种基于多参数指数加权的人脸性别识别方法
US20170262472A1 (en) Systems and methods for recognition of faces e.g. from mobile-device-generated images of faces
CN109190475B (zh) 一种人脸识别网络与行人再识别网络协同训练方法
CN109800643A (zh) 一种活体人脸多角度的身份识别方法
US12223768B2 (en) Adaptive learning and matching of face modalities
CN109214373A (zh) 一种用于考勤的人脸识别系统及方法
JP7337541B2 (ja) 情報処理装置、情報処理方法及びプログラム
CN108549883A (zh) 一种人脸再识别方法
CN113343198A (zh) 一种基于视频的随机手势认证方法及系统
TW201917636A (zh) 一種基於線上學習的人臉辨識方法與系統
CN106295574A (zh) 基于神经网络的人脸特征提取建模、人脸识别方法及装置
CN204791050U (zh) 身份认证设备
Alniemi et al. Class Attendance System Based on Face Recognition.
Varma et al. Human face detection and recognition using artificial intelligence
US10755074B2 (en) Latent fingerprint pattern estimation
Goud et al. Smart attendance notification system using SMTP with face recognition
KR20220156432A (ko) 딥러닝을 활용한 사진 기반 건축공사 하자 검출 시스템
CN110738774A (zh) 一种人脸识别快速自助登机方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16918939

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 04/10/2019)

122 Ep: pct application non-entry in european phase

Ref document number: 16918939

Country of ref document: EP

Kind code of ref document: A1