CN106446754A - Image identification method, metric learning method, image source identification method and devices - Google Patents

Abstract

The application discloses an image identification method, an image identification device, a metric learning method, a metric learning device, an image source identification method and an image source identification device. The image identification method comprises the following steps: getting a to-be-identified object image; extracting the object characteristics of the to-be-identified object image; selecting a similarity measurement model corresponding to the source category of the to-be-identified object image from a pre-trained measurement model set, and calculating the similarity between the object characteristics and registered image object characteristics as the basis of an output object identification result, wherein the measurement model set contains at least one similarity measurement model, and different similarity measurement models correspond to different source categories of object images. By using the method in image identification, the problem about asymmetric object image identification can be solved effectively, and higher robustness and higher accuracy are achieved for identification of to-be-identified object images of changeable sources.

Description

Image identification method, metric learning method, image source identification method and device

Technical Field

The present application relates to pattern recognition technology, and in particular, to an image recognition method and apparatus. The application also provides a metric learning method and device and an image source identification method and device.

Background

Face recognition is one of the hot topics studied in the fields of pattern recognition, image processing, machine vision, neural networks, cognitive science, and the like in recent years. Face recognition generally refers to a computer technology for extracting visual features with identification capability from a face image and determining the identity of a face by using the visual features, and the technology can be specifically divided into two types: face identification and face verification. The face identification refers to identifying the identity of a certain face image, namely determining which person the certain face image is; the face verification means that whether the identity of a face image is a person claimed is judged.

Existing face recognition techniques generally involve two main directions of research: feature learning and metric learning. The purpose of feature learning is to convert the face image into a more separable form with higher identification capability; and the metric learning is used for learning a metric model or a metric function for evaluating the distance or the similarity between samples from the training samples, wherein the combined Bayesian face is a metric learning method which is currently applied and popularized and is derived based on the probability discriminant analysis of Gaussian hypothesis.

The main process of face recognition comprises the following steps: a training process and a recognition process. The training process is to solve parameters of a similarity metric model by using a face image training set, which is also called a metric learning process, wherein the face image training set is composed of face images and identity labels (which images are identified to come from the same person and which images are from different persons); the identification process includes firstly collecting a face image registration set for inquiry, wherein the registration set generally comprises face images, identity labels and identity information, the source of the face images is generally single, the quality of the face images is good, then comparing the features of the face images to be identified with the sample features in the registration set, and calculating the similarity between the features of the face images to be identified and the features of the registration images by using a trained similarity measurement model, so as to determine the identity corresponding to the face images to be identified.

Since the basic assumption of the joint bayesian face is: the face samples x and y participating in the comparison obey the same gaussian distribution, and in specific applications, the image sources in the registered set are usually controllable, and the sources of the face images to be recognized are more complex and have different qualities, such as: video capture, scanned pictures, stickers, etc., namely: the sources of the images in the registered set and the images to be recognized may be different, so that the face samples participating in the comparison may not meet the requirement of obeying the same gaussian distribution (also referred to as asymmetric faces). In the recognition application for other object images, the above problem caused by different image sources (i.e. asymmetric object images) also exists.

Disclosure of Invention

The embodiment of the application provides an image identification method and device, which aim to solve the problem that the existing image identification technology is low in accuracy rate for identifying object images with variable sources. The embodiment of the application also provides a metric learning method and device, and an image source identification method and device.

The application provides an image recognition method, which comprises the following steps:

acquiring an object image to be identified;

extracting object features of the object image to be recognized;

selecting a similarity measurement model corresponding to the source type of the object image to be recognized from a pre-trained measurement model set, and calculating the similarity between the object characteristics and the object characteristics of the registered image to be used as a basis for outputting an object recognition result;

the measurement model set comprises at least one similarity measurement model, and different similarity measurement models respectively correspond to different source types of the object image.

Optionally, each similarity measurement model corresponding to different source categories in the measurement model set is obtained by respectively training a reference object image training set belonging to a preset source category and a comparison object image training set corresponding to different source categories.

Optionally, the object images in the reference object image training set and the registered images belong to the same source category.

Optionally, before the step of selecting the similarity metric model corresponding to the source category of the object image to be recognized from the pre-trained metric model set, the following operations are performed:

and determining the source type of the object image to be recognized by taking the object characteristics as input and utilizing a pre-trained object image source classification model.

Optionally, the object image source classification model is a multi-class classification model obtained by training with the following algorithm:

a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

Optionally, the similarity metric model includes: and establishing an asymmetric measurement model under the assumption that the object features participating in comparison obey respective Gaussian distribution.

Optionally, the asymmetric metric model includes: an asymmetric metric model based on a combined Bayesian face;

the above asymmetric metric model corresponding to a specific source class is obtained by training through the following steps:

extracting object features of images in a reference object image training set belonging to a preset source category to serve as a reference feature sample set;

extracting object features of all images in the comparison object image training set belonging to the specific source category to serve as a comparison feature sample set;

under the assumption that the object features involved in comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters;

and solving parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label for identifying whether the samples belong to the same object, and finishing the training of the model.

Optionally, the asymmetric metric model corresponding to a specific source category is as follows:

A=(S_xx+T_xx)^-1-E

B＝(S_yy+T_yy)^-1-F

G＝-(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1S_xy(S_yy+T_yy)^-1

E＝(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1

F＝(S_yy+T_yy-S_yx(S_xx+T_xx)^-1S_xy)^-1

wherein, assume that sample X in the reference feature sample set X is μ_x+_x，μ_xAnd_xobedience mean 0, covariance matrix S_xxAnd T_xxThe gaussian distribution of (2) is compared with the samples Y ═ mu in the characteristic sample set Y_y+_y，μ_yAnd_yobedience mean 0, covarianceThe difference matrix is S_yyAnd T_yyGaussian distribution of (S)_xyAnd S_yxIs a cross-covariance matrix between X and Y; r (x, y) is the similarity calculated based on the intra-class/inter-class log-likelihood ratio;

the solving for the parameters in the asymmetric metrology model comprises: solving for S_xx、T_xx、S_yy、T_yy、S_xyAnd S_yx。

Optionally, the solving parameters in the asymmetric metric model includes:

estimating parameters in the model using a divergence matrix; or,

and iteratively solving parameters in the model by adopting an expectation-maximization algorithm.

Optionally, the calculating the similarity between the object feature and the object feature of the registration image includes:

calculating the similarity between the object characteristics and the registered image object characteristics corresponding to the specific identity;

after the step of calculating the similarity, the following operations are performed:

judging whether the similarity is larger than a preset threshold value or not;

if yes, the image of the object to be recognized and the registered image corresponding to the specific identity belong to the same object, and the judgment is output as an object recognition result.

Optionally, the calculating the similarity between the object feature and the object feature of the registration image includes:

calculating the similarity between the object characteristics and the registered image object characteristics in the specified range;

after the step of calculating the similarity, the following operations are performed:

judging whether the maximum value in the calculated similarity is larger than a preset threshold value or not;

if yes, the object image to be recognized is judged to be successfully matched in the registered images in the specified range, and the related identity information of the registered image corresponding to the maximum value is output as an object recognition result.

Optionally, the extracting the object feature of the object image to be recognized includes:

extracting the object features by adopting a local binary pattern algorithm; or,

extracting the object features by adopting a Gabor wavelet transform algorithm; or,

and extracting the object features by adopting a deep convolutional network.

Optionally, the image of the object to be recognized includes: a face image to be recognized; the object features include: human face characteristics.

Optionally, the source categories include:

identification photographs, life photographs, video screenshots, scanned images, copied images, or monitored pictures.

Correspondingly, the present application also provides an image recognition apparatus, comprising:

the image acquisition unit is used for acquiring an object image to be identified;

the characteristic extraction unit is used for extracting object characteristics of the object image to be identified;

the similarity calculation unit is used for selecting a similarity measurement model corresponding to the source type of the object image to be recognized from a pre-trained measurement model set, and calculating the similarity between the object characteristics and the object characteristics of the registered image to be used as a basis for outputting an object recognition result;

wherein the similarity calculation unit includes:

the measurement model selection subunit is used for selecting a similarity measurement model corresponding to the source type of the object image to be identified from a pre-trained measurement model set;

and the calculation execution subunit is used for calculating the similarity between the object feature and the object feature of the registration image by using the similarity metric model selected by the metric model selection subunit, and the similarity is used as a basis for outputting an object recognition result.

Optionally, the apparatus includes:

and the measurement model training unit is used for respectively training to obtain each similarity measurement model corresponding to different source categories in the measurement model set by utilizing a reference object image training set belonging to a preset source category and a comparison object image training set corresponding to different source categories.

Optionally, the apparatus includes:

and the source type determining unit is used for determining the source type of the object image to be recognized by taking the object characteristics as input and utilizing a pre-trained object image source classification model before triggering the similarity calculating unit to work.

Optionally, the apparatus includes:

the source classification model training unit is used for training and training the object image source classification model by adopting the following algorithm before triggering the source type determining unit to work: a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

Optionally, the apparatus includes:

a metric model training unit, configured to train each similarity metric model in the metric model set, where the similarity metric model includes: establishing an asymmetric measurement model based on a combined Bayesian face under the assumption that the object features involved in comparison obey respective Gaussian distribution;

the metric model training unit trains the above asymmetric metric model corresponding to a particular source class by the following subunits:

the reference sample extraction subunit is used for extracting object features of images in a reference object image training set belonging to a preset source category to serve as a reference feature sample set;

a comparison sample extraction subunit, configured to extract an object feature of each image in the comparison object image training set belonging to the specific source category, as a comparison feature sample set;

the measurement model establishing subunit is used for establishing an asymmetric measurement model containing parameters under the assumption that the object characteristics involved in comparison obey respective Gaussian distribution;

and the model parameter solving subunit is used for solving the parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label indicating whether the samples belong to the same object, so as to complete the training of the model.

Optionally, the model parameter solving subunit is specifically configured to estimate parameters in the model by using a divergence matrix, or iteratively solve the parameters in the model by using an expectation-maximization algorithm.

Optionally, the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registration image corresponding to the specific identity;

the device further comprises:

the first threshold comparison unit is used for judging whether the similarity is greater than a preset threshold;

and the first identification result output unit is used for judging that the image of the object to be identified and the registered image corresponding to the specific identity belong to the same object and outputting the judgment as an object identification result when the output of the first threshold comparison unit is yes.

Optionally, the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registration image in a specified range;

the device further comprises:

the second threshold comparison unit is used for judging whether the maximum value in the calculated similarity is larger than a preset threshold or not;

and the second identification result output unit is used for judging that the object image to be identified is successfully matched in the registered images in the specified range when the output of the second threshold comparison unit is yes, and outputting the related identity information of the registered image corresponding to the maximum value as an object identification result.

Optionally, the feature extraction unit is specifically configured to extract the object feature by using a local binary pattern algorithm, extract the object feature by using a Gabor wavelet transform algorithm, or extract the object feature by using a deep convolution network.

In addition, the present application also provides a metric learning method, including:

extracting object features of images in a reference object image training set belonging to the same source category to serve as a reference feature sample set;

extracting object features of all images in a comparison object image training set which belong to the same source type and belong to different source types from the reference object image to serve as a comparison feature sample set;

under the assumption that the object features involved in comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters;

and solving parameters in the asymmetric measurement model by using samples in the two types of characteristic sample sets.

Optionally, the asymmetric metric model includes: an asymmetric metric model based on a combined Bayesian face;

the asymmetric metric model is as follows:

A＝(S_xx+T_xx)^-1-E

B＝(S_yy+T_yy)^-1-F

G＝-(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1S_xy(S_yy+T_yy)^-1

E＝(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1

F＝(S_yy+T_yy-S_yx(S_xx+T_xx)^-1S_xy)^-1

wherein, assume that sample X in the reference feature sample set space X is μ_x+_x，μ_xAnd_xobedience mean 0, covariance matrix S_xxAnd T_xxComparing the samples Y in the feature sample set space Y to mu_y+_y，μ_yAnd_yobedience mean 0, covariance matrix S_yyAnd T_yyGaussian distribution of (S)_xyAnd S_yxIs a cross-covariance matrix between X and Y; r (x, y) is the similarity calculated based on the intra-class/inter-class log-likelihood ratio;

the solving for the parameters in the asymmetric metrology model comprises: solving for S_xx、T_xx、S_yy、T_yy、S_xyAnd S_yx。

Optionally, the solving parameters in the asymmetric metric model includes:

estimating parameters in the model using a divergence matrix; or,

and iteratively solving parameters in the model by adopting an expectation-maximization algorithm.

Optionally, the reference object image and the comparison object image include: a face image; the object features include: human face characteristics.

Correspondingly, the present application also provides a metric learning apparatus, comprising:

the reference sample extraction unit is used for extracting object features of images in a reference object image training set belonging to the same source category to serve as a reference feature sample set;

the comparison sample extraction unit is used for extracting object features of all images in a comparison object image training set which belong to the same source type and belong to different source types from the reference object image to serve as a comparison feature sample set;

the asymmetric measurement model establishing unit is used for establishing an asymmetric measurement model containing parameters under the assumption that the object characteristics involved in comparison obey respective Gaussian distribution;

and the measurement model parameter solving unit is used for solving the parameters in the asymmetric measurement model by using the samples in the two types of characteristic sample sets.

Optionally, the metric model established by the asymmetric metric model establishing unit includes: and (3) an asymmetric measurement model based on a combined Bayesian face.

Optionally, the metric model parameter solving unit is specifically configured to estimate parameters in the model by using a divergence matrix, or iteratively solve the parameters in the model by using an expectation-maximization algorithm.

In addition, the present application also provides an image source identification method, including:

acquiring object image sets belonging to different source categories, and extracting object features from the object image sets to form a training sample set;

training an object image source classification model by using the object feature samples and the source types thereof in the training sample set;

extracting object features from an object image to be classified;

and identifying the source type of the object image to be classified by using the extracted object characteristics as input and adopting the object image source classification model.

Optionally, the object image source classification model is a multi-class classification model obtained by training with the following algorithm:

a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

Optionally, the object image includes: a face image; the object features include: human face characteristics.

Correspondingly, the present application further provides an image source identification apparatus, including:

the training sample acquisition unit is used for acquiring object image sets belonging to different source categories and extracting object features from the object image sets to form a training sample set;

the classification model training unit is used for training an image source classification model by using the object feature samples and the source types thereof in the training sample set;

the object image classification device comprises a to-be-classified feature extraction unit, a classification unit and a classification unit, wherein the to-be-classified feature extraction unit is used for extracting object features from an object image to be classified;

and the source type identification unit is used for identifying the source type of the object image to be classified by adopting the object image source classification model by taking the object features extracted by the feature extraction unit to be classified as input.

Optionally, the object image source classification model includes: a multiclass classification model;

the classification model training unit is specifically used for training the object image source classification model by utilizing a Softmax algorithm, a multi-class SVM algorithm or a random forest algorithm.

Compared with the prior art, the method has the following advantages:

according to the image recognition method, firstly, an object image to be recognized is obtained, object features of the object image to be recognized are extracted, then a similarity measurement model corresponding to the source type of the object image to be recognized is selected from a pre-trained measurement model set, and the similarity between the object features and the object features of a registered image is calculated and used as a basis for outputting an object recognition result. The method is adopted to carry out image recognition, and a pre-trained similarity measurement model corresponding to the source type of the object image to be recognized is selected instead of a single similarity measurement model, so that the problem of asymmetric object image recognition can be effectively solved, and the method has better robustness and higher accuracy in recognition of the object image to be recognized with variable sources.

According to the measurement learning method, under the assumption that the face features participating in comparison obey respective Gaussian distribution, an asymmetric measurement model containing parameters is established, and the parameters in the asymmetric measurement model are solved by using object image feature sample sets from different sources, so that the asymmetric measurement model is constructed. The method modifies the assumption in the traditional image recognition technology, namely: the two object samples x and y participating in comparison can respectively obey respective Gaussian distribution without sharing parameters, and a similarity measurement model for identifying the asymmetric object is learned from sample sets belonging to different source categories on the basis, so that a basis is provided for high-performance object identification adapting to various image sources.

According to the image source identification method, firstly, object features are extracted from object images respectively belonging to different source types to form a training sample set, an object image source classification model is trained by using object feature samples and source types in the training sample set, and then the object features extracted from the object images to be classified are used as input, and the object image source classification model is used for identifying the source types of the object images to be classified. The method can effectively identify the source type of the object image, thereby providing a basis for selecting a correct similarity measurement model in the object identification process and ensuring the correctness of the identification result.

Drawings

FIG. 1 is a flow chart of an embodiment of an image recognition method provided herein;

FIG. 2 is a diagram illustrating a training process for a set of metric models provided by an embodiment of the present application;

FIG. 3 is a flowchart of a process for training an asymmetric metric model according to an embodiment of the present application;

FIG. 4 is a schematic diagram of face recognition using a set of metric models according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an embodiment of an image recognition apparatus provided herein;

FIG. 6 is a flow chart of an embodiment of a metric learning method provided herein;

FIG. 7 is a schematic diagram of an embodiment of a metric learning apparatus provided herein;

FIG. 8 is a flow chart of an embodiment of an image source identification method provided by the present application;

fig. 9 is a schematic diagram of an embodiment of an image source identification device provided in the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of implementation in many different ways than those herein set forth and of similar import by those skilled in the art without departing from the spirit and scope of this application, and it is therefore not limited to the specific implementations disclosed below.

In the present application, an image recognition method and apparatus, a metric learning method and apparatus, and an image source recognition method and apparatus are provided, respectively, and are described in detail in the following embodiments one by one.

Although the technical solution of the present application is provided in the context of face recognition, the application field of the technical solution of the present application is not limited to face recognition, and the technical solution provided by the present application can be also applied to recognition applications for other object images.

The existing image recognition technology does not generally consider the source of an object image and adopts a single similarity measurement model for recognition, and the technical scheme of the application provides a new idea of image recognition aiming at the phenomena of complex source and uneven quality of the object image to be recognized: the similarity measurement models corresponding to different source types are trained in advance, and the similarity measurement models corresponding to the source types of the object images to be recognized are selected for recognition during specific application, so that the recognition problem of the asymmetric object images can be solved, and the recognition of the object images belonging to different source types has better robustness and higher accuracy.

The object image is generally an image in which the main display content (for example, a foreground image as an image subject) is an object such as a human face or various articles. The object images from different sources generally refer to images in which the object features follow different data distributions due to different acquisition modes or different acquisition devices, and the different sources may include: video screenshots, scanned images, copied images, and the like.

In view of the popularity of face image recognition applications, the embodiments of the present application are described with face image recognition as an important point.

Please refer to fig. 1, which is a flowchart illustrating an embodiment of an image recognition method according to the present application. The method comprises the following steps:

101, training similarity measurement models corresponding to different source categories to form a measurement model set.

For the face image in this embodiment, various source categories include, but are not limited to: identification photographs, life photographs, video screenshots, scanned images, copied images, monitored pictures, or the like.

Before the technical scheme is adopted for face recognition, similarity measurement models corresponding to different source categories can be trained, all trained similarity measurement models form a measurement model set together, and each member in the set, namely each similarity measurement model corresponds to different source categories of a face image.

Given two face feature samples (called face samples for short) x and y belonging to different source classes, a similarity metric model is used for evaluating the similarity between the two, and in particular, the similarity metric model can be generally represented by a metric function f (x, y, P), where P is a parameter of the model, the purpose of training is to solve a parameter P of the metric model based on a given training set, and once the parameter P is determined, the model training is completed.

Aiming at various source types of the face images, the training process can be repeated for multiple times, so that a plurality of measurement functions are obtained, and each measurement function is suitable for the face images of different source types. When a metric model for a particular source class is trained, the training set consists of three parts: the system comprises a reference facial image training set X serving as a training reference and belonging to a preset source type, a comparison facial image training set Y corresponding to the specific source type, and an identity label Z for identifying which images come from the same person and which images come from different persons. Given a set of training sets (X, Y, Z), a metric function f (X, Y, P) for the (X, Y) space is obtained by training. Fixed trainingTraining set X, by replacing training set Y belonging to different source categories_kThen a plurality of metric functions f can be trained_k(x, Y, P), K is 1 … K, where K is the number of training sets Y and represents the number of categories from which the images originate. Please refer to fig. 2, which is a schematic diagram of a training process of a metric model set.

The above outline of the whole training process, and the following detailed description of the specific steps of training the similarity metric model corresponding to a specific source category, includes: extracting features, establishing a model, solving model parameters and the like. In specific implementation, different algorithms may be used to establish the similarity metric model, and for convenience of understanding, in this embodiment, the similarity metric model is established based on a combined bayesian face which is currently and widely used, and the established model is referred to as an asymmetric metric model. The process of training the asymmetric metrology model is further described below in conjunction with fig. 3, where the training process includes:

step 101-1, extracting the face features of all images in a reference face image training set belonging to a preset source category to serve as a reference feature sample set.

In an implementation, the facial images in the reference facial image training set X as a training reference are usually collected in a controllable environment, and the preset source categories may be: identification photographs, or other source categories for which image quality is often better. After a reference facial image training set is acquired, facial features of all images can be extracted as samples, namely, the samples are called facial samples, and all samples jointly form a reference feature sample set. Please refer to the following text description in step 103 for how to extract the facial features.

And 101-2, extracting the face features of all the images in the comparison face image training set belonging to the specific source category to serve as a comparison feature sample set.

The specific source category may be different from the source category of the training set X of reference face images, for example: x is a certificate photo collected in a controllable environment, and the face image in the comparison face image training set Y can be a life photo collected in an uncontrollable environment. After the comparison face image training set is collected, the face features of all the images can be extracted to serve as samples, and all the samples jointly form a comparison feature sample set. Please refer to the following text description in step 103 for how to extract the facial features.

And step 101-3, under the assumption that the face features involved in comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters.

The embodiment is improved on the basis of the traditional combined Bayesian face, and an asymmetric measurement model is established. For ease of understanding, a Bayesian face and a combined Bayesian face are first briefly described.

The Bayesian face is usually a short for a classical Bayesian face recognition method, the method uses the difference of the characteristics of two human face images as a mode vector, if the two images belong to the same person, the mode is called an intra-class mode, otherwise, the mode is called an inter-class mode, and therefore the multi-classification problem of the human face recognition is converted into a two-classification problem. For any two face samples x and y, if the log-likelihood ratio obtained based on the intra-class/inter-class mode is greater than a preset threshold, the same person can be determined.

The combined Bayesian face is based on Bayesian face, a two-dimensional model is established aiming at the combined probability distribution of x and y, and each face sample is represented as the sum of two independent latent variables: and (4) changing different faces + changing the same face, and then training by using a large number of samples to obtain a similarity measurement model based on the log-likelihood ratio. Although the two bayesian face techniques are proposed for recognizing a face image, the two bayesian face techniques can be applied to recognition of other object images.

The recognition accuracy of the combined Bayes face is improved compared with that of a classic Bayes face, but the basic assumption of the combined Bayes face is that: the face samples x and y participating in the comparison obey the same gaussian distribution, and in specific applications, the image sources in the registered set are usually controllable, and the sources of the face images to be recognized are more complex and have different qualities, that is: the face samples participating in comparison may not meet the requirement of obeying the same gaussian distribution, so that the combined bayesian face technology cannot process the situation well, and the recognition accuracy is low.

In order to solve the above problems, the inventors of the present application propose an asymmetric metric model and a metric learning method using a face image training set of different source categories to train on the basis of modifying the assumption of a combined bayesian face. The asymmetric measurement model is called because the face images corresponding to two face samples compared by the model can belong to different source types, and because the data distribution difference caused by different source types is considered during modeling, a more accurate face recognition result can be obtained according to the similarity estimated by the model.

The asymmetric metric model is based on the following assumptions: the two face samples x and y participating in the comparison can respectively follow respective gaussian distributions without sharing parameters. Assume that a sample X in a reference feature sample set X can be represented by the sum of two independent random variables: x is mu_x+_xIn which μ_xIndicating the randomness imparted by the identity tag of the sample,_xrepresenting randomness due to other factors such as: attitude, expression, illumination, etc., assuming μ_xAnd_xobedience mean 0, covariance matrix S_xxAnd T_xxA gaussian distribution of (a).

Similarly, the sample Y in the face image training set Y can also be represented by the sum of two independent random variables: mu is_y+_yIn which μ_yIndicating the randomness imparted by the identity tag of the sample,_yindicating randomness due to other factors. Suppose μ_yAnd_yobedience mean 0, covariance matrix S_yyAnd T_yyA gaussian distribution of (a).

Since x and y both follow a gaussian distribution, their joint distribution also follows a gaussian distribution. X and Y are connected in space, where the samples are denoted as { X, Y }, the mean of the random variable is still 0, and the variance is analyzed in two cases.

1) For (within class) samples of the same person.

The covariance matrix is:

wherein S is_xyAnd S_yxIs the cross-covariance matrix between X and Y.

The inverse matrix is of the form:

this makes it possible to obtain:

E＝(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1

G＝-(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1S_xy(S_yy+T_yy)^-1

F＝(S_yy+T_yy-S_yx(S_xx+T_xx)^-1S_xy)^-1

2) for (inter-class) samples of different persons.

The covariance matrix is:

the inverse matrix is of the form:

on the basis of the derivation process, for any two samples x and y, the similarity of the samples x and y is evaluated by using the intra-class/inter-class log-likelihood ratio, and the greater the value, the greater the possibility that x and y are the same person is, so that an asymmetric metric model is established as follows:

order to

A＝(S_xx+T_xx)^-1-E

B＝(S_yy+T_yy)^-1-F

Then, the asymmetric metric model can be simplified to the following representation:

- - -equation 1

And step 101-4, solving parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label for identifying whether the samples belong to the same person, and finishing the training of the model.

The main task of training the asymmetric metric model is to solve A, B and G parameters in the model expression shown in formula 1, and as can be seen from the derivation process of step 101-3, these three parameters can be represented by S_xx、T_xx、S_yy、T_yy、S_xyAnd S_yxThe asymmetric metric model is obtained through specific operation, and therefore the core of training the asymmetric metric model is to solve the above covariance matrixes and cross covariance matrix. The embodiment utilizes a reference feature sample set XAnd comparing a large number of face samples in the feature sample set Y, and solving each parameter by adopting a divergence matrix estimation mode, which is explained in detail below.

According to the reference characteristic sample set X and the identity label information (whether the face samples with different identifications belong to the same person or not), using the inter-class divergence matrix pair S_xxFor approximate estimation, using the intra-class divergence matrix pair T_xxFor approximate estimation, the calculation formula is as follows:

where C is the number of categories (face samples belonging to the same person are of the same category),for a set of samples of the i-th class,number of samples, m, representing class i^xIs the average of the whole number of samples,is the mean of the i-th class samples.

Similarly, according to the comparison characteristic sample set Y and the identity label information, the inter-class divergence matrix pair S is used_yyFor approximate estimation, using the intra-class divergence matrix pair T_yyFor approximate estimation, the calculation formula is as follows:

wherein C is the number of the categories,for a set of samples of the i-th class,number of samples, m, representing class i^yIs the average of the whole number of samples,is the mean of the i-th class samples.

Similarly, the cross-covariance matrix between X and Y is estimated using the following calculation:

solving to obtain S by the method of estimating divergence matrix_xx、T_xx、S_yy、T_yy、S_xyAnd S_yxThen, according to the derivation process in step 101-3, the values of the parameters A, B and G can be further calculated, and the values of these parameters are substituted into formula 1 to obtain the trained asymmetric metrology model.

Thus, the specific steps of training the asymmetric metric model corresponding to a particular source class are described through steps 101-1 through 101-4 above. In specific implementation, the above steps may be respectively adopted for training K source types of the face image, so as to obtain K similarity measurement models respectively corresponding to different source types.

It should be noted that, in this embodiment, on the basis of using a large number of face samples, each parameter in the asymmetric metric model is solved in a manner of estimating a divergence matrix, in other embodiments, the parameter in the model may also be solved in a multi-round iterative manner by using an expectation maximization algorithm adopted by a conventional combined bayesian face, and the technical solution of the present application may also be implemented.

In addition, the present embodiment establishes similarity metric models corresponding to different source categories by modifying the assumptions thereof on the basis of the joint bayesian face, and in other embodiments, other methods or techniques may also be used to establish the similarity metric models, for example: the similarity measurement model is established by using a typical correlation Analysis (CCA) technique, an Asymmetric Depth Measurement Learning (ADML) method, or a method based on a multi-modal restricted Boltzmann machine (multimodal restricted Boltzmann Machines). No matter what algorithm or technology is adopted, as long as the corresponding similarity measurement models can be respectively established and trained for the face images with different sources, the method does not deviate from the core of the application, and is within the protection scope of the application.

And 102, acquiring a face image to be recognized.

The facial image to be recognized usually refers to a facial image whose identity is to be determined, and is generally acquired in an uncontrollable environment, and the source types of the facial image are more, and the method may include: live photographs, copying posters, copying televisions, monitoring pictures, scanning images, and the like.

In a specific implementation, the facial image to be recognized may be obtained in various manners, for example, shooting with a camera or a mobile terminal device having a camera, downloading from a resource database of the internet, scanning with a scanner, or receiving a facial image to be recognized uploaded by a client (e.g., a mobile terminal device or a desktop computer) in a wired or wireless manner.

And 103, extracting the face features of the face image to be recognized.

Because the face part usually occupies the main space of the face image to be recognized, the face features can be directly extracted from the face image to be recognized, and in order to improve the recognition accuracy, the specific position where the face is located can be detected from the face image background, for example: and determining the specific position of the face in the image by adopting a skin color-based detection method, a shape-based detection method, a statistical theory-based detection method and the like, and then extracting the face features from the face image corresponding to the specific position.

The process of extracting the features is a process of converting the face image into a vector, the vector is called as the face feature, and the face feature has strong discrimination on the face images from different people and has robustness on external interference factors. In specific implementation, various feature extraction methods can be adopted, such as: local Binary Pattern (LBP), Gabor wavelet transform algorithm, and deep convolutional network, etc., wherein, from the viewpoint of recognition accuracy and execution performance, extracting the face features by using the deep convolutional network is a preferred implementation manner provided by this embodiment.

And step 104, determining the source type of the facial image to be recognized by using a pre-trained facial image source classification model.

In specific implementation, the source type of the facial image to be recognized may be determined according to the manner of acquiring the image to be recognized in step 103, for example: the method comprises the steps that a camera is used for shooting a human face image in ordinary life, and the source type of the human face image is a life photo; if the scanner is adopted to scan the acquired face image, the source type is the scanned image. In addition, for the facial image to be recognized acquired from the client or the network, if the image has the source information labeled in advance, the source type of the facial image can be determined according to the information.

For the face image to be recognized which cannot be obtained in the above manner or the similar manner, the method in this step may be adopted: and determining the source type of the facial image to be recognized by utilizing a facial image source classification model.

The facial image source classification model is a multi-class classification model (also referred to as a multi-class classifier), and in specific implementation, the facial image source classification model may be trained in advance before the step is executed, for example, the classification model is trained by using a Softmax regression algorithm in this embodiment, and the training process is further described below.

Firstly, acquiring a face image set belonging to K different source categories, extracting face features from each face image to form a training sample set, wherein each sample in the training sample set consists of two parts: the face features and the source category labels corresponding to the face features may specifically be represented as follows: { y_i,s_iDenotes (i ═ 1 … N), where y_iIs a characteristic of the human face, s_iIs the source class label, and N is the number of samples.

By adopting a Softmax regression method, for a given face feature, the probability of belonging to the kth class is in the following form:

where θ is a parameter of the model, it can be solved by minimizing the following objective function:

wherein, 1{ } is an index function, when the expression in the brackets is established, the value is 1, otherwise, the value is 0. In practice, for a given training set y_i,s_iAn iterative optimization algorithm (e.g., gradient descent) can be used to minimize the objective function (i-1 … N)And J (theta) is counted, a parameter theta is obtained through solving, and the face image source classification model is trained.

In this step, the facial features of the facial image to be recognized may be used as input, and a trained facial image source classification model is used to calculate a probability P (s ═ k | y) that the facial features belong to each source category, where the source category corresponding to the maximum value is the source category to which the facial image to be recognized belongs.

In this embodiment, a Softmax algorithm is used to implement the face image source classification model, and in other embodiments, other manners different from the above algorithm may also be used, for example, a multi-class SVM algorithm, a random forest algorithm, or the like may also be used.

And 105, selecting a similarity measurement model corresponding to the source type of the face image to be recognized from a pre-trained measurement model set, and calculating the similarity between the face features and the face features of the registered images to serve as a basis for outputting a face recognition result.

The registered image is generally a facial image in a registered set of facial images for querying in a particular application. The images in the face image registration set are usually collected in a controllable environment, the sources of the images are usually single, and the quality of the images is usually better, for example: second generation certificate photos, registration photos and the like, and the scale of the second generation certificate photos is large and can reach tens of thousands to tens of millions. In order to further improve the recognition accuracy of the present technical solution, the facial image registration set and the reference facial image training set used in the training of the similarity metric model in step 101 may use images of the same source type, for example: the certificate photo is adopted.

In specific implementation, after the images used for forming the face image registration set are collected, the face features of each face image can be extracted, the face images, the face features, the corresponding identity labels and the corresponding identity information are stored in a registration image database for query, and meanwhile, the corresponding relation among the various information is established. The identity information generally refers to information that can identify an identity of a person corresponding to the face image, for example: name, identity ID, etc.

Since the set of metric models for face recognition has been trained in advance in step 101, in a specific example of this embodiment, the set of metric models trained in advance includes K similarity metric models, each similarity metric model corresponds to a different source category, and the form of each similarity metric model is f_k(x, y, P), K1.. K, where the parameter P has been solved in step 101.

Selecting a corresponding similarity measurement model from the measurement model set according to the source type of the face image to be recognized, for example, the source type of the face image to be recognized is a scanned image, then selecting a similarity measurement model trained in advance for the source type of the scanned image, calculating the similarity between the face features of the face image to be recognized and the face features of the registered image by using the selected model, and finally outputting a face recognition result according to the similarity. Please refer to fig. 4, which is a schematic diagram of the processing procedure in the specific example.

In specific implementation, for different application requirements of face recognition, there are two different situations when the similarity between the face features and the registered image face features is calculated in this step, which will be described below.

And (I) face verification.

The face verification generally refers to determining whether the identity of a face image is a specific person. In this application scenario, identity information of the specific person, such as a digital identifier (identity ID) representing the identity of the specific person, may be generally known in advance, a registered image database may be queried according to the identity information, that is, a face feature of a registered image corresponding to the identity may be obtained, then a similarity between the face feature of the face image to be recognized and a face feature of the registered image obtained from the database may be calculated, and if the similarity is greater than a preset threshold, it may be determined that the face image to be recognized and the registered image belong to the same person, that is: and the identity of the face image to be recognized is indeed the specific person, and the judgment is output as a face recognition result.

And (II) identifying the human face.

The face identification generally refers to identifying the identity of a face image to be identified, that is, determining which person the face image to be identified is. In this application scenario, the similarity between the facial features of the facial image to be recognized and the facial features of the registered image in the designated range may be calculated, for example, the similarity may be compared with all the facial features of the registered image in a pre-established registered image database one by one, or a part of the facial features of the registered image in the registered image database may be selected according to a preset policy for comparison, and the corresponding similarity is calculated. If the maximum value in the calculated similarity is greater than a preset threshold, it may be determined that the face image to be recognized is successfully matched in the registered images within the specified range, that is, it may be determined that the face image to be recognized is in the registered image set within the specified range, and the related identity information of the registered image corresponding to the maximum value is output as a face recognition result, for example, the identity information, such as the identity ID or the name, of the registered image corresponding to the maximum value may be output.

Now, through the above steps 101 to 105, a specific implementation of the face recognition method provided in this embodiment is described. It should be noted that not all of the above steps are necessary to practice the present method. Step 101 is a training process of a metric model set, and in general, each similarity metric model in the metric model set can be repeatedly used once being trained, without having to train again for an acquired face image to be recognized each time; similarly, step 104 is not necessary, and if the source type of the image to be recognized can be known through the obtaining manner of the image to be recognized, or the image to be recognized carries the source type label, step 104 may not be executed.

The above embodiment takes face recognition as an example, and describes in detail a specific implementation process of the image recognition method provided by the present application. In practical applications, the image recognition method provided by the present application can also be applied to recognition of other object images (for example, images including various articles), and the case image recognition will be briefly described below as an example.

The similarity measurement models corresponding to different image source categories can be trained in advance according to a reference case image training set and comparison case image training sets corresponding to different source categories, after a case image to be identified is obtained, case features in the case image to be identified are extracted, then the similarity measurement model corresponding to the source category of the case image to be identified is selected, the similarity between the case features and the case features of the registered images is calculated, and the identification result of the case image to be identified is output according to the similarity, for example: whether the to-be-identified luggage image and the registered image corresponding to the specific identity belong to the same luggage or not or whether the to-be-identified luggage image and the registered image correspond to the specific identity are related to identity information. Identity information for items such as bags and the like may typically include one or a combination of the following: manufacturer, brand information, model information, etc.

In summary, according to the image recognition method provided by the application, when the object image is recognized, a single similarity measurement model is not adopted, but a pre-trained similarity measurement model corresponding to the source type of the object image to be recognized is selected, so that the recognition problem of the asymmetric object image can be effectively solved, and the recognition of the object image to be recognized with variable sources has better robustness and higher accuracy.

In the above embodiment, an image recognition method is provided, and correspondingly, the application also provides an image recognition device. Please refer to fig. 5, which is a schematic diagram of an embodiment of an image recognition apparatus of the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.

An image recognition apparatus of the present embodiment includes: a metric model training unit 501, configured to respectively train to obtain similarity metric models corresponding to different source categories in the metric model set by using a reference object image training set belonging to a preset source category and a comparison object image training set corresponding to different source categories; an image acquisition unit 502 for acquiring an image of an object to be recognized; a feature extraction unit 503, configured to extract an object feature of the object image to be recognized; a source type determining unit 504, configured to determine a source type of the object image to be recognized by using the object feature as an input and using a pre-trained object image source classification model; a similarity calculation unit 505, configured to select a similarity measurement model corresponding to the source type of the object image to be recognized from a pre-trained measurement model set, and calculate a similarity between the object feature and an object feature of a registered image, as a basis for outputting an object recognition result;

wherein the similarity calculation unit includes:

the measurement model selection subunit is used for selecting a similarity measurement model corresponding to the source type of the object image to be identified from a pre-trained measurement model set;

and the calculation execution subunit is used for calculating the similarity between the object feature and the object feature of the registration image by using the similarity metric model selected by the metric model selection subunit, and the similarity is used as a basis for outputting an object recognition result.

Optionally, the apparatus includes:

the source classification model training unit is used for training and training the object image source classification model by adopting the following algorithm before triggering the source type determining unit to work: a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

Optionally, the metric model training unit is specifically configured to train asymmetric metric models corresponding to different source categories, where the asymmetric metric models are metric models established based on a joint bayesian face under the assumption that object features involved in comparison obey respective gaussian distributions;

the metric model training unit trains an asymmetric metric model corresponding to a particular source class by:

the reference sample extraction subunit is used for extracting object features of images in a reference object image training set belonging to a preset source category to serve as a reference feature sample set;

a comparison sample extraction subunit, configured to extract an object feature of each image in the comparison object image training set belonging to the specific source category, as a comparison feature sample set;

the measurement model establishing subunit is used for establishing an asymmetric measurement model containing parameters under the assumption that the object characteristics involved in comparison obey respective Gaussian distribution;

and the model parameter solving subunit is used for solving the parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label indicating whether the samples belong to the same object, so as to complete the training of the model.

Optionally, the model parameter solving subunit is specifically configured to estimate parameters in the model by using a divergence matrix, or iteratively solve the parameters in the model by using an expectation-maximization algorithm.

Optionally, the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registration image corresponding to the specific identity;

the device further comprises:

the first threshold comparison unit is used for judging whether the similarity is greater than a preset threshold;

and the first identification result output unit is used for judging that the image of the object to be identified and the registered image corresponding to the specific identity belong to the same object and outputting the judgment as an object identification result when the output of the first threshold comparison unit is yes.

Optionally, the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registration image in a specified range;

the device further comprises:

the second threshold comparison unit is used for judging whether the maximum value in the calculated similarity is larger than a preset threshold or not;

and the second identification result output unit is used for judging that the object image to be identified is successfully matched in the registered images in the specified range when the output of the second threshold comparison unit is yes, and outputting the related identity information of the registered image corresponding to the maximum value as an object identification result.

Optionally, the feature extraction unit is specifically configured to extract the object feature by using a local binary pattern algorithm, extract the object feature by using a Gabor wavelet transform algorithm, or extract the object feature by using a deep convolution network.

In addition, the application also provides a metric learning method. Please refer to fig. 6, which is a flowchart illustrating an embodiment of a metric learning method according to the present application, wherein the same steps as those of the embodiment of the image recognition method are not repeated, and the following description focuses on differences. The metric learning method provided by the application comprises the following steps:

step 601, extracting object features of images in a reference object image training set belonging to the same source type to serve as a reference feature sample set.

Step 602, extracting object features of each image in the comparison object image training set which belong to the same source but belong to different source categories from the reference object image, and using the object features as a comparison feature sample set.

Step 603, under the assumption that the object features involved in the comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters.

The asymmetric metrology model comprises: an asymmetric metric model based on a combined Bayesian face; the asymmetric metric model is as follows:

and step 604, solving parameters in the asymmetric human face similarity measurement model by using the samples in the two characteristic sample sets.

In this step, each parameter in the model can be solved by using the samples in the two types of feature sample sets and adopting an algorithm or a mode corresponding to the established model. For example, for the asymmetric metric model based on the joint bayesian face, the parameters in the model may be estimated by using a divergence matrix according to the samples in the two feature sample sets and the identity tag information identifying whether the samples belong to the same object, or the parameters in the model may be solved iteratively by using an expectation-maximization algorithm.

The metric learning method provided in this embodiment may be used to learn a similarity metric model of an asymmetric face image, and in this application scenario, the reference object image and the comparison object image include: a face image; the object features include: human face characteristics. Of course, in practical applications, the metric learning method provided in this embodiment may also be used to learn a similarity metric model of other asymmetric object images.

The metric learning method provided by the application modifies the assumption in the traditional image recognition technology, namely: the two object samples x and y participating in comparison can respectively obey respective Gaussian distribution without sharing parameters, and a similarity measurement model for identifying the asymmetric object is learned from sample sets belonging to different source categories on the basis, so that a basis is provided for high-performance object identification adapting to various image sources.

In the foregoing embodiments, a metric learning method is provided, and correspondingly, the present application also provides a metric learning apparatus. Please refer to fig. 7, which is a schematic diagram of an embodiment of a metric learning apparatus according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.

A metric learning apparatus of the present embodiment includes: a reference sample extraction unit 701, configured to extract a face feature of each image in a reference object image training set belonging to the same source category, as a reference feature sample set; a comparison sample extraction unit 702, configured to extract object features of each image in a comparison object image training set that belong to the same source category but belong to different source categories from the reference object image, as a comparison feature sample set; an asymmetric metric model establishing unit 703, configured to establish an asymmetric metric model including parameters on the assumption that the object features involved in the comparison obey respective gaussian distributions; and a metric model parameter solving unit 704, configured to solve the parameter in the asymmetric metric model by using the samples in the two types of feature sample sets.

Optionally, the metric model established by the asymmetric metric model establishing unit includes: and (3) an asymmetric measurement model based on a combined Bayesian face.

Optionally, the metric model parameter solving unit is specifically configured to estimate parameters in the model by using a divergence matrix, or iteratively solve the parameters in the model by using an expectation-maximization algorithm.

In addition, the application also provides an image source identification method. Please refer to fig. 8, which is a flowchart illustrating an embodiment of an image source identification method according to the present application, wherein the same steps as those in the above embodiment are not repeated, and the following description focuses on differences. The image source identification method provided by the application comprises the following steps:

step 801, acquiring object image sets belonging to different source categories, and extracting object features from the object image sets to form a training sample set.

And step 802, training an object image source classification model by using the object feature samples and the source types thereof in the training sample set.

The object image source classification model is generally a multi-class classification model, and in specific implementation, the following algorithm may be adopted to train the object image source classification model: a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

Step 803, extracting object features from the object image to be classified.

And step 804, using the extracted object features as input, and identifying the source type of the object image to be classified by adopting the object image source classification model.

The image source identification method provided by this embodiment may be used to identify the source type of a face image, and in this application scenario, the object image includes: a face image; the object features include: human face features; the pre-trained object image source classification model is a human face image source classification model. Of course, in practical application, the method can also be used for identifying the source type of other object images.

The image source identification method provided by the application can effectively identify the source type of the object image, thereby providing a basis for selecting a correct similarity measurement model in the object image identification process and ensuring the correctness of the identification result.

In the foregoing embodiment, an image source identification method is provided, and accordingly, an image source identification apparatus is also provided. Please refer to fig. 9, which is a schematic diagram of an embodiment of an image source identification apparatus according to the present application. Since the apparatus embodiments are substantially similar to the method embodiments, they are described in a relatively simple manner, and reference may be made to some of the descriptions of the method embodiments for relevant points. The device embodiments described below are merely illustrative.

An image source identification apparatus of this embodiment includes: a training sample collection unit 901, configured to collect object image sets belonging to different source categories, and extract object features from the object image sets to form a training sample set; a classification model training unit 902, configured to train an object image source classification model by using the object feature samples in the training sample set and the source types thereof; a to-be-classified feature extraction unit 903, configured to extract object features from an object image to be classified; a source type identification unit 904, configured to identify a source type of the object image to be classified by using the object image source classification model with the object feature extracted by the feature extraction unit to be classified as input.

Optionally, the object image source classification model includes: a multiclass classification model;

the classification model training unit is specifically used for training the object image source classification model by utilizing a softmax algorithm, a multi-class SVM algorithm or a random forest algorithm.

Although the present application has been described with reference to the preferred embodiments, it is not intended to limit the present application, and those skilled in the art can make variations and modifications without departing from the spirit and scope of the present application, therefore, the scope of the present application should be determined by the claims that follow.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

1. Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transmyedia), such as modulated data signals and carrier waves.

2. As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

Claims (35)

1. An image recognition method, comprising:

acquiring an object image to be identified;

extracting object features of the object image to be recognized;

selecting a similarity measurement model corresponding to the source type of the object image to be recognized from a pre-trained measurement model set, and calculating the similarity between the object characteristics and the object characteristics of the registered image to be used as a basis for outputting an object recognition result;

the measurement model set comprises at least one similarity measurement model, and different similarity measurement models respectively correspond to different source types of the object image.

2. The image recognition method of claim 1, wherein the similarity metric models corresponding to different source classes in the metric model set are obtained by training a reference object image training set belonging to a preset source class and a comparison object image training set corresponding to different source classes, respectively.

3. The image recognition method of claim 2, wherein the object images in the training set of reference object images and the reference image belong to the same source category.

4. The image recognition method according to claim 1, wherein before the step of selecting the similarity metric model corresponding to the source category of the object image to be recognized from the pre-trained metric model set, the following operations are performed:

and determining the source type of the object image to be recognized by taking the object characteristics as input and utilizing a pre-trained object image source classification model.

5. The image recognition method of claim 4, wherein the object image source classification model is a multi-class classification model trained by the following algorithm:

a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

6. The image recognition method of claim 1, wherein the similarity metric model comprises: and establishing an asymmetric measurement model under the assumption that the object features participating in comparison obey respective Gaussian distribution.

7. The image recognition method of claim 6, wherein the asymmetric metric model comprises: an asymmetric metric model based on a combined Bayesian face;

the above asymmetric metric model corresponding to a specific source class is obtained by training through the following steps:

extracting object features of images in a reference object image training set belonging to a preset source category to serve as a reference feature sample set;

extracting object features of all images in the comparison object image training set belonging to the specific source category to serve as a comparison feature sample set;

under the assumption that the object features involved in comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters;

and solving parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label for identifying whether the samples belong to the same object, and finishing the training of the model.

8. The image recognition method of claim 7, wherein the asymmetric metric model corresponding to a particular source category is as follows:

$r(x,y)=\log \frac{P(x,y|H_I)}{P(x,y|H_E)}=x^{T}Ax+y^{T}By-2x^{T}Gy$

A＝(S_xx+T_xx)^-1-E

B＝(S_yy+T_yy)^-1-F

G＝-(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1S_xy(S_yy+T_yy)^-1

E＝(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1

F＝(S_yy+T_yy-S_yx(S_xx+T_xx)^-1S_xy)^-1

wherein, assume that sample X in the reference feature sample set X is μ_x+_x，μ_xAnd_xobedience mean 0, covariance matrix S_xxAnd T_xxThe gaussian distribution of (2) is compared with the samples Y ═ mu in the characteristic sample set Y_y+_y，μ_yAnd_yobedience mean 0, covariance matrix S_yyAnd T_yyGaussian distribution of (S)_xyAnd S_yxIs a cross-covariance matrix between X and Y; r (x, y) is the similarity calculated based on the intra-class/inter-class log-likelihood ratio;

the solving for the parameters in the asymmetric metrology model comprises: solving for S_xx、T_xx、S_yy、T_yy、S_xyAnd S_yx。

9. The image recognition method of claim 7, wherein the solving the parameters in the asymmetric metric model comprises:

estimating parameters in the model using a divergence matrix; or,

and iteratively solving parameters in the model by adopting an expectation-maximization algorithm.

10. The image recognition method according to claim 1, wherein the calculating of the similarity between the object feature and the object feature of the registered image includes:

calculating the similarity between the object characteristics and the registered image object characteristics corresponding to the specific identity;

after the step of calculating the similarity, the following operations are performed:

judging whether the similarity is larger than a preset threshold value or not;

if yes, the image of the object to be recognized and the registered image corresponding to the specific identity belong to the same object, and the judgment is output as an object recognition result.

11. The image recognition method according to claim 1, wherein the calculating of the similarity between the object feature and the object feature of the registered image includes:

calculating the similarity between the object characteristics and the registered image object characteristics in the specified range;

after the step of calculating the similarity, the following operations are performed:

judging whether the maximum value in the calculated similarity is larger than a preset threshold value or not;

if yes, the object image to be recognized is judged to be successfully matched in the registered images in the specified range, and the related identity information of the registered image corresponding to the maximum value is output as an object recognition result.

12. The image recognition method according to any one of claims 1 to 11, wherein the extracting the object feature of the object image to be recognized comprises:

extracting the object features by adopting a local binary pattern algorithm; or,

extracting the object features by adopting a Gabor wavelet transform algorithm; or,

and extracting the object features by adopting a deep convolutional network.

13. The image recognition method according to any one of claims 1 to 11, wherein the object image to be recognized includes: a face image to be recognized; the object features include: human face characteristics.

14. The image recognition method of claim 13, wherein the source categories include:

identification photographs, life photographs, video screenshots, scanned images, copied images, or monitored pictures.

15. An image recognition apparatus, comprising:

the image acquisition unit is used for acquiring an object image to be identified;

the characteristic extraction unit is used for extracting object characteristics of the object image to be identified;

the similarity calculation unit is used for selecting a similarity measurement model corresponding to the source type of the object image to be recognized from a pre-trained measurement model set, and calculating the similarity between the object characteristics and the object characteristics of the registered image to be used as a basis for outputting an object recognition result;

wherein the similarity calculation unit includes:

the measurement model selection subunit is used for selecting a similarity measurement model corresponding to the source type of the object image to be identified from a pre-trained measurement model set;

and the calculation execution subunit is used for calculating the similarity between the object feature and the object feature of the registration image by using the similarity metric model selected by the metric model selection subunit, and the similarity is used as a basis for outputting an object recognition result.

16. The image recognition device according to claim 15, comprising:

and the measurement model training unit is used for respectively training to obtain each similarity measurement model corresponding to different source categories in the measurement model set by utilizing a reference object image training set belonging to a preset source category and a comparison object image training set corresponding to different source categories.

17. The image recognition device according to claim 15, comprising:

and the source type determining unit is used for determining the source type of the object image to be recognized by taking the object characteristics as input and utilizing a pre-trained object image source classification model before triggering the similarity calculating unit to work.

18. The image recognition device according to claim 17, comprising:

the source classification model training unit is used for training and training the object image source classification model by adopting the following algorithm before triggering the source type determining unit to work: a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

19. The image recognition device according to claim 15, comprising:

a metric model training unit, configured to train each similarity metric model in the metric model set, where the similarity metric model includes: establishing an asymmetric measurement model based on a combined Bayesian face under the assumption that the object features involved in comparison obey respective Gaussian distribution;

the metric model training unit trains the above asymmetric metric model corresponding to a particular source class by the following subunits:

the reference sample extraction subunit is used for extracting object features of images in a reference object image training set belonging to a preset source category to serve as a reference feature sample set;

a comparison sample extraction subunit, configured to extract an object feature of each image in the comparison object image training set belonging to the specific source category, as a comparison feature sample set;

the measurement model establishing subunit is used for establishing an asymmetric measurement model containing parameters under the assumption that the object characteristics involved in comparison obey respective Gaussian distribution;

and the model parameter solving subunit is used for solving the parameters in the asymmetric measurement model according to the samples in the two characteristic sample sets and the identity label indicating whether the samples belong to the same object, so as to complete the training of the model.

20. The image recognition device of claim 19, wherein the model parameter solving subunit is specifically configured to estimate the parameters in the model using a divergence matrix or to iteratively solve the parameters in the model using an expectation-maximization algorithm.

21. The image recognition device according to claim 15, wherein the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registered image corresponding to a specific identity;

the device further comprises:

the first threshold comparison unit is used for judging whether the similarity is greater than a preset threshold;

and the first identification result output unit is used for judging that the image of the object to be identified and the registered image corresponding to the specific identity belong to the same object and outputting the judgment as an object identification result when the output of the first threshold comparison unit is yes.

22. The image recognition device according to claim 15, wherein the calculation execution subunit is specifically configured to calculate a similarity between the object feature and an object feature of a registered image within a specified range;

the device further comprises:

the second threshold comparison unit is used for judging whether the maximum value in the calculated similarity is larger than a preset threshold or not;

and the second identification result output unit is used for judging that the object image to be identified is successfully matched in the registered images in the specified range when the output of the second threshold comparison unit is yes, and outputting the related identity information of the registered image corresponding to the maximum value as an object identification result.

23. The image recognition device according to any one of claims 15 to 22, wherein the feature extraction unit is specifically configured to extract the object feature using a local binary pattern algorithm, extract the object feature using a Gabor wavelet transform algorithm, or extract the object feature using a deep convolutional network.

24. A metric learning method, comprising:

extracting object features of images in a reference object image training set belonging to the same source category to serve as a reference feature sample set;

extracting object features of all images in a comparison object image training set which belong to the same source type and belong to different source types from the reference object image to serve as a comparison feature sample set;

under the assumption that the object features involved in comparison obey respective Gaussian distribution, establishing an asymmetric measurement model containing parameters;

and solving parameters in the asymmetric measurement model by using samples in the two types of characteristic sample sets.

25. The metric learning method of claim 24, wherein the asymmetric metric model comprises: an asymmetric metric model based on a combined Bayesian face;

the asymmetric metric model is as follows:

$r(x,y)=log\frac{P(x,y|H_I)}{P(x,y|H_E)}=x^{T}Ax+y^{T}By-2x^{T}Gy$

A＝(S_xx+T_xx)^-1-E

B＝(S_yy+T_yy)^-1-F

G＝-(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1S_xy(S_yy+T_yy)^-1

E＝(S_xx+T_xx-S_xy(S_yy+T_yy)^-1S_yx)^-1

F＝(S_yy+T_yy-S_yx(S_xx+T_xx)^-1S_xy)^-1

wherein, assume that sample X in the reference feature sample set space X is μ_x+_x，μ_xAnd_xobedience mean 0, covariance matrix S_xxAnd T_xxComparing the samples Y in the feature sample set space Y to mu_y+_y，μ_yAnd_yobedience mean 0, covariance matrix S_yyAnd T_yyGaussian distribution of (S)_xyAnd S_yxIs a cross-covariance matrix between X and Y; r (x, y) is the similarity calculated based on the intra-class/inter-class log-likelihood ratio;

the solving for the parameters in the asymmetric metrology model comprises: solving for S_xx、T_xx、S_yy、T_yy、S_xyAnd S_yx。

26. The metric learning method of claim 25, wherein the solving the parameters in the asymmetric metric model comprises:

estimating parameters in the model using a divergence matrix; or,

and iteratively solving parameters in the model by adopting an expectation-maximization algorithm.

27. The metric learning method of any one of claims 24-26, wherein the reference object image and the comparison object image comprise: a face image; the object features include: human face characteristics.

28. A metric learning apparatus, comprising:

the reference sample extraction unit is used for extracting object features of images in a reference object image training set belonging to the same source category to serve as a reference feature sample set;

the comparison sample extraction unit is used for extracting object features of all images in a comparison object image training set which belong to the same source type and belong to different source types from the reference object image to serve as a comparison feature sample set;

the asymmetric measurement model establishing unit is used for establishing an asymmetric measurement model containing parameters under the assumption that the object characteristics involved in comparison obey respective Gaussian distribution;

and the measurement model parameter solving unit is used for solving the parameters in the asymmetric measurement model by using the samples in the two types of characteristic sample sets.

29. The metric learning apparatus of claim 28, wherein the metric model created by the asymmetric metric model creating unit comprises: and (3) an asymmetric measurement model based on a combined Bayesian face.

30. The metric learning device of claim 29, wherein the metric model parameter solving unit is specifically configured to estimate the parameters in the model using a divergence matrix or to iteratively solve the parameters in the model using an expectation-maximization algorithm.

31. An image source identification method is characterized by comprising the following steps:

acquiring object image sets belonging to different source categories, and extracting object features from the object image sets to form a training sample set;

training an object image source classification model by using the object feature samples and the source types thereof in the training sample set;

extracting object features from an object image to be classified;

and identifying the source type of the object image to be classified by using the extracted object characteristics as input and adopting the object image source classification model.

32. The image source identification method of claim 31, wherein the object image source classification model is a multi-class classification model trained by the following algorithm:

a Softmax algorithm, a multi-class SVM algorithm, or a random forest algorithm.

33. The image source identification method of claim 31 or 32, wherein the object image comprises: a face image; the object features include: human face characteristics.

34. An image source identification device, comprising:

the training sample acquisition unit is used for acquiring object image sets belonging to different source categories and extracting object features from the object image sets to form a training sample set;

the classification model training unit is used for training an image source classification model by using the object feature samples and the source types thereof in the training sample set;

the object image classification device comprises a to-be-classified feature extraction unit, a classification unit and a classification unit, wherein the to-be-classified feature extraction unit is used for extracting object features from an object image to be classified;

and the source type identification unit is used for identifying the source type of the object image to be classified by adopting the object image source classification model by taking the object features extracted by the feature extraction unit to be classified as input.

35. The image source identification device of claim 34, wherein the object image source classification model comprises: a multiclass classification model;

the classification model training unit is specifically used for training the object image source classification model by utilizing a Softmax algorithm, a multi-class SVM algorithm or a random forest algorithm.