JP2012234497A - Object identification device, object identification method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance accuracy of identifying an object included in an image.SOLUTION: An object identification device comprises: objects variation amount detection means for detecting variation amounts between objects which are an input object included in an input image and registered objects included in registered images; partial feature extraction means for extracting at least one or more corresponding partial feature amounts between the input object and the registered objects; partial feature amount composition unit for determining a composition method for combining the partial feature amounts extracted by the partial feature extraction means according to magnitude of each variation amount between the objects to generate composition partial feature amounts by compositing the partial feature amounts by the respective determined composition methods; and input object identification means for identifying which registration object corresponds to the input object by using the composition partial feature amounts.

Description

  The present invention relates to an object identification device, an object identification method, and a program.

Face recognition that identifies an individual's face as an identification technique in pattern recognition, typically a technique that identifies an object that is a subject in image data as being the same as an object that is a subject in another image There is technology. In the following, identification of an object means that a difference between individual objects (for example, a difference between persons as individuals) is determined. On the other hand, the detection of an object means that an object that falls within the same category without distinguishing between individuals is determined (for example, a face is detected without distinguishing between individuals).
As a face identification technique, there is a method as described in Non-Patent Document 1, for example. This is an algorithm that makes it possible to perform face registration and additional learning in real time by replacing an individual identification problem by a face with an identification problem of two classes of feature classes called differential faces.

For example, in the face identification using a well-known support vector machine (SVM), in order to identify the faces of n persons, n faces that are registered and other faces are identified. SVM discriminators are required. When registering a person's face, SVM learning is required. SVM learning requires a large amount of face data of a person to be registered and face data of already registered persons and other persons, and takes a very long calculation time. Was common.
However, according to the method of Non-Patent Document 1, additional learning can be substantially eliminated by replacing the individual identification problem with the following two classes of identification problems. That is,
intra-personal class: Fluctuation feature class such as lighting variation, facial expression and orientation between images of the same person
extra-personal class: Two classes of variation feature classes between images of different persons.

Assuming that the distribution of the two classes is constant regardless of a specific individual, the classifier is configured by reducing the face identification problem of the individual to the classification problem of the two classes. A large number of images are prepared in advance, and a discriminator for discriminating between a variation feature class between the same persons and a variation feature class between different persons is learned. For new registrants, only the face image (or the result of extracting the necessary features) need be retained. When discriminating, the difference feature is extracted from the two images, and the discriminator determines whether the person is the same person or a different person. This eliminates the need for learning such as SVM at the time of personal face registration, and registration processing can be performed in real time.
Moreover, as a conventional technique in face identification, the technique of Patent Document 1 is disclosed.

JP 2009-031991

Baback Moghaddam, Beyond Eigenfaces: Probabilistic Matching for Face Recognition (M.I.T Media Laboratory Perceptual Computing Section Technical Report No.433), ProbabilisticVisual Learning for Object Representation (IEEE Transactions on PatternAnalysis and Machine Intelligence, Vol. 19, No. 7, JULY 1997) Lior Wolf, Tal Hassner, and Yaniv Taigman, Descriptor Based Methods in the Wild.Faces in Real-Life Images Workshop in European Conference on Computer Vision (ECCV), 2008.

In the apparatus and method for identifying a pattern (an object in an image, more specifically, a human face) as described above, fluctuation between a registration pattern and an authentication pattern is a factor that degrades the identification performance. Is mentioned. That is, there are fluctuations in the object (person's face) to be identified, more specifically, lighting conditions, orientation / posture, hiding by other objects, fluctuation due to facial expressions, and the like. Therefore, when the above-described fluctuation increases between the registration pattern and the authentication pattern, the identification performance is greatly deteriorated.
With respect to this problem, in Patent Document 1, a feature vector acquired from a face image is separated into two components, a component representing individual differences and a component representing imaging conditions. By converting the component representing the latter imaging condition in the feature space and adding the converted component to the component representing the individual difference, it is possible to obtain feature vectors corresponding to various imaging conditions from one image. By artificially generating feature vectors of various shooting conditions, it is possible to realize identification robust to fluctuations.
However, this method is based on the premise that the feature vector can be expressed by superimposing the individual difference component and the imaging condition component, and is not applicable to all feature amounts.

In the technique of Non-Patent Document 2, a plurality of feature amounts having different properties such as a Gabor feature and an LBP (Local Binary Pattern) feature are combined to cope with various fluctuations between two patterns.
However, it is limited to a predetermined combination method, and may not work effectively against a large variation in an actual environment.
Furthermore, as another problem, when face authentication is performed, a method of detecting the position of a facial organ in advance and performing comparison based on the position of the organ is generally used. The above two documents are also premised on the fact that the positions of the eyes of the face, the position of the mouth, etc. are preliminarily aligned to some extent between the registration image (registration image) and the authentication image (authentication image). Yes.
However, in an actual environment, facial expression changes and imaging conditions may appear in combination, and it is difficult to detect the position of the facial organ with high accuracy. Therefore, a shift in the position of the facial organ between the registered image and the authentication image is also a major factor that deteriorates the recognition accuracy.

  The present invention has been made in view of such problems, and an object thereof is to further increase the accuracy of identifying an object included in an image.

  Therefore, the object identification device according to the present invention includes an inter-object variation amount detecting means for detecting a variation amount between the input object included in the input image and the registered object included in the registered image, the input object, and the registered object. A combination method of combining the partial feature extraction means for extracting at least one or more corresponding partial feature quantities, and the partial feature quantities extracted by the partial feature extraction means in accordance with the amount of variation between the objects And the partial feature amount combining means for generating the combined partial feature amount by combining the partial feature amounts by the determined combining method, and the registered object to which the input object corresponds using the combined partial feature amount Input object identifying means for identifying the above.

  According to the present invention, the accuracy of identifying an object included in an image can be further increased.

It is a figure which shows an example of a structure of an object identification device. It is a figure which shows an example of the flowchart which concerns on the whole process. It is a figure which shows an example of a structure of an object registration part. It is a figure which shows an example of a structure of a registration object dictionary data generation part. It is a figure which shows an example of the flowchart which concerns on a feature vector extraction conversion process. It is a figure which shows an example of a structure of an input object identification part. It is a figure which shows an example of the flowchart which concerns on the identification process of an input object. It is a figure which shows an example of a structure of the data generation part for input object identification. It is a figure which shows an example of a structure of an input object identification calculating part. It is a figure which shows an example of the flowchart which concerns on an input object identification calculation process. It is a figure which shows an example of a structure of a partial feature synthetic | combination part. It is a figure which shows an example of the flowchart which concerns on a partial feature synthetic | combination process. It is a figure which shows an example of a structure of a synthetic | combination partial feature evaluation part. It is a figure which shows an example of the flowchart which concerns on a synthetic | combination partial feature evaluation process. It is a figure which shows an example of the flowchart which concerns on a synthetic | combination partial feature evaluation value calculation process. It is a figure which shows an example of the flowchart which concerns on the learning process of a partial region. It is a figure which shows an example of a structure of a synthetic | combination partial feature evaluation part. It is a figure which shows an example of the flowchart which concerns on a synthetic | combination partial feature evaluation process. It is a figure which shows an example of the flowchart which concerns on a partial feature quantity measurement process. It is a figure which shows an example of a structure of a synthetic | combination partial feature likelihood calculation part. It is a figure which shows an example of the flowchart which concerns on a synthetic partial feature likelihood calculation process. It is a figure which shows an example of the flowchart which concerns on a synthetic | combination partial feature evaluation value calculation process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The embodiments do not limit the present invention, and all the configurations described in the embodiments are not necessarily essential to the means for solving the problems of the present invention.

<First Embodiment>
In the present embodiment, even when the object to be identified is subject to various changes (in the case of an image, illumination fluctuation, orientation / posture fluctuation, etc.), the identification is performed with high accuracy and high speed. The present invention relates to an object identification device, an object identification method, and a program for the same.
FIG. 1 is a diagram illustrating an example of a configuration of an object identification device 100 according to the present embodiment. The object identification device 100 includes an imaging optical system 1, an imaging unit 2, an imaging control unit 3, an image recording unit 4, an object registration unit 5, an input object identification unit 6, an external output unit 7 that outputs an object identification result, and It has a connection bus 8 for controlling and data connection of each component.
Note that the object registration unit 5 and the input object identification unit 6 may typically be dedicated circuits (such as ASIC) and processors (such as reconfigurable processors, DSPs, and CPUs), respectively. Further, the object registration unit 5 and the input object identification unit 6 may be realized by a program executed in a single dedicated circuit and a general-purpose circuit (PC CPU).

The imaging optical system 1 includes an optical lens having a zoom mechanism. Further, a drive mechanism in the pan / tilt axis direction may be provided. As the image sensor of the imaging unit 2, a CCD or a CMOS image sensor is typically used. The imaging unit 2 outputs a predetermined video signal (for example, a signal obtained by subsampling or block reading) as image data in response to a read control signal from a sensor drive circuit (not shown).
The imaging control unit 3 controls the timing at which actual shooting is performed based on instructions from the photographer (viewing angle adjustment instruction, shutter pressing, etc.) and information from the object registration unit 5 or the input object identification unit 6. To do. The image recording unit 4 is composed of a semiconductor memory or the like, records (holds) the image data transferred from the imaging unit 2, and at a predetermined timing in response to a request from the object registration unit 5, the input object identification unit 6, and the like. Transfer image data.

The object registration unit 5 extracts information of an object to be identified from the image data and records (holds) the information. A more detailed configuration of the object registration unit 5 and the contents of the processing actually performed will be described later. The input object identification unit 6 identifies an object based on the image data and the image data acquired from the object registration unit 5. A more detailed configuration of the input object identification unit 6 and the contents of the processing actually performed will be described later.
The external output unit 7 is typically a monitor such as a CRT or a TFT liquid crystal, and displays image data acquired from the imaging unit 2 and the image recording unit 4 or the object registration unit 5 and the input object identification unit 6. The result output is displayed superimposed on the image data. Further, a configuration may be adopted in which results output from the object registration unit 5 and the input object identification unit 6 are output as electronic data to an external memory or the like. The connection bus 8 is a bus for performing control / data connection between the components of the object identification device 100.

<Overall processing>
FIG. 2 is a diagram illustrating an example of a flowchart relating to the overall processing of the object identification device 100. With reference to FIG. 2, an actual process in which the object identification device 100 identifies an input object will be described. In the following, a case where the object in the input image is a human face will be described, but the present invention is not limited to this case.
First, the object identification device 100 acquires image data from the image recording unit 4 (S0). Subsequently, the object identification device 100 performs a human face detection process on the acquired image data (S1). As a method for detecting a human face from an image, a known technique can be used. For example, a technique proposed in Japanese Patent No. 3078166 and Japanese Patent Laid-Open No. 2002-8032 can be used.

Subsequently, the object identification device 100 performs a process of detecting a human face as a target object from the registered image and the input image, and determines that a human face exists in the image (YES in S2), the input object In this example, an individual identification process is performed (S3). On the other hand, when the object identification device 100 determines that no human face is present in the image (NO in S2), the process ends. The specific contents of the input object identification process will be described later.
Subsequently, the object identification device 100 determines whether there is a face corresponding to the registered person from the result of the identification process of the input object (S4). At this time, if the object identification device 100 determines that the same person as the face detected in S1 is among the registered persons (YES in S4), the object identification apparatus 100 subsequently performs the process of S7. On the other hand, when it is determined that the detected face does not match any registered person (NO in S4), the object identification device 100 determines whether to register the person (S5).

The determination criterion may be set in advance, but for example, the user may determine whether to register on the spot through an external interface or a GUI. At this time, if the object identification device 100 determines to register (YES in S5), the object identification device 100 performs registration processing of an object (such as a person's face) described later (S6). On the other hand, when the object identification device 100 determines not to register (in the case of NO in S5), it continues the process as it is.
Subsequently, when there is a corresponding object in S4, the object identification device 100 determines whether or not the processing has been completed for all detected objects after the object registration processing in S6 or when registration is not performed in S5 (S7). ). At this time, if the object identification device 100 determines that there is an unprocessed object (NO in S7), it performs the process of S3. On the other hand, when the object identification device 100 determines that the processing has been completed for all the detected objects (YES in S7), it outputs a series of input object identification processing results to the external output unit 7 (S8). .
The above is the description of the overall processing of the object identification device 100.

<Object registration part>
Next, object registration processing will be described. FIG. 3 is a diagram illustrating an example of the configuration of the object registration unit 5. As illustrated in FIG. 3, the object registration unit 5 includes a registration object dictionary data generation unit 21, a registration object dictionary data holding unit 22, and a registration object dictionary data selection unit 23.
The registered object dictionary data generation unit 21 generates registered object dictionary data (hereinafter referred to as dictionary data as appropriate) necessary for identifying an individual object from the image data acquired from the image recording unit 4. For example, when discriminating two-class identification problems such as intra-class and extra-class as shown in Non-Patent Document 1, typically, a human face image may be used as dictionary data. Further, the face image data of the person detected by the face detection process may be normalized in size and orientation (in-plane rotation direction) and stored in the registered object dictionary data holding unit 22.

In addition, it is possible to reduce the amount of dictionary data by holding only data necessary for identification, not image data itself. In addition, when the identification calculation is performed by taking the vector correlation of the partial area of the object, it is preferable to cut out only the partial area in advance.
As described above, the registered object dictionary data generation unit 21 appropriately extracts necessary information from the image, performs predetermined conversion described later, and uses it as a feature vector for identifying an object, and holds registered object dictionary data. Stored in the unit 22. Details of specific processing performed by the registered object dictionary data generation unit 21 will be described later.
In response to a request from the input object identifying unit 6, the registered object dictionary data selecting unit 23 reads out necessary registered object dictionary data from the registered object dictionary data holding unit 22, and stores the registered object dictionary data in the input object identifying unit 6. Forward.

<Registered object dictionary data generator>
FIG. 4 is a diagram illustrating an example of the configuration of the registered object dictionary data generation unit 21. As illustrated in FIG. 4, the registered object dictionary data generation unit 21 includes a partial feature extraction unit 30, a feature vector conversion unit 33, and a feature vector conversion data holding unit 34.
The partial feature extraction unit 30 performs a process of extracting a feature vector from an image including a target object. More specifically, the partial feature extraction unit 30 includes a partial region setting unit 31 and a feature vector extraction unit 32.

The partial area setting unit 31 sets a position and a range from which the feature vector extraction unit 32 extracts feature vectors for image data. The position and range of the partial area can be determined in advance using a machine learning method.
For example, a plurality of partial area candidates may be set and selected from the plurality of candidates using AdaBoost. Further, as will be described later, a large number of combinations of the partial areas may be prepared, and the combinations may be selected as one candidate by AdaBoost. A method of determining a partial region and its combination by actually applying AdaBoost will be described later.
Further, the number of partial areas may be determined in advance according to the processing time or the like. In addition, the number of learning samples prepared in advance may be measured and determined so that sufficient identification performance can be obtained.

The feature vector extraction unit 32 extracts feature vectors from registration object data. When the target object is the face of a person in the image, the feature vector extraction unit 32 typically performs a process of extracting data necessary for identification from the image including the face. More specifically, the feature vector extraction unit 32 extracts the luminance value as one of the feature vectors from the partial region set by the partial region setting unit 31 as data necessary for identification.
In addition to the feature vector of the brightness value, the feature vector extraction unit 32 uses a feature amount that is invariant to the position of the partial region of the object or a feature amount that is robust to the position change, for example, a luminance frequency distribution as a feature vector It may be extracted.

It is also conceivable to use, as a feature vector, phase information obtained by Fourier transforming an image including an object as a feature quantity that is invariant to a change in position of a partial region or a robust feature quantity. Further, in addition to features that are robust against position fluctuations as described above, a filter calculation result that extracts edge information of an image may be extracted as a feature vector. For example, a result obtained by applying a typical spatial filter operation such as a Sobel filter or a Gabor filter may be extracted as a feature vector.
Furthermore, the frequency distribution of the edge information by the spatial filter as described above can be taken and used as a feature vector. The frequency distribution is relatively robust against position fluctuations, and has an effect of making feature vectors derived from edge information robust against positional deviation. Details of specific processing performed by the feature vector extraction unit 32 will be described later.

The feature vector conversion unit 33 performs a predetermined conversion on the feature vector extracted by the feature vector extraction unit 32. In the feature vector conversion, for example, dimensional compression by principal component analysis (PCA), dimensional compression by independent component analysis (ICA), or the like is performed. Further, dimension compression by locality preserving projection (LPP), local Fisher discriminant analysis (LFDA), or the like may be performed.
When PCA or the like is used as a feature vector conversion method, there are parameters such as the number of bases (the number of feature vector dimensions to be reduced) and which base to use. Instead of the basis number, a sum of eigenvalues corresponding to the basis vector, that is, a cumulative contribution rate may be used as an index. Also, these parameters can be different for each partial region. What parameters are actually set can be determined by adjusting accuracy and processing time by machine learning in advance.
The feature vector conversion data holding unit 34 holds setting information such as data and conversion parameters necessary when the feature vector conversion unit 33 converts feature vectors. Here, the data and conversion parameters necessary for the feature vector conversion are the setting information such as the base number (dimension reduction number) as described above, the numerical data of the base vector previously obtained by machine learning, and the like. .

FIG. 5 is a diagram illustrating an example of a flowchart relating to processing (feature vector extraction / conversion processing) performed by the feature vector extraction unit 32 and the feature vector conversion unit 33. Hereinafter, a description will be given with reference to FIG.
First, the feature vector extraction unit 32 acquires partial region setting information from the partial region setting unit 31 (S10). Subsequently, the feature vector extraction unit 32 acquires image data of the target object from the image recording unit 4 (S11). Subsequently, the feature vector extraction unit 32 acquires a feature vector related to edge information from the acquired image data based on the setting information of the partial region acquired in S10 (S12). Subsequently, the feature vector extraction unit 32 measures a frequency distribution related to edge information and acquires it as a feature vector (S13).

Here, it is preferable to extract a plurality of types of feature vectors related to edge information and its frequency distribution. As described above, the feature vector extraction unit 32 acquires a feature vector acquired from a result of performing a plurality of filter operations on an image and a frequency vector having the frequency distribution as a feature vector. Typically, the feature vector extraction unit 32 performs LBP (Local Binary Pattern) conversion, differential edge detection, and the like.
Further, the feature vector extraction unit 32 calculates a frequency distribution (histogram) of values at each element after the LBP conversion, and obtains it as a feature vector. Further, a histogram of the luminance gradient image may be taken. This is equivalent to HOG (Histogram of Oriented Gradient).
In this way, it is preferable to acquire a plurality of feature quantities having different properties such as edge information and frequency distribution thereof. As will be described later, it is possible to robustly cope with the case where there is a change between the registered object and the input object by using a combination of feature quantities having greatly different properties for identification.

Subsequently, the feature vector conversion unit 33 performs predetermined conversion on the feature vectors acquired in S12 and S13 according to the setting information acquired from the feature vector conversion data holding unit 34 (S14). As described above, the feature vector conversion unit 33 typically performs dimension reduction by PCA, dimension reduction by ICA, and the like on the feature vector. At this time, a predetermined statistical value, typically an average vector, a maximum value of elements, or the like may be obtained for the acquired feature vector. Further, information on a position cut out from an image may be recorded as a partial feature.
Further, as will be described later, an identifier indicating a correspondence relationship may be recorded in order to compare / evaluate corresponding partial features between a registered object (registered pattern) and an input object (input pattern). . These pieces of information may be output together with the feature vector as an output of the registered object dictionary data generation unit 21.
The above is the description of the processing performed by the feature vector extraction unit 32 and the feature vector conversion unit 33.
The registered object dictionary data generation unit 21 performs the processing as described above, sets a partial area, extracts a feature vector, and then uses the data obtained by converting the feature vector as an output of the registered object dictionary data generation unit 21 for registration. The data is stored in the object dictionary data holding unit 22.

<Input object identification unit>
Next, input object identification processing will be described. FIG. 6 is a diagram illustrating an example of the configuration of the input object identification unit 6. As illustrated in FIG. 6, the input object identification unit 6 includes an input object identification data generation unit 41, a registered object dictionary data acquisition unit 42, and an input object identification calculation unit 43.
The input object identification data generation unit 41 extracts information necessary for identifying the target object from the image data acquired from the image recording unit 4. The registered object dictionary data acquisition unit 42 acquires dictionary data necessary for identifying the input object from the object registration unit 5.
The input object identification calculation unit 43 performs input object identification processing from the identification data acquired from the input object identification data generation unit 41 and the dictionary data acquired from the registered object dictionary data acquisition unit 42. The identification process performed here will be described later.

FIG. 7 is a diagram illustrating an example of a flowchart relating to the input object identification processing performed by the input object identification unit 6.
First, the registered object dictionary data acquisition unit 42 acquires dictionary data of registered objects from the object registration unit 5 (S20). Subsequently, the input object identification data generation unit 41 acquires the image data of the input object from the image recording unit 4 (S21). Subsequently, the input object identification data generation unit 41 performs an input object identification data generation process (S22). The configuration of the input object identification data generation unit 41 will be described later.
Subsequently, the input object identification calculation unit 43 performs an input object identification calculation process (S23). When outputting a match with registered data (dictionary data) in binary (0 or 1), and outputting a normalized output value as a likelihood (a real value between 0 and 1) as an output of the input object identification calculation process You could think so. Furthermore, when there are a plurality of registered objects (registrants), the likelihood may be output for each registered object (registrant), but only the result for the registered object that most closely matches. May be output.
Further, the likelihood for the class to which the registered object belongs may be output instead of the likelihood for the registered object. That is, in the case of a person, the likelihood for the person's ID (name) is output instead of the result of each registered face image. The specific contents of the input object identification calculation process will be described later.
The above is the description of the processing in the input object identification unit 6.

<Input Object Identification Data Generation Unit>
FIG. 8 is a diagram illustrating an example of the configuration of the input object identification data generation unit 41. As illustrated in FIG. 8, the input object identification data generation unit 41 includes a partial feature extraction unit 50, a feature vector conversion unit 53, and a feature vector conversion data holding unit 54. The partial feature extraction unit 50 includes a partial region setting unit 51 and a feature vector extraction unit 52. The configuration of the input object identification data generation unit 41 and the processing performed there are the same as the configuration and processing in the registered object dictionary data generation unit 21, and therefore the details thereof are omitted.

<Input object identification calculation processing>
Next, the input object identification calculation process will be described. Here, as an example, a case will be described in which the identification of the input object is determined based on the similarity between the partial features corresponding to the registered object and the input object.
FIG. 9 is a diagram illustrating an example of the configuration of the input object identification calculation unit 43. The input object identification calculation unit 43 includes an input object identification data acquisition unit 61, a registered object dictionary data acquisition unit 62, an inter-object variation detection unit 63, a partial feature synthesis unit 64, a synthesized partial feature evaluation unit 65, and an evaluation result holding unit. 66 and an evaluation result integration unit 67.

FIG. 10 is a diagram illustrating an example of a flowchart according to the input object identification calculation process. Hereinafter, description will be made with reference to FIG.
First, the input object identification data acquisition unit 61 acquires input object identification data from the input object identification data generation unit 41 (S30). Subsequently, the registration object dictionary data acquisition unit 62 acquires dictionary data of the registration object from the registration object dictionary data acquisition unit 42 (S31). Subsequently, the inter-object variation amount detection unit 63 detects the variation amount between the input object and the registered object based on the data acquired in S30 and S31 (S32).
Here, as a specific variation amount to be detected, when the target object is a human face, the inter-object variation amount detection unit 63 detects an attribute relating to the face and calculates the variation amount. Further, specific detection by the attribute detector includes facial expression detection for detecting facial expressions. The amount of variation can be calculated by performing facial expression detection for each of the registered object and the input object and obtaining the difference between the detected facial expressions.

In addition, a known technique can be used for detecting a human facial expression. For example, the facial expressions may be classified into four categories of emotions, and the detector may be learned so that four class classifiers and the degree of each facial expression can be output. When the registered object and the input object have different facial expressions, the variation amount is increased, and even if the facial expressions are the same, the variation amount between them can be obtained as the variation amount between objects.
The relationship between the output of the expression detector and the actual variation may be formulated by some model, but may be held in advance in a look-up table (LUT). The LUT can be determined in advance by actually identifying the learning data and adjusting so that the subsequent recognition process exhibits the maximum performance.

In addition to facial expressions, the age of the person, the position of the facial feature points, that is, the position of the facial organs such as the eyes, the orientation of the face, how to cast shadows (shadow patterns), lighting conditions, etc. By learning a detector that detects these in advance, the amount of variation can be similarly obtained. In addition, skin quality, makeup, presence / absence of glasses, presence / absence of a mask, and the like can be used as attributes. A known technique can be used as a method for detecting these.
For example, when making a detector using a support vector machine (SVM), learning of SVM is performed by learning SVM based on actual data to be detected, other data, and a tag in which the state is recorded. A detector can be made.

Further, the target of the fluctuation amount can be a camera parameter that determines the shooting condition. The object identification device 100 (for example, an imaging device) has an automatic exposure parameter (AE parameter) and a focus (automatic focus) state (AF parameter) that varies simply by taking a difference in setting values between registration and input. It can be an amount.
In addition, the illumination condition at the time of photographing may be used for measuring the fluctuation amount. In this case, camera parameters can be used for estimation of illumination conditions. For example, it is possible to accurately estimate the illumination condition by acquiring the control AE and AF parameters from the imaging control unit 3. Here, specific examples of camera parameters include exposure conditions, white balance, focus (focusing condition), and object size. For example, it is possible to estimate the color attribute of an object that is not affected by shooting conditions by creating a correspondence table of color components corresponding to the exposure condition and white balance and the skin color component area in advance and holding it as a lookup table. become able to.

Similarly, the size of the object can be used for measuring the amount of variation. The distance to the object that is the subject can be measured by using a distance measuring unit such as AF, and the size of the object can be estimated. More specifically, the size of the object can be estimated according to the following equation (1).
s = (f / d−f) · S (1)
Here, s is the size (number of pixels) of the object on the image, f is the focal length, d is the distance from the device to the object, and S is the actual size of the object. However, it is assumed that (d> f). In this way, it is possible to estimate the size of an object that is not affected by the shooting conditions as an attribute.

The texture information of the object acquired from the image may be used as an attribute. A known technique can be used for texture analysis. Differences in texture information (for example, the amount of edges and the result of classification of luminance patterns) that are analysis results can be defined and detected as fluctuation amounts.
Further, the position of the facial organ may be used for measuring the amount of variation. More specifically, the end points of components such as eyes, mouth, and nose are detected and used. As an algorithm for detecting the end point, for example, a method using a convolutional neural network described in Japanese Patent No. 3078166 can be used. As the end points, parts that are considered to show individual characteristics such as left and right eyes, both end points of the mouth, and the nose are selected in advance. The positional relationship between the end points of the facial organ is detected as the attribute.
Further, as other attributes, attributes such as a person's age, sex, and facial expression may be estimated. A known technique can be used for the attribute estimation. For example, a person's attribute can be estimated by using a method like Unexamined-Japanese-Patent No. 2003-242486. By changing the learning data, not only a person but also a general object can be detected by the above method. This makes it possible to detect elements other than the facial organs on the human face, such as glasses, masks, hands, and other objects that become occlusions. It can be considered as a face of a person who has the occlusion as described above.

As described above, various attributes of an object can be detected and the amount of variation between the registered object and the input object can be calculated, but the example shown here is only an example. That is, in the present embodiment, the amount of change between the registered object and the input object is detected by some method, and the amount of change is reflected in the synthesis of the composite feature amount used for identification described later. Therefore, the attribute used for detecting the variation may be other than the above.
In this way, the amount of variation between the registered object and the input object is obtained, and the value is used as a parameter for partial feature amount synthesis described later. Typically, when the variation is large, the ratio of features that are slightly inferior but robust is high, and when the variation is small, the proportion of features that have low robustness but high peak recognition performance is increased. It is possible to control such as. This synthesis method will be described later.
Further, the feature vector conversion parameter may be changed using the amount of variation between objects. For example, when the illumination variation is detected, if the illumination variation amount is large, a dimension compression method (for example, LFDA) that is robust to the illumination variation may be used. When the amount of illumination variation is small, a dimension compression method (for example, PCA, ICA) that is relatively robust against variations other than illumination variation is used. By doing in this way, it becomes possible to cope with the amount of variation more appropriately, and an improvement in identification performance is expected.

Next, the partial feature synthesis unit 64 performs partial feature synthesis processing based on the input object identification data acquired in S30, S31, and S32, the dictionary data of the registered object, and the amount of variation between the two objects. To generate a synthesized partial feature amount (S33). Here, the combined partial feature amount is a partial feature amount composed of a plurality of feature amounts in the same partial region.
The combined partial feature amount is generated so that the registered object and the input object can be matched by combining the partial feature amounts of the registered object and the input object. Typically, the partial feature synthesis unit 64 concatenates feature amounts mainly including edge information derived from the spatial filter and feature vectors derived from their frequency distributions into one feature vector. Details of the partial feature synthesis processing will be described later.

Subsequently, the combined partial feature evaluation unit 65 evaluates the combined partial feature amount generated in S33 (S34). The combined partial feature evaluation unit 65 calculates a numerical value representing the similarity between the registered object and the input object for a plurality of combined partial feature amounts, and outputs the calculated value as one integrated value. Details of the combined partial feature evaluation processing will be described later. Subsequently, the evaluation result holding unit 66 holds the evaluation result of the combined partial feature acquired in S34 (S35).
Subsequently, the registered object dictionary data acquisition unit 62 determines whether or not the identification calculation has been completed for the dictionary data of all the registered objects (S36). At this time, if the registered object dictionary data acquisition unit 62 determines that there is still dictionary data (NO in S36), it performs the process of S31. On the other hand, when the registered object dictionary data acquisition unit 62 determines that the identification calculation has been completed for all dictionary data (YES in S36), the evaluation result integration unit 67 performs an evaluation result integration process (S37).
As the evaluation result integration process, for example, the process of outputting the dictionary data of the registered object having the largest output value of the combined partial feature evaluation process as the identification result can be considered most simply. Further, the result of dictionary data (higher level registration object dictionary data) having a high degree of coincidence may be output as a list.
The above is the description of the input object identification calculation process.

<Partial feature synthesis processing>
The partial feature synthesis process will be described. FIG. 11 is a diagram illustrating an example of the configuration of the partial feature synthesis unit 64. As illustrated in FIG. 11, the partial feature synthesis unit 64 includes a feature vector acquisition unit 71, an inter-object variation acquisition unit 72, a feature vector combination unit 73, and a partial feature synthesis parameter holding unit 74.
The feature vector acquisition unit 71 acquires a feature vector and its accompanying information from the dictionary data of the registered pattern and the input pattern identification data, and temporarily holds them. Here, the supplementary information is, for example, position information when the image is cut out from the image as a feature vector, parameters for conversion performed on the cut-out vector such as PCA, or an identifier representing these as a character string.

The inter-object variation amount acquisition unit 72 acquires the inter-object variation amount detected by the inter-object variation amount detection unit 63 and temporarily holds it. The feature vector combining unit 73 refers to the parameters acquired from the partial feature synthesis parameter holding unit 74 and the inter-object variation acquired from the inter-object variation acquiring unit 72, and the feature vector stored in the feature vector acquiring unit 71 To combine. This combining process will be described later.
The partial feature synthesis parameter holding unit 74 holds parameters for generating a synthesized partial feature referenced by the feature vector combining unit 73. Here, the parameters include setting information for combining and synthesizing a plurality of feature vectors, coordinate information of a cutout position of a target feature vector, an identifier of the feature vector as described above, and the like. This parameter can be determined by determining an optimal value for the learning data prepared in advance as described above using a machine learning method.

FIG. 12 is a diagram illustrating an example of a flowchart relating to the partial feature synthesis process performed by the partial feature synthesis unit 64. Hereinafter, the contents of the processing will be described with reference to FIG.
First, the feature vector acquisition unit 71 acquires a partial feature vector from dictionary data of a registered object or identification data of an input object (S40). Subsequently, the inter-object variation obtaining unit 72 obtains the inter-object variation (S41). Subsequently, the feature vector combining unit 73 acquires parameters from the partial feature synthesis parameter holding unit 74 (S42).
Subsequently, the feature vector combining unit 73 refers to the obtained variation between objects and the partial feature amount synthesis parameter to determine a feature vector synthesis method (such as a ratio) (S43).

Here, the item to be specifically determined is a ratio when a plurality of feature vectors for one partial region are linked according to the amount of variation between objects. As described above, the proportion of feature vectors suitable for identification is determined based on the balance between the characteristics of feature vectors acquired in advance and the amount of variation between objects.
For example, when the feature amount is relatively close to edge information such as LBP feature, information close to the image is directly compared, so that the descriptive power for identifying the individual is high and the degree of coincidence of the partial regions is high It is known to show good discrimination performance. On the other hand, it is sensitive to the displacement of the partial area between the objects, and if the position is not correct, the accuracy is greatly degraded. For this reason, if facial expression fluctuations, illumination fluctuations, etc. are small, it is better to increase the composition ratio.However, if the fluctuation is large and the displacement of the partial area between objects is expected to increase, the ratio should be It is considered better to make it smaller.

On the other hand, since the feature amount derived from the frequency distribution of edge information, such as the HOG feature, is distributed, it is relatively robust against positional deviation, but the descriptive power for identifying an individual is included in the edge information. There is a tendency to be inferior to the derived feature amount. Therefore, contrary to the feature amount derived from the edge information, when it is expected that the variation between objects is small and the positional deviation is also small, it is better to reduce the ratio. On the other hand, when the variation between the objects is large and the positional deviation is expected to be large, it is considered that the ratio should be larger than the feature amount derived from the edge information.
The actual feature vector composition ratio (composition ratio) can be determined by learning. Therefore, the relationship between the amount of variation between several objects, the temporarily determined composition ratio and the recognition performance is observed in advance with learning data, the composition ratio showing the best performance is determined, and the partial feature composition parameter is retained as an LUT It may be stored in the unit 74.

Subsequently, the feature vector combining unit 73 uses the combination method (concatenation method) and parameters obtained in S42 and S43, which are feature vectors derived from the edge information acquired in S40 and feature vectors derived from the frequency distribution of the edge information. The connection process is performed (S44). As the concatenation process, a plurality of vectors may be combined as one vector.
At this time, the composition ratio is measured by the dimension of each feature vector before composition. For example, when combining the LBP feature with 50% and the LBP histogram feature with 50%, the feature vector combining unit 73 combines the two with the same number of dimensions. Further, the feature vector combining unit 73 adjusts (controls) the number of dimensions of the feature amount derived from the edge information by changing the size of the partial region. In addition, the feature vector combining unit 73 adjusts the dimension of the feature amount derived from the frequency distribution of the edge information by changing the frequency measurement range (bin size).
When feature vectors are used after being dimensionally compressed, the feature vectors can be connected in accordance with the composition ratio by adjusting the dimension after dimension compression. By doing so, it is possible to adjust the dimension of the feature vector without changing the size or frequency range of the partial region, regardless of the feature amount derived from the edge information or the feature amount of the histogram system.
The above is the description of the partial feature synthesis process.

<Synthetic partial feature evaluation processing>
Next, the contents of the combined partial feature evaluation process will be described. FIG. 13 is a diagram illustrating an example of the configuration of the combined partial feature evaluation unit 65. As illustrated in FIG. 13, the combined partial feature evaluation unit 65 includes a combined partial feature vector acquisition unit 81, a combined partial feature evaluation parameter holding unit 82, and a combined partial feature evaluation value calculation unit 83.
The synthesized partial feature vector acquisition unit 81 acquires a synthesized partial feature (synthesized partial feature vector) from the partial feature synthesis unit 64. The combined partial feature evaluation parameter holding unit 82 holds a parameter group used by the combined partial feature evaluation value calculating unit 83. Details of the parameters will be described later. The combined partial feature evaluation value calculation unit 83 includes a combined partial feature vector (registered combined partial feature vector) of the registered object acquired by the combined partial feature vector acquisition unit 81 and a combined partial feature vector (input combined partial feature vector) of the input object. The evaluation value of is calculated. Details of the processing performed by the combined partial feature evaluation value calculation unit 83 will be described later.
Since there may be a plurality of synthesized partial features, in the synthesized partial feature evaluation process, a process of integrating evaluation values of a plurality of synthesized partial features into one value is also performed. For example, the synthesized partial feature evaluation unit 65 outputs the average of the evaluation values of all the synthesized partial features, the average of a predetermined number of high evaluation values, and the like as the final evaluation value.

FIG. 14 is a diagram illustrating an example of a flowchart relating to a combined partial feature evaluation process performed by the combined partial feature evaluation unit 65. Hereinafter, the contents of the processing will be described with reference to FIG.
First, the synthesized partial feature vector acquisition unit 81 acquires a synthesized partial feature vector from the partial feature synthesis unit 64 (S50). In addition, although there exist what was produced | generated from the registration pattern and the thing produced | generated from the input pattern in a synthetic | combination partial feature vector, the synthetic | combination partial feature corresponding by registration and input is each acquired here.
Subsequently, the combined partial feature evaluation value calculation unit 83 acquires an evaluation parameter from the combined partial feature evaluation parameter holding unit 82 (S51). The evaluation parameters are parameters used in subsequent processing, and include, for example, numerical data such as threshold values, evaluation functions for evaluating partial features of registered patterns, and partial features of corresponding input patterns, and the like. . Note that these parameters may be set for each partial feature, or may be common to partial features belonging to the same composite partial feature.

Subsequently, the combined partial feature evaluation value calculation unit 83 calculates an evaluation value of the combined partial feature using the partial feature vector acquired in S50 and the evaluation parameter acquired in S51 (S52). A method for calculating the evaluation value will be described later. Subsequently, the combined partial feature vector acquisition unit 81 determines whether all the combined partial features have been evaluated (S53). At this time, if the synthesized partial feature vector acquisition unit 81 determines that there is an item that has not been evaluated (NO in S53), it performs the process of S50. On the other hand, if the synthesized partial feature vector acquisition unit 81 determines that all the synthesized partial features have been evaluated (YES in S53), the process of the synthesized partial feature evaluation unit 65 ends.
The above is the description of the combined partial feature evaluation process.

<Composite partial feature evaluation value calculation processing>
Next, the composite partial feature evaluation value calculation process will be described. FIG. 15 is a diagram illustrating an example of a flowchart relating to a combined partial feature evaluation value calculation process performed by the combined partial feature evaluation value calculation unit 83. Hereinafter, the contents of the processing will be described with reference to FIG.
First, the composite partial feature evaluation value calculation unit 83 acquires a composite partial feature vector (S60). Subsequently, the combined partial feature evaluation value calculation unit 83 obtains the distance between the registered object and the input object from the acquired feature vector.
Here, the distance may be a simple Euclidean distance, or an inner product between vectors divided by the magnitude of the vector, for example, a cosine similarity.

Further, two vectors of registration and input may be input to an SVM (support vector machine). This SVM can be constructed by learning in advance two vectors acquired from the same person as positive examples and two vectors acquired from different persons as negative examples. At that time, the two vectors may be simply connected, or the difference between the two vectors may be taken. In general, the difference is smaller in input vector dimension and less learning time. The learned SVM parameters and the like are stored in the combined partial feature evaluation parameter holding unit 82.
Subsequently, the combined partial feature evaluation value calculation unit 83 calculates an evaluation value based on the calculated distance (S62). Typically, the distance acquired in S61 may be used as it is as the evaluation value, but it may be output as a binary value of “0” or “1” instead of a real number. In that case, the composite partial feature evaluation value calculation unit 83 determines whether the distance between the registered object and the input object acquired in S61 is larger than the threshold value, and “1” when the threshold value is exceeded, and when the value is below the threshold value Is set to “0”.
The above is the description of the combined partial feature evaluation value calculation process.

<Learning method of partial features and composite partial features>
Next, processing when AdaBoost is used for learning parameters for generating a combined partial feature by combining partial features will be described. Since this process can be performed offline in advance, it is not necessarily performed in the object identification apparatus 100. Moreover, a well-known technique is applicable to the process demonstrated here, The description is abbreviate | omitted as needed.
FIG. 16 is a diagram illustrating an example of a flowchart relating to the learning process of the partial region.

First, the object identification device 100 acquires learning data (S70). When dealing with a human face, a large number of images including a face with a label representing an individual identifier are prepared as learning data. At this time, it is desirable that a sufficient number of images per person is prepared. In addition, in order to learn a method for synthesizing partial features and feature vectors that are robust against illumination fluctuations and facial expression fluctuations, it is important to prepare a sample that sufficiently includes the fluctuations in the learning data.
From the labeled face image, data belonging to a feature class (intra-class) representing a difference due to a variation in an individual's face and data belonging to a feature class (extra-class) representing a difference due to a variation in another human face Kinds can be generated. That is, intra-class data can be created from face images with the same class label, and extra-class data can be created from face images with different class labels.
As described above, one sample of learning data is always generated from a pair of images (or objects). When making this pair, it is preferable to detect the amount of variation between the pair images.

Subsequently, the object identification device 100 performs weak hypothesis selection processing (S71). Here, the weak hypothesis can be associated with the parameter of the partial feature in the present embodiment. In other words, a large number of partial features having different parameters related to partial features such as spatial position, size, feature vector type (edge system, histogram system, etc.), dimension reduction number (PCA, LPP, ICA, etc.) are prepared in advance. deep. The object identification device 100 associates a combination of partial features having different parameters and a combined partial feature vector in which the combination ratio when combining the feature vectors is changed with the weak hypothesis.
When these prepared weak hypotheses are selected by learning, AdaBoost can be used as learning of a two-class classifier that identifies two classes of extra-class and intra-class. In other words, the object identification device 100 selects parameters with partial features (location, size, etc.), and the composition ratio when combining them is optimal for the learning data.

As a specific procedure for performing the performance evaluation, as described in the description of the combined partial feature evaluation unit 65, it is preferable to evaluate the learning data according to an example of an actual identification process. At this time, if the learning data is divided according to the amount of variation between the images detected in advance, the optimum weak hypothesis for the amount of variation, that is, the feature parameter and the composition ratio are obtained. . Then, after performing the evaluation, the object identification device 100 determines whether the correct identification result is obtained between the data between persons with the same label and the data between persons with different labels, and the weighted error of the learning data Find the rate.
When selecting the weak hypothesis with the best performance, the object identification device 100 updates the weighting of the learning data based on the identification result regarding the learning data of the weak hypothesis (S72). Subsequently, the object identification device 100 determines whether the number of weak hypotheses has reached a predetermined number (S73). At this time, when it is determined that the predetermined number has been reached (YES in S73), the object identification device 100 ends the learning process. On the other hand, when it is determined that the predetermined number has not been reached (NO in S73), the object identification device 100 selects a new weak hypothesis.

Further, when learning a plurality of partial features, an optimization method such as a genetic algorithm (GA) may be applied. In this case, the object identification apparatus 100 does not prepare all weak hypotheses before entering the AdaBoost procedure, but dynamically constructs candidates while selecting weak hypotheses. That is, one having a good performance is selected from weak hypothesis candidates partially prepared in advance.
Then, the object identification device 100 generates new weak hypothesis candidates while combining those having good performance, and evaluates the performance. In this way, weak hypothesis candidates can be narrowed down efficiently. According to the above configuration, an increase in learning time can be suppressed.

Further, the object identification device 100 may select only partial features (location, size, etc.) as weak hypotheses, select them with AdaBoost, and select a combination ratio of partial features with another framework. For example, in AdaBoost, the object identification device 100 learns from data including fluctuation amounts between various images, and determines the location and size of the partial feature. Next, the object identification device 100 evaluates the learning data while obtaining the most suitable composition ratio while changing the composition ratio with the learning data in which the amount of variation between images is fixed.
With this configuration, it is possible to cope with the problem that the learning time becomes enormous due to an increase in the combination of the feature parameter and the composition ratio.
As described above, it is possible to determine parameters relating to partial features, combinations thereof, and parameters relating to an evaluation method.
The above is the description of the procedure for learning the parameters related to the combined partial feature.

As described above, the object identification device 100 detects a variation amount between the registered pattern and the input pattern, extracts a plurality of types of partial features corresponding to each, and synthesizes the partial features according to the variation amount. As described above, by evaluating the feature appropriately synthesized according to the variation amount, it is possible to perform robust and high-precision identification with respect to the variation between the pattern of registration and input.
In other words, the object identification device 100 detects a fluctuation amount such as a shooting condition between the registered object and the input object. In accordance with the detected amount of variation, the object identification device 100 dynamically selects a feature amount that is slightly weak in variation between objects but has high absolute identification performance, and a feature amount that is strong in variation but slightly inferior in identification absolute performance. In combination, the identification performance is improved.
The above is the description of the first embodiment.

<Second Embodiment>
The present embodiment is different in the contents of the processing of the input object identifying unit 6 of the first embodiment. More specifically, in the first embodiment, the method for synthesizing partial features is a feature vector concatenation process, whereas in the second embodiment, the co-occurrence of a plurality of partial features is considered. The point of synthesis is different. More specifically, the processing contents of the composite partial feature evaluation unit 65 of the first embodiment are different.
This will be described in more detail below. In the present embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted as appropriate.
Note that the hardware configuration of the entire object identification apparatus 100 according to the present embodiment is the same as that of the first embodiment, and a description thereof will be omitted. Further, in order to distinguish from the first embodiment, in this embodiment, the combined partial feature evaluation unit 65 having different processing contents is referred to as a combined partial feature evaluation unit 165. For convenience of explanation, the object to be identified is the face of a person in the image, but it can also be applied to objects other than the face of the person.

<Synthesized partial feature evaluation unit>
FIG. 17 is a diagram illustrating an example of the configuration of the combined partial feature evaluation unit 165. The combined partial feature evaluation unit 165 includes a combined partial feature vector acquisition unit 181, a combined partial feature evaluation parameter storage unit 182, a combined partial feature evaluation value calculation unit 183, a combined partial feature likelihood calculation unit 184, and a partial feature quantity measurement unit 185. Have The difference from the first embodiment is that a combined partial feature likelihood calculating unit 184 and a partial feature quantity measuring unit 185 are added.
The synthesized partial feature vector acquisition unit 181 acquires a synthesized partial feature from the partial feature synthesis unit 64. The combined partial feature vector acquisition unit 181 also acquires the amount of variation between the input vector and the registered vector. The combined partial feature evaluation parameter holding unit 182 holds a parameter group used in the combined partial feature likelihood calculating unit 184 and the partial feature quantity measuring unit 185. Details of the parameters will be described later.

The combined partial feature evaluation value calculation unit 183 calculates the evaluation value of the combined partial feature using the likelihood of the combined partial feature by the combined partial feature likelihood calculation unit 184 and the measurement result of the partial feature quantity measurement unit 185. Details of processing performed by the combined partial feature evaluation value calculation unit 183 will be described later. The combined partial feature likelihood calculating unit 184 calculates the likelihood of the combined partial feature. A method of calculating the likelihood of the combined partial feature will be described later.
The partial feature quantity measuring unit 185 determines whether or not a partial feature included in one composite partial feature satisfies a predetermined criterion, and measures the quantity of partial features that satisfy the predetermined condition. Details of processing performed by the partial feature quantity measuring unit 185 will be described later.
Since there may be a plurality of composite partial features, in the composite partial feature evaluation process, a process of integrating evaluation values of a plurality of composite partial features into one value is also performed. For example, the composite partial feature evaluation unit 165 outputs an average of evaluation values of all composite partial features, an average of a predetermined number of high evaluation values, and the like as a final evaluation value.

FIG. 18 is a diagram illustrating an example of a flowchart relating to a combined partial feature evaluation process performed by the combined partial feature evaluation unit 165. Hereinafter, the contents of the combined partial feature evaluation process will be described with reference to FIG.
First, the synthesized partial feature vector acquisition unit 181 acquires a synthesized partial feature vector from the partial feature synthesis unit 64 (S80). In addition, although there exist what was produced | generated from the registration pattern and what was produced | generated from the input pattern in a synthetic | combination partial feature vector, the synthetic | combination partial feature corresponding by registration and input is each acquired here.
Subsequently, the combined partial feature likelihood calculating unit 184 obtains an evaluation parameter from the combined partial feature evaluation parameter holding unit 182 (S81). The evaluation parameters are parameters used in subsequent processing, and include, for example, numerical data such as a threshold value, an evaluation function for evaluating a partial feature of a registered pattern and a partial feature of a corresponding input pattern. Note that these parameters may be set for each partial feature, or may be common to partial features belonging to the same composite partial feature.

In S82, the partial feature quantity measuring unit 185 evaluates the partial features belonging to the combined partial feature vector, and measures the quantity of partial features that satisfy a predetermined condition. The contents of the partial feature quantity measurement process will be described later. Subsequently, the combined partial feature likelihood calculating unit 184 calculates the likelihood of the combined partial feature (S83). The likelihood of the combined partial feature can be determined in advance using machine learning, but can also be dynamically calculated according to the combined partial feature corresponding to the input pattern and the registered pattern. A method of obtaining the likelihood of the combined partial feature will be described later.
Subsequently, the composite partial feature evaluation value calculation unit 183 calculates an evaluation value of the composite partial feature using the quantity of partial features obtained in S82 and the likelihood of the composite partial feature calculated in S83 (S84). . A method for calculating the evaluation value will be described later.

Subsequently, the combined partial feature vector acquisition unit 181 determines whether all the combined partial features have been evaluated (S85). At this time, if the synthesized partial feature vector acquisition unit 181 determines that there is an item that has not been evaluated (NO in S85), it performs the process of S80. On the other hand, if the combined partial feature vector acquisition unit 181 determines that all the combined partial features have been evaluated (YES in S85), the processing of the combined partial feature evaluation unit 165 ends.
The above is the description of the combined partial feature evaluation process.

<Partial feature quantity measurement processing>
Next, the content of the partial feature quantity measurement process will be described. FIG. 19 is a diagram illustrating an example of a flowchart according to the partial feature quantity measurement process. Hereinafter, the contents of the processing will be described with reference to FIG.
First, the partial feature quantity measuring unit 185 evaluates partial features (S90). Here, as an example of evaluation, a method of using the similarity between the partial feature of the registered pattern and the partial feature of the corresponding input pattern will be described. Here, the degree of similarity θ can be obtained by an inner product between partial feature vectors, for example. More specifically, the partial feature quantity measuring unit 185 calculates the similarity θ using the following equation (2).

Here, u (h) and v (h) represent partial feature vectors generated from the registered pattern and the input pattern of the feature amount h, respectively. l is the number of dimensions of feature vectors, N is the represents the types of features, w _h represents the weight of the feature quantity h. The above is an inner product between feature vectors. In addition to this, for example, a normalized correlation between feature vectors may be used as a similarity, or an inverse of a distance between feature vectors is used as a similarity. May be.
The weighting w _h of the feature amount h can be determined by the amount of change between the registered vector and the input vector. As described in the first embodiment, when the variation amount is small, it is preferable to increase the weight of the feature amount (LBP feature, Gabor feature, etc.) derived from the edge information. When the variation amount is large, it is preferable that the weight of the histogram type feature amount (HOG feature, SIFT feature, etc.) is increased. As this weighting, an appropriate value is set in advance by evaluating learning data.
Further, as described in the first embodiment, the weighting parameter may be regarded as one of weak hypotheses and determined using AdaBoost.

Then, the partial feature quantity measuring unit 185 evaluates the similarity between the corresponding partial features of the registered pattern and the input pattern obtained in this way (S91). For example, the partial feature quantity measuring unit 185 determines whether the similarity obtained in S90 exceeds a predetermined threshold.
Subsequently, the partial feature quantity measuring unit 185 checks whether all partial features included in the same composite partial feature have been determined (S92). At this time, the partial feature quantity measuring unit 185 performs the process of S90 when the determination has not been completed (NO in S92). On the other hand, the partial feature quantity measuring unit 185 measures the quantity of partial features whose similarity exceeds a predetermined standard when the determination is completed for all the partial features (YES in S92) (S93). Typically, the partial feature quantity measuring unit 185 counts the number of partial features exceeding the threshold. Further, in the measurement process of the quantity n in S93, the partial feature aggregation function can be defined as follows.

Here, m is the number of partial features in the same composite partial feature, and ω is a threshold value. With this configuration, the quantity of partial features can be measured in consideration of partial features in the vicinity of the threshold.
The above is the description of the partial feature quantity measurement process.

<Combined partial feature likelihood calculation process>
Next, the combined partial feature likelihood calculation process will be described. FIG. 20 is a diagram illustrating an example of the configuration of the combined partial feature likelihood calculating unit 184. The combined partial feature likelihood calculation unit 184 includes a partial feature vector information acquisition unit 191, a partial feature vector statistical information acquisition unit 192, a statistical information holding unit 193, and a likelihood calculation unit 194.
The partial feature vector information acquisition unit 191 acquires information on partial feature vectors that are constituent elements of the combined partial feature vector acquired from the combined partial feature vector acquisition unit 181. More specifically, the partial feature vector information acquisition unit 191 uses the number of dimensions of the partial feature vector, the spatial position on the original pattern, the size, the method of converting the feature vector, the accompanying parameter information, and the like. is there. As the parameter information, when the feature vector conversion method is PCA, the number of dimension reductions, the cumulative contribution rate, the number of data used for PCA learning, and the like can be cited.
The partial feature vector statistical information acquisition unit 192 acquires the statistical information based on the information acquired by the partial feature vector information acquisition unit 191. Details of the processing will be described later. The statistical information holding unit 193 holds the statistical information acquired by the partial feature vector statistical information acquisition unit 192. Note that statistical information obtained in advance outside the apparatus may be held. The likelihood calculation unit 194 calculates the likelihood of the combined partial feature vector based on the information obtained from the partial feature vector statistical information acquisition unit 192. Details of the processing will be described later.

FIG. 21 is a diagram illustrating an example of a flowchart according to the combined partial feature likelihood calculation process. Hereinafter, the contents of the process will be described with reference to FIG.
First, the partial feature vector information acquisition unit 191 acquires partial feature vector information that is a component of the combined partial feature vector (S100). Here, as the partial feature vector information, various information such as the spatial position and size on the pattern from which the partial feature vector is cut out as described above can be considered.
Subsequently, the partial feature vector statistical information acquisition unit 192 acquires the statistical information based on the acquired partial feature vector information (S101). Subsequently, the likelihood calculating unit 194 calculates the likelihood of the combined partial feature based on the acquired statistical information (S102). Here, as an example, a case will be described in which partial feature vector information is a spatial position and size on a cut-out original pattern, and statistical information to be acquired is its variance. The likelihood calculating unit 194 obtains a variance according to the following formulas (5) and (6) for each position and size of the partial feature vector.

  Here, l represents the number of partial features included in the composite partial feature, ν and ν with an overline represent the cut-out positions of the partial features and their averages, and s and s with an overline are partial It represents the feature size and its average. In addition, the likelihood calculating unit 194 obtains the likelihood of the combined partial feature from the above statistical information by the following equation (7).

Thus, the following effects can be expected by setting the average of the variance of the position and size of the partial feature as the likelihood of the combined partial feature. In other words, the greater the variance of the incidental information (position, size) of the partial features that make up the composite partial feature, the more the pattern to be identified is observed on various scales. Can be expressed directly. For example, when the pattern to be identified is the face of a person in the image, it is considered more appropriate to use the vicinity of the mouth and nose at the same time for the determination than to determine only the vicinity of the eyes.
Further, instead of the spatial information of the partial feature vectors, the variance of the parameters related to the feature vector conversion may be used as the likelihood. More specifically, when the dimensional compression ratio (PCA cumulative contribution ratio) is different for each partial feature, the variance of the dimensional compression ratio may be used as the likelihood.

Further, the difference between the feature vector extraction methods may be the likelihood. Most simply, the type of extraction method is used as the likelihood as it is. For example, when all the partial features in the combined partial feature are extracted from the luminance image, the likelihood is set to “1”, and when extracted from a plurality of methods, for example, an image obtained by applying the luminance image and the Gabor filter The likelihood may be “2”.
The likelihood may be determined in advance from learning data. The discrimination performance for the learning data for each partial feature can be measured in advance and stored in the statistical information holding unit 193, and the average of these can be used as the likelihood of the combined partial feature. Further, the likelihood obtained from the variance of the partial features and the likelihood obtained from the learning data may be combined and calculated as the likelihood of the combined partial feature.
The above is the description of the combined partial feature likelihood calculation process.

<Composite partial feature evaluation value calculation processing>
Next, the composite partial feature evaluation value calculation process will be described. FIG. 22 is a diagram illustrating an example of a flowchart relating to a combined partial feature evaluation value calculation process performed by the combined partial feature evaluation value calculation unit 183. Hereinafter, the contents of the process will be described with reference to FIG.
First, the composite partial feature evaluation value calculation unit 183 acquires a composite partial feature vector (S110). Subsequently, the combined partial feature evaluation value calculation unit 183 acquires the result of the partial feature quantity measurement process from the partial feature quantity measurement unit 185 (S111). Subsequently, the combined partial feature evaluation value calculation unit 183 acquires the likelihood of the combined partial feature from the combined partial feature likelihood calculation unit 184 (S112). Subsequently, the combined partial feature evaluation value calculation unit 183 calculates an evaluation value of the combined partial feature based on the acquired value (S113). The combined partial feature evaluation value calculation unit 183 obtains the evaluation value V _g using the following equation (8).

Here, n is the result of the partial feature quantity measurement process, and σ is the likelihood of the combined partial feature. By calculating in this way, the evaluation value can be changed greatly depending on the difference in quantity in which the partial features belonging to the combined partial feature satisfy a predetermined criterion. As an effect of performing identification by combining partial features, an evaluation value in a state where a plurality of partial features satisfy a criterion is greatly different from an evaluation value in a state where only one partial feature is satisfied.
In the example of human face identification, it is considered that the partial features coincide with each other by chance due to lighting conditions, facial expression variations, etc., but it is considered that there is a low probability that the partial features coincide at the same time. . In other words, when a plurality of partial areas match at the same time, the possibility that the registered pattern and the input pattern belong to the same class is higher than when one matches. From such an expectation, a state where a plurality of partial features satisfy the standard is represented by the quantity and used for identification.
In addition, by using quantities for evaluation, there is a possibility that more robust identification can be performed against fluctuations. For example, instead of the equation (8), an evaluation function such as the following equation (9) can be used.

Here, r is the similarity of the partial features corresponding to the registered pattern and the input pattern, and w is the weight of each similarity for obtaining a weighted sum. According to this equation (9), when the similarity with a large weight is different from the expected value due to the influence of illumination fluctuation or the like, the influence on the evaluation value is also increased. On the other hand, it is expected that the weight of a specific partial feature is increased by representing the quantity as in Expression (8), and as a result, it is expected to be robust against variation.
Moreover, weighting can be performed between a plurality of synthesized partial features by multiplying the likelihood of the synthesized partial features.
Moreover, the calculation of the evaluation value according to the equations (8) and (9) is merely an example, and other expressions are naturally conceivable. For example, it may be a polynomial having a quantity n of partial features. Further, instead of using the partial feature quantity n directly, it may be used after normalization by dividing by the number of partial features 1 included in the composite partial feature. By doing so, the level of the evaluation value of each synthesized partial feature can be made constant even if the contained partial features vary among a plurality of synthesized partial features.

Further, in the present embodiment, it is also significant that the evaluation value of the combined partial feature is determined according to the partial feature quantity n. Therefore, an optimum evaluation function of the combined partial feature may be selected according to the category to be identified, and is not limited to the function form used in the description here.
Further, when there are a plurality of synthesized partial features, the synthesized partial feature evaluation value calculation unit 183 integrates these and outputs them as one value. The simplest possible method of integration is the average of them. The combined partial feature evaluation value calculation unit 183 sets an average of the evaluation values obtained by the equations (8) and (9) as a final output value.
Moreover, you may make it use only a thing with high likelihood. In addition, before taking the average, only those having a high likelihood of each synthesized partial feature may be selected, and the average may be taken only by them.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

  According to the configuration of the above-described embodiment, it is possible to further improve the accuracy of identifying an object included in an image.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

  1 imaging optical system, 2 imaging unit, 3 imaging control unit, 4 image recording unit, 5 object registration unit, 6 input object identification unit, 7 external output unit, 8 connection bus

Claims (9)

An inter-object variation amount detecting means for detecting a variation amount between objects of the input object included in the input image and the registered object included in the registered image;
Partial feature extraction means for extracting at least one or more corresponding partial feature amounts between the input object and the registered object;
A synthesis method for combining the partial feature amounts extracted by the partial feature extraction unit according to the amount of variation between the objects is determined, and the partial feature amounts are synthesized by combining the partial feature amounts using the determined synthesis method. A partial feature amount synthesis means to be generated;
Input object identifying means for identifying which registered object the input object corresponds to using the synthesized partial feature amount;
An object identification device comprising:   The object identification apparatus according to claim 1, wherein the partial feature extraction unit extracts at least a partial feature quantity that is invariant to a change in position of a partial area or a partial feature quantity that is robust to a change in position.   The object identification device according to claim 1, wherein the partial feature extraction unit extracts a partial feature amount expressing at least information acquired from an image by a frequency distribution.   The object identifying apparatus according to claim 1, wherein the inter-object variation detecting unit calculates the inter-object variation based on a shooting condition.   5. The object identification device according to claim 4, wherein the photographing condition is a parameter related to either automatic exposure or automatic focus.   The inter-object variation amount detection unit further includes an attribute detector that detects attributes of the input object and the registered object, and detects a variation amount of an attribute of the input object and the registered object. Item 1. The object identification device according to Item 1. The partial feature extraction means further comprises conversion means for converting the extracted partial feature value using a conversion parameter,
The partial feature amount combining means, when combining a plurality of partial feature amounts in the same positional relationship between the registered object and the input object, if the amount of variation between the objects is large, information acquired from the image The object identification device according to claim 2, wherein the conversion parameter is controlled so that a ratio of partial feature amounts expressed by a frequency distribution is increased. An inter-object variation amount detecting step for detecting a variation amount between objects of the input object included in the input image and the registered object included in the registered image;
A partial feature extraction step of extracting at least one or more corresponding partial feature quantities between the input object and the registered object;
A synthesis method for combining the partial feature values extracted in the partial feature extraction step according to the magnitude of the amount of variation between the objects is determined, and the partial feature value is synthesized by combining the partial feature values using the determined synthesis method. A partial feature amount synthesis step to be generated; and
An input object identification step for identifying which registered object the input object corresponds to using the synthesized partial feature amount;
An object identification method characterized by comprising:   The program which makes a computer perform each process of the object identification method of Claim 8.

Images