JP5653003B2 - Object identification device and object identification method - Google Patents

Description

  The present invention relates to an object identification device and an object identification method.

As a technique for identifying that an object that is a subject in image data is the same as an object that is a subject in another image, for example, there is a face identification technique for identifying an individual's face. Hereinafter, in this specification, the identification of an object means that a difference between individual objects (for example, a difference between persons as individuals) is determined. On the other hand, detection of an object means that an object that falls within the same category is determined without distinguishing individuals (for example, a face is detected without distinguishing individuals).
As a face identification technique, for example, there is a method as described in Non-Patent Document 1. This is an algorithm that makes it possible to perform face registration and additional learning in real time by replacing an individual identification problem by a face with a two-class identification problem of a feature class called a differential face.

For example, in the face identification using a generally well-known support vector machine (SVM), in order to identify the faces of n persons, n faces that are registered and other faces are identified. SVM discriminators are required. When registering a person's face, SVM learning is required. SVM learning requires a large amount of face data of a person to be registered and face data of already registered persons and other persons, which requires a lot of calculation time. there were.
However, according to the method of Non-Patent Document 1, additional learning can be made substantially unnecessary by replacing the problem of personal identification with the following two classes of identification problems. That is,
Intra-personal class: variation feature class such as illumination variation, facial expression / direction, etc. between images of the same person. Extra-personal class: two classes of variation feature class between images of different people. Assuming that the distribution of the two classes is constant regardless of a specific individual, the classifier is configured by reducing the face identification problem of the individual to the classification problem of the two classes. A large number of images are prepared in advance, and a discriminator for discriminating between a variation feature class between the same persons and a variation feature class between different persons is learned. The new registrant need only hold the face image (or the result of extracting the necessary features). When discriminating, the difference feature is extracted from the two images, and the discriminator determines whether the person is the same person or a different person. This eliminates the need for learning such as SVM at the time of personal face registration, and registration can be performed in real time.

In the apparatus and method for identifying an object (more specifically, the face of a person) as described above, a factor that degrades the identification performance is a change in the registration image and the authentication image. That is, there are fluctuations in the object to be identified (person's face), more specifically, lighting conditions, orientation / posture, hiding by other objects, fluctuations due to facial expressions, and the like. When the above fluctuations become large, the identification performance is greatly reduced.
With respect to this problem, Patent Document 1 performs a plurality of pattern matching for each partial region, removes outliers from those results, and integrates the matching degree of each partial region, thereby improving robustness against fluctuations. Secured.

JP 2003-323622 A

Baback Moghaddam, Beyond Eigenfaces:. Probabilistic Matching for Face Recognition (M.I.T Media Laboratory Perceptual Computing Section Technical Report No.433), ProbabilisticVisual Learning for Object Representation (IEEE Transactions on PatternAnalysis and Machine Intelligence, Vol 19, No. 7 , JULY 1997)

However, it is considered that there is room for improvement in performance by simply removing outliers or taking a weighted average or the like from the similarity of a plurality of partial areas. For example, when there is an error in the setting of the plurality of partial areas, particularly when a partial area is set outside the range of the input image, it is considered that there is a low possibility that the error can be sufficiently corrected only by the above processing. In order to maintain discrimination performance even in environments with large fluctuations such as the human face and various shooting conditions, if there is an error in the multiple partial area settings, the process can absorb the error to some extent Is considered effective.
Assuming application to a digital camera, a web camera, etc., it is desirable that the identification performance does not deteriorate even when the image capturing condition and the variation (size, orientation, facial expression, etc.) of the object are large.

  The present invention has been made in view of such problems, and it is an object of the present invention to improve robustness and identification performance against variations in illumination, size, orientation, and the like.

Accordingly, the present invention provides an object identification device, wherein an input unit that inputs an input image including the object, an acquisition unit that acquires a registered image including the object, affine transformation is performed on the input image, First partial area setting means for setting a partial area in the converted image, and at least a part of the partial area set in the image after the affine transformation protrudes from the area of the input image before the affine transformation. A first determination means for determining whether or not the partial area is not protruding from the area of the input image, a feature vector is set from the partial area, and the partial area is the input If it is determined that extends off region of an image, of the partial region, for a region that protrudes from the region of the input image, sets a random number , Set the region and the first setting means for setting a feature vector, the registered image feature vector and the first including a region not protrude from the region and the input image extends off the said random number is set And an identification unit for calculating a correlation with the feature vector of the partial region set by the unit and identifying the object based on the calculated result.
The present invention may also be an object identification method.

  According to the present invention, it is possible to improve robustness and identification performance against variations in illumination, size, orientation, and the like.

It is a figure which shows an example of the hardware constitutions of an object identification device. It is the flowchart which showed an example of the whole process of the object identification apparatus. It is a figure which shows an example of a structure of an object registration part. It is a figure which shows an example of a structure of an object dictionary data generation part. It is a flowchart showing an example of the process performed in the feature vector extraction part. It is a figure which shows an example of an object identification part. It is the flowchart which showed an example of the identification process. It is the figure which showed an example of the structure of the data generation part for object identification. It is a figure which shows an example of a structure of an object identification calculating part. It is the flowchart which showed an example of the object identification calculation process. It is the flowchart which showed an example of the learning process of the partial area. It is a figure which shows an example of the hardware constitutions of an object identification device. It is a figure which shows an example of a structure of an object registration part. It is the flowchart which showed an example of the process in a feature vector extraction part. It is a figure which shows an example of a structure of an object identification part. It is a schematic diagram at the time of comprising an object discriminator with the tree structure of a weak discriminator.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Embodiment 1
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram (part 1) illustrating an example of a hardware configuration of the object identification device 100. As shown in FIG. 1, the object identification device 100 includes an imaging optical system 1, an imaging unit 2, an imaging control unit 3, an image recording unit 4, an object registration unit 5, and an object identification unit 6. The object identification device 100 also includes an external output unit 7 that outputs an object identification result, and a bus 8 for performing control / data connection of each component.
The object registration unit 5 and the object identification unit 6 may typically be a dedicated circuit (ASIC) and a processor (reconfigurable processor, DSP, CPU, etc.), respectively. Further, the object registration unit 5 and the object identification unit 6 may exist as programs that are executed inside a single dedicated circuit and general-purpose circuit (PC CPU).

The imaging optical system 1 includes an optical lens having a zoom mechanism. Further, the imaging optical system 1 may include a driving mechanism in the pan / tilt axis direction.
A CCD or CMOS image sensor is typically used as the image sensor of the imaging unit 2, and a predetermined image signal (for example, subsampling or block reading is obtained by a read control signal from a sensor drive circuit (not shown). Signal) is output as image data.
The imaging control unit 3 controls the timing of actual shooting based on an instruction from the photographer (viewing angle adjustment instruction, shutter pressing, etc.) and information from the object registration unit 5 or the object identification unit 6. The imaging control unit 3 may include a control device that controls automatic exposure (AE) and automatic focus (AF).
The image recording unit 4 is composed of a semiconductor memory or the like, holds image data transferred from the imaging unit 2, and stores image data at a predetermined timing in response to requests from the object registration unit 5 and the object identification unit 6. Forward.

The object registration unit 5 extracts information on an object to be identified from the image data, and records / holds it. A more detailed configuration of the object registration unit 5 and more specific contents of the processing actually performed will be described later.
The object identification unit 6 identifies an object in the image data based on the image data and the data acquired from the object registration unit 5. A more specific configuration and details of the processing performed on the object identification unit 6 will be described in detail later.
The external output unit 7 is typically a monitor such as a CRT or a TFT liquid crystal, and displays the image data acquired from the imaging unit 2 and the image recording unit 4 or displays the object registration unit 5 and the object identification unit in the image data. The result output of 6 is superimposed and displayed. The external output unit 7 may take a form of outputting the result output of the object registration unit 5 and the object identification unit 6 as electronic data to an external memory or the like.
The connection bus 8 is a bus for performing control and data connection between the above components.

<Overall flow>
FIG. 2 is a flowchart showing an example of the overall processing of the object identification device 100. With reference to FIG. 2, an actual process in which the object identification device 100 identifies an object from an image will be described. In the following, a case where the object to be identified is a human face will be described, but the present embodiment is not limited to this.
First, the object identification unit 6 acquires image data (image data input) from the image recording unit 4 (S00). Subsequently, the object identification unit 6 performs a human face detection process on the acquired image data (S01). A known technique may be used as a method for detecting a human face from an image. The object identification unit 6 can use, for example, a technique proposed in “Patent No. 3078166” or “Japanese Patent Laid-Open No. 2002-8032”.
If the human face exists in the image after the process of detecting the face of the person who is the target object (Yes in S02), the object identification unit 6 performs the object identification process, that is, the individual identification process. . If no human face is present in the image (No in S02), the object identification unit 6 ends the process shown in FIG. More specific processing contents of the object identification processing (S03) will be described later in detail.

The object identification unit 6 determines whether there is a face corresponding to the registered person from the result of the object identification process (S04). When the same person as the face detected in (S01) is among the registered persons (Yes in S04), the object identifying unit 6 proceeds to the process of (S07). If the detected face does not match any registered person (No in S04), the object identifying unit 6 determines whether to register the person (S05). Although this may be set in advance, for example, the user may determine whether to register on the spot through an external interface, a GUI, or the like. If it is determined to register (Yes in S05), the object identifying unit 6 performs registration processing of an object (person's face) described later (S06). When registration is not performed (No in S05), the object identification unit 6 continues the process as it is. After the object registration processing in (S06) and when registration is not performed in (S05), the object identification unit 6 determines whether the processing has been completed for all the detected objects (S07). When there is an unprocessed object (No in S07), the object identification unit 6 returns to (S03). When the processing has been completed for all the detected objects (Yes in S07), the object identification unit 6 outputs a series of object identification processing results to the external output unit 7.
The above is the overall processing flow of the object identification device 100 according to the present embodiment.

<Object registration part>
The object registration process will be described. FIG. 3 is a diagram illustrating an example of the configuration of the object registration unit 5. As shown in FIG. 3, the object registration unit 5 includes an object dictionary data generation unit 21, an object dictionary data holding unit 22, and an object dictionary data selection unit 23.
The object dictionary data generation unit 21 generates object dictionary data necessary for identifying an individual object from the image data acquired from the image recording unit 4. For example, when the object dictionary data generation unit 21 determines a two-class problem of intra-class and extra-class as in Non-Patent Document 1, typically, a person's face image may be used as dictionary data. . The object dictionary data generation unit 21 may normalize the size and direction (in-plane rotation direction) of the object image data detected by the object detection process, and then store the image data in the object dictionary data holding unit 22. Good.
Here, the object dictionary data generation unit 21 may perform the following when normalizing the rotation direction, particularly affine transformation for correcting in-plane rotation. That is, in the affine transformation process for correcting the inclination of the face, the object dictionary data generation unit 21 converts the converted value if the image after the affine transformation refers to the outside of the area with respect to the original image to be referred to. Replace with a random number. Usually, in the above case, a predetermined fixed value is often set in an image after affine transformation. However, if a fixed value is set, there may be a problem if the partial area protrudes from the target object. More specifically, in the case of the image of the object dictionary data and the image of the identification data, when the portion set to a fixed value enters a part of the partial area, the similarity between the two increases, and the identification is performed. To affect. In order to avoid such a situation, the object dictionary data generation unit 21 may replace the data having no corresponding point in the reference source image with a random number in the image after the affine transformation.

It is possible to reduce the amount of dictionary data by holding only the data necessary for identification, not the image data itself. When the identification calculation is performed by taking the vector correlation (similarity) of the partial area of the object, the object dictionary data generation unit 21 may cut out only the partial area in advance.
As described above, the object dictionary data generation unit 21 appropriately extracts necessary information from the image, performs predetermined conversion described later, and then uses the object dictionary data holding unit 22 as a feature vector for identifying the object. To store. More specific processing contents performed by the object dictionary data generation unit 21 will be described in detail later.
The object dictionary data selection unit 23 reads out necessary object dictionary data from the object dictionary data holding unit in response to a request from the object identification unit 6 described later, and transfers the object dictionary data to the object identification unit 6.

<Object dictionary data generator>
FIG. 4 is a block diagram illustrating an example of the configuration of the object dictionary data generation unit 21. As shown in FIG. 4, the object dictionary data generation unit 21 includes a partial region setting unit 31, a feature vector extraction unit 32, a feature vector conversion unit 33, and a feature vector conversion data holding unit 34.
The partial region setting unit 31 sets the position and range where the feature vector extraction unit 32 extracts feature vectors for image data. The position and range of the partial area may be determined in advance using a machine learning method. For example, the partial region setting unit 31 may set a plurality of partial region candidates and select from the plurality of candidates using AdaBoost. A method of actually determining a partial area by applying AdaBoost will be described in detail in the description of the object identification unit described later. The partial area setting unit 31 determines a predetermined number as the number of partial areas in advance according to the processing time or the like. The partial area setting unit 31 may measure and determine the number that can obtain sufficient discrimination performance for a learning sample prepared in advance.

The feature vector extraction unit 32 extracts feature vectors from the registration object data. When the object is a human face, the feature vector extraction unit 32 typically performs a process of extracting data necessary for identification from an image including the face. The feature vector extraction unit 32 extracts data necessary for identification from the partial area set by the partial area setting unit 31 as a feature vector. In addition, the feature vector extraction unit 32 may extract a feature vector from a result of performing some filter operation such as a Gabor filter instead of directly acquiring a luminance value. The contents of the processing performed by the feature vector extraction unit 32 will be described in detail later.
The feature vector conversion unit 33 performs a predetermined conversion on the feature vector extracted by the feature vector extraction unit 32. The feature vector conversion unit 33 performs, for example, dimension compression by principal component analysis (PCA) or dimension compression by independent component analysis (ICA) as the feature vector conversion. The feature vector conversion unit 33 may perform dimensional compression by Fisher discriminant analysis (FDA).

When PCA is used as a feature vector conversion method, parameters such as the number of bases (dimension reduction number of feature vectors) and which base to use are used. Note that the feature vector conversion unit 33 may use, as an index, the sum of eigenvalues corresponding to the basis vectors, that is, the cumulative contribution rate, instead of the basis number. The feature vector conversion unit 33 can make these parameters different for each partial region. What parameters are actually set can be determined in advance by learning.
As described above, the feature vector conversion unit 33 stores the data obtained by converting the feature vector in the object dictionary data holding unit 22 as the output of the object dictionary data.
The feature vector conversion data holding unit 34 holds data necessary for the feature vector conversion unit 33 to convert feature vectors. Here, the data necessary for the feature vector conversion is setting information such as the base number (dimension reduction number) as described above.

  FIG. 5 is a flowchart showing an example of processing performed by the feature vector extraction unit 32. Hereinafter, this will be described. First, the feature vector extraction unit 32 acquires partial region setting information from the partial region setting unit 31 (S10). Subsequently, the feature vector extraction unit 32 acquires image data of the object from the image recording unit 4 (S11). The feature vector extraction unit 32 determines the protrusion of the partial area based on the information of the partial area acquired in (S10) from the acquired image data (S13). Here, the protrusion indicates a state in which the partial area acquired in (S10) is out of the effective range of the image acquired in (S11). For example, when the image data acquired from the image recording unit 4 is an object region cut out based on object detection information, a margin of a fixed value may be added to the image data for subsequent processing. If the partial area is set near the boundary of the object, a fixed value is always acquired as a feature vector in the above case, which has a negative effect on the correlation calculation with the subsequent identification data. there is a possibility.

When there is no protrusion (No in S13), the feature vector extraction unit 32 acquires a feature vector from the image data (S14). Here, the feature vector may be a vector obtained by vectorizing luminance data acquired from image data of a partial region. The feature vector may be converted into a feature vector of a corresponding partial area from a predetermined transformation such as LBP (Local Binary Pattern) transformation. If there is a protrusion (Yes in S13), the feature vector extraction unit 32 once acquires a feature vector from a region other than the protruding region (S15). Next, the feature vector extraction unit 32 performs a feature vector addition process for the protruding portion (S16). Since the feature vector acquired in (S15) has a narrow area, the dimension is smaller than the feature vector acquired in (S14). When adding this small amount, the feature vector extraction unit 32 generates a random number instead of a fixed value, and sets the random value as the feature vector of the protruding portion.
By setting a random number as a feature vector, it is possible to avoid setting a fixed value as a feature vector as described above, and to reduce unfavorable influence on correlation calculation with subsequent identification data.
The above is the description of one processing example performed by the feature vector extraction unit.

<Object identification part>
The object identification process will be described. FIG. 6 is a diagram illustrating an example of the object identification unit 6. As shown in FIG. 6, the object identification unit 6 includes an object identification data generation unit 41, an object dictionary data acquisition unit 42, and an object identification calculation unit 43.
The object identification data generation unit 41 extracts information necessary for object identification from the image data acquired from the image recording unit 4.
The object dictionary data acquisition unit 42 acquires dictionary data necessary for object identification from the object registration unit 5.
The object identification calculation unit 43 performs an object identification process from the identification data acquired from the object identification data generation unit 41 and the dictionary data acquired from the object dictionary data acquisition unit 42. The processing performed here will be described in detail later.

FIG. 7 is a flowchart illustrating an example of identification processing performed by the object identification unit 6. First, the object identification unit 6 acquires object dictionary data from the object registration unit 5 (S20). Next, the object identification unit 6 acquires object image data from the image recording unit 4 (S21). Subsequently, the object identification unit 6 performs an object identification data generation process (S22). The processing performed here will be described in detail later. Next, the object identification unit 6 performs an object identification calculation process (S23). As an output of the object identification calculation process, there are a case where a match with registered data (dictionary data) is output in binary (0 or 1) and a case where a normalized output value (a real value of 0 to 1) is output. Furthermore, when there are a plurality of registered objects (registrants), an output value may be output to each registered object (registrant), but only the registered data that is the best match is output. May be. Note that more specific contents of the object identification calculation process will be described later in detail.
The above is the description of the processing flow example in the object identification unit 6.

<Object identification data generator>
FIG. 8 is a diagram illustrating an example of the configuration of the object identification data generation unit 41. As shown in FIG. 8, the object identification data generation unit 41 includes a partial region setting unit 51, a feature vector extraction unit 52, a feature vector conversion unit 53, and a feature vector conversion data holding unit 54. The configuration of the object identification data generation unit 41 and the processing performed there are almost the same as those in the object dictionary data generation unit 21, and therefore the details are omitted.

<Object identification calculation processing>
The object identification calculation process will be described. Here, as an example, a case where a two-class problem of intra-class and extra-class is determined using an SVM classifier will be described. FIG. 9 is a diagram illustrating an example of the configuration of the object identification calculation unit 43. The object identification calculation unit 43 includes an object identification data acquisition unit 61, an object dictionary data acquisition unit 62, a variation feature extraction unit 63, an SVM classifier 64, an identification result holding unit 65, and an identification result integration unit 66.

FIG. 10 is a flowchart illustrating an example of the object identification calculation process. This will be described below with reference to this figure.
First, the object identification data acquisition unit 61 acquires object identification data (S30). Subsequently, the object dictionary data acquisition unit 62 acquires object dictionary data (S31). Next, the variation feature extraction unit 63 performs variation feature extraction processing from the object identification data and the object dictionary data acquired in (S30) and (S31) (S32). Here, the variation feature is a feature belonging to either variation between the same object or variation between different objects, typically extracted from two images. There are various definitions of the variation feature. Here, as an example, the variation feature extraction unit 63 calculates similarity (correlation value, inner product) between feature vectors corresponding to the same region using dictionary data and identification data, and uses the similarity as a component. Let the vector be a variable feature vector. According to the above definition, the number of dimensions of the variation feature vector matches the number of partial regions.

  The variation feature extraction unit 63 inputs the variation feature vector acquired in (S32) into the support vector machine (SVM) discriminator 64 (S33). The SVM discriminator 64 is trained in advance as a discriminator that discriminates two classes of intra-class variation between the same object and extra-class variation between different objects. In general, when the number of partial regions is increased, the number of dimensions of the variation feature vector is increased accordingly, and the calculation time is increased. For this reason, when priority is given to processing time, a cascade connection type SVM discriminator is effective. In this case, the SVM discriminator is configured by training for each specific partial area. The variation feature extraction unit 63 divides the variation feature vector for each partial region and inputs it to the corresponding SVM classifier. By doing in this way, calculation time can be reduced. Rather than learning an SVM classifier corresponding to only one partial area, a combination of two or more partial areas may be learned as an input of the SVM classifier.

  On the other hand, when importance is attached to the identification accuracy, the SVM classifiers are preferably operated in parallel, and a weighted sum is calculated for the calculation result. Even in this case, it is possible to reduce the calculation time to some extent by applying an algorithm for reducing the number of support vectors. A method for reducing the number of support vectors is described in “Burges, CJC (1996).” Simplified support vector decision rules. A well-known technique as described in "International Conference on Machine Learning (pp.71-77)." Can be used.

The SVM discriminator 64 holds the discrimination result between the dictionary data and the object discrimination data calculated in (S33) in the discrimination result holding unit 65 (S34). Next, for example, the SVM discriminator 64 determines whether or not the discrimination calculation has been completed for all dictionary data (S35). If there is still dictionary data (No in S35), the process returns to (S31). When the identification calculation is completed for all dictionary data (Yes in S35), the identification result integration unit 66 performs the identification result integration process (S36). For example, in the simplest case, when the SVM classifier is a classifier that outputs a regression value, the identification result integration unit 66 performs a process of outputting the dictionary data having the highest value as the identification result. Further, the identification result integration unit 66 may output the results of the higher level object having a high degree of matching as a list.
The above is the description of the object identification calculation process.

<Learning method of partial area>
Next, a procedure when AdaBoost is used for learning the position and range of the partial area will be described.
FIG. 11 is a flowchart illustrating an example of the learning process of the partial area. First, the object identification device 100 acquires learning data (S40). When dealing with a human face, a large number of images including a face with a label representing an individual identifier are prepared as learning data. At this time, it is desirable that a sufficient number of images per person is prepared. In order to learn a method of converting a partial region and a feature vector that are robust against illumination fluctuations and facial expression fluctuations, it is important to prepare a sample that sufficiently includes the fluctuations in the learning data. Two types of data can be generated from a labeled face image: data representing the variation of an individual's face and data representing the variation of another person's face. Next, the object identification device 100 performs weak hypothesis selection processing (S41). Here, the weak hypothesis typically includes a process of calculating the similarity between the partial areas of the registration data and the identification data. The object identification device 100 prepares as many weak hypotheses as the number of combinations of positions and ranges of partial areas. Then, the object identification device 100 selects the weak hypothesis with the best performance, that is, the partial region having the optimum position and range, based on the AdaBoost framework for the learning data acquired in (S40) (S42). ). A more specific procedure for performing the performance evaluation may be as in the example of the variation feature extraction process described in the description of the object identification calculation unit 43. That is, the object identification device 100 obtains the similarity (inner product) of feature vectors with respect to the learning data, generates a variation feature vector, and inputs it to the SVM classifier. The object identification device 100 determines whether a correct identification result is obtained between persons with the same label (images are different) and between persons with different labels, and obtains a weighted error rate of learning data.

When the weak hypothesis with the best performance is selected, the object identification device 100 updates the weighting of the learning data based on the identification result regarding the learning data of the weak hypothesis (S42). Next, the object identification device 100 determines whether the number of weak hypotheses has reached a predetermined number (S43). If the predetermined number has been reached (Yes in S43), the object identification device 100 ends the learning process. If the predetermined number has not been reached (No in S43), the object identification device 100 selects a new weak hypothesis.
The detailed procedure of learning by AdaBoost, such as the calculation of the weighted error rate and the updating method of the weighting of the learning data, is described in “Viola & Jones (2001)” Rap.
id Object Detection using a Boosted Cascade of Simple Features ", Computer Vision
and Pattern Recognition. Or the like may be adopted as appropriate.

The object identification device 100 may configure the weak hypothesis from a combination of a plurality of partial areas. That is, the object identification device 100 makes the number of partial areas included in one weak hypothesis constant (a predetermined number, for example, five, ten, etc.). In this case, since the number of combinations increases exponentially when the number of partial areas in the one weak hypothesis is increased, the object identification device 100 may be learned with some constraint condition. More specifically, the object identification device 100 may refer to the positional relationship of the partial areas so that the partial areas at positions close to each other are not included.
Further, the object identification device 100 may apply an optimization method such as a genetic algorithm (GA) when creating a combination of a plurality of partial regions. In this case, the object identification apparatus 100 does not prepare all weak hypothesis candidates in advance before entering the AdaBoost procedure, but dynamically builds candidates even if a weak hypothesis is selected. In other words, the object identification device 100 selects one with good performance from weak hypothesis candidates prepared in advance (for example, randomly generated by combining region candidates). Then, the object identification device 100 generates new weak hypothesis candidates while combining those having good performance, and evaluates the performance. In this way, the weak hypothesis candidates can be narrowed down efficiently. As described above, it is preferable to suppress an increase in learning time.
The above is the description of the procedure for learning the position and range of the partial region.

<< Embodiment 2 >>
The second embodiment differs from the first embodiment in the processing contents of the object registration unit and the object identification unit.
More specifically, the attribute of the object is not considered in the first embodiment, whereas the attribute of the object is estimated in the second embodiment, and a partial area is set according to the attribute of the object. .
More specific description will be given below. In addition, in order to avoid duplication, the same part as previous embodiment is abbreviate | omitted in the following description. FIG. 12 is a diagram (part 2) illustrating an example of a hardware configuration of the object identification device 100. The basic function of each part is the same as that of the first embodiment, but the following points are different. That is, an object dictionary data input unit 109 and an object dictionary data rewrite unit 110 are added to the object identification device 100.
The object dictionary data input unit 109 executes processing for inputting object dictionary data from the outside, and typically reads the object dictionary data from an external storage device such as a semiconductor memory. The object data rewriting unit 110 rewrites the object dictionary data and the object identification data according to a predetermined procedure. The object data rewriting unit 110 typically performs contrast correction, noise removal, resolution change, and the like when the object data is an image.
For convenience of explanation, the object to be identified is the face of a person in the image, but this embodiment is applicable to objects other than the face of a person.

<Object registration part>
FIG. 13 is a diagram illustrating an example of the configuration of the object registration unit 105. The object registration unit 105 includes an object dictionary data generation unit 111, an object dictionary data holding unit 112, an object dictionary data selection unit 113, and an object attribute estimation unit 114. The difference from the first embodiment is that an object attribute estimation unit 114 is added.
The object attribute estimation unit 114 performs processing for estimating the object attribute from the image data input from the image recording unit 104. Specific attributes for estimation include the size, posture / orientation, lighting conditions, and the like of the object. When the object is a human face, the object attribute estimation unit 114 detects the organ position of the face. More specifically, the object attribute estimation unit 114 detects end points of components such as eyes, mouth, and nose. As an algorithm for detecting an end point, for example, a method using a convolutional neural network described in Japanese Patent No. 3078166 can be used. The object attribute estimation unit 114 selects in advance, as the end points, parts that are considered to show personal features such as left and right eyes, both end points of the mouth, and the nose. The object attribute estimation unit 114 detects the positional relationship between the end points of the facial organ as the attribute. As other attributes, the object attribute estimation unit 114 may estimate attributes such as a person's age, sex, and facial expression. A known technique can be used for the attribute estimation. For example, by using a method such as “Japanese Patent Laid-Open No. 2003-242486”, the attribute of a person can be estimated.

By changing the learning data, not only a person but also a general object can be detected by the above method. Accordingly, it is possible to detect elements other than the facial organs on the face of the person, such as glasses, masks, hands, and other objects that are occluded. It can be considered as a face of a person who has the occlusion as described above.
The object attribute estimation unit 114 may use camera parameters, which are examples of imaging parameters, for attribute estimation. For example, the object attribute estimation unit 114 can accurately estimate attributes such as illumination conditions by acquiring control-related AE and AF parameters from the imaging control unit 103. Here, more specific examples of the camera parameters include exposure conditions, white balance, focus, object size, and the like. For example, the object attribute estimation unit 114 creates in advance a correspondence table of exposure conditions, white balance, and color components corresponding to the skin color component area, and stores the correspondence table as a lookup table. Color attributes can be estimated.

Further, the object attribute estimation unit 114 can measure the distance to the object that is the subject by using a distance measuring means such as AF, and can estimate the size of the object. More specifically, the object attribute estimation unit 114 can estimate the size of the object according to the following equation.
s = (f / d−f) · S
Here, s is the size (number of pixels) of the object on the image. f is a focal length. d is the distance from the device to the object. S is the actual size of the object. However, it is assumed that (d> f).

In this manner, the object attribute estimation unit 114 can estimate the size of an object that is not affected by the shooting conditions as an attribute.
The object attribute information estimated by the object attribute estimation unit 114 is stored in the object dictionary data holding unit 112 together with the object dictionary data output from the object dictionary data generation unit 111.
The processing in the object dictionary data generation unit 111 is also partly different from the above embodiment. FIG. 14 is a flowchart illustrating an example of processing in the feature vector extraction unit in the object dictionary data generation unit 111. Hereinafter, this will be described. First, the object dictionary data generation unit 111 acquires object attribute information from the object attribute estimation unit 114 (S100). The attribute information of the object to be acquired is typically the organ position of the person's face and its end point. Next, the object dictionary data generation unit 111 acquires image data from the image recording unit 104 (S101). The object dictionary data generation unit 111 sets a partial region for the object image data using the object attribute information acquired in (S100) (S102).

  More specifically, the partial area is set as follows. First, the object dictionary data generation unit 111 sets a predetermined one of the plurality of end points of the facial organ acquired as attribute information as a reference point. Further, the object dictionary data generation unit 111 measures a distance between at least two other end points, and sets a partial region at a distance from the reference point by a length that is a predetermined multiple of the distance between the two points. The object dictionary data generation unit 111 uses a predetermined value for the direction from the reference point. Here, the end point serving as the reference point, the two points serving as the reference for the distance, the constant for determining how many times the distance between the two points, the direction from the reference point, and the like can be determined in advance by learning. The learning of these parameters can be realized by the partial region selection method using AdaBoost described in the first embodiment by including them in the partial region parameters. Since the processing after the partial area is set is the same as the processing of the feature vector extraction unit in the first embodiment, the description is omitted.

  When using the method for setting the partial area as described above, that is, the method based on the end points of the facial organ, it is particularly effective to make the feature vector of the protruded area by making it an uncorrelated data. is there. In general, it is difficult to detect the end points of a facial organ with high accuracy when photographing conditions such as oblique light and backlight are severe, or for a human face that has been rotated in the depth direction. If the end point is erroneously detected, it is possible that the setting of the partial area is also incorrect, and as a result, the partial area is set outside the range of the image. If the feature vector of the protruded area is set to a fixed value, if both the registered dictionary data and the identification data are protruded, the degree of similarity of the portion increases, which adversely affects the identification result. The object dictionary data generation unit 111 and the object identification data generation unit 121 set a non-correlated value between the object dictionary data and the object identification data, more specifically, a random number in the protruding area. . As a result, the above problems can be avoided.

The object attribute estimation unit 114 can also use occlusion information as the attribute of the object. The object dictionary data generation unit 111 replaces the range of the object with uncorrelated data, more specifically, a random number when there is another object such as glasses or a mask in the range that should be set in the partial area. End up. By doing so, for example, it is possible to avoid a situation in which the similarity between mouth areas of other people who happen to be masked becomes extremely large.
The object attribute estimation unit 114 can also use a facial expression as an attribute. For example, wrinkles are likely to appear on the cheek of a laughed face, but this may have an adverse effect on identification, so the object dictionary data generation unit 111 has replaced the data with uncorrelated data as described above. Also good. The relationship between the facial expression and the partial area to be replaced with the uncorrelated data may be prepared by creating a lookup table determined in advance by a learning sample, and the object dictionary data generation unit 111 may refer to this.
Similarly, by using the age of the person, the object dictionary data generation unit 111 uses the age of the person as an uncorrelated data so that the identification performance when the time is long between registration and authentication is improved. be able to.
The above is the description of the object registration unit.

<Object identification part>
FIG. 15 is a diagram illustrating an example of the configuration of the object identification unit 106. The object identification unit 106 includes an object identification data generation unit 121, an object identification calculation unit 122, an object dictionary data acquisition unit 123, and an object attribute estimation unit 124. This embodiment is different from the first embodiment in that an object attribute estimation unit 124 is added.
Since the content of the processing of the object attribute estimation unit 124 is the same as that of the object attribute estimation unit 114 of the object registration unit, description thereof is omitted.
The object identification data generation unit 121 uses the input from the image recording unit 104 and the output of the object attribute estimation unit 114 to perform a feature vector and its conversion process. Since this process is almost the same as the process of the object registration unit, the description is omitted.
The object identification calculation unit 122 performs object identification processing based on inputs from the object identification data generation unit 121 and the object dictionary data acquisition unit 123. More specific contents of the processing performed by the object identification calculation unit 122 will be described later.
The object dictionary data acquisition unit 123 acquires object dictionary data from the object dictionary data holding unit 112 in the object registration unit 105 based on a request from the object identification calculation unit 122.

<Object identification calculation processing>
Next, the contents of the object identification calculation process will be described.
The overall process of the object identification process is almost the same as in the first embodiment.
Hereinafter, a case will be described in which object identification processing is performed using an object classifier in which a large number of classifiers (hereinafter referred to as weak classifiers) are configured in a tree shape as object classifiers. Typically, the weak classifier corresponds to one partial area, but the weak classifier may correspond to a plurality of partial areas.
FIG. 16 is a schematic diagram when the object classifier is configured with a tree structure of weak classifiers. One frame in the figure represents one weak classifier. Hereinafter, each weak classifier having a tree structure may be referred to as a node classifier. At the time of identification, processing is performed along the direction of the arrow. That is, processing is performed from the weak classifier at the upper level, and processing is performed at the lower weak classifier as the processing proceeds. In general, weak classifiers at the top have high robustness to fluctuations, but have a high misclassification rate. The weak classifier at the lower level is less robust to fluctuations, while learning is performed so that the classification accuracy when the fluctuation ranges match is higher. Multiple weak classifier series specialized for a specific variation range (face depth direction, facial expression variation, illumination variation, etc.) are prepared and a tree structure is used to secure the corresponding variation range as a whole. . FIG. 16 shows a case where there are five weak classifier sequences. In FIG. 16, finally, five weak classifier sequences are integrated into one node classifier. For example, the final node discriminator may perform processing such as comparing cumulative scores of five sequences and adopting a discrimination result of a sequence having the highest score. Further, instead of being integrated into one identification result and output, the identification result of each series may be output as a vector.

Each weak classifier is a classifier that discriminates two-class problems of intra-class and extra-class, but the node classifier at the base point of the branch determines which weak classifier sequence to proceed to and determines the branch destination I do. Of course, the branch destination may be determined while performing the two-class determination. Further, the processing may be performed for all weak classifier series without determining the branch destination. Further, each node discriminator may determine whether or not to cancel the calculation in addition to the 2-class determination (canceling determination). The abort determination may be performed for each node discriminator alone, or may be performed by performing threshold processing on an accumulation of output values (determination scores) of other node discriminators.
The above is the description of the object identification calculation process.

As described above, according to each embodiment described above, robustness and identification performance can be improved with respect to variations in illumination, size, orientation, and the like.
Further, as described above, the object identification device 100 sets a value that is uncorrelated between the dictionary data of the registered object and the identification data when the setting of the partial area is set outside the range of the image data, for example. Set. As a result, highly accurate identification can be performed.
As described above, when the attribute of the partial area is occlusion, the object identification device 100 sets a value that is uncorrelated between the dictionary data of the registered object and the identification data. Also good. In this way, highly accurate identification can be performed.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

1 imaging optical system, 2 imaging unit, 3 imaging control unit, 4 image seasoning unit, 5 object registration unit, 6 object identification unit, 7 external output unit

Claims (10)

An input means for inputting an input image including an object;
An acquisition unit configured to acquire a registration image including the object,
First partial area setting means for performing affine transformation on the input image and setting a partial area in the converted image;
First determination means for determining whether or not at least a part of the partial area set in the image after the affine transformation protrudes from the area of the input image before the affine transformation;
When it is determined that the partial area does not protrude from the area of the input image, a feature vector is set from the partial area, and when it is determined that the partial area protrudes from the area of the input image , of the partial region, a region for a region that protrudes from the region of the input image, and sets the random number, and a region that does not protrude from the region and the input image extends off the said random number is set First setting means for setting a feature vector from
An identification unit for calculating a correlation between the feature vector of the registered image and the feature vector of the partial area set by the first setting unit, and identifying the object based on the calculated result. Feature object identification device. And further comprising detection means for detecting a specific part of the object in the input image,
It said first setting means, based on the detected position of the said site, the object identification device as claimed in claim 1, characterized in that to set the partial area. And second setting means performs an affine transformation, and sets the partial area in the converted image to the registered image,
Least for the even part of the partial region set in the image after the affine transformation, a second determination means for determining whether protrudes from the area in front of the registered image to the affine transformation,
When it is determined that the partial area does not protrude from the area of the input image, a feature vector is set from the partial area, and it is determined that the partial area of the registered image protrudes from the area of the registered image. in a case where it is, the one of the segment of the registered image, to region protrudes from the region of the registered image, sets a random number, from the region and the input image extends off the said random number is set Second setting means for setting a feature vector from a region including a region that does not protrude,
The identification unit calculates a correlation between feature vectors of the partial areas corresponding to the registered image and the input image, and identifies the object based on the calculated result. Or the object identification device of 2. A dictionary holding means for holding a partial region of the object of the registered image as a dictionary;
4. The object identification device according to claim 1, wherein the identification unit calculates a correlation between the dictionary and the partial area of the input image. 5. The first setting unit sets a random number for the partial area of the input image when the partial area of the input image includes occlusion. The object identification device according to claim 1. The object according to claim 3, wherein the second setting unit sets a random number for the partial area of the registered image when the partial area of the registered image includes occlusion. Identification device. The first setting means sets a random number for the partial region of the input image when the partial region of the input image includes a region that is likely to change. The object identification device according to any one of 6. The said 2nd setting means sets a random number with respect to the said partial area of the said registration image, when the said partial area of the said registration image contains the area | region where change appears easily. The object identification device described. First feature vector extraction means for extracting a feature vector of the partial region of the input image;
Second feature vector extracting means for extracting a feature vector of the partial area of the registered image;
Further comprising
The identification unit is configured to determine the partial area of the registered image and the partial area of the input image based on the feature vector of the partial area of the input image and the feature vector of the partial area of the registered image. The object identification device according to claim 3, wherein the correlation is calculated . An input step for inputting an input image including an object;
An acquisition step of acquiring a registered image including the object;
A first partial area setting step of performing affine transformation on the input image and setting a partial area in the transformed image;
A first determination step of determining whether at least a part of the partial region set in the image after the affine transformation protrudes from the region of the input image before the affine transformation;
When it is determined that the partial area does not protrude from the area of the input image, a feature vector is set from the partial area, and when it is determined that the partial area protrudes from the area of the input image Is a region that includes a region that protrudes from the region of the input image and includes a region that protrudes from the input image and a region that does not protrude from the input image. A first setting step for setting a feature vector from
An identification step of calculating a correlation between the feature vector of the registered image and the feature vector of the partial area set in the first setting step, and identifying the object based on the calculated result;
An object identification method comprising:

Images