JP2012234500A - Pattern identification device, pattern identification method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pattern identification device, a pattern identification method and a program capable of precisely identifying a pattern regardless of variation in entire illumination and partial shading.SOLUTION: The pattern identification device includes: variation image generating means that generates a variation input image in which a predetermined variation is made and a variation registration image; partial feature extraction means that extracts an input partial feature from the variation input image and extracts a registration partial feature corresponding to the input partial feature of the variation registration image; partial feature similarity degree calculating means that calculates the degree of similarity of a partial feature between the input partial feature and the registration partial feature for each predetermined combination of variation images; partial feature similarity degree integrating means that calculates an integrated partial feature degree similarity in which at least one or more partial feature similarities calculated by the partial feature similarity calculating means are integrated based on a predetermined standard; and identification similarity degree calculating means that integrates the integrated partial feature similarities based on a predetermined standard for each partial area and calculates identification similarity degree between the input image and the registered image.

Description

  The present invention relates to a pattern identification device and the like that can perform identification with high accuracy and efficiency even when a pattern to be identified is affected by a difference in illumination conditions.

  As an identification technique in pattern recognition, typically, a technique for identifying that an object that is a subject in image data is the same as an object that is a subject in another image, for example, a face that identifies an individual's face There is identification technology. Hereinafter, pattern identification means determining differences between individual patterns (for example, differences between individuals as individuals). On the other hand, the detection of the pattern means that an object that falls within the same category is determined without distinguishing between individuals (for example, a face is detected without distinguishing between individuals).

As a face identification technique, for example, there is a method as described in Non-Patent Document 1.
In such an apparatus and method for identifying a pattern (specifically, a human face), a difference in illumination conditions between the input pattern and the registered pattern can be cited as a factor that degrades the identification performance. Different lighting conditions change the brightness of the entire face and how the shadow of the face changes, which greatly reduces the identification performance.

  In patent document 1, it respond | corresponds to illumination fluctuation | variation by providing an explicit illumination condition detection means. In other words, a deviation in lighting conditions between the input image and the registered image is detected, pre-processing corresponding to the amount of deviation is performed on the input image and / or the registered image, and face authentication is performed after both lighting conditions are brought close to each other. By doing so, face recognition that is robust to lighting conditions is performed.

  In Patent Document 2, a partial space for absorbing the difference in illumination conditions is prepared in advance, and the pattern of the input image and the registered image is projected onto the partial space at the time of identification, and then face authentication is performed. It is carried out. This makes face recognition robust to lighting conditions.

JP 2006-53774 A JP 2007-79710 A

"Learning Patch Correspondences for Improved Viewpoint Invariant Face Recognition", A. B. Ashraf, S. Lucey, T. Chen, Carnegie Mellon University, IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), June, 2008.

  However, these methods are considered to have practical problems. For example, when an explicit illumination condition detection unit is provided as in Patent Document 1, not only is the amount of calculation large, but the illumination condition detection unit prepares and learns a large number of face images in advance. I have to keep it. Similarly, as in Patent Document 2, a method using a partial space similarly learns a partial space that absorbs illumination fluctuations in advance by preparing a large amount of face images and the like, and stores the base data of the space. Must be. This becomes a practical obstacle.

  In addition, it is considered that these methods alone cannot sufficiently cope with illumination fluctuation in the real environment. For example, in the case of illumination variation that causes partial shading rather than illumination variation over the entire face, it is difficult to cope with only the above-described method that does not consider the partial region of the image.

  The present invention relates to a pattern identification device for identifying a category to which an input image pattern belongs, a variation image generating means for generating a variation input image and a variation registration image to which a predetermined variation is added, and the variation input image. A partial feature extraction unit that extracts an input partial feature and extracts a registered partial feature corresponding to the input partial feature of the variation registration image; and the similarity between the input partial feature and the partial feature of the registered partial feature A partial feature similarity calculating unit that calculates each predetermined combination of images, and an integrated partial feature similarity obtained by integrating at least one of the partial feature similarities calculated by the partial feature similarity calculating unit according to a predetermined criterion A partial feature similarity integrating unit for calculating a degree, and the integrated partial feature similarity is integrated for each partial region according to a predetermined standard, and an identification class between the input image and the registered image Identification similarity calculation means for calculating a degree, and having a.

  According to the present invention, by performing predetermined pre-processing on the input image and the registered image for each partial region, it is possible to perform the overall illumination variation and the detection without requiring explicit illumination condition detection or prior partial space learning. It is possible to perform pattern identification with high accuracy against partial variations in shadows.

It is a block diagram which shows the structure of the pattern identification apparatus of this embodiment. It is a flowchart which shows the process of the pattern identification device of this embodiment. It is a block diagram which shows an example of a structure of a pattern registration part. It is a block diagram which shows an example of a structure of a registration pattern dictionary data generation part. It is a schematic diagram for demonstrating the process of a registration pattern dictionary data generation part. It is a flowchart which shows the process of the fluctuation | variation image generation part and the parameter holding | maintenance part for image conversion. It is a flowchart which shows the process of a feature vector extraction part and a feature vector conversion part. It is a block diagram which shows an example of a structure of an input pattern identification part. It is a flowchart which shows an input pattern identification process. It is a block diagram which shows an example of a structure of the data generation part for input pattern identification. It is a block diagram which shows an example of a structure of an input pattern identification calculating part. It is a flowchart which shows an input pattern identification calculation process. It is a flowchart which shows the process of a partial feature similarity integration part. It is a block diagram which shows an example of a structure of the registration pattern dictionary data generation part of multiple features. It is a schematic diagram for demonstrating the process of the registration pattern dictionary data generation part of multiple features. It is a flowchart which shows a multiple feature vector extraction process. It is a block diagram which shows an example of a structure of the input pattern production | generation part of multiple features. It is a block diagram which shows an example of a structure of the input pattern identification calculating part of multiple features. It is a flowchart which shows the process of the input pattern identification calculating part of multiple features.

Hereinafter, a pattern identification device according to the present embodiment will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a block diagram showing a configuration of a pattern identification apparatus 100 according to this embodiment. The pattern identification device 100 includes an imaging optical system 1, an imaging unit 2, an imaging control unit 3, an image recording unit 4, a pattern registration unit 5, an input pattern identification unit 6, an external output unit 7 that outputs a pattern identification result, and each configuration It includes a connection bus 8 for controlling elements and connecting data. The pattern registration unit 5 and the input pattern identification unit 6 may typically be a dedicated circuit (ASIC) and a processor (reconfigurable processor, DSP, CPU, etc.), respectively. Further, it may exist as a program executed in a single dedicated circuit and general-purpose circuit (PC CPU).

The imaging optical system 1 includes an optical lens having a zoom mechanism. Further, the imaging optical system 1 may include a driving mechanism in the pan / tilt axis direction.
As the image sensor of the imaging unit 2, a CCD or a CMOS image sensor is typically used. The imaging unit 2 outputs a predetermined video signal (for example, a signal obtained by subsampling or block reading) as image data in response to a read control signal from a sensor drive circuit (not shown).
The imaging control unit 3 controls the timing of actual shooting based on an instruction from the photographer (viewing angle adjustment instruction, shutter pressing, etc.) and information from the pattern registration unit 5 or the input pattern identification unit 6.

The image recording unit 4 is configured by a semiconductor memory or the like, holds the image data transferred from the imaging unit 2, and transfers the image data at a predetermined timing in response to a request from the pattern registration unit 5 and the input pattern identification unit 6. To do.
The pattern registration unit 5 extracts information on an object to be identified from the image data, and records and holds it. A more detailed configuration of the pattern registration unit 5 and specific contents of the processing actually performed will be described later.
The input pattern identification unit 6 identifies a pattern, more typically an object in the image data, based on the image data and the data acquired from the pattern registration unit 5. With regard to the input pattern identification unit 6, a specific configuration and details of processing performed will be described later.

The external output unit 7 is typically a monitor such as a CRT or a TFT liquid crystal, and displays the image data acquired from the imaging unit 2 and the image recording unit 4 or identifies the pattern registration unit 5 and the input pattern in the image data. The result output of the unit 6 is displayed in a superimposed manner. Further, the results output from the pattern registration unit 5 and the input pattern identification unit 6 may be output as electronic data to an external memory or the like.
The connection bus 8 is a bus for performing control and data connection between the above components.

<Overall flow>
FIG. 2 is a flowchart showing an example of the overall processing of the pattern identification apparatus according to the present embodiment. With reference to FIG. 2, an actual process in which the pattern identifying apparatus 100 identifies an input pattern will be described. In the following, a case where the pattern to be identified is a human face will be described, but the subject of the present invention is not limited to this.
First, the input pattern identification unit 6 acquires image data from the image recording unit 4 (S1). Subsequently, the input pattern identification unit 6 performs a human face detection process on the acquired image data (S02). A known technique may be used as a method for detecting a human face from an image. For example, a technique proposed in “Patent 3078166” or “JP 2002-8032” can be used.

After performing the process of detecting the face of the person who is the target object, the input pattern identification unit 6 determines whether or not a person's face exists in the image (S3). If it exists (Yes in S3), the input pattern identification unit 6 performs an input pattern identification process, that is, an individual identification process (S4). If no human face is present in the image (No in S3), the input pattern identification unit 6 ends the process. Specific processing contents of the input pattern identification processing (S4) will be described later.
The input pattern identification unit 6 determines whether there is a face corresponding to the registered person from the result of the input pattern identification process (S5). The input pattern identification unit 6 determines whether the same person as the face detected in S1 is among the registered persons (S4). If there is (Yes in S5), the process proceeds to S8. . If the detected face does not match any registered person (No in S5), the input pattern identification unit 6 determines whether to register the person (S6). Although this may be set in advance, for example, the user may determine whether to register on the spot through an external interface or a GUI.

If it is determined to be registered (Yes in S6), the input pattern identification unit 6 performs a pattern (person face) registration process described later (S7). When registration is not performed (No in S6), the process is continued as it is.
After the pattern registration process (S7) and when registration is not performed in S6, the input pattern identification unit 6 determines whether or not the process has been completed for all detected patterns (S8). If there is an unprocessed pattern (No in S8), the process returns to S4. When the processing is completed for all the detected patterns (Yes in S8), the input pattern identification unit 6 outputs a series of input pattern identification processing results to the external output unit 7.
The above is the overall processing flow of the pattern identification apparatus 100 according to the present embodiment.

<Pattern registration part>
The pattern registration process will be described. FIG. 3 is a block diagram illustrating an example of the configuration of the pattern registration unit 5. As shown in FIG. 3, the pattern registration unit 5 includes a registration pattern dictionary data generation unit 21, a registration pattern dictionary data holding unit 22, and a registration pattern dictionary data selection unit 23.

  The registered pattern dictionary data generation unit 21 generates registered pattern dictionary data necessary for identifying an individual pattern from the image data acquired from the image recording unit 4. Here, in order to determine whether the face image is in the same category or a different category, data generated from the person's face image is registered as dictionary data. Although registration with a plurality of images of the same person is conceivable, in this embodiment, one image is used per person. Accordingly, registered pattern dictionary data is generated for the number of registered persons. When registering an image, the face image data of the person detected by the face detection process is normalized in size and orientation (in-plane rotation direction) and then stored in the registered pattern dictionary data holding unit 22. Also good. It is possible to reduce the amount of dictionary data by holding only the data necessary for identification, not the image data itself. When performing the discrimination calculation by taking the vector correlation of the partial area of the pattern, it is sufficient to cut out only the partial area in advance.

  As described above, the registered pattern dictionary data generation unit 21 appropriately stores necessary information from the image, performs predetermined conversion described later, and then stores the registered pattern dictionary data as a feature vector for pattern identification. Stored in the unit 22. Details of specific processing performed by the registered pattern dictionary data generation unit 21 will be described later. The registered pattern dictionary data selecting unit 23 reads necessary registered pattern dictionary data from the registered pattern dictionary data holding unit 22 in response to a request from the input pattern identifying unit 6 described later, and stores the registered pattern dictionary data in the input pattern identifying unit 6. Forward.

<Registered pattern dictionary data generation unit>
FIG. 4 is a block diagram illustrating an example of the configuration of the registered pattern dictionary data generation unit 21. As shown in FIG. 4, the registered pattern dictionary data generation unit 21 includes a variation image generation unit 30, an image conversion parameter holding unit 31, a partial region setting unit 32, a feature vector extraction unit 33, a feature vector conversion unit 34, a feature vector. The conversion data holding unit 35 is included. FIG. 5 shows a schematic diagram of processing in the registered pattern dictionary data generation unit 21, which is also used as an aid for explanation.

  The variation image generation unit 30 acquires a conversion parameter from the image conversion parameter holding unit 31 for the image data, performs image conversion, and generates a variation registration image (image acquisition and variation image generation in FIG. 5). The conversion parameters used here are not only one type, but a plurality of types are prepared, and a plurality of types of variable registration images are generated. Details of this processing will be described later. The partial area setting unit 32 sets the position and range from which the feature vector extraction unit 33 extracts feature vectors for the image data and the fluctuation image data (partial area setting in FIG. 5).

  When the identification target is a human face, the position and range of the partial area may be determined in advance by using a machine learning method for the area based on the facial organ position. For example, a plurality of partial area candidates may be set and selected from the plurality of candidates using AdaBoost. Further, as will be described later, a large number of combinations of the partial areas may be prepared, and the combinations may be selected as one candidate by AdaBoost. A method of determining a partial region and a combination thereof by actually applying AdaBoost will be described in detail in the description of the input pattern identification unit 6 described later. The number of partial areas may be determined in advance according to the processing time or the like. In addition, the number of learning samples prepared in advance may be measured and determined so that sufficient identification performance can be obtained.

  In the present embodiment, the partial image setting unit 32 is arranged after the variation image generating unit 30, but this may be reversed. That is, after the partial area setting unit 32 sets a partial area first, the fluctuation image generation unit 30 may generate a fluctuation image for the partial area.

  The feature vector extraction unit 33 performs partial feature extraction from the variation registration image. In this case, since it is extraction from the registration variation image, it is extraction of the registered partial feature (extraction processing of “feature vector extraction, conversion” in FIG. 5). In this embodiment, an LBP (Local Binary Pattern) feature amount is extracted from a partial region and used as a feature vector. This partial area refers to a rectangular area of a certain size in the image. In addition to this, the brightness value of the image and other feature quantities may be extracted as the feature vector. As described above, since there are a plurality of types of variation images, a feature vector is extracted for each type of variation image. Further, when extracting the feature vector, the feature vector may be extracted from the result of performing some filter operation such as a Gabor filter instead of directly acquiring the luminance value. The contents of the process performed by the feature vector extraction unit 33 will be described later. Further, as in the present embodiment, the feature vector extracted from the original image data (registered image) is converted into the feature vector extracted from the variation image without extracting the feature vector from the variation image. A variation feature generator may be provided. In this way, it is not necessary to generate and store a fluctuation image. However, depending on the type of image conversion performed by the variation image generation unit 30, the conversion performed by the variation feature generator may be non-linear conversion, and thus approximate calculation or the like may be necessary.

The feature vector conversion unit 34 performs predetermined conversion on the feature vector extracted by the feature vector extraction unit 33 (the conversion of the feature vector extraction and conversion in FIG. 5). For the feature vector conversion, for example, dimensional compression by principal component analysis (PCA), dimensional compression by independent component analysis (ICA), or the like is performed. Further, dimensional compression by locality preserving projection (LPP) or local Fisher discriminant analysis (LFDA) may be performed.
When PCA is used as a feature vector conversion method, there are parameters such as the number of bases (the number of feature vector dimensions to be reduced) and which base to use. Instead of the basis number, a sum of eigenvalues corresponding to the basis vector, that is, a cumulative contribution rate may be used as an index. These parameters may be different for each partial region, or may be different for each variation image. What parameters are actually set can be determined in advance by machine learning.

  The feature vector conversion data holding unit 35 holds data necessary for the feature vector conversion unit 34 to convert feature vectors. Here, the data necessary for the feature vector conversion includes setting information such as the number of bases (dimension reduction number) as described above, and numerical data of base vectors obtained in advance by learning.

  FIG. 6 is a flowchart illustrating an example of processing performed by the fluctuation image generation unit 30 and the image conversion parameter holding unit 31. In this flowchart, first, the fluctuation image generation unit 30 acquires input image data (S10). Next, the fluctuation image generation unit 30 acquires the image conversion parameters for the acquired image from the image conversion parameter holding unit 31 (S11). This parameter comprises an image conversion method and its conversion parameters. Subsequently, the fluctuation image generation unit 30 performs image conversion using the acquired image data and image conversion parameters (S12). A conversion formula when an image blur (Gaussian) filter is used as image conversion is shown below.

  x and y represent the x-coordinate and y-coordinate of the pixel position, respectively, and A represents a normalization term for satisfying (Expression 2). σ is a parameter of an image blur (Gaussian) filter. The larger this is, the stronger the degree of extraction of the low frequency component of the luminance, and the smaller the value, the weaker. A variation registration image is generated by such image conversion. In addition to the above processing, a known spatial filter such as an edge enhancement filter or a contrast filter can be used, but other image conversion methods may be used. Further, an original image that has not been subjected to image conversion may also be included as a variation registration image.

  Returning to the flowchart shown in FIG. 6, the fluctuation image generation unit 30 determines whether or not the conversion by the predetermined parameter is completed (S13). If not completed (No in S13), the process returns to S11, and the fluctuation image generation unit 30 performs conversion using the remaining conversion parameters. If it has been completed (Yes in S13), the fluctuation image generation unit 30 holds the fluctuation image data subjected to the image conversion and the parameters used for the image conversion in association with each other (S14), and the processing is completed.

  FIG. 7 is a flowchart illustrating an example of processing performed by the feature vector extraction unit 33, the feature vector conversion unit 34, and the like. First, the partial area setting unit 32 acquires partial area setting information (S20). Subsequently, the partial area setting unit 32 acquires registered variation image data of the target pattern from the image recording unit 4 (S21). The partial region setting unit 32 cuts out the partial region image data from the acquired registered variation image data based on the partial region information acquired in S20 (S22), and the feature vector extraction unit 33 extracts the feature amount from the partial region image data. Obtain (S23). Here, one type of feature amount is extracted.

As described above, the feature vector extraction unit 33 extracts the LBP feature quantity and sets it as the feature vector. Typically, however, the pixel value of the luminance image may be extracted as it is as a feature vector, or other feature quantities. An extraction method may be used. The feature vector conversion unit 34 performs predetermined conversion on the feature vector acquired in S23 according to the setting acquired from the feature vector conversion data holding unit 35 (S24). As described above, typically, dimension reduction by PCA, dimension reduction by ICA, and the like are performed on feature vectors. At this time, a predetermined statistical value, typically an average vector, a maximum value of elements, or the like may be obtained for the acquired feature vector. Further, information on a position cut out from an image may be recorded as a partial feature. As will be described later, in order to compare / evaluate the corresponding partial features between the registered pattern and the input pattern, an identifier indicating the correspondence may be recorded. These pieces of information may be output together with the feature vector as an output of the registered pattern dictionary data generation unit 21. The above is an example of the process performed by the feature vector extraction unit 33, the feature vector conversion unit 34, and the like.
After performing the above processing, setting a partial region, extracting a feature vector, storing the data obtained by converting the feature vector in the registered pattern dictionary data holding unit 22 as an output of the registered pattern dictionary data generating unit 21 To do.

<Input pattern identification unit>
Next, the input pattern identification process will be described. FIG. 8 is a block diagram illustrating an example of the configuration of the input pattern identification unit 6. As shown in FIG. 8, the input pattern identification unit 6 includes an input pattern identification data generation unit 41, a registered pattern dictionary data acquisition unit 42, and an input pattern identification calculation unit 43.
The input pattern identification data generation unit 41 extracts information necessary for identifying the target pattern from the image data acquired from the image recording unit 4.
The registered pattern dictionary data acquisition unit 42 acquires dictionary data necessary for identifying an input pattern from the pattern registration unit 5.
The input pattern identification calculation unit 43 performs input pattern identification processing from the identification data acquired from the input pattern identification data generation unit 41 and the dictionary data acquired from the registered pattern dictionary data acquisition unit 42. The processing performed here will be described later.

  FIG. 9 is a flowchart illustrating an example of identification processing performed by the input pattern identification unit 6. First, the registered pattern dictionary data acquisition unit 42 acquires registered pattern dictionary data from the pattern registration unit 5 (S30). Next, the input pattern identification data generation unit 41 acquires image data of the input pattern from the image recording unit 4 (S31). Subsequently, the input pattern identification calculation unit 43 performs input pattern identification data generation processing (S32). The processing performed here will be described later. Next, the input pattern identification calculation unit 43 performs an input pattern identification calculation process (S33). When outputting a match with registered data (dictionary data) in binary (0 or 1) and outputting a normalized output value as a (0-1 real value) likelihood as the output of the input pattern identification calculation process Can be considered. In addition, when there are a plurality of registration patterns (registrants), the likelihood may be output for each registration pattern (registrant), but only the result for the best matching registration pattern is displayed. It may be output. Also, the likelihood for the category to which the registered pattern belongs may be output instead of the likelihood for the registered pattern. That is, in the case of a person, the likelihood for the person ID (name) is output instead of the result for each registered face image. The specific contents of the input pattern identification calculation process will also be described later. The above is the description of the flowchart showing an example of processing in the input pattern identification unit 6.

<Data generation unit for input pattern identification>
FIG. 10 is a block diagram illustrating an example of the configuration of the input pattern identification data generation unit 41. As illustrated in FIG. 10, the input pattern identification data generation unit 41 includes a variation image generation unit 50, an image conversion parameter holding unit 51, a partial region setting unit 52, a feature vector extraction unit 53, a feature vector conversion unit 54, a feature. A vector conversion data holding unit 55 is included. The configuration of the input pattern identification data generation unit 41 and the processing performed there are almost the same as those of the registered pattern dictionary data generation unit 21, and thus detailed description thereof is omitted. The difference is that an image generated by the variation image generation unit 30 is called a variation input image, and a feature vector extracted by the feature vector extraction unit 53 is called an input partial feature.

<Input pattern identification calculation processing>
The input pattern identification calculation process will be described. Here, as an example, a case will be described in which input pattern identification is determined based on the similarity between corresponding partial features. FIG. 11 is a block diagram illustrating an example of the configuration of the input pattern identification calculation unit 43. The input pattern identification calculation unit 43 includes an input pattern identification data acquisition unit 61, a registered pattern dictionary data acquisition unit 62, a partial feature similarity calculation unit 63, a partial feature similarity integration unit 64, and an identification result calculation unit 65. Has been.

FIG. 12 is a flowchart illustrating an example of the input pattern identification calculation process. Hereinafter, this flowchart will be described.
First, the input pattern identification data acquisition unit 61 acquires input pattern identification data (S40). Next, the registered pattern dictionary data acquisition unit 62 acquires dictionary data of registered patterns (S41). Subsequently, the partial feature similarity calculating unit 63 calculates the partial feature similarity from the feature vectors of the input pattern identification data and the registered pattern dictionary data acquired in S40 and S41 (S42). As described above, this feature vector is a vector obtained by converting the feature amount extracted from the partial region of the fluctuation image. Here, cosine similarity based on the inner product between feature vectors is used as a method for calculating similarity. That is, if the feature vector of the input pattern identification data is V _I , the feature vector of the registered pattern dictionary data is V _R , and the angle formed by V _I and V _R is θ, the cosine similarity S is expressed by the following equation. .

However, V _I · V _R represents the inner product of V _I and V _R , and | V _I | and | V _R | represent the norm of each vector. In this embodiment, this cosine similarity is used, but other methods may be used. Subsequently, the partial feature similarity calculation unit 63 determines whether or not the calculation has been completed for all combinations of the variation images (S43). In this case, in addition to all the combinations of the fluctuation images, a method of calculating only a predetermined combination may be considered. When the calculation for all the combinations is completed (Yes in S43), the partial feature similarity integration unit 64 integrates the partial feature similarities calculated by the number of combinations (S44). Details of the integration method will be described later.

Next, the partial feature similarity integration unit 64 further integrates the partial feature similarity integrated in S44 over the entire partial area, and the identification result calculation unit 65 calculates the identification similarity of the input pattern and the registered pattern ( S45). In the present embodiment, as a method of integration, a method of averaging a predetermined number from the top in descending order of partial feature similarity is used, but other methods may be used.
Finally, the identification result calculation unit 65 determines whether or not the identification calculation has been completed for all dictionary data (S46). If there is registered pattern dictionary data that has not been calculated (No in S46), the process returns to S41, and the registered pattern dictionary data acquisition unit 62 applies to registered pattern dictionary data of a person that has not been processed yet. Process. When the processing of the registered pattern dictionary data for all persons has been completed (Yes in S46), the identification result calculation unit 65 holds the identification similarity (S47). Here, the maximum similarity between the input pattern identification data and each registered pattern dictionary data is stored as the identification result. In addition, a method of holding all similarities in a list is conceivable. The above is the description of the pattern identification calculation process.

<Partial feature similarity integration>
Next, the partial feature similarity integration process will be described. FIG. 13 is a flowchart illustrating an example of the process (S44) performed by the partial feature similarity integration unit 64.
First, the partial feature similarity integration unit 64 acquires partial feature similarity data (S51).
Next, the partial feature similarity integration unit 64 performs a process of integrating a plurality of partial feature similarities into one value (S52). Here, integration is performed based on the maximum similarity criterion. That is, the one with the maximum similarity is selected from the acquired plurality of partial feature similarities, and the partial feature similarity integrating unit 64 outputs it as an integration result. In general, in the process of matching image details such as face authentication, it is unlikely that the similarity between others will increase accidentally. Therefore, as integration processing, by selecting this maximum degree of similarity, after performing conversion that reduces the difference in illumination fluctuation between the input image and the registered image, the person who is really the person is highly similar, If so, it can be expected that a low degree of similarity is obtained.
Finally, the partial feature similarity integration unit 64 holds the integrated partial feature similarity data (S53). The above is the description of the partial similarity integration process.

  As described above, in the present embodiment, feature vectors are extracted for each partial region from a plurality of variation images of the registered image and the input image, and the one having the maximum similarity calculated for the brute force of the plurality of variation images is selected. The similarity for each partial area is integrated. Thereby, it is possible to obtain the similarity between the fluctuation images in which the difference between the illumination conditions is reduced without explicitly calculating the illumination conditions of the registered pattern and the input pattern. In addition, robustness against partial illumination condition differences of images can be expected by processing for each partial region.

(Second Embodiment)
This embodiment differs from the first embodiment in the feature vector extraction unit 33, the input pattern identification data generation unit 41, the input pattern identification calculation unit 43, and the partial feature similarity integration unit 64. Specifically, the feature vector extraction unit 33 of the first embodiment extracts only one type of feature vector, whereas the second embodiment extracts a plurality of types of feature vectors. The point is different. Moreover, the point which performs the input pattern identification calculation according to several types of feature vectors, and partial feature similarity integration also differs from 1st Embodiment.

Hereinafter, this embodiment will be described in detail. In addition, in order to avoid duplication, the same part as 1st Embodiment is abbreviate | omitted in the following description. Since the hardware configuration of the entire pattern identification apparatus 100 according to this embodiment is the same as that of the first embodiment, description thereof is omitted. In the following description, in order to distinguish from the first embodiment, a registration pattern dictionary data generation unit having different processing contents will be referred to as a multi-feature registration pattern dictionary data generation unit 70.
Also in this embodiment, for the sake of convenience, the pattern to be identified is the face of a person in the image, but the identification target may be another object.

<Generation of registered pattern dictionary data for multiple features>
FIG. 14 is a block diagram illustrating an example of the configuration of a registered pattern dictionary data generation unit 70 having a plurality of features. The registered pattern dictionary data generation unit 70 for multiple features includes a variation image generation unit 71, an image conversion parameter holding unit 72, a partial region setting unit 73, a multiple feature vector extraction unit 74, a feature vector conversion unit 75, and feature vector conversion data holding. A portion 76 is included. FIG. 15 is a schematic diagram for explaining the processing in the registered pattern dictionary data generation unit 70 of multiple features, and is also used for explanation. Since the processing in the fluctuation image generation unit 71 is almost the same as that in the first embodiment, it is omitted ("image acquisition" and "fluctuation image generation" shown in FIG. 15).

  FIG. 16 is a flowchart showing processing in the partial region setting unit 73, the multiple feature vector extraction unit 74, and the feature vector conversion unit 75. Hereinafter, this flowchart will be described. First, the partial region setting unit 73 acquires partial region setting information (S60), and acquires variable image data that is a target for extracting feature vectors (S61). The partial region setting unit 73 sets a partial region in the acquired variation image data (partial region setting shown in FIG. 15), and the multiple feature vector extraction unit 74 extracts a plurality of feature vectors from the set partial region ( S62). Here, the LBP feature value, the HOG feature value, and the SIFT feature value used in the first embodiment are extracted as the types of feature vectors (extraction processing of “feature vector extraction / conversion” shown in FIG. 15). Here, HOG stands for Histogram of Oriented Gradients, and SIFT stands for Scale-Invariant Feature Transform. Note that other known methods can be used to extract feature vectors. The feature vector conversion unit 75 converts the extracted plurality of feature vectors using the conversion data acquired from the feature vector conversion data holding unit 76 (S63 and “feature vector extraction and conversion” shown in FIG. 15). Our conversion process). This conversion data is prepared for each type of feature vector to be converted.

Finally, the feature vector conversion unit 75 determines whether or not all predetermined types of feature vectors have been acquired (S64). If there is an unextracted feature vector (No in S64), the process returns to S62. When all the feature vectors have been acquired (Yes in S64), the feature vector conversion unit 75 associates the types of feature vectors and the parameters used for generating the variation image of the extraction source with respect to all the extracted feature vectors. Store and hold (S65).
The above is the description of the registered pattern dictionary data generation unit 70 having a plurality of features.

<Multiple feature input pattern generation>
FIG. 17 is a block diagram illustrating an example of the configuration of the multi-feature input pattern generation unit 80. Since the processing here is almost the same as <Generation of multiple feature registration pattern dictionary data>, detailed description thereof is omitted.

<Multi-feature input pattern identification calculation>
FIG. 18 is a block diagram illustrating an example of the configuration of the multi-feature input pattern identification calculation unit 90. The multi-feature input pattern identification calculation unit 90 includes an input pattern identification data acquisition unit 91, a registered pattern dictionary data acquisition unit 92, a similar partial feature similarity calculation unit 93, a feature amount similarity integration unit 94, and a partial feature similarity. An integration unit 95 and an identification result calculation unit 96 are included. The processes of the input pattern identification data acquisition unit 91 and the registered pattern dictionary data acquisition unit 92 are the same as those in the first embodiment, and thus are omitted.

FIG. 19 is a flowchart illustrating an example of processing of the input pattern identification calculation unit 90 having a plurality of features. Hereinafter, this flowchart will be described.
First, the input pattern identification data acquisition unit 91 acquires input pattern identification data (S70). Next, the registered pattern dictionary data acquisition unit 92 acquires dictionary data of registered patterns (S71). As described above, these data are stored in association with the feature vector and the type of the feature vector, the parameters used for generating the variation image of the extraction source, the extracted partial region information, and the feature vector extracted for each combination thereof. It is. The same type partial feature similarity calculation unit 93 calculates the partial feature similarity (S72). At this time, since the same-type partial feature similarity calculation unit 93 cannot calculate between different types of feature vectors, it calculates only between the same type of feature vectors. Specifically, the same-type partial feature similarity calculating unit 93 first selects a partial area to be processed from the registered pattern dictionary data and the input pattern identification data, and the similarity between the same type of feature vectors in the partial area. Is calculated. Then, this process is performed for all partial areas.

Next, the same-type partial feature similarity calculation unit 93 determines whether or not the calculation has been completed for all predetermined combinations of fluctuation images (S73). When the process is completed (Yes in S73), the process proceeds to S74. When the process is not completed (No in S73), the process returns to S72.
In S72, since the partial feature similarity is calculated by the number of the varying images, the feature-by-feature similarity integration unit 94 performs integration to combine these multiple values into one value (S74). Here, as in the first embodiment, integration is performed by selecting the maximum value of similarity. Since the degree of similarity is calculated by the number of types of feature vectors by the integration, this is further integrated in the next process.

Denoting i an index representing the kind of feature vectors, the number of types of feature vector N, the maximum similarity feature quantity integrated by the selection of the similarity between S _i, S _i is calculated for each feature quantity Because of the similarities, the values cannot be simply integrated. This is because the feature amount has a different tendency depending on the type, such as a feature having a high degree of similarity and a feature having the opposite tendency. Therefore, the partial feature similarity integration unit 95 performs integration using the reciprocal of the sum of the fluctuation amounts when the fluctuation image is generated as a weighting coefficient (S75). The formula for integration is as follows.

However, D _R and D _I are the fluctuation amounts of the fluctuation image from the original image when the fluctuation image is generated in the registered pattern dictionary data and the input pattern identification data, respectively. By integrating such weighting, the following effects can be expected. That is, when the amount of variation when generating a variation image is large, it is considered that the amount of information necessary to identify details such as a human face is lost. The contribution of similarity can be reduced. On the other hand, when the variation amount is small, since the separation from the original image is small, it is considered that the information amount remains, so that the contribution of the similarity between the variation images can be increased. In addition to this integration method, it is simply considered also a method of using ^the sum sigma ^NiSi similarity. As a calculation method of the fluctuation amounts D _R and D _I , the parameter used when generating the fluctuation image, σ in (Expression 1), is used. As another calculation method, a difference average for each pixel may be taken for the fluctuation image and the original image.

  The identification result calculation unit 96 integrates the integrated partial feature similarity calculated by the integration over the entire image to obtain a final identification result (S76). Since this process is the same as that in the first embodiment, a description thereof will be omitted. Finally, the identification result calculation unit 96 determines whether or not the calculation has been completed for all dictionary data (S77). If the calculation has not been completed (No in S77), the process returns to S71 for processing. When the processing is completed (Yes in S77), the identification result calculation unit 96 outputs the identification similarities as many as the number of registered dictionary patterns as a list, as in the first embodiment, A high pattern is output as an identification result, and the result is held (S78). This completes the description of the multi-feature input pattern identification calculation.

  Compared to the first embodiment, the second embodiment extracts a plurality of types of feature vectors instead of a single type from a plurality of variation images, so that differences in lighting conditions that are not robust with a certain feature amount can be obtained. The effect that it can respond with the feature-value of can be expected. In addition, the similarity calculated for each type of feature amount is integrated by weighting the reciprocal of the variation amount between the variation image and its original image, and if the variation amount of the variation image is large, it is calculated from there. If the contribution of similarity is small, and conversely the variation is small, it can be increased. As a result, the influence of the abnormal similarity (for example, the similarity between the fluctuation images of the blur filter applied with extremely high intensity becomes a very high value) calculated by giving an extremely large variation. The effect of not receiving can be expected.

  As mentioned above, although this invention was demonstrated with various embodiment, this invention is not limited only to these embodiment, A change etc. are possible within the scope of the present invention. Each means constituting the pattern identification apparatus and each step of the pattern identification method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable recording medium on which this program is recorded are included in the present invention.

  1: Imaging optical system 2: Imaging unit 3: Imaging control unit 4: Image recording unit 5: Pattern registration unit 6: Input pattern identification unit 7: External output unit 8: Connection bus 21: Registration pattern dictionary generation unit 22: Registration Pattern dictionary data holding unit 23: Registered pattern dictionary data selection unit 30: Fluctuating image generation unit 31: Image conversion parameter holding unit 32: Partial region setting unit 33: Feature vector extraction unit 34: Feature vector conversion unit 35: Feature vector conversion Data storage unit 41: Input pattern identification data generation unit 42: Registered pattern dictionary data acquisition unit 43: Input pattern identification calculation unit 50: Fluctuation image generation unit 51: Image conversion parameter storage unit 52: Partial region setting unit 53: Feature vector extraction unit 54: Feature vector conversion unit 55: Feature vector conversion data holding unit 61: Input parameter Turn identification data acquisition unit 62: Registration pattern dictionary data acquisition unit 63: Partial feature similarity calculation unit 64: Partial feature similarity integration unit 65: Identification result calculation unit 70: Multiple feature registration pattern dictionary data generation unit 71: Variation Image generation unit 72: Image conversion parameter holding unit 73: Partial region setting unit 74: Multiple feature vector extraction unit 75: Feature vector conversion unit 76: Feature vector conversion data storage unit 80: Multiple feature input pattern data generation unit 81 : Fluctuating image generation unit 82: Image conversion parameter holding unit 83: Partial region setting unit 84: Multiple feature vector extraction unit 85: Feature vector conversion unit 86: Feature vector conversion data holding unit 90: Multiple feature input pattern identification calculation unit 91: Input pattern identification data acquisition unit 92: Registered pattern dictionary data acquisition unit 3: homologous partial feature similarity calculation unit 94: feature-amount similarity integration section 95: part feature similarity integration section 96: identification result calculation unit

Claims (6)

A pattern identification device for identifying a category to which an input image pattern belongs,
A fluctuation image generating means for generating a fluctuation input image and a fluctuation registration image to which a predetermined fluctuation is applied;
Partial feature extraction means for extracting an input partial feature from the variable input image and extracting a registered partial feature corresponding to the input partial feature of the variable registered image;
Partial feature similarity calculating means for calculating the similarity between the input partial feature and the partial feature of the registered partial feature for each predetermined combination of the variation images;
Partial feature similarity integration means for calculating an integrated partial feature similarity calculated by the partial feature similarity calculation means and integrating at least one of the partial feature similarities according to a predetermined standard;
An identification similarity calculating unit that integrates the integrated partial feature similarity for each partial area on a predetermined basis and calculates an identification similarity between the input image and the registered image; .   The pattern identification apparatus according to claim 1, wherein the partial feature similarity integration unit sets the maximum value of the partial feature similarity as the integrated partial feature similarity.   The pattern identification apparatus according to claim 1, wherein the partial feature similarity integrating unit integrates the partial feature similarity based on a magnitude of variation applied by the variation image generating unit.   The pattern identification according to claim 1, wherein the predetermined combination of the variation images used when the partial feature similarity calculation unit calculates the partial feature similarity is a round robin of the variation images. apparatus. A pattern identification method for identifying a category to which an input image pattern belongs,
A variation image generation step of generating a variation input image and a variation registration image to which a predetermined variation is added;
A partial feature extraction step of extracting an input partial feature from the variable input image and extracting a registered partial feature corresponding to the input partial feature of the variable registration image;
A partial feature similarity calculating step of calculating the similarity of the partial features of the input partial features and the registered partial features for each predetermined combination of the variation images;
A partial feature similarity integrating step of calculating an integrated partial feature similarity calculated by integrating the at least one or more partial feature similarities calculated in the partial feature similarity calculating step;
An identification similarity calculating step of integrating the integrated partial feature similarity for each partial region on a predetermined basis and calculating an identification similarity between the input image and the registered image . A program for identifying a pattern that identifies a category to which an input image pattern belongs,
A variation image generation step of generating a variation input image and a variation registration image to which a predetermined variation is added;
A partial feature extraction step of extracting an input partial feature from the variable input image and extracting a registered partial feature corresponding to the input partial feature of the variable registration image;
A partial feature similarity calculating step of calculating the similarity of the partial features of the input partial features and the registered partial features for each predetermined combination of the variation images;
A partial feature similarity integrating step of calculating an integrated partial feature similarity calculated by integrating the at least one or more partial feature similarities calculated in the partial feature similarity calculating step;
A program for causing a computer to execute an identification similarity calculation step of integrating the integrated partial feature similarity for each partial area on a predetermined basis and calculating an identification similarity between the input image and the registered image.

Images