JP4803214B2 - Image recognition system, recognition method thereof, and program - Google Patents

Description

  The present invention relates to an image recognition system using an image, a recognition method thereof, and a program, and in particular, identifies whether a target object photographed in an image is an object registered in a dictionary, or includes a plurality of categories registered in a dictionary. The present invention relates to an image recognition system classified into one of the above, a recognition method thereof, and a program.

  An example of a conventional image recognition system is a face image matching device (see, for example, Patent Document 1). This collates human face images, and collation is performed using eigenvectors of Fourier spectrum patterns obtained by Fourier analysis of the entire image.

  Further, in the pattern recognition apparatus and method (for example, see Patent Document 2), collation is performed based on an angle (mutual subspace similarity) between a partial space obtained from a plurality of input images and a partial space stretched by registered images. The method of doing is described. FIG. 29 shows a configuration diagram of an example of a conventional image recognition system. An example of a conventional image recognition system includes an image input unit 210, an angle calculation unit 211 between subspaces, a recognition unit 212, and a dictionary storage unit 213.

  The conventional image recognition system having such a configuration operates as follows. That is, the image input unit 210 inputs a plurality of images taken in a plurality of directions. Next, in the angle calculation part 211 between subspaces, the angle between subspaces is calculated.

  First, an input image group is expressed in an N-dimensional subspace. Specifically, the entire image is regarded as one-dimensional feature data and subjected to principal component analysis, and N eigenvectors are extracted. In the dictionary storage unit 213, dictionary data expressed in advance in an M-dimensional partial space is prepared for each category. The angle calculation unit 211 between subspaces further calculates the angle between the N-dimensional subspace of the input image and the M-dimensional subspace of the dictionary for each category. The recognition unit 212 compares the angles calculated by the angle calculation unit 211 between the subspaces, and outputs a category having the smallest angle as a recognition result.

  When the basis vector of the dictionary subspace is Φm (m = 1,... M) and the basis vector of the input subspace is Ψn (n = 1,... N), xij in the equation (1) or (2) is Calculate the matrix X of the elements.

部分空間間の角度Θの、余弦の二乗は、行列Ｘの最大固有値として求められる。角度が小さいということは、余弦の２乗が大きいことを意味する。すなわち余弦の２乗はパターンの類似度と言い換えることができる。従来の画像認識システムでは行列Ｘの最大固有値を類似度とし、類似度が最大のカテゴリに分類する。

The square of the cosine of the angle Θ between the subspaces is obtained as the maximum eigenvalue of the matrix X. A small angle means that the square of the cosine is large. That is, the square of the cosine can be rephrased as a pattern similarity. In the conventional image recognition system, the maximum eigenvalue of the matrix X is set as the similarity, and the category is classified into the category having the maximum similarity.

  What is common to these conventional image recognition systems is that the similarity calculation or distance calculation at the time of matching is performed only once using the features extracted from the entire image.

  Further, Patent Document 3 discloses an invention related to collation between a two-dimensional image and three-dimensional data. This performs illumination variation and posture variation processing using pre-registered individual three-dimensional data. Examples of other image recognition systems of this type are disclosed in Patent Documents 4 to 9.

Japanese Patent Laid-Open No. 05-020442 JP-A-10-199128 JP 2001-283216 A Japanese Patent Laid-Open No. 03-043877 Japanese Patent Application Laid-Open No. 07-287753 JP 2000-259838 A JP 2001-052182 A JP 2001-236508 A Japanese Patent Laid-Open No. 10-232938

  However, if the target image is partially blackened due to lighting fluctuations, or if occlusion occurs (a part of the target object is hidden behind the object), the feature value acquired from the entire image is abnormal. As a result, there was a problem that it could not be accurately verified.

  On the other hand, the invention disclosed in Patent Document 3 is a well-known example of illumination variation and posture variation processing, but in the present invention, a plurality of variation image groups are generated using standard three-dimensional data from one image. The invention disclosed in Patent Document 3 is a completely different invention from the present invention as a recognition system. Further, none of Patent Documents 4 to 9 discloses the configuration of the present invention.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image recognition system that correctly classifies input images regardless of illumination fluctuations and occlusion conditions.

In order to solve the above problems, an image recognition system according to the present invention provides a plurality of inter-pattern distance values between a plurality of partial areas in an input image and each partial area of a registered image at a position corresponding to each partial area. An image recognition system including an inter-pattern distance calculating means for calculating and an identifying means for identifying the input image based on a plurality of inter-pattern distance values, wherein the inter-pattern distance calculating means is arranged over the entire image The inter-pattern distance value is calculated for the partial area, and the identification unit calculates a single integrated distance value calculated by using only a predetermined number from the calculated inter-pattern distance value in ascending order of the distance value . Based on this , the input image is identified.

Further, the image recognition method according to the present invention is a pattern inter-pattern distance for calculating a plurality of inter-pattern distance values between a plurality of partial areas in an input image and each partial area of a registered image at a position corresponding to each partial area. An image recognition method comprising: a calculation step; and an identification step for identifying the input image based on a plurality of inter-pattern distance values,
In the inter-pattern distance calculating step, the inter-pattern distance value is calculated for the partial region arranged in the entire image, and in the identifying step, the predetermined distance value is determined in ascending order of the distance value from the calculated inter-pattern distance value. The input image is identified based on one integrated distance value calculated using only the number.

Further, the program according to the present invention provides a pattern distance calculation step for calculating a plurality of pattern distance values between a plurality of partial areas in the input image and each partial area of the registered image at a position corresponding to each partial area. And an identification method for identifying the input image based on a plurality of inter-pattern distance values , wherein the image recognition method is arranged over the entire image in the inter-pattern distance calculation step. The inter-pattern distance value is calculated for each partial area,
In the identifying step, the input image is identified based on one integrated distance value calculated using only a predetermined number from the calculated pattern distance values in ascending order of distance values. And

  According to the present invention, it is possible to correctly classify input images regardless of illumination fluctuations and occlusion conditions.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. FIG. 1 is a configuration diagram of a first embodiment of an image recognition system according to the present invention. Referring to the figure, the image recognition system stores collation unit 1 for recognizing an object from an input image, dictionary storage means 5 for storing a dictionary used for collation, and position information for extracting a partial region from the image. The image forming apparatus includes a partial area information storage unit 6 and a registration unit 7 that generates a dictionary from a registered image. An example of the input image is a human face image. In this case, it is desirable that the position and size of the face pattern in the image be constant in advance.

  The input image described here includes not only a natural image obtained by photographing the real world, but also general two-dimensional patterns including those obtained by applying a spatial filter and those generated by computer graphics.

  Next, operation | movement of the collation part 1 is demonstrated. The collation unit 1 includes an input image partial region extraction unit 2, an inter-pattern distance calculation unit 3, a region distance value integration unit 10, and an identification unit 4. For the input image, the input image partial area extraction means 2 sets P (P is an integer of 2 or more) partial areas, and calculates a feature vector based on the pixel values belonging to each partial area. This is called a region-specific input feature vector. As an example of the region-specific input feature vector, a feature vector having pixel values as elements can be cited. Examples of generating other region-specific input feature vectors will be described later.

  When setting the partial area, information on the position, size, and shape of each partial area is acquired from the partial area information storage unit 6.

  Next, extraction of partial areas will be described. FIG. 2 is a conceptual diagram showing a pattern distance calculation method of the image recognition system according to the present invention. Referring to the figure, the input image 300 is equally divided into P rectangular partial areas (see partial area division result 302). The partial area division result 302 is, for example, 20 partial areas of equal horizontal size, 5 horizontal and 4 vertical. The shape of the partial area is not a rectangle, but can be defined by an ellipse or an arbitrary closed curve. Moreover, each size does not need to be uniform. Further, it is possible to define each partial region so as to partially overlap. As in the partial area division result 302, when a partial area is defined in which a rectangle having a uniform size is evenly arranged in the entire image, there is an advantage that the image processing is simplified and speeded up.

  The number of pixels included in the partial area needs to be at least two pixels. This is because the partial area of one pixel is only one pixel, and is equivalent in principle to the conventional method, and the performance cannot be improved. From the experimental results, the number of pixels in the partial area is preferably about 16 pixels or more.

  Next, the operation of the inter-pattern distance calculation unit 3 will be described. The inter-pattern distance calculation means 3 calculates the inter-pattern distance between the registered partial data of each category and the dictionary data using the input characteristic vector for each area obtained by dividing the pattern into partial areas. Dictionary data is read from the dictionary storage means 5. The inter-pattern distance calculation means 3 calculates an inter-pattern distance value dp (p = 1,... P) for every P partial regions.

  As an example of the inter-pattern distance calculating means 3, there is an inter-pattern distance calculating means 40 shown in FIG. Referring to the figure, the inter-pattern distance calculating means 40 includes a norm calculating means 41. The norm distance calculation means 41 calculates the norm of the difference between the input feature vector x (i) and the dictionary feature vector y (i) for each partial region. An example of a norm is the L2 norm. The distance by the L2 norm is the Euclidean distance, and the calculation method is shown in Equation (3). However, the number of dimensions of the vector is n.

ノルムのもう一つの例としてＬ１ノルムがある。Ｌ１ノルムによる距離の算出方法を式（４）に示す。Ｌ１ノルムによる距離は市街地距離と呼ばれる。

Another example of the norm is the L1 norm. A method for calculating the distance based on the L1 norm is shown in Expression (4). The distance by the L1 norm is called city distance.

次に図２を用いてパターン間距離算出手段４０における部分領域ごとの照合について説明する。同図を参照すると、入力画像３００は識別対象として入力された画像である。登録画像３０１はあらかじめ辞書として登録されている画像である。入力画像３００はＰ個の部分領域に分割され、入力画像領域分割結果３０２が得られる。登録画像３０１も同じＰ個の部分領域に分割され、登録画像領域分割結果３０３が得られる。

Next, collation for each partial area in the inter-pattern distance calculation means 40 will be described with reference to FIG. Referring to the figure, an input image 300 is an image input as an identification target. A registered image 301 is an image registered in advance as a dictionary. The input image 300 is divided into P partial areas, and an input image area division result 302 is obtained. The registered image 301 is also divided into the same P partial areas, and a registered image area division result 303 is obtained.

  Image data belonging to each of P partial regions of the input image 300 is extracted as a one-dimensional feature vector and stored as a region-specific input feature vector 309. When attention is paid to partial area A which is one of P, partial area A input image 304 is converted into partial area A input feature vector 307 as a one-dimensional vector.

  Similarly, image data belonging to each of P partial regions of the registered image 301 is extracted as a one-dimensional feature vector and stored as a region-specific dictionary feature vector 310. When attention is paid to partial area A which is one of P, partial area A registered image 305 is converted into partial area A dictionary feature vector 308 as a one-dimensional vector.

  The inter-pattern distance calculation is performed for each partial region. For example, the partial area A input feature data 307 is compared with the partial area A dictionary feature vector 308 to calculate a norm distance. Thus, the norm distance is calculated independently for all P partial regions.

  Another example of the inter-pattern distance calculating unit 3 is the inter-pattern distance calculating unit 50 shown in FIG. Referring to the figure, the inter-pattern distance calculation means 50 includes a partial space projection distance calculation means 51. The partial space projection distance calculation means 51 calculates a partial space projection distance for each partial region.

  Pattern matching based on the subspace projection distance is called a subspace method, for example, Document 1 (Maeda, Murase, “Pattern recognition by kernel nonlinear subspace method”, IEICE Transactions, D-II, Vol. J82). -D-II, No. 4, pp. 600-612, 1999). The partial space projection distance defines a distance value between the partial space spanned by the feature data group registered in the dictionary and the input feature vector. An example of a method for calculating the subspace projection distance will be described. Let X be the input feature vector. Let V be the average vector of the feature vector group registered in the dictionary. The feature vector group registered in the dictionary is subjected to principal component analysis, and a matrix having K eigenvectors having large eigenvalues as columns is denoted by ψi (i = 1,..., K). In the present text, a combination of the average vector V and the matrix Ψi of K eigenvectors is referred to as principal component data. At this time, the subspace projection distance ds is calculated by Expression (5).

次に、パターン間距離算出手段５０における部分領域ごとの照合について説明する。図５は本発明に係る画像認識システムのパターン間距離算出方式を示す概念図である。同図を参照すると、入力画像３４０は識別対象として入力された画像である。登録画像シーケンス３４１は、あらかじめ辞書として登録された、ある１つのカテゴリに属する画像シーケンスである。登録画像シーケンス３４１はＪ個（Ｊは２以上の整数）の登録画像からなる。入力画像３４０はＰ個の部分領域に分割され、入力画像領域分割結果３４２が得られる。登録画像シーケンス３４１も同様にＰ個の部分領域に分割され、登録画像領域分割結果３４３が得られる。

Next, collation for each partial area in the inter-pattern distance calculation means 50 will be described. FIG. 5 is a conceptual diagram showing an inter-pattern distance calculation method of the image recognition system according to the present invention. Referring to the figure, the input image 340 is an image input as an identification target. The registered image sequence 341 is an image sequence that is registered in advance as a dictionary and belongs to a certain category. The registered image sequence 341 includes J registered images (J is an integer of 2 or more). The input image 340 is divided into P partial areas, and an input image area division result 342 is obtained. The registered image sequence 341 is similarly divided into P partial areas, and a registered image area division result 343 is obtained.

  Image data belonging to the P partial regions of the input image 340 is extracted as a one-dimensional feature vector and stored as a region-specific input feature vector 349. When attention is paid to the partial area A which is one of P, the partial area A input image 344 is converted into a partial area A input feature vector 347 as a one-dimensional vector.

  A one-dimensional feature vector is extracted for each partial region from J images of the registered image sequence 341. Principal component data is calculated from the extracted J feature vectors and stored as dictionary-specific dictionary principal component data 350. When attention is paid to the partial area A which is one of the P areas, the partial area A dictionary principal component data 348 is calculated using the partial area A registration sequence 345 which is a feature data string belonging to the partial area A.

  The inter-pattern distance calculation is performed for each partial region. For example, the partial area A input feature data 347 is compared with the partial area A dictionary principal component vector 348 to calculate a partial space projection distance. In this way, the partial space projection distance is calculated independently for all P partial regions.

  Next, the area distance value integration means 10 uses a P distance value for each category to apply a function F (d1, d2, d3... Dp) to calculate one integrated distance value.

  As an example of the distance value integration unit 10, there is an area distance value integration unit 70 of FIG. The area distance value integration unit includes a weighted average value calculation unit 71. When the distance values of the P partial areas are given, the area distance value integration means 70 calculates the weighted average value Dw of the P distance values and outputs it as an integrated distance value. A formula for calculating the weighted average value Dw is shown in Formula (6).

各領域に対応する重み値ｗはあらかじめ決められた値を用いることができるし、領域内の画像データに適当な関数を作用させて求めても良い。

As the weight value w corresponding to each region, a predetermined value can be used, or it may be obtained by applying an appropriate function to the image data in the region.

  Next, as an example of the distance value integration unit 10, there is a region distance value integration unit 80 shown in FIG. The area distance value integration unit includes a distance value sorting unit 82 and a distance value upper average calculation unit 81. When the distance values of the P partial areas are given, the area distance value integration unit 80 sorts the P distance values in ascending order by the distance value sorting unit 82, and the distance value upper average calculation unit 81 decreases the distance value. An average value Dp of P ′ pieces (P ′ is an integer less than P) is calculated and output as an integrated distance value. There is a method of predetermining the value of P ′ according to the value of P. There is also a method of dynamically changing the value of P ′ by defining P ′ as a function of the distance value of each partial region. Further, the value of P ′ can be made variable according to the brightness and contrast of the entire image.

  Further, as an example of the distance value integration unit 10, an average value of P ′ (P ′ is an integer less than P) smaller than a predetermined threshold value among the distance values of the P partial areas is calculated. There is something.

  Next, the advantage of the method of calculating the integrated distance using only the distance values integrating means 80 having a small partial region distance value will be described with reference to FIG. FIG. 8 is a conceptual diagram showing a pattern distance calculation method of the image recognition system according to the present invention. Consider comparing the input image 400 with the registered image 401. The registered image 401 has a shadow on the left side due to the influence of illumination, and the image pattern on the left side is greatly different. When the result of calculating the distance value for each partial area is shown as a gray value, a partial area distance value map 404 is obtained. The black area has a large distance value (low matching score), and the white area has a small distance value (high matching score). The collation score of the partial area where the shadow on the left side of the registered image appears is low. Intuitively, it can be seen that a more accurate collation result can be obtained by ignoring the unclear left partial area and collating only with the right partial area. Therefore, distance value integration means that considers only partial regions with high matching scores (low distance values), such as distance value integration means 80, is effective.

  Next, the identification unit 4 compares the integrated distance value with each category obtained from the distance value integration unit 10 and finally outputs the category to which the input image belongs. As an example of the identification unit 4, there is an identification unit 90 shown in FIG. Referring to the figure, the identification unit 90 includes a minimum value calculation unit 91 and a threshold processing unit 92. First, the minimum value calculation means 91 calculates the minimum value of the integrated distance value with each category. Next, the threshold value processing means 92 compares the minimum value with the threshold value, and if it is smaller than the threshold value, the category from which the minimum value is obtained is output as the identification result. When it is larger than the threshold value, a result that it does not exist in the dictionary is output.

  Next, the operation of the registration unit 7 will be described. The registration unit 7 includes a registered image partial area extraction unit 9 and a dictionary data generation unit 8. A registration image and an ID (identification) of a corresponding category are input to the registration unit 7. This image belongs to the category designated by the category ID. For the registered image, the registered image partial area extraction unit 9 sets P partial areas with reference to the partial area information 6 and generates a region-specific dictionary feature vector based on the pixel values belonging to each partial area. To do. As an example of the region-specific dictionary feature vector, a feature vector having a pixel value as an element can be cited. Examples of generating other region-specific dictionary feature vectors will be described later.

  Then, the dictionary data generation means 8 converts the region-specific dictionary feature vector into an appropriate storage format and outputs it to the dictionary storage means 5. When principal component data is required as a dictionary, principal component analysis of a dictionary feature vector group is performed. An example of the storage format is shown in FIG.

  As an example of the dictionary data generation means 8, there is the dictionary data generation means 190 of FIG. The dictionary data generation unit 190 includes a principal component data generation unit 191. The principal component data generation unit 191 performs principal component analysis of the input plurality of region-specific dictionary feature vectors, and generates region-specific principal component data.

  FIG. 10 shows a configuration diagram of a dictionary storage unit 100 which is an example of the dictionary storage unit 5. FIG. 10A is an overall configuration diagram of the dictionary storage unit 100, and FIG. 10B is a configuration diagram of the record storage unit 101. The dictionary storage means 100 has C record storage units 101, and each record can store a record number 102, area-specific dictionary data 103, and category ID 104. The area-specific dictionary data includes P partial area-specific dictionary data. The dictionary storage means 100 can store a plurality of dictionary records having the same category ID. The specific data of the region-specific dictionary data 103 depends on the distance calculation method of the inter-pattern distance calculation means 3. For example, when the inter-pattern distance calculation means 40 is used, it becomes a one-dimensional feature vector, and the inter-pattern distance calculation means 50. When the inter-pattern distance calculation means 60 is used, the principal component data is an average vector V and K eigenvectors.

  FIG. 11 shows a configuration of a collation unit 21 that is an embodiment for recognizing a target from a plurality of input images such as a video sequence. The input of the present embodiment includes a moving image such as a video image and a plurality of still images obtained by photographing the same object. In addition, the same number is attached | subjected to the component similar to FIG. 1, and the description is abbreviate | omitted. The collation unit 21 averages the input image sequences, generates an input image sequence smoothing means 22, an input image partial area extracting means 2, an inter-pattern distance calculating means 3, and an identifying means 4. It is comprised including. When N input images (N is an integer greater than or equal to 2) are input first, the collating unit 21 takes an average of N images for each pixel and generates one input average image. An operation equivalent to that of the collating unit 1 is performed using the input average image as an input image.

  FIG. 12 shows a configuration of a collation unit 31 that is another embodiment for recognizing a target from a plurality of input images such as a video sequence. The input of the present embodiment includes a moving image such as a video image and a plurality of still images obtained by photographing the same object. In addition, the same number is attached | subjected to the component similar to FIG. 1, and the description is abbreviate | omitted. The collation unit 31 includes an input image partial region extraction unit 2, an input principal component generation unit 32, an inter-pattern distance calculation unit 33, and an identification unit 4. When N input images are input, the collation unit 31 sets P partial areas for each image in the input image partial area extraction unit 2 and extracts image data belonging to each partial area. . When setting the partial area, information on the position, size, and shape of each partial area is acquired from the partial area information storage unit 6. Next, the input principal component generation unit 32 calculates input principal component data for each region, which is input principal component data for each partial region. The inter-pattern distance calculation means 33 calculates the distance value between each category using the obtained P input principal component data and dictionary data. Based on the distance value with each category, the identification unit determines which category the input image sequence belongs to, and outputs a recognition result.

  As an example of the inter-pattern distance calculating means 33, there is an inter-pattern distance calculating means 60 shown in FIG. The inter-pattern distance calculating unit 60 includes a subspace distance calculating unit 61. The inter-subspace distance calculating means 61 receives the area-based input principal component data and the area-specific dictionary principal component data as input, and calculates a distance value for each partial area.

  As a method for realizing the inter-subspace distance calculation means 61, there is a method for obtaining a distance between the subspaces. An example is described below. The dictionary principal component data is composed of a dictionary average vector V1 and K dictionary eigenvectors ψi. The input principal component data is composed of an input average vector V2 and L input eigenvectors Φi. First, the distance value dM1 between the input average vector V2 and the partial space spanned by the dictionary eigenvector is calculated by the equation (7).

次に辞書平均ベクトルＶ１と、入力固有ベクトルで張られる部分空間との距離値ｄＭ２を式（８）によって算出する。

Next, a distance value dM2 between the dictionary average vector V1 and the subspace spanned by the input eigenvector is calculated by the equation (8).

入力と辞書の部分空間同士の距離は、ｄＭ１とｄＭ２の関数Ｇ（ｄＭ１，ｄＭ２）によって算出される。

The distance between the input and dictionary subspaces is calculated by a function G (dM1, dM2) of dM1 and dM2.

  An example of the function G is equation (9).

ただしαは定数である。

Where α is a constant.

  Next, collation for each partial area in the inter-pattern distance calculation means 60 will be described with reference to FIG. FIG. 14 is a conceptual diagram showing an inter-pattern distance calculation method of the image recognition system according to the present invention. Referring to the figure, an input image 320 is an image sequence input as an identification target. The registered image sequence 321 is an image sequence that is registered in advance as a dictionary and belongs to a certain category. The input image sequence includes N registered images, and the registered image sequence 321 includes J registered images. Each image of the input image sequence 320 is divided into P partial areas, and an input image area division result 322 is obtained. The registered image sequence 341 is similarly divided into P partial areas, and a registered image area division result 323 is obtained.

  A one-dimensional feature vector is extracted for each of P partial regions from N images of the input image sequence 320. Principal component data is calculated from the extracted N feature vectors and stored as region-specific input principal component data 329. When attention is paid to the partial area A which is one of the P areas, the partial area A input principal component data 326 is calculated using the partial area A input sequence 324 which is a feature data string belonging to the partial area A.

  On the other hand, a one-dimensional feature vector is extracted for each of P partial regions from J images of the registered image sequence 321. Principal component data is calculated from the extracted J feature vectors and stored as dictionary-specific dictionary principal component data 330. When attention is paid to the partial area A which is one of the P areas, the partial area A dictionary principal component data 327 is calculated using the partial area A registration sequence 325 which is a feature data string belonging to the partial area A.

  The inter-pattern distance calculation is performed for each partial region. For example, the partial area A input principal component data 326 is compared with the partial area A dictionary principal component data 327, and the distance between the partial spaces is calculated by, for example, Expression (9). The inter-subspace distances are calculated independently for all P partial regions.

  Next, the input image partial area extracting unit 2 will be described in detail. As an example of the input image partial area extracting unit 3, there is the input image partial area extracting unit 110 of FIG. The input image partial area extraction unit 110 includes a partial image acquisition unit 111 and a feature extraction unit 112. The partial image acquisition unit 111 refers to the partial area information stored in the partial area information storage unit 6 and acquires the pixel value of the partial image. The feature extraction unit 112 converts the obtained partial image data into a one-dimensional feature vector. As an example of the feature extraction unit 112, there is one that generates a vector whose element is a pixel value of a partial image. As an example of the feature extraction unit 112, there is a feature vector obtained by adding correction processing such as pixel value density normalization, histogram flattening, and filtering to the pixel value of a partial image. Some extract frequency features using Fourier transform, DCT, and wavelet transform. The frequency feature is generally robust to misalignment. The conversion to the frequency feature is a kind of filtering for the pixel value vector.

  As an example of the feature extraction unit 112 that outputs a frequency feature, there is a feature extraction unit 130 shown in FIG. The feature extraction unit 130 includes a Fourier spectrum conversion unit 131. The Fourier spectrum conversion means 131 performs a discrete Fourier transform on the vector of pixel values in the partial area. The feature extraction unit 130 outputs a feature vector whose elements are discrete Fourier transform coefficients of pixel values.

  As another example of the input image partial area extracting unit 2, there is an input image partial area extracting unit 120 of FIG. The input image partial area extraction unit 120 includes an attitude correction unit 121, a partial image acquisition unit 111, and a feature extraction unit 112. The input image partial area extracting unit 120 appropriately corrects the posture of the object in the input image before extracting the feature by the posture correcting unit 121. Specifically, correcting the posture of an object in the image means converting the input image data itself so that the object in the input image is observed by a camera from a predetermined fixed direction. It is. An area-specific input feature vector is generated using the partial image acquisition unit 111 and the feature extraction unit 112 for the image data converted into a fixed posture by the posture correction unit 121. By correcting the posture of the input image, it is possible to improve the deterioration of the collation accuracy due to the posture change of the object in the image. In addition, by correcting the image on the registration side to the same parameter orientation as that of the input, the collation accuracy can be improved.

  The posture correction method will be described with reference to FIG. In general, the posture correction parameters are a total of six parameters: movement along the XYZ axes and rotation around the XYZ axes. In FIG. 21, a face image is taken as an example, and face images facing various directions are shown as input images. The input image A140 is upward, the input image B141 is rightward, the input image C142 is downward, and the input image D143 is leftward. On the other hand, the posture correction image 144 is obtained by converting the input image into a front-facing image.

  As an example of a posture correction method such as conversion to the posture correction image 144, there is a method of affine transformation of image data. A method for correcting the posture of an object by affine transformation is disclosed in, for example, Japanese Patent Application Laid-Open No. 2000-90190.

  As another example of the posture correction method, there is a method using a three-dimensional model shown in FIGS. 22 and 23 show a three-dimensional model assuming a human face. FIG. 22 shows an ellipsoid model 151 and FIG. 23 shows a standard three-dimensional face model 152. The standard 3D face model is a 3D model representing the shape of a standard human face, and can be obtained by using 3D CAD software or measurement by a range finder. Posture correction can be realized by texture-mapping an input image on a three-dimensional model and then moving and rotating the three-dimensional model.

  Next, the registered image partial area extracting unit 9 will be described in detail. As an example of the registered partial area extracting unit 9, there is a registered partial area extracting unit 160 shown in FIG. The registered partial area extraction unit 160 includes a partial image acquisition unit 111 and a feature extraction unit 112. The operations of the partial image acquisition unit 111 and the feature extraction unit 112 have already been described. The registered partial region extraction unit 160 generates a region-specific dictionary feature vector from the registered image.

  Another example of the registered image partial area extracting unit 9 is a registered image partial area extracting unit 170 shown in FIG. The registered image partial area extraction unit 170 includes a posture correction unit 121, a partial image acquisition unit 111, and a feature extraction unit 112. The registered image partial area extracting unit 170 appropriately corrects the posture of the object in the registered image before the feature is extracted by the posture correcting unit 121.

  A region-specific dictionary feature vector is generated by using the partial image acquisition unit 111 and the feature extraction unit 112 for the image data converted into a fixed posture by the posture correction unit 121. By correcting the posture of the registered image, it is possible to improve the deterioration of the collation accuracy due to the posture change of the object in the image.

  As another example of the registered image partial area extracting unit 9, there is a registered image partial area extracting unit 180 shown in FIG. The registered image partial area extraction unit 180 includes a fluctuation image generation unit 181, a partial image acquisition unit 111, and a feature extraction unit 112. The fluctuation image generation means 181 converts the input registered image into a plurality of fluctuation images including itself. The variation image is an image obtained by simulating and converting the change in the appearance of the object caused by various factors such as posture variation and illumination variation with respect to the input image.

  FIG. 28 shows an example of a variation image for a human face. Seven types of variation images including themselves are generated for the input image 250. The posture variation image A251 is an image obtained by converting the face upward. The posture variation image B252 is an image obtained by converting the face to the right. The posture variation image C253 is an image obtained by converting the face downward. The posture variation image D254 is an image obtained by converting the face to the left. These posture variation images can be converted using the method used in the posture correction means 121. The illumination variation image 255 is an image obtained by adding a change in light and darkness due to illumination to the input image 250, and can be realized by, for example, a method of making the pixel values of the input image 250 brighter or darker as a whole. The facial expression variation image 256 is an image converted into a smile by changing the facial expression. The conversion method can be realized by adding a modification such as raising both mouths upward and narrowing both eyes. The original image 257 is data as it is the input image.

  In addition, the fluctuation image generation means 181 can output a plurality of changes of the parameter in a plurality of stages even with the same fluctuation. For example, in the case of a posture change in the right direction, three types of converted images of 15 ° rotation to the right, 30 ° rotation to the right, and 45 ° rotation to the right can be output simultaneously. In addition, it is possible to output an image that has been converted by combining different variation factors such as posture variation and illumination variation.

  The plurality of variation image groups generated by the variation image generation unit 181 is converted into a region-specific dictionary feature vector group by the partial image acquisition unit 111 and the feature extraction unit 112.

  The region-specific dictionary feature vector group generated by the registered image partial region extracting unit 180 is converted into a plurality of dictionary records by the dictionary data generating unit 8, or by the dictionary data generating unit 190 shown in FIG. Is generated. When the principal component data is generated, the input images are collated by the method shown in the conceptual diagram of FIG.

  By creating a fluctuating image on the dictionary registration side, even if the posture or lighting environment of the object in the input image changes, the assumed fluctuating image is registered on the registration side in advance, so it is accurately verified be able to.

  Next, the overall operation of this embodiment will be described in detail with reference to FIG. FIG. 15 is a flowchart showing the overall operation of the image recognition system according to the present invention. First, input image data to be identified is input (step A1). Next, the input image is divided into P partial areas, and feature vectors are extracted for each partial area (step A2). Next, the dictionary data is referred to, and the distance from the registered image is calculated for each partial area (step A3). Next, one integrated distance value is calculated using the distance value for each of the P partial areas (step A4). The minimum distance value in the registered category is calculated (step A5). Next, it is determined whether the minimum distance is smaller than the threshold value (step A6). If the minimum distance is smaller than the threshold value, the category having the minimum distance is output as a recognition result (step A7). When the minimum distance is larger than the threshold value, “no corresponding category” is output (step A8).

  Next, the dictionary data learning operation of this embodiment will be described with reference to FIG. FIG. 16 is a flowchart showing the dictionary data learning operation. A category ID corresponding to the registered image data is input (step B1). Next, the registered image is divided into P partial areas (step B2). Next, using the image data belonging to each partial area, dictionary data is generated for each partial area (step B3). The dictionary data is stored in the dictionary data storage means (step B4). The above operation is repeated as necessary.

  Next, a second embodiment of the present invention will be described in detail with reference to the drawings. FIG. 17 is a block diagram of the second embodiment of the present invention. The second embodiment of the present invention includes a computer 200 that operates under program control, a recording medium 201 that records an image recognition program, a camera 204, an operator console 202, and a display device 203. The recording medium 201 may be a magnetic disk, a semiconductor memory, or other recording medium.

  The computer 200 loads and executes a program that realizes the collation unit 1, the registration unit 7, the dictionary storage unit 5, and the partial area information storage unit 6. The program is stored in the storage medium 201, and the computer 200 reads the program from the storage medium 201 and executes it. The program performs the operations shown in the flowcharts of FIGS. In this embodiment, the input image is input from the camera 204 and the recognition result is displayed on the display device 203. A recognition instruction and learning instruction are given by the operator from the console 202.

  According to the image recognition system of the present invention, the inter-pattern distance between the partial area of the input image and the corresponding partial area of the registered image is calculated, and the input image is calculated based on the inter-pattern distance of each partial area. Since the identification means for identifying is included, it is possible to correctly classify the input images regardless of illumination fluctuations and occlusion conditions. The image recognition method and program according to the present invention also have the same effects as the image recognition system.

  Specifically, the effect of the present invention is to identify a plurality of partial areas in an image independently and integrate the results to reduce the influence of illumination fluctuation and occlusion and correctly identify an input image. It can be done. The reason is that a partial region whose score is abnormal due to illumination variation or occlusion can be eliminated during distance value integration.

1 is a configuration diagram of a first embodiment of an image recognition system according to the present invention. It is a conceptual diagram which shows the distance calculation method between patterns of the image recognition system which concerns on this invention. 3 is a configuration diagram of an inter-pattern distance calculating unit 40. FIG. 3 is a configuration diagram of an inter-pattern distance calculating unit 50. FIG. It is a conceptual diagram which shows the distance calculation method between patterns of the image recognition system which concerns on this invention. It is a block diagram of the area distance value integration means 70. 4 is a configuration diagram of region distance value integration means 80. FIG. It is a conceptual diagram which shows the distance calculation method between patterns of the image recognition system which concerns on this invention. 3 is a configuration diagram of an identification unit 90. FIG. 3 is a configuration diagram of a dictionary storage unit 100. FIG. 3 is a configuration diagram of a collation unit 21. FIG. 3 is a configuration diagram of a collation unit 31. FIG. 4 is a configuration diagram of an inter-pattern distance calculation unit 60. FIG. It is a conceptual diagram which shows the distance calculation method between patterns of the image recognition system which concerns on this invention. It is a flowchart which shows the operation | movement of the whole image recognition system which concerns on this invention. It is a flowchart which shows operation | movement of dictionary data. It is a block diagram of the 2nd Embodiment of this invention. FIG. 3 is a configuration diagram of an input image partial area extraction unit 110. 3 is a configuration diagram of feature extraction means 130. FIG. 4 is a configuration diagram of an input image partial area extraction unit 120. FIG. It is a conceptual diagram which shows an attitude | position correction method. It is a figure which shows an example of an ellipsoidal model. It is a figure which shows an example of a standard 3D face model. FIG. 6 is a configuration diagram of registered image partial area extraction means 160. FIG. 6 is a configuration diagram of registered image partial area extraction means 170. FIG. 6 is a configuration diagram of registered image partial area extraction means 180. 4 is a configuration diagram of dictionary data generation means 190. FIG. It is a conceptual diagram which shows the fluctuation | variation image generation method. It is a block diagram of an example of the conventional image recognition system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Collation part 2 Input image partial area extraction means 3 Inter-pattern distance calculation means 4 Identification means 5 Dictionary storage means 6 Partial area information storage means 7 Registration part 8 Registered image partial area extraction means 9 Dictionary data generation means 10 Area distance value integration means 21 collating unit 22 input image sequence smoothing unit 31 collating unit 32 input principal component generating unit 33 inter-pattern distance calculating means 40 inter-pattern distance calculating means 41 norm calculating means 50 inter-pattern distance calculating means 51 subspace projection distance calculating means 60 patterns Inter-distance calculation means 61 Inter-subspace distance calculation means 70 Area distance value integration means 71 Weighted average value calculation means 80 Area distance value integration means 81 Distance value sorting means 82 Distance value higher average calculation means 90 Identification means 91 Minimum value calculation means 92 Threshold processing means 100 Dictionary storage means 101 Record storage section 102 Code 103-region dictionary data 104 category ID
DESCRIPTION OF SYMBOLS 110 Input image partial area extraction means 111 Partial image acquisition means 112 Feature extraction means 120 Input image partial area extraction means 121 Posture correction means 130 Feature extraction means 131 Fourier spectrum conversion means 140 Input image A
141 Input image B
142 Input Image C
143 Input image D
144 posture correction image 150 ellipsoid model 151 standard three-dimensional face model 160 registered image partial region extraction unit 170 registered image partial region extraction unit 180 registered image partial region extraction unit 181 variation image generation unit 190 dictionary data generation unit 191 principal component data generation Means 200 Computer 201 Storage medium 202 Console 203 Display device 204 Camera 210 Image input unit 211 Angle calculation unit between subspaces 212 Recognition unit 213 Dictionary storage unit 250 Input image 251 Posture change image A
252 Posture change image B
253 Posture change image C
254 Posture change image D
255 Illumination variation image 256 Expression variation image 257 Original image 300 Input image 301 Registered image 302 Input image region segmentation result 303 Registered image region segmentation result 304 Partial region A input image 305 Partial region A registered image 306 Collation pair 307 Partial region A input feature Vector 308 Partial region A dictionary feature vector 309 Regional input feature vector 310 Regional dictionary feature vector 320 Input image sequence 321 Registered image sequence 322 Input image region segmentation result 323 Registered image region segmentation result 324 Partial region A input sequence 325 Partial region A Registration Sequence 326 Partial Area A Input Principal Data 327 Partial Area A Dictionary Principal Data 328 Verification Pair 329 Area-Specific Input Principal Data 330 Area-Specific Dictionary Principal Data 340 Input Image 341 Registered Image System Kens 342 Input image region segmentation result 343 Registered image region segmentation result 344 Partial region A input image 345 Partial region A registration sequence 346 Matching pair 347 Partial region A input feature vector 348 Partial region A dictionary principal component data 349 Regional input feature vector 350 Region-based dictionary principal component data 400 Input image 401 Registered image 402 Low collation score partial region group 403 High collation score partial region group 404 Partial region distance value map

Claims (3)

Inter- pattern distance calculating means for calculating a plurality of inter-pattern distance values between a plurality of partial areas in the input image and each partial area of the registered image at a position corresponding to each of the partial areas ;
An image recognition system including identification means for identifying the input image based on a plurality of inter-pattern distance values,
The inter-pattern distance calculating means calculates the inter-pattern distance value for the partial region arranged in the entire image,
The identifying means identifies an input image based on one integrated distance value calculated using only a predetermined number in order of increasing distance values from the calculated distance values between patterns. Image recognition system. An inter- pattern distance calculating step of calculating a plurality of inter-pattern distance values between a plurality of partial areas in the input image and each partial area of the registered image at a position corresponding to each of the partial areas ;
An identification step for identifying the input image based on a plurality of inter-pattern distance values,
In the inter-pattern distance calculating step, the inter-pattern distance value is calculated for the partial region arranged in the entire image,
In the identifying step, the input image is identified based on one integrated distance value calculated using only a predetermined number from the calculated pattern distance values in ascending order of distance values. An image recognition method. An inter- pattern distance calculating step of calculating a plurality of inter-pattern distance values between a plurality of partial areas in the input image and each partial area of the registered image at a position corresponding to each of the partial areas ;
A program for causing a computer to execute an image recognition method including an identification step of identifying the input image based on a plurality of inter-pattern distance values,
In the inter-pattern distance calculating step, the inter-pattern distance value is calculated for the partial region arranged in the entire image,
In the identifying step, the input image is identified based on one integrated distance value calculated using only a predetermined number from the calculated pattern distance values in ascending order of distance values. Program.

Images