JP4199594B2 - OBJECT IDENTIFICATION DEVICE, ITS PROGRAM, AND RECORDING MEDIUM CONTAINING THE PROGRAM - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention belongs to an image identification technique for identifying what kind of object is shown in an image, and a specific industrial application system includes, for example, an image search system.
[0002]
[Prior art]
The image recognition technique is a technique for specifying to which object a certain area in image data belongs. Image recognition methods are broadly divided into pattern matching methods and statistical identification methods.
[0003]
In the pattern matching method, it is detected whether there is an input image that is the same as or similar to a standard pattern created in advance. The standard pattern is expressed using features that clearly indicate the pattern. In a template matching method as a typical method, a template characterized by a grayscale image or a template characterized by a differential grayscale image obtained by differentiating the grayscale image is generally used as a standard pattern.
[0004]
In the statistical identification method, when two images to be collated do not have a simple translational relationship, or when the input image is described by a feature parameter, the images cannot be directly collated, and matching between feature descriptions is not possible. I need it. The feature vector of the input image and the co-occurrence probability of a certain target are calculated, and the target having the maximum value is determined as the target to which the input image belongs. This means that the most reliable object is selected from the feature vector of the input image.
[0005]
However, these conventional methods have the following two problems.
1. Since the number of patterns to be verified is enormous, the amount of calculation increases and cannot be identified in a realistic time.
2. When the number of objects to be identified becomes enormous, in the statistical method, the dimension of the feature vector needs to be large in order to increase the identification accuracy. In order to obtain the multidimensional occurrence probability, a large amount of learning sample data is essential, which is not realistic. For this reason, a normal distribution is obtained by using a certain number of samples and the occurrence probability is estimated. However, the actual distribution is not a normal distribution, and there is a limit to increasing the identification accuracy.
[0006]
As a conventional method for solving this problem, there is a technique such as Non-Patent Document 1 for compressing a template.
[0007]
[Non-Patent Document 1]
Hiroshi Murase, S. K. Nayer, “Three-dimensional object recognition by two-dimensional matching-Paramelic rick eigenspace method-”, Theory of Science, J77-D-II, No. 11, pp 2179-2187. 1994, November.
[Problems to be solved by the invention]
As mentioned above, in the conventional method,
1. Since the number of patterns to be verified is enormous, the amount of calculation increases and cannot be identified in a realistic time.
2. When the number of objects to be identified becomes enormous, in the statistical method, the dimension of the feature vector needs to be large in order to increase the identification accuracy. In order to obtain the multidimensional occurrence probability, a large amount of learning sample data is essential, which is not realistic. For this reason, a normal distribution is obtained by using a certain number of samples and the occurrence probability is estimated. However, the actual distribution is not a normal distribution, and there is a limit to increasing the identification accuracy.
There are the above two problems, and a conventional method such as Non-Patent Document 1 for compressing a template for solving this problem can cope with a certain number of objects, but cannot cope with a large increase in the number of objects. There is a problem.
[0009]
The present invention has been made in order to solve the above-described problems of the prior art, and does not increase the calculation time required for discrimination even when the number of objects increases enormously, and performs discrimination at high speed. It is an object to provide a method and apparatus capable of performing the above.
[0010]
[Means for Solving the Problems]
In order to solve the above problems, the present invention is an object learning device for registering an object using a plurality of images of an object in order to identify a plurality of registered objects, wherein a feature vector is used as feature data of each object. An input means for inputting; an object classification means for classifying an object as a tree structure based on the similarity between the feature vectors of each object; and a principal component analysis for principal component analysis of feature vectors belonging to each node of the classified tree structure An object learning apparatus characterized by comprising means is a means.
[0011]
Alternatively, in an object identification device for identifying a plurality of registered objects using a plurality of images of the object, in an input means for inputting a target feature vector to be identified and a root node of a tree structure in which the plurality of objects are classified Compression means for compressing feature vectors; child node compression means for further compressing feature vectors compressed at all child nodes connected to each node of the tree structure; and feature vectors compressed at each node of the tree structure An object identification apparatus comprising: a determination unit that determines which node the node belongs to; and an output unit that outputs a corresponding category at the end node of the tree structure.
[0012]
Alternatively, in the object identification device, the input unit includes: an image input unit that inputs image data of each object; and a feature extraction unit that extracts a feature from the image data. That means.
[0013]
Alternatively, in the object identification device, the object classification unit includes a correlation calculation unit that calculates a correlation between feature vectors of each object at a parent node, and a child node that classifies the object into a plurality of child nodes based on a correlation distance value. An object identification device characterized by having classification means is used as the means.
[0014]
Alternatively, in the object identification device, the determination unit calculates a subspace distance, which is a distance between a compressed feature vector and a child node compressed feature vector compressed at a child node, at each node. An object identification apparatus characterized by having a calculation means and a selection means for selecting a child node indicating the minimum subspace distance is used as the means.
[0015]
Alternatively, an object learning method for registering an object using a plurality of images of an object to identify a plurality of registered objects, an input step of inputting a feature vector as feature data of each object; An object classification stage for classifying an object as a tree structure based on a similarity between feature vectors, and a principal component analysis stage for principal component analysis of a feature vector belonging to each node of the classified tree structure The object learning method is used as the means.
[0016]
Alternatively, an object identification method for identifying a plurality of registered objects using a plurality of images of an object, in an input stage for inputting a target feature vector to be identified, and a root node of a tree structure in which the plurality of objects are classified A compression stage for compressing feature vectors; a child node compression stage for further compressing feature vectors compressed at all child nodes connected to each node of the tree structure; and a feature vector compressed at each node of the tree structure The object identifying method is characterized in that it has a determining step of determining which node in the lower layer belongs to and an output step of outputting a corresponding category at the end node of the tree structure.
[0017]
Alternatively, in the object identification method, the input step includes an image input step of inputting image data of each object, and a feature extraction step of extracting features from the image data. That means.
[0018]
Alternatively, in the object identification method, the object classification step includes: a correlation calculation step of calculating a correlation between feature vectors of each object at a parent node; and a child node that classifies the object into a plurality of child nodes according to a correlation distance value. An object identification method characterized by having a classification step is used as the means.
[0019]
Alternatively, in the object identification method, the determination step includes calculating a subspace distance, which is a distance between a compressed feature vector and a child node compressed feature vector compressed at a child node, at each node. The object identifying method is characterized by having a calculation step and a selection step of selecting a child node indicating the minimum subspace distance.
[0020]
Alternatively, an object learning program characterized in that the step in the object learning method is a program for causing a computer to execute is used as the means.
[0021]
Alternatively, the means in the object learning method is a program for causing a computer to execute the program, and the program is recorded on a recording medium readable by the computer. And
[0022]
Alternatively, an object identification program characterized in that the step in the object identification method is a program for causing a computer to execute is used as the means.
[0023]
Alternatively, a means for causing a computer to execute the steps in the object identifying method, and recording the object identifying program, wherein the program is recorded on a recording medium readable by the computer. And
[0024]
In the present invention, in the means and stage for registering a plurality of objects, by classifying the objects as a tree structure based on the similarity between the feature vectors of each object, the calculation time required for discrimination can be reduced even if the number of categories increases. Do not increase. In addition, the principal component analysis is performed on the feature vectors belonging to each node of the classified tree structure, and the feature vector is compressed using the principal component at each node of the tree structure in the means and stage for identifying the object, so that By determining whether the feature vector belongs to the subspace, it is possible to determine the partial space distance at high speed, and to determine which node in the lower layer the feature vector belongs to at high speed.
[0025]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[0026]
FIG. 1 is a diagram showing a system configuration according to an embodiment of the present invention, which is an example in which the present invention is applied to a translation system. With this system, a user can take a photographed character / symbol / trademark / signboard. The translation information of the character / symbol / trademark / signboard can be viewed based on the image of However, it is assumed that a translation dictionary of characters / symbols / trademarks / signboards has been created. This system is composed of a registered information storage / retrieval device such as a learning device 1, an identification device 2, and a translation device 3 according to the present invention. The present invention registers a character / symbol / trademark / signboard using a plurality of images including images of different fonts of characters / symbols / trademarks / signboards by the learning device 1, and a plurality of characters / symbols registered by the identification device 2. Identify symbols / trademarks / signs. The translation device 3 is a device that accumulates characters / symbols / trademarks / signboard names and registration information, and retrieves translation information from the characters / symbols / trademarks / signboard names. Details are not described in the embodiment.
[0027]
The present invention includes the learning device 1 and the identification device 2 as described above. The learning device 1 obtains information necessary for identification from a plurality of viewpoint images of characters / symbols / trademarks / signboards and accumulates the information in a storage device. The identification device 2 identifies the character / symbol / trademark / signboard photographed in the image by using the image input by the user and the information necessary for identification stored in the storage device of the learning device 1.
[0028]
In the following example, a case where six types of sentence examples of “a”, “i”, “u”, “e”, “o”, “ka” are registered / identified will be described as an example. However, this example does not limit the number of objects to six types, and the number of types can be expanded.
[0029]
FIG. 2 is a diagram showing a configuration of the first exemplary embodiment of the present invention, and details of the learning device 1 and the identification device 2 are described.
[0030]
The learning apparatus 1 includes an input unit 11 for inputting a feature vector of each object, an object classifying unit 12 for classifying an object as a tree structure based on a similarity between the feature vectors of each object, and an object belonging to the node of the tree structure The principal component analysis means 13 for performing principal component analysis using the feature vectors and the accumulation means 14 for accumulating eigenvectors and principal components obtained as a result of the principal component analysis.
[0031]
The identification device 2 includes an input unit 21 that inputs a feature vector of an object to be identified, a compression unit 22 that compresses a feature vector and an eigenvector of the root node at the root node of the tree structure, and each node of the tree structure. A child node compression means 23 for further compressing feature vectors compressed in all the connected child nodes; a discrimination means 24 for determining which node in the lower layer the feature vector compressed in each node of the tree structure belongs; The output means 25 outputs the corresponding category at the end node of the tree structure.
[0032]
FIG. 3 is a diagram showing the configuration of the second exemplary embodiment of the present invention, and details the learning device 1 and the identification device 2. The difference from the first embodiment is that the input means 21 is composed of an image input means 211 for inputting an image of an object and a feature extraction means 212 for rendering a feature necessary for identification from the image. Is a point.
[0033]
Hereinafter, embodiments of the method of the present invention will be described.
[0034]
First, the learning stage will be described. FIG. 4 is a diagram illustrating an embodiment of the learning stage according to the present invention, and is a flowchart illustrating an operation example of the learning device 1. The learning device 1 performs the following processing.
1. Repeat for the total number of images H [1]
2. 2. Input of target image Repeat [1] End 4. Repeated for H for all images [2]
5. Feature extraction Repeat [2] End 7. Classification of objects Repeat for J nodes in tree structure [3]
9. Principal component analysis10. Repeat [3] End 11. Accumulation of principal components and compression features.
[0035]
In the following, an example of the operation of each means in FIGS. 2 and 3 will be described in detail with reference to the drawings.
[0036]
FIG. 5 is a diagram for explaining an embodiment of the object classification means 12 of the present invention. The object classification unit 12 includes a correlation calculation unit 121 that calculates a correlation between feature vectors of objects, and a child node classification unit 122 that classifies the objects into a tree structure.
[0037]
FIG. 6 is a flowchart showing an operation example of object classification according to the present embodiment.
1. The number of remaining categories = the number of objects.
2. Repeat until the number of remaining categories is 1 [1]
3. Repeat for the remaining number of categories [2]
4). 4. Select one category from the remaining category group Number of remaining categories-1 minute repeat [3]
6). The correlation between the selected category and all categories belonging to the remaining category group and the feature is calculated.
7). Repeat [3] End 8. The categories whose correlation values are greater than or equal to the threshold are integrated into an integrated category and transferred to a new category group. When there are a plurality of categories having a correlation value equal to or greater than the threshold value, all these categories are integrated.
9. An average of feature vectors in the integrated category is obtained and used as a representative feature.
10. Repeat [2] end.
11. Move all categories that belong to the new category to the remaining categories.
12 The number of remaining categories is updated. If integration has never occurred at 8, the threshold value is updated.
13. Repeat [1] End.
[0038]
However, the threshold and its updating method are entered in advance. When integration does not occur, a threshold is set to be lowered in order to promote integration. For example,
Threshold after update = current threshold−0.1
However, when the threshold value <0, the threshold value = 0.
[0039]
FIG. 7 is a flowchart showing the process of object classification, and each state will be described.
[0040]
State 1 is a state in which “A” is selected from the remaining categories and the correlation with the remaining categories is calculated. Since the correlation value with “I” is high, “A” and “I” are integrated into an integrated category to be a new category. Next, “U” is selected from the remaining categories, and the correlation with the remaining categories is calculated (state 2). Since the correlation value with “e” is high, “u” and “e” are integrated into an integrated category to be a new category. Next, “O” is selected from the remaining categories, and the correlation with the remaining categories is calculated (state 3). Since the correlation with “ka” is high, “o” and “ka” are integrated into an integrated category, which is a new category. The new category is moved to the remaining category (state 4).
[0041]
The above process is repeated until the number of remaining categories becomes one in state 4. An example of the resulting tree structure is shown in FIG. The integrated category at each stage is called a node, the highest node is a root node, the lowest node is a terminal node, and the distance from the root node to the terminal node is called the number of stages in the tree structure.
[0042]
FIG. 9 shows an embodiment of principal component analysis means 13 for principal component analysis at each node. In the root node (Aiueoka) of FIG. 9A, feature vectors D0 (A) and D0 of each category are shown. (I), D0 (U), D0 (E), D0 (O) are subjected to principal component analysis, and the eigenvector S (Aiueo) and each compressed feature vector D1 (A), D1 (I), D1 (U), D1 (E), D1 (O) and D1 (Ka) are obtained. In the node (Ai) of FIG. 9B, the feature vectors D1 (A) and D1 (I) of each category are subjected to principal component analysis, and the eigenvector S (Ai) and the compressed feature vectors D2 (A) and D2 (I) are analyzed. ) In the node “A” in FIG. 9C, the principal component analysis is performed on the feature vector D2 (A) to obtain the eigenvector S (A) and the compressed feature vector D3 (A).
[0043]
Next, the identification step according to an embodiment of the method of the present invention will be described.
[0044]
FIG. 10 is a flowchart showing an operation example in the identification stage.
1. 1. Input of image to be identified Extract features from the picture.
3. Select the root node and compress the feature vector.
4). Repeat for the number of steps in the tree structure (3 in this example) [1]
5. Repeat for the number of child nodes of the selected node [2]
6). Compute the child node compression feature vector at the child node.
7). Repeat [2] End 8. Make a decision.
9. Select the child node that shows the smallest subspace distance.
10. Repeat [1] End 11. Output the category of the selected node.
[0045]
FIG. 11 is a diagram showing an embodiment of the discriminating means 24 of the present invention, in which the distance between the feature vector compressed at each node and the child node compressed feature vector compressed at the child node. It comprises subspace distance calculation means 241 for calculating (subspace distance) and selection means 242 for selecting a child node indicating the minimum subspace distance.
[0046]
FIG. 12 is a flowchart showing an embodiment of the determination flow according to the present invention.
1. Repeat for the number of child nodes [1]
2. A subspace distance is calculated from the compressed feature vector and the child node compressed feature vector.
3. Repeat [1] End 4. Select the child node with the smallest subspace distance.
[0047]
FIG. 13 is a flowchart showing an embodiment of a flow for performing identification.
[0048]
When the feature vector D0 (input) is input, D0 (input) is compressed using the eigenvector S (Aiueoka) to generate D1 (input). Next, on each child node of “Ai”, “Ue”, and “Oka” that are child nodes of the root node, each eigenvector S (Ai), S (Ue), S (Oga) is used and the child node Compression feature vectors D21 (input), D22 (input), and D23 (input) are obtained.
[0049]
A subspace distance L21 (input) is obtained from D1 (input) and D21 (input). The subspace distance L22 (input) is obtained from D2 (human power) and D22 (input). A subspace distance L23 (input) is obtained from D1 (input) and D23 (input). The smallest node (here, “ai”) is selected from L21 (input), L22 (input), and L23 (input).
[0050]
Further, on each child node of “A” and “I” which are child nodes of “Ai”, using each eigenvector S (A), S (I), a child node compressed feature vector D31 (input), D32 ( Input). A subspace distance L31 (input) is obtained from D21 (input) and D31 (input). A subspace distance L32 (input) is obtained from D21 (input) and D32 (input). The smallest node (here, “ai”) is selected from L31 (input) and L32 (input). Finally, “A” is output.
[0051]
It should be noted that some or all of the functions of each unit in each apparatus described with reference to FIGS. 2, 3, 5, and 11 are configured by a computer program, and the program is executed using the computer. Or the processing steps at each step described with reference to FIG. 4, FIG. 6 to FIG. 10, FIG. 12 and FIG. 13 are configured by a computer program and the program is executed on the computer. Needless to say, a program for realizing the function of the computer or a program for causing the computer to execute the processing stage can be recorded on a recording medium that can be read by the computer, for example, a flexible disk, For MO, ROM, memory card, CD, DVD, removable disk, etc. Was recorded, or stored, it is possible to or distribute. It is also possible to provide the above program through a network such as the Internet or electronic mail. In this way, the present invention can be implemented by installing a program provided by a recording medium or a network in a computer.
[0052]
【The invention's effect】
As described above, according to the present invention, in the means and stage for registering a plurality of objects, the objects are classified as a tree structure based on the similarity between the feature vectors of each object, so that the number of categories increases. However, the calculation time required for discrimination does not increase so much. In addition, the feature vector belonging to each node of the classified tree structure is subjected to principal component analysis, and the feature vector is compressed using the principal component at each node of the tree structure in the means and stage for identifying the object. Can be determined at high speed, and it can be determined at high speed which node the feature vector belongs to.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a system configuration according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration of a first embodiment of the present invention. FIG. 3 is a diagram illustrating a second embodiment of the present invention. FIG. 4 is a flow chart illustrating an embodiment of the learning stage of the present invention. FIG. 5 is a diagram illustrating an embodiment of the object classifying means according to the present invention. FIG. 7 is a flowchart showing an operation example of object classification according to an embodiment of the present invention. FIG. 7 is a flowchart showing a process of object classification according to an embodiment of the present invention. FIG. 9 is a diagram illustrating examples of object classification results according to examples. FIGS. 9A, 9B, and 9C are diagrams illustrating a flow of principal component analysis in each node according to an embodiment of the present invention. 10. Identification step according to one embodiment of the present invention. FIG. 11 is a flowchart showing an embodiment of the discrimination means according to the present invention. FIG. 12 is a flowchart showing an embodiment of the discrimination flow according to the present invention. Flowchart showing an embodiment of a flow for performing identification according to the present invention
DESCRIPTION OF SYMBOLS 1 ... Learning apparatus 11 ... Input means 111 ... Image input means 112 ... Feature extraction means 12 ... Object classification means 121 ... Correlation calculation means 122 ... Child node classification means 13 ... Principal component analysis means 14 ... Accumulation means 2 ... Identification apparatus 21 ... Input means 211 ... Image input means 212 ... Feature extraction means 22 ... Compression means 23 ... Child node compression means 24 ... Discrimination means 241 ... Subspace distance calculation means 242 ... Selection means 25 ... Output means 3 ... Translation device

Claims (5)

An object identification device for identifying a plurality of registered objects using a plurality of images of an object,
  An input means for inputting a feature vector of an object to be identified;
  Compression means for compressing a feature vector at a root node of a tree structure in which a plurality of objects are classified;
  Child node compression means for further compressing feature vectors compressed in all child nodes connected to each node of the tree structure;
  Determining means for determining which node in the lower layer the feature vector compressed in each node of the tree structure belongs;
  Output means for outputting a corresponding category at the end node of the tree structure.
  An object identification device characterized by the above. The input means includes
  Image input means for inputting image data of each object;
  And feature extraction means for extracting features from the image data.
  The object identification device according to claim 1. The discrimination means includes
  At each node, a compressed feature vector;
  A subspace distance calculating means for calculating a subspace distance that is a distance between child node compressed feature vectors compressed at the child node;
  Selecting means for selecting a child node exhibiting a minimum subspace distance;
  The object identification device according to claim 1, wherein the object identification device is an object identification device. A program that causes a computer to function as each means that constitutes the object identification device according to claim 1. A recording medium storing the program according to claim 4.

Images