JP5846553B2 - Attribute learning and transfer system, recognizer generation device, recognizer generation method, and recognition device - Google Patents

Description

  The present invention relates to a technology that enables transfer learning by recognizing a class to be identified by an attribute that is a characteristic thereof, and relates to an attribute learning and transfer system for recognizing a class, a recognizer generation device, and a recognizer generation. The present invention relates to a method and a recognition device.

  Individual research on object recognition and object identification has made great progress over the last decade. For tasks that detect specific objects such as faces or vehicles, very powerful detectors and recognizers are available. Such a detector or recognizer is obtained by a combination of a low-dimensional feature amount (for example, SIFT, SURF, etc.) indicating a target feature and a modern machine learning mechanism such as a support vector machine (SVM). be able to. However, such a technique typically requires a large number of manually labeled teacher data to obtain good accuracy, and typically requires several hundred thousand sample images to learn each individual class. I need.

  Also, other problems may occur when it is necessary to recognize many objects. In order to solve the problem of recognizing many objects like this, it is necessary to create new detectors for each target category and learn these detectors with the conventional methods. . However, even when learning each new detector efficiently, in the same way as described above, a large number of manually labeled teacher data is required, and in order to learn each individual class, Therefore, several hundred thousand sample images are required.

  From a computer perspective, for example, if some efficient and automatic labeling tool is available, it may be considered that preparing a large number of training data sets is not that difficult. Many sets of images can be easily accessed via the Internet, and the performance of computer hardware has improved dramatically in recent years. Nevertheless, this does not apply to the use of intelligent agents such as robots. For intelligent robots, robots have limited access to the Internet and limited hardware resources, and practical real-world tasks are too common, It seems unlikely that this task can be solved by using only pre-learned detectors. Therefore, recently, many researchers have become more interested in object recognition by taking into account the attributes of the object (eg, exemplified in Non-Patent Document 1) and the part of the object.

  There is usually some common attribute between multiple subjects (for example, there is a common attribute in lions, tigers, dogs, cats, etc. that all animals are four-legged). As proposed in Non-Patent Document 1, humans can distinguish at least 30000 related object classes by learning from examples and fully abstracting them (humans are high dimensional). If a feature description is given, even a completely unknown target class can be detected.) This means that the knowledge of attributes found in one target class has been transferred for use to other different target classes that contain the same attributes. Many previous valuable achievements in computer vision have already shown that the use of transferred attributes makes it possible to detect unknown target classes (see, for example, Non-Patent Document 1). I want.)

C. Lampert et al., "Learning to detect unseen object classes by between-class attribute transfer," in CVPR, 2009.

  Transferring the possibility of learning common attributes and using them to detect new classes is extremely effective in current robotics. For the sake of simplicity, assume the use of a mobile robot in the office. If you want the robot to move to another room and instruct the robot to pick up the subject B for us, we are prepared to present the image of the subject B to the robot The situation is not very likely. The only way to give instructions to the robot without requiring an image of the object is to describe the attributes of the object in words. This should be solved by letting the robot learn the attributes from one class of objects, and then transfer the learned attributes to be used to recognize new objects belonging to the unknown class.

  However, despite the fact that it has been shown so far that it is possible to detect unknown target classes through attribute learning and transfer, we apply the proposed attribute transfer and learning methods to robotic use. There are problems as described below.

  First, in the conventional method, although the learned attribute can be transferred to use in another target class, the batch process is completely performed in relation to the prior learning stage for learning each attribute detector. The conventional method requires a huge teacher image data set in order to learn a detector having any one attribute. Also, for use with robots, the system should learn more flexibly whenever a teacher image is acquired, and should identify whenever necessary. Therefore, a completely additional attribute learning and transfer approach is needed.

  Here, the problem will be described by taking the conventional method disclosed in Non-Patent Document 1 as an example. In the conventional method disclosed in Non-Patent Document 1, it is necessary to learn a classifier for each individual attribute. In the test phase, each attribute classifier predicts the probability for each attribute, and a final probability score is calculated based on Bayesian theory. Each attribute classifier is learned by SVM and requires several hours for learning. If this is performed for all the attributes (85 attributes), the calculation load becomes too high and cannot be used practically for robotics or other online applications. In addition, since it is unrealistic to learn SVM again for all attributes, new input teacher data cannot be additionally learned.

  The present invention has been made to solve such problems, and it is an object of the present invention to provide an attribute learning and transfer system and a learning and transfer method that are online and can be additionally learned.

  An attribute learning and transfer system according to the present invention includes a feature extraction unit that extracts features from input data and teacher data, a labeling unit that labels given attribute information on the teacher data, and attributes included in the input data An attribute discriminator that comprises a node and a self-propagating neural network including edges connecting the nodes, and the self-propagating neural network is identified by the attribute The teacher data extracted by the feature extraction unit with respect to the part of the self-propagating neural network that is divided into a plurality of parts according to the identification content and specified by the attribute information labeled by the labeling unit Is input as a teacher pattern, and in the self-propagating neural network, When the input data is input, the discriminator generating unit that generates the node and the edge based on the teacher pattern, the discriminator holding unit that holds the attribute discriminator generated by the discriminator generating unit, For each part of the self-propagating neural network constituting the attribute classifier held by the classifier holding unit, a feature extracted from the input data is input as an input pattern, and the input pattern and the A first similarity with the node included in the self-propagating neural network is calculated in each part of the self-propagating neural network, and any one of the identification contents is determined according to the calculated first similarity. An attribute identification unit for identifying whether the input data is included in the input data, and attribute information included for each of the plurality of classes. And comparing the attribute of the input data identified by the attribute identification unit with the attribute information of the class to obtain a second similarity, and according to the calculated second similarity, the plurality of classes A class identifying unit for identifying which class includes the input data.

  Thereby, learning and transfer of attributes that can be performed online and additional learning can be realized.

  The classifier generator may divide the attribute classifier into a first part indicating that the attribute is included and a second part indicating that the attribute is not included. It may be. As a result, binary attribute identification can be realized with a simple configuration.

  Furthermore, the input data and the teacher data may be image data, audio data, time-series data, or a combination of these, and the feature extraction unit may perform SIFT feature values from the image data. , SURF feature value, rg-SIFT feature value, PHOG feature value, cq feature value, Lss-histogram feature value may be extracted.

  The self-propagating neural network may be Self-Organizing and Incremental Neural Networks. Thereby, complete online and additional learning can be realized.

  The recognizer generation device according to the present invention is a recognizer generation device that generates a recognizer for recognizing a class to be identified by an attribute that is a feature by learning a feature amount of teacher data. And a feature extraction unit that extracts a feature quantity as a weight vector from the teacher data in which the attribute is attached as a label weight, and calculates the distance between the input node and each node using the extracted weight vector as an input node A winner node extraction unit that extracts a node closest to the input node and a node closest to the second as a first winner node and a second winner node, respectively, the input node, and the first and second winner nodes. A node insertion determination unit that determines whether or not to insert the input node as a new node based on the distance; and the input node as a new node. Otherwise, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. If there is an edge, the age is set to 0, and the first winner is An edge management unit that increments the age of all edges of a node and deletes an edge that has reached a predetermined age, and when the input node is not inserted as a new node, the weight vector of the first winner node is the input node A node weight updating unit for updating based on a weight vector of the label, and a label weight for diffusing at least a part of the label weight of the input node to the first and second winner nodes when the input node is not inserted as a new node An update unit, and a node deletion unit that deletes a node at a predetermined timing according to the node density, and updates the label weight. Parts are when the node deleting unit deletes the node is at least a portion of the label weights with deletion node intended to diffuse into neighboring nodes of the removed node.

  Thereby, the recognizer production | generation apparatus which can be added online and can be provided can be provided.

  The recognizing device generating method according to the present invention is a recognizing device generating method for generating a recognizing device that recognizes a class to be identified by an attribute that is a feature of the class by learning a feature amount of teacher data. And a feature extraction step of extracting a feature quantity as a weight vector from the teacher data to which the attribute is attached as a label weight, and calculating the distance between the input node and each node using the extracted weight vector as an input node A winner node extracting step of extracting a node closest to the input node and a node closest to the second as a first winner node and a second winner node, respectively, and the input node and the first and second winner nodes. A node insertion determination step for determining whether to insert the input node as a new node based on the distance; and the input node If not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is 0, if there is an edge, its age is 0, An edge management step of incrementing the ages of all edges of the first winner node and deleting edges that have reached a predetermined age; and when the input node is not inserted as a new node, the weight vector of the first winner node A node weight update step for updating the input node based on the weight vector of the input node, and if the input node is not inserted as a new node, at least a part of the label weight of the input node is assigned to the first and second winner nodes The first label weight updating step for spreading and deleting nodes according to the node density at a predetermined timing And a first label weight updating step for diffusing at least a part of the label weight of the deleted node to nodes around the deleted node when the node is deleted in the node deleting step. is there.

  Thereby, it is possible to provide a recognizer generation method capable of online and additional learning.

  A program according to the present invention is a program for causing a computer to execute a process of generating a recognizer that recognizes a class to be identified by an attribute that is a feature by learning a feature amount of teacher data, the class And feature extraction processing for extracting a feature quantity as a weight vector from the teacher data to which the attribute is assigned as a label weight, and using the extracted weight vector as an input node, the distance between the input node and each node is calculated. A winner node extraction process for extracting a node closest to the input node and a node closest to the second as a first winner node and a second winner node, respectively, and the input node and the first and second winner nodes. A node insertion determination process for determining whether to insert the input node as a new node based on the distance; and When a force node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is 0, and if there is an edge, its age is 0. In addition, an edge management process for incrementing the age of all edges of the first winner node and deleting an edge reaching a predetermined age, and when the input node is not inserted as a new node, the first winner node Node weight update processing for updating the weight vector of the input node based on the weight vector of the input node, and when the input node is not inserted as a new node, at least a part of the label weight of the input node is the first and second A first label weight update process that spreads to the winner node and a node that deletes the node according to the node density at a predetermined timing. When the node is deleted in the node deletion step, the computer is caused to execute a first label weight update process for diffusing at least a part of the label weight of the deleted node to nodes around the deleted node. Is.

  As a result, it is possible to provide a program that allows online and additional learning.

  A recognition apparatus according to the present invention is a recognition apparatus capable of transfer learning by using a recognition target to be recognized from input data as a class, and recognizing the class based on an attribute that is a feature of the recognition object. A feature extraction unit that extracts a vector as a weight vector, a recognizer generated by inputting teacher data with the class and the attribute added as label weights to a recognizer generation device, and learning the feature amount; According to the distance between a plurality of learned nodes composed of weight vectors possessed by the recognizer and the weight vectors extracted from the input data, a result output unit that outputs a recognition result; A feature extraction unit that extracts a feature amount from the teacher data as a weight vector, and the extracted weight vector as an input node, and the input node and each node A winner node extraction unit that extracts a node closest to the input node and a node closest to the input node as a first winner node and a second winner node, respectively, the input node, the first and second A node insertion determination unit that determines whether or not to insert the input node as a new node based on a distance from the winner node; and when the input node is not inserted as a new node, the first winner node and the first node If there is no edge between the two winner nodes, an edge is generated and the age is set to 0. If there is an edge, the age is set to 0, and the age of all the edges of the first winner node is incremented. An edge management unit that deletes an edge that has reached the age of, and a weight vector of the first winner node when the input node is not inserted as a new node A node weight update unit that updates based on the weight vector of the input node and, when the input node is not inserted as a new node, diffuses at least a part of the label weight of the input node to the first and second winner nodes A label weight updating unit that deletes the node according to the node density at a predetermined timing, and the label weight updating unit is deleted when the node deleting unit deletes the node. At least a part of the label weight of the node is diffused to nodes around the deleted node, and each node after inputting a predetermined number of the teacher data is set as the learned node, and the recognizer recognizes the recognizer Is constituted.

  Thereby, it is possible to provide a recognition device using a recognizer that is online and capable of additional learning.

  The recognition method according to the present invention generates a recognizer that recognizes a class to be identified by an attribute that is a feature by learning a feature amount of teacher data, and recognizes input data by the recognizer. A first feature extraction step of extracting a feature quantity as a weight vector from teacher data to which the class and the attribute are attached as label weights, and the extracted weight vector as an input node, Calculating a distance to a node, extracting a node closest to the input node and a node closest to the second as a first winner node and a second winner node, respectively, the input node; A node insertion rule for determining whether to insert the input node as a new node based on the distance between the first and second winner nodes If the step and the input node are not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is 0, and if there is an edge When the age is set to 0, the age of all edges of the first winner node is incremented, and an edge management step of deleting an edge that reaches a predetermined age, and when the input node is not inserted as a new node, A node weight update step of updating the weight vector of the first winner node based on the weight vector of the input node; and if the input node is not inserted as a new node, at least a part of the label weight of the input node is A first label weight update step spreading to the first and second winner nodes, and at a predetermined timing, the node A node deletion step for deleting a node according to the degree, and a first label weight for diffusing at least a part of the label weight of the deletion node to nodes around the deletion node when the node is deleted in the node deletion step An update step, each node after inputting a predetermined number of the teacher data as a learned node, a recognizer generating step that configures the recognizer with the learned node, and extracting a feature quantity from the input data as a weight vector A recognition result is output according to the distance between the plurality of learned nodes of the recognizer generated in the recognizer generating step and the weight vector extracted from the input data. And a result output step.

  Thereby, it is possible to provide a recognition method using a recognizer capable of online and additional learning.

  The program according to the present invention generates a recognizer that recognizes a class to be identified by an attribute that is a feature by learning a feature amount of teacher data, and performs processing for recognizing input data by the recognizer. A first feature extraction process for extracting a feature amount as a weight vector from teacher data in which the class and the attribute are assigned as label weights, and the extracted weight vector as an input node Calculating a distance between the input node and each node, and extracting a node closest to the input node and a second closest node as a first winner node and a second winner node, respectively, Based on the distance between the input node and the first and second winner nodes, the input node is inserted as a new node. Node insertion determination processing for determining whether or not the input node is not inserted as a new node, and if there is no edge between the first winner node and the second winner node, an edge is generated and An edge management process for setting the age to 0, if there is an edge, setting the age to 0, further incrementing the age of all edges of the first winner node, and deleting an edge that has reached a predetermined age; and the input node Is not inserted as a new node, a node weight update process for updating the weight vector of the first winner node based on the weight vector of the input node, and when the input node is not inserted as a new node, A first label weight update process for diffusing at least a part of the label weights to the first and second winner nodes; and a predetermined timing In the node deletion process for deleting a node according to the node density, and at the time of deleting a node in the node deletion process, at least a part of the label weight of the deleted node is diffused to nodes around the deleted node. The first label weight updating process, each node after inputting a predetermined number of the teacher data as a learned node, a recognizer generating process for configuring the recognizer by the learned node, and a feature amount from the input data According to the distance between the second feature amount extraction process to be extracted as a weight vector, the plurality of learned nodes of the recognizer generated by the recognizer generation process, and the weight vector extracted from the input data, A result output process for outputting a recognition result is executed by a computer.

  Thereby, it is possible to provide a program using a recognizer capable of online and additional learning.

  A robot apparatus according to the present invention includes an input data acquisition unit, and a recognition apparatus capable of transfer learning by recognizing a class to be recognized from the input data and recognizing the class by an attribute that is a characteristic of the class. The recognition device includes a recognizer generation device, a feature extraction unit that extracts a feature quantity from the input data as a weight vector, and the class and the attribute are attached to the recognizer generation device as label weights. According to the distance between the recognizer generated by inputting the teacher data and learning the feature amount, a plurality of learned nodes composed of the weight vectors of the recognizer and the weight vector extracted from the input data A result output unit that outputs a recognition result, and the recognizer generation device extracts the feature amount from the teacher data as a weight vector, and The extracted weight vector is used as an input node, the distance between the input node and each node is calculated, and the node closest to the input node and the second closest node are respectively the first winner node and the second winner node. And a node insertion determination unit that determines whether to insert the input node as a new node based on the distance between the input node and the first and second winner nodes, When the input node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. An edge management unit that increments the age of all edges of the first winner node and deletes an edge that has reached a predetermined age; and A node weight update unit that updates the weight vector of the first winner node based on the weight vector of the input node, and a node that does not insert the input node as a new node. A label weight update unit that spreads at least a part of the label weight to the first and second winner nodes, and a node deletion unit that deletes a node according to the node density at a predetermined timing, When the node deletion unit deletes a node, the label weight update unit diffuses at least a part of the label weight of the deletion node to nodes around the deletion node and inputs a predetermined number of the teacher data Each of the nodes is the learned node, and the recognizer is configured by the learned node.

  Thereby, it is possible to provide a robot apparatus that is online and capable of additional learning.

  According to the present invention, it is possible to provide an attribute learning and transfer system, a recognizer generation device, a recognizer generation method, and a recognition device that are online and can be additionally learned.

1 is a configuration diagram of an attribute learning and transfer system according to a first exemplary embodiment; 6 is a flowchart for explaining attribute learning and transfer according to the first exemplary embodiment; 3 is a flowchart for explaining unknown object detection according to the first exemplary embodiment; It is a conceptual diagram for demonstrating the whole structure and process of the attribute learning and transfer system concerning Embodiment 1. FIG. FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 6 is a diagram for explaining an effect according to the first embodiment; FIG. 10 is a diagram showing Adjusted-SOINN processing which is a premise of the second embodiment. FIG. 10 is a diagram illustrating a STAR-SONN process which is a premise of the second embodiment. It is a figure which shows the structure of the discriminator production | generation apparatus of Embodiment 2. FIG. It is a figure which shows the process of the discriminator production | generation apparatus of Embodiment 2. FIG. 6 is a diagram illustrating a configuration of an identification apparatus according to Embodiment 2. FIG. FIG. 10 is a diagram illustrating processing of the identification device according to the second embodiment. It is a figure which shows the discriminator production | generation apparatus and discrimination | determination apparatus process of Embodiment 2. FIG. FIG. 10 is a diagram for explaining an effect according to the second embodiment. FIG. 10 is a diagram for explaining an effect according to the second embodiment. FIG. 10 is a diagram for explaining an effect according to the second embodiment. FIG. 10 is a diagram for explaining an effect according to the second embodiment. FIG. 6 is a diagram illustrating a configuration of a robot apparatus according to a third embodiment.

<Embodiment 1. >
<Configuration of learning and transfer system>
Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a configuration diagram of an attribute learning and transfer system according to the present embodiment. The attribute learning and transfer system 1 includes a feature extraction unit 2, a labeling unit 3, a discriminator holding unit 4, a discriminator generating unit 5, an attribute discriminating unit 6, and a class discriminating unit 7. .

  The feature extraction unit 2 extracts features from teacher data during learning and from input data during recognition. For example, when the teacher data and the input data are image data, feature quantities such as SIFT and SURF are extracted. The teacher data and the input data are not limited to image data, and voice data, time series data such as a motor, and a combination of these data may be input. Here, the motor is mounted on the robot, and each joint of the robot is driven by the rotation of the motor. Such motor values are used as teacher data and input data. In addition, as feature amounts to be extracted when the teacher data and the input data are image data, known features such as SIFT feature amounts, SURF feature amounts, rg-SIFT feature amounts, PHOG feature amounts, cq feature amounts, Lss-histogram feature amounts, and the like are known. Any one of the feature amounts may be extracted, or a plurality of types of feature amounts may be extracted.

  The labeling unit 3 labels the given attribute information on the input data (teacher data). The attribute information is information that represents an attribute of the class of input data, and is given by a multi-value vector. In the present embodiment, teacher data and input data are used as images, and the attributes of the image class are labeled as binary vectors (attribute information). In the present embodiment, the labeling unit 3 corresponds to an attribute labeling module described later.

  The discriminator holding unit 4 holds the attribute discriminator generated by the discriminator generating unit 5 in storage means (not shown) such as a memory. The discriminator generation unit 5 generates an attribute discriminator used for identifying an attribute included in the input data. The discriminator generating unit 5 is configured using a self-propagating neural network including nodes and edges connecting the nodes, and the self-propagating neural network includes a plurality of parts according to identification contents identified by attributes. It is divided into In the present embodiment, as will be described later, a first part (positive part) indicating that an attribute is included and a second part (negative part) indicating that an attribute is not included. Divide a self-propagating neural network. When identifying three or more contents, the self-propagating neural network may be divided into three or more parts. Details of the self-propagating neural network will be described later.

  Further, the classifier generation unit 5 uses the feature of the teacher data extracted by the feature extraction unit 2 as a teacher pattern for the part of the self-propagating neural network specified by the attribute information labeled by the labeling unit 3. input. Then, the discriminator generation unit 5 generates nodes and edges based on the teacher pattern in the self-propagating neural network. The attribute information indicates in which part of the self-propagating neural network the teacher data and the input data should be input. The discriminator generation unit 5 uses the attribute information labeled by the labeling unit 3. It is possible to specify for which self-propagating neural network the teacher pattern should be input.

  The attribute identification unit 6 identifies an attribute included in the input data using an attribute identifier when input data is input. The attribute discriminating unit 6 inputs, as an input pattern, features extracted from the input data to each part of the self-propagating neural network of the attribute discriminator held by the discriminator holding unit 4. A first similarity with a node included in the self-propagating neural network is calculated in each self-propagating neural network, and any attribute of the identification content is input data according to the calculated first similarity. To identify whether it is included.

  In the present embodiment, as will be described later, one attribute discriminator that expresses one attribute is used to identify two contents of whether or not the input data includes the one attribute. When a plurality of attribute classifiers are used, an identification result indicating the presence or absence of the attribute is output from each attribute classifier, and it is identified whether or not the input data includes each of the plurality of attributes. .

  In this embodiment, the distance between the input pattern and a plurality of nearest nodes is calculated as the similarity between the input pattern and a node included in the self-propagating neural network. Furthermore, as shown in the number (4) described later, the first similarity calculated for the first part (positive part), the first similarity calculated for the second part (negative part), Are compared, the attributes included in the input data are identified in consideration of the relative magnitude relationship of these first similarities.

  The class identification unit 7 identifies a class including the input data based on the output from the attribute classifier for the input data. More specifically, the class identifying unit 7 is provided with attribute information included for each of the plurality of classes, and compares the attribute of the input data identified by the attribute identifying unit 6 with the attribute information of the class to obtain the second information. The similarity is obtained, and in accordance with the calculated second similarity, which class among the plurality of classes contains the input data is identified.

  Note that the attribute learning and transfer system 1 temporarily stores, for example, a central processing unit (CPU) that performs arithmetic processing, a read only memory (ROM) that stores arithmetic programs executed by the CPU, processing data, and the like. The hardware configuration is centered on a microcomputer comprising a RAM (Random Access Memory) or the like stored in the memory.

<Learning method of attribute classifier>
FIG. 2 is a flowchart for explaining attribute learning and transfer in the learning stage.
S101: The attribute learning and transfer system 1 initializes an attribute classifier.
S102: The feature extraction unit 2 extracts features from an input image that is teacher data.
S103: The labeling unit 3 labels attribute information on the teacher image.
S104: The classifier generation unit 5 inputs the input pattern of the teacher image to the corresponding part of the self-propagating neural network constituting the attribute classifier, and performs clustering.
S105: The attribute learning and transfer system 1 determines whether or not to continue the learning. If it is determined that the learning is to be continued, a new teacher image is input and the processes in and after S102 are repeated. If it is determined not to continue, the learning process (training of the attribute classifier using the teacher image) is terminated.

FIG. 3 is a flowchart for explaining unknown object detection in the recognition stage.
S201: The feature extraction unit 2 extracts features from an input image that is input data.
S202: The attribute identification unit 6 uses an attribute classifier to identify whether the input data includes an attribute represented by the attribute classifier.
S203: The class identifying unit 7 compares the output from the attribute classifier with respect to the input data and the attribute information of the class including the input data.
S204: The class identification unit 7 outputs the identification result of the class including the input data based on the comparison result.

<Adjusted-SOINN>
Next, the self-propagating neural network used in this embodiment will be briefly described.
As a self-propagating neural network, for example, Adjusted-SOINN (Self-Organizing Incremental Neural Network) has been proposed. Adjusted-SOINN is a self-organizing and additionally learning neural network, and is a mechanism for online unsupervised discrimination learning (Japanese Patent Laid-Open No. 2008-217242, non-patent document “F. Shen & O. Hasegawa”). , "An incremental network for on-line unsupervised classification and topology learning," Neural Networks, 19 (1): 90-106, 2006.) and non-patent literature "F. Shen & O. Hasegawa," An on-line learning mechanism for unsupervised classification and topology representation, "in CVPR, 2005.)).

Adjusted-SOINN is composed of a node corresponding to the input pattern and an edge connecting the nodes. Adjusted-SOINN starts with an empty set of nodes and first acquires two input data as the two nodes at the start. Then, for each input pattern ξ∈R ⁿ (n-dimensional vector space), the nearest node s ₁ and the second nearest node s ₂ are obtained by the following numbers (1) and (2). In the following numbers, A ′ is a set of all nodes in Adjusted-SOINN, and W _c indicates an n-dimensional weight vector of node c.

If the distance between the new input pattern and the first and second winner nodes (node s ₁ and node s ₂ ) is smaller than a predetermined threshold, the adjusted-SOINN uses the first input pattern as the first input pattern. Assign to the winner node. In other cases, Adjusted-SOIN determines that the input pattern is too different from the current node, and generates the input pattern as a new node.

When a new pattern is assigned to the closest node s ₁ in the Adjusted-SOINN, the Adjusted-SOINN updates the weight vector W _s1 with the value of the new input pattern. Also, an edge is generated between the first and second winner nodes (if no edge exists). Such processing is significantly different between other clustering methods such as K-Mean and Adjusted-SOINN. In Adjusted-SOINN, the newly entered pattern or data is not added directly to the network to form a cluster. Instead, a cluster is formed by connecting nodes existing in the Adjusted-SOINN using edges. Since a new node is generated only when the current node and the input pattern in Adjusted-SOIN are remarkably different, such cluster formation processing greatly contributes to memory saving for long-term execution. Bring.

  Another key feature of Adjusted-SOINN is that it gives the node some important properties. With this idea, Adjusted-SOINN allows a node to behave like an autonomous agent. Nodes have properties such as age and cumulative error. As a result, at any given time, each node has its own age and accumulated error of the node (obtained by accumulating the distance from the input pattern each time the node is selected as the first winner). Yes. Because of these properties, each node performs two behaviors: node death and its own splitting. With regard to death based on the age of a node, if the node exists for a long time without winning against any new input pattern (if it is a noise or unwanted node), it is connected to that node All edges gradually die out. In addition, with respect to splitting based on the accumulated error of the node, if the accumulated noise is too great, the node splits itself into two. This is similar to the behavior of K-Mean recursion on a large cluster. As described above, Adjusted-SOINN is a powerful learning tool for identification problems that can be learned online and additionally. In this embodiment, adopting Adjusted-SOINN inherits its main advantages and uses it. Change to be applicable to the attribute identification problem in computer vision.

  Note that the self-propagating neural network is not limited to Adjusted-SOINN, but may use Enhanced-SOINN (Japanese Patent Laid-Open No. 2008-217246). From the viewpoint of online and additional learning, although it is necessary to determine the network configuration and size in advance, its performance is limited. However, as a self-propagating neural network, neural gas (NG) (TM Martinetz, and SG Berkovich, and KJ Schulten, "Neural-gas," network for vector quantization and its application to time-series prediction, "IEEE Trans. On Neural Networks, vol. 4, no. 4, pp. 558-569 , 1996.) and Growing neural gas (GAG) (B. Fritzke, "A Growing Neural Gas Network Learns Topologies," In Advances in Neural Information Processing System, vol. 7, pp. 625-632, 1995.) You can also.

<AT-SOINN>
Hereinafter, the attribute learning and transfer system 1 based on Adjusted-SOINN, which is a self-organized and additionally learnable neural network, is referred to as AT-SOINN, and details of AT-SOINN for unknown object class identification will be described. However, in order to facilitate understanding, in the following, the problem to be solved by AT-SOIN will be briefly explained again, the method for generating the attribute classifier will be explained, and how the attribute classifier will be unknown. A description will be given of whether it is used for object recognition.

First, problems to be solved by AT-SOINN will be described.
The problem directed to AT-SOINN is almost similar to the problem described in Non-Patent Document 1. The only and important difference with regard to this problem is how the system is learned. In Non-Patent Document 1, a set of teacher images must be prepared in advance and input to the system at once. Each image of each class is labeled with a binary feature vector attribute. The system first learns the attribute classifier by SVM. (X ₁ , l ₁ ),..., (X _n , l _n ) ⊂X × Y are given as teacher samples. Where X is an arbitrary feature space, Y = {y ₁ ,..., Y _K } is K separated classes, and Z = {z ₁ ,..., Z _L } is Y is a set of test data of disjoint classes (a product set of set Z and set Y is an empty set). The main task here is to identify the input image X → Z for a label set Z = {z ₁ ,..., Z _L } that is completely disjoint from Y.

Even if the classes z ₁ ,..., Z _L are not given in the learning phase, if the attribute representation a is available for each class of z∈Z and y∈Y, Y and Z The attribute can be identified by transferring the attribute aεA existing between the two. Specifically, all the attributes are binary (so that the attribute expression a ^y = (a ^y ₁ ,..., A ^y _m ) for any teacher class y is a fixed-length binary vector. binary value). Learning process begins by learning the probabilistic classifier for each attribute a _m. All images from the teacher class set Y are used as labeled teacher samples, and attribute labels that match the sample labels are entered to determine their labels (ie binary labels for class y samples) a ^y _m is assigned). The learned attribute classifier gives an estimate of p (a _m | x). p _(a m | x) is, p is a model for the layer of the complete image attributes as _{| | (x a m) (} a x) = Π M m = 1 p. Here, M is the total number of given attributes. This estimation term is used to calculate the posterior distribution of the class when an image is given, as shown in the following equation (3).

The technique (AT-SOIN) proposed in the present embodiment solves the problem described in Non-Patent Document 1 described above, and further solves the problem by satisfying the following two conditions. To do.
(Condition 1: Online learning of attributes and realization of transfer) A set of teacher images is not prepared in advance. The system does not know the size of the set of teacher images. From the point of view of use for a robot, the system gradually acquires one teacher image labeled with a class index and a representation of attributes of such a class.
(Condition 2: Realization of learning and transfer of attributes that can be additionally learned) The system can stop and identify any time an input image is input, and a new teacher image can be used in the same way. The learning process can be resumed at any time.
Under these conditions, the method of learning SVM for identifying each attribute becomes unrealistic. Therefore, satisfying these two conditions and solving the problem is the main contribution of this embodiment.

<Method for generating attribute classifier>
Next, a method for generating an attribute discriminator will be described.
Clearly, an effective classifier for each attribute is still needed to obtain high performance in detecting unknown object classes. SVM cannot answer the requirements of online add-on systems. For this reason, in this embodiment, an attribute classifier is configured using Adjusted-SOINN, which is a self-propagating neural network, instead of SVM. However, some important corrections are required for the application of Adjusted-SOINN.

  Basically, Adjusted-SOINN itself operates like a clustering tool that can be additionally learned online. Adjusted-SOINN can be used for multi-class identification, but the main task in the present embodiment is to answer whether or not the image includes an individual attribute a. This is therefore a binary identification problem (ie, identifying whether the image contains attribute a or not). Furthermore, in this embodiment, the number of classes identified using Adjusted-SOINN is fixed; only two classes, positive (+) class and negative (-) class, are considered (the positive class has an image attribute) ) And the negative class does not contain that attribute.

  Therefore, in order to realize such attribute identification, the attribute-identified Adjusted-SOINN is divided into two parts (positive part and negative part). In each part, nodes and edges are generated based on the input pattern by the same process as the original Adjusted-SOINN, and clusters are additionally grown.

  Since one Adjusted-SOINN is required for one attribute discriminator, a total of M Adjusted-SOINNs are required to identify the M attributes of the image. Similar to the content described in Non-Patent Document 1, this number M is required only for one feature space. For this reason, if Q feature values are used, a total of M × Q Adjusted-SOINNs are required. For example, in an experiment using the data set described in Non-Patent Document 1, 85 × 6 Adjusted-SOINNs are required for attribute identification in six different feature spaces. The number of Adjusted-SOINNs according to the present embodiment is exactly the same as the number of SVMs described in Non-Patent Document 1, but Adjusted-SOINN can operate at high speed with less memory consumption, and is most important. As a matter of course, a learning process that enables online and additional learning is performed.

FIG. 4 is a conceptual diagram showing the overall configuration and processing of AT-SOINN.
First, in the learning stage, in an additional method, teacher images from a teacher class (Training Classes Y) are gradually input to M SOINN-based Individual Attribute Classifiers (SOINN). The teacher image is input to each Adjusted-SOINN through a labeling process by an attribute labeling module (Labeling Attributes).

  The attribute labeling module according to the present embodiment is for a system used by a robot. The administrator (here human) labels the attributes of each image class through this module. In the present embodiment, it is assumed that the attribute label is obtained directly in the same manner as the method described in Non-Patent Document 1.

  The teacher image is distinguished as a positive sample (Positive Sample) or a negative sample (Negative Sample) according to the label (attribute information) made through the attribute labeling module, and is input to the corresponding part of each Adjusted-SOINN. . Learning of the attribute classifier by inputting the teacher image can be additionally executed. Also, a new attribute identifier can be simply added by generating a new Adjusted-SOINN.

Whenever it is desired to recognize with AT-SOINN, an input image of an unknown image class is input to AT-SOINN. The input image x is expressed by a feature vector and input to all individual Adjusted-SOINNs. For each Adjusted-SOINN (attribute discriminator) of attribute m, a set of k nearest neighbor nodes is obtained from both the positive part and the negative part (k nearest neighbor nodes S ⁺ from the positive part). _{^{m = {s + m, 1}} , ..., s + m, k} is, k number of nearest node from the negative portion _{^{S - m = {s - m}} , 1, ..., s - m, k } Is obtained). Only when the following number (4) is true, the input image x is considered to include the attribute m. Here, ξ is an input pattern (feature vector of the input image x), and W ⁺ _{m, j} is a weight vector of the node s ⁺ _{m, j} . An appropriate value is set in advance as a predetermined threshold T by the user. When the following number (4) does not apply, the input image x is treated as not including the attribute m.

<Recognition method>
Next, how to use the attribute classifier for unknown object recognition will be described.
With the availability of an identifier for each attribute, AT-SOINN can identify unknown objects in unknown classes whenever necessary. Identification of unknown objects can be performed by a very simple method. Basically, for both Adjusted-SOINN and SVM, the output of the attribute classifier is in the metric space between the vector of the input image and the representative vector. For SVM, an extra teacher image to perform Platt scaling (JC Platt, “Probabilities for SV machines,” in Advances in Large Margin Classifiers. MIT Press, 2000.) that transforms the space into a probability space. It is necessary to prepare a set of Unfortunately, under the conditions assumed by AT-SOIN that a set of teacher data cannot be obtained in advance, the technique of preparing an extra set of teacher images is unrealistic. Therefore, in this embodiment, the unknown object is identified by simply considering the metric space between the feature vector of the input image and the nearest neighbor class of Adjusted-SOINN.

For a two-part Adjusted-SOINN (positive part and negative part) representing a classifier of attribute m, the average distance between the current input image and the k nearest neighbors is d ⁺ _m and d ⁻ , respectively. is given as _m . The input input image is assigned to the unknown target class c based on the following number (5). Here, s ^q _zl is a score of the class z _l in the space of the feature q among the Q features in total, and is calculated based on the number (6) described later.
The similarity score s _zl of each feature space q is obtained by the following number (6). Here, when the condition P is true, [P] = 1, otherwise [P] = 0 (where [P] is described using square brackets. However, correctly, [P] is described using double square brackets as shown in number (6)). All unknown object classes z are assumed to have their attribute vectors a ^z generated by a deterministic approach. This deterministic approach is obtained from Iverson's bracket notation (DE Knuth, "Two notes on notation," Amer. Math. Monthly, 99 (5): 403-422, 1992.). Specifically, in the present embodiment, in the recognition stage, an attribute m expressed by a binary value with respect to the unknown object class z _l is given, and therefore when the value of a ^zl _m is 1 (a ^zl _m = 1), the value of [a ^zl _m = 1] is 1, the value of [a ^zl _m = 0] is 0, and when the value of a ^zl _m is 0 (a ^zl _m = 0) The value of [a ^zl _m = 1] is 0, and the value of [a ^zl _m = 0] is 1.

<Example>
Next, the result and experiment by this Embodiment are demonstrated.
For comparison with the conventional method, an animal data set with attributes described in Non-Patent Document 1 is used. This data set contains 85 attributes for 50 animal classes. In order to focus only on the performance of the learning mechanism, in this embodiment, instead of using the coarse image data set as it is, an experiment is performed using the feature data set extracted in advance in Non-Patent Document 1. Do. Six features can be used for this data set (SIFT, SURF, PHOG, rgSIFT, local self-similarity histograms (LSS), and RGB Color Histogram (CQ). See).

  AT-SOIN can perform additional learning, but here, in order to compare with the results of Non-Patent Document 1 where additional learning is impossible, recognition of performance when 40 image classes have already been learned. To do. That is, out of 50 animal classes, 40 image classes are used as teacher data, and the remaining 10 image classes that are disjoint from the 40 image classes are used for AT-SOINN test (recognition). Data). Since this condition is exactly the same as that described in Non-Patent Document 1, it is possible to make a fair comparison between AT-SOIN and the method described in Non-Patent Document 1.

In the experiment, the adjusted-SOINN parameters are set as age _dead = 100 and λ = 250. For the other parameters, the same settings as those described in the original Adjusted-SOIN document described above are used. These Adjusted-SOINN parameters are selected to achieve the best results with AT-SOINN. Experiments are tried with various values settings (setting λ = 150, 250, 350, 450, 550, age _dead = 50, 100, 150, 250), but these results are not very different. However, setting too small values for λ and age _dead will lead to too frequent node removal processing, and at the same time, existing nodes will die too quickly before getting enough input. , Adjusted-SOINN will not be able to generate clusters. Therefore, as long as a too small value (for example, λ = 25, age _dead = 10) is not set as a parameter, a difference in parameter value does not cause a significant decrease in the identification result.

First, the quality of the AT-SOINN attribute discriminator will be described.
FIG. 5 shows the quality of individual attribute classifiers of AT-SOINN when compared with the method described in Non-Patent Document 1. Although the results of AT-SOINN are lower than the method described in Non-Patent Document 1 for some attributes, as shown in FIG. 9 to be described later, if dramatically reduced calculation time is considered, AT- The quality of the SOINN attribute discriminator is sufficiently high.

  From the viewpoint of calculation time required for learning of the attribute classifier, AT-SOIN is approximately 300 in learning using 40 image classes (24294 teacher images) to learn each classifier for each feature space. Requires seconds. On the other hand, the method described in Non-Patent Document 1 requires several hours to learn each classifier for each feature space. Therefore, the method described in Non-Patent Document 1 requires a total of 100 hours or more in order to learn the classifiers of all the attributes for only one feature space. Further, unlike other conventional methods, AT-SOIN can be additionally learned completely, so that it is possible to additionally input a new input image without restarting the entire learning mechanism.

Next, as an AT-SOIN experiment, an additional online learning of 85 attribute predictors using 40 classes of animals was used to test (recognize) another 10 disjoint classes of animals. I do.
FIG. 6 shows the result of the confusion matrix between the 10 classes. The average accuracy of the multiclass is 28,92%. This performance is clearly much higher than the 10% chance level.

  FIG. 7 shows a comparison between AT-SOINN and the two methods DAP (Direct attribute prediction) and IAP (Indirect attribute prediction) described in Non-Patent Document 1.

  Here, in order to show that the data set selected for evaluation is difficult for unknown object detection, other results described as a baseline technique in Non-Patent Document 1 are also referred to. In Non-Patent Document 1, using the same data set with the same settings, in addition to the two methods of DAP and IAP, there are two more baselines ((i) a simple one-time learning approach, The system learns a diagonal covariance matrix of feature variances from 40 teacher classes and uses the resulting Mahalanobis distance for nearest neighbor identification. (Ii) Standard commonplace. Multi-class identification, where half-image segmentation is given to teacher and recognition data (input data), randomly selected from a set of recognition data by initial baseline (one-time learning) Each target class is represented by a maximum of 10 images).

  However, in the initial baseline, even if several samples (1 to 10 samples) are given, the average accuracy is only 14.3% for one teacher image per class, For 10 teacher images per class, it is only 18.9% average accuracy. This accuracy does not clearly improve beyond the 10% chance level. Also, the second baseline (standard multi-class identification), despite the assumption that many teacher samples from the same class are available, greatly simplifies the problem. The baseline can only achieve multi-class accuracy of 65.9%.

  FIG. 8 is a table summarizing the results of comparison between AT-SOINN and other baselines. Recognition was done in 10 animal classes. The results of the standard method (Ordinary Method) and the one-time learning method (One-shot Learning) are described in Non-Patent Document 1, and the results of these methods are obtained directly from the contents of Non-Patent Document 1. Yes. With respect to unknown object detection, only the last three techniques out of the five comparable techniques can detect without the need for any sample from the class of recognition object. Among them, only AT-SOINN can perform additional learning. Furthermore, AT-SOIN can learn in a much shorter time than required by DAP and IAP. Regarding DAP and IAP, although the learning time required for learning of the SVM (identifier) for each attribute is not clearly shown, it has been confirmed that several hours are required for learning of one SVM. .

  As shown in FIG. 8, based on a comparison with the obtained baseline, even if the one-time learning approach does not require any samples from the class to be recognized, AT-SOINN Exceeds its performance. Further, AT-SOIN exceeds even the IAP method of Non-Patent Document 1. In comparison between the AT-SOINN and the DAP method, the AT-SOINN shows lower performance in terms of accuracy. However, AT-SOIN can additionally process images without the need for a prior teacher data set, and more importantly, with much less learning time than IAP and DAP. I think that the difference in accuracy is not big enough to be discouraged. In addition, AT-SOIN is an algorithm capable of additional learning, and the identification rate can be improved by adding learning data. Further, it is possible to add attributes, thereby improving the identification rate. Conventional transfer learning methods such as DAP do not have functions for adding learning data and adding attributes.

  Further, the attribute identification result shown in FIG. 5 described above includes other interesting points. Considering the quality of the individual attribute classifiers evaluated by the area under the ROC curve (AUC), the results in FIG. 5 show the AT-SOIN performance (0.68) and the DAP performance (0.72). There is no significant difference between and. One of the reasons why the accuracy of object identification by AT-SOINN (28.9%) is lower than that of DAP (40.5%) is that AT-SOIN makes the output space from coarse distance to probability. I think that there is no extra teacher data set suitable for conversion.

FIG. 9 shows the calculation time required for learning each attribute classifier for each feature.
For the same amount of teacher images (40 classes, 24294 images, 6 features), the DAP and IAP methods require several hours to learn SVM for one attribute of one feature. On the other hand, since AT-SOIN requires only about 200 to 500 seconds, the calculation time required to learn one attribute classifier can be dramatically reduced. In addition, since the AT-SOIN learning mechanism is online and additional learning is possible, teacher images can be gradually input to the system.

  As described above, according to the present invention, attribute transfer and learning can be realized by an online and additional learning approach. The success of this approach has a significant impact on the robotics community, especially with respect to robotics operations.

  By virtue of the baseline baseline described above, it is clear that the data sets that can be processed by the present invention are much more difficult than those found in robotics. This data set does not have any segmentation annotations. All features are extracted from the entire image. Some images may include the target animal in its main part as illustrated in FIG. 10, but many images include a background in its main part as illustrated in FIG. It is out. This is why the baseline method (standard method) can only achieve 65.9% accuracy in the recognition test after dividing the data set used for the recognition test in half by the teacher. To clarify.

  In addition, although the above-described experiment shows results for a fixed number of attributes, AT-SOINN can be executed even in a situation where the number of attributes is not fixed. In AT-SOINN, a new attribute classifier can be added by simply adding Adjusted-SOINN.

  Specifically, the AT-SOIN currently has, for example, 85 × 6 Adjusted-SOINN (85 attribute identifications in 6 feature spaces), and some images of each learned class Assume the case of storing (for example, 100 images for each class). In this case, when the user attempts to add a new attribute, the user simply labels one or more attributes (0 or 1) for each class. Therefore, the AT-SOIN can generate a new attribute classifier by using the same amount of teacher images, and can identify 86 attributes.

  The technique for adding attribute discriminators is by adding the same attribute label to each class and reusing the teacher image to learn the SVM for the new discriminator, by the traditional method of the past. May be thought of as something that can be done easily. This idea is correct if calculation time is not taken into account. However, for online addi- tional use in robotics, it is not very reasonable to sacrifice hours to add one or more attributes. In AT-SOINN, using the same technique, additional new attributes can be learned in as little as 38-40 minutes. Furthermore, if the number of features used is fixed, the hardware can be configured to run AT-SOIN in parallel, which reduces the time required for learning to only 10 minutes. Can be reduced to

  In robotics, starting from a well-prepared scene for learning allows the robot to learn attributes more accurately. By learning attributes by a method that can be additionally learned online, the robot can gradually acquire more teacher images while providing services to the user. As a result, the present invention ultimately leads to the imagination ability in robotics, and the robot can go to unknown objects based only on verbal explanations from humans.

<Embodiment 2.>
In the first embodiment described above, attributes are treated as negative and positive binary values. However, if the attributes can be regarded as multi-values, identification can be performed more flexibly. Therefore, as a result of intensive experiments conducted by the inventors of the present application, a method for generating a discriminator capable of discriminating multi-value attribute information was found. In the following description, an algorithm that can generate a discriminator using this multi-valued attribute information is referred to as STAR (STATISTICAL RECOGNITION) -SOINN.

  STAR-SOINN is an improvement of the above-described Adjusted-SOINN (Self-Organizing Incremental Neural Network).

<Adjusted-SOINN>
Here, in order to facilitate understanding of STAR-SOINN, the above-described Adjusted-SOINN will be described in more detail.

  Adjusted-SOINN is a self-propagating neural network that can be additionally learned online. Adjusted-SOINN is characterized in that nodes represented as weight vectors autonomously proliferate and disappear. Nodes are connected by virtual lines called edges when predetermined conditions are satisfied. Adjusted-SOINN performs clustering by regarding nodes that can reach each other by following this edge as the same cluster. Also, the edge has a parameter called age, and Adjusted-SOINN deletes an edge that has reached a predetermined age. Thereby, a node that can be regarded as noise can be deleted at a predetermined timing.

  The adjusted-SOINN learning algorithm, that is, the operation of Adjusted-SOINN when a new node is input to Adjusted-SOINN will be described with reference to FIG.

  S301: An input node having a weight vector is newly input to Adjusted-SOINN.

S302: Adjusted-SOINN calculates the distance between the input node and the existing node, typically the Euclidean distance. The Euclidean distance is a distance defined by Equation 7. In Equation 7, d is an Euclidean distance, and f and g are n-dimensional vectors, respectively.

  Adjusted-SOINN determines the node (first winner) with the shortest Euclidean distance and the second node (second winner) with the closest Euclidean distance from this calculation result.

S303: Adjusted-SOINN calculates the similarity threshold of the first winner node and the second winner node, respectively. Here, when a certain node has an adjacent node, the similarity threshold is a maximum distance from the adjacent node. When a node does not have an adjacent node, it means the minimum distance between that node and other nodes. The similarity threshold is obtained by Equation 8. In Equation 8, N is a set of adjacent nodes of node i, W is a weight vector of node i, and A is a set of all nodes.

  Adjusted-SOINN compares these similarity thresholds with the Euclidean distances between the input node and the first and second winners. As a result of the comparison, when the Euclidean distance between the input node and the first winner and the second winner is larger than the similarity threshold of the first winner or the second winner, the input node is different from the first winner and the second winner. Considered to belong to the cluster. In this case, Adjusted-SOINN determines that a new node should be inserted at the position of the input node. On the other hand, when the Euclidean distance between the input node and the first winner and the second winner is smaller than the similarity threshold of the first winner and the second winner, the input node, the first winner, and the second winner are all in the same cluster. Considered to belong to. In this case, Adjusted-SOIN determines that a new node should not be inserted.

  S304: If it is determined in S303 that a new node should be inserted, Adjusted-SOINN inserts a new node at the position of the input node.

  S305: If it is determined in S303 that a new node should not be inserted, Adjusted-SOINN determines whether an edge exists between the first winner and the second winner.

  S306: If it is determined in S305 that no edge exists between the first winner and the second winner, Adjusted-SOINN generates an edge between them.

  S307: Adjusted-SOINN sets the age of the edge generated in S305 to 0 when it is determined in S305 that no edge exists. On the other hand, if it is determined in S305 that an edge exists, the age of the edge that has already existed is set to zero. In addition, Adjusted-SOINN increments the age of all edges connected to the first winner.

  S308: Adjusted-SOINN deletes an edge having an age exceeding a predetermined threshold (age). “age” is a parameter set to delete an edge that is erroneously generated due to the influence of noise or the like. Setting a small value for age makes it easier to remove edges and prevents the effects of noise. However, if age is extremely small, edges are frequently deleted, resulting in unstable learning results. . On the other hand, if age is too large, edges generated due to noise cannot be removed appropriately. Therefore, it is desirable to set an appropriate value calculated through experiments in age.

S309: Adjusted-SOINN updates the weight vector of the first winner and the adjacent node directly connected to the first winner via the edge by the following formulas (9) and (10). In Equations 9 and 10, ΔWi represents the update amount of the weight vector of node i, and ΔWj represents the update amount of the weight vector of node j. Further, i is the first winner, j is the adjacent node, Wk is the weight vector of the input node, and Mi is the number of times node i has been the first winner so far. As a result, the input node is absorbed by the first winner and adjacent nodes.

  S310: Adjusted-SOINN extracts nodes satisfying the following two conditions from all the nodes, and determines to be deleted.

  The first condition determines whether or not the number of input nodes corresponds to a predetermined setting value, for example, a multiple of a constant λ. This set value is a parameter that is set in order to periodically delete nodes that can be regarded as noise. If a small value is set for λ, noise processing can be performed frequently. However, if λ is extremely small, even nodes that are not actually noise are erroneously deleted. On the other hand, if λ is too large, a node generated due to the influence of noise cannot be removed appropriately. Therefore, it is desirable to set λ to an appropriate value calculated by experiment.

  The second condition is that the number of adjacent nodes of the node is equal to or less than a predetermined threshold value η. The threshold value η is a parameter for defining a low density region of the node group, that is, a node that can be regarded as noise.

  S311: Adjusted-SOINN deletes the node extracted as the deletion target in S405.

  S312: Adjusted-SOINN completes learning when the number of input nodes reaches a predetermined constant ρ. If not reached yet, the input of the next input node is accepted and the learning is continued by the above-described procedure.

<STAR-SOINN>
Next, STAR-SOINN will be described.

  As described above, STAR-SOINN is an improved version of Adjusted-SOINN. The main differences between STAR-SOINN and Adjusted-SOINN are shown below.

  STAR-SOINN aims to improve the recognition rate and reduce the amount of information by applying an extension for incorporating statistical information to Adjusted-SOINN. In order to realize this, in STAR-SOINN, some changes are made to the weight vector management method such as addition and deletion of nodes. In addition, the concept of a label, which is information added to a node, is introduced. STAR-SOINN can hold, for example, information such as a class to which the node belongs and attributes provided as a label. In addition, as this attribute value, it is possible to hold a multi-value (not a binary value of 0 or 1 but a continuous value). Note that the label is added to the node in advance by some method when the node is input to the STAR-SOINN.

  Here, the class indicates an identification target of the classifier. If the classifier identifies animals such as lions or tigers, pandas or bears, they are classes. The attribute indicates the characteristics of the class. If the name of the animal is a class, the attribute indicates information such as whether or not the carnivorous, the herbivorous, or the long hair. In addition, when the identification target (class) is a room such as a living room, a bathroom, or a bedroom, the room name is a class. In this case, the attributes are whether there is a table, a bed There is information such as whether there is a bathtub or a bathtub.

  The label is information on the class and attribute of the node given to each node. In Embodiment 1 described above, one AT-SOINN can handle only one attribute of one class as a label. In other words, one AT-SOINN learns only the attribute of “meat”, for example, of the class “lion”, for example. On the other hand, in STAR-SOINN, all attribute information (for example, 85 attributes) of one class “Lion” can be assigned as labels. For example, for each of a plurality of attributes such as “meat” and “herbivore” possessed by the class “lion”, “meat” is “1” (positive), and “herbivore” is “0” ( It is possible to assign a value such as negative). In the present embodiment, description will be made assuming that data of all attributes of one class is given as a label.

  The label value can be given as a binary value of “0” and “1” as in the above example, but can also be a continuous value, for example, a normalized value of 1 to 0. . For example, in the case of an animal that sometimes eats meat, the attribute “carnivore” is not simply set to “1” (positive), but the attribute “carnivore” is defined as “40% related”. be able to. In this way, it is possible to define the attribute with high accuracy by corresponding to the continuous value input. For example, when expressing an omnivorous animal such as a human being, it is necessary to give attribute values as binary values of “0” and “1” in the case of the above-described AT-SONN, so that “carnivorous animals” and “herbivores” Both attributes of “animal” had to be positive. On the other hand, since STAR-SOINN can handle continuous values as attribute values, for example, both attributes can be set to 0.5. By setting in this way, omnivorous humans can be learned separately from the animals that are originally meant by “carnivores”. Note that STAR-SOINN is also used in the same manner as AT-SOINN by setting both attributes to 100% and binarizing attribute values that are continuous values before and after a predetermined threshold. It is also possible. The attribute value does not need to be normalized, and the data value as it is may be used.

  Note that the label corresponding to the continuous value is very useful not only at the time of learning described here but also at the time of recognition described later. For example, an unrecognized animal given as a recognition target can output an ambiguous recognition result such as “eating a little meat” because the attribute values are continuous. Due to such characteristics, STAR-SOINN can provide a recognizer that can simulate human recognition and senses.

  In STAR-SOINN, it is possible to easily add attributes. Specifically, by increasing the labels given to the input nodes, it is possible to increase the attributes possessed by SOINN. Unlike AT-SOINN described above, the increase or decrease in attributes does not affect the number of SOINNs in STAR-SOINN. Further, STAR-SOINN can execute such an attribute addition operation online. Therefore, it is possible to flexibly add a new attribute even during operation of the apparatus. Thus, STAR-SOINN has an online learning property suitable for a robot working in a human living environment that requires learning and recognition in accordance with the environment and the demands of the commander.

  Next, a learning algorithm of STAR-SOINN, that is, the operation of STAR-SOIN when a new node is input to STAR-SOINN will be described with reference to FIG.

  S401: An input node having a weight vector and a label of the input node are newly input to STAR-SOINN.

  S402: STAR-SOIN increments all ages of existing nodes.

  S302: STAR-SOINN determines the first winner node and the second winner node for the input node in the same manner as Adjusted-SOINN.

  S303: STAR-SOINN calculates the similarity threshold values of the first winner node and the second winner node, respectively, similarly to Adjusted-SOINN, and uses the same cluster for the input node, the first winner, and the second winner. It is judged whether it belongs to.

  S403: If it is determined in S303 that the clusters are not the same, a new node is generated at the same position as the input node. Here, in STAR-SOINN, it is desirable to set the label weight of the input node for a new node. The label weight is a label value given to a node. That is, if the class name and attribute value are assigned to the input node as labels, the class name and attribute value are assigned to the new node as labels.

  S305 to S309: When the same cluster is determined in S303, STAR-SOINN generates an edge between the first winner and the second winner, and sets the age of the edge to 0, similarly to Adjusted-SOINN. If an edge already exists, the age of the edge is set to zero. In addition, the age of all edges connected to the first winner is incremented. Thereafter, an edge having an age exceeding a predetermined threshold (age) is deleted. Then, STAR-SOINN updates the weight vector of the first winner and its neighboring nodes.

S404: STAR-SOINN spreads the label information of the input node. That is, based on the label weight of the input node, the label weights of the first winner and its adjacent node, and the second winner node and its adjacent node are updated. The label weight can be updated according to the equations 11 and 12, for example. In STAR-SOINN, the recognition rate is improved by storing the label information of the input node as statistical information in STAR-SOINN.

  S405: If a new node is generated in S403 described above, and if the label weight is diffused in S404, the process proceeds to step S405.

  STAR-SOINN extracts nodes satisfying the following two conditions from all the nodes and determines that they are to be deleted.

  The first condition is that the age of the node corresponds to a predetermined set value, for example, a multiple of a constant λ. This set value is a parameter that is set in order to periodically delete nodes that can be regarded as noise. If a small value is set for λ, noise processing can be performed frequently. However, if λ is extremely small, even nodes that are not actually noise are erroneously deleted. On the other hand, if λ is too large, a node generated due to the influence of noise cannot be removed appropriately. Therefore, it is desirable to set λ to a value obtained by experiments, for example. Note that λ in STAR-SOINN has a different meaning from λ in Adjusted-SOINN. In Adjusted-SOINN, the number of nodes input in the past is evaluated by λ and the deletion node is extracted. However, in this method, a node having a relatively short time after generation is likely to be deleted. On the other hand, in STAR-SOINN, the concept of age is introduced to each node, and the age of each node is evaluated by λ instead of the number of input nodes, and the deleted node is extracted. That is, when each node reaches a certain age, it is determined whether or not to delete. As a result, noise can be removed without being affected by the node generation timing.

  The second condition is that the number of adjacent nodes (number of nodes connected by an edge) is equal to or less than a predetermined threshold η. The threshold value η is a parameter for defining a low density region of the node group, that is, a node that can be regarded as noise. Η represents an integer of 0 or more. Depending on the class to be identified or the number of teacher data used for learning, an optimal η may be set by experiments or the like.

  As described above, for example, when λ = 100 and η = 2 are set according to the two conditions, a node (adjacent node) is connected to an edge that is a multiple of 100 with an edge connected to two or less nodes. Nodes having 2 or less) are deleted.

  S311: STAR-SOINN deletes the node extracted as the deletion target in S405.

S406: STAR-SOIN transfers at least part of the label weight of the deleted node to nodes around the deleted node (label diffusion). The assignment of the label weight can be performed, for example, by updating the label weights of the node closest to the deleted node and the second closest node according to Equations 13 and 14, respectively. In Expressions 13 and 14, ΔL is an increase amount of the label weight. Also, Κ is the label information of the deleted node, c is the attribute, Kc is the label weight of the attribute c of the deleted node, and Tω and Tsω are the similarity thresholds of the node closest to the deleted node and the second closest node, respectively. Dω and Dsω are defined by Equations 15 and 16, respectively. In Equations 15 and 16, Wω and Wsω are the weight vectors of the node closest to the deleted node and the second closest node, respectively, and Wd is the weight vector of the deleted node.

  In the STAR-SOINN, label diffusion is performed in this manner, whereby label information is accumulated as statistical information in the STAR-SOINN, and the recognition rate can be improved.

  S312: If the number of input nodes reaches a predetermined constant ρ, STAR-SOIN completes learning. If not reached yet, the input of the next input node is accepted and the learning is continued by the above-described procedure.

<Configuration of recognizer generator>
Next, the configuration of the recognizer generation device 100 according to the present embodiment will be described with reference to FIG. The recognizer generation device 100 is typically realized by a computer such as a dedicated computer or a personal computer (PC).

  The constituent elements 101 to 107 of the recognizing device generating apparatus 100 have a function of executing various controls based on various programs stored in a storage means (not shown), etc., and a central processing unit (CPU), a read-only memory (ROM), a random access memory (RAM), an input / output port (I / O), and the like.

  The feature extraction unit 101 extracts feature amounts from input data (teacher data). A process of inputting this feature quantity to the STAR-SOINN as a weight vector (input node) together with a label described later is performed.

  As the teacher data, for example, arbitrary information input from various sensors including an image sensor can be used. In this embodiment, the case where animal image information is used as the teacher data is mainly exemplified. When the teacher data is image information, a feature amount can be extracted from the image information using a known technique such as SIFT, SURF, HOG, Haar-like, and the like.

  Note that as teacher data (input data), not only images but also voice and various sensor information can be input. For example, when there is an attribute of “a scary animal”, it is very difficult to determine the value of such an attribute (a numerical value indicating the degree of “a scary animal”) from the characteristics of the image. In such a case, if voice information such as the cry of the animal is used as input data, it is considered that the recognition can be performed with higher accuracy than the image. Furthermore, if there are attributes such as “soft skin” and “gritty surface”, the information of the pressure sensor that can directly touch the animal is also used, so that these attributes that are difficult to understand in the image can be accurately detected. It can be handled well. The method of recognizing an object using such various senses (sensor information) is very close to the recognition performed by humans. For this reason, this embodiment is considered to be very useful if applied to a robot that can be active in the same way as a human being. A robot can acquire the same concept as a human by installing a sensor (sensor) similar to a human. This makes it possible for robots to handle vague and complex attributes that humans recognize when receiving commands from humans, for example, thus promoting efficiency and simplification of interaction between humans and robots. be able to.

  Further, in the present embodiment, it is assumed that at least an attribute of the teacher data is given to the input teacher data as a label. In addition to this, information on a class to which the teacher data belongs may be held as a label. The label is typically given in advance by hand, but may be automatically given by a labeling unit (not shown) according to a predetermined algorithm.

  As described above, the attribute refers to a value representing the property or characteristic of the class indicating the identification target. For example, if the teacher data is animal image information, a plurality of attributes such as brown, large, long hair, and carnivorous are given as labels together with attribute values indicating the degree according to the class of the teacher data. The In the present embodiment, this attribute value is multivalued. In this way, since the attribute value can be handled as a continuous value in STAR-SOINN, not only the label is manually input, but also, for example, a value obtained by normalizing data input from the sensor can be used as it is. Is possible.

  Here, usually, different classes have different attribute types, combinations thereof, attribute value combinations, and the like. The performance of the discriminator varies depending on what attributes are set in the class to be identified and how many attributes are set. That is, if a large number of attributes are set, the class can be expressed in more detail, but the calculation amount increases. Also, if the number of attributes is too small, the identification capability will be reduced. By setting an appropriate number of optimal attributes for identifying the class, a higher performance recognizer can be generated.

  The winner node extraction unit 102 calculates the distance between the input node and each existing node when the weight vector (input node) is input to the STAR-SOINN, and the node closest to the input node and the second A process of extracting close nodes as a first winner node and a second winner node is performed.

  The node insertion determining unit 103 performs processing for determining whether to insert the input node as a new node in the STAR-SOINN based on the distance between the input node and the first and second winner nodes.

  The edge management unit 104 performs processing related to edge generation and deletion. Specifically, when the input node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. Let the age be 0. Also, the ages of all the edges of the first winner node are incremented, and the edges that have reached a predetermined age are deleted.

  When the input node is not inserted as a new node, the node weight update unit 105 performs a process of updating the weight vectors of the first winner node and the second winner node based on the weight vector of the input node. In this embodiment, the weights of the first and second winner nodes are described as being updated. However, only the weight vector of the first winner node may be updated.

  When the input node is not inserted as a new node, the label weight update unit 106 spreads the label weight of the input node to at least the first winner and the second winner, and when deleting the node, the label weight of the node A process of updating the label weight of a node existing around the deleted node is performed on at least a part of the node.

  When a node reaches an age that is a multiple of λ, the node deletion unit 107 determines the node density of the node, that is, how many nodes are connected to the node by an edge, etc. In response, the node is deleted.

  Note that the recognizer generation device 100 may include an attribute optimization unit (not shown). The attribute optimization unit can delete an attribute unnecessary for the node at a predetermined timing. The predetermined timing may be, for example, when the age of the node in the STAR-SOIN reaches a predetermined age, or when the attribute of the input node is spread to the first winner node and the second winner node. May be. The unnecessary attribute may be, for example, an overlapping attribute or an attribute whose attribute value is lower than a predetermined threshold. Alternatively, for an attribute whose attribute relevance is lower than a predetermined threshold, the effectiveness as an attribute may be calculated by a predetermined evaluation formula or the like, and an attribute with a low effectiveness may be determined as unnecessary. In addition, if the upper limit number of attributes is set and the number of attributes exceeds the upper limit because a new attribute has been added, the attribute with the lowest attribute value is selected from all attributes. May be.

  By reducing the attributes in this way, it is possible to suppress over-learning, shorten the recognition time, reduce the amount of information, and the like. In addition, when the attribute is deleted, it is desirable to determine by the recognition work that the recognition rate does not decrease. Specifically, it is possible to detect attributes similar to the attribute to be deleted and quantitatively evaluate them to determine whether deletion is possible.

  Unlike AT-SOINN described above, the increase or decrease in attributes does not affect the number of SOINNs in STAR-SOINN. Also, STAR-SOINN can execute such attribute deletion work online. Therefore, it is possible to flexibly reduce new attributes even during operation of the apparatus. That is, STAR-SOINN has an online learning property suitable for a robot working in a human living environment that requires learning and recognition in accordance with the environment and the demands of the commander.

<Recognizer generation method>
Next, the operation of the recognizer generation device 100 according to the present embodiment will be specifically described with reference to FIGS. 15 and 18. FIG. 15 is a flowchart showing processing of the recognizer generation device 100. FIG. 18 is a conceptual diagram of processing performed by the recognizer generation device 100 and a recognition device 200 described later.

  S501: Teacher data is input to the recognizer generation device 100.

  S502: The feature extraction unit 101 extracts feature amounts from the input teacher data.

  S401: The extracted feature amount is input to STAR-SOINN as a weight vector. This weight vector is called an input node. In the learning stage (recognition generator generation stage), a label is input together with the input node, and the label is given to each input node.

  S402: The node deletion unit 107 increments the ages of all existing nodes in the STAR-SOINN.

  S302: The winner node extraction unit 102 calculates a Euclidean distance indicating a distance from an existing node, typically a distance between vectors, as an input node. As a result of this calculation, the node closest to the input node is extracted as the first winner, and the second closest node is extracted as the second winner.

  S303: The node insertion determining unit 103 determines whether or not to insert the input node as a new node in the STAR-SOINN.

  S403: When it is determined in S303 that a new node should be inserted, the node insertion determination unit 103 inserts the input node as a new node in the STAR-SOINN. That is, a new node is created at the same position as the input node. At this time, the node insertion determination unit 103 gives the label that the input node has to the new node.

  S305: When it is determined in S303 that a new node should not be inserted, the edge management unit 104 determines whether there is an edge between the first winner and the second winner.

  S306: When it is determined in S305 that there is no edge, the edge management unit 104 generates an edge between the first winner and the second winner.

  S307: The edge management unit 104 sets the age to 0 when an edge is generated in S306. If no edge is generated, the age of the existing edge is set to zero. Furthermore, the edge management unit 104 increments the ages of all edges to which the first winner is connected.

  S308: If there is an edge whose age has reached a predetermined threshold (age), the edge management unit 104 deletes the edge.

  S309: The node weight update unit 105 updates at least the weight vector of the first winner node based on the weight vector of the input node. Or it is good also as updating the weight vector of a 1st winner and its adjacent node. The update amount of the weight vector can be obtained by, for example, Equation 3 and Equation 4.

  S404: The label weight update unit 106 spreads the label weight of the input node to the first winner node and the second winner node. That is, the label weight update unit 106 updates the label weights of the first winner node and the second winner node based on at least a part of the label weight of the input node. The update amount of the label weight can be obtained from the number A and the number B, for example. In addition, the range of spreading | diffusion is good also as a 1st winner and its adjacent node, and a 2nd winner and its adjacent node, for example.

  S405: The node deletion unit 107 extracts a node whose node age is equal to a predetermined setting value and whose number of adjacent nodes is equal to or less than a predetermined threshold λ from all the nodes of the STAR-SOINN. Here, the set value can be a multiple of a constant λ, for example.

  S311: The node deletion unit 107 deletes the node extracted as the deletion target in S405.

  S406: The label weight update unit 106 transfers at least a part of the label weight of the deleted node to nodes around the deleted node, for example, the node closest to the deleted node and the second closest node. The assignment of the label weight can be performed by updating the label weight of the node on the assignment side according to the number C and the number D, respectively. Here, the label weight updating unit 106 may be configured to diffuse only the label weight of an arbitrary label. For example, it is also possible to spread only the labels relating to attributes. Also, attributes and classes may be diffused.

  S312: The recognizer generating apparatus 100 determines that learning is completed when the number of input nodes reaches a predetermined constant ρ, and completes the process. If not reached yet, the input of the next input node is accepted and the processing is continued according to the above-described procedure. The STAR-SOINN node group for which learning has been completed can be used as a recognizer included in the recognition apparatus 200 described later.

  Further, in the above-described embodiment, the configuration has been described in which the recognizer generation device 100 generates one recognizer corresponding to one feature amount extracted from input data. However, when a plurality of feature amounts can be extracted from the input data, a plurality of recognizer generation units each having a function equivalent to the recognizer generation device 100 corresponding to each of the feature amounts is prepared, and each feature amount is supported. Thus, an independent recognizer can be generated. In this case, the feature extraction units 101 of these recognizer generation units can be configured to extract different feature amounts. More specifically, when teacher data is input to the discriminator generation device 100, the teacher data is input to the feature extraction units 101 of the plurality of recognizer generation units. These feature extraction units 101 each have a corresponding winner node extraction unit 102, node insertion determination unit 103, edge management unit 104, node weight management unit 105, label weight update unit 106, and node deletion unit 107. These components perform the same processing as in steps S401 to S315 described above.

  With this configuration, the recognizer generation device 100 can generate a plurality of recognizers corresponding to a plurality of feature amounts.

<Configuration of recognition device>
Next, a recognition device 200 capable of performing transfer learning using the recognizer generated by the recognizer generation device 100 will be described. The recognition apparatus 200 recognizes the attribute of the input data using the recognizer, and performs transfer learning by recognizing the class that is the recognition target based on the attribute.

  The configuration of the recognition apparatus 200 according to the present embodiment will be described with reference to FIG. The recognizer generation device 200 is typically realized by a computer such as a dedicated computer or a personal computer (PC).

  The constituent elements 201 to 203 of the recognition apparatus 200 have a function of executing various controls based on various programs stored in a storage unit (not shown), and are provided with a central processing unit (CPU) and a read-only memory (ROM). ), A random access memory (RAM), an input / output port (I / O), and the like.

  The feature extraction unit 201 performs a process of extracting feature amounts as weight vectors from input data.

  Note that the feature extraction unit 201 may be the same as the feature extraction unit 101 described above. That is, the feature extraction units 101 and 201 may function to extract the feature amount of the teacher data during learning and to extract the feature amount of the input data.

  The recognizer 202 can perform transfer learning by using a recognition target to be recognized from input data as a class, and recognizing the class by an attribute that is a characteristic of the class. The recognizer generating apparatus 100 has a class and an attribute. It is generated by inputting the teacher data attached as the label weight and learning the feature amount.

  That is, the recognizer 202 is configured by each node (learned node) after inputting a predetermined number of the teacher data to the recognizer generation device 100.

  The result output unit 203 recognizes the attribute and class of the input node according to the distance between the plurality of learned nodes included in the recognizer 202 and the weight vector extracted from the input data, and performs a process of outputting the recognition result. . Below, an example of the recognition method is shown.

  First, the result output unit 203 calculates the distance between the weight vector (input node) input to the recognizer 202 and the existing node, typically the Euclidean distance. Using the result of this calculation, k nodes (k is an arbitrary natural number) closest to the input node are extracted. Next, the result output unit 203 recognizes the attribute and class of the input node based on the label weights of these k nodes.

  An example of a method for recognizing attributes and classes of input nodes is shown below. The following example is a recognition method in the case where all attribute values are represented by binary values of negative or positive.

  First, the recognizer generation device 100 executes the learning step described as the recognizer generation method for a class (animal) prior to the ρ-th class (animal) to be learned. On the other hand, when learning a class (animal) subsequent to the ρ-th class (animal), in addition to the learning step, a threshold value T (used in Equation 17 described later) for determining attributes is determined. Perform the process. The process for determining the threshold value T is calculated by calculating the probability of positive and negative for each attribute through recognition work and taking the average. Specifically, the probability of being positive and negative is obtained, and using this value, the average value of positive is updated for the attribute that should be positive, and the average value of negative is updated for the attribute that should be negative. For example, let us consider a case where the image characteristic amount of a lion is input and the values of positive and negative are calculated for the attribute “carnivorous”. Here, since the lion is originally a “carnivorous animal” (positive), in this case, the value of positive is used to update the average value of the positive relating to the attribute of “carnivorous animal”. When the same processing is performed for all animals after the ρth, the average value of positive and the average value of negative relating to this attribute are obtained. Then, an intermediate value between the average value of the positive and the average value of the negative, that is, a value obtained by adding and dividing by 2, is set as a threshold value T.

After such processing, the recognizer 202 recognizes the class of the input node by using the determination formula shown in Equation 17. Here, c is a recognition result class. Z is a set of classes to be compared, M is the number of attributes, Q is the number of features to be extracted from the recognition target, and a is an attribute m (positive or negative) of class z. T is an intermediate value of average values when recognition is performed during learning, and D is a value obtained by subtracting the negative average value from the positive average value. U is obtained by Equation 18. In Equation 18, P when the attribute is m and the feature quantity of the input node of feature q is I is obtained by Equation 19 and Equation 20. Here, N is a node having the feature vector I of the input node and the t-th closest weight vector in the STAR-SOIN of the feature q, and W is a weight vector of N. Further, P on the right side of Equations 19 and 20 is obtained by Equations 21 and 22. Equations 21 and 22 indicate the probabilities that the attribute m of the node N is positive or negative. L is the label weight of the attribute m of the node N.

  Here, class recognition is performed based on dictionary data defining the correspondence between attributes and classes. In this dictionary data, one class name is associated with a combination of a plurality of attributes and their values. The result output unit 203 compares the type and value of the attribute recognized as possessed by the input node with this dictionary data, and if there is a match, outputs the class name as a recognition result. Can do. Even if the matching class name is not stored in the dictionary data, the result output unit 203 outputs the recognition result as an undefined class composed of the recognized attribute type and its value. Can do.

<Recognition method>
Next, the operation of the recognition apparatus 200 according to the present embodiment will be specifically described with reference to FIG.

  S601: Input data is input to the recognition apparatus 200. As input data, for example, arbitrary information input from various sensors including an image sensor can be used.

  S502: The feature extraction unit 201 extracts a feature amount from this input data. The feature amount extraction processing can be performed in the same manner as the feature extraction unit 101 of the recognizer generation device 100. The feature amount extracted from the input data is input to the recognizer 202 as a weight vector.

  S602: The result output unit 203 extracts k learned nodes closest to the input node (k is an arbitrary natural number) among the existing nodes.

  S603: The result output unit 203 recognizes the attribute and class of the input node based on the label weights of these k learned nodes, and outputs the recognition result.

  In the above-described embodiment, a configuration in which the recognition apparatus 200 performs recognition using one recognizer corresponding to one feature amount extracted from input data (that is, when Q is one type). explained. However, when a plurality of feature amounts can be extracted from the input data (when Q is a plurality), a configuration using independent recognizers corresponding to the feature amounts may be used. For example, a plurality of feature extraction units 201 may be provided, and the feature extraction units 201 may be configured to extract different feature amounts. In this case, when data is input to the recognition apparatus 200, the input data is input to the plurality of feature extraction units 201, respectively. Each of these feature extraction units 201 includes a feature extraction unit 201 and a recognizer 202 corresponding to each other, and a result output unit 203 outputs a recognition result using parameters obtained from these recognizers 202.

  Such a recognizer generation device and a recognition device can be realized by a computer such as a dedicated computer or a personal computer (PC). However, the computer does not need to be physically single, and a plurality of computers may be used when performing distributed processing. As shown in FIG. 1, the computer 10 includes a CPU 11 (Central Processing Unit), a ROM 12 (Read Only Memory), and a RAM 13 (Random Access Memory), which are connected to each other via a bus 14. Although explanation of OS software for operating the computer is omitted, it is assumed that a computer for constructing the information processing apparatus is also provided.

  An input / output interface 15 is also connected to the bus 14. The input / output interface 15 includes, for example, an input unit 16 including a keyboard, a mouse, and a sensor, a display including a CRT and an LCD, an output unit 17 including headphones and speakers, a storage unit 18 including a hard disk, A communication unit 19 including a modem and a terminal adapter is connected.

  The CPU 11 performs various processes according to various programs stored in the ROM 12 or various programs loaded from the storage unit 18 to the RAM 13, and in this embodiment, for example, processes in the nearest prototype selection unit 34 and the prototype deletion unit 35. Execute. The RAM 13 also appropriately stores data necessary for the CPU 11 to execute various processes.

  The communication unit 19 performs, for example, communication processing via the Internet (not shown), transmits data provided from the CPU 11, and outputs data received from the communication partner to the CPU 11, the RAM 13, and the storage unit 18. The storage unit 18 exchanges with the CPU 11 to save and erase information. The communication unit 19 also performs communication processing of analog signals or digital signals with other devices.

  The input / output interface 15 is also connected to the drive 20 as necessary. For example, a magnetic disk 201, an optical disk 202, a flexible disk 203, or a semiconductor memory 204 is appropriately mounted, and a computer program read from them is required. Is installed in the storage unit 18 according to the above.

<Example>
The recognizer generation apparatus 100 and the recognition apparatus 200 (hereinafter referred to as AT-STAR-SOINN (Attribute Transfer-STAR-SOINN)) according to the present embodiment are DAP and IAP of Lampert et al., Which are techniques described in Non-Patent Document 1. The result evaluated in comparison with the above-mentioned AT-SOINN is shown in FIGS. The parameters used in the evaluation of this embodiment are λ = 600, age = 100, η = 0, k = 13, and ρ = 25. These were determined based on the results of preliminary experiments. There are 24,295 learning images and 6,180 recognition images. In all experiments, a personal computer with a CPU of 2.93 GHz and a memory of 8 GB was used.

  FIG. 19 shows a comparison result of the recognition rates of the AT-STAR-SOINN of this embodiment and the other methods described above. From FIG. 19, it can be seen that the recognition rate in AT-STAR-SOINN is inferior to DAP learned in batch, but is equivalent to AT-SOINN.

  FIG. 20 shows a comparison result of the number of nodes of SOINN (STAR-SOINN or Adjusted-SOINN) possessed by AT-STAR-SOINN and AT-SOINN, respectively. According to FIG. 22, in AT-STAR-SOINN, 99.47% of the number of nodes in AT-SOINN could be reduced. In AT-SOINN, the number of Adjusted-SOINN was proportional to the number of attributes, whereas in AT-STAR-SOINN, the number of attributes was increased or decreased by managing the attributes using the labels in STAR-SOINN. This is because learning can be performed without changing the number of SOINNs. In the present embodiment, the recognition rate hardly decreased as described above, despite such a large amount of information reduction (SOINN).

  21 and 22 show the comparison results of the learning time and the recognition time of AT-STAR-SOINN and the other methods described above. According to FIG. 21, the learning time of AT-STAR-SOINN is about 13,616 times faster than DAP and about 47 times faster than AT-SOINN. Further, according to FIG. 22, the time required for recognition in AT-STAR-SOINN is about 1,892 times faster than DAP and about 156 times faster than AT-SOINN.

  Thus, according to the present embodiment, it is possible to realize learning and transfer of attributes that can be performed online and additional learning at high speed without reducing the recognition rate.

<Third Embodiment>
Next, a robot apparatus equipped with a discriminator generated by the discriminator generation apparatus according to Embodiment 2 will be described. FIG. 23 is a block diagram showing the robot apparatus according to the present embodiment.

  The robot apparatus 300 includes an input data acquisition unit 301 and a recognition apparatus 200. Here, the recognition apparatus 201 may be the same as the recognition apparatus 200 described in the second embodiment.

  The input data acquisition unit 301 is an imaging unit including a camera, for example. In this case, the input data acquisition unit 301 can acquire image data as input data. The input data acquisition unit 301 may be capable of acquiring other types of data using various sensors or the like.

  The recognition device 201 recognizes the input data class, attributes, and the like using the data acquired by the input data acquisition unit 301 as input data. This recognition can be performed by the same procedure as in the second embodiment.

  Note that the robot apparatus 300 may be configured to perform additional learning using input data as teacher data by further including a configuration similar to that of the recognizer generation apparatus 100 described above. At this time, it is desirable that the input data acquisition unit 301 acquires label information given to the input data. The label information may be input by a person every time input data is acquired, or may be automatically provided by a labeling unit (not shown) according to a predetermined algorithm.

  The robot apparatus 300 may include a moving unit such as a self-propelled wheel, and may acquire input data by the input data acquiring unit 301 while moving by the moving unit.

  The robot apparatus 300 includes a communication unit capable of communicating with another robot apparatus, a recognition apparatus, a recognizer generation apparatus, or a storage device or database in which various learned nodes are registered. A learned node may be obtained from the above device or the like via

  In general, the types of attributes and dictionary data necessary for recognizing an object can change according to the environment or the like. For example, the attribute group and dictionary data necessary for recognizing the kind of animal and the attribute group and dictionary data necessary for recognizing the place will be different. Therefore, in such a case, the robot apparatus 300 can be configured to download a node group that has been learned with teacher data having a specific attribute necessary for the recognition target via the communication unit. The robot apparatus 300 can appropriately recognize the class and the attribute by using the node group as a recognizer. As described above, by providing an appropriate recognizer according to the situation through the network, it is possible to reuse the learning result in another apparatus and to provide the robot apparatus 300 that efficiently performs appropriate recognition. it can.

  In the present embodiment, the robot apparatus 300 can learn and recognize at high speed using STAR-SOINN. Such an increase in processing time is an indispensable feature for a robot that requires real-time performance, and in this respect, the present invention is suitable for application to a robot.

<Other embodiments>
In addition to the above-described embodiment, the present invention can be applied to a mobile terminal. For example, the mobile terminal includes an input unit including a camera and a microphone, and the recognition device 200 described above. The portable terminal recognizes the data input from the input unit by the recognition device 200 described above. As a result, even if something that has never been seen or heard, this mobile terminal can be used to infer what it is.

  At this time, the recognition apparatus 200 may be arranged on a network such as the Internet. The recognition apparatus 200 may perform learning and recognition in accordance with the environment and culture of the country using a database of learned nodes arranged on a network such as the Internet. Further, this database may store such learned node data in association with the position information.

  Many mobile terminals are equipped with a function for identifying the position such as GPS. Therefore, using the position information acquired by such a function as a key, an appropriate learned node is acquired from the database, and this is recognized. By using this, the recognizer 200 can perform appropriate recognition using location information. For example, when a towel is considered as a recognition target, using a learned node specialized in a “place where a part of the body is washed” such as a washroom or a bath, recognition including the meaning of an object that wipes the body is possible. It will be possible, and if we use learned nodes specialized for “places with desks” such as dining and living, it will be possible to recognize the meaning of wiping desks and furniture. Let's go. Even such a thing having a different meaning depending on a place can be recognized by using the method of the present embodiment.

  Such a learning method using a network can be a learning method that does not use a conventional transfer, but it is not realistic to have learned data on everything in the daily environment on the network. In this regard, according to the present invention using the STAR-SOINN that performs transfer learning, it is possible to recognize the learning target based on the learned data. In this respect, the present invention is suitable for application to the portable terminal that requires recognition processing in a daily environment. In addition, the present invention requires a smaller amount of calculation than conventional learning methods, and can secure real-time performance even with a low processing capability, so that it is also suitable for mounting on a portable terminal.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

  In the above-described embodiment, an example in which Adjusted-SOINN is used as a self-propagating neural network has been shown. However, the present invention is not limited to this, and Enhanced-SOINN (Japanese Patent Laid-Open No. 2008-217246), k-means, etc. A known clustering tool may be used.

  In the above-described embodiment, an example is shown in which one class (animal) is recognized from one image, but it is also possible to recognize one animal from a plurality of continuous images such as moving images. . The learning of time series data using SOINN is, for example, the non-patent literature “Shoko Okada and Osamu Hasegawa, On-line Learning of Sequence Data Based on SelfNeural Incremental Incremental. Can be performed using the techniques described.

  Furthermore, in the above-described embodiment, animals are mainly exemplified as identification objects (classes). However, the present invention is not limited to this, and can be applied to identification of all objects, spaces, events, and the like. For example, objects such as “cups”, “paper cups”, “bottles”, “kettles” in the kitchen are identified by attributes such as “water-filled”, “made of paper”, “made of metal”, etc. It is possible. The class of stationery "pen", "ballpoint pen", "scissors", "cutter", etc. has the object "bicycle", " Classes such as “automobile”, “bike”, and “truck” can be identified by attributes such as “tire”, “handle”, “loading platform”, and “exhaust port”.

  Moreover, although the example which uses an attribute as a label was mainly shown in the above-mentioned embodiment, this invention is not limited to this, It is possible to apply this to transfer learning of every label.

  Further, arbitrary processing in the above-described recognizer generation device and recognizer can be realized by causing a CPU (Central Processing Unit) to execute a computer program, for example. In this case, the computer program can be stored and provided to the computer using various types of non-transitory computer readable media. Non-transitory computer readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (for example, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (for example, magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / W, semiconductor memory (for example, mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)). The program may also be supplied to the computer by various types of transitory computer readable media. Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

DESCRIPTION OF SYMBOLS 1 Attribute learning and transfer system 2 Feature extraction part 3 Labeling part 4 Classifier holding part 5 Classifier production | generation part 6 Attribute identification part 7 Class identification part 100 Recognizer production | generation apparatus 101 Feature extraction part 102 Winner node extraction part 103 Node insertion determination Unit 104 edge management unit 105 node weight management unit 106 label weight update unit 107 node deletion unit 200 recognition device 201 feature extraction unit 202 recognizer 203 result output unit 300 robot device 301 input data acquisition unit

Claims (18)

A recognizer generating device that generates a recognizer that recognizes a class to be identified by an attribute that is a feature by learning a feature amount of teacher data,
A feature extraction unit that extracts a feature amount as a weight vector from teacher data in which the class and the attribute are attached as label weights;
The extracted weight vector is used as an input node, a distance between the input node and each node is calculated, and a node closest to the input node and a node closest to the second are respectively a first winner node and a second winner node. A winner node extraction unit that extracts as:
A node insertion determination unit that determines whether to insert the input node as a new node based on a distance between the input node and the first and second winner nodes;
When the input node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. An edge management unit that increments the ages of all edges of the first winner node and deletes edges that have reached a predetermined age;
A node weight updating unit that updates the weight vector of the first winner node based on the weight vector of the input node when not inserting the input node as a new node;
When not inserting the input node as a new node, a label weight updating unit that spreads at least part of the label weight of the input node to the first and second winner nodes;
A node deletion unit that deletes the node according to the node density at a predetermined timing;
The said label weight update part is a recognizer production | generation apparatus which spread | diffuses at least one part of the label weight which a deletion node has to the surrounding node of the said deletion node, when the said node deletion part deletes a node.   The recognizer generation device according to claim 1, wherein the attribute is multi-value data. The class be one in which a plurality of attributes are set, may have different attributes depending on the class, learner generation apparatus according to claim 1 or 2, wherein.   Any one of the Claims 1 thru | or 3 which further has an attribute optimization part which deletes an attribute unnecessary for the said node among the attributes contained in the label weight which the said node has whenever the age of the said node reaches predetermined age. The recognizer generation device according to claim 1.   5. The recognizer generation device according to claim 1, wherein the label weight update unit diffuses only the attribute among the label weights of the deleted node to the neighboring nodes. 6.   The node deletion unit increments the ages of all nodes every time a new node is input, and deletes nodes that are connected by edges with less than a predetermined number of nodes every time a predetermined age is reached. The recognizer generation device according to any one of claims 1 to 5.   A plurality of recognizer generation units including the feature extraction unit, the winner node extraction unit, the node insertion determination unit, the edge management unit, the node weight update unit, the label weight update unit, and a node deletion unit; The recognizer generation device according to claim 1, wherein the extraction unit extracts different feature amounts from the teacher data.   The node insertion determination unit calculates a similarity threshold, which is a threshold for determining whether or not to add the input node as a new node, based on the distances between the first and second winner nodes and surrounding nodes, The recognizer generation device according to claim 1, wherein it is determined whether to insert the input node as a new node based on a similarity threshold.   If there is a neighboring node that is connected to the first and second winner nodes by an edge, the node insertion determining unit determines the distance to the farthest node among the neighboring nodes, and the neighboring node exists. Otherwise, the distance from the nearest node is set as the first and second similarity thresholds, respectively, and when the input node is larger than one of the first and second similarity thresholds, the input node The recognizer generation device according to claim 1, wherein the recognizer generation device is inserted as a node.   The recognizer generation device according to claim 1, further comprising a labeling unit that labels the teacher data with the class and the attribute as label weights.   The recognizer generation device according to any one of claims 1 to 10, wherein each node after inputting a predetermined number of teacher data is set as a learned node, and a discriminator is configured by the learned node. A recognizer generating method for generating a recognizer that recognizes a class to be identified by an attribute that is a feature by learning a feature amount of teacher data,
A feature extraction step of extracting a feature quantity as a weight vector from teacher data to which the class and the attribute are attached as label weights;
The extracted weight vector is used as an input node, a distance between the input node and each node is calculated, and a node closest to the input node and a node closest to the second are respectively a first winner node and a second winner node. A winner node extraction step to extract as
A node insertion determination step of determining whether to insert the input node as a new node based on a distance between the input node and the first and second winner nodes;
When the input node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. An edge management step that increments the ages of all edges of the first winner node and deletes edges that have reached a predetermined age;
A node weight update step of updating the weight vector of the first winner node based on the weight vector of the input node when not inserting the input node as a new node;
A first label weight update step of spreading at least a part of the label weight of the input node to the first and second winner nodes when not inserting the input node as a new node;
A node deletion step of deleting a node according to the node density at a predetermined timing;
And a first label weight updating step of diffusing at least a part of the label weight of the deleted node to nodes around the deleted node when the node is deleted in the node deleting step. A recognition device that can perform transfer learning by recognizing a class to be recognized from input data and recognizing the class by an attribute that is a characteristic of the class,
A feature extraction unit that extracts a feature quantity as a weight vector from the input data;
The classifier is generated by inputting the teacher data with the class and the attribute added as label weights to the recognizer generation device, and learning the feature amount between the learned node and the learned node. A recognizer configured using a self-propagating neural network including connected edges;
A result output unit that outputs a recognition result according to a distance between a plurality of learned nodes composed of weight vectors of the recognizer and a weight vector extracted from the input data;
The recognizer generation device includes:
A feature extraction unit for extracting a feature amount from the teacher data as a weight vector;
The extracted weight vector is used as an input node, a distance between the input node and each node is calculated, and a node closest to the input node and a node closest to the second are respectively a first winner node and a second winner node. A winner node extraction unit that extracts as:
A node insertion determination unit that determines whether to insert the input node as a new node based on a distance between the input node and the first and second winner nodes;
When the input node is not inserted as a new node, if there is no edge between the first winner node and the second winner node, an edge is generated and its age is set to 0. An edge management unit that increments the ages of all edges of the first winner node and deletes edges that have reached a predetermined age;
A node weight updating unit that updates the weight vector of the first winner node based on the weight vector of the input node when not inserting the input node as a new node;
When not inserting the input node as a new node, a label weight updating unit that spreads at least part of the label weight of the input node to the first and second winner nodes;
A node deletion unit that deletes the node according to the node density at a predetermined timing;
The label weight update unit, when the node deletion unit deletes a node, spreads at least a part of the label weight of the deletion node to nodes around the deletion node,
Each node after inputting a predetermined number of the teacher data is regarded as the learned node,
The recognizing device recognizes the attribute of the input data according to the similarity between the weight vector extracted from the input data and the learned node.   The recognition apparatus according to claim 13, wherein the feature extraction unit extracts a feature amount of the teacher data during learning and extracts a feature amount of the input data during recognition.   The result output unit extracts N (N is a natural number) learned nodes closest to the weight vector extracted from the input data, and based on the label weights of the N learned nodes, 15. The recognition apparatus according to claim 13 or 14, which outputs class and attribute information as a recognition result. The result output unit has dictionary data indicating a correspondence relationship between the attribute and the class, and refers to the dictionary data based on a label weight of the N learned nodes, and the class of the input data The recognition apparatus according to claim 15 , which outputs a recognition result as a recognition result. A feature extraction unit for extracting features from input data and teacher data;
A labeling unit for labeling the attribute information given to the teacher data;
An attribute discriminator for identifying an attribute included in the input data, wherein the attribute discriminator is configured using a self-propagating neural network including a node and an edge connecting the nodes, and the self-propagating neural network Is divided into a plurality of parts according to the identification contents identified by the attribute, and the feature extracting unit is applied to the part of the self-propagating neural network specified by the attribute information labeled by the labeling unit A classifier generating unit that inputs the characteristics of the teacher data extracted in step (b) as an input pattern and generates the node and the edge based on the input pattern in the self-propagating neural network;
A discriminator holding unit for holding the attribute discriminator generated by the discriminator generating unit;
When the input data is input, a feature extracted from the input data is input to each part of the self-propagating neural network constituting the attribute classifier held by the classifier holding unit. A first similarity between the input pattern and the node included in the self-propagating neural network is calculated in each part of the self-propagating neural network, and the calculated first similarity Depending on the degree, an attribute identification unit for identifying which attribute of the identification content is included in the input data;
Attribute information included in each of a plurality of classes is given, the attribute of the input data identified by the attribute identification unit is compared with the attribute information of the class to obtain a second similarity, and the calculated second A class identifying unit for identifying which class of the plurality of classes includes the input data according to the similarity of
An attribute learning and transfer system comprising: The discriminator generator is
Claim 17, wherein the dividing a first portion indicating that it contains the attribute, and a second portion indicating that does not contain the attribute, the said attribute identifier Attribute learning and transfer system.