JP5802916B2 - Sensory data identification device and program - Google Patents

Description

  The present invention relates to a sensory data identification device and program, and more particularly, to a sensory data identification device and program for learning sensory feature quantities for identifying sensory data and identifying sensory data.

  2. Description of the Related Art Conventionally, an article discrimination method is known in which a feature vector is mapped by a transformation matrix obtained by canonical correlation analysis to calculate a new feature vector (Patent Document 1).

JP 2006-260310 A

  However, the technique described in Patent Document 1 has a problem that the feature vector is merely dimensionally compressed by canonical correlation analysis, and a feature amount that improves the identification performance cannot be calculated.

  The present invention has been made to solve the above-described problems, and sensory data identification that can improve the discrimination performance for sensory data using the kernel version of canonical correlation analysis (kernel canonical correlation analysis). An object is to provide an apparatus and a program.

In order to achieve the above object, a sensory data identification device according to a first aspect of the present invention provides brain activity data representing signals measured at each measurement point of a biological brain when sensory data or related sensory data is given, and Kernel canonical correlation analysis is performed on the sensory data or the related sensory data, and is expressed as a linear sum of kernel functions so that the correlation between the brain activity data and the sensory data or the related sensory data becomes high. Learning means that learns weights in the canonical correlation variable and uses the learned weights as the canonical feature variable for identifying the sensory data, and a plurality of sensory data prepared in advance Corresponding to the sensory feature quantity calculated by the second feature quantity calculating means and the second feature quantity calculating means for calculating the sensory feature quantity learned by the learning means. And the identification result obtained in advance with respect to sensory data as learning data, the identification model learning means for learning the identification model to identify the sensory data from the input sensory data, which is learned by the learning means A feature amount calculating unit for calculating the sensory feature amount; and an identification unit for identifying the sensory data based on the sensory feature amount and the identification model calculated by the feature amount calculating unit. ing.

According to a second aspect of the invention, there is provided a program comprising: a brain activity data representing a signal measured at each measurement point of a living organism's brain when sensory data or related sensory data is given; and the sensory data or the related sensory data; Is subjected to kernel canonical correlation analysis, and weights in the canonical correlation variable represented by a linear sum of kernel functions are set so that the correlation between the brain activity data and the sensory data or the related sensory data is high. Learning means that uses the learned canonical correlation variables using the learned weights as sensory feature quantities for identifying the sensory data, and learns from each of a plurality of sensory data prepared in advance. A second feature value calculating unit for calculating the sensory feature value, and a sensory data corresponding to the sensory feature value calculated by the second feature value calculating unit. And the identification result obtained because as learning data, the identification model learning means for learning the identification model to identify the sensory data from the input sensory data, the sensory characteristic amount learned by the learning means It is a program for functioning as an identification means for identifying the sensory data based on the feature quantity calculation means to be calculated and the sensory feature quantity calculated by the feature quantity calculation means and the identification model .

  According to the first invention and the second invention, the brain activity data representing the signals measured at each measurement point of the brain of the organism when the sensory data or the related sensory data is given by the learning means and the sensory data or the related data. Kernel canonical correlation analysis is performed on sensory data, and weights in the canonical correlation variable expressed by a linear sum of kernel functions are set so that the correlation between brain activity data and sensory data or related sensory data becomes high. learn. Then, the canonical correlation variable using the learned weight is used as a sensory feature amount for identifying sensory data.

  Then, the sensory feature amount learned by the learning unit is calculated from the input sensory data by the feature amount calculating unit. The identification means identifies sensory data based on the sensory feature quantity calculated by the feature quantity calculation means.

  In this way, by performing kernel canonical correlation analysis on brain activity data and sensory data or related sensory data, a canonical correlation variable represented by a linear sum of kernel functions is used as a sensory feature amount. The discrimination performance for sensory data can be improved.

  Note that sensory data used for learning sensory feature values does not have to be exactly the same as sensory data to be identified, and may be anything that is relevant to some extent. In this specification, data for learning sensory feature quantities and data related to sensory data to be identified is referred to as “related sensory data”, “related image data”, and the like.

According to a fourth aspect of the present invention, the computer gives the sensory data or the related sensory data to a neural network that has been learned by using a combination of sensory data and an identification result obtained in advance for the sensory data as learning data. A pseudo-canonical correlation analysis is performed on the pseudo brain activity data representing the output observed in each neuron of the intermediate layer of the neural network and the sensory data or the related sensory data, and the pseudo brain activity data is obtained. And learning the weight in the canonical correlation variable represented by a linear sum of kernel functions so that the correlation between the sensory data or the related sensory data is high, and the canonical correlation variable using the learned weight, each of the plurality of sensory data learning means, prepared in advance to sensory characteristic quantity for identifying the sensory data A second feature quantity calculating means for calculating the sensory feature quantity learned by the learning means, and sensory data corresponding to the sensory feature quantity calculated by the second feature quantity calculating means. A discrimination model learning means for learning an identification model for identifying the sensory data, using the identification result as learning data, and a feature for calculating the sensory feature quantity learned by the learning means from the input sensory data A program for functioning as an identification unit for identifying the sensory data based on a quantity calculation unit, and the sensory feature amount calculated by the feature amount calculation unit and the identification model .

  According to the third and fourth inventions, sensory data or related sensory data is stored in a neural network that has been learned by the learning means using a combination of sensory data and a discrimination result obtained in advance for sensory data as learning data. The brain canonical correlation analysis is performed on the simulated brain activity data and sensory data or related sensory data representing the output observed in each neuron of the intermediate layer of the neural network. A weight in a canonical correlation variable represented by a linear sum of kernel functions is learned so that the correlation with sensory data or related sensory data is high. Then, the canonical correlation variable using the learned weight is used as a sensory feature amount for identifying sensory data.

  In this way, by performing kernel canonical correlation analysis on pseudo brain activity data and sensory data or related sensory data, the canonical correlation variable represented by the linear sum of kernel functions is used as a sensory feature quantity. The identification performance for sensory data can be improved.

  The sensory feature amount can be expressed by the following equation.

Here, I _a ⁽ⁿ⁾ (x) is a sensory feature quantity of the input sensory data x, a ^T _(n) φ (x) is the nth canonical correlation variable, and x ^(j) Is the jth sensory data or related sensory data previously given for kernel canonical correlation analysis. k _a (x, x ′) is a kernel function, and α _j ⁽ⁿ⁾ is a weight for the j-th kernel function.

  The learning means can learn weights in each of a plurality of canonical correlation variables, and can use a plurality of canonical correlation variables using the learned weights as sensory feature quantities.

  The sensory data can be image data, sound data, odor data, or tactile data.

  The sensory data identification device may further include display control means for causing the display means to display the identification result obtained by the identification means in association with the input sensory data.

  As described above, according to the sensory data identification apparatus and program of the present invention, kernel canonical correlation analysis is performed on brain activity data or pseudo brain activity data and sensory data or related sensory data to obtain a kernel function. By using the canonical correlation variable represented by the linear sum of the sensation feature quantity, the effect of improving the discrimination performance for the sensation data can be obtained.

It is the schematic which shows the structure of the target object identification apparatus which concerns on the 1st Embodiment of this invention. It is the schematic which shows the structure of the brain activity data measuring device which concerns on the 1st Embodiment of this invention. It is a figure which shows a mode that the cap type | mold electrode was mounted | worn with the test subject. It is a flowchart which shows the content of the feature-value learning process routine in the computer of the target object identification apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the content of the identification processing routine in the computer of the target object identification apparatus which concerns on the 1st Embodiment of this invention. It is an image figure of the screen which displays an identification result. It is the schematic which shows the structure of the pseudo brain activity data measuring device which concerns on the 2nd Embodiment of this invention. It is a figure for demonstrating a neural network. It is a flowchart which shows the content of the pseudo brain signal acquisition process routine in the pseudo brain activity data measuring device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the example of handwritten numeral image data. (A) A graph showing the relationship between the first canonical correlation variable on the image side and the first canonical correlation variable on the brain side, and (B) the second canonical correlation variable on the image side and the second canonical value on the brain side. It is a graph which shows the relationship with a correlation variable. (A) A graph showing the relationship between the third canonical correlation variable on the image side and the third canonical correlation variable on the brain side, and (B) the fourth canonical correlation variable on the image side and the fourth canonical side on the brain side. It is a graph which shows the relationship with a correlation variable. (A) A graph showing the distribution of the first canonical correlation variable on the image side, and (B) a graph showing the distribution of the second canonical correlation variable on the image side. (A) A graph showing the distribution of the first component of PCA on the image side, and (B) a graph showing the distribution of the second component of PCA on the image side. It is a graph which shows the correct answer rate with respect to the handwritten number with a shield in conventional SVM and KCCA-SVM, and the correct answer rate by KCCA-SVM when handwritten hiragana is used as an input stimulus. It is a figure which shows the example of the collection of the image data showing a handwritten hiragana.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In addition, the case where this invention is applied to the target object identification apparatus which identifies whether a captured image is an image which imaged the pedestrian as an identification target object is demonstrated to an example.

  As shown in FIG. 1, an object identification device 10 according to the first embodiment is attached to a vehicle (not shown), and captures an image of the front of the vehicle to generate an image, and an imaging The computer 14 which identifies whether the captured image obtained from the apparatus 12 is the image which imaged the pedestrian, and the display apparatus 16 which displays the identification result of the computer 14 are provided.

  The imaging device 12 images the front of the vehicle and generates an image signal of the image (not shown), and an A / D conversion unit (not shown) that A / D converts the image signal generated by the imaging unit. And an image memory (not shown) for temporarily storing the A / D converted image signal.

  The computer 14 includes a CPU, a RAM, and a ROM that stores a program for executing a feature amount learning processing routine and an identification processing routine, which will be described later, and is functionally configured as follows. The computer 14 includes an image brain signal storage unit 20 that stores a plurality of pairs of image data and brain activity data, which will be described later, and an image feature amount learning unit that learns image feature amounts based on the plurality of pairs of image data and brain activity data. 22 and a learning result storage unit 24 that stores a learning result obtained by the image feature amount learning unit 22.

  A plurality of pairs of image data and brain activity data stored in the image brain signal storage unit 20 is obtained by the brain activity data measuring device 30 shown in FIG. The brain activity data measuring device 30 includes a cap-type electrode 32 for detecting an electroencephalogram signal from a plurality of parts on the subject's head surface, a display device 34 for displaying image data, and displaying image data on the display device 34. And a computer 36 for recording brain activity data representing the electroencephalogram signal of each part detected by the cap-type electrode 32.

  As shown in FIG. 3, the cap-type electrode 32 is attached to a plurality of sites on the subject's head surface, and detects electroencephalogram signals from the plurality of sites on the head surface. The display screen of the display device 34 is suitable for the subject.

  The computer 36 includes a CPU, a RAM, and a ROM, and is functionally configured as follows. The computer 36 stores in advance a plurality of image data including a pedestrian image and a non-pedestrian image, and reads from the image brain signal storage unit 38 and an image brain signal storage unit 38 for storing corresponding brain activity data. A display control unit 40 for displaying the image data on the display device 34, a signal acquisition unit 42 for acquiring an electroencephalogram signal output from the cap-type electrode 32 when the image data is displayed on the display device 34, and an A / D An A / D conversion unit 44 that converts the brain wave signal acquired by the signal acquisition unit 42 into a digital signal by performing conversion, and the obtained brain activity data representing the brain wave signal of each part is imaged in correspondence with the image data. And a storage control unit 46 for storing in the brain signal storage unit 38.

  The signal acquisition unit 42 acquires an electroencephalogram signal output from each of the electrodes attached to a plurality of sites on the subject's head surface, and performs noise removal and amplification processing.

  The computer 36 causes the display control unit 40 to sequentially display a plurality of image data stored in the image brain signal storage unit 38 on the display device 34, and detects when each image data is displayed by the storage control unit 46. The brain activity data thus obtained is stored in the image brain signal storage unit 38 in correspondence with the image data. As a result, the image brain signal storage unit 38 stores a plurality of pairs of image data and brain activity data.

  A plurality of pairs of image data and brain activity data stored in the image brain signal storage unit 38 are input to the computer 14 of the object identification device 10, and the image brain signal storage unit 20 stores the brain activity data measurement device 30. A plurality of pairs of the same image data and brain activity data as in the image brain signal storage unit 38 are stored.

  Next, the principle of learning image feature amounts from a plurality of pairs of image data and brain activity data will be described.

  The visual discrimination task such as whether or not the object that is visible in front of you is a human being becomes a very difficult task for existing devices if the object is hidden by noise or shielding. On the other hand, living organisms must respond correctly to such visual obstacles for survival, and the brain can perform the above tasks well. Therefore, consider learning the method from the brain. Recent progress in rapid brain activity measurement technology has made it possible to simultaneously measure the activity of many brain cells when the brain is making visual discrimination. Therefore, the image data viewed by the subject is x, the brain activity at that time is y, and the brain judgment as to whether the image is a pedestrian is z. Since x represents an image, it represents a vector having the dimension of the number of pixels, and y represents an activity of a plurality of parts, so that y is represented as a vector having the dimension of the number of parts simultaneously measured. Finally, z is a scalar variable that takes 0 or 1 because it is a decision as to whether it is a pedestrian or not.

  Note that it is possible to expand not only the values at a certain point in time of the image and brain activity but also the values at a plurality of times, and examine the correspondence between them. In that case, the number of dimensions is as described above. The number of dimensions is multiplied by the number of times to consider.

Conventionally, a large number of image ^{x (1), x (2} ), x (3), ··· x (N) determination of the brain for ^{z (1), z (2} ), z (3), ··· A method for learning an SVM model as a visual discrimination algorithm using z ^(N) as learning data is known. In this conventional method, the main method is to find the feature quantity of the image that is important for image discrimination from the set of x ^(j) and z ^(j). there were.

A new place of the method proposed in the present embodiment is that the brain activity y ⁽¹⁾ , y ⁽²⁾ , y ⁽³⁾ ,... Y ^{(N) at} that time is used. Focusing on the fact that the correct judgment of z ^(j) is made by the cranial nerve activity y ^(j) , it is considered that the image features that correlate with the neural activity are the image features important for image identification. Therefore, the brain activity data for identifying the image or the brain activity data for inputting the related image data is used.

Further, the relationship between the image x ^(j) and the brain activity y ^(j) is extremely non-linear and difficult to obtain by normal canonical correlation analysis. However, if the image x ^(j) and the brain activity y ^(j) are mapped onto the high-dimensional feature space as φ (x ^(j) ) and Ψ (y ^(j) ), respectively, φ (x ^(j) ) And Ψ (y ^(j) ) are linear. In order to utilize this, in the present embodiment, the correlation between the image x ^(j) and the brain activity y ^(j) is obtained by using kernel canonical correlation analysis (KCCA).

To search for a one-dimensional space having the highest linear correlation between the space where {φ (x ^(j) )} exists and the space where {Ψ (y ^(j) )} exists, respectively Consider a ^T φ (x ^(j) ) and b ^T Ψ (y ^(j) ), which are projections in one dimension. In the kernel canonical correlation analysis, the correlation C (a ^T φ (x ^(j) )) between a ^T φ (x ^(j) ) and b ^T Ψ (y ^(j) ) expressed by the following equation (1): Determine a ^T and b ^T that maximize b ^T Ψ (y ^(j) )).

Here, a ^T and b ^T are transposes of vectors having the same dimensions as φ (x ^(j) ) and ψ (y ^(j) ), respectively. Also, Cov (X, Y) is covariance and Var (X) is dispersion.

Further, when obtaining the a ^T and b ^T, correlation becomes extremely high for a finite number of data provided to determine the a ^T and b ^T, dangerous (overfitting to poor correlation for unknown data In order to prevent this problem, the regularization term expressed by the following equation (2) is added and the maximization problem is solved.

In a specific calculation, a ^T φ (x ^(j) ) and b ^T Ψ (y ^(j) ) are specifically expressed as a linear sum of a predetermined kernel function (here, using a Gauss kernel). As shown in equations (3) and (4), the expression that can be expressed using α and β is used (kernel trick).

  That is, in the present embodiment, α and β that maximize the formula obtained by substituting the formulas (3) and (4) into the formula (1) are obtained. In addition, for a detailed formula with the addition of the regularization term represented by the above formula (2) for calculating α and β, non-patent literature (kernel multivariate analysis, Shotaro Akaho, Iwanami Shoten, 2008, 76) Since the expression similar to the expression described in page -77, expression (3.80) may be used, the description is omitted.

A set of vectors (a, b) as described above (in a specific calculation, a set of (α, β)) is obtained as a solution to the generalized eigenvalue problem, but the obtained eigenvector (a, b) is used as a correlation value. c ₍₁₎ , c ₍₂₎ , c ₍₃₎ ,... in descending order (a ₍₁₎ , b ₍₁₎ ), (a ₍₂₎ , b ₍₂₎ ), (a _{(3 )} , B ₍₃₎ ), ... Further, since the correlation value of the above equation (1) is an amount that does not depend on the vector (a, b), the length of the vector (a _(j) , b _(j) ) is not specified in the maximization procedure here. . However, since a vector corresponding to a large correlation value is considered to be more important, information on the importance of the vector is incorporated by making the length of these vectors equal to the correlation value according to the following equation (5). .

The following (a _(j) , b _(j) ) means the length adjusted in this way.

A ^T ₍₁₎ φ (x) and b ^T ₍₁₎ Ψ (y) determined as above are called the first canonical correlation variable of the image and the first canonical correlation variable of the brain activity, respectively. , A ^T ₍₂₎ φ (x), b ^T ₍₂₎ Ψ (y) are sequentially called the second canonical correlation variable of image and brain activity.

The image-side canonical correlation variable obtained in this way is an image feature amount useful for image discrimination. Image feature amount I _a for the image x (x) can be calculated as specifically Equation (6) below as a linear sum of the kernel function k _a (a, b).

Here, x ^(j) is image data given in advance for kernel canonical correlation analysis.

Incidentally, the feature value I _b (x) on the brain activity side can also be calculated by the following equation (7) as a linear sum of the brain function on the brain activity side k _b (y, y ′).

By the way, since | a _(j) | and | b _(j) | required in the above equation (2) can also be written by a kernel function, substantial calculation is possible.

  The image feature amount represented by the above equation (6) obtained in this way is created by reflecting the brain activity that can solve the visual discrimination task that is currently a problem with high accuracy. Therefore, the visual discrimination algorithm using the image feature amount shows high performance reflecting the ability of the brain in principle.

In this embodiment, n ₀ canonical correlation variables {I ⁽¹⁾ _a (x), I ⁽²⁾ _a (x),..., I ⁽ⁿ⁰⁾ _a (x) on the image side } Is used as an image feature amount.

As described above, the image feature amount learning unit 24 learns weights α and β in each of n ₀ canonical correlation variables from a plurality of pairs of image data and brain activity data, and the weight α And n ₀ canonical correlation variables on the image side calculated by the above equation (6) using the image data set and the image data set are determined as image feature values, and the learning result storage unit 24 stores the image feature values. As a learning result, for each of the n ₀ canonical correlation variables, the parameters including the weight α and the image data x ^(j) in the above equation (6) are stored.

  In addition, the computer 14 of the object identification device 10 includes a feature amount calculation unit 50 that calculates a learned image feature amount from each of the image data stored in the image brain signal storage unit 20, and a calculated image feature amount. And an identification model learning unit 52 for learning the identification model based on the teacher label given to the corresponding image data, and an identification model storage unit 54 for storing the learned identification model.

The feature amount calculation unit 50 calculates n ₀ canonical correlation variables on the image side represented by the above equation (6) as image feature amounts from each of the image data stored in the image brain signal storage unit 20. .

  The identification model learning unit 52 performs SVM as an identification model based on the calculated image feature amount and a teacher label given according to whether the corresponding image data is a pedestrian image or a non-pedestrian image. (Support vector machine) The model is learned, and the learned SVM model is stored in the identification model storage unit 54.

  The computer 14 is based on the feature amount calculation unit 56 that calculates the learned image feature amount from the image data captured by the imaging device 12, the calculated image feature amount, and the learned SVM model. The discriminator 58 further includes a discriminator 58 that discriminates whether or not the captured image data is an image of a pedestrian.

The feature amount calculation unit 56 acquires the image data captured by the imaging device 12, and from the acquired image data, as the image feature amount, n ₀ canonical correlation variables on the image side represented by the above equation (6). Is calculated.

The identification unit 58 is conventionally known based on the SVM model stored in the identification model storage unit 54 and the n ₀ canonical correlation variables on the image side as the image feature amount calculated by the feature amount calculation unit 56. The SVM discriminator is used to identify whether or not the captured image data is an image of a pedestrian, and the identification result is displayed on the display device 16 in association with the captured image data.

For example, the SVM classifier, linear separation in space mapped with higher dimensional image side of the n ₀ pieces of canonical correlation spanned by variable space linearly separable or image side n ₀ amino canonical correlation variable, I do. In these linear separations, images are identified using kernel functions represented by the following equations (8) and (9), respectively.

  Next, the operation of the present embodiment will be described. First, prepare a plurality of pedestrian image data obtained by imaging pedestrians and non-pedestrian image data obtained by imaging other than pedestrians, and give a teacher label to each image data The image data is stored in the image brain signal storage unit 38 of the brain activity data measuring device 30.

  Then, each image data stored in the image brain signal storage unit 38 is displayed on the display device 34 by the computer 36 of the brain activity data measuring device 30 with the cap-type electrode 32 attached to the head of the subject. At the same time, brain activity data representing the electroencephalogram signal of each part detected at this time is acquired, and the brain activity data is stored in the image brain signal storage unit 38 in association with the image data. As a result, a plurality of pairs of image data and brain activity data are stored in the image brain signal storage unit 38.

  Further, the data stored in the image brain signal storage unit 38 of the brain activity data measuring device 30 is input to the object identification device 10 and stored in the image brain signal storage unit 20. Then, a feature amount learning process routine shown in FIG. 4 is executed in the computer 14 of the object identification device 10.

In step 100, a plurality of pairs of image data and brain activity data are read from the image brain signal storage unit 20, and in the next step 102, based on the plurality of pairs of image data and brain activity data read in step 100 above. The kernel function weights α and β that maximize the correlation represented by the formulas obtained by substituting the formulas (3) and (4) into the formula (1) are learned, and the learned weight α is applied. The n ₀ canonical correlation variables on the image side represented by the equation (6) are determined as image feature amounts. For example, all canonical correlation variables on the image side where the correlation value is greater than or equal to a predetermined value are determined as image feature amounts.

In step 104, the parameters including the weight α and the image data x ^(j) in the image feature amount represented by the above equation (6) learned in step 102 are stored in the learning result storage unit 24.

  In the next step 106, each image data is read from the image brain signal storage unit 20, and the image feature amount learned in the above step 102 is calculated for each image data. In step 108, the teacher label given for each image data is read from the image brain signal storage unit 20, and based on the image feature amount of each image data calculated in step 106 and the teacher label, the SVM model. In step 110, the learning result in step 108 is stored in the identification model storage unit 54, and the feature amount learning processing routine is terminated.

  Then, when a predetermined area in front of the vehicle is imaged by the imaging device 12 while the vehicle equipped with the object identification device 10 is traveling, the identification processing routine shown in FIG.

  First, in step 120, image data is acquired from the imaging device 12, and in the next step 122, a learned image feature amount is calculated for the image data acquired in step 120. In step 124, whether or not the captured image data is pedestrian image data representing a pedestrian using the learned SVM model and SVM classifier based on the image feature amount calculated in step 122. To identify. In step 126, the identification result in step 124 is displayed on the display device 16 in association with the captured image data as shown in FIG. 6, and the identification processing routine is terminated.

  As described above, according to the object identification device according to the first embodiment, kernel canonical correlation analysis is performed on image data and brain activity data obtained when the image data is displayed. To learn the kernel function weights, and to learn image feature quantities that improve the discrimination performance for image data by using the image-side canonical correlation variables as image feature quantities Can do.

  Up to now, in order to assist human beings, devices for making various cognitive judgments imitating human beings have been developed, but they often do not reach the human brain ability. In recent years, the development of mechanical devices based on new computational algorithms inspired by the knowledge of brain science has been sought in order to get close to the high ability of the brain, but these attempts are still at an early stage. Development of devices based on new ideas applying brain science knowledge in various industries is desired, but the principles of signal processing and calculation principles are completely different between the brain and machine, and brain science knowledge is used in machines. It is not easy to apply. In particular, brain research ranges from molecular biological research to behavioral and psychological research. Since the brain is extremely complex, it is not clear which knowledge is useful for the development of new devices. Thus, while learning from brain science is considered important, no specific means of applying knowledge of brain science has been shown.

  In this embodiment, brain activity data during task execution obtained in brain science experiments is used to make use of the advanced characteristics of the brain and provide a computational algorithm that can be implemented to improve discrimination performance for image data. Added the ability to learn image features. In addition, by using the image feature values learned in the present embodiment, problems with conventional recognition machines (for example, problems of misrecognition and non-recognition due to the influence of shielding or noise in pedestrian recognition using images) Can be solved.

  In the present embodiment, a realistic method of learning from the brain is defined. In this embodiment, when the intellectual tasks performed by the brain are divided into 1) which information is used and 2) how to calculate from the information, in this embodiment, only 1) is learned from the brain, and 2) I applied the existing mathematical and engineering methods to avoid learning from the brain. By clearly limiting the objects to be learned in this way, it is possible to develop a device that incorporates the knowledge of brain science realistically and precisely.

  Next, a second embodiment will be described. In addition, since the structure of the target object identification apparatus which concerns on 2nd Embodiment is the same as that of 1st Embodiment, it attaches | subjects the same code | symbol and abbreviate | omits description.

  The second embodiment is mainly different from the first embodiment in that pseudo brain activity data when image data is input is acquired from a learned neural network.

  In the second embodiment, a plurality of pairs of image data and simulated brain activity data are stored in the image brain signal storage unit 20 of the object identification device 10.

  A plurality of pairs of image data and simulated brain activity data stored in the image brain signal storage unit 20 are obtained by the simulated brain activity data measuring device 230 shown in FIG. The simulated brain activity data measuring device 230 is a computer including a CPU, a RAM, and a ROM that stores a program for executing a simulated brain signal acquisition processing routine to be described later. It is configured. The simulated brain activity data measuring device 230 stores in advance a plurality of image data to which a teacher label is assigned, and also stores an image brain signal storage unit 238 for storing corresponding pseudo brain activity data, and an image brain signal storage unit 238. A neural network learning unit 240 that reads each image data and teacher label from the image and learns a neural network (pseudo-brain) for identifying whether the image data is a pedestrian image or a non-pedestrian image; An image data acquisition unit 242 that reads each image data from the signal storage unit 238 and inputs it to the input layer of the neural network, and based on the learned neural network, whether the input image data is a pedestrian image or a non-pedestrian Neural network identification unit 244 for identifying whether the image is an image and when image data is input A pseudo brain signal acquisition unit 246 that acquires pseudo brain activity data representing an output (pseudo brain signal) observed in each neuron of the intermediate layer of the neural network, and the obtained pseudo brain activity data is inputted as image data and And a storage control unit 248 that stores the image brain signal storage unit 238 in a corresponding manner.

  As shown in FIG. 8, the neural network model used as a pseudo brain in the present embodiment is, for example, an artificial neural network model (ANN), which is composed of an input layer, an intermediate layer, and an output layer. , Comprising at least one neuron.

  For each image data read from the image brain signal storage unit 238, the neural network learning unit 240 inputs the pixel value of each pixel of the image data to each neuron of the input layer of the neural network, from the neurons of the output layer. A value to be output is given based on a teacher label for the image data, and the neural network is learned.

The neural network identifying unit 244 is a pedestrian image or a non-pedestrian image according to the output of the output layer when the image data is input based on the neural network learned as described above. In the pseudo brain activity data measuring device 230, the image data acquisition unit 242 sequentially inputs a plurality of image data stored in the image brain signal storage unit 238 to the learned neural network, and the pseudo brain signal The acquisition unit 246 acquires the simulated brain activity data observed when each piece of image data is input to the neural network, and the storage control unit 248 stores the simulated brain activity data corresponding to the image data in the image brain signal storage unit 238. Remember me. As a result, the image brain signal storage unit 238 stores a plurality of pairs of image data and simulated brain activity data.

  A plurality of pairs of image data and simulated brain activity data stored in the image brain signal storage unit 238 are input to the computer 14 of the object identification device 10, and the simulated brain activity data measurement device is stored in the image brain signal storage unit 20. A plurality of pairs of the same image data and simulated brain activity data as the image brain signal storage unit 238 of 230 are stored.

  Next, the operation of the present embodiment will be described. First, prepare a plurality of pedestrian image data obtained by imaging pedestrians and non-pedestrian image data obtained by imaging other than pedestrians, and give a teacher label to each image data The image data is stored in the image brain signal storage unit 38 of the brain activity data measuring device 30.

  Then, the simulated brain activity data measurement device 230 executes a simulated brain signal acquisition processing routine shown in FIG.

  First, in step 250, each image data and teacher label stored in the image brain signal storage unit 238 is read. In step 252, a neural network is learned based on each image data and teacher label read in step 250. To do.

  In step 254, one image data stored in the image brain signal storage unit 238 is acquired. In step 256, the image data acquired in step 254 is input to the input layer of the neural network.

  In step 258, a pseudo brain signal output from each neuron in the intermediate layer of the neural network when image data is input to the neural network in step 256 is obtained, and a pseudo brain representing the pseudo brain signal of each neuron is obtained. Get activity data. In step 260, the simulated brain activity data is stored in the image brain signal storage unit 238 in association with the image data. In step 262, it is determined whether or not the processing in steps 254 to 260 has been executed for all image data stored in the image brain signal storage unit 238. If there is image data for which the processing in steps 254 to 260 has not been performed, the process returns to step 254, and the processing in steps 254 to 260 is repeated for the image data. On the other hand, when the processes in steps 254 to 260 are executed for all image data, the pseudo brain signal acquisition process routine is terminated.

  In addition, since it is the same as that of 1st Embodiment about the structure and effect | action of the target object identification apparatus 10 which concern on 2nd Embodiment, description is abbreviate | omitted.

  As described above, according to the object identification device according to the second embodiment, kernel correction is performed on image data and pseudo brain activity data obtained from a neural network when the image data is given. Image feature quantity that improves identification performance for image data by performing quasi-correlation analysis, learning kernel function weights, and using image-side canonical correlation variables as learned image weights Can learn.

  Next, a third embodiment will be described. A case where the present invention is applied to a sound identification device for identifying sound data will be described as an example. Note that the configurations of the sound identification device and the brain activity data measurement device according to the third embodiment are substantially the same as those of the first embodiment, and thus the same reference numerals are given and description thereof is omitted. .

  The third embodiment is different from the first embodiment in that sound data is given to the subject instead of image data, and the sound feature amount is learned from the obtained brain activity data.

  In the third embodiment, in the brain activity data measuring device, the sound output device outputs a sound to the subject based on the sound data, and the brain activity data representing the electroencephalogram signal of each part obtained at this time is acquired. .

In addition, the sound identification apparatus according to the third embodiment has a kernel function weight α that maximizes the correlation expressed by the above equation (1) from a plurality of pairs of frequency spectrum data of sound data and brain activity data. , Β, and using the learned weight α and learning sound data x ^(j) , n ₀ canonical correlation variables on the sound side represented by the above equation (6) are used as sound feature quantities. Each parameter including the weight α and the sound data x ^(j) determined and learned in the equation (6) is stored in the learning result storage unit 24.

  In addition, the sound identification device learns the SVM model using the sound feature amount learned as described above, and the frequency spectrum of the sound data input from the sound input device, as in the first embodiment. A sound feature amount is calculated from the data, and the sound data is identified using the learned SVM model.

  Thus, according to the sound identification device according to the third embodiment, kernel canonical correlation analysis is performed on sound data and brain activity data obtained when the sound data is output. By learning the weight of the kernel function and using the learned canonical correlation variable on the sound side as the sound feature value, it is possible to learn the sound feature value that improves the discrimination performance for the sound data.

  In the third embodiment, a pseudo brain such as a neural network may be used as in the method described in the second embodiment. In this case, the neural network is learned using the frequency spectrum data of the sound data, and the pseudo brain activity data when the frequency spectrum data of the sound data is input to the learned neural network is acquired. Good.

In the first to third embodiments, the image feature amount for identifying image data or sound data or the case of learning the sound feature amount has been described. In addition to hearing, sensory signals such as olfaction, touch, and taste are also discriminated, and the present invention can be applied to such other senses. For example, the present invention may be applied to an odor discriminating apparatus for discriminating whether an odor is appetizing with respect to odor data representing each signal detected by n chemical substance sensors. In the case of vision, the image data is a vector of the number of pixels, but in the case of the olfaction, the odor data is a vector vector of the number of sensors, and for the odor data and the brain activity data, A kernel canonical correlation analysis may be performed to learn olfactory features. As a result, an olfactory feature quantity I _a (x) advantageous for odor discrimination is obtained, and the odor can be identified with high accuracy based on the olfactory feature quantity.

  In addition, for example, the present invention may be applied to a tactile sensation identification device that identifies whether a material is wooden or not with a texture felt by a fingertip. In this case, kernel canonical correlation analysis is performed on the tactile data representing the outputs of the n tactile sensors and the brain activity data to learn tactile feature quantities advantageous for material discrimination.

  In the second embodiment, the case where pseudo brain activity data is acquired using a neural network (pseudo brain) has been described as an example. However, the present invention is not limited to this. It is also possible to examine how the devices exhibit a certain discrimination performance by applying the technique proposed in the present invention to many existing artificial visual processing signal devices. For example, if the artificial visual processing signal device is A => B => C =>. . . => E, if there are several processing steps in the middle, the kernel canonical correlation analysis is performed between the processing steps of interest, for example, A => C, and the data input to the A stage , It may be useful for understanding what is a characteristic quantity advantageous for identification.

  In the first to third embodiments, the case where the kernel canonical correlation analysis (KKCA) is used is described. However, it is sufficient to use the normal canonical correlation analysis (CCA). In some cases, a good performance can be obtained. Therefore, as an implementation procedure, first, the image feature amount is learned using CCA, and when the identification performance using the learned image feature amount is insufficient, the image feature amount is learned using KCCA. You may do it.

  In the first embodiment and the third embodiment, the case where brain activity data is measured for the human brain has been described as an example. However, the present invention is not limited to this, and other organisms ( For example, brain activity data may be measured for the brains of rats and monkeys. For example, if the organism can be trained so that handwritten numbers can be discriminated and the brain activity (for example, primary visual cortex) at the time of discrimination can be measured using a two-photon microscope or a multipoint electrode, it can be used as brain activity data. it can.

  Moreover, although the case where image identification processing is performed using an SVM classifier has been described as an example, the present invention is not limited to this, and other conventionally known identification methods (for example, nearest neighbor identification, linear discrimination, etc.) May be used to perform image identification processing.

  Moreover, although the case where the captured image is acquired from the imaging device and the captured image is identified as to whether it is a pedestrian image has been described as an example, the present invention is not limited thereto, and the image data is acquired from the hard disk device. You may identify whether it is a pedestrian image with respect to the read image data and read.

  In addition, a window image is extracted from the captured image using cutout windows of various sizes, and a learned image feature amount is calculated for each window image to identify whether it is a pedestrian image or not. May be.

  Moreover, although the case where the identification target object is a person has been described as an example, the present invention is not limited to this, and an object other than a person may be used as the identification target object.

  In the specification of the present application, the embodiment has been described in which the program is installed in advance. However, the program may be provided by being stored in a storage medium such as a CDROM.

  An embodiment of an object identification device to which the present invention is applied will be described. Hereinafter, examples of the object identification device 10 and the simulated brain activity data measurement device 230 described in the second embodiment will be described.

  Since handwritten numbers have various habits depending on the writer, identifying which number each handwritten number means is one of the difficult image identification problems. In particular, when a part of a handwritten character is shielded, the recognition is extremely difficult for an automatic recognition machine, but even in such a situation, the human brain can somehow identify which number. Here, an artificial neural network model (ANN) given a sufficient training set in place of the human brain and trained for a sufficient amount of time is used as a pseudo brain, and this pseudo brain is used in the second implementation described above. An embodiment will be described in which an object identification device that acquires the ability of numerical discrimination by learning by the method described in the above embodiment is manufactured.

First, as shown in FIG. 10, image data {x ⁽¹⁾ , x ⁽²⁾ ,... X ^(N) } (referred to as basic numeral image data ⁾ indicating normal handwritten numerals and shielded handwriting. Image data indicating numbers {￣x ⁽¹⁾ , ￣x ⁽²⁾ ,... ^' X ( ^{N ′)} } (referred to as shielded numeric image data) is given as a training set, and correct answers with a correct answer rate of 95% or more are given. Thus, ANN learning was performed (the largest value among the ten output layers of ANN was used as the output).

As the database of handwritten characters, the ETL6 database (numbers) and the ETL7l database (Hiragana) of the Electronic Technology Research Institute (currently the National Institute of Advanced Industrial Science and Technology) are used. I used a downsampled image. Therefore, {x ^(j) } and {￣x ^(j) } are 14 × 14 = 196-dimensional vectors, respectively.

  The trained ANN was used as a neural network of the simulated brain activity data measuring device 230.

Next, the correlation between the simulated brain activity data representing the output of 60 neurons in the intermediate layer representing the brain activity of the simulated brain and the image data is calculated by the method described in the first embodiment. As a data set at that time, image data {x ⁽¹⁾ , x ⁽²⁾ ,... X ^(N) }, {￣x ⁽¹⁾ , ￣x ⁽²⁾ ,. ￣x ^{(N ′)} } and pseudo brain activity data {y ⁽¹⁾ , y ⁽²⁾ ,... Y ^(N) } representing the output of each neuron in the corresponding intermediate layer, {￣y ^{( 1)} , ￣y ⁽²⁾ , ... ￣y ^{(N ')} }. Based on these data sets, a canonical correlation variable that maximizes the correlation was learned as an image feature amount.

As described in the first embodiment, after mapping both the image side and the brain activity side to the high-dimensional feature space, the direction to a direction a ₍₁₎ and b ₍₁₎ in each feature space is changed. Consider the projected pairs, a ^T ₍₁₎ φ (x), b ^T ₍₁₎ Ψ (y), and a ₍₁₎ and b ₍₁₎ such that these are the most correlated one-dimensional pairs. Can be found. That is a pair of first canonical correlation variables. As shown in FIG. 11A, it is confirmed that the a ^T ₍₁₎ φ (x) and b ^T ₍₁₎ Ψ (y) pairs actually calculated for this example are well correlated. (Correlation value = 0.997). The x-axis is a variable on the image side, and the y-axis is a variable on the brain activity side.

  Specifically, among the total 900 handwritten numeric image data, the first canonical correlation variable is determined using the pseudo brain activity data when 500 numeric image data is input to the ANN, and is not used for the determination. A pair of canonical correlation variables was determined for the remaining 400 handwritten numeral image data. The values of the obtained canonical correlation variable pairs are shown in FIG. 11A and indicate what numbers (0 to 9) each is. Similarly, FIGS. 11B, 12A, and 12B show pairs of second, third, and fourth canonical correlation variables. All pairs were found to correlate very well.

Of the many handwritten numeral image data, the image data representing “0” is assigned the back number from 1 to 40, the image data representing “1” is assigned the back number from 41 to 80, In this way, a total number of 400 handwritten numeral image data is assigned with a back number from 1 to 400, and FIG. 13A shows the first canonical image on the image side from the bottom to the top in ascending order of the back number. The value a ^T ₍₁₎ φ (x) of the correlation variable is shown on the horizontal axis. FIG. 13B similarly shows the value a ^T ₍₂₎ φ (x) of the second canonical correlation variable on the horizontal axis.

  The value of the canonical correlation variable is specifically calculated as the following equation (10) as a linear combination of kernel functions. The value of the canonical correlation variable is a value that can be calculated every time the image x is given. Therefore, it can be regarded as an image feature amount. It can be understood from FIGS. 13A and 13B that the image feature amount is an effective feature amount for discrimination of handwritten numerals. Because, for example, if the first canonical correlation variable is smaller than −0.02, the handwritten number is found to be {1, 4, 7} or {2, 9} with a low probability, and the first canonical correlation variable The variable is. If it is greater than 01, it can be seen that the handwritten digit is {0, 5, 6} or {2, 8} with a low probability. After limiting to this point with the first canonical correlation variable, {1, 4, 7} that could not be distinguished before can be identified with certainty by looking at the value of the second canonical correlation variable.

  In this way, the canonical correlation variable created by learning from the pseudo brain that discriminates handwritten numerals is a useful image feature amount. On the other hand, FIGS. 14A and 14B show the first and second components of PCA (principal component analysis) of an image, which is a conventional image feature quantity that cannot be learned directly from the brain. These components are extracted in the direction of the greatest dispersion from the original number of pixels of 196 dimensions. However, as can be seen from FIGS. 14A and 14B, the values of the feature quantities are greatly overlapped between different numbers, and it can be seen that they are not useful as image feature quantities for identifying the numbers.

  Thus, the significance of the algorithm learned from the brain proposed in the present invention was confirmed with respect to the conventional image discrimination algorithm not learned directly from the brain.

  Next, using all canonical correlation variables on the image side having a correlation of 0.4 or more, constructed by learning from the brain using kernel canonical correlation analysis (KCCA), the above equation (9) is used. The performance of the image discrimination algorithm KCCA-SVM using the kernel function shown in FIG.

  The issue of evaluation is the ability to respond to shielding. That is, the performance of the SVM classifier trained so that the classification of basic handwritten numerals (no occlusion) as shown in the upper left part of FIG. We tested how much it declined against handwritten numbers. Specifically, the test was performed with a two-choice question whether the given handwritten handwritten numeral is “5” or not.

  It should be noted that the evaluation here is to what extent SVM and KCCA-SVM that are not trained with shield characters can cope with shield characters, but ANN has the ability to distinguish even shielded characters. Since it is assumed that it is a “pseudo-brain”, the ANN training set includes a shield character, and the ANN is learned so that the shield character can be identified.

  First, looking at the results of a conventional SVM that uses the kernel function expressed by the following equation (11), as shown in FIG. 15, the accuracy rate drops significantly from 100% due to the shielding of handwritten characters.

  On the other hand, the accuracy rate in the KCCA-SVM proposed in the present invention is hardly affected by the shielding and is almost 100%. Both discriminant algorithms give only basic handwritten characters that do not include occlusion as the training set of the SVM model, but KCCA-SVM has learned from the pseudo-brain the general guidelines for successful character identification. As a result, it was found that it was able to effectively cope with shielding against numbers.

  Here, we prepared a pseudo brain (ANN) that can correctly discriminate both the basic character group and the character group added with the visual hindrance factor (distractor) that shields it. Is extracted using the KCCA proposed in the present invention, a character group including a shield character is used as a stimulus input to the ANN, and a canonical correlation variable is obtained using the brain activity at this time. This method can be concretely applied in the case of visual discrimination in which it is determined from the beginning what the disturbing factor that the user wants to deal with is known, and it is known in advance that the brain can cope with it.

  On the other hand, the brain generally has not only shielding but also ability to cope with various interference factors such as ink stains and changes in lighting conditions. If it is possible to learn the ability of the brain to respond to several disturbing factors without prior identification of the disturbing factors, it would be more useful as a method of learning from the brain.

  Therefore, we investigated whether it was possible to learn the ability to cope with various disturbing factors without prior identification of the disturbing factors. Since the interfering factor is not specified as “shielding”, a shielded number cannot be used as an input stimulus (hereinafter referred to as a correlation variable extraction stimulus) to the ANN when a canonical correlation variable is obtained by KCCA. Instead, here, a collection of handwritten hiragana characters as shown in FIG. 16 (and rotations, inversions, and shieldings added) is used as a correlation variable extraction stimulus. In addition, this handwritten hiragana (and the one added with rotation, inversion, and shielding) is an example of related sense data. Based on the correlation variable extraction stimulus shown in FIG. 16 above, the canonical correlation variable is obtained by the same procedure as above, and the KCCA-SVM handwritten numeral (shielded one) using the canonical correlation variable on the image side as the image feature amount. 15, it was found that even when the above handwritten hiragana is used as an input stimulus, the discrimination performance is extremely high. Even if it is not, it means that the ability of the brain to cope with disturbing factors can be read with KCCA, and an SVM classifier (CKCA-SVM) with that ability can be constructed. The usefulness was shown.

DESCRIPTION OF SYMBOLS 10 Object identification apparatus 14 Computer 20 Image brain signal memory | storage part 22 Image feature-value learning part 24 Learning result memory | storage part 26 Target object detection part 30 Brain activity data measuring device 32 Cap-type electrode 34 Display device 36 Computer 38, 238 Image brain signal Storage unit 50 Feature amount calculation unit 52 Identification model learning unit 54 Identification model storage unit 56 Feature amount calculation unit 58 Identification unit 230 Pseudo brain activity data measuring device 240 Neural network learning unit 244 Neural network identification unit

Claims (8)

Kernel canonical correlation analysis is performed on brain activity data representing the signals measured at each measurement point of the brain of the organism when sensory data or related sensory data is given, and the sensory data or the related sensory data. Learning weights in a canonical correlation variable represented by a linear sum of kernel functions so that the correlation between the brain activity data and the sensory data or the related sensory data is high, and using the learned weights A learning means that uses a quasi-correlation variable as a sensory feature amount for identifying the sensory data,
Second feature amount calculating means for calculating the sensory feature value learned by the learning means from each of a plurality of sensory data prepared in advance;
An identification model for learning an identification model for identifying the sensory data using, as learning data, an identification result obtained in advance for the sensory data corresponding to the sensory feature quantity calculated by the second feature quantity calculating means. Learning means,
Feature quantity calculating means for calculating the sensory feature quantity learned by the learning means from input sensory data;
Identification means for identifying the sensory data based on the sensory feature quantity calculated by the feature quantity calculation means and the identification model ;
Sensory data identification device including When the sensory data or the related sensory data is given to the neural network learned as learning data, a combination of the sensory data and the identification result obtained in advance for the sensory data, each neuron in the intermediate layer of the neural network A kernel canonical correlation analysis is performed on the simulated brain activity data representing the observed output and the sensory data or the related sensory data to correlate the simulated brain activity data with the sensory data or the related sensory data. So that the weight in a canonical correlation variable represented by a linear sum of kernel functions is learned, and the canonical correlation variable using the learned weight is a sensory feature amount for identifying the sensory data Learning means to
Second feature amount calculating means for calculating the sensory feature value learned by the learning means from each of a plurality of sensory data prepared in advance;
An identification model for learning an identification model for identifying the sensory data using, as learning data, an identification result obtained in advance for the sensory data corresponding to the sensory feature quantity calculated by the second feature quantity calculating means. Learning means,
Feature quantity calculating means for calculating the sensory feature quantity learned by the learning means from input sensory data;
Identification means for identifying the sensory data based on the sensory feature quantity calculated by the feature quantity calculation means and the identification model ;
Sensory data identification device including The sensory data identification device according to claim 1, wherein the sensory feature amount is represented by the following expression.


Where I _a ⁽ⁿ⁾ (x) is the sensory feature quantity of the input sensory data x, a ^T _(n) φ (x) is the nth canonical correlation variable, and x ^(j) is the j-th sensory data or the related sensory data given in advance for kernel canonical correlation analysis. k _a (x ^(j) , x) is the j-th kernel function, and α _j ⁽ⁿ⁾ is the weight for the j-th kernel function. Said learning means learns the weights in each of the plurality of canonical correlation variables, the plurality of canonical correlation variables using learned weights, one of claims 1 to 3, wherein the sensory characteristic quantity The sensory data identification device according to claim 1. The sensory data identification device according to any one of claims 1 to 4 , wherein the sensory data is image data, sound data, odor data, or tactile data. The sensory data identification device according to any one of claims 1 to 5 , further comprising a display control unit that causes the display unit to display the identification result obtained by the identification unit in association with the input sensory data. Computer
Kernel canonical correlation analysis is performed on brain activity data representing the signals measured at each measurement point of the brain of the organism when sensory data or related sensory data is given, and the sensory data or the related sensory data. Learning weights in a canonical correlation variable represented by a linear sum of kernel functions so that the correlation between the brain activity data and the sensory data or the related sensory data is high, and using the learned weights A learning means that uses a quasi-correlation variable as a sensory feature amount for identifying the sensory data,
Second feature amount calculating means for calculating the sensory feature value learned by the learning means from each of a plurality of sensory data prepared in advance;
An identification model for learning an identification model for identifying the sensory data using, as learning data, an identification result obtained in advance for the sensory data corresponding to the sensory feature quantity calculated by the second feature quantity calculating means. Learning means,
Based on the input sensory data, the feature quantity calculating means for calculating the sensory feature quantity learned by the learning means, and the sensory feature quantity calculated by the feature quantity calculating means and the identification model , A program for functioning as an identification means for identifying sensory data. Computer
When the sensory data or the related sensory data is given to the neural network learned as learning data, a combination of the sensory data and the identification result obtained in advance for the sensory data, each neuron in the intermediate layer of the neural network A kernel canonical correlation analysis is performed on the simulated brain activity data representing the observed output and the sensory data or the related sensory data to correlate the pseudo brain activity data with the sensory data or the related sensory data. So that the weight in a canonical correlation variable represented by a linear sum of kernel functions is learned, and the canonical correlation variable using the learned weight is a sensory feature amount for identifying the sensory data Learning means to do,
Second feature amount calculating means for calculating the sensory feature value learned by the learning means from each of a plurality of sensory data prepared in advance;
An identification model for learning an identification model for identifying the sensory data using, as learning data, an identification result obtained in advance for the sensory data corresponding to the sensory feature quantity calculated by the second feature quantity calculating means. Learning means,
Based on the input sensory data, the feature quantity calculating means for calculating the sensory feature quantity learned by the learning means, and the sensory feature quantity calculated by the feature quantity calculating means and the identification model , A program for functioning as an identification means for identifying sensory data.