JP5909943B2 - Information processing apparatus, estimator generation method, and program - Google Patents

Description

  The present technology relates to an information processing device, an estimator generation method, and a program.

  In recent years, attention has been focused on a technique for mechanically extracting feature values of an arbitrary data group from which it is difficult to quantitatively determine characteristics. For example, there is known a method of automatically constructing an algorithm that takes arbitrary music data as input and mechanically extracts the genre of music to which the music data belongs. The genre of music, such as jazz, classical music, and pop music, is not quantitatively determined according to the type of musical instrument and the form of performance. For this reason, it has been generally considered that it is generally difficult to mechanically extract music genres from music data given arbitrary music data.

  However, in reality, the music genre is divided into various combinations of information, such as the combination of pitches included in the music data, the combination of pitches, the combination of instrument types, and the structure of the melody line and bass line. Is potentially included. Therefore, a feature quantity extractor has been studied from the viewpoint of whether an algorithm for extracting this feature (hereinafter, feature quantity extractor) can be automatically constructed by machine learning. As one of the research results, for example, an automatic construction method of a feature quantity extractor based on a genetic algorithm described in Patent Document 1 below can be cited. A genetic algorithm is a model that takes into account the elements of selection, crossover, and mutation in the process of machine learning, following the evolutionary process of organisms.

  By using the feature quantity extractor automatic construction algorithm described in the same document, it is possible to automatically construct a feature quantity extractor that extracts music genre to which the music data belongs from arbitrary music data. In addition, the feature quantity extractor automatic construction algorithm described in the same document is very versatile and automatically uses a feature quantity extractor that extracts the feature quantity of an arbitrary data group, not limited to music data. Can be built. Therefore, the feature quantity extractor automatic construction algorithm described in this document is expected to be applied to feature quantity analysis of artificial data such as music data and video data, and feature quantity analysis of various observed quantities existing in nature. Has been.

JP 2009-48266 A

  The feature quantity extraction formula automatic construction algorithm described in the above document automatically constructs a feature quantity extraction formula using learning data prepared in advance. In addition, as the number of learning data increases, the performance of the feature quantity extraction formula that is automatically constructed improves. However, there is a limit to the amount of memory that can be used to construct a feature quantity extraction formula. In addition, when the number of learning data is large, high calculation performance is required to realize the construction of the feature quantity extraction formula. Therefore, there is a demand for a mechanism that preferentially uses useful learning data that contributes to improving the performance of the feature quantity extraction formula among learning data supplied in large quantities. By realizing this mechanism, a more accurate feature quantity extraction formula can be obtained, so it is expected to improve the performance of the estimator that estimates the result using the feature quantity extraction formula. The

  Therefore, the present technology has been devised in view of the above-described circumstances, and is a new and improved information processing apparatus capable of generating a higher performance estimator, an estimator generation method, and Intended to provide a program.

  According to an aspect of the present technology, when a plurality of learning data configured by input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, A feature vector calculation unit that calculates a feature vector that includes the output value of each basis function as an element, and a distribution of the points so that the distribution of the points specified by the feature vector in the feature space approaches a predetermined distribution An information processing apparatus comprising: a distribution adjustment unit that adjusts the function; and a function generation unit that generates an estimation function that outputs an estimated value of the objective variable according to an input of the feature quantity vector for the plurality of pieces of learning data Is provided.

  According to another aspect of the present technology, when a plurality of learning data including input data and an objective variable corresponding to the input data are given, the input data is input to a plurality of basis functions. Calculating a feature quantity vector having the output value of each basis function as an element, and setting the distribution of the point so that the distribution of the point specified by the feature quantity vector in the feature quantity space approaches a predetermined distribution. An estimator generation method is provided that includes adjusting and generating an estimation function that outputs an estimated value of the objective variable with respect to an input of the feature quantity vector for a plurality of the learning data. .

  According to another aspect of the present technology, when a plurality of learning data including input data and an objective variable corresponding to the input data are given, the input data is input to a plurality of basis functions. A feature amount vector calculation function for calculating a feature amount vector having an output value of each basis function as an element, and the distribution of points specified by the feature amount vector in the feature amount space so as to approach a predetermined distribution. A distribution adjustment function that adjusts the distribution of points; and a function generation function that generates an estimation function that outputs an estimated value of the objective variable in response to an input of the feature vector for a plurality of pieces of learning data. A program for realizing this is provided.

  Moreover, according to another viewpoint of this technique, the computer-readable recording medium with which said program was recorded is provided.

  As described above, according to the present technology, a higher-performance estimator can be generated.

It is explanatory drawing for demonstrating the structure of the system which estimates a result using the estimator constructed | assembled by machine learning. It is explanatory drawing for demonstrating the structure of the data for learning utilized for construction of an estimator. It is explanatory drawing for demonstrating the structure of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating the construction method of an estimator. It is explanatory drawing for demonstrating online learning. It is explanatory drawing for demonstrating the problem which should be solved regarding the construction method of the estimator based on offline learning, and the construction method of the estimator based on online learning. It is explanatory drawing for demonstrating the function structure of the information processing apparatus which concerns on this embodiment. It is explanatory drawing for demonstrating the detailed functional structure of the estimator construction part which concerns on this embodiment. It is explanatory drawing for demonstrating the relationship between the distribution of the data for learning in the feature-value space, and the precision of an estimator. It is explanatory drawing for demonstrating the relationship between the distribution of the data for learning in the feature-value space, the precision of an estimator, and the effect of online learning. It is explanatory drawing for demonstrating the sampling method of the data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the sampling method of the efficient data for learning which concerns on this embodiment. It is explanatory drawing for demonstrating the efficient weighting method which concerns on this embodiment. It is explanatory drawing for demonstrating the efficient weighting method which concerns on this embodiment. It is explanatory drawing for demonstrating the efficient weighting method which concerns on this embodiment. It is explanatory drawing for demonstrating the efficient weighting method which concerns on this embodiment. It is explanatory drawing for demonstrating the efficient sampling / weighting method which concerns on this embodiment. It is explanatory drawing for demonstrating the selection method of the data for learning which concerns on the modification of this embodiment. It is explanatory drawing for demonstrating the selection method of the data for learning which concerns on the modification of this embodiment. It is explanatory drawing for demonstrating the weighting method which concerns on the modification of this embodiment. It is explanatory drawing for demonstrating the selection method of the data for learning which concerns on the modification of this embodiment. It is explanatory drawing for demonstrating the weighting method which concerns on the modification of this embodiment. It is explanatory drawing for demonstrating the production | generation method of the data for learning utilized for construction of an image recognizer. It is explanatory drawing for demonstrating the production | generation method of the data for learning utilized for construction of a language analyzer. It is explanatory drawing for demonstrating the effect acquired by application of online learning. It is explanatory drawing which showed the hardware structural example which can implement | achieve the function of the information processing apparatus which concerns on this embodiment.

  Hereinafter, preferred embodiments according to the present technology will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

[About the flow of explanation]
Here, the flow of explanation described below will be briefly described.

  First, an automatic construction method for an estimator will be described with reference to FIGS. Next, an automatic construction method for an estimator based on online learning will be described with reference to FIGS. 13 and 14. Next, the functional configuration of the information processing apparatus 10 according to the present embodiment will be described with reference to FIGS. 15 and 16. Next, a learning data integration method according to the present embodiment will be described with reference to FIGS.

  Next, an efficient learning data sampling method according to the present embodiment will be described with reference to FIGS. Next, an efficient weighting method according to the present embodiment will be described with reference to FIGS. Next, a method of combining the efficient learning data sampling method and weighting method according to the present embodiment will be described with reference to FIG.

  Next, a learning data sampling method according to a modified example (modified example 1) of the present embodiment will be described with reference to FIG. Next, a learning data sampling method and weighting method according to a modified example (modified example 2) of the present embodiment will be described with reference to FIGS. 33 and 34. Next, a learning data sampling method and weighting method according to a modified example (modified example 3) of the present embodiment will be described with reference to FIGS. 35 and 36.

  Next, a method of applying the technology according to the present embodiment to the image construction machine automatic construction method will be described with reference to FIG. Next, a method of applying the technology according to the present embodiment to the language analyzer automatic construction method will be described with reference to FIG. Next, the effect of online learning according to the present embodiment will be described with reference to FIG. Next, a hardware configuration example capable of realizing the functions of the information processing apparatus 10 according to the present embodiment will be described with reference to FIG.

  Finally, the technical idea of the embodiment will be summarized and the effects obtained from the technical idea will be briefly described.

(Description item)
1: Introduction 1-1: Estimator automatic construction method
1-1-1: Configuration of estimator
1-1-2: Flow of construction process 1-2: Toward realization of online learning 2: Embodiment 2-1: Functional configuration of information processing apparatus 2-2: Method of integrating learning data
2-2-1: Distribution of learning data in feature space and accuracy of estimator
2-2-2: Configuration for sampling during data integration
2-2-3: Configuration weighted during data integration
2-2-4: Configuration for sampling and weighting during data integration 2-3: Efficient sampling / weighting method
2-3-1: Sampling method
2-3-2: Weighting method
2-3-3: Combination method 2-4: Modification examples regarding sampling processing and weighting processing
2-4-1: Modification 1 (Processing Based on Distance)
2-4-2: Modification 2 (Processing Based on Clustering)
2-4-3: Modification 3 (processing based on density estimation method)
3: Application example 3-1: Automatic construction method of image recognizer 3-2: Automatic construction method of language analyzer 4: Hardware configuration example 5: Summary

<1: Introduction>
The embodiment described later relates to an automatic construction method for an estimator. The embodiment also relates to a mechanism (hereinafter referred to as online learning) that enables addition of learning data used for construction of an estimator. Therefore, prior to describing the technology according to the embodiment in detail, problems to be solved in realizing the automatic construction method of the estimator and online learning will be described. In the following, description will be given by taking an example of an automatic construction method for an estimator based on a genetic algorithm, but the scope of application of the technique according to the embodiment is not limited to this.

[1-1: Automatic construction method of estimator]
An automatic construction method of the estimator will be described.

(1-1-1: Configuration of estimator)
First, the configuration of the estimator will be described with reference to FIGS. FIG. 1 is an explanatory diagram showing a system configuration example of a system that uses an estimator. FIG. 2 is an explanatory diagram showing a configuration example of learning data used for construction of an estimator. And FIG. 3 is explanatory drawing which showed the structure of the estimator, and the outline | summary of the construction method.

First, refer to FIG. As illustrated in FIG. 1, the construction of the estimator and the calculation of the estimated value are executed by, for example, the information processing apparatus 10. The information processing apparatus 10 constructs an estimator using a plurality of learning data (X ₁ , t ₁ ),..., (X _N , t _N ). In the following description, a set of learning data may be referred to as a learning data set. Further, the information processing apparatus 10 calculates the estimated value y from the input data X using the constructed estimator. This estimated value y is used for recognition of the input data X. For example, the recognition result YES is obtained when the estimated value y is larger than the predetermined threshold Th, and the recognition result NO is obtained when the estimated value y is smaller than the predetermined threshold Th.

Consider the configuration of the estimator more specifically with reference to FIG. The learning data set illustrated in FIG. 2 is used to construct an image recognizer that recognizes an image of “the sea”. In this case, the estimator constructed by the information processing apparatus 10 outputs an estimated value y representing the “sea likeness” of the input image. As shown in FIG. 2, the learning data, pairs of data _{X k} and target variable _{t k} (where, k = 1 to N) by constructed. Data X _k is the k-th image data (image #k). Further, the target variable t _k, 1 if the image #k is the image of the "sea", image #k is a variable which becomes 0 when not the image of the "sea".

In the example of FIG. 2, the image # 1 is an “sea” image, the image # 2 is an “sea” image, and the image #N is not an “sea” image. In this case, t ₁ = 1, t ₂ = 1,..., T _N = 0. When this learning data set is input, the information processing apparatus 10 outputs an estimator that outputs an estimated value y that represents the “sea likeness” of the input image by machine learning based on the input learning data set. To construct. The estimated value y is closer to 1 as the “sea likeness” of the input image is higher, and approaches 0 as the “sea likeness” is lower.

  Further, when input data X (image X) is newly input, the information processing apparatus 10 inputs the image X to an estimator constructed using the learning data set, and the “sea likeness” of the image X is input. An estimated value y representing is calculated. Using this estimated value y, it is possible to recognize whether the image X is an “ocean” image. For example, when the estimated value y ≧ the predetermined threshold Th, the input image X is recognized as the “sea” image. On the other hand, when the estimated value y <predetermined threshold value Th, it is recognized that the input image X is not an “ocean” image.

  The present embodiment relates to a technique for automatically constructing an estimator as described above. In addition, although the estimator used for the construction of the image recognizer has been described here, the technology according to the present embodiment can be applied to various automatic estimator construction methods. For example, it can be applied to the construction of a language analyzer, and can also be applied to the construction of a music analyzer that analyzes the melody line and chord progression of music. Furthermore, it can also be applied to the construction of a motion predictor that reproduces natural phenomena such as butterfly movement and cloud flow, and predicts natural behavior.

  For example, algorithms described in Japanese Patent Application Laid-Open No. 2009-48266, Japanese Patent Application No. 2010-159598, Japanese Patent Application No. 2010-159597, Japanese Patent Application No. 2009-277083, Japanese Patent Application No. 2009-277084, etc. Can be applied to. The present invention can also be applied to an ensemble learning method such as AdaBoost or a learning method using a kernel such as SVM or SVR. When applied to an ensemble learning method such as AdaBoost, a weak learning machine (Weak Learner) corresponds to a basis function φ described later. When applied to a learning method such as SVM or SVR, the kernel corresponds to a basis function φ described later. Note that SVM is an abbreviation for Support Vector Machine, SVR is an abbreviation for Support Vector Registration, and RVM is an abbreviation for Relevance Vector Machine.

Here, the structure of the estimator will be described with reference to FIG. As shown in FIG. 3, the estimator includes a basis function list (φ ₁ ,..., Φ _M ) and an estimation function f. The basis function list (φ ₁ ,..., Φ _M ) includes M basis functions φ _k (k = _{1 to M} ). The basis function φ _k is a function that outputs a feature value z _k according to the input of the input data X. Further, the estimation function f outputs an estimated value y according to an input of a feature vector Z = (z ₁ ,..., Z _M ) including M feature values z _k (k = 1 to M) as elements. It is a function. The basis function φ _k is generated by combining one or a plurality of processing functions prepared in advance.

As processing functions, for example, trigonometric functions, exponential functions, four arithmetic operations, digital filters, differentiation operations, median filters, normalization operations, white noise addition processing, image processing filters, and the like can be used. For example, when the input data X is an image, a basis function φ _j (X) = AddWhiteNoise (Media (Blur (X)) combining white noise addition processing AddWhiteNoise (), median filter Median (), and blurring Blur () )) Etc. are used. The basis function φ _j means that the input data X is sequentially subjected to blurring processing, median filtering processing, and white noise addition processing.

(1-1-2: Flow of construction process)
Now, the configuration of the basis function φ _k (k = 1 to M), the configuration of the basis function list, and the configuration of the estimation function f are determined by machine learning based on the learning data set. Hereinafter, the construction process of the estimator by machine learning will be described in more detail.

(overall structure)
First, the overall processing flow will be described with reference to FIG. FIG. 4 is an explanatory diagram for explaining the overall processing flow. Note that the processing described below is executed by the information processing apparatus 10.

  As shown in FIG. 4, first, a learning data set is input to the information processing apparatus 10 (S101). Note that a set of data X and an objective variable t is input as the learning data. When the learning data set is input, the information processing apparatus 10 generates a basis function by combining processing functions (S102). Next, the information processing apparatus 10 calculates the feature vector Z by inputting the data X to the basis function (S103). Next, the information processing apparatus 10 performs basis function evaluation and generation of an estimation function (S104).

  Next, the information processing apparatus 10 determines whether or not a predetermined end condition is satisfied (S105). If the predetermined end condition is satisfied, the information processing apparatus 10 advances the process to step S106. On the other hand, when the predetermined end condition is not satisfied, the information processing apparatus 10 returns the process to step S102 and repeatedly executes the processes of steps S102 to S104. When the process proceeds to step S106, the information processing apparatus 10 outputs an estimation function (S106). As described above, the processes in steps S102 to S104 are repeatedly executed. Therefore, in the following description, the basis function generated in step S102 in the τ-th iteration is referred to as a τ-th generation basis function.

(Generation of basis functions (S102))
Here, the processing in step S102 (basis function generation) will be described in more detail with reference to FIGS.

  First, referring to FIG. As illustrated in FIG. 5, the information processing apparatus 10 determines whether the current generation is the second generation or later (S111). That is, the information processing apparatus 10 determines whether or not the process in step S102 that is currently being executed is a second or subsequent repetition process. If it is the second generation or later, the information processing apparatus 10 advances the process to step S113. On the other hand, when it is not the second generation or later (when it is the first generation), the information processing apparatus 10 advances the processing to step S112. When the process proceeds to step S112, the information processing apparatus 10 randomly generates a basis function (S112). On the other hand, when the process proceeds to step S113, the information processing apparatus 10 evolutionarily generates a basis function (S113). And the information processing apparatus 10 will complete | finish the process of step S102, if the process of step S112 or S113 is completed.

(S112: Randomly generate basis functions)
Next, the process of step S112 will be described in more detail with reference to FIGS. The process of step S112 relates to a process of generating a first generation basis function.

First, referring to FIG. As illustrated in FIG. 6, the information processing apparatus 10 starts a processing loop related to the index m (m = 0 to M−1) of the basis function (S121). Next, the information processing apparatus 10 randomly generates a basis function φ _m (x) (S122). Next, the information processing apparatus 10 determines whether or not the basis function index m has reached M−1. If the basis function index m has not reached M−1, the information processing apparatus 10 The index m is incremented and the process returns to step S121 (S124). On the other hand, when the index m of the basis function is m = M−1, the information processing apparatus 10 ends the processing loop (S124). When the processing loop is ended in step S124, the information processing apparatus 10 completes the process of step S112.

(Details of step S122)
Next, the process of step S122 will be described in more detail with reference to FIG.

  When the process of step S122 is started, as illustrated in FIG. 7, the information processing apparatus 10 randomly determines a prototype of a basis function (S131). As a prototype, processing functions such as a linear term, a Gaussian kernel, and a sigmoid kernel can be used in addition to the processing functions exemplified above. Next, the information processing apparatus 10 randomly determines the parameters of the determined prototype, and generates a basis function (S132).

(S113: Evolutionary generation of basis functions)
Next, the process of step S113 will be described in more detail with reference to FIGS. The processing in step S113 relates to processing for generating basis functions of the τ generation (τ ≧ 2). Accordingly, when executing step S113, the τ-1 generation basis functions φ _{m, τ-1} (m = _{1 to} M) and the evaluation values v _m, τ− of the basis functions φ _{m, τ−1} are obtained. ₁ is obtained.

First, referring to FIG. As illustrated in FIG. 8, the information processing apparatus 10 updates the number M of basis functions (S141). That is, the information processing apparatus 10 determines the number M _τ of basis functions of the τ generation. Next, the information processing apparatus 10 evaluates the evaluation value v _τ−1 = {v _{1, τ−1} ,..., V _{M, for} the τ−1 generation basis function φ _{m, τ−1} (m = _{1 to M).} Based on _τ-1 }, e useful basis functions are selected from the τ-1 generation basis functions and set to the τ generation basis functions φ _{1, τ} ,..., φ _{e, τ} ( S142).

Next, the information processing apparatus 10 randomly selects a method of generating the remaining (M _τ −e) basis functions φ _{e + 1, τ} ,..., _{ΦMτ, τ} from cross, mutation, and random generation ( S143). When the intersection is selected, the information processing apparatus 10 proceeds with the process to step S144. If a mutation is selected, the information processing apparatus 10 advances the process to step S145. If random generation is selected, the information processing apparatus 10 advances the process to step S146.

When the process proceeds to step S144, the information processing apparatus 10, the basis functions phi ₁ is selected in step _{S142, tau,} ..., phi _e, new basis function by intersecting the basis functions selected from the _tau φ _{m ′, τ} (m ′ ≧ e + 1) is generated (S144). Also, when the process proceeds to step S145, the information processing apparatus 10, the basis functions phi ₁ is selected in step _{S142, tau,} ..., phi _e, newly mutated basis functions selected from the _tau A basis function φ _{m ′, τ} (m ′ ≧ e + 1) is generated (S145). On the other hand, when the process proceeds to step S146, the information processing apparatus 10 randomly generates a new basis function φ _{m ′, τ} (m ′ ≧ e + 1) (S146).

When any one of steps S144, S145, and S146 is completed, the information processing apparatus 10 advances the process to step S147. When the process proceeds to step S147, the information processing apparatus 10 determines whether the number of τ th generation basis functions has reached M (M = M _τ ) (S147). If the number of τ-th generation basis functions has not reached M, the information processing apparatus 10 returns the process to step S143 again. On the other hand, when the number of τ th generation basis functions reaches M, the information processing apparatus 10 ends the process of step S113.

(Details of S144: Intersection)
Next, the process of step S144 will be described in more detail with reference to FIG.

When the process of step S144 is started, as shown in FIG. 9, the information processing apparatus 10 selects a basis function having the same prototype from the basis functions φ _{1, τ} ,..., Φ _{e, τ} selected in step S142. Two are selected at random (S151). Next, the information processing apparatus 10 generates a new basis function by intersecting the parameters of the two selected basis functions (S152).

(Details of S145: Mutation)
Next, the process of step S145 will be described in more detail with reference to FIG.

When the process of step S145 is started, as shown in FIG. 10, the information processing apparatus 10 randomly selects _one basis function from the basis functions φ _{1, τ} ,..., Φ _{e, τ} selected in step S142. Select (S161). Next, the information processing apparatus 10 randomly changes some of the parameters of the selected basis function to generate a new basis function (S162).

(Details of S146: random generation)
Next, the process of step S146 will be described in more detail with reference to FIG.

  When the process of step S122 is started, as illustrated in FIG. 7, the information processing apparatus 10 randomly determines a prototype of a basis function (S131). As a prototype, processing functions such as a linear term, a Gaussian kernel, and a sigmoid kernel can be used in addition to the processing functions exemplified above. Next, the information processing apparatus 10 randomly determines the parameters of the determined prototype, and generates a basis function (S132).

  The processing in step S102 (basis function generation) has been described in detail above.

(Calculation of basis function (S103))
Next, the process of step S103 (basis function calculation) will be described in more detail with reference to FIG.

As illustrated in FIG. 11, the information processing apparatus 10 starts a processing loop related to the index i of the i-th data X ⁽ⁱ⁾ included in the learning data set (S171). For example, when a set of N data {X ⁽¹⁾ ,..., X ^(N) } is input as a learning data set, a processing loop is executed for i = 1 to ^N. Next, the information processing apparatus 10 starts a processing loop regarding the index m of the basis function φ _m (S172). For example, when M basis functions are generated, a processing loop is executed for m = 1 to M.

Next, the information processing apparatus 10 calculates a feature quantity z _mi = φ _m (x ⁽ⁱ⁾ ) (S173). Next, the information processing apparatus 10 proceeds with the process to step S <b> 174 and continues the processing loop regarding the index m of the basis function. Then, when the processing loop related to the index m of the basis function ends, the information processing apparatus 10 proceeds with the process to step S175 and continues the processing loop related to the index i. When the processing loop related to the index i ends, the information processing apparatus 10 ends the process of step S103.

  The processing in step S103 (basis function calculation) has been described in detail above.

(Generation of basis function evaluation / estimation function (S104))
Next, the process of step S104 (basis function evaluation / estimation function generation) will be described in more detail with reference to FIG.

As illustrated in FIG. 12, the information processing apparatus 10 calculates parameters w = {w ₀ ,..., W _M } of the estimation function by regression / discriminative learning based on the AIC standard increase / decrease method (S181). That is, the information processing apparatus 10 is configured so that a set (i = 1 to N) of the feature quantity z _mi = φ _{m, τ} (x ⁽ⁱ⁾ ) and the objective variable t ⁽ⁱ⁾ is fitted by the estimation function f. Vector w = {w ₀ ,..., W _M } is obtained by regression / discriminative learning. However, the estimation function f (x) is assumed to be f (x) = Σw _m φ _{m, τ} (x) + w ₀ . Next, the information processing apparatus 10 sets the evaluation value v of the basis function for which the parameter w is 0 to 0, and sets the evaluation value v of the other basis functions to 1 (S182). That is, a basis function having an evaluation value v of 1 is a useful basis function.

  The processing in step S104 (basis function evaluation / estimation function generation) has been described in more detail above.

  The flow of processing related to the construction of the estimator is as described above. As described above, the processes in steps S102 to S104 are repeatedly executed, and the basis function is sequentially updated by an evolutionary method, thereby obtaining an estimation function with high estimation accuracy. That is, a high-performance estimator can be automatically constructed by applying the above method.

[1-2: Toward realization of online learning]
As described above, in the case of an algorithm that automatically constructs an estimator by machine learning, the performance of the constructed estimator increases as the number of learning data increases. Therefore, it is preferable to construct an estimator using as much learning data as possible. However, the memory capacity of the information processing apparatus 10 that can be used for storing learning data is limited. In addition, when the number of learning data increases, the calculation performance required for realizing the construction of the estimator also increases. For these reasons, as long as the above method for constructing an estimator by batch processing (hereinafter, offline learning) is used, the performance of the estimator is limited by the resources of the information processing apparatus 10.

  Therefore, the present inventors have devised a mechanism (hereinafter referred to as online learning) that allows learning data to be added sequentially. The construction of the estimator for online learning is performed along the flow of processing as shown in FIG. As shown in FIG. 13, first, a learning data set is input to the information processing apparatus 10 (Step 1). Next, the information processing apparatus 10 uses the input learning data set, and constructs an estimator by the estimator automatic construction method already described (Step 2).

  Next, the information processing apparatus 10 acquires additional learning data at any time or at a predetermined timing (Step 3). Next, the information processing apparatus 10 integrates the learning data set input in (Step 1) and the learning data acquired in (Step 3) (Step 4). At this time, the information processing apparatus 10 performs a learning data sampling process and a weighting process, and generates an integrated learning data set. Then, the information processing apparatus 10 constructs an estimator again using the integrated learning data set (Step 2). At this time, the information processing apparatus 10 constructs an estimator using the estimator automatic construction method described above.

  Note that the estimator constructed in (Step 2) may be output every time it is constructed. Further, the processing of (Step 2) to (Step 4) is repeatedly executed. The learning data set is updated each time the process is repeated. For example, if learning data is added each time it is repeated, the number of pieces of learning data used for the estimator construction process increases, so that the performance of the estimator improves. However, since the resources of the information processing apparatus 10 are limited, the integration process of learning data executed in (Step 4) is performed so that more useful learning data is used for the construction of the estimator. It is necessary to devise a way.

(Organization of issues)
As shown in FIG. 14, when offline learning is applied, there is a limit to the further performance improvement of the estimator because the number of learning data available for construction of the estimator is limited. On the other hand, when online learning is applied, it becomes possible to add learning data, so that further improvement in the performance of the estimator can be expected. However, since the resources of the information processing apparatus 10 are limited, it is necessary to devise a method for integrating learning data so that the performance of the estimator can be further improved within the limited resources. The technology according to the following embodiments has been devised to solve these problems.

<2: Embodiment>
An embodiment of the present technology will be described.

[2-1: Functional Configuration of Information Processing Apparatus 10]
First, the functional configuration of the information processing apparatus 10 according to the present embodiment will be described with reference to FIGS. 15 and 16. FIG. 15 is an explanatory diagram showing an overall functional configuration of the information processing apparatus 10 according to the present embodiment. On the other hand, FIG. 16 is an explanatory diagram showing a detailed functional configuration of the estimator construction unit 12 configuring the information processing apparatus 10 according to the present embodiment.

(Overall functional configuration)
First, an overall functional configuration will be described with reference to FIG. As illustrated in FIG. 15, the information processing apparatus 10 mainly includes a learning data acquisition unit 11, an estimator construction unit 12, an input data acquisition unit 13, and a result recognition unit 14.

  When the estimator construction process is started, the learning data acquisition unit 11 acquires learning data used for construction of the estimator. For example, the learning data acquisition unit 11 reads learning data stored in a storage device (not shown). Alternatively, the learning data acquisition unit 11 acquires learning data via a network from a system that provides learning data. In addition, the learning data acquisition unit 11 may acquire data with a tag and generate learning data including a set of data and an objective variable based on the tag.

  A set of learning data (learning data set) acquired by the learning data acquisition unit 11 is input to the estimator construction unit 12. When the learning data set is input, the estimator construction unit 12 constructs an estimator by machine learning based on the input learning data set. For example, the estimator construction unit 12 constructs an estimator using the method for automatically constructing an estimator based on the genetic algorithm already described. When additional learning data is input from the learning data acquisition unit 11, the estimator construction unit 12 integrates the learning data and constructs an estimator using the integrated learning data set. .

  The estimator constructed by the estimator construction unit 12 is input to the result recognition unit 14. This estimator is used to obtain a recognition result for arbitrary input data. When the input data to be recognized is acquired by the input data acquisition unit 13, the acquired input data is input to the result recognition unit 14. When input data is input, the result recognition unit 14 inputs the input data to the estimator, and generates a recognition result based on the estimated value output from the estimator. For example, as illustrated in FIG. 1, the result recognition unit 14 compares the estimated value y with a predetermined threshold Th and outputs a recognition result according to the comparison result.

  The overall functional configuration of the information processing apparatus 10 has been described above.

(Functional configuration of the estimator construction unit 12)
Next, the functional configuration of the estimator construction unit 12 will be described in detail with reference to FIG. As illustrated in FIG. 16, the estimator construction unit 12 includes a basis function list generation unit 121, a feature amount calculation unit 122, an estimation function generation unit 123, and a learning data integration unit 124.

  When the estimator construction process is started, the basis function list generation unit 121 first generates a basis function list. The basis function list generated by the basis function list generation unit 121 is input to the feature amount calculation unit 122. A learning data set is input to the feature amount calculation unit 122. When the basis function list and the learning data set are input, the feature amount calculation unit 122 calculates the feature amount by inputting the data of the input learning data set to each basis function included in the basis function list. A set of feature amounts (feature amount vector) calculated by the feature amount calculation unit 122 is input to the estimation function generation unit 123.

  When the feature quantity vector is input, the estimation function generation unit 123 generates an estimation function by regression / discriminant learning based on the input feature quantity vector and the objective variable constituting the learning data. In addition, when applying the construction method of the estimator based on the genetic algorithm, the estimation function generation unit 123 calculates the contribution rate (evaluation value) of each basis function with respect to the generated estimation function, and ends based on the contribution rate It is determined whether or not the above is satisfied. When the termination condition is satisfied, the estimation function generation unit 123 outputs an estimator including a basis function list and an estimation function.

  On the other hand, when the termination condition is not satisfied, the estimation function generation unit 123 notifies the basis function list generation unit 121 of the contribution rate of each basis function to the generated estimation function. Upon receiving this notification, the basis function list generation unit 121 updates the basis function list based on the contribution rate of each basis function using a genetic algorithm. When the basis function list is updated, the basis function list generation unit 121 inputs the updated basis function list to the feature amount calculation unit 122. When the updated basis function list is input, the feature amount calculation unit 122 calculates a feature amount vector using the updated basis function list. The feature amount vector calculated by the feature amount calculation unit 122 is input to the estimation function generation unit 123.

  As described above, when applying the construction method of the estimator based on the genetic algorithm, the estimation function generation process by the estimation function generation unit 123 and the update of the basis function list by the basis function list generation unit 121 until the termination condition is satisfied The processing and the feature amount vector calculation processing by the feature amount calculation unit 122 are repeatedly executed. When the end condition is satisfied, the estimation function generation unit 123 outputs an estimator.

  When the additional learning data is input, the input additional learning data is input to the feature amount calculation unit 122 and the learning data integration unit 124. When the additional learning data is input, the feature amount calculation unit 122 generates the feature amount by inputting the data constituting the additional learning data to each basis function included in the basis function list. Then, the feature amount vector corresponding to the additional learning data and the feature amount vector corresponding to the existing learning data are input to the learning data integration unit 124. It is assumed that existing learning data is also input to the learning data integration unit 124.

  The learning data integration unit 124 integrates an existing learning data set and additional learning data based on a learning data integration method described later. For example, the learning data integration unit 124 thins out the learning data so that the distribution of coordinates (hereinafter referred to as feature amount coordinates) indicated by the feature amount vector in the feature amount space becomes a predetermined distribution, or weights the learning data. Or setting. When the learning data is thinned out, the thinned learning data set is used as the integrated learning data set. On the other hand, when the weight is set for the learning data, the weight set for each learning data is taken into account in the regression / discriminative learning by the estimation function generation unit 123.

  When the learning data is integrated, the estimator automatic construction process is executed using the integrated learning data set. Specifically, the learning data set after integration and the feature vector corresponding to the learning data included in the learning data set after integration are input from the learning data integration unit 124 to the estimation function generation unit 123. The estimation function generation unit 123 generates an estimation function. In addition, when applying an estimator construction method based on a genetic algorithm, processing such as generation of an estimation function, calculation of a contribution rate, and update of a basis function list is performed using the integrated learning data set.

  The detailed functional configuration of the estimator construction unit 12 has been described above.

[2-2: Method for integrating learning data]
Next, a learning data integration method according to the present embodiment will be described. The learning data integration method described here is realized by the function of the learning data integration unit 124.

(2-2-1: Distribution of learning data in feature space and accuracy of estimator)
First, the relationship between the distribution of learning data in the feature amount space and the accuracy of the estimator will be considered with reference to FIG. FIG. 17 is an explanatory diagram showing a distribution example of learning data in the feature amount space.

  One feature vector is obtained by inputting data constituting one learning data to each basis function included in the basis function list. That is, one feature vector (feature amount coordinate) corresponds to one learning data. For this reason, the distribution of feature amount coordinates is referred to as the distribution of learning data in the feature amount space. The distribution of learning data in the feature amount space is, for example, as shown in FIG. For the convenience of expression, the example of FIG. 17 considers a two-dimensional feature amount space, but the number of dimensions of the feature amount space is not limited to this.

  Now, referring to the distribution of the feature amount coordinates illustrated in FIG. 17, it will be noticed that a sparse region exists in the fourth quadrant. As already described, the estimation function is generated by regression / discriminant learning so that the relationship between the feature vector and the objective variable is well expressed for all learning data. For this reason, there is a high possibility that the estimation function cannot express the relationship between the feature quantity vector and the objective variable well in an area where the density of feature quantity coordinates is sparse. Therefore, it is difficult to expect a highly accurate recognition result when the feature amount coordinates corresponding to the input data to be recognized are located in the sparse region.

  As shown in FIG. 18, an estimator capable of outputting a recognition result with high accuracy even if input data corresponding to any region is input is less likely to occur when the number of learning data increases. Expected to be able to build. In addition, even if the number of learning data is relatively small, it is expected that an estimator capable of outputting recognition results with high accuracy can be constructed if the feature amount coordinates are evenly distributed in the feature amount space. The Therefore, the present inventor considers the distribution of feature amount coordinates when integrating learning data, and the distribution of feature amount coordinates corresponding to the integrated learning data set is a predetermined distribution (for example, uniform distribution or We have devised a mechanism to adjust it so that it has a Gaussian distribution.

(2-2-2: Configuration for sampling during data integration)
First, a method for sampling learning data will be described with reference to FIG. FIG. 19 is an explanatory diagram for explaining a method of sampling learning data.

  As described above, when online learning is applied, learning data can be added sequentially, so that an estimator can be constructed using a large amount of learning data. However, when the memory resources of the information processing apparatus 10 are limited, it is necessary to narrow down the number of learning data used for constructing the estimator when integrating the learning data. At this time, it is possible to narrow down the number of learning data without degrading the accuracy of the estimator by thinning out the learning data in consideration of the distribution of the feature amount coordinates instead of randomly thinning out the learning data. it can. For example, as shown in FIG. 19, many feature amount coordinates included in a dense region are thinned out, and feature amount coordinates included in a sparse region are left as much as possible.

  By thinning out the learning data by such a method, the density of the feature amount coordinates corresponding to the integrated learning data set becomes uniform. In other words, the number of learning data is reduced, but the feature amount coordinates are distributed evenly throughout the feature amount space. Therefore, the entire feature amount space is used in the regression / discriminative learning executed when the estimation function is generated. Will be considered. As a result, even if the memory resource of the information processing apparatus 10 is limited, it is possible to construct an estimator that can estimate a correct recognition result with high accuracy.

(2-2-3: Configuration weighted during data integration)
Next, a method for setting weights for learning data will be described.

  When the memory resources of the information processing apparatus 10 are limited, a method of thinning out learning data when integrating learning data is effective. On the other hand, when there is a margin in memory resources, it is possible to improve the performance of the estimator by setting weights on the learning data instead of thinning out the learning data. For example, a large weight is set for learning data in which feature amount coordinates are included in a sparse region, and a small weight is set in learning data in which feature amount coordinates are included in a dense region. Then, the weight set for each learning data is taken into consideration in the regression / discriminative learning executed when the estimation function is generated.

(2-2-4: Configuration for sampling and weighting during data integration)
Further, a method of sampling learning data may be combined with a method of setting weights on learning data. For example, after thinning out the learning data so that the distribution of the feature amount coordinates becomes a predetermined distribution, a weight corresponding to the density of the feature amount coordinates is set for the learning data included in the thinned learning data set To do. Thus, by combining the thinning process and the weighting process, it is possible to construct a more accurate estimator even if the memory resources are limited.

[2-3: Efficient sampling / weighting method]
Next, an efficient sampling / weighting method for learning data will be described.

(2-3-1: Sampling method)
First, an efficient sampling method for learning data will be described with reference to FIG. FIG. 20 is an explanatory diagram for explaining an efficient sampling method of learning data.

  As illustrated in FIG. 20, the information processing apparatus 10 calculates feature amount vectors (feature amount coordinates) for all the learning data using the function of the feature amount calculation unit 122 (S201). Next, the information processing apparatus 10 normalizes the calculated feature amount coordinates by the function of the feature amount calculation unit 122 (S202). For example, as shown in FIG. 21, the feature quantity calculation unit 122 normalizes the values so that the variance is 1 and the average is 0 for each feature quantity. The feature amount coordinates normalized in this way are input to the learning data integration unit 124.

Next, the information processing apparatus 10 randomly generates a hash function g by the function of the learning data integration unit 124 (S203). For example, the learning data integration unit 124 generates a plurality of hash functions g that output a 5-bit value as shown in the following equation (1). At this time, the learning data integration unit 124 generates Q hash functions g _q (q = 1 to Q). However, the function h _j (j = 1 to 5) is defined by the following equation (2). D and Threshold are determined by random numbers.

However, when the distribution of the feature amount coordinates is made closer to the uniform distribution, a uniform random number is used as a random number used for determination of Threshold. Further, when the distribution of the feature amount coordinates is made close to a Gaussian distribution, a Gaussian random number is used as a random number used for determination of Threshold. The same applies to other distributions. Also, the determination of d is carried out using a random number that is biased in accordance with the contribution ratio of the basis functions used to calculate the z _d. For example, a random number whose probability of occurrence of d increases as the contribution rate of the basis function used for calculating z _d increases is used.

When the hash function g _q (q = 1 to Q) is generated, the learning data integration unit 124 inputs the feature quantity vector Z corresponding to each learning data to the hash function g _q and calculates a hash value. Then, the learning data integration unit 124 assigns learning data to the bucket based on the calculated hash value (S204). However, the bucket mentioned here means an area associated with values that can be taken as hash values.

For example, consider the case where the hash value is 5 bits and Q = 256. In this case, the configuration of the bucket is as shown in FIG. As shown in FIG. 22, since the hash value is 5 bits, 32 buckets (hereinafter referred to as bucket sets) are provided for one hash function g _q . Since Q = 256, 256 bucket sets are provided. A method of assigning learning data to buckets will be described along this example.

Given a feature vector Z corresponding to certain learning data, 256 hash values are calculated using ₂₅₆ hash functions g _{1 to} g ₂₅₆ . For example, when g ₁ (Z) = 2 (decimal number display), the learning data integration unit 124 assigns the learning data to the bucket corresponding to 2 in the bucket set corresponding to g ₁ . Similarly, g _q (Z) (q = 2 to 256) is calculated, and learning data is assigned to the bucket corresponding to each value. In the example of FIG. 22, two types of learning data are expressed by white circles and black circles, and the corresponding relationship with each bucket is schematically expressed.

  When each learning data is assigned to the bucket in this way, the learning data integration unit 124 selects one learning data from the bucket in a predetermined order (S205). For example, as shown in FIG. 23, the learning data integration unit 124 scans the bucket in order from the upper left (the side where the index q of the hash function is small and the value assigned to the bucket is small), and learning assigned to the bucket. Select one data.

  The rules for selecting learning data from the bucket are as shown in FIG. First, the learning data integration unit 124 skips empty buckets. Second, when one learning data is selected, the learning data integration unit 124 excludes the same learning data from all other buckets. Third, the learning data integration unit 124 randomly selects one learning data when a plurality of learning data is assigned to one bucket. Note that the information of the selected learning data is held by the learning data integration unit 124.

  After selecting one piece of learning data, the learning data integration unit 124 determines whether or not a predetermined number of pieces of learning data have been selected (S206). When the selection of the predetermined number of learning data has been completed, the learning data integration unit 124 outputs the selected predetermined number of learning data as an integrated learning data set, and a series of learning data integration series. The process ends. On the other hand, if the predetermined number of learning data has not been selected, the learning data integration unit 124 proceeds with the process to step S205.

  In the foregoing, an efficient sampling method for learning data has been described. The correspondence relationship between the feature amount space and the bucket is as shown in the image diagram of FIG. Further, the result of sampling the learning data by the above method is, for example, as shown in FIG. 26 (an example of uniform distribution). Referring to FIG. 26, it can be seen that the feature amount coordinates included in the sparse region remain and the feature amount coordinates included in the dense region are thinned out. Note that when the above bucket is not used, the computation load required for sampling the learning data is significantly increased.

(2-3-2: Weighting method)
Next, an efficient weighting method for learning data will be described with reference to FIG. FIG. 27 is an explanatory diagram for explaining an efficient weighting method for learning data.

  As illustrated in FIG. 27, the information processing apparatus 10 calculates feature amount vectors (feature amount coordinates) for all learning data using the function of the feature amount calculation unit 122 (S211). Next, the information processing apparatus 10 normalizes the calculated feature amount coordinates by the function of the feature amount calculation unit 122 (S212). For example, as shown in FIG. 21, the feature quantity calculation unit 122 normalizes the values so that the variance is 1 and the average is 0 for each feature quantity. The feature amount coordinates normalized in this way are input to the learning data integration unit 124.

Next, the information processing apparatus 10 randomly generates a hash function g by the function of the learning data integration unit 124 (S213). For example, the learning data integration unit 124 generates a plurality of hash functions g that output a 5-bit value as shown in the above equation (1). At this time, the learning data integration unit 124 generates Q hash functions g _q (q = 1 to Q). However, the function h _j (j = 1 to 5) is defined by the above equation (2). D and Threshold are determined by random numbers.

However, when the distribution of the feature amount coordinates is made closer to the uniform distribution, a uniform random number is used as a random number used for determination of Threshold. Further, when the distribution of the feature amount coordinates is made close to a Gaussian distribution, a Gaussian random number is used as a random number used for determination of Threshold. The same applies to other distributions. Also, the determination of d is carried out using a random number that is biased in accordance with the contribution ratio of the basis functions used to calculate the z _d. For example, a random number whose probability of occurrence of d increases as the contribution rate of the basis function used for calculating z _d increases is used.

When the hash function g _q (q = 1 to Q) is generated, the learning data integration unit 124 inputs the feature quantity vector Z corresponding to each learning data to the hash function g _q and calculates a hash value. The learning data integration unit 124 then assigns learning data to the bucket based on the calculated hash value (S214). Next, the learning data integration unit 124 calculates the density for each learning data (S215). For example, assume that learning data is assigned to a bucket as shown in FIG. Also, pay attention to the learning data expressed by white circles.

In this case, the learning data integration unit 124 first counts the number of learning data assigned to buckets including white circles for the bucket set corresponding to each hash function. For example, referring to the bucket set corresponding to the hash function g ₁ , the number of learning data assigned to the bucket including the white circle is 1. Similarly, referring to bucket set corresponding to a hash function g _2, the number of learning data that are assigned to a bucket containing white circle is 2. The learning data integration unit 124 counts the number of learning data assigned to buckets including white circles for the bucket sets corresponding to the hash functions g _{1 to} g ₂₅₆ .

  Then, the learning data integration unit 124 calculates the average value of the counted numbers, and regards the calculated average value as the density of the learning data corresponding to the white circles. Similarly, the learning data integration unit 124 calculates the density of all learning data. The density of each learning data is expressed as shown in FIG. 29B. However, the density of the dark part is high and the density of the light part is low.

  Now, after calculating the density for all the learning data, the learning data integration unit 124 proceeds with the process to step S217 (S216). When the process proceeds to step S217, the learning data integration unit 124 calculates a weight to be set for each learning data from the calculated density (S217). For example, the learning data integration unit 124 sets the inverse of the density as the weight. The distribution of weights set for each learning data is expressed as shown in FIG. 30B. However, the weight of the dark portion is large and the weight of the light portion is small. Referring to FIG. 30, it can be seen that the weight of the dense area is small and the weight of the sparse area is large.

  When the calculation of the weights to be set for each learning data is completed as described above, the learning data integration unit 124 ends the series of processes related to the weighting. The efficient weighting method for learning data has been described above. Note that if the above bucket is not used, the computational load required for weighting the learning data is significantly increased.

(2-3-3: Combination method)
Next, a method of combining the efficient sampling method and the efficient weighting method will be described with reference to FIG. FIG. 31 is an explanatory diagram for explaining a method of combining the efficient sampling method and the efficient weighting method.

  As illustrated in FIG. 31, the learning data integration unit 124 first performs a sampling process of learning data (S221). This sampling process is executed along the flow of the process shown in FIG. When a predetermined number of learning data is obtained, the learning data integration unit 124 performs weighting processing on the obtained learning data (S222). This weighting process is executed along the process flow shown in FIG. Note that a feature vector or hash function calculated during the sampling process may be used. When the sampling process and the weighting process are finished, the learning data integration unit 124 ends the series of processes.

  The efficient sampling / weighting method of learning data has been described above. Here, the efficient sampling / weighting method for efficiently bringing the distribution of the feature amount coordinates close to the predetermined distribution has been described, but the application range of the data sampling / weighting method using the bucket is not limited to this. . For example, by assigning data to a bucket based on a hash function for an arbitrary data group and sampling data from the bucket according to the rules shown in FIG. 24, the distribution of the arbitrary data group can be efficiently converted to a predetermined distribution. You can get closer. The same applies to the weighting process.

[2-4: Modified example regarding sampling process and weighting process]
Next, modified examples related to sampling processing and weighting processing will be introduced.

(2-4-1: Modification 1 (Processing Based on Distance))
First, a learning data sampling method based on the distance between feature amount coordinates will be described with reference to FIG. FIG. 32 is an explanatory diagram for describing a method of sampling learning data based on the distance between feature amount coordinates.

As shown in FIG. 32, the learning data integration unit 124 first randomly selects one feature amount coordinate (S231). Next, the learning data integration unit 124 initializes the index j to 1 (S232). Next, the learning data integration unit 124 sets the j-th feature amount coordinate among the J feature amount coordinates not yet selected as the target coordinate (S233). Next, the learning data integration unit 124 calculates the distance D between all the feature amount coordinates already selected and the target coordinates (S234). Next, the learning data integration unit 124 extracts the calculated minimum value D _min of the distance D (S235).

Next, the learning data integration unit 124 determines whether j = J (S236). If j = J, the learning data integration unit 124 proceeds with the process to step S237. On the other hand, if j ≠ J, the learning data integration unit 124 advances the process to step S233. When the process has proceeded to step S237, the learning data integration unit 124 selects target coordinates (feature amount coordinates) that maximize the minimum value _Dmin (S237). Next, the learning data integration unit 124 determines whether or not the number of feature amount coordinates selected in steps S231 and S237 has reached a predetermined number (S238).

  When the number of feature amount coordinates selected in steps S231 and S237 reaches a predetermined number, the learning data integration unit 124 sets the learning data corresponding to the selected feature amount coordinates as a learning data set after integration. Output, and the series of processing ends. On the other hand, if the number of feature amount coordinates selected in steps S231 and S237 does not reach the predetermined number, the learning data integration unit 124 advances the process to step S232.

  Heretofore, the learning data sampling method based on the distance between the feature amount coordinates has been described.

(2-4-2: Modification 2 (Processing Based on Clustering))
Next, a learning data sampling / weighting method based on clustering will be described. In the following, the sampling method and the weighting method will be described separately, but these methods may be combined.

(Selecting learning data)
First, a learning data sampling method based on clustering will be described with reference to FIG. FIG. 33 is an explanatory diagram for describing a method of sampling learning data based on clustering.

  As shown in FIG. 33, first, the learning data integration unit 124 classifies the feature vector into a predetermined number of clusters (S241). As a clustering method, for example, a method such as a k-means method or hierarchical clustering can be used. Next, the learning data integration unit 124 selects feature quantity vectors one by one from each cluster in order (S242). Then, the learning data integration unit 124 outputs a learning data set corresponding to the selected feature vector as a learning data set after integration, and the series of processing ends.

(Weight setting)
Next, a learning data weighting method based on clustering will be described with reference to FIG. FIG. 34 is an explanatory diagram for describing a learning data weighting method based on clustering.

  As shown in FIG. 34, first, the learning data integration unit 124 classifies the feature vector into a predetermined number of clusters (S251). As a clustering method, for example, a method such as a k-means method or hierarchical clustering can be used. Next, the learning data integration unit 124 counts the number of elements in each cluster and calculates the reciprocal of the number of elements (S252). Then, the learning data integration unit 124 outputs the reciprocal of the calculated number of elements as a weight, and ends the series of processes.

  The learning data sampling / weighting method based on clustering has been described above.

(2-4-3: Modification 3 (Processing Based on Density Estimation Method))
Next, a learning data sampling / weighting method based on the density estimation method will be described. In the following, the sampling method and the weighting method will be described separately, but these methods may be combined.

(Selecting learning data)
First, a learning data sampling method based on a density estimation method will be described with reference to FIG. FIG. 35 is an explanatory diagram for describing a learning data sampling method based on the density estimation method.

  As shown in FIG. 35, first, the learning data integration unit 124 models the density of feature amount coordinates (S261). For the density modeling, for example, a density estimation method such as GMM (Gaussian Mixture Model) is used. Next, the learning data integration unit 124 calculates the density of each feature amount coordinate based on the constructed model (S262). Next, the learning data integration unit 124 randomly selects feature amount coordinates with a probability proportional to the reciprocal of the density from the feature amount coordinates not yet selected (S263).

  Next, the learning data integration unit 124 determines whether a predetermined number of feature amount coordinates has been selected (S264). If the predetermined number of feature amount coordinates has not been selected, the learning data integration unit 124 proceeds with the process to step S263. On the other hand, when a predetermined number of feature amount coordinates are selected, the learning data integration unit 124 outputs a set of learning data corresponding to the selected feature amount coordinates as an integrated learning data set, and performs a series of processing. finish.

(Weight setting)
Next, a learning data weighting method based on the density estimation method will be described with reference to FIG. FIG. 36 is an explanatory diagram for describing a learning data weighting method based on the density estimation method.

  As shown in FIG. 36, first, the learning data integration unit 124 models the density of feature amount coordinates (S271). For the density modeling, for example, a density estimation method such as GMM is used. Next, the learning data integration unit 124 calculates the density of each feature amount coordinate based on the constructed model (S272). Then, the learning data integration unit 124 sets the reciprocal of the calculated density as a weight, and ends a series of processing.

  The learning data sampling / weighting method based on the density estimation method has been described above.

<3: Application example>
Next, application examples of the technology according to the present embodiment will be described. The application range of the technology according to this embodiment is wide, and it can be used for automatic construction of various discriminators and analyzers such as image data discriminators, text data discriminators, audio data discriminators, and signal data discriminators. Can be applied. Here, as an example, the application to the automatic construction method of the image recognizer and the application to the automatic construction method of the language analyzer are introduced.

[3-1: Automatic construction method of image recognizer]
First, application to an automatic construction method for an image recognizer will be described with reference to FIG. FIG. 37 is an explanatory diagram for describing a method for generating a learning data set used for constructing an image recognizer. Note that the image recognizer here automatically recognizes whether an image is a “flower” image, an “empty” image, or a “sushi” image, for example. It means the algorithm to do.

  In the above description, the description has been made on the assumption that learning data composed of the data X and the objective variable t is given. However, when online learning is assumed, it is desirable that the learning data set be automatically generated from information obtained by crawling a Web service (hereinafter, acquired information), for example. For example, assume that acquired information as shown in FIG. 37A is obtained. This acquired information includes an image and a tag attached to the image. For example, when constructing an image recognizer that recognizes whether or not an input image is an image of “flower”, the information processing apparatus 10 sets an objective variable t = for an image including “flower” in the tag. 1 is assigned, and the objective variable t = 0 is assigned to the other images (see FIG. B).

  Similarly, when constructing an image recognizer that recognizes whether or not the input image is an “empty” image, the information processing apparatus 10 performs the objective variable t for an image including “empty” in the tag. = 1 is assigned, and the objective variable t = 0 is assigned to the other images (see FIG. C). Further, when constructing an image recognition device that recognizes whether or not the input image is an image of “sushi”, the information processing apparatus 10 sets a target variable t = for an image including “sushi” in the tag. 1 is assigned, and the objective variable t = 0 is assigned to other images (see FIG. D). By using the tag in this way, a learning data set that can be used to construct a desired image recognizer can be generated.

  If the learning data set can be generated, the learning data integration process and the estimator construction process described above are executed and used for an image recognizer (means for obtaining a recognition result from the estimated value y). An estimator (calculating means for the estimated value y) can be automatically constructed. The application to the automatic construction method of the image recognizer has been described above.

[3-2: Automatic construction method of language analyzer]
Next, application of the language analyzer to the automatic construction method will be described with reference to FIG. FIG. 38 is an explanatory diagram for describing a method for generating a learning data set used for constructing a language analyzer. Note that the language analyzer here refers to, for example, when a sentence is entered, it automatically recognizes whether the sentence is a sentence related to "politics", a sentence related to "economics", or a sentence related to "entertainment" It means the algorithm to do.

  In the above description, the description has been made on the assumption that learning data composed of the data X and the objective variable t is given. However, when online learning is assumed, it is desirable that the learning data set be automatically generated from information (acquired information) obtained by crawling a Web service, for example. For example, assume that acquired information as shown in FIG. 38A is obtained. This acquired information includes a sentence and a tag attached to the sentence. For example, when constructing a language analyzer that recognizes whether or not an input sentence is a sentence related to “politics”, the information processing apparatus 10 sets an objective variable t = 1 for a sentence whose tag is “politics”. Assign the target variable t = 0 to the other sentence (see FIG. B).

  Similarly, when constructing a language analyzer that recognizes whether or not the input sentence is a sentence related to “economy”, the information processing apparatus 10 sets the objective variable t = 1 for the sentence whose tag is “economy”. And the objective variable t = 0 is assigned to the other sentences (see FIG. C). By using the tag in this way, a learning data set that can be used to construct a desired language analyzer can be generated. When the learning data set can be generated, the learning data integration process and the estimator construction process described above are executed and used for a language analyzer (means for obtaining a recognition result from the estimated value y). An estimator (calculating means for the estimated value y) can be automatically constructed.

(About the effects of online learning)
Experiments were conducted using the above-mentioned automatic construction method for language analyzers. The experimental results are shown in FIG. The horizontal axis of the graph shown in FIG. 39 is the elapsed time (the unit is days), and the vertical axis is the average F value (Average F-Measures). In addition, a solid line (Online, 1k) and a broken line (Online, 4k) indicate experimental results when the learning data set is continuously updated by online learning. On the other hand, a chain line ( Offline , 1k) and an alternate long and short dash line ( Offline , 4k) indicate the experimental results by offline learning. Note that 1k indicates that the number of learning data used to construct the estimator is set to 1000. On the other hand, 4k indicates that the number of learning data used to construct the estimator is set to 4000.

  As is clear from FIG. 39, the more accurate the estimator is obtained as the number of learning data used for constructing the estimator increases. In addition, in the case of offline learning, the accuracy quickly reaches a limit, but in the case of performing online learning, it can be seen that the accuracy improves as time passes. And after a certain amount of time has passed, the results of online learning are much better than the results of offline learning. From this experimental result, it became clear that the accuracy of the estimator can be improved by updating the learning data set by online learning. In addition, although the experimental result was shown about the automatic construction method of a language analyzer here, the same effect is anticipated also about the automatic construction method of another recognizer.

(Summary of effects)
As described above, the accuracy of the estimator can be improved by enabling online learning. Further, as a construction method of the estimator, for example, Japanese Patent Application Laid-Open No. 2009-48266, Japanese Patent Application No. 2010-159598, Japanese Patent Application No. 2010-159597, Japanese Patent Application No. 2009-277083, Japanese Patent Application No. 2009- Various methods such as an algorithm described in the specification of No. 277084 can be applied. Therefore, it is possible to improve accuracy for various types of recognizers. Further, by providing a mechanism for automatically generating a learning data set using acquired information acquired from a Web service or the like, the accuracy of the estimator can be improved continuously without maintenance. In addition, by updating the learning data set sequentially, the estimator is always constructed from the new learning data set, so it is possible to flexibly cope with the appearance of new tags and changes in the meaning of tags as the times change. Is possible.

<4: Hardware configuration example>
The function of each component included in the information processing apparatus 10 can be realized by using, for example, a hardware configuration illustrated in FIG. That is, the function of each component is realized by controlling the hardware shown in FIG. 40 using a computer program. The form of the hardware is arbitrary, and includes, for example, a personal computer, a mobile phone, a portable information terminal such as a PHS, a PDA, a game machine, or various information appliances. However, the above PHS is an abbreviation of Personal Handy-phone System. The PDA is an abbreviation for Personal Digital Assistant.

  As shown in FIG. 40, this hardware mainly includes a CPU 902, a ROM 904, a RAM 906, a host bus 908, and a bridge 910. Further, this hardware includes an external bus 912, an interface 914, an input unit 916, an output unit 918, a storage unit 920, a drive 922, a connection port 924, and a communication unit 926. However, the CPU is an abbreviation for Central Processing Unit. The ROM is an abbreviation for Read Only Memory. The RAM is an abbreviation for Random Access Memory.

  The CPU 902 functions as, for example, an arithmetic processing unit or a control unit, and controls the overall operation of each component or a part thereof based on various programs recorded in the ROM 904, the RAM 906, the storage unit 920, or the removable recording medium 928. . The ROM 904 is a means for storing a program read by the CPU 902, data used for calculation, and the like. In the RAM 906, for example, a program read by the CPU 902, various parameters that change as appropriate when the program is executed, and the like are temporarily or permanently stored.

  These components are connected to each other via, for example, a host bus 908 capable of high-speed data transmission. On the other hand, the host bus 908 is connected to an external bus 912 having a relatively low data transmission speed via a bridge 910, for example. As the input unit 916, for example, a mouse, a keyboard, a touch panel, a button, a switch, a lever, or the like is used. Further, as the input unit 916, a remote controller (hereinafter referred to as a remote controller) capable of transmitting a control signal using infrared rays or other radio waves may be used.

  As the output unit 918, for example, a display device such as a CRT, LCD, PDP, or ELD, an audio output device such as a speaker or a headphone, a printer, a mobile phone, or a facsimile, etc. Or it is an apparatus which can notify audibly. However, the above CRT is an abbreviation for Cathode Ray Tube. The LCD is an abbreviation for Liquid Crystal Display. The PDP is an abbreviation for Plasma Display Panel. Furthermore, the ELD is an abbreviation for Electro-Luminescence Display.

  The storage unit 920 is a device for storing various data. As the storage unit 920, for example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, a magneto-optical storage device, or the like is used. However, the HDD is an abbreviation for Hard Disk Drive.

  The drive 922 is a device that reads information recorded on a removable recording medium 928 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable recording medium 928. The removable recording medium 928 is, for example, a DVD medium, a Blu-ray medium, an HD DVD medium, or various semiconductor storage media. Of course, the removable recording medium 928 may be, for example, an IC card on which a non-contact type IC chip is mounted, an electronic device, or the like. However, the above IC is an abbreviation for Integrated Circuit.

  The connection port 924 is a port for connecting an external connection device 930 such as a USB port, an IEEE 1394 port, a SCSI, an RS-232C port, or an optical audio terminal. The external connection device 930 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder. However, the above USB is an abbreviation for Universal Serial Bus. The SCSI is an abbreviation for Small Computer System Interface.

  The communication unit 926 is a communication device for connecting to the network 932. For example, a wired or wireless LAN, Bluetooth (registered trademark), or a WUSB communication card, an optical communication router, an ADSL router, or various types It is a modem for communication. The network 932 connected to the communication unit 926 is configured by a wired or wireless network, such as the Internet, home LAN, infrared communication, visible light communication, broadcast, or satellite communication. However, the above LAN is an abbreviation for Local Area Network. The WUSB is an abbreviation for Wireless USB. The above ADSL is an abbreviation for Asymmetric Digital Subscriber Line.

  The hardware configuration example has been described above.

<5: Summary>
Finally, the technical idea of this embodiment will be briefly summarized. The technical ideas described below can be applied to various information processing apparatuses such as PCs, mobile phones, game machines, information terminals, information appliances, car navigation systems, and the like.

  The functional configuration of the information processing apparatus can be expressed as follows. For example, the information processing apparatus described in the following (1) adjusts the distribution of feature amount coordinates so that the distribution of feature amount coordinates in the feature amount space approaches a predetermined distribution. Specifically, as described in (2) below, the learning data is thinned out so that the distribution of feature coordinates approaches a predetermined distribution, or for each learning as described in (3) below. A process of weighting data is performed. Of course, as shown in the following (4), it is possible to combine the thinning process and the weighting process. By making the feature amount coordinate distribution in the feature amount space close to a predetermined distribution (for example, uniform distribution or Gaussian distribution) by such a method, the performance of the estimator can be further improved.

(1)
When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element A feature amount vector calculating unit for calculating a feature amount vector to be
A distribution adjusting unit that adjusts the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
For a plurality of learning data, a function generation unit that generates an estimation function that outputs an estimated value of the objective variable in accordance with an input of the feature vector;
Comprising
Information processing device.

(2)
The distribution adjustment unit thins out the learning data so that a distribution of points specified by the feature vector in a feature space approaches a predetermined distribution;
The information processing apparatus according to (1) above.

(3)
The distribution adjusting unit weights the learning data so that a distribution of points specified by the feature vector in the feature space approaches a predetermined distribution;
The information processing apparatus according to (1) above.

(4)
The distribution adjustment unit thins the learning data so that the distribution of points specified by the feature vector in the feature amount space approaches a predetermined distribution, and for each of the learning data remaining after the thinning And weight,
The information processing apparatus according to (1) above.

(5)
The predetermined distribution is a uniform distribution or a Gaussian distribution.
The information processing apparatus according to any one of (1) to (4) above.

(6)
When the distribution adjustment unit is additionally provided with new learning data, the distribution adjustment unit targets the learning data group including the new learning data and the existing learning data in the feature amount space. The learning data is thinned out so that the distribution of points specified by the quantity vector approaches a predetermined distribution.
The information processing apparatus according to (2) or (4) above.

(7)
A basis function generation unit that generates the basis function by combining a plurality of functions prepared in advance;
The information processing apparatus according to any one of (1) to (6) above.

(8)
The basis function generation unit updates the basis function based on a genetic algorithm,
When the basis function is updated, the feature amount vector calculation unit calculates the feature amount vector by inputting the input data to the updated basis function,
The function generation unit generates an estimation function that outputs an estimated value of the objective variable in accordance with an input of a feature vector calculated using the updated basis function;
The information processing apparatus according to (7) above.

(9)
When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element Calculating a feature vector to be
Adjusting the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
Generating an estimation function that outputs an estimated value of the objective variable with respect to an input of the feature quantity vector for a plurality of the learning data;
including,
Estimator generation method.

(10)
When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element A feature vector calculation function for calculating a feature vector to be
A distribution adjusting function for adjusting the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
A function generating function for generating an estimation function that outputs an estimated value of the objective variable in response to an input of the feature vector for a plurality of the learning data;
A program to make a computer realize.

(Remarks)
The feature amount calculation unit 122 is an example of a feature amount vector calculation unit. The learning data integration unit 124 is an example of a distribution adjustment unit. The estimation function generation unit 123 is an example of a function generation unit. The basis function list generation unit 121 is an example of a basis function generation unit.

  The preferred embodiments according to the present technology have been described above with reference to the accompanying drawings, but it is needless to say that the present technology is not limited to the configuration examples disclosed herein. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present technology. Understood.

DESCRIPTION OF SYMBOLS 10 Information processing apparatus 11 Learning data acquisition part 12 Estimator construction part 121 Basis function list generation part 122 Feature quantity calculation part 123 Estimation function generation part 124 Learning data integration part 13 Input data acquisition part 14 Result recognition part

Claims (8)

When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element A feature amount vector calculating unit for calculating a feature amount vector to be
A distribution adjusting unit that adjusts the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
For a plurality of learning data, a function generation unit that generates an estimation function that outputs an estimated value of the objective variable in accordance with an input of the feature vector;
Equipped with a,
When the distribution adjustment unit is additionally provided with new learning data, the distribution adjustment unit targets the learning data group including the new learning data and the existing learning data in the feature amount space. The learning data is thinned out so that the distribution of points specified by the quantity vector approaches a predetermined distribution.
Information processing device. The distribution adjusting unit weights the learning data so that a distribution of points specified by the feature vector in the feature space approaches a predetermined distribution;
The information processing apparatus according to claim 1. The distribution adjusting unit, the distribution of points specified by the feature vector in a feature quantity space so as to approach the predetermined distribution, weighted for each of the learning data remaining after during pulling,
The information processing apparatus according to claim 1. The predetermined distribution is a uniform distribution or a Gaussian distribution.
The information processing apparatus according to any one of claims 1 to 3 . A basis function generation unit that generates the basis function by combining a plurality of functions prepared in advance;
The information processing apparatus according to any one of claims 1 to 4 . The basis function generation unit updates the basis function based on a genetic algorithm,
When the basis function is updated, the feature amount vector calculation unit calculates the feature amount vector by inputting the input data to the updated basis function,
The function generation unit generates an estimation function that outputs an estimated value of the objective variable in accordance with an input of a feature vector calculated using the updated basis function;
The information processing apparatus according to claim 5 . When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element Calculating a feature vector to be
Adjusting the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
Generating an estimation function that outputs an estimated value of the objective variable with respect to an input of the feature quantity vector for a plurality of the learning data;
When new learning data is additionally provided, the learning data group including the new learning data and the existing learning data is specified by the feature vector in the feature space. Thinning out the learning data so that the distribution of points approaches a predetermined distribution;
including,
Estimator generation method. When a plurality of learning data composed of input data and an objective variable corresponding to the input data is given, the input data is input to a plurality of basis functions, and an output value of each basis function is defined as an element A feature vector calculation function for calculating a feature vector to be
A distribution adjusting function for adjusting the distribution of the points so that the distribution of the points specified by the feature vector in the feature amount space approaches a predetermined distribution;
A function generating function for generating an estimation function that outputs an estimated value of the objective variable in response to an input of the feature vector for a plurality of the learning data;
When new learning data is additionally given, the learning data group including the new learning data and the existing learning data is specified by the feature vector in the feature space. A function of thinning out the learning data so that the distribution of points approaches a predetermined distribution;
A program to make a computer realize.