JP2001272994A - Device and method for study, device and method for recognizing pattern, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a device for study enabling a prescribed model (for example, a recognition model) to stabilize itself at an early stage and reach a local optimal state independently of a model to be handled and also by reducing a calculation amount. SOLUTION: This device comprises a normalizing means 11 for normalizing a discrimination function to each class by converting it into at least a primarily differentiable and stochastically limited function, a loss calculating means 12 for calculating a loss to a correct answer by using a discrimination function value normalized by the normalizing means 11, an optimal parameter adjustment amount calculating means 13 for calculating an optimal adjustment amount of a prescribed model 100 to minimize the loss calculated by the loss calculating means 12, and an adjusting means 14 for adjusting the parameter of the prescribed model 100 by the optimal parameter adjusting amount calculated by the optimal parameter adjustment amount calculating means 13.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a learning device, a learning method, a pattern recognition device, a pattern recognition method, and a recording medium.

[0002]

2. Description of the Related Art Conventionally, for example, the document "The Journal of the
Acoustical Society of Japan (E) vol.13 no.6, pp.34
1-349, Nov. 1992 ", there is disclosed a technique for classifying an input pattern into any one of classes 1 to U having a class number U. That is, according to the technique disclosed in this document, first, a discriminant function for a class u when an input pattern x and a parameter の of a recognition model are given is g _u (x, Λ), (u = 1,
..., U). Here, the input pattern x is a class u
If it belongs to is assumed to be designed such that the value of the discriminant function g _u increases. In this case, the misclassification measure d _p (x,
Λ) is represented by the following equation (Equation 1).

[0003]

(Equation 1)

Here, ζ represents an index for controlling the comparison operation (that is, an adjustment amount). From Equation 1, the misclassification measure d _p (x, ｘ) indicates a correct answer when it is negative (−), and indicates an incorrect answer when it is positive (+).

Next, individual losses E _u (x, Λ) for the u-th class when the m-th pattern is input are defined in a smooth form as in the following equation (Equation 2).

[0006]

(Equation 2)

The loss E _u (x, Λ) in Equation 2 is a sigmoid function, and takes a value very close to “0” when the answer is correct and “1” when the answer is completely wrong. Using the loss expression E _u (x, Λ) of Equation 2, the empirical loss L (Λ) can be expressed as the following Equation (Equation 3).

[0008]

(Equation 3)

Here, x _m is the m-th pattern input from the M patterns. In order to minimize the experience loss L (Λ), the correction amount ΔΛ of the model parameter Λ is calculated as in the following equation (Equation 4).

[0010]

(Equation 4)

Here, η is a small positive learning coefficient.
By executing the iterative calculation of the following equation (Equation 5) based on the stochastic descent theorem, the local minimum state of Equation 3 is guaranteed.

[0012]

５ (t + 1) = Λ (t) + ΔΛ (t)

In equation (5),) (t) is a parameter after the repetitive calculation is applied t times.

[0014]

By the way, according to the technical contents disclosed in the above-mentioned literature, since the equation (1) is calculated for each of the classes 1 to U, the calculation amount is large and the total calculation load is large. There are inconveniences.

On the other hand, the above document also discloses that ζ, which controls the comparison operation, is set to infinity, and a simplified misclassification degree is used as in the following equation (Equation 6).

[0016]

## EQU6 ## d _p (x, Λ) = − g _p (x, Λ) + g _q (x, Λ)

Here, q is a class having the largest discriminant function value except p. That is, Equation 6 is a pattern x
The comparison is performed only between the class p to which belongs and the closest class q. Using Equation 6,
Although the amount of calculation can be reduced, the operation for minimizing the loss is not reflected on the model parameters other than p and q, so that it takes a long time to reach an optimum state. Further, the value of the discriminant function varies in a possible range depending on a model to be handled (for example, a hidden Markov model or a neural network model). Therefore, in Equation 2, the shape of the loss function is controlled using the variables h and b. However, since these values are determined empirically, if they are not set appropriately, there is a problem that the time to reach the optimum state becomes longer.

The present invention is independent of the model to be treated and
Predetermined model stably at an early stage by reducing the amount of calculation
It is an object of the present invention to provide a learning device, a learning method, a pattern recognition device, a pattern recognition method, and a recording medium capable of reaching a local optimum state (for example, a recognition model).

[0019]

In order to achieve the above object, according to the first aspect of the present invention, a class to which an input pattern belongs is determined by using a discriminant function, and a predetermined class is determined in order to minimize a loss from a correct answer. A learning device that adjusts the parameters of the model of (1), normalizing means for converting a discriminant function for each class into a function having at least first-order differentiable and stochastic constraints and normalizing the discriminant function, and discriminant normalized by the normalizing means Loss calculation means for calculating a loss to a correct answer using a function value, and an optimum parameter adjustment amount calculation means for calculating an optimum parameter adjustment amount of a predetermined model by minimizing the loss calculated by the loss calculation means; Adjusting means for adjusting the parameters of a predetermined model based on the optimum parameter adjustment amount calculated by the optimum parameter adjustment amount calculating means. It is characterized in.

According to a second aspect of the present invention, in the learning apparatus according to the first aspect, the loss calculating means distributes the discriminant function value normalized by the normalizing means to a class to which the input pattern belongs. Using a scale that evaluates as
It is characterized in that a loss for the correct answer class is obtained.

According to a third aspect of the present invention, in the learning apparatus according to the first aspect, the loss calculating means includes a discriminant function value for the correct answer and a discriminant for the correct answer among the discriminant function values normalized by the normalizing means. It is characterized in that a loss for a correct answer class is obtained using at least one discriminant function value having a plausible value other than the function value using a scale evaluated as a distribution of the possibility that the input pattern belongs.

According to a fourth aspect of the present invention, in the learning apparatus according to the second or third aspect, the distribution of the correct answer used for the scale is a distribution having a possibility only in the class to which the correct answer belongs. It is characterized by:

According to a fifth aspect of the present invention, in the learning apparatus according to any one of the first to fourth aspects,
When the input pattern is a variable-length input pattern, a variable-length input is performed by using a state transition model represented by a parameter for evaluating the characteristic amount of the pattern and a parameter for evaluating the continuation length of the partial parameter as a predetermined model. It is characterized by scoring patterns.

According to a sixth aspect of the present invention, in the learning apparatus according to the fifth aspect, the state transition model can adjust a ratio between a score based on a parameter for evaluating a feature amount and a score based on a parameter for evaluating a continuation length. It is characterized by being.

According to a seventh aspect of the present invention, there is provided a learning method for determining a class to which an input pattern belongs by using a discriminant function and adjusting parameters of a predetermined model in order to minimize a loss from a correct answer. After converting the discriminant function for the class to a function having at least first-order differentiable and stochastic constraints and normalizing it, find the loss for the correct answer and minimize the loss for the correct answer to obtain the optimal parameters for the given model. It is characterized by seeking.

The invention according to claim 8 is characterized in that, in the learning method according to claim 7, parameters of a predetermined model are obtained by a stochastic descent method in order to minimize a loss from a correct answer. .

According to a ninth aspect of the present invention, there is provided the learning method according to the seventh aspect, wherein a standardized discriminant function value is determined by using a scale for evaluating the normalized discriminant function value as a probability distribution to a class to which the input pattern belongs. It is characterized by finding the loss for

The invention according to claim 10 is the same as the invention according to claim 7.
In the learning method described, the input pattern may belong to the discriminant function value for the correct answer and at least one discriminant function value having a plausible value other than the discriminant function value for the correct answer among the normalized discriminant function values. It is characterized in that a loss for a correct answer class is obtained using a scale evaluated as a gender distribution.

Further, the invention described in claim 11 is the same as the ninth invention.
Alternatively, in the learning method according to the tenth aspect, the distribution of the correct answer used as the scale is a distribution having a possibility only in the class to which the correct answer belongs.

The invention according to claim 12 is the same as claim 7.
In the learning method according to any one of claims 11 to 11, when the input pattern is a variable length input pattern,
As a predetermined model, a variable-length input pattern is scored by using a state transition model represented by a parameter for evaluating a pattern characteristic amount and a parameter for evaluating a continuation length of a partial parameter.

The invention according to claim 13 is the first invention.
2. In the learning method described in 2, the state transition model is characterized in that the ratio between the score based on the parameter for evaluating the characteristic amount and the score based on the parameter for evaluating the continuation length can be adjusted.

The invention according to claim 14 is the first invention.
In the learning method according to the second or thirteenth aspect, an average vector of a parameter for evaluating a feature amount is adjusted using the learning method according to any one of the seventh to eleventh aspects.

Further, the invention according to claim 15 provides the invention according to claim 1.
In the learning method according to the second or the thirteenth aspect, the variance of the parameter for evaluating the feature amount is determined.
The adjustment is performed using the learning method described in any one of the above.

The invention according to claim 16 is the first invention.
In the learning method according to the second or thirteenth aspect, the average vector of the parameter for evaluating the continuation length is adjusted using the learning method according to any one of the seventh to eleventh aspects.

The invention according to claim 17 is the first invention.
In the learning method according to the second or the thirteenth aspect, the variance of the parameter for evaluating the continuation length is determined.
The adjustment is performed using the learning method described in any one of the above.

The invention according to claim 18 is the first invention.
In the learning method according to the third aspect, a ratio between a score based on the parameter for evaluating the feature amount and a score based on the parameter for evaluating the continuation length is adjusted using the learning method according to any one of claims 7 to 11. It is characterized by:

According to a nineteenth aspect of the present invention, there is provided a pattern recognition apparatus which obtains a class to which an input pattern belongs by using a discriminant function and adjusts parameters of a recognition model in order to minimize a loss from a correct answer. A normalizing means for converting a discriminant function for a class into a function having at least first differentiable and having a stochastic constraint to normalize the discriminant function, and a loss for calculating a loss for a correct answer using the discriminant function value normalized by the normalizing means Calculating means, an optimum parameter adjustment amount calculating means for calculating an optimum parameter adjustment amount of the recognition model by minimizing the loss calculated by the loss calculating means, and an optimum parameter adjustment amount calculated by the optimum parameter adjustment amount calculating means. Adjusting means for adjusting the parameters of the recognition model by the parameter adjustment amount, and adjusting the parameters of the recognition model by the adjusting means. When it is adjusted to the optimum parameters, it is characterized by being adapted to perform pattern recognition for the input pattern by using the recognition models parameter adjustment.

According to a twentieth aspect of the present invention, in the pattern recognition method for determining a class to which an input pattern belongs by using a discriminant function and adjusting parameters of a recognition model in order to minimize a loss from a correct answer, After converting the discriminant function for the class to a function with at least first-order differentiable and stochastic constraints and normalizing it, find the loss for the correct answer and minimize the loss for the correct answer to obtain the optimal parameters of the recognition model. When the parameters of the recognition model are adjusted to the optimum parameters, the recognition of the input pattern is performed using the parameter-adjusted recognition model.

According to a twenty-first aspect of the present invention, the discriminant function for each class is converted to a function having at least first-order differentiable and stochastic constraints and normalized, and the loss for the correct answer is obtained. Is a computer-readable recording medium on which a program for causing a computer to execute a process of obtaining an optimal parameter of a predetermined model by minimizing the parameter is recorded.

[0040]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a configuration example of a learning device according to the present invention. Referring to FIG. 1, the learning apparatus obtains a class to which an input pattern belongs using a discriminant function, and uses a predetermined model 1 in order to minimize a loss from a correct answer.
00 (for example, a recognition model described above or below) that adjusts (learns) the parameters, and converts a discriminant function for each class into a function having at least first-order differentiable and stochastic constraints to normalize it. Converting means 11, a loss calculating means 12 for calculating a loss for a correct answer using the discriminant function value normalized by the normalizing means 11, and a predetermined value for minimizing the loss calculated by the loss calculating means 12. An optimal parameter adjustment amount calculating unit 13 for calculating an optimal parameter adjustment amount of the model 100; an adjusting unit 14 for adjusting a parameter of a predetermined model 100 based on the optimal parameter adjustment amount calculated by the optimal parameter adjustment amount calculating unit 13; have.

Here, the loss calculating means 12 obtains a loss for the correct answer class by using a scale for evaluating the discriminant function value normalized by the normalizing means 11 as a distribution of possibility to the class to which the input pattern belongs. It has become.

Next, a learning method in the learning device having such a configuration will be described. In this learning device, the input pattern x is classified into one of classes 1 to U of the number of classes U. That is, first, the discriminant function for the class u when the input pattern x and the parameter の of the predetermined model 100 are given is g _u (x, Λ), (u = 1,..., U). Here, when the input pattern x belongs to a class u is assumed to be designed such that the value of the discriminant function g _u increases.

FIG. 2 is a diagram for explaining such a predetermined model 100. Referring to FIG. 2, class 1,
The discriminant function for class 2,..., Class U is g ₁ (x,
Λ), g ₂ (x, Λ),..., G _U (x, Λ), the discriminant function values of the predetermined model 100 when the pattern x is input are g ₁ , g _{2 ,.} Becomes Here, for example, when the input pattern x belongs to class 1, as the discriminant function value g ₁ is largest, i.e., g _1> g _2,
.., G _{U are} assumed to be a predetermined model 100.

The predetermined model 100 (discriminating function g)
_{1 (x, Λ) ~g U} (x, Λ) for learning the parameters lambda in), the learning apparatus of FIG. 1, first, the normalization unit 11, with respect to i-th discriminant function values g _i The normalized function value a _i is derived by the following equation (Equation 7).

[0045]

(Equation 7)

The function value a _i expressed by the equation (7) is expressed by a function (exponential function) that can be linearly differentiated.
It has a probabilistic constraint as expressed by (Equation 8).

[0047]

(Equation 8)

When the normalized function value a _i is derived in the normalizing means 11 as described above, the loss calculating means 12
Then, the loss for the correct answer is determined. The function values a _i (i = 1 to U) of the equations (7) and (8) are converted into the input pattern x by the probability expression.
Can be interpreted as a distribution of the likelihood of the class to which it belongs. Therefore,
The loss calculating means 12 can calculate the loss E (x, Λ) as in the following equation (Equation 9), with the measure for evaluating this distribution being the loss.

[0049]

(Equation 9)

Here, t _u is a function value representing the possibility that the correct answer belongs to the u-th, and has a probability expression similar to Equation 8 as in _ai .

The loss calculation means 12 calculates the loss E
When (x, Λ) is calculated, the optimal parameter adjustment amount calculating means 13 sets the optimum parameter adjustment amount ΔΛ of the predetermined model 100 so as to minimize the calculated loss E (x, Λ).
Is calculated. Then, the adjusting unit 14 uses the optimum parameter adjustment amount ΔΛ calculated by the optimum parameter adjustment amount calculating unit 13 to obtain a predetermined model 100
Adjust the parameters of.

As described above, in the learning apparatus of FIG. 1, by normalizing the discriminant function value of each class using a function having a probabilistic constraint, the model parameters can be stably set regardless of the model to be handled. After making adjustments, the model 100
(For example, a recognition model) can be stably reached to a local optimum state.

Further, in the learning apparatus of FIG. 1, the loss for the correct answer class is obtained by using a scale for evaluating the possibility of the class to which the input pattern belongs as a distribution.
A predetermined model (for example, a recognition model) can reach a local optimum state at an early stage. That is, by calculating the discriminant function value of each class as one distribution, the correction amount of the model parameter corresponding to each class can be determined at the same time, and a predetermined model (for example, a recognition model) Can reach a local optimal state.

Although the learning apparatus of FIG. 1 takes into account the total number of classes U in Equations 7 to 9, in order to reduce the calculation load, the normalized discriminant function values for the correct answer are reduced. Finding the loss for the correct class using a measure that evaluates the distribution of the possibility that the input pattern belongs to the discriminant function value and at least one discriminant function value having a plausible value other than the discriminant function value for the correct answer. Can also.

FIG. 3 is a diagram showing a modification of the learning apparatus of FIG. 1. In the learning apparatus of FIG. 3, the learning apparatus of FIG.
In order to reduce the calculation load, a class selection unit 15 is further provided before the normalization unit 11. The class selecting means 15 selects a predetermined number U 'of classes from classes 1 to U of the total number U of classes. Specifically, in order to obtain the class included in U ′ in this case, the magnitudes of the discriminant function values g _{1 to} g _U are determined, and the sum of the correct class and the class selected in the order of the highest score other than the correct class is calculated as What is necessary is just to perform a class selection operation so that it may become U '.

As described above, by determining the loss only for the correct answer and some classes close to the correct answer, it is possible to reduce the amount of calculation required for adjusting the parameters of the predetermined model 100 (for example, the recognition model). That is, since the number of classes required for calculating the loss can be controlled, the amount of calculation required for parameter adjustment can be reduced.

In the learning apparatus and the learning method of the present invention described above, as will be described later, the distribution of the correct answer used for the scale may be a distribution having a possibility only in the class to which the correct answer belongs.

In the learning apparatus and the learning method of the present invention described above, when the input pattern x is a variable-length pattern, a predetermined model (for example,
Local optimization of the recognition model can be performed at high speed. Specifically, as described later, when the input pattern is a variable-length input pattern, a state represented by a parameter for evaluating the feature of the pattern and a parameter for evaluating the continuation length of the partial parameter as a predetermined model A variable-length input pattern can be scored using a transition model. Here, the state transition model can adjust the ratio between the score based on the parameter for evaluating the feature amount and the score based on the parameter for evaluating the duration.

Further, the learning apparatus and the learning method of the present invention described above are applicable not only to the design of a predetermined model (for example, a recognition model) for a static pattern in which the input pattern x is not only a variable-length pattern but also an input pattern x. Can be applied.

The above-described learning apparatus and learning method of the present invention can be applied to pattern recognition such as voice recognition and character recognition. FIG. 4 and FIG. 5 are diagrams showing a configuration example of the pattern recognition device according to the present invention. Note that the pattern recognition devices of FIGS. 4 and 5 apply the learning device of FIGS. 1 and 3, respectively. Therefore, in FIG. 4 and FIG. 5, the same reference numerals are given to portions corresponding to FIG. 1 and FIG.

In the pattern recognition apparatus shown in FIGS. 4 and 5, the predetermined model 100 in the learning apparatus shown in FIGS. 1 and 3 is a recognition model (for example, as described later, a state length control type used for voice recognition). Transition (DST) model)
Is used.

That is, the pattern recognition apparatus shown in FIGS. 4 and 5 obtains the class to which the input pattern x belongs by using the discriminant function, and minimizes the loss from the correct answer.
A learning device that adjusts (learns) the parameters of 00, and converts a discriminant function for each class into a function having at least first-order differentiable and stochastic constraints and normalizes the discriminant function; A loss calculating unit 12 that calculates a loss for a correct answer using the discriminant function value normalized by the normalizing unit 11 and an optimal parameter of the recognition model 100 by minimizing the loss calculated by the loss calculating unit 12. An optimal parameter adjustment amount calculation unit for calculating an adjustment amount; and an adjustment unit for adjusting the parameters of the recognition model 100 based on the optimum parameter adjustment amount calculated by the optimum parameter adjustment amount calculation unit.
When the parameters of the recognition model 100 are adjusted to the optimum parameters by the adjustment means 14, the recognition of the unknown input pattern x is performed using the parameter-adjusted recognition model 100.

In the pattern recognition apparatus shown in FIGS. 4 and 5, the feature extraction means 17 generates a predetermined feature pattern as the input pattern x to the recognition model 100 (for example, extracts a voice feature pattern from an input voice). Is provided.

The pattern recognition apparatuses shown in FIGS. 4 and 5 can handle a variable-length pattern such as a voice pattern as a model (recognition model) that can be expressed by the parameter Λ. Specifically, for example, a description will be given below using a duration length control type state transition (DST) model described in Japanese Patent No. 2804265 as a basic model.

The number of DST models is the same as the number U of classes, and the score when the voice pattern x is measured by the u-th model u is defined as the value g _u (x, 値) of the discriminant function. It is expressed by Equation 10).

[0066]

(Equation 10)

In the equation (10), x is the characteristic extracting means 17
Is a voice feature pattern generated by the above. A well-known LPC (linear prediction) analysis or the like can be used for the feature extraction in the feature extraction unit 17. For example, the feature extraction conditions are as follows: sampling frequency: 8 kHz, high-frequency emphasis: first-order difference, 256-point Hamming window, moving width: 16 ms, LPC analysis order: 20, 10-dimensional mel-cepstral coefficient + first-order difference of logarithmic power + logarithmic power Is extracted for each frame. Note that the feature extraction conditions are not limited to those described above, and any extraction method such as frequency analysis can be used.

In Equation 10, r (·) represents the correspondence between the voice pattern obtained by the matching and each state of the model, and r (n) represents the end of the partial pattern corresponding to the n-th state. Frame number.

[0069] Further, in the equation 10, S _n is a parameter for evaluating the characteristic quantity (scores of the n state related features), is defined as follows (number 11).

[0070]

[Equation 11]

In Equation 11, T _n is a bias value, and D represents a local distance in each state. The weighted square Euclidean distance of the following equation (Equation 12) is used for D.

[0072]

(Equation 12)

In Equation 12, μ _n = (μ _nk ), σ ² _n =
(Σ ² _nk ) and (k = 1,..., K) are the mean and variance of the n-th state, respectively. Note that k represents an element number of a K-dimensional vector. The pattern x is composed of M frames, and x _m = (x _mk ), (m = 1,...,
M) represents the audio feature amount of the frame number m.

In Equation 10, R _n is a parameter for evaluating the duration (distance relating to the duration of the n-th state)
And is represented by the following equation (Equation 13).

[0075]

(Equation 13)

In Expression 13, J is a weighted squared Euclidean distance, and is expressed by the following expression (Expression 14).

[0077]

[Equation 14]

Here, τ _n and ζ _n ² are the average and the variance of the duration of each state, respectively. Further, l _n is the length of time of the partial pattern associated with each state.

Further, in Expression 10, z _n (0 ≦ z _n ≦
1) is a weight to adjust proportion of scores obtained from the S _n and R _n, as the value of z _n is large, the influence of the score on evaluation duration decreases.

The discriminant function value of Equation 10 is disclosed in Japanese Patent No. 280402.
As shown in No. 65, it can be obtained by performing a state search while incorporating a score regarding the duration evaluation into the dynamic programming.

That is, the loss E (x, Λ) defined by equation 9
To minimize the recognition model g_u(X, Λ)
Adjust parameters with optimal parameter adjustment amount calculating means 13
Adjustment (learning) by the stochastic descent method using the means 14
You. In the following, the recognition model g _uEach parameter of (x, Λ)
The description will be made by adding u representing the class to the data.

First, an average vector related to a feature amount is targeted from among the model parameters. In this case, the amount of parameter correction can be calculated from the loss function E (x, Λ) of Equation 9 as in the following Equation (Equation 15).

[0083]

(Equation 15)

In Expression 15, x _{c (n) k} indicates the _k- th element of the audio feature amount associated with the n-th state.

Similarly, the variance correction amount related to the feature amount can be calculated as in the following equation (Equation 16).

[0086]

(Equation 16)

Next, the correction amount is calculated for the parameter relating to the continuation length. Mean vector for duration,
When the variance vector is obtained, it can be calculated as in Equations 17 and 18, respectively.

[0088]

[Equation 17]

[0089]

(Equation 18)

Subsequently, the parameter for determining the evaluation ratio of the feature amount and the duration can also be calculated as in the following equation (Equation 19) by obtaining the correction amount.

[0091]

[Equation 19]

If equations 15 to 19 can be calculated, each model parameter can be adjusted using equation 5. At this time, how to give the distribution of the correct answer becomes a problem, but the following equation (Equation 20) exists only in the class p to which the correct answer belongs.
It can be used for distribution t _u.

[0093]

(Equation 20)

From Equations 15 to 20, the parameters of the model (the average vector of the parameter for evaluating the feature, the variance of the parameter for evaluating the feature, the average vector of the parameter for evaluating the duration, and the parameter for evaluating the duration) (The ratio between the score of the parameter for evaluating the feature amount and the score of the parameter for evaluating the continuation length) using the learning method of the present invention described above (that is, the adjusting means 1).
4) can be adjusted as follows.

That is, the average vector of the parameter for evaluating the characteristic amount can be adjusted by the following equation (Equation 21).

[0096]

(Equation 21)

The variance of the parameter for evaluating the feature can be adjusted by the following equation (Equation 22).

[0098]

(Equation 22)

The average vector of the parameter for evaluating the continuation length can be adjusted by the following equation (Equation 23).

[0100]

(Equation 23)

The variance of the parameter for evaluating the continuation length can be adjusted by the following equation (Equation 24).

[0102]

(Equation 24)

The ratio between the score obtained by the parameter for evaluating the characteristic amount and the score obtained by the parameter for evaluating the continuation length can be adjusted by the following equation (Equation 25).

[0104]

(Equation 25)

As described above, in the pattern recognition apparatus and the pattern recognition method of the present invention, when the input pattern x is a variable length pattern, local optimization of a recognition model for the variable length pattern is performed at high speed and with high accuracy. Pattern recognition for the input pattern x can be performed. Further, the present invention can be applied to design of a recognition model for not only a variable-length pattern but also a static pattern.

FIG. 6 is a diagram showing an example of a hardware configuration of the learning device or the pattern recognition device shown in FIG. 1, FIG. 3, FIG. 4 or FIG. Referring to FIG. 6, FIGS.
Alternatively, the learning device or the pattern recognition device of FIG.
For example, a CPU 21 that is realized by a workstation, a personal computer, or the like and controls the whole,
ROM 22 storing the control program of
RAM 23 used as a work area of PU 21
And a hard disk 24 for storing data, and a pattern input unit (or a voice input unit) 25.

Here, the predetermined model 100 is, for example, R
It is set in the AM 23 or the like, and is read and used by the CPU 21. Further, the CPU 21
It has the functions of the normalizing means 11, the loss calculating means 12, the optimal parameter adjustment amount calculating means 13, the adjusting means 14, the class selecting means 15, and the feature extracting means 17 shown in FIG. 1, FIG. 3, FIG. .

The functions of the CPU 21 such as the normalizing means 11, the loss calculating means 12, the optimum parameter adjustment amount calculating means 13, the adjusting means 14, the class selecting means 15, the feature extracting means 17 and the like are, for example, software packages. (Specifically, an information recording medium such as a CD-ROM) can be provided. For this reason, in the example of FIG. 6, when the information recording medium 30 is set, the medium driving device 31 drives the information recording medium 30. Is provided.

In other words, the learning apparatus and the pattern recognition apparatus of the present invention allow a general-purpose computer system having an image scanner, a display, and the like to read a program recorded on an information recording medium such as a CD-ROM. The present invention can also be implemented in an apparatus configuration in which a microprocessor of a general-purpose computer system executes a learning process and a pattern recognition process. In this case, a program for executing the learning process and the pattern recognition process of the present invention (that is, a program used in the hardware system) is provided in a state recorded on a medium. The information recording medium on which the program or the like is recorded is not limited to a CD-ROM, but may be a ROM, a RAM, a flexible disk, a memory card, or the like. The program recorded on the medium is installed in a storage device incorporated in the hardware system, for example, a hard disk device, so that the program can be executed to realize the learning function and the pattern recognition function of the present invention. it can.

The program for realizing the learning function and the pattern recognition function of the present invention may be provided not only in the form of a medium but also downloaded from a network.

[0111]

As described above, according to the present invention, the class to which the input pattern belongs is obtained by using the discriminant function, and the loss from the correct answer is minimized. In order to adjust the parameters of the predetermined model, the discriminant function for each class is converted to a function having at least first-order differentiable and stochastic constraints and normalized, and the loss to the correct answer is obtained. , The optimum parameters of the predetermined model are obtained, so that the model parameters can be adjusted stably regardless of the model to be handled, and the local optimum state of the predetermined model can be reached. .

In particular, according to the second, fourth, ninth, and eleventh aspects of the present invention, a scale for evaluating a normalized discriminant function value as a distribution of possibility to a class to which an input pattern belongs. Is used to determine the loss for the correct answer class, so that a local optimal state of a predetermined model (for example, a recognition model) can be reached at an early stage. That is, by calculating the discriminant function value of each class as one distribution, the correction amount of the model parameter corresponding to each class can be determined at the same time, and a predetermined model (for example, a recognition model) Can reach the local optimal state of

Further, according to the third and tenth aspects of the present invention, the loss is obtained by limiting the loss to the correct class and some classes close to the correct one to obtain a predetermined model (for example, a recognition model). The amount of calculation required for parameter adjustment can be reduced. That is, since the number of classes required for calculating the loss can be controlled, the amount of calculation required for parameter adjustment can be reduced.

Further, claim 5, claim 6, claim 12,
According to the thirteenth aspect, when the input pattern is a variable-length input pattern, the predetermined model is represented by a parameter for evaluating the characteristic amount of the pattern and a parameter for evaluating the continuation length of the partial parameter. Using a state transition model to score variable-length input patterns,
When the input pattern x is a variable-length pattern, local optimization of a recognition model for the variable-length pattern can be performed at high speed, and pattern recognition for the input pattern x can be performed with high accuracy. Further, the present invention can be applied to design of a recognition model for not only a variable-length pattern but also a static pattern.

Further, according to the present invention, the parameters can be adjusted efficiently by adjusting the adjustable model patterns using the same loss.

According to the nineteenth and twentieth aspects of the present invention, the discriminant function for each class is converted to a function having at least first-order differentiable and stochastic constraints, and is normalized. Is determined, and the optimal parameter of the recognition model is determined by minimizing the loss to the correct answer. When the parameters of the recognition model are adjusted to the optimal parameters, the pattern for the input pattern is determined using the parameter-adjusted recognition model. Since recognition is performed, pattern recognition accuracy can be improved.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration example of a learning device according to the present invention.

FIG. 2 is a diagram for explaining a predetermined model.

FIG. 3 is a diagram showing a modification of the learning device of FIG. 1;

FIG. 4 is a diagram showing a configuration example of a pattern recognition device according to the present invention.

FIG. 5 is a diagram showing a modification of the pattern recognition device of FIG. 4;

FIG. 6 is a diagram illustrating an example of a hardware configuration of the learning device of FIG. 1 or 3 or the pattern recognition device of FIG. 4 or 5;

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 11 Normalization means 12 Loss calculation means 13 Optimum parameter adjustment amount calculation means 14 Adjustment means 15 Class selection means 17 Feature extraction means 100 Predetermined model (recognition model) 21 CPU 22 ROM 23 RAM 24 Hard disk 25 Pattern input unit or voice input unit Reference Signs List 30 recording medium 31 medium driving device

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 15/14 G10L 3/00 535Z 15/16 539 F term (Reference) 5B049 DD00 DD03 DD05 EE03 EE08 EE31 FF03 FF04 FF09 5B056 BB01 BB71 BB91 5D015 HH23 JJ00 5L096 BA16 BA17 CA22 EA17 FA34 GA09 GA30 KA04 9A001 GG05 HH21 JJ74 KK09

Claims (21)

[Claims] 1. A learning apparatus for determining a class to which an input pattern belongs by using a discriminant function and adjusting a parameter of a predetermined model in order to minimize a loss from a correct answer. Normalization means for converting to a function having a probable and stochastic constraint and normalizing the loss function, and a loss calculation means for calculating a loss for a correct answer using the discriminant function value normalized by the normalization means,
An optimal parameter adjustment amount calculating unit that calculates an optimal parameter adjustment amount of a predetermined model by minimizing the loss calculated by the loss calculating unit; and an optimal parameter adjustment amount calculated by the optimal parameter adjustment amount calculating unit. And an adjusting means for adjusting a parameter of a predetermined model by using the learning device. 2. The learning apparatus according to claim 1, wherein the loss calculation means uses a scale for evaluating the discriminant function value normalized by the normalization means as a distribution of the possibility of a class to which the input pattern belongs. A learning device for determining a loss for a correct answer class. 3. The learning device according to claim 1, wherein the loss calculating means is plausible among the discriminant function values for the correct answer and the discriminant function values for the correct answer among the discriminant function values normalized by the normalizing means. A learning apparatus for determining a loss for a correct answer class using at least one discriminant function value having a value and a scale that is evaluated as a distribution of a possibility that an input pattern belongs to. 4. The learning device according to claim 2, wherein the distribution of the correct answer used for the scale is a distribution having a possibility only in the class to which the correct answer belongs. . 5. The learning apparatus according to claim 1, wherein, when the input pattern is a variable-length input pattern, a parameter for evaluating a feature of the pattern is used as the predetermined model. A learning device, wherein a variable-length input pattern is scored using a state transition model expressed by a parameter for evaluating the continuation length of a partial parameter. 6. The learning apparatus according to claim 5, wherein the state transition model is capable of adjusting a ratio between a score based on the parameter for evaluating the feature amount and a score based on the parameter for evaluating the duration. Characteristic learning device. 7. A learning method for determining a class to which an input pattern belongs by using a discriminant function and adjusting parameters of a predetermined model in order to minimize a loss from a correct answer, wherein a discriminant function for each class is at least a first derivative. After converting to a function having a probable and stochastic constraint and normalizing, a loss for the correct answer is obtained, and by minimizing the loss for the correct answer, an optimal parameter of the predetermined model is obtained. Learning method. 8. The learning method according to claim 7, wherein a parameter of a predetermined model is obtained by a stochastic descent method in order to minimize a loss from a correct answer. 9. The learning method according to claim 7, wherein a loss for a correct answer class is obtained by using a scale for evaluating the normalized discriminant function value as a distribution of possibility to a class to which an input pattern belongs. And learning method. 10. The learning method according to claim 7, wherein, among the normalized discriminant function values, at least one discriminant function value having a plausible value other than the discriminant function value for the correct answer and the discriminant function value for the correct answer. A learning method for determining a loss for a correct answer class using a scale that is evaluated as a distribution of a possibility that an input pattern belongs. 11. The learning method according to claim 9, wherein the distribution of the correct answer used for the scale is a distribution having a possibility only in the class to which the correct answer belongs. 12. The learning method according to claim 7, wherein when the input pattern is a variable-length input pattern, a parameter for evaluating a feature of the pattern is used as the predetermined model. A learning method characterized in that a variable-length input pattern is scored using a state transition model represented by a parameter for evaluating the continuation length of a partial parameter. 13. The learning method according to claim 12, wherein
The learning method, wherein the state transition model is capable of adjusting a ratio between a score based on the parameter for evaluating the feature amount and a score based on the parameter for evaluating the duration. 14. The learning method according to claim 12, wherein an average vector of the parameter for evaluating the feature amount is calculated using the learning method according to any one of claims 7 to 11. A learning method characterized by adjusting. 15. The learning method according to claim 12, wherein a variance of a parameter for evaluating the feature amount is adjusted using the learning method according to any one of claims 7 to 11. Learning method characterized by doing. 16. The learning method according to claim 12, wherein an average vector of the parameter for evaluating the continuation length is determined by using the learning method according to claim 7. A learning method characterized by adjusting. 17. The learning method according to claim 12, wherein a variance of the parameter for evaluating the continuation length is adjusted using the learning method according to any one of claims 7 to 11. Learning method characterized by doing. 18. The learning method according to claim 13, wherein
12. A learning method comprising: adjusting a ratio of a score based on a parameter for evaluating the feature amount and a score based on a parameter for evaluating a continuation length by using the learning method according to any one of claims 7 to 11. Method. 19. A pattern recognition apparatus for obtaining a class to which an input pattern belongs by using a discriminant function and adjusting a parameter of a recognition model in order to minimize a loss from a correct answer. Normalization means for converting the function into a function having a probable and stochastic constraint and normalizing the loss function; loss calculation means for calculating a loss for a correct answer using the discriminant function value normalized by the normalization means; By calculating the optimal parameter adjustment amount of the recognition model by minimizing the loss calculated in step (a), an optimal parameter adjustment amount is calculated by the optimal parameter adjustment amount calculated by the optimal parameter adjustment amount calculation unit. Adjusting means for adjusting parameters, whereby the parameters of the recognition model are adjusted to optimal parameters by the adjusting means. A pattern recognition device configured to perform pattern recognition on an input pattern by using a recognition model adjusted for parameters when the recognition is performed. 20. A pattern recognition method for determining a class to which an input pattern belongs using a discriminant function and adjusting parameters of a recognition model in order to minimize a loss from a correct answer, wherein a discriminant function for each class is at least a first derivative. After converting to a function having a probable and stochastic constraint and normalizing, the loss for the correct answer is obtained, and the loss for the correct answer is minimized, thereby obtaining the optimal parameters of the recognition model. A pattern recognition method comprising: performing pattern recognition on an input pattern using a recognition model adjusted for parameters when the parameters are adjusted to optimal parameters. 21. A discriminant function for each class is converted to a function having at least first-order differentiability and a stochastic constraint and normalized, and then a loss for a correct answer is determined. And a computer-readable recording medium on which a program for causing a computer to execute a process of obtaining an optimal parameter of the model is recorded.