JP4864783B2 - Pattern matching device, pattern matching program, and pattern matching method - Google Patents

Description

  The present invention relates to a pattern matching device, a pattern matching program, and a pattern matching method.

  The speech recognition device compares the time-series acoustic feature extracted from the input speech signal with an acoustic model in which the probability density distribution in the acoustic feature space is learned in advance, with phonemes such as vowels and consonants as units. Get recognition result. The acoustic model, which is a probabilistic model, outputs a score (acoustic likelihood) of the phoneme-likeness with respect to the input of the acoustic feature amount. The speech recognition device accumulates phoneme-likeness scores (acoustic likelihood) over the entire utterance according to the restrictions of the grammar and the word dictionary, and outputs a word sequence having the highest accumulated score as a recognition result.

The acoustic feature amount is time-series data of multi-dimensional vectors, and when the frequency distribution of data corresponding to each phoneme in each dimension is aggregated, it becomes a shape close to a normal distribution or a shape close to the sum of a plurality of normal distributions. In order to express such a distribution of acoustic features, the probability density distribution of the acoustic model is represented by a multidimensional normal distribution or a plurality of multidimensional normal distributions. However, in actual collation, due to variations in microphone characteristics, speaker differences, background noise, etc., a mismatch occurs between the distribution of input acoustic features and the probability density distribution of the acoustic model, which causes the recognition rate to decrease. Become. A method called Cepstral Mean Normalization (CMN) is widely used as a method to eliminate this mismatch in matching input acoustic features with acoustic models, and the average value is a further development of CMN. -Variance normalization (MVN) has been proposed. The CMN subtracts the average value of the entire utterance from the acoustic feature amount at each time of utterance, and makes the average of the acoustic feature amount zero, thereby aligning the distribution of the input acoustic feature amount and the probability density distribution of the acoustic model, This is a technique for reducing mismatches. When the acoustic feature quantity of each dimension before CMN is x (t) and the acoustic feature quantity after CMN is x _c (t), the operation of the CMN is represented by Expressions (1) and (2). T represents the number of frames of the entire utterance.

MVN, on the other hand, normalizes the acoustic feature value at each time of utterance with the average value and variance of the entire utterance, and aligns it with the normal distribution N (average 0, variance 1) of the reference system, and so on. This is a technique for reducing the mismatch between the distribution of input acoustic features and the probability density distribution of the acoustic model. Assuming that the acoustic feature quantity of each dimension before MVN is x (t) and the acoustic feature quantity after MVN is x _m (t), the operation of MVN is expressed by equations (3) to (5).

Further, not only audio but also still images and moving images can be normalized by CMN and MVN. In the case of a still image, assuming that the image feature amount of each dimension is x _{i, j} and the image feature amount after CMN is x _{ci, j} , the operation of the CMN is expressed by equations (6) and (7). I and J represent the number of blocks on the vertical and horizontal axes of a still image.

On the other hand, in MVN, if the image feature quantity of each dimension before MVN is x _{i, j} and the image feature quantity after MVN is x _{mi, j} , the operation of MVN is expressed by equations (8) to (10). .

In the case of a moving image, if the moving image feature amount of each dimension is x _{i, j, t} and the moving image feature amount after CMN is x _{ci, j, t} , the operation of the CMN is expressed by equations (11) and (12). . I and J are the vertical and horizontal axes of the moving image, and T is the number of frames.

On the other hand, in MVN, if the moving image feature amount of each dimension before MVN is x _{i, j, t} , and the moving image feature amount after MVN is x _{mi, j, t} , the operation of MVN is expressed by equations (13) to (15). It is represented by

  However, CMN and MVN using the average value and variance of the entire utterance delay the start of the collation process because the normalized acoustic feature quantity is not obtained until the utterance is finished, and the waiting time from the end of the utterance to the output of the recognition result There is a demerit that makes it longer. As a technique for reducing this processing delay, a technique has been proposed in which an average value or variance is calculated from a local interval of several tens to several hundreds of milliseconds instead of the entire utterance and used for normalization. Hereinafter, the method for normalizing the acoustic feature using the average value calculated from the entire utterance is called batch CMN, and the method for normalizing the acoustic feature using the average calculated from a part of the utterance is called segmental CMN. . Similarly, batch MVN is a method for normalizing acoustic features using the average and variance values calculated from the entire utterance, and normalizes acoustic features using the average and variance values calculated from some sections of the utterance. This technique is called segmental MVN.

There is also known a method in which dispersion normalization is not performed when the dispersion value calculated in the mean value / dispersion normalization (MVN) that does not assume feature quantity quantization is zero or a small value close to zero ( For example, see Patent Document 1).
JP 2002-278586 A

  However, in the CMN, since the batch CMN calculates the average value of feature values from a speech section longer than the segmental CMN, the accuracy is high and the recognition rate improvement effect is high. It is impossible to align even the variation of probability density distribution of the acoustic model to be referred to.

  Also, in MVN, batch MVN normalizes the distribution of acoustic features of the entire utterance to 0 on average and 1 on variance, but focusing on the distribution for each phoneme as a unit of speech recognition, variance is not normalized. On the other hand, if the average / variance calculation interval is set to a time length equivalent to one phoneme (several tens to hundreds of milliseconds) in the segmental MVN, an effect close to normalizing the variance of the distribution of each phoneme can be obtained. . However, since the average value for a short time is also normalized to 0, the average value of all phoneme distributions approaches 0, so that the overlap becomes large (see FIG. 2), leading to a decrease in phoneme identification ability.

  In Patent Document 1, only the application to the batch MVN that normalizes using the average value and the variance value of the entire utterance and the segmental MVN that normalizes the acoustic feature value using the local average value and the variance value. However, it is impossible to solve the above-mentioned problem that the phoneme discrimination ability is deteriorated.

  That is, CMN cannot normalize distribution variation (dispersion), and segmental MVN has an effect close to normalization of dispersion for each phoneme, but the average value of the distribution approaches between phonemes, and the phonemes There is a problem that the discriminating ability of the is reduced.

  In addition, the above-described problem is not limited to speech, but is also applied to a pattern matching device that calculates feature amounts of data input from the outside, normalizes the calculated feature amounts, and performs pattern matching based on the normalized feature amounts. apply.

  The present invention has been made to solve the above-described problem, and a pattern matching device, a pattern matching program, and a pattern matching method capable of normalizing feature quantities without degrading the feature quantity identification capability The purpose is to provide.

The present invention provides an analysis means for calculating feature values of audio data or image data input from the outside, a normalization means for normalizing the feature values calculated by the analysis means, and a normalization by the normalization means A pattern matching unit that performs pattern matching on the basis of the normalized feature quantity that has been normalized, wherein the normalization means includes the total number of frames of the audio data or the feature quantity of the entire image data. An overall average acquisition means for acquiring an overall average value, which is an average value of the audio data, and a local average calculation for calculating a local average value that is an average value of the feature quantity of the local number of frames of the audio data or the local range of the image data and means, based on said local average value, a variance value of the characteristic amount of the local range of the local frame number or the image data of the voice data locality Pattern matching comprising: local variance calculation means for calculating a variance value; and normalization processing calculation means for normalizing the feature quantity based on the overall average value and the plurality of local variance values Device.

Further, the overall average acquisition means of the present invention is characterized in that the overall average value is calculated from the total number of frames of the audio data or the feature amount of the entire image data .

  The overall average acquisition means of the present invention is characterized in that a predetermined value stored in advance is used as the overall average value.

The present invention further includes a range identification unit that identifies a range including the feature quantity that is a target of pattern matching, and the overall average acquisition unit includes the audio data based on the range identified by the range identification unit . 3. The pattern matching device according to claim 2, wherein the overall average value is calculated from the total number of frames or the feature amount of the entire image data .

  The local average calculating means of the present invention calculates the local average value based on a value weighted by the local average value calculated in the past, and the local variance calculating means calculates the local variance calculated in the past. The local variance value is calculated based on a value weighted by the value.

Further, the present invention provides an analysis unit that calculates a feature amount of audio data or image data input from the outside, a normalization unit that normalizes the feature amount calculated by the analysis unit, and the normalization unit. In a pattern matching program for causing a computer to function as a pattern matching unit that performs pattern matching based on a normalized feature amount that has been normalized, the normalization unit includes the total number of frames of the audio data or the image the overall average obtaining means for obtaining an overall average value is an average value of the feature amount of the total data, the local average value is an average value of the feature amount of the local range of the local frame number or the image data of the audio data A local average calculating means for calculating the local average value, and a local value that is a variance value of the feature amount included in the second range based on the local average value A pattern matching program for causing a computer to function as local variance calculation means for calculating a variance value, and normalization processing calculation means for normalizing the feature quantity based on the overall average value and the plurality of local variance values It is.

Further, the present invention provides an analysis step for calculating the feature amount of audio data or image data input from the outside, a normalization step for normalizing the feature amount calculated in the analysis step, and the normalization step. A pattern matching step of performing pattern matching based on the normalized normalized feature quantity, wherein the normalization step includes the total number of frames of the audio data or the whole of the image data. An overall average acquisition step of acquiring an overall average value that is an average value of feature amounts; and a local average value that calculates a local average value that is an average value of the feature amounts of a local range of the audio data or a local range of the image data an average calculation step, on the basis of the local average value, the number of frames the local of the voice data or the image data Normalization processing calculating step of normalizing the feature amount based on the local variance calculation step of calculating a local variance value is the variance value of the characteristic quantity of Tokoro range, the overall average value and a plurality of said local variance And a pattern matching method characterized by comprising:

  According to the present invention, it is possible to normalize the feature amount without reducing the feature amount identification capability.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of a speech recognition apparatus according to an embodiment of the present invention. The acoustic analysis unit 101 performs acoustic analysis on voice data input from a microphone or the like, and calculates an acoustic feature amount. Input can also be controlled by push-to-talk. The acoustic analysis unit 101 temporarily stores the calculated acoustic feature amount in a buffer. The normalization processing unit 102 normalizes the acoustic feature amount stored in the buffer by the acoustic analysis unit 101 using the average value and the variance value of the acoustic feature amount. The normalization process will be described later. The acoustic model learning unit 103 performs the same acoustic analysis on the learning speech data as the speech data to be recognized by the acoustic analysis unit 101, and normalization by the normalization processing unit 102. The acoustic feature quantity of the data is stored in the acoustic model storage unit 104. The language model storage unit 105 stores a word dictionary and grammar. The recognition processing unit 106 performs acoustic analysis on the speech data to be recognized by the acoustic analysis unit 101 and performs normalization processing on the normalization processing unit 102 to obtain the acoustic feature amount and sound of the recognition target speech data. Pattern matching is performed using the acoustic features of the learning speech data stored in the model storage unit 104 and the word dictionary or grammar stored in the language model, and the recognition result is output.

[First embodiment]
First, a first embodiment of the present invention will be described. FIG. 3 shows a configuration of the normalization processing unit 102 according to the present embodiment. The acoustic feature quantity of the entire utterance input to the speech recognition device from a microphone or the like is stored in a buffer (not shown) by the acoustic analysis unit 101. The overall average calculation unit 301 reads the acoustic feature amount within the frame number T corresponding to the entire utterance from the buffer, and calculates the average value. As the length of the entire utterance, it is possible to use the length of the word, the length from the speech break, the length from the punctuation mark to the punctuation mark, the length of the entire input speech, and the like. The average value E (x) of the acoustic features of the entire utterance is obtained by the calculation formula (16).

The local average calculation unit 302 reads an acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the average value. The local frame number τ is a number corresponding to, for example, tens to hundreds of milliseconds as the phoneme length. Since it is the length of a phoneme, it varies depending on the word or person uttered, but in this embodiment, a fixed value is used. The average acoustic feature value E _τ (x) of the local number of frames τ is obtained by the calculation formula (17).

The local variance calculation unit 303 reads an acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the variance value based on the average value calculated by the local average calculation unit 302. The dispersion value V _τ (x) of the acoustic feature quantity of the local frame number τ is obtained by the calculation formula (18).

The normalization processing calculation unit 304 subtracts the average value of the acoustic feature amount for the entire utterance calculated by the overall average calculation unit 301 from the speech feature amount before normalization, and calculates the local value calculated by the local variance calculation unit 303. By dividing by the dispersion value of the acoustic feature quantity of the number of frames τ, the normalized acoustic feature quantity x _τ (t) can be obtained (see the calculation formula (19)).

  As described above, by performing normalization processing based on the average value of the entire utterance on the input acoustic feature amount, the positions of all phoneme distributions are aligned with the corresponding phoneme distribution of the acoustic model, and further according to the local variance value. The normalization process has an effect of bringing the distribution close to the normal distribution while suppressing the overlap of the distribution of all phonemes (see FIG. 4). As a result, it is possible to reduce the mismatch component between the acoustic model due to background noise, reverberation, etc. and the input acoustic feature amount without reducing the accuracy of discrimination between phonemes, and to reduce degradation of speech recognition accuracy. .

  Note that the overall average calculation unit 301 uses the number of frames for which the average is calculated as the number corresponding to the utterance time, but instead, the number of frames τ ′ may be set in advance.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. FIG. 5 shows the configuration of the normalization processing unit 102 according to this embodiment. In the present embodiment, when calculating the local average value and local variance value for the number of local frames of interest, the local average value calculated from the acoustic feature quantity of the previous local frame number (hereinafter referred to as one). And a local variance value calculated from the acoustic feature quantity of the number of local frames immediately before (hereinafter referred to as the previous local variance value). When sudden noise is input to the speech recognition apparatus, the local average value and the local variance value change greatly, making it difficult to correctly recognize the input speech data, but the previous local average value and 1 By using the previous local variance value, even when noise is suddenly input to the speech recognition device, the local average value and the local variance value do not change greatly, and the degradation of the speech recognition system can be reduced.

Similar to the first embodiment, the overall average calculation unit 501 reads the acoustic feature amount corresponding to the entire utterance input to the speech recognition apparatus from the buffer, and calculates the average value E (x). The local average calculation unit 502 reads the acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the average value. At that time, the forgetting factor α is set in advance, and the previous local average value is weighted and added. The local average value E _p (t) of the acoustic feature quantity of the number of local frames τ obtained by weighted addition of the previous local average value is obtained by the calculation formula (20).

The local variance calculation unit 503 reads the acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the variance based on the average value calculated by the local average calculation unit 502. At that time, the forgetting factor α is set in advance, and the previous local average value is weighted and added. The local variance value V _p (t) of the acoustic feature quantity of the number of local frames τ obtained by weighted addition of the previous local average value is obtained by the calculation formula (21).

The normalization processing calculation unit 504 subtracts the average value of the acoustic feature amount of the entire utterance calculated by the overall average calculation unit 501 from the speech feature amount before normalization, and the number of local frames τ calculated by the local variance calculation unit 503. The acoustic feature value x _p (t) after normalization can be obtained by dividing by the variance value of the acoustic feature value (see formula (22)).

  As described above, by performing normalization processing based on the average value of the entire utterance on the input acoustic feature amount, the positions of all phoneme distributions are aligned with the corresponding phoneme distribution of the acoustic model, and further according to the local variance value. The normalization process has an effect of bringing the distribution close to the normal distribution while suppressing the overlap of the distribution of all phonemes (see FIG. 4). As a result, it is possible to reduce the mismatch component between the acoustic model due to background noise, reverberation, etc. and the input acoustic feature amount without reducing the accuracy of discrimination between phonemes, and to reduce degradation of speech recognition accuracy. . Furthermore, when sudden noise is input to the speech recognition device, it becomes difficult to recognize the input speech data, but by using the previous local average value and the previous local variance value, Even when noise is suddenly input to the speech recognition apparatus, the average value does not change greatly, and deterioration of speech recognition accuracy can be reduced.

[Third embodiment]
Next, a third embodiment of the present invention will be described. FIG. 6 shows the configuration of the normalization processing unit 102 according to this embodiment. The present embodiment is characterized in that a fixed average value calculated in advance is used instead of calculating the average value of the acoustic feature values of the entire utterance in the first embodiment. This eliminates the need to calculate the average value of acoustic features from the entire utterance, so the waiting time until the normalization of acoustic features is completed is the time required to calculate local variance, enabling real-time processing. Become.

The fixed average value storage unit 601 stores a preset average value E _f (x) of the acoustic feature amount. The fixed value can be obtained by using an average value of previous utterances or obtained from a huge amount of past voice data.

The local average calculation unit 602 and the local variance calculation unit 603 calculate the local average value and the local variance value as in the first embodiment. The normalization processing calculation unit 602 uses the fixed average value stored in the fixed average value storage unit 601, subtracts the fixed average value from the speech feature value before normalization, and calculates the local frame calculated by the local variance calculation unit 603. By dividing by the variance of the acoustic feature quantity of several τ, the normalized acoustic feature quantity x _f (t) can be obtained (see the calculation formula (23)).

  As described above, instead of calculating the average value of the acoustic feature amount of the entire utterance, it is not necessary to calculate the average value of the acoustic feature amount from the entire utterance in real time by using the fixed average value calculated in advance. Thereby, the waiting time until the normalization of the acoustic feature amount is completed becomes a time required for the calculation of the local variance, and real-time processing is possible. In addition, by normalizing the input acoustic features using the average value of the entire utterance, the positions of all phoneme distributions are aligned with the corresponding phoneme distribution of the acoustic model, and further normalized by local variance values. The processing has an effect of bringing the distribution close to the normal distribution while suppressing the overlap of the distribution of all phonemes (see FIG. 4). As a result, it is possible to reduce the mismatch component between the acoustic model due to background noise, reverberation, etc. and the input acoustic feature amount without reducing the accuracy of discrimination between phonemes, and to reduce degradation of speech recognition accuracy. .

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described. FIG. 7 shows the configuration of the normalization processing unit 102 according to this embodiment. In the present embodiment, a speech section is identified by providing a speech detection unit in front of the overall average calculation unit 702, and the average of the number of frames τ ′ corresponding to the time obtained by adding the speech section and several tens of milliseconds before and after the speech section. It is characterized by normalization using values. Thereby, even when the silent section continues for a long time after the end of the utterance, the waiting time until the normalization process can be shortened.

The voice detection unit 701 detects that voice-specific features are included in the input acoustic feature quantity, and identifies a voice section. As the voice-specific features, voice power, cepstrum value, frequency, and the like can be used. The overall average calculation unit 702 reads out from the buffer the acoustic feature amount corresponding to the utterance in the number of frames τ ′ corresponding to the time obtained by adding the speech section identified by the speech detection unit 701 and the preceding and succeeding tens of milliseconds. Calculate the average value. The average value E _{τ ′} (x) of acoustic feature values of τ ′ is obtained by the calculation formula (24).

The local average calculation unit 703 reads the acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the average value. The number of local frames is a number corresponding to, for example, tens to hundreds of milliseconds as the phoneme length. Since it is the length of a phoneme, it varies depending on the word or person uttered, but in this embodiment, a fixed value is used. The average acoustic feature value E _τ (x) of the local number of frames τ is obtained by the calculation formula (17).

The local variance calculation unit 704 reads an acoustic feature amount corresponding to the utterance within the preset local frame number τ from the buffer, and calculates the variance value based on the average value calculated by the local average calculation unit 703. The dispersion value V _τ (x) of the acoustic feature quantity of the local frame number τ is obtained by the calculation formula (18).

The normalization processing calculation unit 705 subtracts the average value of the acoustic feature amount of the entire utterance calculated by the overall average calculation unit 702 from the speech feature amount before normalization, and the number of local frames τ calculated by the local variance calculation unit 704 The acoustic feature value x _{τ ′} (t) after normalization can be obtained by dividing by the variance value of the acoustic feature value (see formula (25)).

  As described above, the speech section is identified by providing the speech detection unit 701 in the preceding stage of the overall average calculation unit 702, and the average at the number of frames τ ′ corresponding to the time obtained by adding the speech section and several tens of milliseconds before and after the speech section. By normalizing using the value, it is possible to shorten the waiting time until the normalization process even when the silent period continues for a long time after the end of utterance.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described. FIG. 8 is a configuration diagram showing the configuration of the image recognition apparatus according to the present embodiment. In FIG. 4, an image input from a camera is used instead of voice data input from a microphone, an object model is used instead of a word dictionary / grammar and an acoustic model, and an image recognition result is used instead of a voice recognition result. Application to image recognition is also possible.

  An image feature amount analysis unit 801 performs image feature amount analysis on image data input from a camera, and calculates an image feature amount. The normalization processing unit 802 normalizes the image feature amount calculated by the image feature amount analysis unit 801 using the average value and the variance value of the image feature amount. The normalization process will be described later. The object model learning unit 803 obtains the image data for learning by performing the same image feature amount analysis as the image data to be recognized by the image feature amount analysis unit 801 and normalizing by the normalization processing unit 802. Then, the image feature amount of the learning image data is stored in the object model 804. The recognition processing unit 805 performs image feature amount analysis on the image data to be recognized by the image feature amount analysis unit 801, and performs normalization processing on the normalization processing unit 802. Recognition processing is performed using the feature amount and the image feature amount of the learning image data stored in the object model 804, and a recognition result is output.

The image normalization process in this embodiment will be described with reference to FIG. FIG. 9 shows the configuration of the normalization processing unit 102 according to this embodiment. The overall average calculation unit 901 reads the image feature amount of the entire image data input from the camera or the like to the image recognition apparatus from the buffer, and calculates the average value. The average value E (x _{i, j} ) of the image feature values of the entire image data is obtained by the calculation formula (26). I and J represent the number of blocks on the vertical and horizontal axes of a still image.

The local average calculation unit 902 calculates an average value of image feature amounts in a local range of preset image data. As the local range, it is possible to use a peripheral block including the normalization target image range. Local range (k, l) mean E (x _{k, l)} of the image feature amount is obtained by equation (27). I and J represent the number of blocks on the vertical axis and the horizontal axis in the local range of the still image.

The local variance calculation unit 903 calculates the variance value of the image feature amount in the local range of the preset image data based on the average value calculated by the local average calculation unit 902. Local range (k, l) the image feature amount of variance V (x _{k, l)} of obtaining by equation (28).

The normalization processing calculation unit 904 subtracts the average value of the image feature amount of the entire image calculated by the overall average calculation unit 901 from the image feature amount before normalization, and sets the preset image data calculated by the local variance calculation unit 903 By dividing by the variance value of the image feature amount in the range of (2), the normalized image feature amount x _{k, l} can be obtained (see the calculation formula (29)).

  As described above, even in image recognition, by performing normalization processing based on the average value of the entire image for image feature amounts, the positions of all image feature amount distributions are aligned with the corresponding image feature amount distributions of the object model. Furthermore, the normalization process using the local variance value has an effect of bringing the distribution close to the normal distribution while suppressing the overlap of the distributions of all the image feature amounts. As a result, it is possible to reduce the mismatch component of the input image feature quantity and the object model due to shadows, brightness, etc. without reducing the identification accuracy of the image feature quantity, and to reduce the degradation of the image recognition precision. .

  Note that image recognition can be performed not only on a planar image but also on a 3D image. When creating a 3D image, the shadow of the object changes depending on the position of the camera. By using the normalization of the present invention, the mismatch component of the image feature amount can be reduced, and the degradation of the image recognition accuracy is reduced. can do.

[Sixth Embodiment]
In addition, by incorporating a time element into image recognition, it is possible to reduce the mismatch component of the moving image feature quantity for moving images, and to reduce degradation of moving image recognition accuracy.

  A sixth embodiment of the present invention will be described. FIG. 10 is a configuration diagram showing the configuration of the moving image recognition apparatus according to the present embodiment. In FIG. 4, a moving image input from a camera is used instead of voice data input from a microphone, an object model storage unit is used instead of a word dictionary / grammar storage unit and an acoustic model storage unit, and a moving image recognition is performed instead of a voice recognition result. By replacing the result, it can be applied to moving image recognition.

  The moving image feature amount analysis unit 1001 performs moving image feature amount analysis on the moving image data input from the camera, and calculates the moving image feature amount. The normalization processing unit 1002 normalizes the moving image feature amount calculated by the moving image feature amount analysis unit 1001 using the average value and the variance value of the moving image feature amount. The normalization process will be described later. The object model learning unit 1003 is obtained by performing the same moving image feature amount analysis as the recognition target moving image data with the moving image feature amount analyzing unit 1001 and normalizing with the normalization processing unit 1002 with respect to the moving image data for learning. The moving image feature amount of the learning moving image data is stored in the object model 1004. The recognition processing unit 1005 performs the moving image feature amount analysis on the moving image data to be recognized by the moving image feature amount analysis unit 1001 and the normalization processing by the normalization processing unit 1002 to obtain the moving image of the moving image data to be recognized. Recognition processing is performed using the feature amount and the moving image feature amount of the learning moving image data stored in the object model 1004, and a recognition result is output.

With reference to FIG. 11, the normalization process of the moving image in the present embodiment will be described. FIG. 11 shows the configuration of the normalization processing unit 102 according to this embodiment. The overall average calculation unit 1101 reads the moving image feature amount of the entire moving image data input from the camera or the like to the moving image recognition apparatus, and calculates the average value. The average value E (x _{i, j, T} ) of the moving image feature values of the entire moving image data is obtained by the calculation formula (30). I and J are the vertical and horizontal axes of the moving image, and T is the number of frames.

A local average calculation unit 1102 calculates an average value of image feature amounts in a local range of preset moving image data. As the local range, it is possible to use a surrounding block including the normalization target moving image range and the number of local frames. The average value E (x _{k, l, τ} ) of the moving image feature amount of the local range (k, l) and the local frame number τ is obtained by the calculation formula (31). I and J represent the number of blocks in the vertical and horizontal axes in the local range of the moving image, and τ represents the number of local frames.

The local variance calculation unit 1103 calculates the variance value of the moving image feature amount in the local range of the preset moving image data based on the average value calculated by the local average calculation unit 1102. The distribution value V (x _{k, l, τ} ) of the moving image feature quantity of the local range (k, l) and the local frame number τ is obtained by the calculation formula (32).

The normalization processing calculation unit 1104 subtracts the average value of the moving image feature amount of the entire moving image calculated by the overall average calculation unit 1101 from the moving image feature amount before normalization, and preset moving image data calculated by the local variance calculation unit 1103 The moving image feature value x _{k, l, τ} after normalization can be obtained by dividing by the variance value of the moving image feature value in the range and the number of local frames (see the calculation formula (33)).

    The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

  For example, audio, image, and video have been described in detail. However, the present invention is not limited to audio, image, and video, and the present invention can also be applied to a recognition device that performs pattern matching based on the feature amount of input data. .

  Moreover, although voice recognition has been described for the second to fourth embodiments, it can also be applied to image recognition and video recognition.

  Also, a program for realizing the function of the normalization processing unit 102 shown in FIG. 1 or the like is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. Thus, normalization processing may be performed. Here, the “computer system” may include an OS and hardware such as peripheral devices.

  Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used. The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

  Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

It is the block diagram which showed the structure of the speech recognition apparatus by one Embodiment of this invention. It is the figure which showed the mode of the change of the distribution by the normalization process of a segmental MVN method. It is the figure which showed the structure of the normalization process part by the 1st Embodiment of this invention. It is the figure which showed the mode of the change of the distribution by the normalization process of this invention. It is the figure which showed the structure of the normalization process part by the 2nd Embodiment of this invention. It is the figure which showed the structure of the normalization process part by the 3rd Embodiment of this invention. It is the figure which showed the structure of the normalization process part by the 4th Embodiment of this invention. It is the block diagram which showed the structure of the image recognition apparatus by the 5th Embodiment of this invention. It is the figure which showed the structure of the normalization process part by the 5th Embodiment of this invention. It is the block diagram which showed the structure of the image recognition apparatus by the 6th Embodiment of this invention. It is the figure which showed the structure of the normalization process part by the 6th Embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 101 ... Speech analysis part, 102,802,1002 ... Normalization process part, 103 ... Acoustic model learning part, 104 ... Acoustic model, 105 ... Language model, 106,805 ... Recognition processing unit, 301, 501, 702, 901, 1101... Overall average calculation unit, 302, 502, 602, 703, 902, 1102 ... Local average calculation unit, 303, 503, 603, 704, 903, 1103: Local variance calculation unit, 304, 504, 604, 705, 904, 1104 ... Normalization processing calculation unit, 601 ... Fixed average value storage unit, 701 ... Voice detection unit, 801 ... Image feature amount analysis unit, 803, 1003 ... object model learning unit, 804, 1004 ... object model, 1001 ... moving image feature amount analysis unit

Claims (7)

An analysis means for calculating a feature amount of audio data or image data input from the outside;
Normalizing means for normalizing the feature amount calculated by the analyzing means;
Pattern matching means for performing pattern matching based on the normalized feature value normalized by the normalization means;
In the pattern matching device with
The normalizing means includes
An overall average acquisition means for acquiring an overall average value that is the total number of frames of the audio data or the average value of the feature values of the entire image data ;
A local average calculating means for calculating a local average value that is an average value of the feature quantity of the local number of frames of the audio data or the local range of the image data ;
Local variance calculation means for calculating a local variance value which is a variance value of the feature quantity of the local number of frames of the audio data or the local range of the image data based on the local average value;
Normalization processing calculation means for normalizing the feature amount based on the overall average value and the plurality of local variance values;
A pattern matching device characterized by comprising: The pattern matching apparatus according to claim 1, wherein the overall average acquisition unit calculates the overall average value from the total number of frames of the audio data or the feature amount of the entire image data . The pattern matching apparatus according to claim 1, wherein the overall average acquisition unit sets a predetermined value stored in advance as the overall average value. A range identifying means for identifying a range including the feature quantity as a pattern matching target;
The overall average acquisition unit calculates the overall average value from the total number of frames of the audio data or the feature amount of the entire image data based on the range identified by the range identification unit. 2. The pattern matching device according to 2. The local average calculation means calculates the local average value based on a value obtained by weighting the local average value calculated in the past,
The pattern matching device according to claim 1, wherein the local variance calculation unit calculates the local variance based on a value obtained by weighting the local variance calculated in the past. An analysis means for calculating a feature amount of audio data or image data input from the outside;
Normalizing means for normalizing the feature amount calculated by the analyzing means;
Pattern matching means for performing pattern matching based on the normalized feature value normalized by the normalization means;
As a pattern matching program to make a computer function as
The normalizing means includes
An overall average acquisition means for acquiring an overall average value that is the total number of frames of the audio data or the average value of the feature values of the entire image data ;
A local average calculating means for calculating a local average value that is an average value of the feature quantity of the local number of frames of the audio data or the local range of the image data ;
Local variance calculation means for calculating a local variance value which is a variance value of the feature quantity of the local number of frames of the audio data or the local range of the image data based on the local average value;
Normalization processing calculation means for normalizing the feature amount based on the overall average value and the plurality of local variance values;
Pattern matching program to make the computer function as An analysis step for calculating feature values of audio data or image data input from the outside;
A normalizing step of normalizing the feature amount calculated in the analyzing step;
A pattern matching step for performing pattern matching based on the normalized feature value normalized in the normalization step;
In the pattern matching method with
The normalizing step includes
An overall average acquisition step of acquiring an overall average value that is the total number of frames of the audio data or the average value of the feature values of the entire image data ;
A local average calculation step of calculating a local average value which is an average value of the feature amount of the local number of frames of the audio data or the local range of the image data ;
A local variance calculation step of calculating a local variance value, which is a variance value of the feature quantity in the local number of frames of the audio data or the local range of the image data , based on the local average value;
A normalization processing calculation step of normalizing the feature amount based on the overall average value and a plurality of the local variance values;
A pattern matching method characterized by comprising:

Images