JPH1091789A - Device for recognizing word - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a device in which sufficient word recognition can be attained even under a high noise. SOLUTION: A lip tracing part 102 traces the movement of an upper lip and a lower lip from a facial picture obtained by an inputting part 101. A pre-processing part 103 converts it into a pattern based on the difference of the movement of the upper lip and the lower lip, and operates linear complementation and smoothing to the pattern. A matching executing part 104 compares the distances of the inclination of patterns based on an arbitrary part between the pattern processed by the pre-processing part 103 and a dictionary pattern 108, and defines the pattern with the shortest distance as a recognized word.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a word recognition apparatus for recognizing words based on the movement of the lips of a speaker.

[0002]

2. Description of the Related Art Conventionally, there has been known a voice recognition technique for recognizing a speaker's word by comparing a speaker's voice pattern with a dictionary pattern of words created in advance.

[0003]

However, in the conventional speech recognition, a sufficient recognition rate can be secured in a low noise environment such as a laboratory, but in an environment under high noise such as an actual street corner, for example. Sufficient recognition performance could not be obtained. In other words, even in a relatively quiet environment, even if the system can be used practically, sufficient performance cannot be exhibited in an actual office or the like due to the effect of the surrounding noise environment. there were.

[0004] In view of the above, it has been desired to realize a word recognition device capable of performing sufficient word recognition even under high noise.

[0005]

The present invention employs the following structure to solve the above-mentioned problems. <Structure of Claim 1> An input unit for acquiring the face image of the speaker, a lip tracking unit for tracking the movement of each of the upper lip and the lower lip from the face image acquired by the input unit, and a lip tracking unit A pre-processing unit that converts the movement of the lips into a pattern for matching, a dictionary that stores a dictionary pattern corresponding to each word, and a pattern that is converted by the pre-processing unit against the dictionary pattern, and the closest dictionary A matching execution unit that recognizes a pattern as a word spoken by the speaker.

<Explanation of Claim 1> The invention of claim 1 tracks the movement of the lips of a speaker and performs word recognition by comparing the movement pattern with a dictionary pattern registered in advance. It is. Thus, a word recognition device that can be used even under high noise can be obtained.

<Structure of claim 2> In the word recognition device according to claim 1, the pre-processing unit is configured to calculate a difference between a Y coordinate of a coordinate located on the upper lip and a Y coordinate of a coordinate located on the lower lip. It is a word recognition device characterized by converting it into a pattern.

<Explanation of Claim 2> In the invention of claim 2, in the invention of claim 1, the up and down movement of the lips is made a pattern. As a result, the word content of the speaker can be accurately extracted, and thus the recognition accuracy can be improved.

<Structure of claim 3> In the word recognition device according to claim 1 or 2, the preprocessing unit converts the pattern by performing linear interpolation and smoothing of the coordinate value at an arbitrary time. Is a word recognition device.

<Explanation of Claim 3> In the invention of claim 3, the preprocessing section linearly complements the coordinate values of the pattern and performs smoothing processing. As a result, the approximation accuracy of the pattern is improved, and the efficiency of the matching is increased. As a result, it is possible to contribute to the improvement of the recognition rate.

<Structure of Claim 4> In the word recognition device according to any one of Claims 1 to 3, the matching execution unit calculates a distance of a pattern inclination based on a change amount of the pattern at an arbitrary time, The word recognition apparatus is characterized by recognizing a pattern having the shortest distance as a word spoken by a speaker.

<Explanation of Claim 4> When comparing the degree of opening of the mouth when the same word is uttered, the outlines of the waveforms are similar, and when the distance between these patterns is obtained, the distance between the patterns is based on the amount of change. It was confirmed by experiments that the calculation was effective. In other words, if the change in the degree of opening of the mouth when the same word is uttered is graphed, the similarity of the shapes of the peaks and valleys is high, but the height of the peaks and the depths of the valleys vary, except for the time shift. It turned out to be. Therefore, even if DP matching is performed using the height of the waveform, the similarity between the waveforms cannot be accurately represented. However, if the angle calculated from the change amount is used for the matching distance calculation, this angle becomes an approximation of the inclination of the tangent line.
Value close to. Thus, if DP matching is performed using this angle, the peak positions of the peaks and valleys work to match the temporal positions, and the similarity between the waveforms can be accurately represented.

The invention of claim 4 focuses on this point, and therefore, the accuracy of matching can be improved and the recognition rate can be improved.

<Structure of Claim 5> In the word recognition device according to any one of Claims 1 to 5, in each time, a representative pattern serving as a reference and a plurality of patterns having the same utterance content as the representative pattern are used. A dictionary having statistical information of distances indicating the average and variance of distances, and a matching execution unit that calculates a distance for matching based on a value obtained by converting an angle calculated from a change amount into a deviation value by statistical information on distances. A word recognition apparatus characterized in that:

<Explanation of Claim 5> The graph of the change in the degree of opening of the mouth when the same word is uttered has a high similarity in the shape of a mountain or a valley. However, depending on the content of the utterance, there may be a portion where a peak or a valley is always formed regardless of the manner of utterance, and a portion where a hill or a valley is formed or not depending on the manner of utterance. This can be said to be a partial (temporal) feature of the utterance content. The statistical information of the distance indicates this partial feature. That is, a portion where a peak or a valley is always formed has a small variance value, and a portion where a peak or a valley is formed or cannot be formed has a large variance value. If the deviation value obtained from the variance is used for the DP matching distance calculation, this partial feature can be reflected in the calculation result. The invention of claim 5 focuses on this point. Therefore, a high recognition rate can be obtained without being affected by the utterance content.

<Structure of claim 6> In the word recognition device according to any one of claims 1 to 5, each time of a representative pattern of the degree of opening of the mouth serving as a reference and a plurality of patterns of the degree of opening of the mouth. A dictionary having statistical information of the height representing each distance and variance, and the degree of opening of the mouth at each vertex of the waveform in the recognition pattern is converted to a deviation value using the statistical information of the height,
A word recognition device comprising: a matching execution unit that determines a word by using the converted value.

<Explanation of Claim 6> The invention of claim 6 utilizes the opening itself of the mouth.
For example, most of the vowel parts appear as peaks or valleys in the pattern,
When vowels such as “a” and “e” are pronounced, the degree of opening of the mouth varies considerably. But roughly,
The relationship of “a”>“e”>“i”>“o”> “u” is maintained. In other words, even if "a"<"e" sometimes occurs, it is unlikely that "a"<"i". When the same word is pronounced in this way, the range that the vertex can take is limited to some extent. Accordingly, the similarity between patterns can be expressed by expressing this range using statistical information and expressing whether or not the pattern is well within this range using a deviation value. Such statistical information is height statistical information.

Since the range of the vertices is represented by using such statistical information, and the deviation value is used to indicate whether or not the range is well within the range, the similarity between the patterns can be accurately represented. .

[0019]

Embodiments of the present invention will be described below in detail with reference to the drawings. << Specific Example 1 >><Configuration> FIG. 1 is a configuration diagram showing a specific example 1 of the word recognition apparatus of the present invention. The illustrated device includes an input unit 101 and a lip tracking unit 1
02, a pre-processing unit 103, a matching execution unit 104, a dictionary 105, and an output unit 106.

The input section 101 acquires a face image of a speaker, and is composed of an image pickup device such as a television camera. The lip tracking unit 102 is a functional block that tracks upper and lower lips of a face image input from the input unit 101. Pre-processing unit 103
Is a functional block that converts the lip position obtained by the lip tracking unit 102 into a pattern that can be processed by the matching execution unit 104. The matching execution unit 104 is a functional block that executes matching between the pattern in the dictionary 105 and the pattern obtained from the preprocessing unit 103 to recognize words. That is, the matching execution unit 104 has a distance calculation unit 107, and the distance calculation unit 107 calculates the distance between the dictionary pattern 108 stored in the dictionary 105 and the pattern obtained by the pre-processing unit 103, thereby obtaining a word. Is to be recognized. The lip tracking unit 102 to the matching execution unit 104 are functionally configured by hardware, a processor or a memory of a microcomputer, software executed by the processor, and the like.

The dictionary 105 is a dictionary having a dictionary pattern 108 that is a pattern of a recognized word, and the output unit 106 is an output device that outputs the word recognized by the matching execution unit 104.

<Operation> First, a face image photographed by the input unit 101 is sent to the lip tracking unit 102. The lip tracking unit 102 outputs, from this image, a temporal change in coordinates located on the upper lip and coordinates located on the lower lip. The lip tracking unit 102 is, for example, a Japanese Patent Application No. 7-34 filed earlier.
The lip tracking process described in No. 9151 “lip tracking device” is performed.

FIG. 2 is a diagram showing an output example of the lip tracking unit 102. In this figure, the horizontal axis represents time (the unit is a frame), the vertical axis represents the Y coordinate in the face image, and the two curves represent the movements of the upper lip and the lower lip, respectively.

The output of the lip tracking unit 102 is sent to a pre-processing unit 103. The pre-processing unit 103 calculates the difference between the value of the Y coordinate of the upper lip and the value of the Y coordinate of the lower lip, thereby converting the opening state of the mouth. FIG. 3 is a waveform diagram when converted to the degree of opening of the mouth.

Next, normalization of the enlargement ratio of the face image is performed. This can be realized, for example, based on the distance between the eyes or the like. Next, n points are linearly complemented between each point. FIG. 4 is a waveform diagram in which this linear interpolation is performed. The example shown in FIG. 4 shows a case where the points between the points in the waveform of FIG. 3 are complemented by four points. Note that the recognition performance can be improved as the n values for linear interpolation become larger, but the calculation time becomes longer).

Finally, smoothing is performed so that these points are connected smoothly. FIG. 5 is a waveform diagram after smoothing. Smoothing can be achieved, for example, by utilizing a low pass filter.

The waveform that has been subjected to the preprocessing in the preprocessing unit 103 is sent to the matching execution unit 104. The matching execution unit 104 uses the received waveform as a recognition pattern,
A matching process is performed with the pattern of each word recorded in the dictionary 105. Then, as a result of the matching, the word having the smallest distance between the patterns is output to the output unit 106 as a recognition result.

Here, comparing the degree of opening of the mouth when the same word is uttered, the outlines of the waveforms are similar, and when calculating the distance between these patterns, it is effective to calculate based on the amount of change. Was confirmed by experiments. In other words, if the change in the degree of opening of the mouth when the same word is uttered is graphed, the similarity of the shapes of the peaks and valleys is high, but the height of the peaks and the depths of the valleys vary, except for the time shift. It turned out to be. Therefore, even if DP matching is performed using the height of the waveform, the similarity between the waveforms cannot be accurately represented. However, if the angle calculated from the amount of change is used for the calculation of the matching distance, this angle is an approximation of the inclination of the tangent, and therefore, the closer to the peak of a mountain or a valley, the closer to zero. Thus, if DP matching is performed using this angle, the peak positions of the peaks and valleys work to match the temporal positions, and the similarity between the waveforms can be accurately represented.

For example, the change amount of the dictionary pattern 108 is δ
_a, the variation of the pattern to be recognized is taken as [delta] _b,
When (tan ⁻¹ (δ _a / Z) −tan ⁻¹ (δ _b / Z)) ² is used as the distance of each part, when the same word is uttered, the distance between the patterns is reduced, and different words are uttered. In this case, the distance between the patterns works to increase, and excellent recognition performance can be exhibited. Therefore, the distance calculation unit 107
Is configured to calculate the above equation. Here, Z is a coefficient that changes depending on the speaker, the shooting environment, and the like.

Since the amount of change is used, a general D
Although P matching may be used, the use of the technique disclosed in the “pattern matching processing method and apparatus” filed at the same time is more effective.

In the case of the first embodiment, when the utterance content is known, the dictionary pattern 108 is created by using the output of the above-described preprocessing unit 103 and extracting only the utterance part. Extraction of the utterance portion can be easily realized by using the presence or absence of speech if an environment free from noise is used when acquiring the dictionary pattern.

<Effect> As described above, according to the specific example 1, since the matching distance is calculated using the angle calculated from the change amount, the similarity between the waveforms can be accurately represented. Therefore, the recognition rate can be improved.

Further, since linear interpolation and smoothing are performed as pre-processing, the accuracy of approximation is improved, and the angle changes smoothly, so that the efficiency of DP matching is improved, which further contributes to the improvement of the recognition rate. be able to.

<< Specific Example 2 >> In the matching of Specific Example 1,
Although only the dictionary pattern is used, in the specific example 2, the recognition rate is further improved by adding the statistical information of the distance to the dictionary.

That is, as described in the first embodiment, the graph of the change in the degree of opening of the mouth when the same word is uttered has a high similarity in the shape of a mountain or a valley. However, depending on the content of the utterance, there are parts where peaks and valleys can always be formed regardless of the manner of utterance
There may be parts where peaks and valleys may or may not be formed depending on the way of speaking. This can be said to be a partial (temporal) feature of the utterance content. Distance statistics are:
This partial feature is shown. That is, a portion where a peak or a valley is always formed has a small variance value, and a portion where a peak or a valley is formed or not formed has a large variance value. If the deviation value obtained from the variance is used for the DP matching distance calculation, this partial feature can be reflected in the calculation result. Example 2 focuses on this point.

<Structure> FIG. 6 is a diagram showing the structure of the second embodiment. The illustrated device includes an input unit 101 to an output unit 106,
The dictionary 105 has the dictionary pattern 108, the distance statistical information 109, and the matching execution unit 1
The fourth embodiment is characterized in that the distance calculation unit 107 includes a deviation value calculation unit 110. Note that the same reference numerals are given to components equivalent to those of the first embodiment, and description thereof will be omitted.

The distance statistical information 109 is the above-described statistical information, and details thereof will be described later. The deviation value calculation unit 110 is a calculation unit existing inside the distance calculation unit 107, and is used at the time of a recognition operation in this specific example.

First, a method for creating the dictionary 105 will be described. In this specific example, when creating the dictionary 105,
First, a plurality of patterns of the same utterance content are prepared. This pattern uses the output of the preprocessing unit 103 as in the case of creating a dictionary in the specific example 1 described above. Next, one of the most standard patterns is selected as a representative pattern from the plurality of patterns. As a method for selecting this standard pattern, for example, it can be determined by comparing the distance between patterns obtained by performing matching of all patterns with each other.

After the representative pattern is determined, matching of the representative pattern with all other patterns is performed.
(A pattern for performing matching with a representative pattern is hereinafter referred to as a comparison pattern). When the matching is performed, the comparison pattern is expanded and contracted so as to be closest to the representative pattern, so that the time axis is normalized. Therefore, after all matching is completed, the distances between all patterns and the representative pattern are totaled for each time, the average value and the variance are calculated, and these values are used as the distance statistical information 109. The dictionary 105 is configured with the representative pattern as the dictionary pattern 108. However, the matching performed at the time of creating the dictionary described above is performed with the start and end fixed.

Next, the distance statistical information 109 will be described in more detail. Regarding how to obtain the distance statistical information 109,
For example, "Speech Recognition by Probabilistic Model" by Seiichi Nakagawa
An example in which DP matching is performed in an asymmetric form described in IEICE p19 will be described.

The representative pattern is T, the comparison pattern is C ₁ ,
_{C 2, ..., C n,} ..., and C _N. Since the pattern is a time series of degree of opening of the _{mouth, T = a 1, a 2} , ..., a i, ... a I C 1 = b 11, b 12, ..., b 1i, ... b 1J1 C 2 = b 21, _{b 22, ..., b 2i,} ... b 2J2 · · · C n = b n1, b n2, ..., b ni, ... b nJn · · · C N = b N1, b N2, ..., b Ni, ... b _NJN . Here, a _i represents the i-th data of T (the degree of opening of the mouth). Similarly, b _ni represents the i-th data of C _n , and b _NJN represents the JN-th data of C _N.

Now, the time conversion function F _n when T and C _n are most matched is represented by F _n = c _n (1), c _n (2),..., C _n (i),.
c _n (I). This is because the pattern T
Indicates that the i-th data of the pattern C _{n and} the c _n (i) -th data of the pattern C _n correspond to each other. That is, a _i and b _ncn (i)
Respectively correspond. Since the start and end are fixed, c _n (1) = 1c _n (I) = j _n is always satisfied.

At this time, the dictionary pattern T and the comparison pattern C
_n of the distance D (T, C _n) is as follows. FIG. 7 is an explanatory diagram of a calculation regarding the statistical information of the distance. Here, the distance D (T, C _n ) is expressed as in equation (1), and when expanded, it becomes as in equation (2). Therefore, assuming that the average of the distance for each time is μ _i and the variance is σ _i , μ _i is expressed by equation (3), σ _i is expressed by equation (4), and the distance statistical information 10
9, (μ ₁ , μ ₂ ,..., Μ _I ) and (σ ₁ , σ ₂ ,
..., σ _I ).

Next, a recognition method in the matching execution unit 104 using the dictionary 105 will be described. Here, the recognition method is basically the same as that of the first embodiment, but the distance calculation method used in the matching processing is different.

First, as in the first embodiment, the distance calculation unit 107 calculates the distance of each part. Next, a deviation value is calculated using the distance statistical information 109 in the dictionary 105, and this is used as the distance. That is, when the distance calculated in the same manner as in Example 1 and d (c (i)), the distance dl _i actually utilized is as the following equation (5).

[0046] Figure 8 is an operation explanatory view of a distance dl _i for actual use. In general, a matching algorithm does not depend on a method of calculating a distance, and therefore can be applied to a case where any matching method of the algorithm is used in the matching execution unit 104.

<Effects> According to the specific example 2 as described above,
Since the statistical information of the distance was used in addition to the configuration of the specific example 1,
Since the partial characteristics of each pattern can be reflected in the calculation result, the recognition rate can be further improved.

<< Specific Example 3 >> In the specific examples 1 and 2, only the amount of change in the degree of opening of the mouth is used, but in the specific example 3, the degree of opening of the mouth itself is used.

For example, most of vowel-producing portions appear as vertices. However, when vowels such as "a" and "e" are produced, the degree of opening of the mouth varies considerably.
However, roughly, "a">"e">"i">"o"
>"U". In other words, even if "a"<"e" sometimes occurs, it is unlikely that "a"<"i". When the same word is pronounced in this way, the range that the vertex can take is limited to some extent. Accordingly, the similarity between patterns can be expressed by expressing this range using statistical information and expressing whether or not the pattern is well within this range using a deviation value.

<Structure> FIG. 9 is a view showing the structure of the third embodiment. The illustrated device includes an input unit 101 to an output unit 106,
The dictionary 105 has height statistical information 111 in addition to the distance statistical information 109 of the specific example 2, and includes a height score calculating unit 112 and a total score calculating unit 113.

The height statistical information 111 is statistical data constituting the dictionary 105, and a method for creating the statistical information will be described later. The height score calculation unit 112 includes the matching execution unit 10
A unit that scores the degree of opening of the mouth using the result of No. 4 and the information of the statistical information 111 of height. The total score calculation unit 113 is a unit that calculates a total score from the scores obtained by the distance and height score calculation units 112 obtained by the matching execution unit 104.

First, a method for creating the dictionary 105 will be described. In the specific example 2, the representative pattern and the comparison pattern are matched to normalize the time axis, and the distance between all the patterns and the representative pattern is defined as the average value and the variance value for each time as the distance statistical information 109. Similarly, an average value and a variance value for each time of the degree of opening of the mouth are obtained, and this is set as the height statistical information 111. Furthermore, the statistical information 111 of this height
Then, a dictionary 105 is constructed by adding the dictionary pattern 108 and the statistical information 109 of the distance obtained by the same method as in the specific example 2.

Next, a method of obtaining the height statistical information 111 will be described by taking as an example a case where the above-described asymmetric matching is performed.

The representative pattern is T, the comparison pattern is C ₁ ,
_{C 2, ..., C n,} ..., and C _N. Since the pattern is a time series of degree of opening of the _{mouth, T = a 1, a 2} , ..., a i, ... a I C 1 = b 11, b 12, ..., b 1i, ... b 1J1 C 2 = b 21, _{b 22, ..., b 2i,} ... b 2J2 · · · C n = b n1, b n2, ..., b ni, ... b nJn · · · C N = b N1, b N2, ..., b Ni, ... b _NJN . Here, a _i represents the i-th data of T (the degree of opening of the mouth). Similarly, b _ni represents the i-th data of C _n , and b _NJN represents the JN-th data of C _N.

Now, the time conversion function F _n when T and C _n are most matched is represented by F _n = c _n (1), c _n (2),..., C _n (i),.
c _n (I). This is because the pattern T
Indicates that the i-th data of the pattern C _{n and} the c _n (i) -th data of the pattern C _n correspond to each other. That is, a _i and b _ncn (i)
Respectively correspond. Since the start and end are fixed, c _n (1) = 1c _n (I) = j _n is always satisfied.

Therefore, the average value of the degree of opening of C ₁ ,..., C _N corresponding to a _i in the representative pattern T is represented by m _i ,
If the variance value is s _i , it can be expressed by the following equation 6. FIG. 10 is an explanatory diagram of the calculation of Expression 6. Therefore, the height statistical information 111 is (m ₁ , m ₂ ,..., M _I ) and (s
₁ , s ₂ ,..., S _I ).

Next, a recognition method using the dictionary 105 will be described. Basically, it is the same as the specific example 2. However, in addition to the distance obtained by the same method as the specific example 2, the score of the degree of opening of the mouth is obtained, and the total score is obtained from these values. They differ in that they determine the result.

First, processing is performed in the same manner as in the second embodiment until the matching execution unit 104 obtains the distance between the recognition pattern and the dictionary pattern. Next, in the height score calculation unit 112, as a result of the matching, which vertex of the dictionary pattern is associated with which point in the recognition pattern is obtained from the time conversion function F in the matching execution unit 104. next,
Using the average value and the variance of the degree of opening of the mouth when corresponding to the vertices in the same dictionary pattern in the height statistical information 111, the deviation value of the degree of opening of the mouth is obtained. The above calculation is performed for all vertices in the dictionary pattern, and the average value is used as the height score. Here, the apex indicates a peak or a valley in a waveform when the pattern is graphed as time on the horizontal axis and the degree of opening of the mouth on the vertical axis.

FIG. 11 is an explanatory diagram for calculating the height score. As shown in the figure, using a mathematical expression, at T, the data numbers at which vertices exist are represented by p (1), p (2),.
p (P), and the recognition pattern R is R = e ₁ , e ₂ ,.
e _l, ..., when the e _L, the score S of the height can be represented by the illustrated formula (7).

Next, the total score calculation unit 113 obtains a total score V from the distance obtained by the matching execution unit 104 and the height score obtained by the height score calculation unit 112. This is obtained from the following equation. V = D (T, R) + S · K where K is a coefficient that changes depending on the speaker, the shooting environment, and the like.

In this way, the total score is calculated using the pattern of each word recorded in the dictionary 105 and the recognition pattern. Then, the word having the smallest total score is output to the output unit 106 as a recognition result.

<Effects> As described above, in the third embodiment, the DP matching is performed using the angle to eliminate the time lag, and then the degree of opening of the mouth is compared. Therefore, the comparison target is clear.

Further, since the range of the vertices is represented by using the statistical information, and the deviation value is used to indicate whether or not the range falls within the range, the similarity between the patterns can be represented.

Further, similarity between patterns is obtained by using the total score calculated using both the distance between patterns when DP matching is performed using an angle and the score obtained from the degree of opening of the mouth. The performance can be comprehensively evaluated from the two viewpoints of the outline of the waveform and the position of each vertex, and the recognition rate can be further improved.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a specific example 1 of the word recognition device of the present invention.

FIG. 2 is a diagram illustrating an output example of a lip tracking unit in the word recognition device of the present invention.

FIG. 3 is a waveform diagram when converted to a degree of opening of a mouth in a preprocessing unit in the word recognition device of the present invention.

FIG. 4 is a diagram illustrating a 4 in a preprocessing section in the word recognition device of the present invention.
FIG. 9 is a waveform diagram when points are linearly complemented.

FIG. 5 is a waveform diagram after smoothing is performed in a preprocessing unit in the word recognition device of the present invention.

FIG. 6 is a configuration diagram of a specific example 2 of the word recognition device of the present invention.

FIG. 7 is an explanatory diagram of calculation relating to statistical information of distance in the specific example 2 of the word recognition device of the present invention.

FIG. 8 is an explanatory diagram of a calculation of a distance actually used in a specific example 2 of the word recognition device of the present invention.

FIG. 9 is a configuration diagram of Example 3 in the word recognition device of the present invention.

FIG. 10 is an explanatory diagram illustrating a calculation representing an average value and a variance value of the degree of opening of a mouth in the specific example 3 of the word recognition device of the present invention.

FIG. 11 is an explanatory diagram of a calculation of a height score in the specific example 3 of the word recognition device of the present invention.

[Explanation of symbols]

 Reference Signs List 101 input part 102 lip tracking part 103 preprocessing part 104 matching execution part 105 dictionary 106 output part 107 distance calculation part 108 dictionary pattern 109 distance statistical information 110 deviation value calculation part 111 height statistical information 112 height score calculation part 113 Total Score Calculator

Claims (6)

[Claims] An input unit for obtaining a face image of a speaker; a lip tracking unit for tracking movement of each of an upper lip and a lower lip from the face image obtained by the input unit; A preprocessing unit that converts the movement of the lips into a pattern for performing matching; a dictionary that stores a dictionary pattern corresponding to each word; and a pattern that is converted by the preprocessing unit is compared with the dictionary pattern. A word recognition device comprising: a matching execution unit that recognizes a close dictionary pattern as a word spoken by the speaker. 2. The word recognition device according to claim 1, wherein the preprocessing unit converts the word into a pattern based on a difference between a Y coordinate of coordinates located on the upper lip and a Y coordinate of coordinates located on the lower lip. A word recognition device characterized by the following. 3. The word recognition device according to claim 1, wherein the pre-processing unit converts a pattern by performing linear interpolation and smoothing of coordinate values at arbitrary time intervals. . 4. The word recognition apparatus according to claim 1, wherein the matching execution unit calculates a distance of the inclination of the pattern based on an arbitrary amount of change in the pattern, and determines a pattern having the smallest distance. A word recognition device that recognizes a word as a word spoken by a speaker. 5. The word recognition apparatus according to claim 1, wherein an average and a variance of a distance at each time between a reference representative pattern and a plurality of patterns having the same utterance content as the representative pattern are set. And a matching execution unit that calculates a distance for matching based on a value obtained by converting an angle calculated from the amount of change into a deviation value by the distance statistical information. Word recognition device. 6. The word recognition apparatus according to claim 1, wherein a distance and a variance at each time between a representative pattern of the degree of opening of the mouth serving as a reference and a plurality of patterns of the degree of opening of the mouth. A dictionary having statistical information of the height representing the height of the word at each apex of the waveform in the recognition pattern is converted to a deviation value using the statistical information of the height, and the converted value is used for the word using the converted value. A word recognition device, comprising: a matching execution unit that makes a determination.