JPH05165494A - Voice recognizing device - Google Patents

Abstract

PURPOSE:To obtain a low-priced system facilitating the study of a learning person without being hourly limited by providing a correspondence display means to display the prescribed acoustic feature amount of a voice signal and a language symbol corresponding to the amount while making them correspondent. CONSTITUTION:This device is equipped with an input means 50 to input the voice of an unspecified speaker, voice recognizing means 51 to obtain the language symbol by executing voice recognition according to the voice signal, and display means 54 to display the contents of making the prescribed acoustic characteristic amount of the voice signal correspondent to the language symbol corresponding to the amount by normalizing them concerning the two kinds of voices at least while making those amount and symbol correspondent. Namely, a normalizing part and graphic display part 54 inputs the voice data of the learning person outputted from the voice recognizing means 51 and model voice data from a model data acquisition part 52, normalizes speaking time for each word to which both data are correspondent, and graphically displays the voice data of the learning person and the model voice data (intonation, stress).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice recognition device applicable to fields such as education.

[0002]

2. Description of the Related Art Conventionally, for example, the following methods have been known as methods for training pronunciation (speaking) in foreign language education. (1) Ask a foreign language education expert or native speaker to listen to their pronunciation, receive advice, and point out the learner's shortcomings. (2) Listen to commercially available foreign language learning tapes and bilingual broadcasts, and imitate them. Also, practice based on the charts that show mouth shapes, articulation points, etc., described in reference books. Record your pronunciation on a magnetic tape and compare it with the voice recorded on the learning tape. (3) Practice using a pronunciation practice device that uses a computer or the like. That is, a CAI (computer assisted instruction) system for pronunciation practice is known. It listens to native speaker pronunciation. Record and play back the learner's voice. Graphic display of native speaker's voice waveform and learner's voice waveform. Also,
Formants are displayed, and changes in frequency and sound pressure are displayed over time.

[0003]

However, the conventional pronunciation training method as described above has the following problems. (1) Training methods by foreign language education specialists and native speakers can be expected to have great effects, but they are time-consuming and costly. (2) With a method using a voice tape, etc., although it is possible to practice anytime and it is inexpensive, it is difficult for the learner to grasp where his or her pronunciation is not good because the pronunciation that is produced cannot be evaluated objectively. is there. (3) In the method of practicing pronunciation using a computer, the native speaker's voice waveform and the learner's voice waveform are displayed, but if you read what from that, the learner's pronunciation will be pronounced. There is a problem that it is difficult to know if it approaches the pronunciation of a native speaker.

In consideration of the above problems of the conventional pronunciation training method, the present invention provides a speech recognition apparatus which can realize a system that is not restricted by time, is inexpensive, and is easy for learners to learn. It is intended to be provided.

[0005]

The first aspect of the present invention is to provide an input means for inputting voice, a recognition means for performing voice recognition from the voice signal to obtain a language symbol, and a predetermined acoustic characteristic of the voice signal. It is a voice recognition device provided with correspondence display means for displaying an amount and the language symbol corresponding thereto in association with each other.

A second aspect of the present invention is an input means for inputting a voice of an unspecified speaker, a recognition means for performing voice recognition from the voice signal to obtain a language symbol, and a predetermined acoustic feature amount of the voice signal. And a display means for displaying the content of correspondence between the language symbol and the corresponding language symbol, for at least two types of voices in a normalized manner and in correspondence.

A third aspect of the present invention is an input means for inputting a voice of an unspecified speaker, a recognition means for performing voice recognition from the voice signal to obtain a language symbol, and a predetermined acoustic feature amount of the voice signal. And a comparing means for comparing the content of correspondence between the language symbol and the corresponding language symbol with respect to at least two types of voices.

[0008]

In the first aspect of the present invention, speech recognition is performed from the speech signal input by the input means by the recognition means to obtain a language symbol. Further, the correspondence display means displays the predetermined acoustic feature amount of the voice signal and the corresponding language symbol in association with each other.

In the second aspect of the present invention, speech recognition is performed from the speech signal of the unspecified speaker input by the input means to obtain a language symbol. The display means normalizes and displays at least two types of sounds the correspondence content between the predetermined acoustic feature amount of the sound signal and the corresponding language symbol.

In the second aspect of the present invention, speech recognition is performed from the speech signal of the unspecified speaker input by the input means to obtain a language symbol. The comparing means compares the correspondence between the predetermined acoustic feature amount of the voice signal and the corresponding language symbol for at least two types of voice.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing a system using a computer for graphically displaying voice data which is an embodiment of the voice recognition apparatus of the present invention. In the figure, the learner's voice is input from the microphone 15 and analyzed by the unspecified speaker continuous voice recognition means 14. The program for performing the processing of the present invention and necessary data are the magnetic disk 13
Stored in the memory 12 and loaded into the memory 12 by the CPU 11. The keyboard 17 and mouse 18 are used when selecting a practice sentence, which will be described later.
The stress or the like is graphically displayed on the display 16.

FIG. 2 shows voice data as an example of the output of the unspecified speaker continuous voice recognition means used in the present invention. Number of recognized words 21, array of character strings of each word 22, array of number of segments (recognized phonemes) 23 of each word, array of average pitch frequency 24 of each segment, array of average sound pressure of each segment 25, and an array 26 of utterance times of each segment.

Example 1 below is an example of the first aspect of the present invention and shows an apparatus for displaying the intonation or stress of a voice signal to be recognized in association with a word as a recognition result.

Further, in the following description, the nth element of the array means that the first element is the 0th element, and the first element, the second element ,. ． ． And

(First Embodiment) FIG. 3 is a block diagram of a first embodiment of the present invention. Input means 3 such as a microphone 15
A voice recognition processing unit 3 as an example of a recognition unit that inputs a learner's voice from 0 and outputs the voice data shown in FIG.
1, and the voice data output from the voice recognition processing unit 31 as an input, and the intonation of the input voice,
The graphic display unit 32 is an example of correspondence display means for displaying the stress on the display in association with the word string of the recognition result.

FIG. 4 is a flow chart showing the contents of the voice recognition processing section 31. That is, in FIG. 4, in step 310, analog processing and A / D conversion are performed on the audio signal from the microphone 15. here,
Input: analog continuous audio signal, output: digital audio signal, processing outline: anti-alias filter, with a sampling frequency of 16 kHz, to convert to a digital signal.

At step 311, acoustic parameter conversion is performed. Here, input: digital audio signal, output: 23 acoustic feature values for each frame (6.6 ms), processing outline: the digital signal is divided into 6.6 ms frames, and 23 The acoustic feature quantity of is extracted.

At step 312, phoneme code (phoneme piece) conversion is performed. Here, input: acoustic feature amount for each frame, output: phoneme code (phoneme piece) for each segment, and acoustic feature amount for each segment, processing outline:
A phoneme code (about 1800 kinds) is assigned to each frame based on the acoustic feature amount of each frame. Adjacent frames with similar characteristics are combined into one segment.

In step 313, phoneme code (phoneme piece) / phoneme conversion is performed. Here, input: phoneme code (phoneme piece) for each segment and acoustic feature amount for each segment, output: phoneme candidate for each segment, and acoustic feature amount for each segment, external reference data: phoneme codebook 315, Outline of processing: With reference to the phoneme codebook, a possibility that the segment corresponds to each phoneme (about 50 types) is assigned to each segment.

In step 314, phoneme / word string conversion is performed. Here, input: phoneme candidates for each segment and acoustic feature amount for each segment, output: acoustic feature amount for each word string and segment, external reference data: word dictionary 316, processing outline: refer to word dictionary, The distance between the input segment sequence and the phoneme sequence corresponding to each word is calculated, and the closest one is output as the recognition result.

The phoneme codebook 315 is a table having a possibility that each phoneme code (phoneme piece) has a phoneme code, and the word dictionary 3 is used.
Reference numeral 16 is a dictionary having word character strings and corresponding phoneme strings.

5 and 6 are examples of the processing flow of the graphic display unit 32 of FIG.

Reference numeral 401 is a pre-processing which displays the X-axis and the Y-axis on the display and initializes each variable. That is, a variable L_word_cnt showing which number of the word string of the recognition result is the word being processed, a variable L_seg_cnt showing what number of the array of the segment of the recognition result is the segment being processed, and a line graph to be displayed graphically (see below. 0) L_x of the variable (array) that indicates the value of each sample point
Substitute 0 for each of [0] and L_y [0].

At 402, the number of words in the voice data of the voice recognition result is substituted into L_word_num.

At 403, L_word_cnt and L_word_num are compared. If L_word_cnt is smaller than L_word_num, it means that there is a word to be processed, and step 4
Go to 04. If L_word_cnt is greater than or equal to L_word_num, it means that the graphic display has been completed for all words, and all processing is completed.

At 404, X is displayed at the next line graph display.
The flag disp_word_flag indicating that the word currently being processed is displayed at the bottom of the axis is set to ON.

At 405, the L_word_cnt th element of the array of the segment number of each word of the audio data is added to the variable L_seg_num which indicates up to what segment the word currently being processed is.

At 406, L_seg_cnt and L_seg_num are compared. If L_seg_cnt is smaller than L_seg_num, it means that there is an unprocessed segment in the segment corresponding to the word currently being processed, and the process proceeds to step 407. If L_seg_cnt is greater than or equal to L_seg_num, it means that processing has been completed for all the segments corresponding to the word currently being processed, and the routine proceeds to step 410.

At 407, sample points of the line graph to be displayed are acquired. That is, L_x [L_seg_cnt + 1] becomes {L
_x [L_seg_cnt] + (L_seg_cnt th element of the utterance time array of the voice data segment) is substituted, and L_y [L_seg_cn
Substituting (L_seg_cnt th element of average array of pitch frequencies of audio data segments) or (L_seg_cnt th element of average array of sound pressure of audio data segments) to [t + 1]. When the line graph of the intonation is displayed, the elements of the average array of the pitch frequencies of the segments are substituted, and when the line graph of the stress is displayed, the elements of the average array of the sound pressure of the segments are substituted.

At 408, a line graph is actually displayed. That is, {(L_x [L_seg_cnt], L_y [L_seg_cnt),
Display the line segment connecting the two points (L_x [L_seg_cnt + 1], L_y [L_seg_cnt +1])}.

At 409, it is determined whether to display the word character string corresponding to the lower part of the X axis. That is, disp_w
It is checked whether ord_flag is ON, and if it is ON, the process proceeds to step 410, and if it is OFF, the process proceeds to step 412.

At 410, the character string which is the L_word_cnt th element of the word character string array 22 of the voice data is displayed immediately below (L_x [L_seg_cnt + 1], 0).

In 411, since the display of the word character string need only be performed once for one word, disp_word_flag is turned off.

At 412, 1 is added to L_seg_cnt to move the processing to the next segment, and the process returns to step 406.

At 413, since the processing is moved to the next word,
Add 1 to L_word_cnt and return to step 403.

Next, the above processing will be described more specifically by using a specific example of data.

When a voice is input using the learner's microphone 15, the voice recognition means 31 performs a voice recognition process by a technique such as spectrum analysis. Here, the voice recognition means 31
It is assumed that the audio data shown in FIG.

The graphic display unit 32 receives the voice data and performs the following processing.

First, as preprocessing, display of X-axis and Y-axis,
0 for L_word_cnt, L_seg_cnt, L_x [0], and L_y [0] respectively
Is substituted (step 401). Then, 4 which is the value of the number of words of the voice data is substituted into L_word_num (step 402).

Here, L_word_cnt and L_word_num are compared (step 403). Now, L_word_cnt is 0, L_word
Since _num is 4, the process proceeds to step 404.

In step 404, the disp_word_flag indicating that the character string of the word to be processed now is displayed at the bottom of the X axis at the time of displaying the next polygonal line is turned ON. Then L_seg_nu
An array L_word_ of the number of segments of each word of audio data in m
Add the cnt th element. Here, L_seg_num = 0, L_wo
rd_cnt = 0. Since the 0th element of the array of the segment number of each word of the voice data is 3, L_seg_num = 3 (step 405).

In step 406, L_seg_cnt and L_seg_n
Compare um. Since L_seg_cnt = 0 and L_seg_num = 3 now, the process proceeds to step 407.

Here, the sample points for actually displaying the line graph are acquired. That is, the value of L_x [0] and the value of the 0th element of the utterance time array of each segment of the audio data are added to L_x [1], and 24 is substituted. Further, for example, if an intonation graph is displayed, the value 104 of the 0th element of the average array of pitch frequencies of each segment of audio data is substituted into L_y [1] (step 40).
7). Then, the line graph is actually displayed. That is, it is the value of {(L_x [0], L_y [0]), (L_x [1], L_y [1])}
A line segment connecting two points {(0,0), (24,104)} is displayed (step 408).

Next, it is checked whether or not disp_word_flag is ON (step 409). Since it is ON now, processing for displaying the word character string corresponding to the lower part of the X axis is performed.
That is, "how", which is the 0th value of the word character string array of the voice data, is displayed immediately below (x [1], 0) (step 41).
0). While processing "how", there is no need to display this character string anymore, so disp_word_flag is set to OFF (step 411).

Since the processing for the 0th segment is completed, 1 is added to L_seg_cnt to make it 1 and step 40
Return to 6. A line graph is displayed for the first segment as well (steps 407 and 408).

At step 409, disp_word_flag is set to O.
Since it is FF, the display process of the word character string is not performed, and the process proceeds to step 412.

In this way, the processing up to the second segment is completed. At this time, L_seg_cnt is 3 (step 412). As a result of the comparison in step 406, L_seg_cnt is equal to or more than L_seg_num. Therefore, the process proceeds to step 413, and the process is performed on the next word.
Add 1 to L_word_cnt and return to step 403.

Similarly, it is assumed that all the processes up to the fourth word have been completed. Then, L_word_cnt becomes 4 (step 413), and as a result of the comparison in 403, it is determined that the processing has been completed for all the words, and the processing of this embodiment ends.

(Embodiment 2) The embodiment 2 is an embodiment of the speech recognition apparatus of the second invention, and is the intonation or stress of the speech in which a certain sentence is input in the practice of pronunciation of a foreign language. Is displayed graphically in correspondence with the recognized words, and the intonation or stress of the learner's voice in the same sentence is normalized by matching the recognized words and displayed in correspondence with the model graphic for each word. It is an example of a device to do.

FIG. 7 is a block diagram of the second embodiment.

The voice recognition means 51 is a means for a learner to input a voice and output voice data as a recognition result, and has the same function as the voice recognition means 31 of the first embodiment.
Further, the word string obtained as a result of the voice recognition processing is assumed to be the same as the practice sentence character string 61 (see FIG. 8) of the practice sentence selected by the learner, except for the silent parts.

The practice sentence presenting section 52 displays a list of practice sentences using the practice sentence number 60 and the practice sentence character string 61 of the teaching material file stored in the teaching material data 55, and presents the learner with a sentence to be practiced. And urge you to enter your gender. When the learner selects a practice sentence and inputs the sex, the practice sentence number and the gender corresponding to the selected practice sentence are output.

The model data acquisition unit 53 is a practice sentence presentation unit 5.
It is a means for inputting the number of the practice sentence selected by the learner and the gender of the learner, which is the output of 2, and reading the teaching material data 55 to obtain the voice data file corresponding to the number of the practice sentence from the model voice data 56. ..

The normalization section and the graphic display section 54 as an example of the display means of the present invention include the learner's voice data and the model data acquisition section 5 output from the voice recognition means 51.
It is a means for inputting the model voice data from No. 3, normalizing the utterance time for each word corresponding to both, and graphically displaying the voice data of the learner and the voice data of the model (intonation, stress). 15 and 16 are examples of screens of such a graphic display means 54.

FIG. 8 shows an example of a teaching material file, which comprises a practice sentence number 61, a practice sentence character string 62, a voice data file name 63 for men, and a voice data file niece 4 for women.

FIG. 9 is an example of a model voice data file, in which the model voice data having the same structure as the voice data obtained by recognizing the learner's voice is stored. The data of these examples are obtained in advance as described in the first aspect of the present invention.

FIG. 10 shows an example of the process flow of the normalization and graphic display unit 54.

Reference numeral 801 denotes preprocessing, which is the display of the X-axis and Y-axis, L_word_cnt indicating the number of the word in the voice data of the learner currently being processed, and the voice data of the learner currently being processed. L_seg_cnt indicating the number of the segment, the 0th element L_x [0], L_y [0] of the array that is the sample point when the learner's intonation (stress) line graph is displayed,
M_word_cnt indicating the number of the word of the model voice data currently being processed, M_seg_cnt indicating the number of the segment of the model voice data currently being processed, and the line graph display of the intonation (stress) of the model 0 is assigned to each of the 0th element M_x [0] and M_y [0] of the array which becomes the sample point of.

At 8011, the number of words of the voice data of the voice recognition result is substituted into L_word_num.

802 checks L_word_cnt and L_word_n in order to check whether processing has been completed for all words.
Compare um. If L_word_cnt is smaller than L_word_num, it means that there are still words to be processed, and the process proceeds to step 803. If not, all the processes are terminated.

In 803, M_word_cnt of the model voice data
Display the line graph and the corresponding word string for the th word. An example of the detailed flow of this step is shown in FIG.
1 and FIG.

At 805, a magnification for displaying the intonation or stress corresponding to the L_word_cnt th learner's word in association with the model intonation or stress is determined. An example of the detailed flow of this processing is shown in FIG.
3 shows.

At 806, the intonation or stress corresponding to the L_word_cnt th learner's word is displayed as a line graph. At this time, it is displayed at the same position (X coordinate) as the corresponding word in the model according to the magnification obtained in 805.

At 807, 1 is added to the values of L_word_cnt and M_word_cnt so that the learner and the example can proceed to the next word.

11 and 12 show step 8 of FIG.
It is an example of a detailed flow of 03.

In 901, X is displayed at the next line graph display.
Disp_word_, which indicates to display the word string at the bottom of the axis
Turn flag on. Also, 0 is substituted for the variable M_word_dur indicating the time required to speak this word.

At 902, the M_word_cnt th element of the array of the segment number of each word of the audio data is added to the variable M_seg_num which indicates up to what segment the word currently being processed is.

At 903, M_seg_cnt and M_seg_num are compared. If M_seg_cnt is smaller than M_seg_num, it means that there is an unprocessed segment in the segment corresponding to the word currently processed, and the process proceeds to step 904. If M_seg_cnt is greater than or equal to M_seg_num, it means that processing has been completed for all the segments corresponding to the word currently being processed, and the processing ends.

At 904, the sample points of the line graph to be displayed are acquired. That is, in M_x [M_seg_cnt + 1],
_x [M_seg_cnt] + (M_seg_cnt th element of the utterance time array of the voice data segment) is substituted, and M_y [M_seg_cn
Substitute (M_seg_cnt th element of average array of pitch frequencies of audio data segments) or (M_seg_cnt th element of average array of sound pressure of audio data segments) to [t + 1]. When the line graph of the intonation is displayed, the elements of the average array of the pitch frequencies of the segments are substituted, and when the line graph of the stress is displayed, the elements of the average array of the sound pressure of the segments are substituted.

At the same time, the M_seg_cnt th element of the utterance time array of the voice data segment is added to M_word_dur.

At 905, a line graph is actually displayed. That is, ((M_x [M_seg_cnt], M_y [M_seg_cnt), (M
_x [M_seg_cnt + 1], M_y [M_seg_cnt + 1])} is displayed.

At 906, it is determined whether to display the word character string corresponding to the lower part of the X axis. That is, disp_w
It is checked whether ord_flag is ON. If it is ON, the process proceeds to step 907. If it is OFF, the process proceeds to step 909.

At 907, the character string which is the M_word_cnt th element of the recognition result word character string array 22 is displayed immediately below (M_x [M_seg_cnt], 0).

In 908, the word character string need only be displayed once for one word, so disp_word_flag is set to OFF.

In 909, in order to move the processing to the next segment, 1 is added to M_seg_cnt, and the process returns to step 903.

FIG. 13 is an example of the detailed flow of step 805 in FIG.

In 1001, the L_word_cnt-th value of the array of the segment number of each word of the voice data is substituted into the variable L_word_seg_num which indicates the number of the segment of the word of the learner who is currently processing, and this word is uttered. 0 for the variable L_word_dur, which indicates the time required for
Initialize to.

At 1002, i is compared with L_word_seg_num. If i is smaller than L_word_seg_num, step 1
The process proceeds to step 003, and if i is greater than or equal to L_word_seg_num, the process proceeds to step 1005.

At 1003, i of the word currently being processed is displayed.
Add the time of the th segment to L_word_dur. That is, the (L_seg_cnt + i) th value of the utterance time array of each segment of the learner's voice data is added to L_word_dur.

At 1004, 1 is added to i to obtain the speech time of the next segment.

At 1005, the magnification when displaying the word of the learner who is currently processing is calculated, and the processing is ended. In other words, the variable rate that indicates the magnification is (M_word_dur / L_word_dur)
Is assigned and the processing ends.

FIG. 14 is an example of the detailed flow of step 806 in FIG.

At 1101, the value of the L_word_cnt th element of the array of the number of segments of each word of the audio data is added to the variable L_seg_num which indicates up to which segment in total the segment corresponding to the word currently being processed.

At 1102, it is checked whether or not all segments corresponding to the word currently being processed have been processed. That is, the values of L_seg_cnt and L_seg_num are compared, and if L_seg_cnt is smaller than L_seg_num, step 1
If not, the process ends.

At 1103, the sample points of the line graph to be displayed are acquired. That is, in L_x [L_seg_cnt + 1],
Substitute {L_x [L_seg_cnt] + (L_seg_cnt th element of the utterance time array of the audio data segment) * rate} and set L_y
In [L_seg_cnt + 1], (L_seg_cnt th element of the average array of pitch frequencies of the audio data segment) or
(L_seg_c of the average array of sound pressures of segments of audio data
nt element). When the line graph of the intonation is displayed, the elements of the average array of the pitch frequencies of the segments are substituted, and when the line graph of the stress is displayed, the elements of the average array of the sound pressure of the segments are substituted.

At 1104, a line graph is actually displayed. That is, {(L_x [L_seg_cnt], L_y [L_seg_cnt),
Display the line segment connecting the two points (L_x [L_seg_cnt + 1], L_y [L_seg_cnt + 1])}.

At 1105, 1 is added to M_seg_cnt to move the processing to the next segment, and the process returns to step 1102.

Next, the above processing will be described using a specific example of data.

It is assumed that the practice sentence presenting section 52 presents the practice sentence list to the learner based on the teaching material file shown in FIG.

Then, the learner gives a practice sentence "h
ow do you do "is selected and the learner is male.

The model data acquisition unit 53 uses the teaching material file 5
Referring to step 5, obtain a model voice data file name "M_m1.dat" for men of "how do you do". The content of "M_m1.dat" is
It is assumed that it is the same as the example of the model data file shown in FIG.

Further, it is assumed that the output of the voice recognition means 51 by the voice input of the learner is as shown in FIG.

In the normalization and graphic display section 54,
The learner voice data and the model voice data are input, and the processing is performed as follows.

First, the X and Y axes are displayed and various variables are initialized (step 801).

Next, the number of words in the learner's voice data is substituted into L_word_num. Here, 4 is substituted (step 8011).

Here, L_word_cnt and L_word_num are compared (step 802), but now L_word_cnt is 0, L_word
Since _num is 4, the process proceeds to step 803.

At step 803, the intonation (stress) corresponding to the word of the L_word_cnt th model voice data is displayed.

That is, in step 901, the variable M_word_dur indicating the utterance time of the word in the model is initialized to 0, and the character string of the word to be processed now is displayed at the bottom of the X axis when the next line graph is displayed. Indicating disp
Turn on _word_flag.

Next, the L_word_cnt th element of the array of the number of segments of each word of the audio data is added to M_seg_num.
Here, M_seg_num = 0 and L_word_cnt = 0. Since the 0th element of the array of the segment number of each word of the voice data is 2, M_seg_num = 2 (step 90).
2).

At step 903, M_seg_cnt and M_seg_n
Compare um. Since M_seg_cnt = 0 and M_seg_num = 2 now, the process proceeds to step 904.

Here, the sample points for actually displaying the line graph are acquired. That is, the value of M_x [0] is added to M_x [1] and M_seg_ of the array of the utterance time of each segment of audio data.
The value of the cnt = 0th element is added and 14 is substituted. Further, for example, when displaying a graph of intonation, the value 114 of the 0th element of the average array of pitch frequencies of each segment of audio data is substituted into M_y [1].
Further, M_word_dur is added to the current M_word_dur by adding 14 to the M_seg_cnt = 0th value of the array of the utterance time of each segment of the model voice data. (Step 904).

Then, a line graph is actually displayed.
That is, the line segment connecting the two points {(0,0), (14,114)}, which are the values of {(L_x [0], L_y [0]), (L_x [1], L_y [1])}, It is displayed (step 905).

Next, it is checked whether or not disp_word_flag is ON (step 906). Since it is ON now, processing for displaying the word character string corresponding to the lower part of the X axis is performed.
That is, "how", which is the 0th value of the word character string array of the voice data, is displayed immediately below (x [1], 0) (step 90).
7). While processing "how", there is no need to display this character string anymore, so disp_word_flag is set to OFF (step 908).

Since the processing for the 0th segment is completed, 1 is added to M_seg_cnt to make it 1 (step 90
9) and returns to step 903. A line graph is displayed for the first segment as well (steps 905 and 906).

At step 409, disp_word_flag is set to O.
Since it is FF, the display processing of the word character string is not performed, and the process proceeds to step 909.

At this time, M_seg_cnt is 2 (step 909). As a result of the comparison in step 903,
Since M_seg_cnt has become equal to or larger than M_seg_num, the process for the 0th word of the example is finished, and the process proceeds to step 805. Here, M_word_dur = 27.

Next, the magnification for displaying the data of the segment corresponding to the L_word_cnt th word of the learner in correspondence with the model is calculated (step 805).

That is, in the variable L_word_seg_num indicating the number of segments corresponding to the learner's L_word_cnt th word, L_word_cnt = of the array of the segment number of each word of the audio data is set.
The 0th value 3 is substituted, and 0 is substituted for L_word_dur, i.

Next, i is compared with L_word_seg_num in order to check whether or not all segments corresponding to the word L_word_cnt = 0th word have been processed. I =
Since 0 and L_word_seg_num = 3, the process is step 100.
Go to 3.

Here, the current L_word_dur in L_word_dur =
0 and the (L_s
eg_cnt + i) = 24, which is the sum of the 0th value 24, is substituted (step 1003).

Further, 1 is added to i to check the speech time of the next segment (step 1004).

In this way, it is assumed that the processing is completed up to the second segment. At this time, i = 3, L_word_dur =
46, and the process proceeds to step 1005 by comparing step 1002.

At step 1005, (M_word_dur = 2
7) / (L_word_dur = 46) is substituted for rate. That is, about 0.59 is substituted for rate.

Since the scale factor for displaying the learner's voice data is obtained (step 805), the scale factor is used to graphically display the intonation (stress) corresponding to the L_word_cnt = 0-th learner's word. Is performed (step 806).

In step 1101, the L_word_cnt th element of the array of the segment number of each word of the audio data is added to L_seg_num. Here, L_seg_num = 0, L_word_cnt
= 0. Since the 0th element of the array of the number of segments of each word of the voice data is 3, L_seg_num = 3.

At step 1102, L_seg_cnt and L_seg
Compare _num. Now L_seg_cnt = 0, L_seg_num = 3
Therefore, the process proceeds to step 1103.

Here, sample points for actually displaying the line graph are acquired. That is, the value of L_x [0] is added to L_x [1] by the product of the value of the 0th element of the array of the speech time of each segment of the audio data and the scaling factor rate. Here, 0 + 24 *
Substitute 0.59 = 14.16. If an intonation graph is to be displayed, the value 104 of the 0th element of the average pitch frequency array of each segment of the audio data is assigned to L_y [1] (step 1103). Then, the line graph is actually displayed. That is, {(L_x [0],
The value of L_y [0]), (L_x [1], L_y [1])} is {(0,0), (14.1
6, 104)} is displayed (step 110)
4).

Since the display of the learner's L_seg_cnt = 0th segment data is finished, 1 is added to L_seg_cnt in order to advance the processing to the next segment (step 11).
05).

In this way, the processing is advanced and the learner's 2
L_seg_cnt = 3 after processing up to the th segment
Then, as a result of the comparison in step 1102, the learner's L_wo
rd_cnt = 0 processing of the 0th word ends, and step 807
Go to.

In step 807, 1 is added to L_word_cnt to set 1 in order to advance the processing to the next word, and step 8
Return to 02.

When the processing is advanced in this way and the processing is completed up to the third word, L_word_cnt = 4, and by the comparison in step 802, all the processing of this embodiment is ended.

As the third aspect of the present invention, the correspondence between the acoustic feature amount of the person who serves as a model and the language symbol and that of the learner are normalized and displayed as in the above embodiment. Alternatively, or in addition to the display, the comparison means may compare the contents of correspondence between the two and display an instruction to lower the intonation of a certain word in the mismatched portion. 130 of FIG.
Is an example of the instruction.

Further, the voice data as a model is not stored in the storage means in advance and can be input at any time by a native speaker or the like, and a predetermined acoustic feature amount of the voice signal is associated with a corresponding language symbol. The content may be available.

The acoustic feature amount of the present invention is not limited to intonation and stress, and may be another acoustic feature amount such as formant.

The linguistic symbol of the present invention may be any symbol related to the language, such as words, phonetic symbols, phoneme codes, and phoneme pieces.

Each means of the present invention may be realized by software using a computer or by using a dedicated hardware circuit having those functions.

[0128]

As is apparent from the above description, the first aspect of the present invention includes the correspondence display means for displaying the predetermined acoustic feature amount of the voice signal and the corresponding language symbol in association with each other. It has an advantage that the correspondence between the acoustic feature amount and the language symbol is easy to understand.

The second aspect of the present invention relates to a correspondence means for associating a predetermined acoustic feature quantity of a voice signal with a language symbol corresponding to the correspondence means, and a correspondence content obtained by the correspondence means for at least two kinds of voices. Since there is provided a comparison means for comparing, for example, when one type of voice is the voice of the native speaker and the other type of voice is the voice of the learner, by comparing them, the bad place of the learner can be accurately identified. It becomes possible to point out. Further, according to the third aspect of the present invention, associating means for associating a predetermined acoustic feature amount of a voice signal with a corresponding language symbol, and at least two types of speech, the associating content obtained by each corresponding means is normalized. Since the display means for displaying the converted sound is provided, for example, when one kind of sound is a native speaker sound and the other kind of sound is a learner's sound, the associated contents are normalized and displayed. Therefore, it is possible to accurately understand the bad places of the learner by looking at the display.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of a voice recognition device of the present invention.

FIG. 2 is voice data of a voice recognition means of the present invention.

FIG. 3 is a block diagram showing an embodiment of a voice recognition device of the first present invention.

FIG. 4 is a flowchart showing the operation of the voice recognition means of the present invention.

FIG. 5 is a flow chart for explaining the operation of the correspondence display means of the first present invention.

FIG. 6 is a flowchart for explaining the operation of the correspondence display means of the first present invention.

FIG. 7 is a block diagram showing an embodiment of a voice recognition device of the second invention.

FIG. 8 is a configuration diagram showing an example of a teaching material file of the embodiment.

FIG. 9 is a configuration diagram showing an example of a model voice data file of the embodiment.

FIG. 10 is a flowchart for explaining the operation of the embodiment.

11 is a detailed flowchart of step 803 of FIG.

12 is a detailed flowchart of step 803 in FIG.

FIG. 13 is a detailed flowchart of step 805 in FIG.

FIG. 14 is a detailed flowchart of step 806 in FIG.

FIG. 15 is a diagram showing a display screen of a voice recognition device according to a second aspect of the present invention.

FIG. 16 is a diagram showing a display screen of a voice recognition device according to a second aspect of the present invention.

[Explanation of Codes] 30 Input Means 31 Voice Recognition Means 32 Graphic Display Unit 50 Input Means 51 Voice Recognition Means 52 Practice Sentence Presentation Unit 53 Model Data Acquisition Unit 54 Normalization and Graphic Display Unit 55 Teaching Material Data 56 Model Voice Data

 ─────────────────────────────────────────────────── ─── Continued Front Page (72) Inventor Tadamichi Tagawa 1-7-12 Toranomon, Minato-ku, Tokyo Within Oki Electric Industry Co., Ltd. (72) Inventor Masaaki Kato 3-8-10 Uchiyama, Chikusa-ku, Nagoya, Aichi Prefecture Oki Techno Systems Laboratory Co., Ltd.

Claims (5)

[Claims] 1. An input unit for inputting voice, a recognition unit for performing voice recognition from the voice signal to obtain a language symbol, and a predetermined acoustic feature amount of the voice signal and the corresponding language symbol. A voice recognition device comprising: a corresponding display means for displaying while attaching. 2. An input unit for inputting a voice of an unspecified speaker, a recognition unit for performing voice recognition from the voice signal to obtain a language symbol, a predetermined acoustic feature amount of the voice signal and the corresponding acoustic feature amount. The contents associated with the language symbol are
A voice recognition device, comprising: a display unit that normalizes and displays at least two types of voices. 3. An input unit for inputting a voice of an unspecified speaker, a recognition unit for performing voice recognition from the voice signal to obtain a language symbol, a predetermined acoustic feature amount of the voice signal and the corresponding acoustic feature amount. The contents associated with the language symbol are
A voice recognition device comprising: a comparison means for comparing at least two types of voices. 4. The voice recognition device according to claim 2, wherein the two types of voices are a voice that serves as a model and a voice of a learner, and are used for educational purposes. 5. The voice recognition apparatus according to claim 4, wherein the correspondence content of the model voice is acquired in advance and stored in a storage unit.