JP6786065B2 - Voice rating device, voice rating method, teacher change information production method, and program - Google Patents

Description

The present invention relates to a voice rating device or the like that evaluates voice.

Conventionally, there have been the following pronunciation learning support devices (see, for example, Patent Document 1). This pronunciation learning support device is stored in the region-specific pronunciation information storage means that stores words and phrases for each area in association with the model pronunciation information of the words, and in the region-specific pronunciation information storage means based on the user operation. A phrase area input means for inputting any of the words and areas as a designated phrase and a designated area, a user voice input means for capturing a user voice for the designated word, and a model pronunciation corresponding to the designated phrase and the designated area. The device is characterized by comprising a user voice evaluation means for evaluating the pronunciation of the user voice incorporated in the user voice input means based on the information.

Japanese Unexamined Patent Publication No. 2008-8346

However, in the conventional device, the voice cannot be evaluated in consideration of the flow of the sounded input voice, so that the input voice cannot be evaluated appropriately.

The voice evaluation device of the first invention includes a teacher change information storage unit that stores teacher change information regarding changes in the feature amounts of two or more partial voice information constituting the teacher voice information, which is voice information to be a teacher. A reception unit that receives input voice information that is voice information having two or more partial voices, an acquisition unit that acquires input change information regarding a change in the feature amount of each of two or more partial voice information that the input voice information has, and input. It is a voice rating device including a rating unit that evaluates input voice information using change information and teacher change information and acquires a score, and an output unit that outputs a score.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the sounded input voice, so that the input voice can be appropriately evaluated.

Further, the voice rating device of the second invention is a voice rating device in which the teacher change information and the input change information are information regarding the order of the magnitude of the feature amount of the partial voice information with respect to the first invention. ..

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the sounded input voice, so that the input voice can be appropriately evaluated.

Further, the voice rating device of the third invention is information possessed by the teacher change information with respect to the second invention, and information regarding the order of the magnitude of the feature amount of at least two or more evaluation target partial voice information. If the information is the same, the acquisition unit is the information possessed by the input change information, and is the order of the magnitudes of the feature amounts of at least two evaluation target partial voice information at positions corresponding to the same information adjacent to each other? If it is determined whether or not they are adjacent to each other, it is a voice evaluation device that acquires input change information by regarding at least the size of the feature amount of the two evaluation target partial voice information as the same size. ..

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the input voice that is pronounced, so that it is possible to evaluate the input voice appropriately. In particular, since the voice can be evaluated in consideration of the flow of the input voice of the sentence, the input voice of the sentence can be appropriately evaluated.

Further, in the voice evaluation device of the fourth invention, with respect to the second invention, the acquisition unit acquires the feature amount of each of two or more partial voice information possessed by the input voice information, and the input voice information has 2 Of the above partial information, at least two or more parts to be evaluated are voices that acquire the order of the magnitude of two or more feature quantities and acquire the input change information having the rank of the magnitude of two or more feature quantities. It is a rating device.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the input voice that is pronounced, so that it is possible to evaluate the input voice appropriately. In particular, since the voice can be evaluated in consideration of the flow of the input voice of the sentence, the input voice of the sentence can be appropriately evaluated.

Further, in the voice evaluation device of the fifth invention, with respect to the second invention, the acquisition unit acquires the feature amounts of two or more partial voice information possessed by the input voice information, and the input voice information has 2 Distinguish between the evaluation target partial voice information corresponding to the largest feature amount and the other evaluation target partial voice information for two or more feature quantities of at least two or more evaluation target partial voice information among the above partial information. It is a voice rating device that acquires input change information that is information.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the input voice that is pronounced, so that it is possible to evaluate the input voice appropriately. In particular, since the voice can be evaluated in consideration of the flow of the input voice of the word in the word, the input voice of the word can be appropriately evaluated.

Further, in the voice evaluation device of the sixth invention, with respect to any one of the second to fifth inventions, the information regarding the ranking is the feature amount of two or more partial voice information of the teacher voice information or the input voice information. It is a voice rating device that is a feature quantity pattern that is information on the order of the size of.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the sounded input voice, so that the input voice can be appropriately evaluated.

Further, in the voice rating device of the seventh invention, for any one of the first to sixth inventions, the input voice information is the voice information of the sentence, and the partial voice information is the voice information of the words constituting the sentence. It is a voice rating device that is voice information.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the input voice of the pronounced sentence, so that the input voice of the sentence can be appropriately evaluated.

Further, in the voice rating device of the eighth invention, with respect to any one of the first to sixth inventions, the input voice information is the voice information of the word, and the partial voice information is the phoneme constituting the word. It is a voice rating device that is voice information.

With such a configuration, it is possible to evaluate the voice in consideration of the flow of the input voice of the pronounced word, so that the input voice of the word can be appropriately evaluated.

Further, the voice rating device of the ninth invention is a voice rating device in which the feature amount of the partial voice information is the accent strength, which is information on the accent strength, with respect to any one of the first to eighth inventions. is there.

With such a configuration, it is possible to evaluate the voice in consideration of the change in the accent intensity of the input voice pronounced, so that the input voice of the word can be appropriately evaluated.

Further, in the voice rating device of the tenth invention, for any one of the first to eighth inventions, the feature amount of the partial voice information is the voice rating which is the rhythm amount which is the information about the length of the voice information. It is a device.

With such a configuration, it is possible to evaluate the voice in consideration of the change in the rhythm amount of the input voice pronounced, so that the input voice of the word can be appropriately evaluated.

Further, in the voice rating device of the eleventh invention, for any one of the first to tenth inventions, the rating unit acquires the rank correlation coefficient between the input change information and the teacher change information as a score. It is a voice rating device.

With such a configuration, an appropriate score can be calculated.

In addition, the voice rating device of the twelfth invention is a second rating unit that evaluates the pronunciation of the input voice information for any one of the first to eleventh inventions and obtains the second score. , A calculation unit for calculating a representative score, which is a representative score, using the score acquired by the rating unit and the second score acquired by the second rating unit is further provided, and the output unit outputs the representative score. It is a voice rating device.

With such a configuration, it is possible to evaluate the sounded input voice from various angles, so that the input voice can be appropriately evaluated.

Further, the teacher change information production device of the thirteenth invention includes a reception unit that receives teacher voice information, a dividing means that divides teacher voice information into two or more partial voice information, and two or more partial voice information. A feature amount acquisition means for acquiring two or more feature amounts from each of two or more evaluation target partial voice information, a change information acquisition means for acquiring teacher change information using two or more feature amounts, and a teacher change information. It is a teacher change information production device including a storage unit that stores in a recording medium.

With such a configuration, teacher data for evaluating the voice in consideration of the flow of the sounded input voice can be automatically generated.

According to the voice evaluation device according to the present invention, it is possible to evaluate the voice in consideration of the flow of the sounded input voice, so that the input voice can be evaluated appropriately.

Block diagram of the voice rating device 1 according to the first embodiment A flowchart explaining the operation of the voice rating device 1. Flow chart explaining the change information acquisition process Diagram showing the teacher change information management table Diagram showing the same output example Block diagram of the voice rating device 2 according to the second embodiment Flow chart explaining the operation of the voice rating device 2 Block diagram of the production apparatus 3 according to the third embodiment Flow chart explaining the operation of the production apparatus 3 Overview of the computer system according to the above embodiment Block diagram of the computer system

Hereinafter, embodiments of the voice rating device and the like will be described with reference to the drawings. In addition, since the components having the same reference numerals perform the same operation in the embodiment, the description may be omitted again.

(Embodiment 1)
In the present embodiment, input voice information having two or more partial voice information is received, input change information relating to a change in the feature amount of the partial voice information is acquired, and the input change information and the teacher change information of the teacher voice are used. The voice rating device for rating the input voice will be described.

The input change information and the teacher change information (hereinafter, may be collectively referred to as "change information") are, for example, information related to the ranking among two or more partial voice information. Further, the information regarding the ranking is, for example, a feature quantity pattern described later. The input voice information is, for example, a sentence, a word, or the like. The feature amount is, for example, the accent intensity and the rhythm amount described later. When the feature amount is the accent intensity, the voice rating device 1 is a device for performing accent rating. When the feature amount is a rhythm amount, the voice rating device 1 is a device that performs rhythm rating.

FIG. 1 is a block diagram of the voice rating device 1 according to the present embodiment.

The voice evaluation device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14.

The storage unit 11 includes a teacher change information storage unit 111. The processing unit 13 includes an acquisition unit 131 and a rating unit 132. The acquisition unit 131 includes a division unit 1311, a feature amount acquisition unit 1312, and a change information acquisition unit 1313.

The storage unit 11 can store various types of information. The various types of information include, for example, teacher change information described later, input voice information described later, teacher voice information described later, and the like.

The teacher change information storage unit 111 stores one or more teacher change information. The teacher change information is information related to a change in the feature amount of each of two or more partial voice information constituting the teacher voice information. The partial voice information is, for example, a phoneme, a word, or the like. The two or more teacher change information is, for example, change information of a sentence of one teacher voice information and change information of two or more words constituting the sentence. Further, the two or more teacher change information is, for example, change information of two or more teacher voice information. Further, the two or more teacher change information is, for example, two or more teacher voice information sentence change information and two or more word change information constituting the sentence.

The teacher voice information is voice information that serves as a teacher. Teacher audio information is usually word or sentence audio information. A sentence can be called a sentence. The teacher change information is, for example, information regarding the rank of the magnitude of the feature amount of two or more partial voice information. The information regarding the rank of the feature amount of the partial voice information is, for example, a feature amount pattern which is a sequence of information regarding the rank of the feature amount of each of two or more partial voice information. The feature quantity pattern is, for example, an accent intensity pattern and a rhythm quantity pattern. The accent strength pattern may be called an accent pattern. The accent pattern is information on the order of the magnitude of the accent intensity of the partial voice information. The accent pattern is information representing the magnitude relationship of the accent intensity of a word or phoneme in speech information, and is, for example, information obtained by patterning the accent intensity of a word or phoneme with an integer value. The accent strength is information on the strength of the accent. The accent intensity includes, for example, the accent intensity for each phoneme and the accent intensity for each word. A technique for calculating the accent intensity for each phoneme is shown in, for example, Japanese Patent No. 4716116, and is a known technique. The accent intensity for each phoneme is a representative value of the accent intensity for each frame in the phoneme section (usually, it is the maximum value, and may be an average value, a median value, or the like). The phoneme to be calculated is usually a vowel. That is, phonemes other than vowels are excluded from the calculation (rating), and for example, the zero value is set as the accent intensity. Further, the accent intensity for each word is calculated as a representative value of the accent intensity for each phoneme in the word for each word (usually, the maximum value, which may be an average value or a median value). Further, for each word, a representative value of the accent intensity for each frame in the word (usually, the maximum value, which may be an average value or a median value) may be calculated. For words that are not subject to evaluation (basically only silent sections or words without vowels), for example, a zero value is set as the accent intensity.

The rhythm amount is information related to the length of voice information. The information regarding the length of the voice information is information regarding the length of the partial voice information. The rhythm amount pattern is information on the order of the magnitude of the rhythm amount of the partial voice information. If the feature pattern is a rhythm pattern, a rhythm rating is possible. Rhythm evaluation evaluates whether the length of utterance of a word or phoneme is correct (similar to native utterance). The length of vocalization of words and phonemes can be obtained using forced alignment. The length is called the rhythm amount. A rhythm pattern is generated from the rhythm amount of the teacher's voice. The similarity between the rhythm amount pattern obtained from the teacher voice information and the rhythm amount of the input voice information is calculated using the rank correlation coefficient, and the rhythm rating score is obtained.

Further, the teacher change information may be, for example, a feature quantity tendency. The feature amount tendency is information indicating whether the feature amount of each of two or more partial voice information tends to increase or decrease. As for the feature amount tendency, information may be obtained in which the feature amount of each of the two or more partial voice information has an increasing tendency, a decreasing tendency, or the same information. The feature amount tendency is, for example, an accent tendency and a rhythm amount tendency. The accent tendency is information indicating whether the accent intensity of each of two or more partial voice information tends to increase or decrease. As for the accent tendency, information may be obtained in which the accent intensity of each of the two or more partial voice information is either an increasing tendency, a decreasing tendency, or the same. The rhythm amount tendency is information indicating whether the rhythm amount of each of two or more partial voice information tends to increase or decrease. The two or more partial voice information for which the feature amount is acquired may be two or more partial voice information to be evaluated, excluding the partial voice information not to be evaluated. Further, the two or more partial voice information for which the feature amount is acquired may be all partial voice information. The partial voice information that is not subject to evaluation is, for example, information in a silent section or a section without vowels.

Usually, the data structure of the teacher change information and the data structure of the input change information described later are the same. The teacher change information of the teacher change information storage unit 111 may correspond to, for example, an identifier. Further, the teacher change information may correspond to, for example, teacher voice information. The teacher voice information is voice information that serves as a teacher. The voice information that serves as a teacher is a model voice information.

It is preferable that the teacher change information such as the feature quantity pattern stored in the teacher change information storage unit 111 is information generated from the teacher voice information. Such a generation process is performed by, for example, the teacher change information production apparatus described in the third embodiment. However, the teacher change information of the teacher change information storage unit 111 may be manually created based on the scientific knowledge of phonetics and linguistics. In addition, teacher change information such as feature pattern can be basically generated from teacher voice information, but according to the points to be evaluated (what to focus on) of sentences and words to be evaluated, It may be adjusted manually.

The reception unit 12 receives voice information which is voice information having two or more partial voices. Such voice information is, for example, input voice information or teacher voice information. The audio information is usually the audio information of a word or sentence. Here, acceptance means acceptance of information input from an input device such as a microphone, reception of information transmitted via a wired or wireless communication line, and reading from a recording medium such as an optical disk, a magnetic disk, or a semiconductor memory. It is a concept that includes the acceptance of information that has been received.

The processing unit 13 performs various processes. The various processes are, for example, processes performed by the acquisition unit 131, the rating unit 132, and the like.

The acquisition unit 131 acquires change information regarding a change in the feature amount of each of two or more partial voice information possessed by the voice information. The acquisition unit 131 acquires the input change information regarding the change in the feature amount of each of the two or more partial voice information possessed by the input voice information. Further, as described in the third embodiment, the acquisition unit 131 may acquire the teacher change information regarding the change in the feature amount of each of two or more partial voice information possessed by the teacher voice information. In the present embodiment, the acquisition unit 131 will be described as acquiring the input change information, but in the third embodiment, the acquisition unit 131 acquires the teacher change information, and the operation is the same.

Further, here, when the input voice information is a sentence, the partial voice information is, for example, a word. However, when the input voice information is a sentence, the partial voice information may be a phoneme. When the input voice information is a word, the partial voice information is, for example, a phoneme. The feature amount is, for example, an accent intensity or a rhythm amount.

Further, the input change information includes, for example, an accent pattern of the input voice information, an accent tendency of the input voice information, and the like.

The acquisition unit 131 acquires, for example, the feature amount of each of two or more partial voice information possessed by the voice information, and at least two or more of the two or more partial information possessed by the voice information and two or more of the evaluation target partial voice information. The rank of the magnitude of the feature amount is acquired, and the change information having the rank of the magnitude of two or more feature quantities is acquired. Such a method is called the first change information acquisition method. The two or more evaluation target partial voice information is the partial information of the two or more partial information possessed by the voice information, excluding the partial voice information not subject to the evaluation. Further, "acquiring the rank of the magnitude of the feature amount of at least two or more evaluation target partial voice information" may be obtained by acquiring the rank of the feature quantity of only the rating target partial voice information. However, it is also possible to acquire the rank of the magnitude of all the features of the two or more partial information possessed by the voice information.

When the information is the information possessed by the teacher change information and the information regarding the rank of the feature amount of the two or more evaluation target partial voice information is the same information, the acquisition unit 131 is, for example, the information possessed by the input change information. , It is judged whether or not the ranks of the features of the two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other, and if it is determined that they are adjacent to each other, the two evaluations are made. Input change information is acquired by regarding the magnitude of the feature amount of the target partial voice information as the same magnitude. Such a method is called a special method.

For example, the word transcription of the input voice information is "/ sil/alice/sil/looked/sil/up/sil/", and the teacher word accent pattern (teacher change information) corresponding to the input voice information is "/". The case of "0/2/0/1/0/1/0 /" will be explained. It is assumed that the sequence of input word accent intensities in such a case is "/0.000000/60.396744/0.000000/53.130833/0.000000/48.609158/0.000000/". In that case, for example, the following operations 1) to 5) are performed.
1) The acquisition unit 131 excludes zero values that are not subject to evaluation from the teacher word accent pattern "/ 0/2/0/1/0/1/0 /". Then, the teacher word accent pattern "2 1 1" is acquired.
2) The acquisition unit 131 excludes zero values that are not subject to evaluation from the input word accent intensity. Then, the input word accent intensity "60.396744 53.130833 48.609158" is acquired.
3) The acquisition unit 131 obtains the ranking data from the teacher word accent pattern. At this time, it is the information possessed by the teacher change information, the information regarding the rank of the feature amount of the two or more evaluation target partial voice information is the same information, and the adjacent information (the second and third places are the same). Since it is a value), the acquisition unit 131 gives the average of the ranks (2 + 3) / 2 = 2.5 as an intermediate rank. Then, the acquisition unit 131 obtains the teacher word accent rank “1 2.5 2.5”.
4) When there is a tie (equal rank) in the teacher word accent rank, if the ranks of the input word accent strength corresponding to the position of the tie data are adjacent to each other, the acquisition unit 131 determines the input word accent strength. Convert to tie data according to the large intensity value. That is, the acquisition unit 131 acquires the input word accent intensity “60.396744 53.130833 53.130833”.
5) The acquisition unit 131 obtains the ranking data from the input word accent intensity. Here, since the tie (equal rank) is included, the acquisition unit 131 gives an intermediate rank. That is, the acquisition unit 131 obtains the input word accent order “1 2.5 2.5”.

For example, the acquisition unit 131 acquires the feature amount of each of the two or more partial voice information possessed by the input voice information, and excludes the partial voice information not subject to evaluation from the two or more partial information possessed by the input voice information. For two or more feature quantities of two or more evaluation target partial voice information, input change information which is information for distinguishing the evaluation target partial voice information corresponding to the largest feature quantity from other evaluation target partial voice information is acquired. To do. Such a method is called a second change information acquisition method. The partial voice information is, for example, voice information of words constituting a sentence. Further, the partial voice information is, for example, voice information of phonemes constituting a word.

The dividing means 1311 divides the input voice information into two or more partial voice information. The dividing means 1311 calculates, for example, the section information for each phoneme of the input voice information by using the forced alignment. The section information is information indicating a section in which the phoneme is in the input voice information. The section information is, for example, information indicating the number of milliseconds to the number of milliseconds of the input voice information. For example, the dividing means 1311 aligns the input voice information received by the reception unit 12 with the stored teacher voice information. The teacher voice information is information stored in the storage unit 11, and is the same phoneme data as the input voice information. Here, the alignment is usually a force alignment (also referred to as a force alignment). The force alignment is a process of forcibly associating the phoneme of the voice information with the phoneme of the teacher data, and since it is a known technique, detailed description thereof will be omitted. Further, the dividing means 1311 may divide the input voice information into two or more words. The algorithm in which the dividing means 1311 divides the input voice information into two or more partial voice information does not matter.

The feature amount acquisition means 1312 acquires the feature amount from each of two or more partial voice information divided by the dividing means 1311. The feature amount acquisition means 1312 calculates, for example, the accent intensity for each frame of the partial voice information. Then, the feature amount acquisition means 1312 calculates, for example, the accent intensity for each phoneme of the partial voice information from the section information for each phoneme and the accent intensity for each frame. The feature amount acquisition means 1312 acquires, for example, representative values of a plurality of accent intensities of a plurality of frames in one phoneme as the accent intensities of the phonemes. The representative value is, for example, a maximum value, an average value, a median value, or the like. The phoneme for which the feature amount such as the accent intensity is calculated is usually a vowel. It is preferable that the feature amount acquisition means 1312 sets the feature amount of phonemes other than vowels to zero (0). Further, the feature amount acquisition means 1312 calculates, for example, the accent intensity of each word of the input voice information from the accent intensity of each phoneme in each word. The statistic acquisition means 1312 acquires, for example, a representative value of a plurality of accent intensities of a plurality of phonemes possessed by one word as a word accent intensity. The representative value is described above. The metering acquisition means 1312 may acquire, for example, a representative value of the accent intensity of a plurality of frames possessed by one word as the accent intensity of the word.

The process of calculating the accent strength for each frame is described in Japanese Patent No. 4716116 and the like, and is a known technique. Therefore, detailed description thereof will be omitted.

The change information acquisition means 1313 acquires input change information relating to a change in the feature amount of each of the two or more partial voice information acquired by the feature amount acquisition means 1312.

The change information acquisition means 1313 acquires, for example, the rank of the feature amount of each of the two or more partial voice information acquired by the feature amount acquisition means 1312, and the input change having the rank of the magnitude of the feature amount of two or more. Get information. Such a method is the first change information acquisition method.

In the first change information acquisition method, for example, the word transcription of the input voice information of a sentence is "/ sil / alice / sil / looked / sil / up / sil /", and the word accent intensity of each word is arranged. When is "/0.000000/60.396744/0.000000/53.130833/0.000000/48.609158/0.000000/", the change information acquisition means 1313 is, for example, the word accent pattern "/ 0/3/0/2/0/1/0 /". To get. That is, since the word accent strength / 60.396744 / of the word / alice / is the largest accent strength in the change information acquisition means 1313, the maximum number "3" is given as the rank information for the word / alice /. The maximum number is the number of words to be evaluated (the number of partial voice information to be evaluated). Further, in the change information acquisition means 1313, since the word accent strength / 53.130833 / of the word / looked / is the second largest accent strength, "2" is added as the rank information with respect to the word / looked /. Further, in the change information acquisition means 1313, since the word accent strength / 48.609158 / of the word / up / is the third largest accent strength, "1" is added as the rank information with respect to the word / up /. Further, the change information acquisition means 1313 gives rank information "0" to the silent section having an accent intensity of /0.000000/. As described above, the change information acquisition means 1313 acquires the word accent pattern "/ 0/3/0/2/0/1/0 /". Here, the word transcription expresses a speech sequence of a word when the speech of a sentence is divided into the speech of the word. The word accent strength is an example of a word feature amount, and is a word accent strength. Further, the word accent pattern is an example of input change information, and is information related to the order of the accent strength of words. The slash "/" is a word delimiter. Since "sil" is a symbol representing silence and is not subject to evaluation, the accent intensity is usually set to zero. Also, silence generally exists between words, but it does not always exist. The basic rules for patterning here are, for example, the following 1) and 2).
1) Give a large integer pattern to a large accent intensity.
2) Give zero to words and phonemes that are not subject to rating.

By patterning the accent strength as described above, information showing only the magnitude relation (strength relation) with respect to the accent of a word or phoneme can be obtained. In the accent rating, the information about what the accent intensity value of a word or phoneme is is not important. In a sentence (or word), the magnitude relationship of which word (or phoneme) has a large accent intensity and which word (or phoneme) has a low accent intensity is important, and the word (or phoneme) accent intensity of the teacher's voice ( The purpose of accent evaluation is to see the degree of similarity with the magnitude relationship between phonemes). In other words, if there is accent pattern data that serves as an accent teacher (even if there is no accent intensity data), accent rating can be achieved with sufficient accuracy.

Also, rather than comparing the accent intensities of the teacher's voice information and the input voice information, by introducing an accent pattern, it is possible to make adjustments as described below, and how to configure the teacher's accent. In other words, the degree of freedom in deciding from what point of view the accent of the input voice information is evaluated is increased.

For example, in the special method of the accent pattern generation method described later, the phoneme accent pattern (excluding the zero value that is not subject to evaluation) is / 2 1 1 /. This is a correct answer if the phoneme with the maximum intensity is the first, and it is a teacher pattern that the difference in the intensity of the second and third is irrelevant to the quality of the accent. However, looking at the phoneme accent intensity of the teacher's voice, there is little difference between the first and second values, and the third value is far apart. This indicates that the difference between the first and second intensities is irrelevant whether it is good or bad, and the point that we want to see is that the third is a small intensity. At this time, if the teacher accent pattern is 2 2 1 /, you can see the point and it will be reflected in the grade score. As described above, by introducing simple integer value data called a pattern into the accent information of the teacher, it becomes possible to flexibly adjust the points to be evaluated.

Further, in the first change information acquisition method, for example, the phoneme transcription of the word "understand" is "/ sil / ah nd er st ae nd / sil /", and the phoneme accent intensity of the phonemes constituting the word is When the sequence is "/0.000000/62.717609 0.000000 0.000000 62.379860 0.000000 0.000000 51.971569 0.000000 0.000000 / 0.000000 /", the change information acquisition means 1313 is, for example, the phoneme accent pattern "/ 0/3 0 0 2 0 0 1 0 0/0". / ”Is acquired. That is, the change information acquisition means 1313 gives the maximum number "3" as the rank information with respect to the phoneme "ah" because the phoneme accent strength "62.717609" of the phoneme "ah" is the largest accent strength. The maximum number is the number of phonemes to be evaluated (the number of partial voice information to be evaluated). Further, in the change information acquisition means 1313, since the phoneme accent intensity "62.379860" of the phoneme "er" is the second largest accent intensity, "2" is added as the rank information with respect to the phoneme "er". Further, in the change information acquisition means 1313, since the phoneme accent intensity “51.971569” of the phoneme “ae” is the third largest accent intensity, “1” is added as ranking information for the word / up /. Further, the change information acquisition means 1313 adds rank information "0" to a silent section or a consonant having an accent intensity of /0.000000/. As described above, the change information acquisition means 1313 acquires the phoneme accent pattern "/ 0/3 0 0 2 0 0 1 0 0/0 /". Here, the phoneme transcription expresses a phoneme sequence when the speech of a word sentence is divided into phonemes. Further, the phoneme accent pattern is an example of input change information, and is information related to the order of phoneme accent intensities.

The change information acquisition means 1313 determines, for example, the information possessed by the teacher change information corresponding to the input voice information, and the information regarding the rank of the feature amount of the two or more evaluation target partial voice information is the same information. However, if it is determined that the information is possessed by the input change information and the ranks of the features of the two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other, the two Input change information is acquired by regarding the size of the feature amount of the evaluation target partial voice information as the same size. Such a method is a special method in the change information acquisition method.

The change information acquisition means 1313 distinguishes between the evaluation target partial voice information corresponding to the largest feature amount and the other evaluation target partial voice information with respect to two or more feature quantities of two or more evaluation target partial voice information, for example. Acquires input change information, which is information to be input. In such a case, the value for the evaluation target partial voice information corresponding to the largest feature amount is set to "2", the value for the other evaluation target partial voice information is set to "1", and the numerical string in the order of the partial voice information is input and changed. Get as information. In addition, such a method is the second change information acquisition method.

In the second change information acquisition method, for example, the word transcription of the input voice information of a sentence is "/ sil / alice / sil / looked / sil / up / sil /", and the word accent intensity of each word is arranged. When is "/0.000000/60.396744/0.000000/53.130833/0.000000/48.609158/0.000000/", the change information acquisition means 1313 is, for example, the word accent pattern "/ 0/2/0/1/0/1/0 /". To get. That is, since the word accent strength / 60.396744 / of the word / alice / is the largest accent strength, the change information acquisition means 1313 gives the maximum number "2" as the rank information for the word / alice /. Further, in the change information acquisition means 1313, since the word accent intensity / 53.130833 / of the word / looked / is the second largest accent intensity, "1" is added as the rank information with respect to the word / looked /. Further, in the change information acquisition means 1313, since the word accent strength / 48.609158 / of the word / up / is the second and subsequent large accent strength, "1" is added as the rank information with respect to the word / up /. Further, the change information acquisition means 1313 gives rank information "0" to the silent section having an accent intensity of /0.000000/. As described above, the change information acquisition means 1313 acquires the word accent pattern "/ 0/2/0/1/0/1/0 /".

Further, in the second change information acquisition method, for example, the phoneme transcription of the word "understand" is "/ sil / ah nd er st ae nd / sil /", and the phoneme accent intensity of the phonemes constituting the word is When the sequence is "/0.000000/62.717609 0.000000 0.000000 62.379860 0.000000 0.000000 51.971569 0.000000 0.000000 / 0.000000 /", the change information acquisition means 1313 is, for example, the phoneme accent pattern "/ 0/2 0 0 1 0 0 1 0 0/0". / ”Is acquired. That is, since the phoneme accent intensity "62.717609" of the phoneme "ah" is the largest accent intensity, the change information acquisition means 1313 gives the maximum number "2" as the rank information with respect to the phoneme "ah". Further, in the change information acquisition means 1313, since the phoneme accent intensity “62.379860” of the phoneme “er” is the second largest accent intensity, “1” is added as the rank information with respect to the phoneme “er”. Further, in the change information acquisition means 1313, since the phoneme accent intensity "51.971569" of the phoneme "ae" is the second and subsequent largest accent intensity, "1" is added as the rank information for the word / up /. Further, the change information acquisition means 1313 adds rank information "0" to a silent section or a consonant having an accent intensity of /0.000000/. As described above, the change information acquisition means 1313 acquires the phoneme accent pattern "/ 0/2 0 0 1 0 0 1 0 0/0 /".

The first method of acquiring change information is suitable for evaluating sentences. In the sentence task, words with strong accents to words with weak accents are evaluated as good pronunciation when they are uttered with the correct accent. In addition, it is necessary to look at the magnitude (strength) relationship of all word accent intensities. You can see it if you rate it with an accent pattern according to the accent intensity of the teacher's voice as in the first change information acquisition method. In addition, the second change information acquisition method is suitable for word evaluation. In the word task, it is evaluated whether or not the phoneme that should have the first (or up to several) accent intensity (maximum intensity) has the correct accent (maximum intensity). If only the phoneme with the first accent intensity (maximum intensity) has a large accent pattern value as in the second change information acquisition method, and the other phonemes have a flat accent pattern value, it is evaluated. be able to.

The rating unit 132 evaluates the input voice information using the input change information and the teacher change information, and acquires a score. The rating unit 132 usually acquires information on the difference between the input change information and the teacher change information (which may be called information on the degree of similarity) as a score. For example, the rating unit 132 acquires the rank correlation coefficient between the input change information and the teacher change information as a score. The rank correlation coefficient is, for example, Spearman's rank correlation coefficient. Since Spearman's rank correlation coefficient is a known technique, detailed description thereof will be omitted.

An example of the process in which the rating unit 132 calculates the score using Spearman's rank correlation coefficient is as follows. For example, the teacher change information (teacher word accent rank) is x = {x ₁ , x ₂ , ... , x _N }, and there are n _x ties (equal rank) in _x, and the i-th spot. Let t _i (i = 1, 2, ... , n _x ) be the number of ties, and y = {y ₁ , y ₂ , ... , y _N } for the input change information (input word accent order). If there is a tie (equal rank) in _y and the number of ties in the jth place is t _j (j = 1, 2, ... , n _y ), the rating unit 132 is a Spearman. The rank correlation coefficient of is calculated by Equation 1. The accent rating score is calculated from the rank correlation coefficient by Equation 4. In this case, the rating score is 1.0, that is, a perfect score.

The rank correlation coefficient may be a rank correlation coefficient other than Spearman's rank correlation coefficient (for example, Kendall's rank correlation coefficient). The rank correlation coefficient is in the range of -1 to 1. Then, the score acquired by the rating unit 132 is, for example, a positive rank correlation coefficient (0.0 to 1.0), and 0.0 when it is 0 or less.

The rating unit 132 calculates a sentence score from, for example, the word accent pattern of the teacher voice information and the word accent intensity of the input voice information. It can be said that this score is the accent score of sentences. Similarly, the rating unit 132 divides, for example, the phoneme accent pattern and the phoneme accent intensity for each word, and calculates the score for each word. It can be said that this score is a word accent score.

The output unit 14 outputs the score acquired by the rating unit 132. Here, the output means display on a display, projection using a projector, printing by a printer, sound output, transmission to an external device, storage on a recording medium, storage on another processing device, another program, or the like. It is a concept that includes delivery of processing results.

A non-volatile recording medium is suitable for the storage unit 11 and the teacher change information storage unit 111, but a volatile recording medium can also be used.

The process of storing information in the storage unit 11 or the like does not matter. For example, the information may be stored in the storage unit 11 or the like via the recording medium, or the information transmitted via the communication line or the like may be stored in the storage unit 11 or the like. Alternatively, the information input via the input device may be stored in the storage unit 11 or the like.

The processing unit 13, the acquisition unit 131, the rating unit 132, the division means 1311, the feature amount acquisition unit 1312, and the change information acquisition unit 1313 can usually be realized from an MPU, a memory, or the like. The processing procedure of the processing unit 13 is usually realized by software, and the software is recorded on a recording medium such as ROM. However, it may be realized by hardware (dedicated circuit).

The output unit 14 may or may not include an output device such as a display or a speaker. The output unit 14 can be realized by the driver software of the output device, the driver software of the output device, the output device, or the like.

Next, the operation of the voice rating device 1 will be described with reference to the flowchart of FIG.

(Step S201) The reception unit 12 determines whether or not the input voice information has been received. If the input voice information is accepted, the process proceeds to step S202, and if the input voice information is not received, the process returns to step S201.

(Step S202) The acquisition unit 131 acquires change information of the voice information received in step S201. The change information acquisition process will be described with reference to the flowchart of FIG. Here, the input change information of the input voice information received in step S201 is acquired. Further, for example, the acquisition unit 131 acquires the input change information of the sentence and the input change information of two or more words constituting the input voice information with respect to the input voice information of the sentence.

(Step S203) The rating unit 132 is change information corresponding to the input voice information received in step S201, and acquires the teacher change information of the sentence from the teacher change information storage unit 111.

(Step S204) The rating unit 132 acquires a score by using the input change information of the sentence acquired in step S202 and the teacher change information of the sentence acquired in step S203.

(Step S205) The output unit 14 outputs the score acquired in step S204. This score is the total score of the input voice information of the sentence.

(Step S206) The rating unit 132 substitutes 1 for the counter i.

(Step S207) The rating unit 132 determines whether or not the voice information of the i-th word exists in the input voice information received in step S201. If the voice information of the i-th word exists, the process proceeds to step S208, and if the voice information of the i-th word does not exist, the process returns to step S201. It should be noted that whether or not the voice information of the i-th word exists has the same meaning as whether or not the i-th word exists in the sentence corresponding to the input voice information. That is, in the loop from step S206 to step S212, it suffices if the process of outputting the score for each word can be performed, and the information used for determining whether or not the i-th word exists does not matter.

(Step S208) The rating unit 132 acquires the input change information of the i-th word among the input change information acquired in step S202.

(Step S209) The rating unit 132 acquires the teacher change information of the i-th word from the teacher change information storage unit 111.

(Step S210) The rating unit 132 acquires a score by using the input change information of the i-th word acquired in step S208 and the teacher change information of the i-th word acquired in step S209. This score is the voice score of the i-th word in the input voice information.

(Step S211) The output unit 14 outputs the score acquired in step S210. This score is the score of the i-th word of the input voice information.

(Step S212) The rating unit 132 increments the counter i by 1. Return to step S207.

In the flowchart of FIG. 2, the rating unit 132 may calculate the representative score by using the score of the sentence of the input voice information and the score of two or more words. Then, the output unit 14 may output this representative score. The representative score is usually an increasing function with the score of a sentence and the score of two or more words as parameters. The representative score is, for example, the average value, the median value, the maximum value, etc. of the score of the sentence and the score of two or more words.

Further, in the flowchart of FIG. 2, the processing is terminated by an interrupt of power off or processing termination.

Next, an example of the input change information acquisition process in step S202 will be described with reference to the flowchart of FIG.

(Step S301) The dividing means 1311 of the acquisition unit 131 divides the input voice information into two or more phonemes. Usually, the dividing means 1311 acquires the section information for each phoneme from the input voice information.

(Step S302) The feature amount acquisition means 1312 of the acquisition unit 131 substitutes 1 for the counter i.

(Step S303) The feature amount acquisition means 1312 determines whether or not the i-th phoneme exists among the two or more phonemes divided in step S301. If the i-th phoneme is present, the process goes to step S304, and if the i-th phoneme is not present, the process goes to step S310.

(Step S304) The feature amount acquisition means 1312 substitutes 1 for the counter j.

(Step S305) The feature amount acquisition means 1312 determines whether or not the j-th frame exists in the i-th phoneme. If the j-th frame exists, the process goes to step S306, and if the j-th frame does not exist, the process goes to step S308.

(Step S306) The feature amount acquisition means 1312 acquires the feature amount of the j-th frame. The feature amount is, for example, the accent intensity.

(Step S307) The feature amount acquisition means 1312 increments the counter j by 1. Return to step S305.

(Step S308) The feature amount acquisition means 1312 acquires the representative feature amount of the j-th phoneme from the feature amounts of two or more phonemes acquired in step S306.

(Step S309) The feature amount acquisition means 1312 increments the counter i by 1. Return to step S303.

(Step S310) The change information acquisition means 1313 of the acquisition unit 131 substitutes 1 for the counter k.

(Step S311) The change information acquisition means 1313 determines whether or not the kth word exists. If the k-th word exists, the process goes to step S312, and if the k-th word does not exist, the process goes to step S316.

(Step S312) The change information acquisition means 1313 acquires representative feature quantities of two or more phonemes in the k-th word in the order of phoneme arrangement.

(Step S313) The change information acquisition means 1313 acquires the change information of the k-th word by using the representative features of two or more phonemes acquired in step S312.

(Step S314) The feature amount acquisition means 1312 acquires the representative feature amount of the k-th word by using the representative feature amounts of two or more phonemes acquired in step S312. The representative feature of the k-th word is usually a feature that represents a representative feature of two or more phonemes.

(Step S315) The change information acquisition unit 1313 increments the counter k by 1. Return to step S311.

(Step S316) The change information acquisition means 1313 acquires change information of a sentence by using the representative feature amounts of two or more words acquired in step S314. Return to higher processing. The text change information is input change information of voice information which is a text.

Hereinafter, the specific operation of the voice rating device 1 in the present embodiment will be described.

(Specific example 1)
Now, it is assumed that the teacher change information storage unit 111 stores the teacher change information management table shown in FIG. The teacher change information management table contains the teacher change information of the entire teacher voice information (sentence) of the sentence "Alice looked up.", And the words "Alice", "looked", and each word constituting the sentence "Alice looked up." The teacher change information corresponding to "up" is stored.

In such a situation, it is assumed that the user reads out the English sentence "Alice looked up." To the voice rating device 1. Then, the reception unit 12 of the voice rating device 1 receives the input voice information, which is the voice information of the sentence “Alice looked up.”.

Next, the dividing means 1311 divides the input voice information into two or more words. That is, the dividing means 1311 acquires the section information for each phoneme constituting the input voice information by using forced alignment or the like.

Next, the feature amount acquisition means 1312 acquires the feature amount of each of two or more frames possessed by each phoneme in the order in which the phonemes are arranged for each phoneme. Here, it is assumed that the feature amount is, for example, the accent intensity. Then, the feature amount acquisition means 1312 acquires a representative feature amount (for example, the maximum value) from the feature amounts of two or more frames for each phoneme. And this representative feature is the feature of each phoneme.

Next, the change information acquisition means 1313 acquires the feature amounts (representative feature amounts) of two or more phonemes in the word for each word in the order of phoneme arrangement. That is, first, the change information acquisition means 1313 has a sequence of features (phoneme accent intensity) "/ 55.148270 0.000000 60.396744 0.000000 /" for the phoneme transcription "/ ae l ax s /" corresponding to the word "Alice". "Is obtained. Then, the change information acquisition means 1313 obtains the input change information “/1 0 2 0 /” of the word “Alice” from the arrangement of the phoneme accent intensities. Here, the change information acquisition means 1313 acquired the input change information by the second change information acquisition method.

Similarly, the change information acquisition means 1313 obtains a sequence of features (phoneme accent intensities) "0.000000 53.130833 0.000000 0.000000" for the phoneme transcription "l uh k t" corresponding to the word "looked". Then, the change information acquisition means 1313 acquires the input change information "0 1 0 0" of the word "looked" from the arrangement of the phoneme accent intensities.

Similarly, the change information acquisition means 1313 obtains a sequence of features (phoneme accent intensities) "48.609158 0.000000" for the phoneme transcription "ah p" corresponding to the word "up". Then, the change information acquisition means 1313 acquires the input change information "10" of the word "up" from the arrangement of the phoneme accent intensities.

Next, the change information acquisition means 1313 acquires the input change information of the sentence by using the representative feature amounts of the acquired two or more words. That is, the change information acquisition means 1313 has a word accent intensity “/0.000000/60.396744” which is a sequence of feature quantities of each word constituting the word transcription “/ sil/alice/sil/looked/sil/up/sil/”. From "/0.000000/53.130833/0.000000/48.609158/0.000000/", the word accent pattern "/ 0/3/0/2/0/1/0 /" is acquired by the first change information acquisition method. This word accent pattern is an example of input change information.

Next, the rating unit 132 is change information corresponding to the received input voice information (voice information corresponding to "Alice looked up."), And is text teacher change information "/ 0/3/0/2 /". 0/1/0 / ”is acquired from the teacher change information management table (Fig. 4).

Next, the rating unit 132 receives the input change information "/ 0/3/0/2/0/1/0 /" of the acquired sentence "Alice looked up." And the teacher change information "/" of the acquired sentence. The score, which is information on the degree of similarity with "0/3/0/2/0/1/0 /", is acquired using Spearman's rank correlation coefficient. Here, the input change information "/ 0/3/0/2/0/1/0 /" and the teacher change information "/ 0/3/0/2/0/1/0 /" are the same. , The rating unit 132 acquires the score "1". Next, the rating unit 132 multiplies the acquired score "1" by 100 to calculate the score "100" to be output.

Next, the output unit 14 outputs the score "100" acquired by the rating unit 132. An example of such an output is shown in FIG. In FIG. 5, the score is displayed as a rating score 501.

Next, the rating unit 132 evaluates each word. That is, the rating unit 132 acquires the phoneme accent intensity sequence “/1 0 2 0 /” which is the input change information of the first word “Alice”. Next, the rating unit 132 acquires the teacher change information “/1 0 2 0 /” paired with “Alice” from the teacher change information management table (FIG. 4). Then, the rating unit 132 uses the input change information “/1 0 2 0 /” of the first word and the teacher change information “/1 0 2 0 /” of the acquired first word to obtain a score “1 0 2 0 /”. 1 ”is acquired. Then, the rating unit 132 multiplies the score "1" by 100 to obtain a rating score "100" for the word "Alice". Then, the output unit 14 outputs the rating score “100” of the word “Alice”.

The above processing is also performed for the words "looked" and "up", and the rating score "100" for the words "looked" and "up" is also output. The output mode of the rating score does not matter.

As described above, according to the present embodiment, the voice can be evaluated in consideration of the flow of the sounded input voice, so that the input voice can be evaluated appropriately.

According to the present embodiment, the teacher change information was prepared in advance. However, the teacher change information may also be dynamically generated from the teacher voice information. For such generation, for example, the production apparatus 3 described in the third embodiment is used. Specific examples of the processing in such a case are the following 1) to 16).
1) Calculate the section information for each phoneme of the teacher's voice information using forced alignment.
2) The accent intensity for each frame of the teacher voice information is calculated from the teacher voice information.
3) The accent intensity for each phoneme of the teacher voice information is calculated from the section information for each phoneme and the accent intensity for each frame.
4) The accent intensity for each word of the teacher's voice information is calculated from the accent intensity for each phoneme in the word.
5) The word accent rank of the teacher voice information is calculated from the word accent intensity of the teacher voice information.
6) The phoneme accent rank of the teacher voice information is calculated from the phoneme accent intensity of the teacher voice information for each word.
7) The section information for each phoneme of the input voice information is calculated using forced alignment.
8) The accent intensity for each frame of the input voice information is calculated from the input voice information.
9) The accent intensity for each phoneme of the input voice information is calculated from the section information for each phoneme and the accent intensity for each frame.
10) The accent intensity for each word of the input voice information is calculated from the accent intensity for each phoneme in the word.
11) The word accent rank of the input voice information is calculated from the word accent intensity of the input voice information.
12) The phoneme accent rank of the input voice information is calculated from the phoneme accent intensity of the input voice information for each word.
13) The sentence accent rank correlation coefficient is calculated from the word accent rank of the teacher voice information and the word accent rank of the input voice.
14) The rank correlation coefficient of the word accent for each word is calculated from the phoneme accent rank of the teacher voice information and the phoneme accent rank of the input voice information.
15) The sentence accent rating score is obtained from the sentence accent rank correlation coefficient.
16) The word accent rating score for each word is obtained from the rank correlation coefficient of the word accent.

Further, in the present embodiment, the accent rating and the rhythm rating may be performed by using both the accent intensity and the rhythm amount as the feature amount. Then, the representative score calculated by using both the accent rating score and the rhythm rating score may be calculated and output.

Further, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded on a recording medium such as a CD-ROM and disseminated. It should be noted that this also applies to other embodiments herein. The software that realizes the information processing device in this embodiment is the following program. That is, in this program, the recording medium accessible to the computer is a teacher change in which the teacher change information regarding the change in the feature amount of each of two or more partial voice information constituting the teacher voice information which is the teacher voice information is stored. An information storage unit is provided, and a computer is provided with a reception unit that receives input audio information that is audio information having two or more partial voices, and an input regarding a change in the feature amount of each of the two or more partial audio information possessed by the input audio information. Using the acquisition unit for acquiring change information, the input change information and the teacher change information, the input voice information is evaluated, and the evaluation unit for acquiring the score and the output unit for outputting the score are made to function. Program for.

Further, in the above program, it is preferable that the teacher change information and the input change information are information relating to the order of the magnitude of the feature amount of the partial voice information.

Further, in the above program, the acquisition unit acquires the feature amount of each of the two or more partial voice information possessed by the input voice information, and the portion of the two or more partial information possessed by the input voice information that is not subject to evaluation. Assuming that the order of the magnitudes of two or more feature quantities of two or more evaluation target partial audio information excluding the audio information is acquired, and the input change information having the rank of the magnitudes of the two or more feature quantities is acquired. , It is preferable that it is a program that makes a computer function.

Further, in the above program, when the information possessed by the teacher change information and the information regarding the rank of the magnitude of the feature amount of the two or more evaluation target partial voice information is the same information, the acquisition unit performs the input change. When it is judged whether or not the ranks of the features of the two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other and are judged to be adjacent to each other. Is a program that operates a computer by regarding the magnitudes of the feature amounts of the two evaluation target partial voice information as the same magnitude and acquiring the input change information.

Further, in the above program, the acquisition unit acquires the feature amount of each of the two or more partial voice information possessed by the input voice information, and the portion of the two or more partial information possessed by the input voice information that is not subject to evaluation. Information that distinguishes between the evaluation target partial voice information corresponding to the largest feature amount and other evaluation target partial voice information for two or more feature quantities of two or more evaluation target partial voice information excluding the voice information. It is preferable that the program operates a computer to acquire certain input change information.

Further, in the above program, the information regarding the ranking is assumed to be an accent pattern which is information regarding the ranking of the magnitude of the feature amount of each of two or more partial voice information of the teacher voice information or the input voice information. It is preferable that the program operates the computer.

Further, in the above program, the input voice information is the voice information of a sentence, and the partial voice information is the voice information of the words constituting the sentence, and it is preferable that the program functions the computer. ..

Further, in the above program, it is preferable that the input voice information is the voice information of the word and the partial voice information is the voice information of the phonemes constituting the word, so that the program functions the computer. ..

Further, in the above program, it is preferable that the feature amount of the partial voice information is a program that causes the computer to function, assuming that the feature amount is the accent intensity, which is information on the intensity of the accent.

Further, in the above program, it is preferable that the feature amount of the partial voice information is a rhythm amount which is information on the length of the voice information, and the program functions the computer.

(Embodiment 2)
In the present embodiment, in addition to the evaluation performed in the first embodiment, the pronunciation evaluation is performed on the input voice information, and the final evaluation result and the pronunciation evaluation result performed in the first embodiment are used. A voice rating device for calculating a score will be described. The evaluation performed in the first embodiment is, for example, an accent evaluation and a rhythm evaluation. That is, in the present embodiment, a voice rating device that performs two or more of the accent rating, the rhythm rating, and the pronunciation rating will be described.

FIG. 6 is a block diagram of the voice rating device 2 according to the present embodiment.

The voice evaluation device 2 includes a storage unit 11, a reception unit 12, a processing unit 23, and an output unit 24.

The processing unit 23 includes an acquisition unit 131, a rating unit 132, a second rating unit 231 and a calculation unit 232.

The processing unit 23 performs various processes. The various processes are, for example, processes performed by the acquisition unit 131, the rating unit 132, the second rating unit 231 and the calculation unit 232.

The second rating unit 231 evaluates the pronunciation of the input voice information and acquires the second score. The second score is the pronunciation rating score. The second rating unit 231 performs the same processing as the pronunciation rating performed by the pronunciation rating device or the like described in, for example, Patent No. 4859125, Patent No. 4962930, Patent No. 5007401, etc., and evaluates the quality of pronunciation. Get a second score that indicates. It is assumed that the teacher voice information is stored in the storage unit 11. Further, in the storage unit 11, one or more teacher data, which is an acoustic model for each one or more phonemes, is usually stored. Further, the algorithm in which the second rating unit 231 evaluates the sound quality of the input voice information and obtains the second score does not matter. Since the pronunciation evaluation algorithm for obtaining the second score is a known technique, detailed description thereof will be omitted.

The calculation unit 232 calculates a representative score, which is a representative score, by using the score acquired by the rating unit 132 and the second score acquired by the second rating unit 231. The representative score is, for example, the average value of the score and the second score. The representative score is, for example, the value of the weighted average of the score and the second score. Further, here, the score acquired by the rating unit 132 is, for example, a sentence score. However, the score acquired by the rating unit 132 here may be, for example, a sentence score and a word score of 1 or more. Further, the score acquired by the rating unit 132 here may be, for example, the score of one or more words.

The output unit 24 outputs the representative score calculated by the calculation unit 232. The output unit 24 may also output a score or a second score.

The processing unit 23, the second rating unit 231 and the calculation unit 232 can usually be realized from an MPU, a memory, or the like. The processing procedure of the processing unit 23 and the like is usually realized by software, and the software is recorded on a recording medium such as ROM. However, it may be realized by hardware (dedicated circuit).

The output unit 24 may or may not include an output device such as a display or a speaker. The output unit 24 can be realized by the driver software of the output device, the driver software of the output device, the output device, or the like.

Next, the operation of the voice rating device 2 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 7, the same steps as the flowchart of FIG. 2 will be omitted.

(Step S701) The second rating unit 231 evaluates the pronunciation of the input voice information received in step S201 and acquires the second score.

(Step S702) The calculation unit 232 calculates a representative score, which is a representative score, by using the score acquired by the rating unit 132 and the second score acquired in step S701.

(Step S703) The output unit 24 outputs the representative score calculated by the calculation unit 232 calculated in step S702. Return to step S201.

In the flowchart of FIG. 7, only the representative score may be output. That is, the score calculated in the first embodiment does not have to be output.

Further, in the flowchart of FIG. 7, the processing is terminated by an interrupt of power off or processing termination.

As described above, according to the present embodiment, since the pronounced input voice can be evaluated from various angles, the input voice can be appropriately evaluated. Specifically, according to the present embodiment, for example, accent evaluation and pronunciation evaluation can be performed on the pronounced input voice.

The software that realizes the information processing device in this embodiment is the following program. That is, in this program, the recording medium accessible to the computer is a teacher change in which the teacher change information regarding the change in the feature amount of each of two or more partial voice information constituting the teacher voice information which is the teacher voice information is stored. An information storage unit is provided, and a computer is provided with a reception unit that receives input audio information that is audio information having two or more partial voices, and an input regarding a change in the feature amount of each of the two or more partial audio information possessed by the input audio information. Using the acquisition unit for acquiring change information, the input change information and the teacher change information, the input voice information is evaluated, and the evaluation unit for acquiring the score and the output unit for outputting the score are made to function. Program for.

Further, in the above program, the computer evaluates the pronunciation of the input voice information and obtains a second score, a second rating unit, a score acquired by the evaluation unit, and a second evaluation unit acquired by the second evaluation unit. It is preferable that the output unit is a program that functions a computer to output the representative score by further functioning as a calculation unit for calculating a representative score which is a representative score by using the score. ..

(Embodiment 3)
In the present embodiment, the production apparatus that automatically generates the teacher change information of the teacher change information storage unit 111 will be described.

FIG. 8 is a block diagram of the production apparatus 3 according to the present embodiment.

The production device 3 includes a teacher change information storage unit 111, a reception unit 12, an acquisition unit 131, and a storage unit 31. The acquisition unit 131 includes a division unit 1311, a feature amount acquisition unit 1312, and a change information acquisition unit 1313.

The voice information received by the reception unit 12 here is teacher voice information. Further, the processing target of the acquisition unit 131 here is the teacher voice information received by the reception unit 12.

The dividing means 1311 divides the teacher voice information received by the reception unit 12 into two or more partial voice information.

The feature amount acquisition means 1312 acquires two or more feature amounts from each of the two or more evaluation target partial voice information possessed by the two or more partial voice information.

The change information acquisition means 1313 acquires teacher change information using two or more features.

The storage unit 31 stores the teacher change information in the recording medium. The recording medium here is usually the teacher change information storage unit 111. The storage unit 31 may store the teacher change information in the recording medium in association with the teacher voice information, for example. Further, the storage unit 31 may store the teacher change information in the recording medium in association with the word transcription acquired from the teacher voice information, for example. Further, the storage unit 31 may store the teacher change information of the word in the recording medium in association with the phoneme transcription of the word constituting the teacher voice information.

The storage unit 31 can usually be realized from an MPU, a memory, or the like. The processing procedure of the storage unit 31 is usually realized by software, and the software is recorded on a recording medium such as ROM. However, it may be realized by hardware (dedicated circuit).

Next, the operation of the production apparatus 3 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 9, the same steps as the flowchart of FIG. 3 will be omitted.

(Step S901) The reception unit 12 determines whether or not the teacher voice information has been received. If the teacher voice information is received, the process proceeds to step S301, and if the teacher voice information is not received, the process returns to step S901.

(Step S902) The storage unit 31 stores the change information of the text acquired in step S316 in the recording medium. The change information here is the teacher change information of the text.

(Step S903) The storage unit 31 stores the change information of one or more words acquired in step S313 in the recording medium. End the process. The change information here is word teacher change information.

Hereinafter, the specific operation of the production apparatus 3 in the present embodiment will be described. Specific operation examples of the production apparatus 3 are the following operations 1) to 6).
1) The section information for each phoneme of the teacher voice information is calculated using forced alignment.
2) The accent intensity for each frame of the teacher voice information is calculated from the teacher voice data.
3) The accent intensity for each phoneme of the teacher voice information is calculated from the section information for each phoneme and the accent intensity for each frame.
4) The accent intensity for each word of the teacher's voice information is calculated from the accent intensity for each phoneme in the word.
5) Generate a word accent pattern of teacher voice information from the accent intensity of each word.
6) Generate a phoneme accent pattern of teacher voice information from the accent intensity of each phoneme.

Further specific examples of the above operation will be described below. Now suppose that a teacher with a model pronunciation reads out the sentence "Alice looked up." Then, the reception unit 12 of the production device 3 receives the teacher voice information which is the voice data of the sentence “Alice looked up.”.

Next, the dividing means 1311 divides the teacher voice information into two or more words. That is, the dividing means 1311 acquires the section information for each phoneme constituting the teacher voice information by using forced alignment or the like.

Next, the feature amount acquisition means 1312 acquires the feature amount of each of two or more frames possessed by each phoneme in the order in which the phonemes are arranged for each phoneme. Here, it is assumed that the feature amount is the accent intensity. Then, the feature amount acquisition means 1312 acquires a representative feature amount (for example, the maximum value) from the feature amounts of two or more frames for each phoneme.

Next, the change information acquisition means 1313 acquires the feature amounts (representative feature amounts) of two or more phonemes in the word for each word in the order of phoneme arrangement. That is, first, the change information acquisition means 1313 has a sequence of features (phoneme accent intensities) "/50.041230 0.000000 65.123454 0.000000 /" for the phoneme transcription "/ ae l ax s /" corresponding to the word "Alice". "Is obtained. Then, the change information acquisition means 1313 obtains the teacher change information “/1 0 2 0 /” of the word “Alice” from the arrangement of the phoneme accent intensities. Here, the change information acquisition means 1313 acquired the teacher change information by the second change information acquisition method.

Similarly, the change information acquisition means 1313 obtains a sequence of features (phoneme accent intensity) for the phoneme transcription corresponding to the word “looked” and the word “up”. Then, the change information acquisition means 1313 acquires the teacher change information of the word "looked" and the word "up" by the second change information acquisition method.

Next, the change information acquisition means 1313 acquires the input change information of the sentence by using the representative feature amounts of the acquired two or more words. That is, the change information acquisition means 1313 has a word accent intensity “/0.000000/65.123454” which is a sequence of feature quantities of each word constituting the word transcription “/ sil/alice/sil/looked/sil/up/sil/”. From "/0.000000/54.012354/0.000000/45.987661/0.000000/", the word accent pattern "/ 0/3/0/2/0/1/0 /" is acquired by the first change information acquisition method. This word accent pattern is an example of teacher change information.

Then, the storage unit 31 stores the teacher change information "/ 0/3/0/2/0/1/0 /" of the acquired sentence in the recording medium. Here, it is assumed that the storage unit 31 stores, for example, the word (phoneme) transcription "Alice looked up." And the teacher change information of the sentence as a pair.

In addition, the storage unit 31 stores the teacher change information of the acquired word in the recording medium. Here, it is assumed that the storage unit 31 stores the word (phoneme) transcription "Alice" or the like of the teacher voice information and the teacher change information of the word as a pair.

Through the above processing, the production apparatus 3 obtains the teacher change information management table shown in FIG.

As described above, according to the present embodiment, it is possible to automatically generate teacher data for evaluating the voice in consideration of the flow of the sounded input voice.

The software that realizes the information processing device in this embodiment is the following program. That is, this program divides the computer into a reception unit that receives teacher voice information, a dividing means that divides the teacher voice information into two or more partial voice information, and two or more of each of the two or more partial voice information. A feature amount acquisition means for acquiring two or more feature amounts from the evaluation target partial voice information, a change information acquisition means for acquiring teacher change information using the two or more feature amounts, and the teacher change information as a recording medium. It is a program for functioning as a storage unit for storage.

In addition, FIG. 10 shows the appearance of a computer that executes the program described in the present specification to realize the voice rating device and the like of various embodiments described above. The above embodiments can be realized in computer hardware and computer programs running on it. FIG. 10 is an overview view of the computer system 300, and FIG. 11 is a block diagram of the system 300.

In FIG. 10, the computer system 300 includes a computer 301 including a CD-ROM drive 3012, a keyboard 302, a mouse 303, a monitor 304, and a microphone 305.

In FIG. 11, the computer 301 includes a CD-ROM drive 3012, an MPU 3013, an MPU 3013, a bus 3014, a ROM 3015, a RAM 3016, and a hard disk 3017. The ROM 3015 stores a program such as a boot-up program. The RAM 3016 is connected to the MPU 3013 to temporarily store the instructions of the application program and provide a temporary storage space. The hard disk 3017 usually stores application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card that provides a connection to the LAN.

The program for causing the computer system 300 to execute the functions of the voice rating device 1 and the like according to the above-described embodiment may be stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. .. Further, the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017. The program is loaded into RAM 3016 at run time. The program may be loaded directly from the CD-ROM3101 or network.

The program does not necessarily include an operating system (OS) that causes the computer 301 to execute the functions of the voice rating device 1 and the like according to the above-described embodiment, or a third-party program. The program only needs to include a portion of the instructions that call the appropriate module in a controlled manner to achieve the desired result. It is well known how the computer system 300 works, and detailed description thereof will be omitted.

Further, the number of computers that execute the above program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

Further, in each of the above embodiments, it goes without saying that the two or more communication means existing in one device may be physically realized by one medium.

Further, in each of the above-described embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be done.

It goes without saying that the present invention is not limited to the above embodiments, and various modifications can be made, and these are also included in the scope of the present invention.

As described above, the voice rating device according to the present invention has the effect of being able to appropriately evaluate the input voice because it can evaluate the voice in consideration of the flow of the input voice pronounced, and is in a foreign language such as English. It is useful as a learning device for.

1, 2 Voice rating device 3 Production device 11 Storage section 12 Reception section 13, 23 Processing section 14, 24 Output section 31 Storage section 111 Teacher change information storage section 131 Acquisition section 132 Rating section 231 Second rating section 232 Calculation section 1311 Division Means 1312 Feature amount acquisition means 1312 Collection amount acquisition means 1313 Change information acquisition means

Claims (10)

A teacher change information storage unit that stores teacher change information related to changes in the features of two or more partial voice information that constitutes teacher voice information, which is voice information that serves as a teacher.
A reception unit that accepts input voice information, which is voice information having two or more partial voice information,
An acquisition unit that acquires input change information relating to a change in the feature amount of each of two or more partial voice information possessed by the input voice information, and an acquisition unit.
Using the input change information and the teacher change information, the evaluation unit that evaluates the input voice information and acquires a score,
It is provided with an output unit that outputs the score.
The teacher change information and the input change information are
This is information regarding the order of the magnitude of the feature amount of the partial voice information.
When the information is the information possessed by the teacher change information and the information regarding the rank of the feature amount of at least two or more evaluation target partial voice information is the same information.
The acquisition unit
It is the information possessed by the input change information, and it is determined whether or not the ranks of the features of at least two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other, and they are adjacent to each other. A voice rating device that acquires input change information by regarding at least the size of the feature amount of the two evaluation target partial voice information as the same size when it is determined that the information is present. The acquisition unit
The feature amount of each of two or more partial voice information possessed by the input voice information is acquired, and at least two or more of the two or more partial information possessed by the input voice information of the two or more feature amounts of the evaluation target partial voice information. get the size of the order, the sound evaluation apparatus according to claim 1, wherein acquiring the input change information having the two or more feature quantity of the size of the order. The acquisition unit
The feature amount of each of two or more partial voice information possessed by the input voice information is acquired, and at least two or more of the two or more partial information possessed by the input voice information are converted into two or more feature amounts of the evaluation target partial voice information. against it, the greatest rating corresponding to the characteristic amount target portion speech information and other assessment target portion speech information and voice evaluation apparatus according to claim 1, wherein acquiring the input change information is information for distinguishing. The input voice information is text voice information, and is
The voice rating device according to any one of claims 1 to 3 , wherein the partial voice information is voice information of words constituting a sentence. The input voice information is word voice information, and is
The voice rating device according to any one of claims 1 to 3 , wherein the partial voice information is voice information of phonemes constituting a word. The feature amount of the partial voice information is
The voice rating device according to claim 1, wherein the accent strength is information on the strength of the accent, or the amount of rhythm is the information on the length of the voice information. The rating section
The voice rating device according to any one of claims 1 to 6 , wherein the rank correlation coefficient between the input change information and the teacher change information is acquired as a score. The second rating section, which evaluates the pronunciation of the input voice information and obtains the second score,
A calculation unit for calculating a representative score, which is a representative score, is further provided by using the score acquired by the rating unit and the second score acquired by the second rating unit.
The output unit
The voice rating device according to any one of claims 1 to 7, which outputs the representative score. The recording medium is
It is provided with a teacher change information storage unit for storing teacher change information regarding changes in the feature amount of each of two or more partial voice information constituting the teacher voice information which is the voice information to be a teacher.
A voice rating method realized by the reception section, acquisition section, rating section, and output section.
A reception step in which the reception unit receives input voice information which is voice information having two or more partial voice information,
An acquisition step in which the acquisition unit acquires input change information relating to a change in the feature amount of each of two or more partial voice information possessed by the input voice information.
A rating step in which the rating unit evaluates the input voice information using the input change information and the teacher change information and acquires a score.
The output unit includes an output step for outputting the score.
The teacher change information and the input change information are
This is information regarding the order of the magnitude of the feature amount of the partial voice information.
When the information is the information possessed by the teacher change information and the information regarding the rank of the feature amount of at least two or more evaluation target partial voice information is the same information.
In the acquisition step
It is the information possessed by the input change information, and it is determined whether or not the ranks of the features of at least two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other, and they are adjacent to each other. A voice rating method for acquiring input change information by regarding at least the size of the feature amount of the two evaluation target partial voice information as the same size. A computer that can access the teacher change information storage unit that stores teacher change information related to changes in the feature amount of each of the two or more partial voice information that constitutes the teacher voice information, which is the voice information that serves as a teacher.
A reception unit that accepts input voice information, which is voice information having two or more partial voice information,
An acquisition unit that acquires input change information relating to a change in the feature amount of each of two or more partial voice information possessed by the input voice information, and an acquisition unit.
Using the input change information and the teacher change information, the evaluation unit that evaluates the input voice information and acquires a score,
It is a program for functioning as an output unit that outputs the score.
The teacher change information and the input change information are
This is information regarding the order of the magnitude of the feature amount of the partial voice information.
When the information is the information possessed by the teacher change information and the information regarding the rank of the feature amount of at least two or more evaluation target partial voice information is the same information.
The acquisition unit
It is the information possessed by the input change information, and it is determined whether or not the ranks of the features of at least two evaluation target partial voice information at the positions corresponding to the same information are adjacent to each other, and they are adjacent to each other. If it is determined that the information is present, at least the magnitudes of the feature amounts of the two evaluation target partial voice information are regarded as the same magnitude, and the input change information is acquired.

Images