JP6314884B2 - Reading aloud evaluation device, reading aloud evaluation method, and program - Google Patents

Description

  The present invention relates to a technical field such as a system that evaluates reading of a sentence based on speech uttered when a speaker reads out a sentence.

  In recent years, various systems have been proposed for the purpose of supporting announcement evaluation and singing evaluation. For example, Patent Document 1 discloses a technique for scoring and evaluating the skill of a song correctly by scoring a song for each song section based on an inflection (pitch) or volume extracted from a singer's singing voice signal. It is disclosed. On the other hand, the evaluation items in announcement evaluation include inflection, voice volume, smooth tongue, speed, etc. In announcements, it is important to convey information that should be conveyed, for example, even if part of the sentence is not heard If such information is transmitted to the listener, it can be said that it makes sense as an announcement.

JP-A-10-78749

  However, in conventional systems, for example, when scoring announcement evaluation, it is not considered whether it is important in context, and if a mistake occurs in one place, regardless of whether the location of the mistake is important in context , I could only get the same score. For this reason, the practitioner is not convinced of the displayed score, and in order to obtain a high score, it is necessary to practice the entire announcement regardless of the actual importance.

  The present invention has been made in view of the above points, and provides a reading aloud evaluation apparatus, a reading aloud evaluation method, and a program capable of performing an evaluation according to whether or not the part is important in context.

Speech in order to solve the above problems, the invention according to claim 1, emitted when the speaker has read aloud a plurality including statements phrase is a group of clause spoken in one breath Ri Do from one or more clauses An acquisition means for acquiring a score calculated for each phrase based on speech waveform data indicating a waveform of the phrase, a first determination means for determining a weight for each phrase, and the phrase determined by the first determination means A second deciding unit that decides, for each phrase, a scoring ratio corresponding to each weight, and the score acquired by the acquiring unit is adjusted for each phrase by the scoring ratio determined by the second determining unit And calculating means for calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the adjusted score for each phrase. The invention according to claim 2 is based on speech waveform data indicating a waveform of speech uttered when a speaker reads a sentence including a plurality of phrases that are composed of one or more phrases and are spoken at a time. A first acquisition means for acquiring the score calculated for each phrase, a first determination means for determining a weight for each phrase, and a score according to the weight for each phrase determined by the first determination means A second determining means for determining a ratio for each phrase; and a second score for obtaining a score between intervals calculated from an end timing of one of the plurality of phrases to a start timing of the next phrase. Adjust the score acquired by the acquisition means and the first acquisition means for each phrase according to the scoring ratio determined by the second determination means, and adjust And calculating means for calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the score for each phrase and the score of the interval acquired by the second acquisition means, To do.

According to a third aspect of the present invention, in the reading aloud evaluation device according to the first or second aspect , the sentence is divided into the plurality of phrases to be displayed on a screen, and is relatively high among the plurality of phrases. It further comprises display control means for displaying the weighted phrase in a display mode different from other phrases.

Invention according to claim 4, in the reading-aloud evaluation apparatus according to claim 3, wherein the display control unit, a front Symbol phrase acquired the score for each, one at least of the adjusted score for each of the phrases One of them is displayed on the screen in association with each of the phrases.

According to a fifth aspect of the present invention, in the reading aloud evaluation device according to any one of the first to fourth aspects, the first input means for inputting the text data of the sentence and the first input means are input. Decomposing means for decomposing the sentence indicated by the text data into a plurality of words; first specifying means for specifying the importance of each of the decomposed words with reference to reference information that defines the importance of the word; The first determining means determines the weight of the phrase composed of the words for each phrase based on the importance of the word specified by the first specifying means. .

According to a sixth aspect of the present invention, in the reading aloud evaluation apparatus according to the fifth aspect , the second input means for inputting reference voice waveform data indicating a waveform of a voice as a reference for score calculation by reading the sentence aloud, and The first determining means further comprises second specifying means for specifying, for each phrase, the time length of the phrase specified based on the waveform of the voice indicated by the reference voice waveform data input by the second input means. Based on the importance of the word specified by the first specifying means and the time length of the phrase specified by the second specifying means, the weight of the phrase composed of the words is determined for each phrase. It is characterized by determining.

The invention according to claim 7 is a first acquisition for acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice uttered when a speaker reads a sentence including a plurality of phrases aloud. means, a first determination means for determining a weight for each of the clauses, the second determining means for determining the allocated points ratio corresponding to the weight of each of the clauses which are determined by the first determining means for each of the clauses, wherein A second acquisition means for acquiring a score of the interval calculated in an interval section from the end timing of any one of the plurality of phrases to the start timing of the next phrase; and the score acquired by the first acquisition means a second adjusted by the allotted points ratio determined by the determining means for each said clause, the scores for each adjusted clause, based on the scores of Maai acquired by the second acquisition means, said plurality of Characterized in that it comprises calculating means for calculating the total score for the sentence entire reading aloud containing section a.

According to an eighth aspect of the present invention, in the reading aloud evaluation device according to the seventh aspect , the sentence is divided into the plurality of phrases and displayed on a screen, and a relatively high weighting is given among the plurality of phrases. It is further characterized by further comprising display control means for displaying the made clause in a display mode different from other clauses.

Invention according to claim 9, in reading aloud evaluation apparatus according to claim 8, wherein the display control unit, before SL and the scores obtained for each clause, either at least the adjusted score for each of the clauses One of them is displayed on the screen in association with each of the phrases.

A tenth aspect of the present invention is the reading aloud evaluation device according to any one of the seventh to ninth aspects, wherein the first input means for inputting the text data of the sentence and the first input means are input. Decomposing means for decomposing the sentence indicated by the text data into a plurality of words; first specifying means for specifying the importance of each of the decomposed words with reference to reference information that defines the importance of the word; The first determining means determines, for each phrase, the weight of the phrase composed of the words based on the importance of the word specified by the first specifying means. .

The invention according to claim 11 is the reading aloud evaluation apparatus according to claim 10 , wherein the second input means for inputting reference speech waveform data indicating a waveform of a speech as a reference for score calculation by reading the sentence aloud, and Second specifying means for specifying, for each phrase, a time length of the phrase specified based on the waveform of the voice indicated by the reference voice waveform data input by the second input means, the first determining means Is based on the importance of the word specified by the first specifying means and the time length of the phrase specified by the second specifying means, and sets the weight of the phrase composed of the words for each phrase. It is characterized by determining to.

Statement invention according to claim 12, a reading aloud evaluation method performed by one or more computers, including a plurality of phrases a collection of clause speaker spoken in one breath Ri Do from one or more clauses An acquisition step for acquiring a score calculated for each phrase based on speech waveform data indicating a waveform of a speech uttered when reading aloud, a first determination step for determining a weight for each phrase, and the first A second determination step for determining for each phrase a scoring ratio according to the weight for each phrase determined by the determination step; and the score acquired by the acquisition step, the determination by the second determination step. Adjust for each phrase by the scoring ratio, and calculate the total score for reading aloud the entire sentence including the plurality of phrases based on the adjusted score for each phrase A calculation step that, characterized in that it comprises a. The invention according to claim 13 is a reading aloud evaluation method executed by one or more computers, wherein a speaker is composed of one or more phrases and includes a plurality of phrases that are a group of phrases spoken at a breath. A first acquisition step of acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice generated when reading aloud; a first determination step of determining a weight for each phrase; A second determination step for determining for each phrase a scoring ratio according to the weight for each phrase determined in one determination step; and a start timing of the next phrase from an end timing of any one of the phrases The second acquisition step of acquiring the score of the interval calculated in the interval section until, and the acquired in the first acquisition step The points are adjusted for each phrase by the scoring ratio determined in the second determination step, and the plurality of points are calculated based on the adjusted score for each phrase and the score for the interval acquired in the second acquisition step. And a calculating step for calculating a total score for reading aloud the entire sentence including the phrase.

The invention according to claim 14 is based on speech waveform data indicating a waveform of speech uttered when a speaker reads a sentence including a plurality of phrases that are composed of one or more phrases and are spoken at a time. An acquisition step for acquiring the score calculated for each phrase, a first determination step for determining a weight for each phrase, and a scoring ratio according to the weight for each phrase determined by the first determination step The second determination step determined for each phrase, and the score acquired by the acquisition step is adjusted for each phrase by the scoring ratio determined by the second determination step, and the adjusted score for each phrase And a calculation step of calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the computer. . The invention according to claim 15 is based on speech waveform data indicating a waveform of speech uttered when a speaker reads a sentence including a plurality of phrases that are composed of one or more phrases and are spoken at a time. A first acquisition step for acquiring a score calculated for each phrase, a first determination step for determining a weight for each phrase, and a score according to the weight for each phrase determined by the first determination step A second determination step of determining a ratio for each of the phrases; and a second step of acquiring a score of a gap calculated in an interval section from an end timing of one of the phrases to a start timing of the next phrase The score obtained in the acquisition step and the first acquisition step is determined according to the scoring ratio determined in the second determination step. A calculation step of adjusting for each phrase and calculating a total score for aloud reading of the whole sentence including the plurality of phrases based on the score for each adjusted phrase and the score of the interval acquired by the second acquisition step And making the computer execute.

The invention according to claim 16 is a speech reading evaluation method executed by one or more computers, wherein speech waveform data indicating a speech waveform generated when a speaker reads a sentence including a plurality of phrases aloud. A first acquisition step for acquiring a score calculated for each phrase based on the first determination step for determining a weight for each phrase, and a weight for each phrase determined by the first determination step A second determination step of determining a scoring ratio for each of the clauses, and obtaining a score of the interval calculated in an interval section from the end timing of any one of the plurality of clauses to the start timing of the next clause and second acquisition step, has been the scores obtained by the first obtaining step, by the allocated points ratio determined by said second determining step to adjust for each of the clauses, tone And scores for each clause that, based on the scores of Maai acquired by the second acquisition step, a calculation step of calculating the total score for the sentence entire reading aloud including the plurality of clauses, and characterized in that it comprises To do.

The invention according to claim 17 is a first acquisition for acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice uttered when a speaker reads a sentence including a plurality of phrases aloud. a step, a first determining step of determining a weight for each of the clauses, and a second determination step of determining the allocated points ratio corresponding to the weight of each of the clauses which are determined by the first determining step for each of the clauses, wherein A second acquisition step of acquiring a score of the interval calculated in an interval section from the end timing of any one of the plurality of clauses to the start timing of the next clause; and the score acquired by the first acquisition step while a second was adjusted by the allotted points ratio determined by the determining step for each of the clauses, the scores for each adjusted clause, acquired by the second acquisition step Based on the had scores, characterized in that to execute a calculation step, to a computer which calculates the total score for the sentence entire reading aloud including the plurality of clauses.

According to invention of Claim 1, 2 , 7, 12-17 , it can evaluate according to whether it is an important part in context.

According to the third and eighth aspects of the present invention, it is possible to cause the user to confirm at a glance a phrase that is given a relatively high weight among a plurality of phrases, phrases, or words.

According to the fourth and ninth aspects of the present invention, the score for each phrase, phrase, or word can be confirmed at a glance by the user.

According to the fifth and tenth aspects of the present invention, it is possible to reflect the importance of the phrase or phrase in the weight in units of words.

According to the inventions described in claims 6 and 11 , it is possible to increase the appropriateness of the weight of the phrase or phrase.

It is a figure which shows the example of a schematic structure of the reading aloud evaluation apparatus S which concerns on this embodiment. It is a conceptual diagram which shows the example by which the score for every phrase about a certain evaluation item is adjusted with a scoring ratio, and the total score with respect to the reading of the whole sentence is calculated. It is a conceptual diagram which shows the example in which importance is determined for every phrase based on the importance of each of the some word which comprises a phrase within the weighting engine shown in FIG. It is a figure which shows the example of a screen which displays the information which shows evaluation with respect to a speaker's reading aloud. 3 is a flowchart illustrating an example of a reading aloud evaluation process performed by a control unit 3 according to the first embodiment. (A) is a conceptual diagram which shows the example in which the importance of a phrase is determined based on the importance and the word length of a word. (B) is a conceptual diagram showing an example in which a phrase weight and a score ratio are determined based on a phrase importance and a phrase length. 10 is a flowchart illustrating an example of a reading aloud evaluation process performed by a control unit 3 according to the second embodiment. It is a figure which shows the example by which the score for every word is adjusted with the scoring ratio, and the total score with respect to the reading of the whole sentence is calculated.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[1. Configuration and function of reading aloud evaluation device S]
First, with reference to FIG. 1, the structure and function of the reading aloud evaluation apparatus S which concerns on one Embodiment of this invention are demonstrated. FIG. 1 is a diagram illustrating a schematic configuration example of the reading aloud evaluation apparatus S according to the present embodiment. In addition, a personal computer, a portable information terminal (smartphone, etc.) etc. are mentioned as an example of a reading aloud evaluation apparatus. As shown in FIG. 1, the reading aloud evaluation device S includes a communication unit 1, a storage unit 2, a control unit 3, an operation unit 4, an interface (IF) unit 5, and the like. It is connected to the. The operation unit 4 receives an operation instruction from the user and outputs a signal corresponding to the received operation to the control unit 3. The interface unit 5 is connected to a microphone M, a display D, and the like. The microphone M collects a sound produced when a speaker who performs speech learning such as language learning, announcement, reading, etc. reads a sentence (sentence) aloud. The sentence includes a plurality of phrases or a plurality of clauses. A phrase is composed of one or more phrases, and is a group of phrases that are generally spoken at a breath. Each phrase and phrase includes one or more words. Therefore, the sentence includes one or more words. Words include independent words such as nouns, verbs, adjectives, adverbs, and conjunctions (parts of speech that can constitute a phrase alone), adjuncts such as auxiliary verbs and particles (parts of speech that cannot constitute a phrase alone), and the like. Examples of sentences to be read aloud include, for example, sentences used for language learning, announcement training, reading training, and lyrics used for singing. The display D displays information (for example, a score) indicating, for example, evaluation of the speaker's reading aloud on the screen according to a display command from the control unit 3. This evaluation includes evaluation of voice volume, evaluation of inflection (also referred to as pitch or pitch), evaluation of smooth tongue, evaluation of speed (reading speed), evaluation of interval (interval), and overall evaluation. The microphone M and the display D may be integrated with the reading aloud evaluation device S or may be separate.

  The communication unit 1 communicates with a server or the like by connecting to a network (not shown) by wire or wireless. The storage unit 2 includes, for example, a hard disk drive and stores an OS (Operating System), a reading aloud evaluation processing program (an example of the program of the present invention), and the like. The reading aloud evaluation processing program is a program for causing the control unit 3 as a computer to execute a reading aloud evaluation processing described later. The reading aloud evaluation processing program may be downloaded from a predetermined server as an application, or may be provided by being stored in a recording medium such as a CD or a DVD. The storage unit 2 also stores voice data (hereinafter referred to as “voice data”) indicating the waveform of the above-described sentence text data and the voice that serves as a reference for score calculation by reading the sentence (for example, the voice that serves as a model for reading aloud). "Reference voice waveform data"). Here, the text data includes, for example, the sound generation timing of each character (for example, the elapsed time from the start of sound generation) in association with each character or each word. The reference voice waveform data is stored in a predetermined voice file format.

  In the storage unit 2, a word importance database (DB) for registering reference information that defines the importance of words is constructed. The importance of a word is a degree indicating how important a word is in context when the word is included in a sentence. For example, proper nouns and words indicating numerical values can be said to be important information to be communicated to listeners such as announcements. Therefore, such words have a high importance (relative to other words). High importance is given). In particular, the number representing the quantity and the order by the number is important information, so it is preferable that the importance is unconditionally set high. The importance of the word may be expressed by letters such as “small (low)”, “medium”, “large (high)”, or “1”, “2”, “3”, “ It may be expressed numerically such as “4” and “5”. Thereby, for example, a difference is made between words that are important and words that are not. In the reference information, the importance level may be defined for each word for all the words included in the sentence, or only words with high importance levels (or only words with low importance levels) may be specified.

  The control unit 3 includes a CPU (Central Processing Unit) as a computer, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The control unit 3 functions as a speech processing unit 31, a reading aloud evaluation unit 32, a score adjustment unit 33, and a display control unit 34 by a reading aloud evaluation processing program. The speech processing unit 31, the reading aloud evaluation unit 32, and the score adjustment unit 33 are an acquisition unit, a first input unit, a second input unit, an input unit, a first determination unit, a second determination unit, a determination unit, and a calculation in the present invention. It is an example of a means, a disassembling means, a first specifying means, a second specifying means, and a specifying means. The display control unit 34 is an example of display control means in the present invention.

  The control unit 3 inputs reference audio waveform data stored in a predetermined audio file format from the storage unit 2 to the audio processing unit 31. In addition, the control unit 3 is voice waveform data (hereinafter referred to as “speaker voice waveform data”) indicating a waveform of a voice collected by the microphone M when the speaker reads the above sentence aloud. Is input to the voice processing unit 31. The reference speech waveform data and speaker speech waveform data are collectively referred to as speech waveform data. The voice waveform data is discretized time-series sound pressure waveform data, for example, sampling rate 44.1 kHz, quantization 16 bits, and monaural waveform data. Note that the sound pressure refers to a change (Pa) in air pressure due to sound waves. In this embodiment, the sound pressure level (dB) that represents the magnitude of effective sound pressure (Pa), which is the root mean square (RMS) of instantaneous sound pressure (Pa), is expressed as a numerical value that is easy to handle in calculation. To do. The sound pressure level (dB) is also called volume in a broad sense. The sound pressure level is used to evaluate the voice volume.

  The voice processing unit 31 calculates the sound pressure level (dB) from the data cut out from the voice waveform data every predetermined time (for example, 10 ms), for example, as a voice volume every predetermined time (at predetermined time intervals). Further, the voice processing unit 31 calculates a fundamental frequency (Hz) from data cut out at predetermined time intervals from the voice waveform data, and calculates the calculated fundamental frequency (Hz) at predetermined time intervals as inflections. For example, a known method such as a zero-cross method or a vector autocorrelation can be applied to the intonation calculation method. In addition, the voice processing unit 31 calculates a feature amount (acoustic characteristic) indicating a vocal tract characteristic used for evaluation of the smooth tongue for each word. For example, the voice processing unit 31 cuts out the voice waveform data for each word (word section) (for example, cut out based on text data of a sentence read aloud), and divides the data of the cut out word section by windowing (for example, Amplitude spectrum is obtained by performing a Fourier analysis (FFT). Then, the speech processing unit 31 multiplies the obtained amplitude spectrum by a mel filter bank and performs a discrete cosine transform (DCT) on a logarithmic value of the output of the mel filter bank to obtain a MFCC (mel frequency cepstrum coefficient). It is calculated for each word as a feature quantity indicating the characteristic.

  In addition, the voice processing unit 31 specifies a phrase section (hereinafter referred to as “reference phrase section”) from the start timing to the end timing of each phrase based on the reference voice waveform data for each phrase. Here, the start timing and the end timing may be recognized from the sound waveform, or may be recognized from the sound pressure level (dB) calculated as described above. For example, the voice processing unit 31 recognizes the time point when the amplitude width of the voice waveform becomes a predetermined value or more as the start timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes a predetermined value or more as the start timing. Further, for example, the voice processing unit 31 recognizes the time point when the amplitude width of the voice waveform is less than a predetermined value as the end timing. Alternatively, the sound processing unit 31 recognizes the time point when the sound pressure level (dB) becomes less than a predetermined value as the end timing. In addition, the voice processing unit 31 specifies an interval section (hereinafter referred to as a “reference interval section”) from the end timing of one of the phrases to the start timing of the next phrase. The speech processing unit 31 may specify a reference interval section from the end timing of any one of the plurality of clauses to the start timing of the next clause. For example, the voice processing unit 31 may specify a word section of each word (hereinafter referred to as “reference word section”) for each word based on the pronunciation timing indicated by the text data of the sentence read aloud. Furthermore, the speech processing unit 31 determines, for example, a phrase section (hereinafter referred to as “reference phrase section”) from the start timing to the end timing of each phrase based on the pronunciation timing indicated by the text data of the sentence read aloud. You may specify every.

  Further, the speech processing unit 31 specifies a phrase section (hereinafter referred to as “speaker phrase section”) from the start timing to the end timing of each phrase based on the speaker speech waveform data. In addition, the voice processing unit 31 identifies an interval section (hereinafter referred to as “speaker interval section”) from the end timing of any one of the phrases to the start timing of the next phrase. Further, the speech processing unit 31 may specify a speaker interval section from the end timing of any one of a plurality of clauses to the start timing of the next clause. For example, the speech processing unit 31 may specify a word section of each word (hereinafter referred to as “speaker word section”) for each word based on the pronunciation timing indicated by the text data of the sentence read aloud. . Furthermore, the speech processing unit 31, for example, based on the pronunciation timing indicated by the text data of the sentence read aloud, the phrase section (hereinafter referred to as “speaker phrase section”) from the start timing to the end timing of each phrase, You may specify for each phrase.

  The phrase section (or phrase section), word section, and interval section data identified as described above are stored in the storage unit 2 in association with the sound file of each sound waveform data, for example. Each identified section (phrase section (or phrase section), word section, and interval section) is represented by, for example, a time range (for example, 01: 00-03: 00) from the start of the waveform. . Also, serial numbers are assigned to the identified sections in order, for example, from the top.

  Next, the reading aloud evaluation unit 32 compares the voice volume of the reference phrase section with the voice volume of the speaker phrase section corresponding to the reference phrase section (for example, the serial numbers match), and performs voice volume evaluation for each phrase. . For example, the above-described sound pressure level (dB) is used as the voice volume to be compared. The reading aloud evaluation unit 32 calculates, for example, a difference between the volume of the reference phrase section and the volume of the speaker phrase section as a comparison result of the volume, and calculates a score (evaluation score) based on the difference, thereby calculating the voice volume. Evaluate. For example, the score is calculated such that the evaluation is higher (that is, the score is higher) as the difference is closer to 0. That is, since the difference becomes larger as the speaker's voice volume is larger or smaller than the reference voice volume, the evaluation becomes lower. On the other hand, since the difference becomes smaller as the speaker's voice volume approaches the reference voice volume, the evaluation becomes higher. In this way, the voice volume is evaluated for each phrase section (that is, a score is calculated). By the way, when comparing the amount of voice calculated at each predetermined time in each phrase section, the evaluation of the voice volume is performed by adjusting the time position of the start of the reference phrase section and the speaker phrase section, and expanding and contracting the phrase section. It is good to carry out according to the time length of a section. At this time, the length may be adjusted by simply expanding and contracting, or the position to be dynamically evaluated in the phrase section may be adjusted using a technique such as DP matching. Alternatively, the voice volume to be compared may be an average voice volume calculated every predetermined time in each phrase section. The reading aloud evaluation unit 32 compares the voice volume of the reference phrase section with the voice volume of the speaker phrase section corresponding to the reference phrase section, and evaluates the voice volume for each phrase by the same evaluation method as the voice volume for each phrase. (That is, a score is calculated for each phrase). Moreover, the reading aloud evaluation unit 32 compares the voice volume of the reference word section with the voice volume of the speaker word section corresponding to the reference word section by the same evaluation method as the voice volume of each phrase, and evaluates the voice volume for each word. (That is, a score is calculated for each word).

  Moreover, the reading aloud evaluation unit 32 compares the inflection (pitch) of the reference phrase section with the inflection of the speaker phrase section corresponding to the reference phrase section, and performs the inflection evaluation for each phrase. For example, the reading aloud evaluation unit 32 calculates the difference between the inflection of the reference phrase section and the inflection of the speaker phrase section as the comparison result of the inflection, and evaluates the inflection by calculating the score based on the difference. For example, the score is calculated such that the evaluation is higher (that is, the score is higher) as the difference is closer to 0. That is, the higher the lower or lower speaker inflection, the greater the difference and the lower the evaluation. On the other hand, the evaluation becomes higher because the difference becomes smaller as the inflection of the speaker approaches the inflection of the reference. In this way, inflection is evaluated for each phrase section (that is, a score is calculated). Further, as in the case of the voice volume, the inflection evaluation may be performed by matching the start time positions of the reference phrase section and the speaker phrase section, expanding and contracting the phrase section, and adjusting the time length of the phrase section. Alternatively, the inflection to be compared may be an average value of the inflection calculated every predetermined time in each phrase section. The reading aloud evaluation unit 32 compares the inflection of the reference phrase section with the inflection of the speaker phrase section corresponding to the reference phrase section, and evaluates the inflection evaluation for each phrase by the same evaluation method as the inflection for each phrase. (That is, a score is calculated for each phrase). The reading aloud evaluation unit 32 compares the inflection of the reference word section with the inflection of the speaker word section corresponding to the reference word section, and evaluates the inflection evaluation for each word by the same evaluation method as the inflection for each phrase. (That is, a score is calculated for each word).

  The reading aloud evaluation unit 32 compares the smooth tongue of the reference word section with the smooth tongue of the speaker word section corresponding to the reference word section, and evaluates the smooth tongue for each word. The reading aloud evaluation unit 32 uses, for example, a feature value (MFCC) indicating vocal tract characteristics calculated for each word in the smooth tongue evaluation. The reading aloud evaluation unit 32 calculates, for example, the similarity between the feature amount of the reference word section and the feature amount of the speaker word section as a smoothing tongue comparison result, and calculates a score based on the similarity. Perform tongue evaluation. This score is calculated, for example, such that the higher the similarity, the higher the evaluation (that is, the higher the score). In this way, the smooth tongue is evaluated for each word section (that is, a score is calculated). Furthermore, the reading aloud evaluation part 32 calculates the feature-value of a reference phrase area for every reference phrase area based on the feature-value of each of the some reference word area contained in a reference phrase area (for example, average). Further, the reading aloud evaluation unit 32 calculates the feature amount of the speaker phrase section for each speaker phrase section based on (for example, on average) the feature amount of each of the plurality of speaker word sections included in the speaker phrase section. To do. Then, the reading aloud evaluation unit 32 calculates, for example, the similarity between the feature amount of the reference phrase section and the feature amount of the speaker phrase section, and calculates a score based on the similarity, thereby evaluating the smooth tongue. Do it every time. The reading aloud evaluation unit 32 compares the smooth tongue of the reference phrase section with the smooth tongue of the speaker phrase section corresponding to the reference phrase section by the same evaluation method as the smooth tongue for each phrase, The evaluation may be performed for each phrase (that is, a score is calculated for each phrase).

  The reading aloud evaluation unit 32 compares the time length (temporal length) of the reference phrase section with the time length of the speaker phrase section, and evaluates the speed of reading the sentence (reading speed) as a phrase (ie, reading speed). This is done for each phrase section. The reading aloud evaluation unit 32 calculates, for example, a time difference between the time length of the reference phrase section and the time length of the speaker phrase section for each phrase as a comparison result of the time length, and obtains a score based on the absolute value of this time difference. The speed is evaluated by calculating. For example, the evaluation is higher (that is, the score is higher) as the absolute value of the time difference is closer to 0. That is, as the speaker speed is faster or slower than the reference speed, the absolute value of the time difference increases, so the evaluation becomes lower. On the other hand, since the absolute value of the time difference decreases as the speaker speed approaches the reference speed, the evaluation increases. In this way, the speed is evaluated for each phrase (that is, a score is calculated). The reading aloud evaluation unit 32 compares the time length of the reference phrase section with the time length of the speaker phrase section corresponding to the reference phrase section in the same evaluation method as the speed for each phrase, and evaluates the speed. It may be performed for each phrase (that is, a score is calculated for each phrase). Moreover, the reading aloud evaluation unit 32 compares the time length of the reference word section with the time length of the speaker word section corresponding to the reference word section by the same evaluation method as the speed for each phrase, and evaluates the speed. You may carry out for every word (that is, a score is calculated for every word).

  Note that the score for each phrase calculated in each of the above-described evaluations of voice volume, intonation, and speed may be calculated based on the scores of a plurality of words constituting each phrase. For example, the score of a phrase is calculated as the average score of the scores of a plurality of words constituting the phrase. Alternatively, for example, the score of a phrase is calculated based on the score and importance of each of a plurality of words constituting the phrase (importance assigned to each word as described above). For example, it is assumed that the scores of a plurality of words constituting the phrase are 5, 5, 6, and 7, respectively, and the importance of the plurality of words constituting the phrase is small, small, large, and small, respectively. . In this case, when the weight ratio is “large: small = 2: 1”, the weights of the plurality of words constituting the phrase are determined as 1, 1, 2, 1, respectively. And the score of the phrase in this case is 5 × 1/5 [1 / (1 + 1 + 2 + 1)] + 5 × 1/5 [1 / (1 + 1 + 2 + 1)] + 6 × 2 / 5 [2 / (1 + 1 + 2 + 1)] + 7 × 1/5 [1 / (1 + 1 + 2 + 1)] = 5.8. Similarly, the score for each phrase calculated in each of the evaluation of voice volume, intonation, and speed may be calculated based on the score of each of a plurality of words constituting each phrase.

  Moreover, the reading aloud evaluation unit 32 compares the time length of the reference interval section with the time length of the speaker interval section, and evaluates the time during which the sentence is read aloud (interval). The reading aloud evaluation unit 32 calculates, for example, a time difference between the time length of the reference interval section and the speaker interval section as a comparison result of time lengths, and calculates a score based on the absolute value of the time difference. Interim assessment. Moreover, the reading aloud evaluation part 32 performs the evaluation between all the interval sections based on the evaluation for every interval section. In the evaluation between all interval sections, for example, the average value of the scores calculated for each interval section is calculated as the total score between all interval sections.

  The score adjustment unit 33 acquires the score calculated for each phrase based on the speech waveform data by the reading aloud evaluation unit 32 for the evaluation item set as the adjustment target. For example, a score for each phrase is acquired for each voice volume, intonation, speed, and smooth tongue. And the score adjustment part 33 determines the weight for every phrase (namely, weighting is carried out for every phrase), and determines the score ratio according to each determined weight for every phrase. The points are inclined according to the point ratio. Note that the weight of the phrase is determined for each phrase based on the importance of each of a plurality of words constituting the phrase, for example, using a word importance database. Thereby, the importance of a phrase can be reflected in a weight for every word. In addition, if the phrase weight is determined for each phrase based on the importance of each of the plurality of words constituting the phrase and the time length of the phrase (for example, the reference phrase section), the phrase weight can be appropriately set. Can be increased. And the score adjustment part 33 adjusts the said acquired score for every phrase by the determined scoring ratio, and calculates the total score with respect to the reading of the whole sentence containing several phrases based on the score for every adjusted phrase. To do. For example, the total score for reading aloud the entire sentence is calculated based on the amount of voice, the inflection, the speed, and the score for each phrase adjusted for each tongue. Note that the score after the adjustment and the score before the adjustment are not necessarily different and may be the same.

  FIG. 2 is a conceptual diagram illustrating an example in which the score for each phrase for a certain evaluation item is adjusted by the scoring ratio to calculate the total score for reading aloud the entire sentence. In the example of FIG. 2, the sentence read aloud is composed of phrases F1 to F4. For example, the importance of the phrases F1 to F4 is “small” by the weighting engine (software module) included in the score adjustment unit 33, for example. , “Large”, “Small”, and “Large”. When the weight ratio is set to “large: small = 2: 1”, the weights of the phrases F1 to F4 are “1”, “2”, “1”, “1”, respectively, as shown in FIG. 2 ". Thereby, the scoring ratios according to the weights of the phrases F1 to F4 are “1/6 [1 / (1 + 2 + 1 + 2)]” and “1/3 [2 / (1 + 2 + 1), respectively. +2)] ”,“ 1/6 [1 / (1 + 2 + 1 + 2)] ”,“ 1/3 [2 / (1 + 2 + 1 + 2)] ”. And the score is adjusted by multiplying each score of phrases F1 to F4 (10 points, 9 points, 8 points, and 7 points) by each of the score ratios of phrases F1 to F4, and the score for each adjusted phrase Is calculated as the total score (8.33 points). Here, in the example of FIG. 2, the scores (10 points, 9 points, 8 points, and 7 points) of the phrases F1 to F4 are points when the score (= full score) is 10 points. As another example, the importance of phrases F1 to F4 is “medium”, “large”, “small”, and “large”, respectively, and the weight ratio is “large: medium: small = 3”. : 2: 1 ”, the weights of the phrases F1 to F4 are determined as“ 2 ”,“ 3 ”,“ 1 ”, and“ 3 ”, respectively. In this case, the scoring ratios according to the weights of the phrases F1 to F4 are “2/9 [2 / (2 + 3 + 1 + 3)]” and “1/3 [3 / (2 + 3 + 1), respectively. +3)] ”,“ 1/9 [1 / (2 + 3 + 1 + 3)] ”,“ 1/3 [3 / (2 + 3 + 1 + 3)] ”.

  FIG. 3 is a conceptual diagram showing an example in which the importance is determined for each phrase based on the importance of each of a plurality of words constituting the phrase in the weighting engine shown in FIG. In the example of FIG. 3, the importance levels of the plurality of words constituting the phrase F1 are “small” and “small”, respectively, based on the word importance database. Thereby, the weight of the word which comprises the phrase F1 is determined to be "1" and "1", respectively. The weights of a plurality of words constituting the phrases F2 to F4 are similarly determined. The average value of the weights of the plurality of words constituting the phrase F1 is “1”, and the average value of the weights of the plurality of words constituting the phrase F2 is “1.2 (for convenience, the second and subsequent digits after the decimal point are The average value of the weights of the plurality of words constituting the phrase F3 is “1”, and the average value of the weights of the plurality of words constituting the phrase F4 is “1.2”. Accordingly, the importance levels of the phrases F1 to F4 are determined as “small”, “large”, “small”, and “large”, respectively. In addition, how to take the numerical value of the weight of a word is an example, and what kind of numerical value may be taken if there is a difference between words according to the importance of a word.

  Moreover, the score adjustment part 33 may acquire the score calculated for every clause by the reading aloud evaluation part 32 based on speech waveform data about a predetermined evaluation item. In this case, as in the case of the phrase, the score adjustment unit 33 determines the weight for each phrase (that is, the phrase is weighted), and the scoring ratio corresponding to each determined weight is determined for each phrase. To do. Then, as in the case of the phrase, the score adjustment unit 33 adjusts the acquired score for each phrase by the determined scoring ratio, and the entire sentence including a plurality of phrases based on the adjusted score for each phrase. Calculate the total score for reading aloud. Moreover, the score adjustment part 33 may acquire the score calculated for every word based on speech waveform data by the reading aloud evaluation part 32 about a predetermined evaluation item. In this case, the score adjustment unit 33 identifies the importance for each word from the word importance database, and determines a score ratio for each word according to the weight reflecting the importance for each identified word. And the score adjustment part 33 adjusts the acquired score for every word by the determined score ratio, and calculates the total score with respect to the reading of the whole sentence containing several words based on the adjusted score for every word.

  The aloud reading evaluation unit 32 performs a comprehensive evaluation on the aloud reading of the entire sentence based on the total score calculated for each evaluation item (for example, voice volume, intonation, smooth tongue, speed, interval). In this comprehensive evaluation, for example, the total score calculated for each evaluation item (for example, voice volume, intonation, smooth tongue, speed, interval) is calculated as an overall score for reading aloud the entire sentence.

  The display control unit 34 divides a sentence into a plurality of phrases and displays the phrase on a screen, and displays a phrase with a relatively high weight among the phrases in a display mode different from other phrases. Thereby, the user can confirm at a glance the phrase with relatively high weighting among the plurality of phrases. At this time, the display control unit 34 obtains at least one of the score acquired (calculated) for each phrase (that is, the score before being adjusted by the score adjusting unit 33) and the score adjusted for each phrase. It is good to display on the screen in association with each phrase. Thereby, the user can also confirm at a glance the score for each phrase that has been given a relatively high weight.

  FIGS. 4A and 4B are diagrams illustrating examples of screens that display information indicating evaluation of a speaker for reading aloud. The screen shown in FIG. 4A is provided with a graph display section 51, a phrase display section 52, a phrase score display section 53, and a total score display section 54. The graph display unit 51 includes a graph 51a representing a time-series change of inflection calculated every predetermined time based on the reference speech waveform data, and an inflection calculated every predetermined time based on the speaker speech waveform data. 51b representing a time-series change of the above, a graph 51c representing a time-series change of the voice volume calculated every predetermined time based on the reference voice waveform data, and every predetermined time based on the speaker voice waveform data And a graph 51d representing a time-series change in the volume of voice calculated in FIG.

  The phrase display unit 52 displays the sentence divided into a plurality of phrases. The phrase score display unit 53 is provided with display fields 53a to 53d that display scores of evaluation items (for example, smooth tongues) corresponding to each phrase displayed on the phrase display unit 52 in association with each phrase. Yes. The display control unit 34 compares the colors (that is, the background color of the score) in the display fields 53b and 53d corresponding to the phrase having a relatively high weight among these display fields 53a to 53d. The display fields 53b and 53d are highlighted in a different manner from the colors in the display fields 53a and 53c corresponding to the phrase with a low weight. Thereby, the user can discriminate | determine the phrase by which relatively high weight was made | formed among the some phrases contained in a sentence. It should be noted that the patterns in the display fields 53b and 53d corresponding to the phrase having a relatively high weight are different from the patterns in the display fields 53a and 53c corresponding to the phrase having a relatively low weight. You may display as follows. Each score (10 points, 9 points, 8 points, 7 points) displayed in the display fields 53a to 53d is multiplied by a corresponding scoring ratio to adjust the score, and each adjusted phrase Will be calculated as the total score (8.33 / 10 points).

  In addition, although the score before adjusting by the score adjustment part 33 is displayed on the display fields 53a-53d corresponding to a phrase, the score after adjusting by the score adjustment part 33 may be displayed. Alternatively, in the display columns 53a to 53d corresponding to the phrases, the score before being adjusted by the score adjusting unit 33 and the score after being adjusted by the score adjusting unit 33 may be displayed together. In addition, the display control unit 34 displays, for example, the phrase score display unit 53 corresponding to the evaluation item (intonation in this example) selected from the evaluation items displayed on the total score display unit 54 (that is, switching). Display). As another example, the screen may be provided with a phrase score display unit 53 corresponding to all evaluation items that have been evaluated.

  The total score display unit 54 displays the total score calculated for all the evaluation items that are evaluation targets and the total score calculated for the comprehensive evaluation. Of the displayed total score, the total score of the evaluation items (for example, voice volume, intonation, smooth tongue, and speed) to be adjusted is the score for each phrase adjusted by the score adjustment unit 33 as described above. The total score calculated based on this. In the total score display section 54, the total score and the total score are displayed on the right side (right side of /). In this example, the score for each of the five types of evaluation items is 20 points, and the score for comprehensive evaluation is 100 points. For this reason, in the example of FIG. 4A, the total score (8.33 / 10 points) calculated for the tongue is displayed as a doubled total score (16.7 points / 20 points). Yes.

  On the other hand, the configuration of the screen shown in FIG. 4B is basically the same as the configuration of the screen shown in FIG. 4A. However, in the display fields 55a to 55d in the phrase score display section 55, “score / In particular, the score and the score in the display fields 55b and 55d corresponding to the phrase having a relatively high weight are multiplied by the weight “2” and displayed. As a result, it is possible to explicitly indicate to the user which phrase has a high score (ie, which is important).

  The display control unit 34 divides the sentence into a plurality of clauses and displays them on the screen, and displays a relatively high weighted clause among the plurality of clauses in a display mode different from other clauses. May be. At this time, the display control unit 34 may display on the screen at least one of the score acquired for each phrase and the score adjusted for each phrase in association with each phrase. In addition, the display control unit 34 divides the sentence into a plurality of words and displays them on the screen, and among the plurality of words, a word given a relatively high importance is displayed in a display mode different from other words. It may be displayed. At this time, the display control unit 34 may display on the screen at least one of the score acquired for each word and the score adjusted for each word in association with each word.

[2. Example of operation of the reading aloud evaluation device S]
Next, an example of the operation of the reading aloud evaluation apparatus S will be described separately in the first embodiment and the second embodiment. Example 1 is an example in which the total score for reading aloud the entire sentence for a predetermined evaluation item is calculated based on the score for each phrase or phrase. Example 2 is an example in which the total score for reading aloud the entire sentence for a predetermined evaluation item is calculated based on the score for each word.

Example 1
First, with reference to FIG. 5 etc., the reading aloud evaluation process of the control part 3 in Example 1 is demonstrated. FIG. 5 is a flowchart illustrating an example of the reading aloud evaluation process of the control unit 3 according to the first embodiment. Note that, in the reading aloud evaluation process described below, a sentence including a plurality of phrases will be described as an example. Moreover, as a premise of the reading aloud evaluation process described below, calculation is performed at predetermined intervals based on the data of the reference phrase section, the reference interval section and the reference word section identified based on the reference speech waveform data, and the reference speech waveform data. The voice volume and inflection data that have been obtained and the vocal tract feature quantity (MFCC) data calculated for each reference phrase section based on the reference voice waveform data are associated with the voice file of the reference voice waveform data, for example. Are stored in the storage unit 2.

  The process shown in FIG. 5 is started, for example, when the speaker designates a desired voice file as a reference (example) of score calculation for reading aloud via the operation unit 4 and gives a start instruction. When the process shown in FIG. 5 is started, the control unit 3 turns on the microphone input, the text data of the sentence associated with the designated audio file, the reference phrase section, the reference interval section, the reference word section, Voice volume, intonation, and vocal tract feature value (MFCC) data are input from the storage unit 2 (step S1). The input data is stored in the RAM. A serial number is assigned to each of the reference phrase section and the reference interval section. The reference word section is associated with a reference phrase section including the reference word section. When the speaker starts reading the sentence, the voice generated during the reading of the sentence is collected by the microphone M, and the speaker voice waveform data indicating the waveform of the collected voice is sent to the interface unit 5. Is input to the reading aloud evaluation device S.

  As described above, the control unit 3 of the reading aloud evaluation apparatus S stores (records) the input speaker voice waveform data in the storage unit 2 and, based on the input speaker voice waveform data, as described above. Sections and speaker interval sections are sequentially identified (step S2). Serial numbers are assigned to the data of the specified speaker phrase section and speaker interval section, and are stored in the RAM. The data of each speaker phrase section and each speaker interval section stored in this way are used for evaluation described later.

  Next, as described above, the control unit 3 calculates the voice volume and the inflection every predetermined time based on the input speaker voice waveform data, and the vocal tract feature quantity (MFCC) for each speaker phrase section. ) Is calculated (step S3). The calculated voice volume, intonation and vocal tract feature quantity (MFCC) data are stored in the RAM. The voice volume, intonation, and vocal tract feature quantity (MFCC) data stored in this way are used for evaluation described later. Next, the control unit 3 evaluates the inflection by comparing the inflection of the reference phrase section and the inflection of the speaker phrase section in the order of serial numbers (step S4). As described above, the inflection score is calculated for each phrase, and is stored in the storage unit 2 in association with each phrase indicated by the text data of the sentence, for example.

  Next, the control unit 3 evaluates the voice volume by comparing the voice volume of the reference phrase section and the voice volume of the speaker phrase section in the order of serial numbers (step S5). As described above, the voice volume score for each phrase is calculated by the evaluation of the voice volume, and is stored in the storage unit 2 in association with each phrase indicated by the text data of the sentence, for example. Next, the control unit 3 compares the feature quantity (MFCC) indicating the vocal tract characteristic of the reference phrase section and the feature quantity (MFCC) indicating the vocal tract characteristic of the speaker phrase section in the order of serial numbers, and evaluates the smooth tongue. It performs (step S6). As described above, the smooth tongue score is calculated for each phrase and is stored in the storage unit 2 in association with each phrase indicated by the text data of the sentence.

  Next, the control unit 3 evaluates the speed by comparing the time length of the reference phrase section and the time length of the speaker phrase section in the order of serial numbers (step S7). As described above, the speed score for each phrase is calculated by the speed evaluation, and is stored in the storage unit 2 in association with each phrase indicated by the text data of the sentence, for example. Next, the control unit 3 compares the time length of the reference interval section and the time length of the speaker interval section in the order of serial numbers and performs an evaluation between them (step S8). As described above, the total score between all the interval sections is calculated and stored in the storage unit 2.

  Next, the control unit 3 decomposes the sentence indicated by the text data input in step S1 into a plurality of words for each phrase (step S9). Next, the control unit 3 refers to the reference information that defines the importance of the word registered in the word importance database, and identifies the importance of each word decomposed in step S9 (step S10). Next, the control unit 3 determines the importance of the phrase for each phrase, for example, as shown in FIG. 3, based on the importance of the word specified in step S10 (step S11). The importance of the phrase may be determined based on the importance of the word and the time length of the reference word section of the word (hereinafter referred to as “word length”).

  FIG. 6A is a conceptual diagram illustrating an example in which the importance of a phrase is determined based on the importance of a word and the word length. In the example of FIG. 6A, the importance levels of the plurality of words constituting the phrase F11 are “small”, “small”, and “small”, respectively. Thereby, the weights of the plurality of words constituting the phrase F11 are determined as “1”, “1”, and “1”, respectively. The word lengths of the plurality of words constituting the phrase F11 are “0.5 seconds”, “0.75 seconds”, and “0.75 seconds”, respectively. Based on the weights and word lengths of these words, the importance corresponding to the value calculated by the calculation formula (1) shown in FIG. 6A is determined as the importance of the phrase F11. . Similarly, the importance corresponding to the value calculated by the calculation formula (2) shown in FIG. 6A is determined as the importance of the phrase F12.

  Next, the control unit 3 determines the time length of the reference phrase section (the time length of the phrase specified based on the speech waveform indicated by the reference speech waveform data, from the data of the reference phrase section input in step S1, hereinafter “ (Referred to as “phrase length”) for each phrase (step S12). Subsequently, the control part 3 determines the weight of a phrase for every phrase based on the importance of the phrase determined by step S11, and the phrase length specified by step S12 (step S13). Subsequently, the control part 3 determines the score ratio according to the weight determined by step S13 for every phrase (step S14).

  FIG. 6B is a conceptual diagram illustrating an example in which phrase weights and scoring ratios are determined based on phrase importance and phrase length. The importance levels of the phrases F11 to F13 are small, large, and large, respectively. On the other hand, the phrase lengths of the phrases F11 to F13 are “2 seconds”, “1 second”, and “2 seconds”, respectively. When the weight ratio is set to “large: small = 2: 1” and “2 seconds: 1 second = 2: 1”, as shown in FIG. 6B, the entire phrases F11 to F13 The weights are determined as “2”, “2”, and “4”, respectively. Thereby, the scoring ratios according to the weights of the phrases F11 to F13 are determined to be “1/4”, “1/4”, and “1/2”, respectively.

  Note that the control unit 3 may be configured to determine the weight of the phrase for each phrase based only on the importance of the phrase determined in step S11.

  Subsequently, the control part 3 acquires the score (namely, score of an inflection, a voice volume, a smooth tongue, and speed) for every evaluation item calculated by step S4-S7, respectively (step S15). Next, the control unit 3 adjusts the score for each phrase by multiplying the score acquired in step S15 by the scoring ratio determined in step S14 (step S16). Such adjustment of the score is executed for each evaluation item.

  Next, the control unit 3 calculates a total score for reading aloud the entire sentence based on the score for each phrase adjusted in step S16 (step S17). For example, the total score adjusted for each phrase is calculated as the total score. Such calculation of the total score is executed for each evaluation item. Next, the control unit 3 calculates a total score for the reading of the whole sentence based on the total score calculated in step S8 and the total score of each evaluation item calculated in step S17 (step S18). Next, based on the information such as the scores obtained in steps S1 to S18, the control unit 3 displays information indicating evaluation of the speaker's reading aloud as shown in FIG. 4 (A) or (B). Is displayed on the display D (step S19).

  As described above, according to the first embodiment, the reading aloud evaluation device S determines a weight for each phrase (or phrase), and assigns a scoring ratio according to each determined weight for each phrase (or phrase). The score that is determined and evaluated for each phrase (or phrase) for a given evaluation item for reading aloud is adjusted for each phrase (or phrase) according to the determined ratio of points, and the score for each adjusted phrase (or phrase) is adjusted. Based on the above, the total score for reading aloud the entire sentence including a plurality of phrases (or phrases) is calculated, so that the evaluation can be performed according to whether it is an important part in context. Thereby, since the speaker (practitioner) knows the important position (phrase part etc.) in the context, he / she can know where to focus and the satisfaction of the score calculation is improved. Therefore, for example, it is possible not to practice the entire announcement uniformly, but to focus on important positions in the context.

(Example 2)
Next, with reference to FIG. 7 etc., the reading aloud evaluation process of the control part 3 in Example 2 is demonstrated. FIG. 7 is a flowchart illustrating an example of a reading aloud evaluation process performed by the control unit 3 according to the second embodiment. As a premise of the reading aloud evaluation process described below, the data of the reference word interval and the reference interval interval specified based on the reference speech waveform data, the voice volume calculated every predetermined time based on the reference speech waveform data, and Inflection data and vocal tract feature value (MFCC) data calculated for each word (speaker word interval) based on the reference speech waveform data are associated with the speech file of the reference speech waveform data, for example. Are stored in the storage unit 2.

  The process shown in FIG. 7 is started in the same manner as the process shown in FIG. When the process shown in FIG. 7 is started, the control unit 3 turns on the microphone input, the text data of the sentence associated with the designated voice file, the reference word interval, the reference interval interval, the voice volume, the inflection, and Data of the vocal tract feature value (MFCC) is input from the storage unit 2 (step S21). A serial number is assigned to each of the reference word section and the reference interval section. When the speaker starts reading the sentence, the voice generated during the reading of the sentence is collected by the microphone M, and the speaker voice waveform data indicating the waveform of the collected voice is sent to the interface unit 5. Is input to the reading aloud evaluation device S.

  The control unit 3 of the reading aloud evaluation device S stores the input speaker voice waveform data in the storage unit 2 and, as described above, based on the input speaker voice waveform data, Speaker interval intervals are sequentially identified (step S22). Serial numbers are assigned to the data of the specified speaker word section and speaker interval section, and are stored in the RAM.

  Next, as described above, the control unit 3 calculates the voice volume and the inflection every predetermined time based on the input speaker voice waveform data, and the vocal tract feature quantity (MFCC) for each speaker word section. ) Is calculated (step S23). The calculated voice volume, intonation and vocal tract feature quantity (MFCC) data are stored in the RAM. Next, the control unit 3 evaluates the inflection by comparing the inflection of the reference word section and the inflection of the speaker word section in the order of serial numbers (step S24). As described above, the inflection score for each word is calculated and stored in the storage unit 2 by the evaluation of the inflection.

  Next, the control unit 3 evaluates the voice volume by comparing the voice volume of the reference word section and the voice volume of the speaker word section in the order of serial numbers (step S25). As described above, the voice volume score for each word is calculated and stored in the storage unit 2 by the evaluation of the voice volume. Next, the control unit 3 compares the feature quantity (MFCC) indicating the vocal tract characteristic of the reference word section and the feature quantity (MFCC) showing the vocal tract characteristic of the speaker word section in order of serial numbers, and evaluates the smooth tongue. This is performed (step S26). As described above, the smooth tongue score for each word is calculated and stored in the storage unit 2 by the evaluation of the smooth tongue.

  Next, the controller 3 evaluates the speed by comparing the time length of the reference word section and the time length of the speaker word section in the order of serial numbers (step S27). As described above, the speed score for each word is calculated and stored in the storage unit 2 by the speed evaluation. Next, the control unit 3 compares the time length of the reference interval section and the time length of the speaker interval section in the order of serial numbers and performs an evaluation between them (step S28). As described above, the total score between all the interval sections is calculated and stored in the storage unit 2.

  Next, the control unit 3 breaks down the sentence indicated by the text data input in step S21 into a plurality of words (step S29). Next, the control unit 3 refers to the reference information that defines the importance of the word registered in the word importance database, and specifies the importance of each word decomposed in step S29 (step S30). Next, the control unit 3 specifies the word length for each word from the data of the reference word section input in step S21 (step S31).

  Next, the control unit 3 determines the word weight for each word based on the importance of the word determined in step S30 and the word length specified in step S31 (step S32). Subsequently, the control part 3 determines the score distribution according to the weight determined by step S32 for every word (step S33). The control unit 3 may be configured to determine the word weight for each word based only on the importance of the word determined in step S30.

  Next, the control unit 3 obtains a score for each word (that is, a score for inflection, voice volume, smooth tongue, and speed) calculated in steps S24 to S27 (step S34). Next, the control unit 3 adjusts the score for each word by multiplying the score acquired in step S34 by the score ratio determined in step S33 (step S35). Such adjustment of the score is executed for each evaluation item.

  Next, the control unit 3 calculates a total score for reading aloud the entire sentence based on the score for each word adjusted in step S35 (step S36). For example, the total score adjusted for each word is calculated as the total score. Such calculation of the total score is executed for each evaluation item.

  FIG. 8 is a conceptual diagram showing an example in which the score for each word is adjusted by the scoring ratio, and the total score for reading the whole sentence is calculated. In the example of FIG. 8, the weight ratios are set to “large: small = 2: 1” and “1.0 seconds: 0.5 seconds = 2: 1”, whereby the weights of the words W1 to W7 are set. Are determined to be “2”, “2”, “1”, “1”, “1”, “2”, and “1”, respectively. Then, the scores of the words W1 to W7 (6 points, 5 points, 4 points, 8 points, 7 points, 10 points, and 6 points out of 10 points) are respectively multiplied by the stipulation ratios of the words W1 to W7. Thus, the score is adjusted, and the total score for each adjusted word is calculated as a total score (7.23 / 10 points).

  Next, the control unit 3 calculates a total score for the reading of the whole sentence based on the total score calculated in step S28 and the total score of each evaluation item calculated in step S36 (step S37). Next, the control unit 3 causes the display D to display a screen that displays information indicating evaluation of the speaker's reading aloud based on the information such as the scores obtained in steps S21 to S37 (step S38).

  As described above, according to the second embodiment, the reading aloud evaluation device S identifies the importance for each word from the word importance database, and responds to the weight reflecting the importance for each identified word. A scoring ratio is determined for each word, and the score evaluated for each word for a predetermined evaluation item for reading aloud is adjusted for each word according to the determined scoring ratio, and a plurality of words are determined based on the adjusted score for each word. Since it is configured to calculate the total score for reading aloud for the entire sentence, it is possible to quickly identify the importance for each word from the word importance database and evaluate it according to whether it is an important part in context . As a result, the speaker knows an important position (word part) in the context, so that the speaker can know where to focus, and the satisfaction of the score calculation increases. Therefore, for example, it is possible not to practice the entire announcement uniformly, but to focus on important positions in the context.

DESCRIPTION OF SYMBOLS 1 Communication part 2 Memory | storage part 3 Control part 4 Operation part 5 Interface part 6 Bus S Reading aloud evaluation apparatus

Claims (17)

Speakers is calculated for each of the phrases on the basis of the speech waveform data representing the speech waveform emitted when reading aloud a plurality including statements phrase is a group of clause spoken in one breath Ri Do from one or more clauses An acquisition means for acquiring a score;
First determining means for determining a weight for each phrase;
Second deciding means for deciding for each phrase a scoring ratio according to the weight for each phrase decided by the first deciding means;
The score obtained by the obtaining means is adjusted for each phrase by the scoring ratio determined by the second determining means, and the entire sentence including the plurality of phrases is adjusted based on the score for each adjusted phrase. A calculation means for calculating a total score for reading aloud;
A reading aloud evaluation apparatus comprising: Scores calculated for each phrase based on speech waveform data indicating the waveform of speech uttered when a speaker reads a sentence containing a plurality of phrases that are composed of one or more phrases and are spoken at a time. First acquisition means for acquiring
First determining means for determining a weight for each phrase;
Second deciding means for deciding for each phrase a scoring ratio according to the weight for each phrase decided by the first deciding means;
A second acquisition means for acquiring a score of a gap calculated in an interval section from an end timing of any one of the phrases to a start timing of the next phrase;
The score acquired by the first acquisition means is adjusted for each phrase by the scoring ratio determined by the second determination means, and the score for each adjusted phrase is acquired by the second acquisition means. A calculation means for calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the score of the gap;
A reading aloud evaluation apparatus comprising: Display control means for dividing the sentence into the plurality of phrases and displaying the same on the screen, and displaying the phrase having a relatively high weight among the plurality of phrases in a display mode different from other phrases. reading aloud evaluation apparatus according to claim 1 or 2, further comprising. Wherein the display control unit, a front Symbol phrase acquired the score for each, and wherein the display at least one of the screen in association with each of the phrases in the adjusted score for each of the phrases The reading aloud evaluation apparatus according to claim 3 . First input means for inputting text data of the sentence;
Decomposing means for decomposing a sentence indicated by the text data input by the first input means into a plurality of words;
Referring to reference information that defines the importance of a word, first specifying means for specifying the importance of each of the decomposed words;
Further comprising
The first determining unit determines a weight of a phrase composed of the words for each phrase based on the importance of the word specified by the first specifying unit. 5. The reading aloud evaluation apparatus according to any one of 4 above. A second input means for inputting reference speech waveform data indicating a speech waveform that serves as a reference for score calculation by reading the sentence aloud;
A second specifying unit that specifies, for each phrase, a time length of the phrase specified based on a waveform of a voice indicated by the reference voice waveform data input by the second input unit; Is based on the importance of the word specified by the first specifying means and the time length of the phrase specified by the second specifying means, and the weight of the phrase composed of the words is determined for each phrase. The reading aloud evaluation apparatus according to claim 5 , wherein: First acquisition means for acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice uttered when a speaker reads a sentence including a plurality of phrases;
First determining means for determining a weight for each phrase;
Second deciding means for deciding for each clause a scoring ratio according to the weight for each clause determined by the first deciding means;
Second acquisition means for acquiring a score of a gap calculated in an interval section from an end timing of any one of the plurality of clauses to a start timing of the next clause;
The score acquired by the first acquisition unit is adjusted for each phrase by the scoring ratio determined by the second determination unit, and the score for each adjusted phrase is acquired by the second acquisition unit. A calculation means for calculating a total score for reading aloud the entire sentence including the plurality of clauses based on the score of the gap;
A reading aloud evaluation apparatus comprising: Display control means for dividing the sentence into the plurality of phrases and displaying the same on the screen, and for displaying the phrase having a relatively high weight among the plurality of phrases in a display mode different from other phrases. The reading aloud evaluation apparatus according to claim 7 , further comprising: Wherein the display control unit, before SL and the scores obtained for each clause, and characterized by displaying at least one of the screen in association with each of the clauses of the adjusted score for each of the clauses The reading aloud evaluation apparatus according to claim 8 . First input means for inputting text data of the sentence;
Decomposing means for decomposing a sentence indicated by the text data input by the first input means into a plurality of words;
Referring to reference information that defines the importance of a word, first specifying means for specifying the importance of each of the decomposed words;
Further comprising
Wherein the first determination means, based on said word importance identified by the first identification means, to claim 7 the weight of clause constituted by the word and determines for each of the clauses The reading aloud evaluation apparatus according to any one of 9 . A second input means for inputting reference speech waveform data indicating a speech waveform that serves as a reference for score calculation by reading the sentence aloud;
Second specifying means for specifying, for each phrase, a time length of the phrase specified based on a speech waveform indicated by the reference speech waveform data input by the second input means;
The first determination unit is configured to determine a phrase composed of the words based on the importance level of the word specified by the first specification unit and the time length of the phrase specified by the second specification unit. 11. The reading aloud evaluation apparatus according to claim 10 , wherein a weight is determined for each phrase. A reading aloud evaluation method executed by one or more computers,
Speakers is calculated for each of the phrases on the basis of the speech waveform data representing the speech waveform emitted when reading aloud a plurality including statements phrase is a group of clause spoken in one breath Ri Do from one or more clauses An acquisition step for acquiring a score;
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each of the phrases, a scoring ratio according to the weight of each of the phrases determined in the first determination step;
The score acquired in the acquisition step is adjusted for each phrase by the scoring ratio determined in the second determination step, and the entire sentence including the plurality of phrases is adjusted based on the adjusted score for each phrase. A calculation step for calculating a total score for reading aloud;
A reading aloud evaluation method characterized by including: A reading aloud evaluation method executed by one or more computers,
Scores calculated for each phrase based on speech waveform data indicating the waveform of speech uttered when a speaker reads a sentence containing a plurality of phrases that are composed of one or more phrases and are spoken at a time. A first acquisition step of acquiring
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each of the phrases, a scoring ratio according to the weight of each of the phrases determined in the first determination step;
A second acquisition step of acquiring a score of the interval calculated in the interval section from the end timing of any one of the phrases to the start timing of the next phrase;
The score acquired in the first acquisition step is adjusted for each phrase by the scoring ratio determined in the second determination step, and the score for each adjusted phrase is acquired in the second acquisition step. A calculation step of calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the score of the gap;
A reading aloud evaluation method characterized by including: Scores calculated for each phrase based on speech waveform data indicating the waveform of speech uttered when a speaker reads a sentence containing a plurality of phrases that are composed of one or more phrases and are spoken at a time. An acquisition step to acquire,
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each of the phrases, a scoring ratio according to the weight of each of the phrases determined in the first determination step;
The score acquired in the acquisition step is adjusted for each phrase by the scoring ratio determined in the second determination step, and the entire sentence including the plurality of phrases is adjusted based on the adjusted score for each phrase. A calculation step for calculating a total score for reading aloud;
A program that causes a computer to execute. Scores calculated for each phrase based on speech waveform data indicating the waveform of speech uttered when a speaker reads a sentence containing a plurality of phrases that are composed of one or more phrases and are spoken at a time. A first acquisition step of acquiring
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each of the phrases, a scoring ratio according to the weight of each of the phrases determined in the first determination step;
A second acquisition step of acquiring a score of the interval calculated in the interval section from the end timing of any one of the phrases to the start timing of the next phrase;
The score acquired in the first acquisition step is adjusted for each phrase by the scoring ratio determined in the second determination step, and the score for each adjusted phrase is acquired in the second acquisition step. A calculation step of calculating a total score for reading aloud the entire sentence including the plurality of phrases based on the score of the gap;
A program that causes a computer to execute. A reading aloud evaluation method executed by one or more computers,
A first acquisition step of acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice uttered when a speaker reads a sentence including a plurality of phrases;
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each phrase, a scoring ratio according to the weight of each phrase determined in the first determination step;
A second obtaining step of obtaining a score of the interval calculated in an interval section from an end timing of any one of the plurality of clauses to a start timing of the next clause;
The score acquired in the first acquisition step is adjusted for each phrase according to the scoring ratio determined in the second determination step, and the score for each adjusted phrase is acquired in the second acquisition step. A calculation step of calculating a total score for reading aloud the entire sentence including the plurality of clauses based on the score of the gap;
A reading aloud evaluation method characterized by including: A first acquisition step of acquiring a score calculated for each phrase based on voice waveform data indicating a waveform of a voice uttered when a speaker reads a sentence including a plurality of phrases;
A first determination step of determining a weight for each phrase;
A second determination step of determining, for each phrase, a scoring ratio according to the weight of each phrase determined in the first determination step;
A second obtaining step of obtaining a score of the interval calculated in an interval section from an end timing of any one of the plurality of clauses to a start timing of the next clause;
The score acquired in the first acquisition step is adjusted for each phrase according to the scoring ratio determined in the second determination step, and the score for each adjusted phrase is acquired in the second acquisition step. A calculation step of calculating a total score for reading aloud the entire sentence including the plurality of clauses based on the score of the gap;
A program that causes a computer to execute.