JP6531654B2 - Speech reading evaluation device, display control method, and program - Google Patents

Description

  The present invention relates to the technical field such as a system for evaluating the reading of a sentence based on the speech emitted when the speaker aloud the sentence.

  In recent years, for the purpose of practice support such as language learning, announcement, or singing, evaluation points such as intonation and volume are calculated for each of plural sentence elements (for example, phrases and words), and overall evaluation for reading aloud based on the result Systems for calculating and displaying points are known. For example, in the technique disclosed in Patent Document 1, a song is scored for each section of the song based on intonation, volume, etc. extracted from the singing voice signal of the singer, and the total is obtained based on the score of each section The score is displayed (see FIG. 3 of Patent Document 1).

Japanese Patent Application Laid-Open No. 10-078749

  However, in the prior art, in a state in which the overall evaluation point for the reading by the speaker is displayed, any of the sentence element sections corresponding to the sentence elements constituting the read aloud sentence, for example, any one It has been difficult to display details of evaluation points such as intonation and volume calculated in a sentence element section in a more easily viewable display mode in association with text etc. representing the sentence element in the sentence element section.

  The present invention has been made in view of the above points, and, for example, it is a speech reading evaluation that can display details of evaluation points calculated in any section that the speaker particularly wants to confirm in a more easily viewable display mode An apparatus, a display control method, and a program are provided.

In order to solve the above-mentioned problems, the invention according to claim 1 is characterized in that each sentence constituting the read speech is based on speech waveform data showing a waveform of speech emitted when the speaker aloud the sentence At least one of a sentence element section from the start timing to the end timing of the element and an interval section from the end timing of any of the sentence elements of the plurality of sentence elements to the start timing of the next sentence element Comprehensive evaluation point calculation means for calculating an overall evaluation point for the speech based on the evaluation point calculation means for calculating an evaluation point for the phonetic reading for each section and the evaluation point for each section calculated by the evaluation point calculation means And at least a portion of the text representing each sentence element constituting the aloud sentence in the first display area of a length corresponding to the time length of each of the sentence element sections on the time axis And at least one of a pitch and a sound pressure specified on the basis of the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section. A second display control means for displaying a graph representing a time-series change of one sound element in the second display area along the time axis, and an overall evaluation point calculated by the overall evaluation point calculation means A third display control means for displaying in a third display area, a space area located along the time axis between the first display area, the second display area, and the plurality of first display areas If any one of the regions is designated by the user operation, the first window displaying the evaluation point calculated in the section corresponding to the designated one of the regions is popped up . Said If the display area of is designated by a user operation, and the fourth display control means for the third window to see all of the text at least part of which is displayed in the specified the first display area is pop up , And.

According to the second aspect of the present invention, from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence Score for the reading aloud at least one of the sentence element section and the interval section from the end timing of the sentence element of any of the plurality of sentence elements to the start timing of the next sentence element Evaluation point calculating means for calculating a score, comprehensive evaluation point calculating means for calculating an overall evaluation point for the reading on the basis of the evaluation points for each of the sections calculated by the evaluation point calculating means, At least a portion of the text representing each of the constituent sentence elements is listed for each of the sentence element intervals along the time axis in the first display area of a length corresponding to the time length of each of the sentence element intervals. Time-series of at least one sound element of a pitch and a sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section; A second display control means for displaying a graph showing various changes in the second display area along the time axis, and a third display area for displaying the comprehensive evaluation point calculated by the comprehensive evaluation point calculation means One of the three display control means, the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas Fourth display control means for pop-up displaying a first window for displaying the evaluation point calculated in the section corresponding to the designated one of the areas when designated by a user operation; versus The evaluation points corresponding to each of at least one of intonation, volume, glide tongue, and speed set in advance as the evaluation items, the evaluation points being calculated based on the evaluation points calculated for each of the sentence element sections Interval evaluation point calculation means for calculating an interval evaluation point for the reading aloud for each sentence element interval, and an icon representing each interval evaluation point calculated by the interval evaluation point calculation means, for each of the first display areas And a fifth display control unit configured to display in the fourth display area arranged corresponding to each of the sentence element sections along the time axis, and the fourth display control unit is configured such that the icon is operated by a user operation. When it is designated, it is an evaluation point calculated in the sentence element section corresponding to the designated icon, which is previously set as an evaluation item for the reading aloud, volume, slip tongue, and A fourth window pops up to display the evaluation point for at least one of the speeds .

According to the third aspect of the present invention, from the start timing to the end timing of each sentence element constituting the read sentence on the basis of voice waveform data indicating the waveform of the voice emitted when the speaker aloud the sentence Score for the reading aloud at least one of the sentence element section and the interval section from the end timing of the sentence element of any of the plurality of sentence elements to the start timing of the next sentence element Evaluation point calculating means for calculating a score, comprehensive evaluation point calculating means for calculating an overall evaluation point for the reading on the basis of the evaluation points for each of the sections calculated by the evaluation point calculating means, At least a portion of the text representing each of the constituent sentence elements is listed for each of the sentence element intervals along the time axis in the first display area of a length corresponding to the time length of each of the sentence element intervals. Time-series of at least one sound element of a pitch and a sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section; A second display control means for displaying a graph showing various changes in the second display area along the time axis, and a third display area for displaying the comprehensive evaluation point calculated by the comprehensive evaluation point calculation means One of the three display control means, the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas And fourth display control means for pop-up displaying a first window for displaying the evaluation point calculated in the section corresponding to the designated one of the areas when designated by a user operation; Said [4] The display control means causes the first window to be popped up by superimposing the mouse pointer on the one area in response to the user operation, and the mouse pointer is separated from the one area. The first window is erased and the display of the first window is continued by clicking the operation button of the mouse in a state where the mouse pointer is superimposed on the one area, and the first window is erased. The evaluation point calculated in the section corresponding to the other area is displayed by superimposing the mouse pointer on the other area different from the one area while the display of the window is continued. The second window to be displayed is popped up, and the mouse pointer is moved away from the other area. (C) erasing the image, and continuing the display of the second window by clicking the operation button of the mouse while the mouse pointer is superimposed on the other area .

According to a fourth aspect of the present invention, in the speech reading evaluation apparatus according to the third aspect, the fourth display control means sets the first area of the mouse in a case where the display of the first window continues. It is characterized in that the first window is erased by clicking again on the operation button of the mouse while the pointer is superimposed.

The invention according to claim 5 is the speech reading evaluation apparatus according to any one of claims 1 to 4, wherein the fourth display control means is a user of the first display area or the second display area. When it is designated by an operation, it is an evaluation point calculated in the section corresponding to the designated first display area or the second display area, which is previously set as an evaluation item for the reading aloud Pop-up a first window displaying the evaluation points for at least one of volume, tongue, and speed .

The invention according to claim 6 is the speech reading evaluation apparatus according to any one of claims 1 to 4 , wherein the fourth display control means is specified when the space area is specified by a user operation. And a pop-up display of a first window for displaying the evaluation point corresponding to the interval set in advance as the evaluation item for the reading aloud, the evaluation point being calculated in the section corresponding to the space area. .

The invention according to claim 7 is a display control method executed by one or more computers, wherein the reading of the sound is performed based on voice waveform data indicating a waveform of a sound emitted when the speaker aloud the sentence. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence, and an interval from the end timing of any sentence element of the plurality of sentence elements to the start timing of the next sentence element Based on the evaluation point for each section calculated at the evaluation point calculation step for calculating the evaluation point for the reading aloud for at least one of the sections with the section, and the comprehensive evaluation point for the reading aloud Calculating an overall evaluation point for calculating at least a part of the text representing each of the sentence elements constituting the sound-read sentence, The first display control step of displaying in the first display area of a length according to the time length for each sentence element section along the time axis, and the voice at predetermined time intervals shorter than the time length of the sentence element section Second display control for displaying a graph representing time-series change of at least one sound element of a pitch and a sound pressure specified based on waveform data in the second display area along the time axis A third display control step of displaying a total evaluation point calculated in the total evaluation point calculation step in a third display area, the first display area, the second display area, and a plurality of the above When any one of the space regions located along the time axis between the first display regions is designated by the user operation, calculation is performed in the section corresponding to the designated one of the regions Said When the first window displaying the valence point is popped up and the first display area is designated by the user operation, all of the text at least a part of which is displayed in the designated first display area And a fourth display control step of pop-up displaying a third window for displaying. The invention according to claim 8 is a display control method executed by one or more computers , wherein the reading of the sound is performed based on voice waveform data indicating a waveform of the sound emitted when the speaker aloud the sentence. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence, and an interval from the end timing of any sentence element of the plurality of sentence elements to the start timing of the next sentence element Based on the evaluation point for each section calculated at the evaluation point calculation step for calculating the evaluation point for the reading aloud for at least one of the sections with the section, and the comprehensive evaluation point for the reading aloud Calculating an overall evaluation point for calculating at least a part of the text representing each of the sentence elements constituting the sound-read sentence, A first display control step of displaying in the first display area of a length corresponding to the time length for each of the sentence element sections along the time axis, and the predetermined time interval shorter than the time length of the sentence element sections A second display for displaying a graph representing a time-series change of at least one sound element of a pitch and a sound pressure specified based on audio waveform data in the second display area along the time axis A control step, a third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculation step in a third display region, the first display region, the second display region, and a plurality of Of the space areas located along the time axis between the first display areas, when any one of the space areas is designated by a user operation, in the section corresponding to the designated one of the areas The above calculated A fourth display control step of pop-up displaying a first window for displaying a valence point, and at least one of intonation, volume, glide tongue, and speed preset as evaluation items for the reading aloud A section evaluation point calculating step of calculating a section evaluation point for the reading aloud for each of the sentence element sections based on the evaluation points which are the evaluation points and calculated for each of the sentence element sections; A fifth display area for displaying each of the section evaluation points calculated in the step, for each of the sentence element sections along the time axis in a fourth display area arranged corresponding to each of the first display areas; A display control step, and in the fourth display control step, when the icon is designated by a user operation, the icon corresponds to the designated icon The evaluation points calculated in the sentence element section, the evaluation points for at least one of intonation, volume, syntony, and speed preset as evaluation items for the reading aloud It is characterized in that the window 4 is popped up. The invention according to claim 9 is a display control method executed by one or more computers , wherein the reading of the sound is performed based on voice waveform data indicating a waveform of the sound emitted when the speaker aloud the sentence. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence, and an interval from the end timing of any sentence element of the plurality of sentence elements to the start timing of the next sentence element Based on the evaluation point for each section calculated at the evaluation point calculation step for calculating the evaluation point for the reading aloud for at least one of the sections with the section, and the comprehensive evaluation point for the reading aloud Calculating an overall evaluation point for calculating at least a part of the text representing each of the sentence elements constituting the sound-read sentence, A first display control step of displaying in the first display area of a length corresponding to the time length for each of the sentence element sections along the time axis, and the predetermined time interval shorter than the time length of the sentence element sections A second display for displaying a graph representing a time-series change of at least one sound element of a pitch and a sound pressure specified based on audio waveform data in the second display area along the time axis A control step, a third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculation step in a third display region, the first display region, the second display region, and a plurality of Of the space areas located along the time axis between the first display areas, when any one of the space areas is designated by a user operation, in the section corresponding to the designated one of the areas The above calculated And a fourth display control step of pop-up displaying a first window for displaying a valence point, and in the fourth display control step, a mouse pointer is superimposed on the one area in accordance with the user operation. Causes the first window to be popped up, and the mouse pointer is erased from the one area to cause the first window to disappear, and the mouse pointer is superimposed on the one area. The display of the first window is continued by clicking on the operation button of the mouse, and while the display of the first window continues, in another area different from the one area. A second display of the evaluation point calculated in the section corresponding to the other area by superimposing a mouse pointer When the mouse is popped up and the mouse pointer leaves the other area, the second window is erased, and the mouse operation button is in a state where the mouse pointer is superimposed on the other area. The display of the second window is continued by being clicked.

The invention according to claim 10 is characterized in that, from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating the waveform of the speech emitted when the speaker read the sentence aloud Score for the reading aloud at least one of the sentence element section and the interval section from the end timing of the sentence element of any of the plurality of sentence elements to the start timing of the next sentence element Evaluation point calculating step for calculating a total evaluation point calculating step for calculating the comprehensive evaluation point for the reading on the basis of the evaluation point for each section calculated by the evaluation point calculating step; At least a portion of the text representing each of the constituent sentence elements in the first display area of a length corresponding to the time length of each of the sentence element sections along the time axis A first display control step of displaying each raw segment, and at least one of a pitch and a sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section A second display control step of displaying a graph representing a time-series change of an element in a second display area along the time axis; and a third display of the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step Any of a third display control step of displaying in an area, the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas or if one area is designated by the user operation, to pop up a first window displaying the evaluation points calculated in the section corresponding to the specified the any one of the regions, before If the first display area is designated by a user operation, the fourth display control to pop up a third window that displays all of the text at least part of which is displayed in the specified the first display regions And causing the computer to execute the steps. The invention according to claim 11 is from the start timing to the end timing of each sentence element constituting the read sentence based on voice waveform data indicating the waveform of the voice emitted when the speaker read the sentence aloud Score for the reading aloud at least one of the sentence element section and the interval section from the end timing of the sentence element of any of the plurality of sentence elements to the start timing of the next sentence element Evaluation point calculating step for calculating a total evaluation point calculating step for calculating the comprehensive evaluation point for the reading on the basis of the evaluation point for each section calculated by the evaluation point calculating step; At least a portion of the text representing each of the constituent sentence elements in the first display area of a length corresponding to the time length of each of the sentence element sections along the time axis A first display control step of displaying each raw segment, and at least one of a pitch and a sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section A second display control step of displaying a graph representing a time-series change of an element in a second display area along the time axis; and a third display of the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step Any of a third display control step of displaying in an area, the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas The first window for displaying the evaluation point calculated in the section corresponding to the specified one area is popped up when the area is specified by a user operation. A display control step, the evaluation point corresponding to each of at least one of an inflection, a volume, a tongue, and a speed preset as evaluation items for the reading aloud, calculated for each of the sentence element sections A section evaluation point calculating step of calculating a section evaluation point for the reading aloud for each sentence element section based on the evaluation points, and an icon representing each section evaluation point calculated in the section evaluation point calculating step; A program for causing a computer to execute a fifth display control step of causing a fourth display area arranged corresponding to each first display area to be displayed for each sentence element section along the time axis, In the fourth display control step, when the icon is designated by a user operation, the sentence element section corresponding to the designated icon is displayed. Pops up a fourth window for displaying the evaluation points for at least one of intonation, volume, glide tongue, and speed preset as evaluation items for the reading aloud. It is characterized in that it is displayed. The invention according to claim 12 is from the start timing to the end timing of each sentence element constituting the read sentence based on voice waveform data indicating the waveform of the voice emitted when the speaker aloud the sentence Score for the reading aloud at least one of the sentence element section and the interval section from the end timing of the sentence element of any of the plurality of sentence elements to the start timing of the next sentence element Evaluation point calculating step for calculating a total evaluation point calculating step for calculating the comprehensive evaluation point for the reading on the basis of the evaluation point for each section calculated by the evaluation point calculating step; At least a portion of the text representing each of the constituent sentence elements in the first display area of a length corresponding to the time length of each of the sentence element sections along the time axis A first display control step of displaying each raw segment, and at least one of a pitch and a sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element section A second display control step of displaying a graph representing a time-series change of an element in a second display area along the time axis; and a third display of the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step Any of a third display control step of displaying in an area, the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas The first window for displaying the evaluation point calculated in the section corresponding to the specified one area is popped up when the area is specified by a user operation. A program for causing a computer to execute a display control step, wherein, in the fourth display control step, a mouse pointer is superimposed on the one area according to the user operation, and the first window Is popped up and the mouse pointer is moved away from the one area to erase the first window, and the mouse operation button is clicked while the mouse pointer is superimposed on the one area. To continue the display of the first window, and while the display of the first window continues, the mouse pointer is superimposed on another area different from the one area. Pops up a second window displaying the evaluation points calculated in the section corresponding to the other area by The mouse operation button is clicked in a state where the second window is erased when the mouse pointer leaves the other area, and the mouse pointer is superimposed on the other area. To continue the display of the second window.

According to the first, seventh and tenth aspects of the invention, even if the text does not fit in the display area, it is possible to allow the speaker or the like to grasp the content of the sentence element represented by the text .

According to the second , eighth and eleventh aspects of the present invention, it is possible to make the speaker or the like effectively grasp the breakdown of the section evaluation point indicated by the icon pattern .

According to the inventions described in claims 3 , 4, 9 and 12 , it becomes possible for the speaker etc. to compare the evaluation points for each section, and it is at first glance to the speaker etc. in which section the evaluation is good or bad. It can be made to grasp .

According to the fifth aspect of the present invention, the details of the evaluation points calculated in any sentence element section that the speaker or the like particularly wants to confirm are associated with the text or the like representing the sentence element in the sentence element section. It can be displayed in an easy-to-see display mode .

According to the sixth aspect of the present invention, the details of the evaluation point calculated in any interval section that the speaker or the like particularly wants to check can be displayed in a more easily viewable display mode .

It is a figure which shows the example of a outline | summary structure of the reading aloud evaluation apparatus S which concerns on this embodiment. It is a figure which shows the example of a display of the display screen by which the model display area 51, the speaker display area 52, and the comprehensive evaluation point display area 53 grade | etc., Are arrange | positioned. It is a figure which shows the example of a display of the display screen by which the model display area 51, the speaker display area 52, and the comprehensive evaluation point display area 53 grade | etc., Are arrange | positioned. It is a figure which shows the example of a display of the display screen by which the model display area 51, the speaker display area 52, and the comprehensive evaluation point display area 53 grade | etc., Are arrange | positioned. 5 is a flowchart showing a reading aloud evaluation display process of the control unit 3 in the first embodiment. 10 is a flowchart showing a reading aloud evaluation display process of the control unit 3 in the second embodiment.

  Hereinafter, embodiments of the present invention will be described based on the drawings.

[1. Configuration and Function of Reading Evaluation Apparatus S]
First, with reference to FIG. 1, the configuration and function of a reading assessing apparatus S according to an embodiment of the present invention will be described. FIG. 1 is a diagram showing an example of a schematic configuration of a reading assessing apparatus S according to the present embodiment. In addition, a personal computer, a portable information terminal (smartphone etc.), etc. are mentioned as an example of a reading aloud evaluation apparatus. As shown in FIG. 1, the reading evaluation apparatus S includes a communication unit 1, a storage unit 2, a control unit 3, an operation unit 4, an interface (IF) unit 5, and the like. It is connected to the. The operation unit 4 receives an operation (user operation) from a user, and outputs a signal corresponding to the user operation to the control unit 3. An example of user operation is mouse operation. When the display D is a touch panel display, the user operation may be a touch operation with a user's finger or a pen. A microphone M, a display D, and the like are connected to the interface unit 5. The microphone M collects a voice emitted when a speaker who is a practitioner who performs language learning, an utterance, a speech practice such as reading, etc. reads a sentence (sentence) including a plurality of sentence elements. A sentence element is a unit that constitutes a sentence. Examples of sentence elements include phrases, clauses, words, and combined phrases in which a plurality of phrases are combined as described later. Here, a phrase is generally a unit of reading in a single breath when reading a sentence. A phrase consists of one or more clauses. That is, one phrase may be composed of one clause or one phrase may be composed of a plurality of clauses. A clause is, for example, a group of one or more words. The words include independent words such as nouns, verbs, adjectives, adverbs, and conjunctions (parts of speech that can compose a clause alone), and appendages such as auxiliary verbs and particles (parts of speech that can not compose a clause alone). Examples of sentences to be read aloud include sentences used in language learning, announcements, readings and the like. The display D displays a display screen on which a display area or the like to be described later is arranged in accordance with a display command from the control unit 3. The microphone M and the display D may be integrated with or separately from the reading evaluation device S.

  The communication unit 1 communicates with a server or the like by connecting to a network (not shown) by wire or wirelessly. The storage unit 2 includes, for example, a hard disk drive, and stores an OS (Operating System), a reading and reading evaluation display processing program (an example of a program of the present invention), and the like. The reading aloud evaluation display processing program is a program that causes the control unit 3 as a computer to execute a reading aloud evaluation display processing described later. The reading aloud evaluation display processing program may be downloaded from a predetermined server as an application, or may be stored and provided on a recording medium such as a CD or a DVD. In addition, the storage unit 2 stores text data of a sentence including a plurality of sentence elements, and model speech waveform data indicating a waveform of speech serving as a model when reading a sentence aloud. Here, in the text data, for example, texts (characters) representing each sentence element constituting the sentence to be read aloud are divided and defined for each sentence element. For example, it is separated by punctuation marks inserted between sentence elements. Alternatively, serial numbers may be assigned to the text representing sentence elements in order from the beginning. The sample speech waveform data is stored in a predetermined speech file format.

  The control unit 3 is configured by a central processing unit (CPU) as a computer, a read only memory (ROM), a random access memory (RAM), and the like. The control unit 3 functions as an audio processing unit 31, a read aloud evaluation unit 32, and a display processing unit 33 according to the aloud evaluation display processing program. The reading aloud evaluation unit 32 is an example of evaluation point calculation means, section evaluation point calculation means, and comprehensive evaluation point calculation means in the present invention. The display processing unit 33 is an example of a first display control unit, a second display control unit, a third display control unit, a fourth display control unit, and a fifth display control unit in the present invention.

  The voice processing unit 31 inputs model voice waveform data stored in a predetermined voice file format from the storage unit 2 as a processing target. The input sample speech waveform data is stored in the RAM. Further, the voice processing unit 31 inputs speaker voice waveform data indicating a waveform of a voice emitted when the speaker aloud the sentence and is a voice collected by the microphone M. The input speaker voice waveform data is stored in the RAM. The audio waveform data is discrete time-series sound pressure waveform data, and is, for example, waveform data of sampling rate 44.1 kHz, quantization 16 bits, and monaural. The sound pressure refers to the change (Pa) in the pressure of air due to sound waves. In the present embodiment, as the sound pressure, a sound pressure level (dB) representing the magnitude of the effective sound pressure (Pa), which is the root mean square (RMS) of the instantaneous sound pressure (Pa) Do. Sound pressure level (dB) is also referred to as volume in a broad sense.

  The speech processing unit 31 specifies, for each sentence element, a handwritten text element section from the start timing to the end timing of each sentence element based on the sample speech waveform data. Then, the voice processing unit 31 associates the text representing each sentence element with the hand-text element area data indicating the hand-text element area specified for each sentence element, and stores it in the RAM. The text representing these sentence elements is extracted from, for example, text data associated with the sample speech waveform data to be processed. The hand text element section data includes, for example, a serial number indicating the order from the beginning of the specified hand text element section, and a time range of the hand text element section (for example, 01: 00-03: 00). Be done. Similarly, the speech processing unit 31 specifies, for each sentence element, a speaker sentence element section from the start timing to the end timing of each sentence element based on the speaker speech waveform data. Then, the speech processing unit 31 stores, in the RAM, the speaker sentence element section data indicating the speaker sentence element section specified for each sentence element in association with the text indicating each sentence element. The text representing these sentence elements is extracted from, for example, a phoneme specified from the speech waveform indicated by the speaker speech waveform data. There are three phoneme examples: vowel only, consonant only, and a combination of consonant and vowel. In addition, since the identification method of a phoneme is known by the labeling method etc., detailed description is abbreviate | omitted. In addition, the speaker sentence element section data is constituted of, for example, a serial number indicating an order from the beginning of the specified speaker sentence element section, and a time range of the speaker sentence element section.

  Here, each of the start timing and the end timing may be recognized from the sound waveform or may be recognized from the sound pressure level (dB) calculated as described above. For example, the voice processing unit 31 recognizes a point in time when the amplitude of the voice waveform reaches a predetermined value or more as the start timing. Alternatively, the audio processing unit 31 recognizes a point in time when the sound pressure level (dB) becomes equal to or higher than a predetermined value as the start timing. Also, for example, the voice processing unit 31 recognizes a point in time when the amplitude width of the voice waveform becomes less than a predetermined value as the end timing. Alternatively, the sound processing unit 31 recognizes a point in time when the sound pressure level (dB) becomes less than a predetermined value as the end timing. Note that, for example, only when the time (silence time) from when the sound pressure level (dB) becomes lower than the predetermined value to when the sound pressure level (dB) becomes higher than the predetermined value is the threshold or more, It is preferable that the time when the sound pressure level (dB) falls below a predetermined value is recognized as the end timing, and the time when the sound pressure level (dB) exceeds the predetermined value is recognized as the start timing (the sound waveform The same is true for the amplitude). This is the effect that if the silence time is shorter than the threshold, the sentence element is not separated in that section.

  By the way, if you have voice waveform data of a model that you can decipher the segment slowly and say "inside the car (set up) (set up) please refrain from making calls (set up) in silent mode (set up), start timing according to the above method And the end timing, "in the car," "mobile phone is", "set in manner mode", "please refrain from calling", etc., corresponding to the text body elements of each of the four phrases It is divided into sections and identified. In addition, when the speaker reads the same sentence in the same way as the example sentence, breaks and reads aloud, “in the car” and “mobile phone” by recognizing the start timing and the end timing by the above method. For example, "set in manner mode" and "please refrain from calling", each of the four phrases is identified and divided into corresponding speaker sentence element sections. On the other hand, the speaker, for example, in the above sentence, "Please set to silent mode" and "Please refrain from calling" in a short pause, "Please refrain from calling on silent mode" In the case of reading aloud, this part becomes a phrase read in a single breath, and when the start timing and the end timing are recognized by the above method, the above-mentioned part is not particularly divided, and the speaker sentence element section is specified. As described above, when a plurality of phrases read aloud by the model correspond to one phrase read aloud by the speaker, the hand-text element section corresponding to the phrase read aloud by the model and the speaker It becomes difficult to compare with the speaker sentence element section of the aloud phrase. Therefore, in such a case, the voice processing unit 31 sets a plurality of phrases read aloud by the speaker ("Please refrain from calling after setting in the manner mode") to the phrases read aloud by the model. It is preferable to identify speaker sentence element intervals by dividing into clauses or words.

  Contrary to the above, for example, if there is voice waveform data of a model that reads aloud in part quickly, such as "Please set the mobile phone to silent mode in the car (in the car)" By recognizing the start timing and the end timing in, the hand text element section corresponding to each of two phrases is separated as "in the car" and "Please refrain from calling after setting the mobile phone in the manner mode". Be identified. On the other hand, if the speaker splits aloud and reads slowly in the car, "Please refrain from making calls in the car (set up) (set up) in the mobile mode (set up)" (set up) in the car mode, start timing and end according to the above method Recognizing the timing, the speaker sentence element interval corresponding to every 4 phrases such as "in the car", "mobile phone", "set in manner mode", "please refrain from calling" and so on. It will be separated and identified. As described above, even when one phrase read aloud according to the model corresponds to a plurality of phrases read aloud by the speaker, the hand-text element section corresponding to the phrase read aloud by the model, and the speaker Makes it difficult to compare the speaker sentence element section of the aloud phrase. Therefore, in such a case, the voice processing unit 31 may, for example, set "mobile phone is" and "set in manner mode" and "please refrain from calling" to match the phrase read aloud by the model. It is preferable to specify a speaker sentence element interval corresponding to a combined phrase including three phrases.

  Further, the voice processing unit 31 specifies a model interval section from the end timing of any one sentence element of the plurality of sentence elements to the start timing of the next sentence element based on the model voice waveform data. Then, the audio processing unit 31 stores, in the RAM, model interval section data indicating the specified model interval section. A serial number is assigned to the identified sample interval section in order from the top, for example. Similarly, the voice processing unit 31 specifies a speaker interval section from the end timing of any one sentence element of the plurality of sentence elements to the start timing of the next sentence element based on the speaker voice waveform data. Then, the speech processing unit 31 stores, in the RAM, speaker interval section data indicating the specified speaker interval section. A serial number is given to the identified speaker interval section in order from the head, for example.

  Further, the sound processing unit 31 specifies a sound pressure level (dB) as a model sound pressure from data sampled at predetermined time intervals from the sample sound waveform data at predetermined time intervals. Then, the audio processing unit 31 stores, in the RAM, model sound pressure data indicating the model sound pressure specified at each predetermined time interval. Similarly, the voice processing unit 31 specifies the sound pressure level (dB) as the speaker sound pressure at predetermined time intervals from the data extracted at predetermined time intervals from the speaker voice waveform data. Then, the voice processing unit 31 stores, in the RAM, speaker sound pressure data indicating the speaker sound pressure specified at predetermined time intervals. Further, the voice processing unit 31 calculates a fundamental frequency (Hz) from data extracted at predetermined time intervals from the model voice waveform data, and specifies the calculated fundamental frequency (Hz) as a model pitch at predetermined time intervals. . Note that, as a method of specifying the pitch (also referred to as intonation or pitch), for example, a known method such as a zero crossing method or vector autocorrelation can be applied. Then, the voice processing unit 31 stores, in the RAM, model pitch data indicating a model pitch specified at each predetermined time interval. Similarly, the voice processing unit 31 calculates a fundamental frequency (Hz) from data extracted at predetermined time intervals from the speaker voice waveform data, and uses the calculated fundamental frequency (Hz) as a speaker's voice pitch at predetermined time intervals. Identify. Then, the voice processing unit 31 stores, in the RAM, speaker pitch data indicating the speaker pitches specified at predetermined time intervals. The predetermined time for specifying the sound pressure and the sound pitch is shorter than the time length (temporal length) of the sentence element section, and is set to, for example, about 10 ms.

  Further, the voice processing unit 31 divides the data extracted from the sample voice waveform data at predetermined time intervals by windowing (for example, framing every 25 ms), and obtains an amplitude spectrum by Fourier analysis (FFT). Then, the voice processing unit 31 calculates a MFCC (mel frequency cepstrum coefficient) by applying a mel filter bank to the obtained amplitude spectrum and performing discrete cosine transformation (DCT) on a value obtained by logarithmically converting the output of the mel filter bank. Then, it is specified for each hand text region as a feature indicating the vocal tract characteristics of the example. Then, the voice processing unit 31 stores, in the RAM, model feature quantity data that indicates the feature quantity that indicates the vocal tract characteristics of the model that has been identified for each of the handwritten text element sections. Similarly, the voice processing unit 31 divides the data extracted from the speaker voice waveform data at predetermined time intervals by windowing, and obtains an amplitude spectrum by Fourier analysis. Then, the voice processing unit 31 applies a mel filter bank to the obtained amplitude spectrum, and discrete cosine transforms a value obtained by converting the output of the mel filter bank into a logarithmic value to calculate MFCC, whereby the vocal tract characteristic of the speaker is obtained. The feature amount to be indicated is specified for each speaker sentence element section. Then, the speech processing unit 31 stores, in the RAM, the speaker feature amount data indicating the feature amount indicating the vocal tract characteristics of the speaker specified for each speaker sentence element section.

  Next, the reading aloud evaluation unit 32 (an example of the evaluation point calculation means) generates a speaker for each of at least one of the speaker sentence element section and the speaker interval section, and for each evaluation item set in advance. Calculate the evaluation score for reading aloud. Here, examples of evaluation items in the speaker sentence element section include intonation, volume, syntony, and speed. Further, as an example of the evaluation item in the speaker interval section, a gap may be mentioned. For example, the reading evaluation unit 32 determines the difference between the sample pitch specified by the speech processing unit 31 and the speaker pitch for each sentence element section (that is, the text portion and the speaker sentence element having the same serial number). Based on the calculated difference calculated for each section, evaluation points for the speaker's intonation are calculated for each sentence element section. The evaluation score is calculated, for example, such that 30 points are full marks, and the closer the difference is to 0, the higher (closer to full marks). In addition, the reading evaluation unit 32 calculates the difference between the model sound pressure and the speaker sound pressure for each sentence element section, and calculates an evaluation point for the volume of the speaker for each sentence element section based on the calculated difference. . This evaluation score is calculated, for example, so that 30 points are full marks, and the closer the difference is to 0, the higher. Further, the reading aloud evaluation unit 32 calculates, for each sentence element section, the similarity between the feature amount indicating the vocal tract characteristics of the model and the feature amount indicating the vocal tract characteristics of the speaker, and based on the calculated similarity, An evaluation point for the speaker's tongue tongue is calculated for each sentence element interval. This evaluation score is calculated, for example, so that 30 points are full marks, and the higher the degree of similarity, the higher. Further, the reading evaluation unit 32 calculates the time difference between the time length of the hand text element section and the time length of the speaker sentence element section for each sentence element section, and based on the calculated absolute value of the time difference, the speed of the speaker An evaluation point for (reading speed) is calculated for each sentence element section. This evaluation point is calculated, for example, so that 30 points are full marks, and the closer the absolute value of the time difference is to 0, the higher. Further, the reading evaluation unit 32 calculates the time difference between the time length of the sample interval section and the time length of the speaker interval section for each interval section, and based on the calculated absolute value of the time difference, the speaker interval is calculated. Evaluation points are calculated for each interval section. This evaluation point is calculated, for example, so that 30 points are full marks, and the closer the absolute value of the time difference is to 0, the higher.

  The reading evaluation unit 32 (an example of a section evaluation point calculation means) calculates a section evaluation point for the speaker's reading aloud for each sentence element section based on the evaluation points for each evaluation item calculated for each sentence element section. calculate. For example, when the evaluation items are intonation and volume, the reading evaluation unit 32 calculates an average value (may be a total value) of the evaluation point for intonation and the evaluation point for volume as a section evaluation point for each sentence element section Do. In addition, when the evaluation item is only intonation, the reading evaluation unit 32 may use an evaluation point for intonation as a section evaluation point. In addition, the reading aloud evaluation unit 32 (an example of the integrated evaluation point calculation means) calculates the integrated evaluation points by evaluation item in the full text element section based on the evaluation points classified by evaluation item for each sentence element section calculated as described above. Calculate For example, the reading aloud evaluation unit 32 calculates an average value (or a total value) of evaluation points for each evaluation item calculated for each sentence element section as a comprehensive evaluation point for each evaluation item in the full text element section. Further, the reading aloud evaluation unit 32 calculates a comprehensive evaluation point for the gaps in all the interval sections based on the evaluation points for each interval section calculated as described above. For example, the reading aloud evaluation unit 32 calculates the average value or the total value of the evaluation points for the gaps calculated for each interval section as a total evaluation point for the gaps in all the interval sections. Then, the read aloud evaluation unit 32 (an example of the integrated evaluation point calculating means) calculates the total value (may be an average value) of the integrated evaluation points by evaluation items in the full text element section and the integrated evaluation points for intervals in all interval sections. , It is calculated as a comprehensive evaluation point in all sections (that is, sentence element sections and interval sections).

  Next, the display processing unit 33 displays a model display area in which information on a voice that is a model when reading a sentence aloud is displayed, and a speech in which information on a voice emitted when the speaker aloud the sentence is displayed A display screen is displayed on which the person display area, the comprehensive evaluation point display area where the comprehensive evaluation point for the speaker's aloud reading is displayed, and the like are arranged. FIGS. 2 to 4 are diagrams showing display examples of a display screen on which a sample display area 51, a speaker display area 52, an overall evaluation point display area 53 and the like are arranged. On the display screen shown in FIG. 2A, by setting the time axis t to, for example, the horizontal axis (X axis), the vertical display direction in which the model display area 51 and the speaker display area 52 are orthogonal to the time axis t Are arranged side by side). At this time, in the sample display area 51, the display processing unit 33 has a length corresponding to the time length of each hand-text element section at least a part of the text representing each sentence element constituting the sentence to be read aloud The model text display areas 51a1 to 51a3 are displayed along the time axis t for each hand-text element section. Here, the texts displayed in the model text display areas 51a1 to 51a3 are texts associated with the body-of-text element zone data of the respective body-of-text element zones.

  Further, the display processing unit 33 (an example of the first display control means) is configured to, in the speaker display area 52, at least a part of the text representing each sentence element constituting the sentence read aloud by the speaker The speaker text display area (an example of the first display area) 52a1 to 52a3 having a length corresponding to the time length of the sentence element section is displayed for each speaker sentence element section along the time axis t. Here, the texts displayed in the contours of the speaker text display areas 52a1 to 52a3 are texts associated with the speaker sentence element segment data of the respective speaker sentence element segments. Also, for example, the length D1 (in other words, the number of pixels) in the direction parallel to the time axis t of the speaker text display area 52a1 corresponds to the time length of the speaker sentence element section corresponding to the speaker text display area 52a1. The length is set. That is, as the time length of the sentence element section is longer, the length of the text display area corresponding to this is set longer in distance. If the text does not fit within the text display area, in the present embodiment, as in the speaker text display area 52a3 shown in FIG. 2A, the text in the portion extending out of the speaker text display area 52a3 is not displayed. Configured Further, as shown in FIG. 2A, a speaker space area 52s1 is located along the time axis t between the speaker text display area 52a1 and the speaker text display area 52a2. Further, a speaker space area 52s2 is located along the time axis t between the speaker text display area 52a2 and the speaker text display area 52a3. For example, the length D2 in the direction parallel to the time axis t of the speaker space area 52s1 is set to a length corresponding to the time length of the speaker interval section corresponding to the speaker space area 52s1. That is, as the time length of the speaker interval section is longer, the length of the corresponding speaker space area is set longer in distance.

  In addition, the display processing unit 33 (an example of the fifth display control means) corresponds the icons 52a11 to 52a31 representing the section evaluation points calculated by the reading evaluation unit 32 to the speaker sentence element sections corresponding to the respective icons. In each of the speaker text display areas 52a1 to 52a3 to be displayed, an area (an example of a fourth display area) arranged corresponding to each speaker text element area is displayed along the time axis t. In the example of FIG. 2A, the icons 52a11 to 52a31 are disposed at the right end in the speaker text display areas 52a1 to 52a3. As an example of the pattern of the icon showing a section evaluation point, the pattern which shows fine, the pattern which shows cloudy, and the pattern which shows rain are mentioned. Here, when the full score of the section evaluation point is 30 points, the pattern showing fine indicates that the section evaluation point is, for example, 21 to 30 points. The design showing fogging indicates that the section evaluation point is, for example, 11 to 20 points. The pattern indicating rain indicates that the section evaluation point is, for example, 0 to 10 points. In addition, the pattern of the icon showing a section evaluation point and the number of patterns can be set arbitrarily.

  Further, the display processing unit 33 displays a model graph representing a time-series change of at least one sound element of the model pitch and the model sound pressure specified by the voice processing section 31 in a model graph display area 51b. To display along the time axis. In addition, the display processing unit 33 (an example of the second display control unit) generates a time-series change in at least one of the sound elements of the speaker pitch and the speaker sound pressure specified by the sound processing unit 31. A speaker graph to be displayed is displayed in the speaker graph display area 52b (an example of a second display area) along the time axis t. At this time, the display processing unit 33 displays the same kind of sound element (for example, the same kind of sound element as the example sound pitch as the sound element represented by the example graph (that is, the example sound pitch or the model sound pressure) A speaker graph representing time-series changes in pitch) is displayed along the time axis t. In the example of FIG. 2 (A), the model graph display area 51b includes a model graph (broken line graph) 51b11 to 51b31 representing a time-series change in model pitch and a time-series change in model sound pressure. A model graph (bar graph) 51b12 to 51b32 representing the symbol is displayed separately for each hand text element section along the time axis t. In the speaker graph display area 52b, a speaker graph (broken line graph) 52b11 to 52b31 showing time-series changes in speaker pitch and a speaker graph showing time-series changes in speaker sound pressure (Bar graphs) 52b12 to 52b32 are displayed separately for each speaker sentence element section along the time axis t.

  Further, on the display screen shown in FIG. 2A, a comprehensive evaluation point display area 53 (an example of a third display area), a scroll bar 54, and a "close" key 55 for closing the display screen are arranged. There is. The display processing unit 33 (an example of the third display control unit) causes the comprehensive evaluation point display area 53 to display the comprehensive evaluation point calculated by the reading evaluation unit 32. In the example of FIG. 2 (A), the overall evaluation points for the evaluation items (abduction, volume, syntony, and speed) in all speaker sentence element intervals, and the overall evaluation points for the gap (interval) in all speaker interval intervals And the comprehensive evaluation points in all sections. The display contents in the sample display area 51 and the speaker display area 52 are scrolled and displayed in the direction parallel to the time axis t in accordance with the user operation of the scroll bar 54.

  Then, when any one of the speaker text display areas 52a1 to 52a3 and the speaker space areas 52s1 and 52s2 is specified by the user operation, the display processing unit 33 indicates any one of the specified areas. The window W11 (an example of the first window) for displaying the evaluation point calculated in the section (the sentence element section or the interval section) corresponding to is popped up. This makes it possible to display the details of the evaluation points calculated in any of the sections that the speaker or the like particularly wants to confirm in a more easily viewable display mode.

  The example of FIG. 2B shows a case where the speaker text display area 52a1 is designated by superimposing the mouse pointer P (cursor) on the speaker text display area 52a1. In this case, the display processing unit 33 displays evaluation points for each of the speaker's intonation, volume, glide tongue, and speed, which are calculated in the speaker sentence element section corresponding to the specified speaker text display area 52a1. The window W11 is popped up. In this way, the detail of the evaluation point calculated in any speaker sentence element section that the speaker or the like particularly wants to confirm is associated with the text or the like representing the sentence element in the speaker sentence element section to make the display easier to view Can be displayed. The window W11 displayed in this way is erased when the designation of the speaker text display area 52a1 is canceled by the pointer P leaving the speaker text display area 52a1 (that is, the window W11 is closed). On the other hand, the display of the window W11 is continued (continuously displayed) by clicking the mouse operation button in a state where the mouse pointer P is superimposed on the speaker text display area 52a1. The window W11 thus continuously displayed is erased by clicking the mouse operation button again while the mouse pointer P is superimposed on the speaker text display area 52a1. Note that the evaluation points displayed on the window W11 may be evaluation points for a part of the intonation, the volume, the tongue, and the speed (for example, the volume and the speed).

  The example of FIG. 3A shows the case where the speaker space area 52s1 is designated by superimposing the mouse pointer P on the speaker space area 52s1. In this case, the display processing unit 33 pops up and displays a window W21 for displaying the evaluation score for the speaker gap calculated in the speaker interval section corresponding to the designated speaker space area 52s1. This makes it possible to display the details of the evaluation point calculated in any speaker interval section that the speaker or the like particularly wants to confirm in a more easily viewable display mode. The window W21 displayed in this way is erased when the designation of the speaker space area 52s1 is canceled by the pointer P leaving the speaker space area 52s1. On the other hand, the display of the window W21 is continued by clicking the mouse operation button while the mouse pointer P is superimposed on the speaker space area 52s1. The window W21 thus continuously displayed is erased by clicking the mouse operation button again while the mouse pointer P is superimposed on the speaker space area 52s1.

  The windows W11 and W21 may be configured to be popped up when the speaker graph display area 52b is designated by the user operation. In this case, the speaker graph display area 52b is divided into a plurality of areas (for example, divided by a line orthogonal to the time axis t) so as to correspond to each of the above sections (speaker sentence element section and speaker interval section). Be done. Then, the display processing unit 33 displays the evaluation points for each of the speaker's intonation, the volume, the glide tongue, and the speed, which are calculated in the speaker sentence element section corresponding to the area designated in the speaker graph display area 52b. Window W11 is popped up. Alternatively, the display processing unit 33 pops up and displays a window W21 for displaying the evaluation score for the speaker gap calculated in the speaker interval section corresponding to the area designated in the speaker graph display area 52b.

  In addition, as described above, the display processing unit 33 causes the display of the window W11 to be continued, and while the display of the window W11 is continued, another region different from any one region (that is, the speaker text display When the area, the speaker graph display area, or the speaker space area is newly specified by the user operation, the window W12 (the second display area) displays the evaluation point calculated in the section corresponding to the newly specified other area. 2) is popped up, and the display of the window W11 and the display of the window W12 are continued. As a result, it becomes possible for the speaker etc. to compare the evaluation points for each section, and it is possible to make the speaker etc. understand at a glance which section the evaluation is good or bad. The example of FIG. 3B shows an example in which the pop-up display of the window W11 and the window W12 continues. The window W12 is an evaluation point for each of the speaker's intonation, volume, glide tongue, and speed, which is calculated in the speaker sentence element section corresponding to the speaker text display area 52a2 specified after the speaker text display area 52a1. Is displayed.

  In addition, when the icon (any of 52a11 to 52a31) is designated by the user operation, the display processing unit 33 calculates the speaker's intonation, the volume, and the like calculated in the speaker sentence element section corresponding to the designated icon. A window W31 (an example of a fourth window) for displaying an evaluation point for each of at least one of a tongue and a velocity is popped up. The example of FIG. 4A shows a case where the icon 52a31 is designated by superimposing the mouse pointer P on the icon 52a31. In this case, the display processing unit 33 pops up a window W31 for displaying evaluation points for each of the speaker's intonation, volume, glide tongue, and speed, calculated in the speaker sentence element section corresponding to the designated icon 52a31. Display. Here, each evaluation point displayed in the window W31 is an evaluation point used to calculate a section evaluation point in the speaker sentence element section corresponding to the icon 52a31 (that is, a section evaluation point indicated by the design of the icon 52a31). is there. Therefore, the pop-up display of the window W31 allows the speaker or the like to effectively grasp the breakdown of the section evaluation point indicated by the pattern of the icon 52a31. The window W31 displayed in this manner is erased when the designation of the icon 52a31 is released by the pointer P leaving the icon 52a31. On the other hand, the display of the window W31 is continued by clicking the mouse operation button while the mouse pointer P is superimposed on the icon 52a31. The window W31 continuously displayed in this manner is erased by clicking the mouse operation button again while the mouse pointer P is superimposed on the icon 52a31. Also in this case, when another icon different from the icon 52a31 is newly designated by the user operation while the display of the window W31 is continued, the speaker corresponding to the newly designated other icon A new window displaying the evaluation points calculated in the sentence element section may be popped up to continue the display of the window W31 and the display of the new window.

  In addition, when the speaker text display area (any of 52a1 to 52a3) is designated by the user operation, the display processing unit 33 performs the whole of the text at least a part of which is displayed in the designated speaker text display area. A window WT (an example of a third window) to be displayed may be popped up. The example of FIG. 4B shows the case where the speaker text display area 52a3 is designated by superimposing the mouse pointer P on the speaker text display area 52a3. In this case, the display processing unit 33 pops up a window WT that displays the entire text associated with the speaker sentence element segment data of the speaker sentence element segment corresponding to the specified speaker text display area 52a3. . As a result, even if the text does not fit in the speaker text display area 52a3, it is possible to make the speaker or the like understand the content of the sentence element represented by the text. The window WT displayed in this way is erased when the designation of the speaker text display area 52a3 is canceled by the pointer P leaving the speaker text display area 52a3. On the other hand, the display of the window WT is continued by clicking the mouse operation button while the mouse pointer P is superimposed on the speaker text display area 52a3. The window WT continuously displayed in this manner is erased by clicking the mouse operation button again while the mouse pointer P is superimposed on the speaker text display area 52a3.

[2. Operation Example of Reading Evaluation System S]
Next, an operation example of the reading evaluation apparatus S will be described by being divided into the first embodiment and the second embodiment. It should be noted that, as a premise of the operation described below, the body text section data and the speaker voice waveform data are taken into the control unit 3 and the speech processing section 31 outputs the body text section, the model interval section, the model sound pressure, A sample pitch, a model feature, a speaker sentence element interval, a speaker interval interval, a speaker sound pressure, a speaker pitch and a speaker feature are specified, and these data and each hand text element interval are specified. It is assumed that the text and the text for each speaker sentence element section are stored in the storage unit 2 in association with, for example, identification information of a desired voice file serving as a model of the reading target. Furthermore, the reading aloud evaluation unit 32 calculates evaluation points for each evaluation item, a section evaluation point, and an overall evaluation point, and these evaluation points are associated with identification information of a desired voice file to be a model for reading aloud It is assumed that it is stored in the storage unit 2.

Example 1
First, the first embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing the reading aloud evaluation display processing of the control unit 3 in the first embodiment. The process shown in FIG. 5 is started, for example, by the speaker designating a desired voice file to be a model for reading aloud through the operation unit 4 and giving a display start instruction. When the process shown in FIG. 5 is started, the control unit 3 (display processing unit 33) generates data (text element section data, interval section data, text data, etc.) associated with identification information of the specified audio file. And data of evaluation points are read from the storage unit 2 and, as shown in FIG. 2A, the display screen on which the model display area 51, the speaker display area 52, the comprehensive evaluation point display area 53, etc. are arranged The display D is displayed (step S1). In the example of FIG. 2A, the sample display area 51, the speaker display area 52, and the comprehensive evaluation point display area 53 are arranged on the same screen. In this way, it is possible to have the speaker etc. simultaneously refer to the comprehensive evaluation points in all the sections and the evaluation points in the partial section that the speaker etc wants to check. However, the comprehensive evaluation point display area 53 may be arranged on a screen different from the screen on which the model display area 51 and the speaker display area 52 are arranged. That is, for example, although the comprehensive evaluation point is displayed on another screen, the comprehensive evaluation point may not be displayed on the screen on which the sample display area 51 and the speaker display area 52 are displayed.

  Next, the control unit 3 determines whether or not any one of the speaker text display areas 52a1 to 52a3 and the speaker space areas 52s1 and 52s2 is designated by the user operation (step S2). For example, when the user operates the mouse and the mouse pointer P is superimposed on the speaker text display area or the speaker space area, it is determined that the area is designated by the user operation. Alternatively, in the case where the display D is a touch panel display, when the user touches the speaker text display area or the speaker space area with a finger or a pen, it is determined that the area is designated by the user operation. In step S2, as described above, it may be determined whether or not the speaker graph display area is designated by the user operation. When the control unit 3 determines that one of the areas is designated by the user operation (step S2: YES), the process proceeds to step S3. On the other hand, when it is determined that one of the areas is not designated by the user operation (step S2: NO), the control unit 3 proceeds to step S10.

  In step S3, the control unit 3 determines whether the window corresponding to the specified area is being displayed continuously. If the control unit 3 determines that the window is being displayed continuously (step S3: YES), the process proceeds to step S10. On the other hand, when the control unit 3 determines that the window is not continuously displayed (step S3: NO), the process proceeds to step S4. In step S4, the control unit 3 specifies the evaluation point calculated in the section corresponding to the designated area (that is, the area determined to be designated). Next, the control unit 3 pops up a window for displaying the evaluation point identified in step S4 on the display screen (step S5). For example, when the speaker text display area 52a1 is designated, a window W11 for displaying evaluation points for each of the speaker's intonation, volume, glide tongue, and speed calculated in the speaker sentence element section corresponding to this area Is popped up as shown in FIG. 2 (B). On the other hand, when the speaker space area 52s1 is designated, the window W21 for displaying the evaluation points for the speaker gap calculated in the speaker interval section corresponding to this area is as shown in FIG. 3 (A). Pop up is displayed.

  Next, the control unit 3 determines whether or not the designation to the area is released (step S6). For example, when the user operates the mouse and the pointer P of the mouse leaves the speaker text display area or the speaker space area, it is determined that the designation to the area is released. Alternatively, in the case where the display D is a touch panel display, when the user removes a finger, a pen or the like from the speaker text display area or the speaker space area, it is determined that the designation to the area is cancelled. Then, when it is determined that the designation to the area is canceled (step S6: YES), the control unit 3 erases the display of the window corresponding to the determined area (step S7), and proceeds to step S10.

  On the other hand, when it is determined that the designation to the area is not canceled (step S6: NO), the control unit 3 determines whether or not there is an instruction to continue displaying the window corresponding to the determined area (step S8). . For example, in the state where the mouse pointer P is superimposed on the area determined in step S6, when the user clicks the operation button of the mouse, it is determined that the continuous display instruction of the window corresponding to the area is given. Alternatively, if the display D is a touch panel display, when the user touches the area determined in step S6 with a finger or a pen with a finger or a pen etc., the finger or pen is temporarily released and quickly taps the area. It is determined that there is a continuous display instruction of the corresponding window. Then, when it is determined that the continuous display instruction of the window corresponding to the area is given (step S8: YES), the control unit 3 causes the window to be continuously displayed (step S9), and proceeds to step S10. On the other hand, when it is determined that the control unit 3 has not instructed to continuously display the window corresponding to the area (step S8: NO), the process returns to step S6.

  In step S10, the control unit 3 determines whether or not there is a continuous display cancellation instruction of the window being continuously displayed. If the window is not continuously displayed in step S10, the instruction to cancel the continuous display is not received from the start, so the process proceeds to step S12. If the control unit 3 determines that the instruction to cancel the continuous display of the continuously displayed window has been issued (step S10: YES), the control unit 3 proceeds to step S11. For example, when the user operates the mouse and superimposes the mouse pointer P on the window being continuously displayed, and the user clicks the operation button of the mouse, it is determined that the instruction to cancel the continuous display of the window has been issued. Ru. Alternatively, in the case where the display D is a touch panel display, when the user taps on the continuously displayed window with a finger, a pen, or the like, it is determined that there is a continuous display cancellation instruction of the window. In step S11, the control unit 3 erases the display of the window for which the continuous display cancellation instruction is issued, and the process proceeds to step S12. On the other hand, when the control unit 3 determines that the continuous display cancellation instruction of the window being continuously displayed is not received (step S10: NO), the control unit 3 proceeds to step S12.

  In step S12, the control unit 3 determines whether or not there is a display end instruction. When the control unit 3 determines that the display end instruction has been given according to the user operation of the close button 55 (step S12: YES), the process shown in FIG. 5 is ended. On the other hand, when it is determined that the display end instruction has not been issued (step S12: NO), the control unit 3 returns to step S2. Although not illustrated, in the process illustrated in FIG. 5, the control unit 3 scrolls the sample display area 51 and the speaker display area 52 in the left direction or the right direction according to the user operation of the scroll bar 54.

(Example 2)
A second embodiment will now be described with reference to FIG. FIG. 6 is a flowchart showing the reading aloud evaluation display processing of the control unit 3 in the second embodiment. In the second embodiment, the contents displayed in the window are switched between when the text area is specified in the speaker text display area and when the icon area is specified in the speaker text display area. is there. In the process shown in FIG. 6, the process when the speaker space area is designated by the user operation is omitted. The process shown in FIG. 6 is started in the same manner as the process shown in FIG. Step S21 shown in FIG. 6 is the same as step S1 shown in FIG.

  In step S22, the control unit 3 determines whether the text area in any one of the speaker text display areas of the speaker text display areas 52a1 to 52a3 is designated by the user operation. When the control unit 3 determines that the text area is designated by the user operation (step S22: YES), the control unit 3 proceeds to step S23. On the other hand, when the control unit 3 determines that the text area is not designated by the user operation (step S22: NO), the control unit 3 proceeds to step S30.

  In step S23, the control unit 3 determines whether the window corresponding to the designated text area is being displayed continuously. If the control unit 3 determines that the window is continuously displayed (step S23: YES), the control unit 3 proceeds to step S30. On the other hand, when the control unit 3 determines that the window is not continuously displayed (step S23: NO), the process proceeds to step S24. In step S24, the control unit 3 specifies the text associated with the speaker sentence element segment data of the speaker sentence element segment corresponding to the designated text region. Next, the control unit 3 pops up a window for displaying all of the text specified in step S24 on the display screen (step S25). As a result, a window WT displaying all of the text at least partially displayed in the designated speaker text display area is popped up as shown in FIG. 4 (B). The process of steps S26 to S29 is the same as that of steps S6 to S9 shown in FIG.

  In step S30, the control unit 3 determines whether any one of the icons 52a11 to 52a31 is designated by the user operation. If the control unit 3 determines that any one icon is designated by the user operation (step S30: YES), the process proceeds to step S31. On the other hand, when it is determined that one of the icons is not designated by the user operation (step S30: NO), the control unit 3 proceeds to step S38.

  In step S31, the control unit 3 determines whether or not the window corresponding to the designated icon is being displayed continuously. If the control unit 3 determines that the window is continuously displayed (step S31: YES), the control unit 3 proceeds to step S38. On the other hand, when the control unit 3 determines that the window is not continuously displayed (step S31: NO), the process proceeds to step S32. In step S32, the control unit 3 specifies the evaluation point calculated in the speaker sentence element section corresponding to the designated icon. Next, the control unit 3 pops up a window displaying the evaluation point identified in step S32 on the display screen (step S33). As a result, a window W31 for displaying the evaluation point calculated in the speaker sentence element section corresponding to the designated icon is popped up as shown in FIG. 4 (A). The process of steps S34 to S37 is similar to that of steps S6 to S9 shown in FIG. Moreover, the process of step S38-S40 is the same as that of step S10-S12 shown in FIG.

  As described above, according to the above-described embodiment, the reading evaluation apparatus S displays the speaker text display areas 52a1 to 52a3 and the speaker space areas 52s1 and 52s1 in a state where the comprehensive evaluation point for the reading by the speaker is displayed. When any one of the 52s2 is designated by the user operation, a window etc. displaying the evaluation point calculated in the section corresponding to the designated one is displayed in a pop-up manner, so that the speaker etc. In particular, the details of the evaluation points calculated in any of the sections to be confirmed can be displayed in a more easily viewable display mode.

Reference Signs List 1 communication unit 2 storage unit 3 control unit 4 operation unit 5 interface unit 6 bus 31 voice processing unit 32 speech reading evaluation unit 33 display processing unit S speech reading evaluation device

Claims (12)

A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements Evaluation point calculation means for calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
Comprehensive evaluation point calculation means for calculating an overall evaluation point for the reading on the basis of the evaluation points for each section calculated by the evaluation point calculation means;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. First display control means for displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval Second display control means for displaying in the display area of
Third display control means for displaying in the third display area the comprehensive evaluation point calculated by the comprehensive evaluation point calculation means;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case where the first window displaying the evaluation points calculated in the section corresponding to the designated one of the regions is displayed in a pop-up manner , and the first display region is designated by a user operation, Fourth display control means for pop-up displaying a third window displaying all of the text at least a part of which is displayed in the specified first display area ;
A speech reading and evaluating apparatus comprising: A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements Evaluation point calculation means for calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
Comprehensive evaluation point calculation means for calculating an overall evaluation point for the reading on the basis of the evaluation points for each section calculated by the evaluation point calculation means;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. First display control means for displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval Second display control means for displaying in the display area of
Third display control means for displaying in the third display area the comprehensive evaluation point calculated by the comprehensive evaluation point calculation means;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. Fourth display control means for pop-up displaying a first window for displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
It is the evaluation point corresponding to each of at least one of intonation, volume, glide tongue, and speed set in advance as an evaluation item for the aloud reading, and the evaluation point calculated for each of the sentence element sections Interval evaluation point calculation means for calculating an interval evaluation point for the reading aloud for each sentence element interval based on the following;
The icon representing each section evaluation point calculated by the section evaluation point calculation means is arranged for each of the sentence element sections along the time axis in a fourth display area arranged corresponding to each of the first display areas. A fifth display control means for displaying on the
Equipped with
The fourth display control means is an evaluation point calculated in the sentence element section corresponding to the designated icon when the icon is designated by a user operation, and is previously set as an evaluation item for the reading aloud And a fourth window for displaying the evaluation point for each of at least one of the selected intonation, the volume, the tongue, and the speed as a pop-up display. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements Evaluation point calculation means for calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
Comprehensive evaluation point calculation means for calculating an overall evaluation point for the reading on the basis of the evaluation points for each section calculated by the evaluation point calculation means;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. First display control means for displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval Second display control means for displaying in the display area of
Third display control means for displaying in the third display area the comprehensive evaluation point calculated by the comprehensive evaluation point calculation means;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. Fourth display control means for pop-up displaying a first window for displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
Equipped with
The fourth display control means causes the mouse pointer to be superimposed on the one area in response to the user operation to cause the first window to pop up, and the mouse pointer leaves the one area. Thus, the first window is erased, and the display of the first window is continued by clicking the operation button of the mouse while the mouse pointer is superimposed on the one area;
While the display of the first window continues, the mouse pointer is superimposed on another area different from any one of the areas, thereby calculating in the section corresponding to the other area. A second window displaying evaluation points is popped up, and the mouse pointer is erased from the other area so as to erase the second window, and the mouse pointer is superimposed on the other area. The sound reading evaluation apparatus according to claim 1, wherein the display of the second window is continued by clicking the operation button of the mouse in a state. When the display of the first window continues, the fourth display control means clicks again the operation button of the mouse while the pointer of the mouse is superimposed on the one area. 4. The reading evaluation apparatus according to claim 3, wherein the first window is erased. When the first display area or the second display area is designated by a user operation, the fourth display control means corresponds to the designated first display area or the second display area. An evaluation point calculated in the section, a first window for displaying the evaluation point for at least one of an inflection, a volume, a tongue tongue, and a speed preset as an evaluation item for the reading aloud The speech reading evaluation apparatus according to any one of claims 1 to 4, wherein the pop-up display is performed. When the space area is designated by a user operation, the fourth display control means is an evaluation point calculated in the section corresponding to the designated space area, and is previously set as an evaluation item for the reading aloud The speech reading evaluation apparatus according to any one of claims 1 to 4 , wherein a first window for displaying the evaluation point for the determined gap is popped up. A display control method implemented by one or more computers, comprising:
A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case where the first window displaying the evaluation points calculated in the section corresponding to the designated one of the regions is displayed in a pop-up manner , and the first display region is designated by a user operation, A fourth display control step of pop-up displaying a third window for displaying all of the text at least a part of which is displayed in the specified first display area ;
A display control method comprising: A display control method implemented by one or more computers, comprising:
A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case, a fourth display control step of pop-up displaying a first window displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
It is the evaluation point corresponding to each of at least one of intonation, volume, glide tongue, and speed set in advance as an evaluation item for the aloud reading, and the evaluation point calculated for each of the sentence element sections A section evaluation point calculation step of calculating a section evaluation point for the reading aloud for each sentence element section based on the following;
In the fourth display area arranged corresponding to each of the first display areas, the icon representing each of the section evaluation points calculated in the section evaluation point calculation step for each of the sentence element sections along the time axis A fifth display control step to be displayed on the
Including
In the fourth display control step, when the icon is designated by a user operation, it is an evaluation point calculated in the sentence element section corresponding to the designated icon, and is previously determined as an evaluation item for the reading aloud A display control method characterized by pop-up displaying a fourth window for displaying the evaluation point for each of at least one of the set intonation, the volume, the tongue, and the speed. A display control method implemented by one or more computers, comprising:
A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case, a fourth display control step of pop-up displaying a first window displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
Including
In the fourth display control step, the mouse pointer is superimposed on the one area in response to the user operation to cause the first window to be popped up, and the mouse pointer starts from the one area. The first window is erased by being separated, and the display of the first window is continued by clicking the operation button of the mouse while the mouse pointer is superimposed on the one area;
While the display of the first window continues, the mouse pointer is superimposed on another area different from any one of the areas, thereby calculating in the section corresponding to the other area. A second window displaying evaluation points is popped up, and the mouse pointer is erased from the other area so as to erase the second window, and the mouse pointer is superimposed on the other area. A display control method characterized by continuing display of the second window by clicking an operation button of the mouse in a state. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case where the first window displaying the evaluation points calculated in the section corresponding to the designated one of the regions is displayed in a pop-up manner , and the first display region is designated by a user operation, A fourth display control step of pop-up displaying a third window for displaying all of the text at least a part of which is displayed in the specified first display area ;
A program characterized by causing a computer to execute. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case, a fourth display control step of pop-up displaying a first window displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
It is the evaluation point corresponding to each of at least one of intonation, volume, glide tongue, and speed set in advance as an evaluation item for the aloud reading, and the evaluation point calculated for each of the sentence element sections A section evaluation point calculation step of calculating a section evaluation point for the reading aloud for each sentence element section based on the following;
In the fourth display area arranged corresponding to each of the first display areas, the icon representing each of the section evaluation points calculated in the section evaluation point calculation step for each of the sentence element sections along the time axis A fifth display control step to be displayed on the
A program that causes a computer to execute
In the fourth display control step, when the icon is designated by a user operation, it is an evaluation point calculated in the sentence element section corresponding to the designated icon, and is previously determined as an evaluation item for the reading aloud A program for pop-up displaying a fourth window for displaying the evaluation point for each of at least one of the set intonation, volume, glide tongue, and speed. A sentence element interval from the start timing to the end timing of each sentence element constituting the sentence read aloud based on speech waveform data indicating a waveform of speech emitted when the speaker aloud the sentence, and a plurality of the sentence elements An evaluation point calculating step of calculating an evaluation point for the reading aloud for at least one of interval segments from the end timing of any one of the sentence elements to the start timing of the next one of the sentence elements;
An overall evaluation point calculating step of calculating an overall evaluation point for the reading on the basis of the evaluation point for each section calculated in the evaluation point calculating step;
At least a portion of the text representing each sentence element constituting the aloud sentence is placed along the time axis in the first display area of a length corresponding to the time length of each of the sentence element sections. A first display control step of displaying each time;
A graph representing a time-series change of at least one sound element of the pitch and the sound pressure specified based on the voice waveform data at predetermined time intervals shorter than the time length of the sentence element interval A second display control step of displaying on the display area along the time axis;
A third display control step of displaying the comprehensive evaluation point calculated in the comprehensive evaluation point calculating step in a third display area;
One of the first display area, the second display area, and a space area located along the time axis between the plurality of first display areas is designated by a user operation. In the case, a fourth display control step of pop-up displaying a first window displaying the evaluation point calculated in the section corresponding to the designated one of the areas;
A program that causes a computer to execute
In the fourth display control step, the mouse pointer is superimposed on the one area in response to the user operation to cause the first window to be popped up, and the mouse pointer starts from the one area. The first window is erased by being separated, and the display of the first window is continued by clicking the operation button of the mouse while the mouse pointer is superimposed on the one area;
While the display of the first window continues, the mouse pointer is superimposed on another area different from any one of the areas, thereby calculating in the section corresponding to the other area. A second window displaying evaluation points is popped up, and the mouse pointer is erased from the other area so as to erase the second window, and the mouse pointer is superimposed on the other area. A program characterized by continuing display of the second window by clicking an operation button of the mouse in a state.