TW202403695A - Voice correction auxiliary method and system characterized by providing the practice of articulation and pronunciation at home and providing references for speech therapists - Google Patents

Abstract

A voice correction auxiliary method includes a training program executed by a training device and an execution program executed by an execution device. The execution program includes: playing a first sound data corresponding to a first word in a voice sample data when it is determined that a first pronunciation button is triggered; recording a first recording data corresponding to the first word and generating a data to be analyzed on the basis of the first recording data, when it is determined that a first recording button is triggered; and comparing the similarity between the data to be analyzed and the first sound data to generate an analysis result when it is determined that the start analysis button is triggered. The present invention is a tool for practicing articulation and pronunciation at home, and can analyze the user's articulation status on the basis of the data to be analyzed, which records the user's pronunciations, so as to provide references for speech therapists. The training program includes: using a training data to train an AI mode to establish a sound comparison mode; and transmitting the sound comparison mode to the execution device.

Description

Speech correction auxiliary method and system

A speech correction auxiliary method and system, especially a speech correction auxiliary method and system that can assist in analyzing and correcting pronunciation.

Some children will encounter difficulties in articulating and pronouncing words as they grow up, and the difficulties and errors encountered in the pronunciation process can be called language barriers. Speech disorder is a disorder that can improve with treatment.

The treatment of speech disorders needs to be presided over by a professional speech therapist, and the children's parents further need to cooperate in supervising the children's pronunciation practice at home to effectively improve the children's speech disorders. However, when parents cooperate in supervising their children's pronunciation practice at home, they are unable to discern the pronunciation errors their children make during practice as carefully as professional speech therapists. Therefore, parents may fail to provide immediate feedback to their children when their children make mispronunciation errors, causing the children to repeatedly practice incorrect pronunciation, causing resistance to overall language disorder treatment.

Furthermore, even if parents are aware of the errors in their children's pronunciation, most parents are unable to teach their children how to change their pronunciation skills in a targeted manner. In other words, even if parents can detect their children's pronunciation errors, they do not understand how their children make them, and therefore cannot provide targeted improvement methods. This will cause a burden on both children and parents.

In view of the above problems, the present invention provides a speech correction method and system.

The speech correction assistance system of the present invention includes a training device and an execution device. The training device further includes a display module, an audio module, a memory module, a communication module and a processing module. The processing module is electrically connected to the display module, the audio module, the memory module and the communication module respectively.

The memory module stores a first image information, a first word information corresponding to the first image information, and a voice sample data. The first word information includes a first word, and the speech sample data includes a first sound data corresponding to the first word. The communication module is connected to a network to communicate with the training device.

The speech correction auxiliary method of the present invention includes a training program and an execution program. The training program is executed by the training device, and the execution program is executed by the processing module of the execution device, and the execution program includes the following steps: a. Display a start analysis button, a first image information and a first word information corresponding to the first image information through the display module of the execution device, and display a first word information included in the first word information. A word, and a first pronunciation button and a first recording button corresponding to the first word; b. When it is determined that the first pronunciation button is triggered, play a first sound data corresponding to the first word in a voice sample data through the audio module of the execution device; c. When it is determined that the first recording button is triggered, record a first recording data corresponding to the first word through the audio module, and generate a data to be analyzed based on the first recording data; d. When it is determined that the start analysis button is triggered, compare the similarity between the data to be analyzed and the first sound data to generate an analysis result.

The present invention provides an auxiliary tool for parents to assist a child with speech impairment in correcting pronunciation. When the present invention plays the first sound data of the first word and presents the first image information through the execution program, children will be able to understand how to pronounce it correctly. When the present invention records the first recording data corresponding to the first word and generates the analysis result, the parent can understand whether the child's pronunciation is correct through the assistance of the machine. Furthermore, after using the present invention multiple times to record the first recording data and generate the analysis results, the parent can understand whether the child's pronunciation has improved with the help of the machine, that is, understand the first sound data and the first recording Whether the similarity of the data is getting closer and closer to similarity.

The present invention can provide tools to assist parents in analyzing and correcting the child's pronunciation during home practice, and can also provide reference analysis results to speech therapists to assist in the judgment of the child's language disorder.

Referring to Figure 1, the present invention provides a speech correction auxiliary method and system. The speech correction assistance system of the present invention includes a training device 100 and an execution device 200. The execution device 200 includes a display module 10, an audio module 20, a memory module 30, a processing module 40 and a communication module 50. The processing module 40 is electrically connected to the display module 10, the audio module 20 and the memory module 30 respectively.

The memory module 30 stores a first image information, a first word information corresponding to the first image information, and a voice sample data. The first word information includes a first word, and the speech sample data includes a first sound data corresponding to the first word. The communication module 50 of the execution device 200 is connected to a network to communicate with the training device 100 .

Please refer to Figure 2. The speech correction assistance method includes a training program S100 and an execution program S200. The training program S100 is executed by the training device 100, and the execution program S200 is executed by the processing module 40 of the execution device 200. The execution procedure S200 includes the following steps: Step S210: Display a start analysis button, the first image information and the first word information corresponding to the first image information through the display module 10 of the execution device 200, and display the first word information included The first word, and the first pronunciation button and the first recording button corresponding to the first word; Step S220: When it is determined that the first pronunciation button is triggered, play the first sound data corresponding to the first word in the speech sample data through the audio module 20 of the execution device 200; Step S230: When it is determined that the first recording button is triggered, record a first recording data corresponding to the first word through the audio module 20, and generate a data to be analyzed based on the first recording data; Step S240: When it is determined that the start analysis button is triggered, compare the similarity between the data to be analyzed and the first sound data to generate an analysis result.

In an embodiment of the present invention, the execution device 200 further includes a human-computer interaction module 60 , and the human-computer interaction module 60 is electrically connected to the processing module 40 . The human-computer interaction module 60 generates a selection signal. A user using the execution device 200 can create interactions through the human-computer interaction module 60, that is, by selecting the first pronunciation button to listen to the first sound data played by the present invention, and by selecting the first recording button. The first recording material corresponding to the first word is recorded.

When the processing module 40 receives the selection signal generated by the human-computer interaction module 60, and the selection signal corresponds to the selection of the first pronunciation button, the processing module 40 determines that the first pronunciation button is triggered, and further The first audio data is played through the audio module 20 . When the processing module 40 receives the selection signal generated by the human-computer interaction module 60, and the selection signal corresponds to the selection of the first recording button, the processing module 40 determines that the first recording button is triggered, and further The first recording data corresponding to the first word is recorded through the audio module 20 .

In this embodiment, the first word information is a word, and the first image information is a static pattern or a dynamic pattern corresponding to the word. The dynamic graphic is, for example, a Graphics Interchange Format (GIF). The first word is one of the words in the word, and the first sound data is a sound file that correctly pronounces the first word. The first recording data is a sound file of the user reading the first word.

In this embodiment, the network connected to the communication module 50 is an encrypted network, that is, the user must first log in to the network and verify his or her identity before downloading a software update data. The processing module 40 of the present invention downloads the software update data from the network through the communication module 50, and the processing module 40 updates the voice sample data stored in the memory module 30 according to the downloaded software update data. , to achieve the purpose of expanding data. When the communication module 50 stops connecting to the network, the present invention can still operate normally offline because the processing performed by the processing module 40 of the execution device 200 does not need to rely on the function of cloud computing.

For example, in this embodiment, the execution device 200 is a smart phone, that is, the processing module 40 is a processor, the communication module 50 is a network module, and the display module 10 and the The human-computer interaction module 60 is a touch screen, the audio module 20 is a speaker and a microphone, and the memory module 30 is a memory. In another embodiment, the execution device 200 is a tablet computer. In another embodiment, the execution device 200 is a computer, the display module 10 is a screen, and the human-computer interaction module 60 is a keyboard and a mouse. In addition, in this embodiment, the training device 100 is a computer or a cloud server that can be connected to the network, and the execution device 200 is connected to the communication module 50 of the training device 100 through the network communication.

Referring to Figure 3, the execution program S200 of the speech correction auxiliary method further includes the following steps:

Step S201: Download a software update data through the communication module 50, and update the voice sample data stored in the memory module 30 according to the software update data.

Step S202: Download a questionnaire data through the communication module 50, and display the questionnaire data through the display module 10.

Step S203: When the questionnaire data is selected, a user data is generated, and the user data is stored in the memory module 30.

Displaying the questionnaire data means that the present invention asks the user questions about personal information and health information through the display module 10 . Similarly, the user can select the answer to the multiple-choice question through the human-computer interaction module 60, and the human-computer interaction module 60 will correspondingly generate the selection signal to make a selection in the questionnaire data. Between step S202 and step S203, the processing module 40 determines whether all questions in the questionnaire data have been selected by the selection signal and have been selected. When it is judged that the selection has not been completed, the user data is not generated. The user information generated by the present invention will only be stored in the memory module 30 and not sent through the communication module 50. Therefore, the user information is only held by the user himself to protect the user. privacy. This user information is used to assist the user in organizing and presenting his or her own information and physical and mental status.

In this embodiment, the questionnaire data includes plural question information, and the question information may be fill-in questions, single-choice questions or multiple-choice questions. If the question information is a single-choice question or a multiple-choice question, it further includes corresponding plural answers for selection. Among them, the fill-in questions include filling in the name of the child, and the multiple-choice questions include selecting one of the options of good eating status, picky eating, poor appetite, difficulty chewing, or drooling. For single-choice questions, for example, you can select one of the options of breathing status: normal, noisy, or breathing through the mouth. For multiple-choice questions, for example, you can select unclear articulation, speech disorder (big tongue), voice disorder (hoarseness), Options include stuttering or delayed speech development. When each corresponding answer of the question information is selected, the present invention generates the user data including a plurality of selected answers, and stores the user data in the memory module 30 .

Please refer to FIG. 4 . In this embodiment, the first word information stored in the memory module 30 further includes a second word, and the speech sample data further includes a second sound corresponding to the second word. material.

When the processing module 40 executes step S210, the processing module 40 further displays the second word included in the first word information, and a second pronunciation button and a second recording button corresponding to the second word. And the execution program S200 further includes the following steps before step S240:

Step S231: When it is determined that the second pronunciation button is triggered, play the second sound data corresponding to the second word in the voice sample data through the audio module 20.

Step S232: When it is determined that the second recording button is triggered, record a second recording data corresponding to the second word through the audio module 20, and update the data to be analyzed based on the second recording data. The data to be analyzed includes the first recording data and the second recording data after being updated.

In addition, in this embodiment, the training device 100 stores an artificial intelligence model (AI model) and a training data for training the artificial intelligence model. In the training device 100, the artificial intelligence model is a convolutional neural network model (CNN model), and the training device 100 trains the convolutional neural network model based on the training data to A sound comparison model is established, and after the sound comparison model training is completed, the training device 100 then stores the sound comparison model in the memory module 30 of the execution device 200 through a communication method, such as the network. , for use by the processing module 40 of the execution device 200 . The training device 100 is connected to the communication module 50 of the execution device 200, so the trained sound comparison model can be transmitted to the processing module 40 through the communication module 50 and stored in the memory module 30. . The processing module 40 of the execution device 200 uses the sound comparison model to compare the similarity between the data to be analyzed and the first sound data to generate the analysis result.

Please refer to FIG. 5 . The training device 100 also has values such as an adjustment semitone number, a translation time, a speed adjustment percentage, a sound amplification percentage, and an environmental noise value. The training data includes a plurality of child voice files used to train the artificial intelligence model. Moreover, the training program S100 further includes the following steps:

Step S110: Train the artificial intelligence model through a pitch shift step, that is, adjust the audio pitch of each child's voice file up or down by the adjusted semitone number to train the artificial intelligence model.

For example, if the number of adjusted semitones is two, step S110 is to adjust the audio pitch of each of the children's voice files by two semitones up and down to train the artificial intelligence model, thereby simulating the difference in pitch between different human voices, thereby training the artificial intelligence model. The artificial intelligence model recognizes the difference in pitch between different human voices.

Step S120: Train the artificial intelligence model through a time shift step, that is, randomly shift the audio timeline of each of the children's sound files by the translation time to train the artificial intelligence model.

For example, the translation time is one second. Step S120 is to randomly translate the audio timeline of each of the children's voice files back and forth by one second, so as to train the artificial intelligence model to recognize the changes in the audio after being translated. , to enhance the artificial intelligence model’s ability to interpret the audio files of these children.

Step S130: Train the artificial intelligence model through a speed scaling step, that is, randomly scale the audio speed of each of the children's voice files by the speed adjustment percentage to train the artificial intelligence model.

For example, the speed adjustment percentage is 25%. Step S130 is to randomly scale the audio speed of each of the children's voice files by 25% of the original audio speed, so as to simulate changes in different speaking speeds and train the artificial intelligence model to recognize Changes in different speaking speeds.

Step S140: Train the artificial intelligence model through a volume increasing step, that is, increase the audio volume of each of the children's voice files by the amplification percentage to train the artificial intelligence model.

For example, if the amplification percentage is 15%, step S140 is to increase the audio volume of each of the children's voice files by 15% of the original volume, so as to simulate the changes in loudness and volume of different speakers to train the artificial intelligence model. Understand the changes in different speaking styles.

Step S150: Train the artificial intelligence model through a step of adding white noise, that is, adding the environmental noise to the audio of each of the children's voice files to train the artificial intelligence model.

For example, because the environmental noise is full-frequency noise, the changes in all frequencies in the children's voice files that are affected by the noise can be added to train the artificial intelligence model to recognize that the children's voice files are affected by the noise. Diversity of changes after the impact of information.

Step S160: End training the artificial intelligence model, and establish the trained voice comparison model.

Step S170: Send the trained sound comparison model to the execution device 200.

In step S170, the training device 100 transmits the trained sound comparison model to the communication module 50 of the execution device 200 through the network. The processing module 40 of the execution device 200 receives the trained sound comparison model through the communication module 50 to establish or update the trained sound comparison model in the memory module 30 . When the processing module 40 of the execution device 200 executes the program S240, the processing module 40 compares the similarity between the data to be analyzed and the first recording data through the sound comparison model to generate the analysis result. The order of the above-mentioned steps S110 to step S170 and the training combination used are not limited to this embodiment.

Referring to FIG. 6 , in another embodiment, the training data includes the children's voice files and a plurality of adult voice files used to train the artificial intelligence model. Moreover, the training program S100 further includes the following steps:

Step S100A: Train the artificial intelligence model through a Mel spectrum step, that is, after time-frequency conversion of the adult voice files and the child voice files, capture the frequency band audio in the plurality of signal windows, and convert the signal windows into After the frequency band audio is filtered, the time-frequency conversion is performed again and used to train the artificial intelligence model.

Step S100B: End training the artificial intelligence model, and establish the trained voice comparison model.

Step S100C: Send the trained sound comparison model to the execution device 200 .

In this embodiment, step S100A is executed by the training device 100 through the MelSpectrogram instruction of the Matrix Laboratory (Matlab). The signal windows, specifically, include three signal windows with different window sizes. Each signal window captures the time-frequency converted spectra of the adult voice files and the child voice files. of different frequency bands. When these captured frequency bands are filtered, noise can be removed based on the frequency of the noise to optimize the signal quality of these captured frequency bands, and thereby train the artificial intelligence model to improve the accuracy of the audio comparison. In other words, when the processing module 40 of the execution device 200 executes the program S240, the processing module 40 can more accurately compare the similarity between the data to be analyzed and the first recording data through the sound comparison model. produce the results of this analysis. Furthermore, when performing step S100A to train the artificial intelligence model through the Mel-scale step, the training device 100 will first obtain Mel-scale Frequency Cepstral Coefficients (MFCC), and then use the obtained Mel-scale Frequency Cepstral Coefficients (MFCC). MFCC is used to establish the coefficients of the Mel-Frequency Cepstrum (MFC) to obtain the spectrum on the nonlinear Mel scale (Mel scale), that is, a Mel spectrum is obtained. Among them, there is a mathematical conversion formula of logarithmic scale (logarithmic scale; log scale) between the Mel scale and the linear frequency scale Hertz (Hz).

Further, when filtering the frequency band audio within the signal windows, the training device 100 filters the adult voice files and the children's voice files after time-frequency conversion through a filter bank (FBank). Frequency distribution to filter out the noise outside the signal windows and retain the frequency band audio within the signal windows. Noise outside these signal windows, such as high-frequency noise higher than the frequency of human vocalization, or low-frequency noise lower than the frequency of human vocalization. In this embodiment, the filter bank is a digital filter bank, and the frequency filtered by the filter bank can be set by the training device 100 .

In another embodiment, the similarity generated according to the sound comparison model in the analysis result is the so-called Percentage of Consonants Correct (PCC). The consonant accuracy is obtained as a result of inputting a plurality of incorrect tones and a plurality of correct tones when training the artificial intelligence model. The artificial intelligence model can acoustically understand different degrees of speech impairment and perform subsequent analysis and error type classification. After the sound comparison model is generated, the sound comparison model can present the categories of error sounds that occur. For example, in terms of training the artificial intelligence model, it is familiar with the logical relationship between different types of mispronunciation according to the Chinese phonetic alphabet (International Phonetic Alphabet; IPA), for example:

Africiate ㄐ = stop ㄉ + fricative ㄒ.

Africiate ㄓ = stop ㄉ + fricative ㄕ.

Africiate ㄗ = stop ㄉ + fricative ㄙ, etc.

Furthermore, in terms of acoustics, when mispronunciations such as stops, affricates, fricatives or root sounds are time-frequency converted, the spectral characteristics of each mispronunciation can be observed on the spectrogram. For example, the spectrum corresponding to the root sound is Lower frequencies have more energy. Low frequencies in the spectrum have more energy, which can be expressed by, for example, higher intensity at low frequencies or higher sample density at low frequencies. As far as the time-domain acoustic spectrum characteristics of stops are concerned, the characteristics that can be observed on the acoustic spectrum are the so-called aspiration, voice onset time (VOT), and formant transition. Isotemporal sound spectrum characteristics. The time domain sound spectrum characteristics of stops mainly include several relatively similar formants, and obvious VOT can also be observed at the beginning of pronunciation.

After training, the artificial intelligence model of the present invention can integrate the above-mentioned acoustic knowledge and logic to generate the sound comparison model. The above acoustic knowledge and logic are only simple examples of this embodiment, and are therefore not limited thereto. When the sound comparison model is used by the execution device 200, the execution device 200 does not need to perform time-frequency conversion of the recorded sound to obtain the analysis results summarized by the artificial intelligence model based on its learned acoustic knowledge and logic.

Please refer to FIG. 7 . In this embodiment, when the processing module 40 of the execution device 200 executes the program S240 , the processing module 40 performs the following steps:

Step S241: Determine whether the start analysis button is triggered. When it is determined that the start analysis button has not been triggered, program S241 is executed.

Step S242: When it is determined that the start analysis button is triggered, compare the similarity between the first recording data and the first sound data in the data to be analyzed through the sound comparison model to generate the first result.

Step S243: Compare the similarity between the second recording data and the second sound data in the data to be analyzed through the sound comparison model to generate a second result.

Step S244: Generate the analysis result according to the first result and the second result.

Further, in this embodiment, step S244 averages the first result and the second result to generate the analysis result. In other words, this embodiment uses step S242 and step S243 to perform two comparison analyzes using the sound comparison model, and then averages the results produced by the two analyzes to obtain the analysis result. What is referred to here is the average result produced by the two analyses, that is, the analysis result = (the first result + the second result)/2.

In another embodiment of the present invention, the first word information includes a third word, and a third pronunciation button and a third recording button corresponding to the third word. According to the aforementioned logic of this case, when the third pronunciation button is triggered, a third sound data corresponding to the third word in the voice sample data is played through the audio module 20. When the third recording button is triggered, a third recording data corresponding to the third word is recorded through the audio module 20, and the data to be analyzed is updated according to the third recording data. By analogy with this logic, after the processing module 40 executes the program S243, the system will further compare the similarity between the third recording data and the third sound data in the data to be analyzed through the sound comparison model to generate a first Three results. Then, the analysis result is generated based on all generated results, that is, the first result, the second result, and the third result. In other words, the analysis result = (the sum of all generated results)/(the number of all generated results) = (the first result + the second result + the third result)/3.

Please refer to FIG. 8 . In another embodiment of the present invention, when the processing module 40 of the execution device 200 executes program S240, the processing module 40 performs the following steps:

Step S240A: Determine whether the start analysis button is triggered. When it is determined that the start analysis button has not been triggered, program S240A is executed.

Step S240B: When it is determined that the start analysis button is triggered, compare the similarity between the first recording data and the first sound data in the data to be analyzed, and the second voice data in the data to be analyzed through the sound comparison model. The similarity between the recorded data and the second sound data is used to generate the analysis result.

In other words, this embodiment uses the sound comparison model to compare and analyze the similarity between the first recording material and the first sound data and the similarity between the second recording material and the second sound data together at once, so as to only Using the sound comparison model produces a comprehensive result as the analysis result.

In yet another embodiment of the present invention, the third word included in the first word information, and the third pronunciation button and the third recording button corresponding to the third word, when the third recording button is When triggered, the third recording data corresponding to the third word is recorded through the audio module 20, and the data to be analyzed is updated according to the third recording data. Moreover, when it is determined that the start analysis button is triggered, the same logic as in the previous embodiment is used to compare the similarity between the first recording data and the first sound data in the data to be analyzed through the sound comparison model. The similarity between the second recording data and the second sound data in the data to be analyzed, and the similarity between the third recording data and the third sound data in the data to be analyzed are used to comprehensively generate the analysis result.

The analysis result includes a normal sound similarity and a plurality of abnormal sound similarities, and the normal sound similarity and the abnormal sound similarity are each a percentage generated through the sound comparison model. The percentage can numerically represent whether the user's pronunciation is normal, that is, whether the first recording data is similar to the first sound data, whether the second recording data is similar to the second sound data and the third recording Whether the data is similar to the analysis result of the third sound data. When the similarity of the normal sounds is 0%, it means that they are completely dissimilar, and when the similarity of the normal sounds is 100%, it means that they are completely similar.

Specifically, the present invention further generates different interpretations of the normal sound similarity based on a normal sound similarity threshold data stored in the memory module 30 . When the normal sound similarity is determined to be greater than 85% based on the normal sound similarity threshold data, the user is judged to have a mild language disorder. When the similarity of the normal sounds is determined to be greater than or equal to 65% and less than or equal to 85%, it is determined that the user has a mild to moderate speech impediment. When it is determined that the similarity between normal sounds is greater than or equal to 50% and less than or equal to 64%, it is determined that the user has a moderate to severe language disorder. When it is determined that the similarity of the normal pronunciation is less than 85%, it is determined that the user has severe language impairment. Based on the different interpretations of the similarity of the normal sounds, the present invention generates a language disorder interpretation data and stores it in the memory module 30 .

Please refer to Figure 9. After executing step S240, the processing module 40 of the present invention further executes the following steps of the execution program S200:

Step S250: Display the similarity of the normal sounds and the similarities of the abnormal sounds of the analysis results through the display module 10 .

Step S260: Interpret the normal sound similarity based on the normal sound similarity threshold data to generate the language disorder interpretation data.

Specifically, these abnormal sound similarities can be subdivided into one stop consonant similarity, one vowel consonant similarity, one vowel omission similarity, one tongue front consonant similarity, one tongue base consonant similarity, and one unaspirated similarity. Phonetic similarity, vowel similarity, unilateral phonetic similarity, interdental similarity, consonant omission similarity, fricative similarity, medial omission similarity, and affricate similarity Degrees and complex vowels omit numerical values such as similarity. When the processing module 40 of the execution device 200 executes the program S240, the present invention can analyze the numerical distribution composition of the similarities of the abnormal sounds through the sound comparison model, that is, using the trained artificial intelligence model. Why. When the similarity percentage of these abnormal sounds is higher, it means that according to the analysis of the sound comparison model, there is a higher probability of facing a pronunciation correspondence dilemma. For example, when the similarity of the stop consonantization is 99% and the similarity of the fricative consonantization is 1%, the user's abnormal pronunciation has a very high probability of facing the pronunciation problem of stop consonantization, but only a very small probability of facing a fricative problem. cultural voice issues.

In this embodiment, when the partial component similarity among the abnormal sound similarities is zero, 0% of the components are omitted from display on the display module 10 . For example, suppose that after one analysis, the user's similarity of the stop consonantization is 0.56%, the similarity of the root consonantization is 1.95%, the similarity of the consonant finals is 0.31%, and the similarity of the fricative consonantization is 0.56%. is 91.17%, and the remaining similarity is the vowelization similarity, the vowel omission similarity, the prelingual vowelization similarity, the unaspirated vowelization similarity, the lateralization similarity, the interdental similarity, the If the consonant omission similarity, the fricativeization similarity, the medial omission similarity and the compound vowel omission similarity are all 0%, then the display module 10 will only display the representative stop consonantization with a percentage greater than 0%. The similarity, the similarity of the root of the tongue, the similarity of the sound and the final and the similarity of the affricate.

In another embodiment of the present invention, after executing step S260, the execution device 200 of the present invention further transmits the data to be analyzed and the analysis results to the training device 100 through the communication module 50 to feed back the results of an analysis. Related information. The training device 100 can review and adjust the artificial intelligence model based on the feedback data to be analyzed and the analysis results, so as to have more data as teaching materials for training the artificial intelligence model in the future.

Please refer to FIG. 10 . FIG. 10 illustrates the screen displayed by the display module 10 of the execution device 200 , and the speech correction auxiliary method of the present invention is implemented by an application program (Application; APP).

In Figure 10, the display module 10 displays a start screen 10A. The start screen 10A includes a download option 2 and a start test option 3. When the download option 2 is selected, the process S201 is executed, and the download progress is displayed by a progress percentage of 1.

Please refer to FIG. 11 . When the start test option 3 is selected, the present invention jumps out of the start screen 10A and enters a test screen 10B. In the test screen 10B, the display module 10 displays the first image information 4 and the first word information 5 corresponding to the first image information 4, which is pudding. The first word 5A of the first word information 5 is the word "bu" for pudding, and the second word 5B of the first word information 5 is the word "D" for pudding. The display module 10 also displays the first pronunciation button 5AS and the first recording button 5AMic corresponding to the first word 5A, and the second pronunciation button 5BS and the second recording button corresponding to the second word 5B. 5BMic. Below the first image information 4 is a second image information 6, and the second image information 6 is a green vegetable. The pudding of the first image information 4 and the vegetables of the second image information 6 are for the same reason, that is, they are different objects that assist the user in analyzing and correcting pronunciation of words. In addition, the display module 10 displays a start analysis option 7. When the start analysis option 7 is selected, sampling is stopped and analysis is performed based on all currently recorded sound samples.

In another embodiment of the present invention, when the start test option 3 is selected, the present invention jumps out of the start screen 10A and first enters a simulation screen, and then jumps to the test screen 10B. A plurality of imitating words are displayed in the imitation speaking screen to assist the user in imitating speaking. These imitative words include "A" and "1, 2, 3, 4, 5, 6, 7, 8, 9, 10" and so on. The purpose of this is to induce the user to get used to reading the words displayed by the display module 10, so that the quality of recording the user's speech after entering the test screen 10B can be better, that is, to facilitate the use of the audio module 20 Recording the first recording material corresponding to the first word and the second recording material corresponding to the second word can have better quality due to the user's habit of reading, so as to record the user's normal and relaxed situation. Pronunciation made.

Please refer to FIG. 12 . When the start analysis option 7 is selected, the present invention jumps out of the test screen 10B and enters an analysis result screen 10C. In the analysis result screen 10C, the display module 10 displays the analysis result, that is, a first item 8N and its corresponding normal sound similarity 8 are displayed to represent the data to be analyzed and the normal sound recorded by the user. The pronunciation is about 99.11% similar.

The display module 10 further displays a plurality of abnormal pronunciation information and corresponding similarities of the abnormal pronunciations. The display module 10 displays a first abnormal pronunciation information 8AN and the corresponding stop consonantization similarity 8A, a second abnormal pronunciation information 8BN and the corresponding tongue base consonantization similarity 8B, a third abnormal pronunciation information 8CN and The corresponding consonant-final similarity 8C, a fourth abnormal pronunciation information 8DN and the corresponding affricate similarity 8D.

The display module 10 further displays a storage and upload option 9 and a next page option 11 . When the storage and upload option 9 is selected, the processing module 40 uploads the data to be analyzed and the analysis results to a cloud database for recording through the encrypted network connected to the communication module 50 . In addition, when the abnormal pronunciation information displayed by the display module 10 is not enough to be displayed on one page, the next page option 11 can be selected to update the display page to the next page to continue to display the remaining abnormalities. Pronunciation information.

Please refer to Figure 13. When the storage and upload option 9 is selected, the present invention jumps out of the analysis result screen 10C and enters a coaching information screen 10D. The processing module 40 downloads a plurality of health knowledge information 12 through the communication module 50, and in the counseling information screen 10D, the processing module 40 displays the health knowledge information 12 and an end option through the display module 10 13. The health knowledge information 12 displayed by the display module 10 can help the user increase their knowledge, so that the user understands the meaning of the abnormal pronunciation information 8AN, 8BN, 8CN, and 8DN. For example, abnormal pronunciation of affricates may originate from a certain error in the pronunciation process of a certain articulatory part. After the user understands the meaning of the abnormal pronunciation information 8AN, 8BN, 8CN, and 8DN, the user can better understand how to correct the pronunciation error. For example, because the user's similarity of affricate 8D is 91.17%, which is a very high percentage, it means that the user faces greater difficulties in pronunciation of affricate. Therefore, compared with other pronunciation problems, affricate The problem of fricativeization needs to be improved as a priority. In this way, the present invention can not only enable the user to receive assistance in correcting pronunciation, but also provide the user with reference analysis data to record pronunciation correction.

When the end option 13 is selected, the present invention jumps out of the tutoring information screen 10D and returns to the start screen 10A. That is, it ends all processes of recording and analyzing voice similarity and returns to the start screen 10A to standby for the next process. Start. When the start test option 3 in the start screen 10A is selected, a new process of recording and analyzing the sound similarity is started.

The present invention provides an auxiliary tool for parents to assist a child with speech impairment in correcting pronunciation. The user of the present invention can be the parent and the child, and the parent is responsible for operating the human-computer interaction module 60 of the present invention, and the children who may have language barriers watch the first image information 4 and interact with each other. The first word information 5 corresponding to the first image information 4, listening to the first sound data corresponding to the first word 5A, and recording the first recording data corresponding to the first word 5A. When the present invention plays the first sound data of the first word 5A and presents the first image information 4, children will be able to understand how to pronounce it correctly. When the present invention records the first recording data corresponding to the first word 5A and generates the analysis result, the parent can understand whether the child's pronunciation is correct through the assistance of the machine. Furthermore, after using the present invention multiple times to record the first recording data and generate the normal sound similarity 8, the parent can help understand whether the child's pronunciation has improved through the analysis and long-term recording results of the present invention, that is, Understand whether the similarity between the first sound material and the first recording material is getting closer and closer, or the similarity between the first sound material and the first recording material and the similarity between the second sound material and the second recording Is the comprehensive similarity of the data similarity becoming more and more normal? When the percentage of normal sound similarity 8 becomes higher and higher, it means that the child's pronunciation is becoming more and more correct.

After the display module 10 displays the analysis result, the parent integrates the health knowledge information 12 displayed by the display module 10 to help the child correctly practice articulation and pronunciation, and the parent provides the record of the present invention to a child. The speech therapist provides reference materials for the child's pronunciation.

1: Progress percentage 2:Download options 3: Start quiz option 4:First image information 5: First language word information 5A:First word 5Amic: First recording button 5AS: First pronunciation button 5B: The second word 5Bmic: Second recording button 5BS: Second pronunciation button 6: Second image information 7: Start analyzing options 8: Normal sound similarity 8N: The first project 8A: Stop consonant similarity 8AN: The first abnormal pronunciation information 8B: Similarity of tongue base phoneticization 8BN: Second abnormal pronunciation information 8C: Similarity of sounds and finals 8CN: Third abnormal pronunciation information 8D: Africative similarity 8DN: The fourth abnormal pronunciation information 9:Save and upload options 10:Display module 10A:Start screen 10B: Test screen 10C: Analysis result screen 10D: Counseling information screen 11: Next page options 12: Health knowledge information 13:End option 20: Audio module 30:Memory module 40: Processing module 50: Communication module 60:Human-computer interaction module 100:Training device 200: Execution device S100: Training program S110, S120, S130, S140, S150, S160, S170: steps S100A, S100B, S100C: Steps S200: Execute program S201~S203, S210, S220, S230~S232, S240~S244: steps S240A, S240B: steps S250, S260: steps

Figure 1 is a block diagram of a speech correction auxiliary system of the present invention. Figure 2 is a flow chart of a speech correction auxiliary method according to the present invention. FIG. 3 is a flow chart of an execution program of the speech correction auxiliary method of the present invention. FIG. 4 is another flow chart of the execution program of the speech correction auxiliary method of the present invention. Figure 5 is a flow chart of a training program of the speech correction auxiliary method of the present invention. Figure 6 is another flow chart of the training program of the speech correction auxiliary method of the present invention. Figure 7 is another flow chart of the execution program of the speech correction auxiliary method of the present invention. FIG. 8 is another flow chart of the execution program of the speech correction auxiliary method of the present invention. FIG. 9 is another flow chart of the execution program of the speech correction auxiliary method of the present invention. Figure 10 is a schematic diagram of an execution device of the speech correction auxiliary system of the present invention displaying a start screen. Figure 11 is a schematic diagram of the execution device of the speech correction auxiliary system of the present invention displaying a test screen. Figure 12 is a schematic diagram of the execution device of the speech correction assistance system of the present invention displaying an analysis result screen. Figure 13 is a schematic diagram of the execution device of the speech correction assistance system of the present invention displaying a coaching information screen.

S100: Training program

S200: Execute program

S210, S220, S230, S240: steps

Claims (11)

A speech correction auxiliary method, including: An execution program is executed by a processing module of an execution device and includes the following steps: a. Display a start analysis button, a first image information and a first word information corresponding to the first image information through a display module of the execution device, and display a first word information included in the first word information. A word, and a first pronunciation button and a first recording button corresponding to the first word; b. When it is determined that the first pronunciation button is triggered, play a first sound data corresponding to the first word in a voice sample data through an audio module of the execution device; c. When it is determined that the first recording button is triggered, record a first recording data corresponding to the first word through the audio module, and generate a data to be analyzed based on the first recording data; d. When it is determined that the start analysis button is triggered, compare the similarity between the data to be analyzed and the first sound data to generate an analysis result. The speech correction auxiliary method as described in claim 1, wherein: The first word information includes a second word; When performing step a, further display the second word included in the first word information, and a second pronunciation button and a second recording button corresponding to the second word; The execution procedure further includes the following steps before step d: c1. When it is determined that the second pronunciation button is triggered, play a second sound data corresponding to the second word in the voice sample data through the audio module; c2. When it is determined that the second recording button is triggered, record a second recording data corresponding to the second word through the audio module, and update the data to be analyzed based on the second recording data; wherein, the data to be analyzed Including the first recording material and the second recording material. The speech correction auxiliary method as described in claim 1 or 2 further includes: A training program is executed by a training device, and the training program includes the following steps: A. Use a training data to train an artificial intelligence model to build a sound comparison model; B. Send the sound comparison model to the execution device; When step d is executed, the similarity between the data to be analyzed and the first recording data is compared through the sound comparison model to generate the analysis result. The speech correction auxiliary method as described in claim 3, wherein: When executing step d, when it is determined that the start analysis button is triggered, the following steps are executed: d1. Compare the similarity between the first recording data and the first sound data in the data to be analyzed through the sound comparison model to generate a first result; d2. Compare the similarity between the second recording data and the second sound data in the data to be analyzed through the sound comparison model to generate a second result; d3. Generate the analysis result based on the first result and the second result. The speech correction auxiliary method as described in claim 4, wherein: Step d3 averages the first result and the second result to generate the analysis result. The speech correction auxiliary method as described in claim 3, wherein: When step d is executed, when it is determined that the start analysis button is triggered, the similarity of the first recording data and the first sound data in the data to be analyzed, and the data to be analyzed are compared together through the sound comparison model. The similarity between the second recording data and the second sound data is used to generate the analysis result. The speech correction auxiliary method as described in claim 3, wherein: The analysis result includes a normal sound similarity and a plurality of abnormal sound similarities, and the normal sound similarity and the abnormal sound similarity are each a percentage generated through the sound comparison model. The speech correction auxiliary method as described in claim 3, wherein: The training material includes multiple children's voice files; When executing program A, the artificial intelligence model is trained through a pitch shift step, a time shift step, a speed scaling step, a volume increase step and a white noise increase step; The pitch shifting step is to adjust the audio pitch of each of the children's voice files up and down by a number of semitones to train the artificial intelligence model; The time shifting step is to randomly shift the audio timeline of each of the children's voice files by a shifting time to train the artificial intelligence model; The speed scaling step is to randomly scale the audio speed of each of the children's voice files by a speed adjustment percentage to train the artificial intelligence model; The step of increasing the volume is to increase the audio volume of each of the children's voice files to train the artificial intelligence model; The step of adding white noise is to add an environmental noise to the audio of each of the children's voice files to train the artificial intelligence model. The speech correction auxiliary method as described in claim 3, wherein: The training data includes multiple adult voice files and multiple child voice files; When executing program A, the artificial intelligence model is trained through a Mel spectrum step; The Mel spectrum step converts the adult voice files and the child voice files into time-frequency conversions, captures the frequency band information within the plurality of signal windows, and filters the frequency band information within the signal windows, and then re-times frequency conversion and used to train the artificial intelligence model. The speech correction auxiliary method as described in claim 9, wherein: When filtering the frequency band audio within the signal windows, a filter bank (FBank) is used to filter the noise outside the signal windows; The filter bank is a digital filter bank, and the frequency filtered by the filter bank can be set by the training device. A speech correction auxiliary system, including: A training device that executes the training program of the speech correction auxiliary method described in any one of claims 3 to 10; An execution device, including: a display module; an audio module; A memory module that stores a first image information, a first word information corresponding to the first image information, and a voice sample data; wherein the first word information includes a first word, and the voice sample The data includes a first sound data corresponding to the first word; A processing module is electrically connected to the display module, the audio module and the memory module respectively; a communication module electrically connected to the processing module and connected to a network to communicate with the training device; The processing module executes the execution program of the speech correction auxiliary method described in any one of claims 1 to 10.

Images