KR102610871B1 - Speech Training System For Hearing Impaired Person - Google Patents

Abstract

The disclosed speech language training system for the hearing impaired includes a storage unit that stores learning target information including learning characters, learning pronunciation organ shapes, learning waveforms, and learning frequencies; a recognition unit that generates practice information comparable to the learning object information from the learner's language practice according to the learning object; a correction unit that generates correction information to minimize the difference between the learning target information and the practice information; and a display unit that displays the learning object information and the correction information.

Description

Speech Training System For Hearing Impaired Person}

The present invention (Disclosure) relates to a speech language training system for the hearing impaired, and specifically, by analyzing the voice of the hearing impaired during language training in the form of a sound wave and also analyzing it in a language form composed of letters, This is about a speech language training system for the hearing impaired that can maximize learning or training efficiency.

Here, background information related to the present invention is provided, and this does not necessarily mean prior art (This section provides background information related to the present disclosure which is not necessarily prior art).

Methods for hearing impaired people to communicate with the general public include lip reading or speech reading, and sign language.

However, most of these are intended to communicate to the general public from the perspective of the hearing impaired or to learn the contents of the general public's communication.

In other words, there are limits to the general public's ability to understand the communications of hearing-impaired people.

Accordingly, not only do the general public have difficulty communicating with the hearing impaired, but problems may also arise that make it difficult to have close interpersonal relationships.

Hearing impaired people basically have a physical handicap in which their hearing function is significantly lower or absent compared to the general population.

However, in many hearing-impaired people, the function of the vocal organs, including the vocal cords, and the pronunciation organs, including the tongue, is often normal.

Hearing-impaired people whose vocal and pronunciation organs function normally use sign language or sign language to express their opinions, but at the same time, they often speak spoken language on their own, albeit inaccurately.

However, because one cannot hear the sound produced by one's own voice, one cannot judge the accuracy of the language being pronounced, let alone the pitch of the voice.

Therefore, since vocalization and pronunciation cannot be corrected on their own, it may sound like a series of incorrect words or syllables to the general public.

Also, for this reason, it is often difficult to easily learn how to skillfully use the vocal organ or pronunciation organ to produce the correct pronunciation that is in mind.

In this way, many technologies have been introduced to help hearing-impaired people who can speak spoken language using their vocal organs or pronunciation organs learn and practice accurate language speaking methods.

However, these conventional technologies are based on much research on how to analyze the language of hearing impaired people, and lack practical practice or consideration of methods to increase learning efficiency.

1. Korean Patent Publication No. 10-2011-0065276

The purpose of the present invention (Disclosure) is to provide a speech language training system for the hearing impaired that can maximize the efficiency of language training and practice for the hearing impaired.

Here, a general summary of the present invention is provided, and this should not be construed as limiting the scope of the present invention (This section provides a general summary of the disclosure and is not a comprehensive disclosure of its full scope or all of its features).

In order to solve the above problem, a speech language training system for the hearing impaired according to one of the various aspects describing the present invention includes the learning letters of the learning target, the learning pronunciation organ shape, the learning waveform, and the learning frequency. A storage unit in which learning target information is stored; a recognition unit that generates practice information comparable to the learning object information from the learner's language practice according to the learning object; a correction unit that generates correction information to minimize the difference between the learning target information and the practice information; and a display unit that displays the learning object information and the correction information.

In the speech language training system for the hearing impaired according to one aspect of the present invention, the correction unit derives a practice waveform and a practice frequency from the learner's voice and compares them with the learning waveform and the learning frequency, respectively, to match the voices. and a voice analysis unit that calculates the degree of suitability as quantitative information, and the correction information may include the degree of suitability for voice matching.

In the speech language training system for the hearing impaired according to an aspect of the present invention, the correction information may further include a comparison graph that compares the learning waveform and the learning frequency with the practice waveform and the practice frequency, respectively. .

In the voice language training system for the hearing impaired according to an aspect of the present invention, the correction unit analyzes a plurality of voice matching suitability generated from repeated language practice of the learner, and as the voice matching suitability increases or decreases, It may further include a correction evaluation unit that generates voice matching trend information indicating whether the learner's language practice is improving or worsening.

A speech language training system for the hearing impaired according to an aspect of the present invention, the correction unit further includes a alphabet analysis unit that derives a practice alphabet from the learner's voice, compares it with the learning alphabet, and calculates alphabet matching suitability; , the correction information may include the letter matching suitability.

In the speech language training system for the hearing impaired according to one aspect of the present invention, the correction unit generates a practice pronunciation organ shape including a mouth shape through which the learner pronounces the learning cap, and the correction information is used to determine the learning pronunciation. It may include a comparison video comparing the organ shape and the practice pronunciation organ shape.

A speech language training system for the hearing impaired according to an aspect of the present invention. The correction unit analyzes the plurality of letter matching suitability generated from repeated language practice of the learner, and as the letter matching suitability increases or decreases, the correction unit analyzes the letter matching suitability. It may further include a correction evaluation unit that generates letter matching trend information indicating whether the learner's language practice is improving or worsening.

According to the present invention, the efficiency of language practice and learning can be maximized by providing various types of correction information in language skills, including speech and pronunciation, so that hearing-impaired people can practice accurate pronunciation and vocalization.

Figure 1 is a diagram showing a first embodiment of a speech language training system for the hearing impaired according to the present invention.
Figure 2 is a diagram showing a second embodiment of the speech language training system for the hearing impaired according to the present invention.
Figure 3 is a diagram showing a third embodiment of the speech language training system for the hearing impaired according to the present invention.

Hereinafter, an embodiment implementing the speech language training system for the hearing impaired according to the present invention will be described in detail with reference to the drawings.

However, the essential technical idea of the present invention cannot be said to be limited to the embodiments described below, and based on the essential technical idea of the present invention, it cannot be said that the possible embodiments are limited by the embodiments described below. It is disclosed that the embodiment described in covers the scope that can be easily proposed by way of replacement or change.

In addition, the terms used below are selected for convenience of explanation, so in understanding the essential technical idea of the present invention, they are not limited to dictionary meanings and are appropriately interpreted as meanings that correspond to the technical idea of the present invention. It should be.

Figure 1 is a diagram showing a first embodiment of a speech language training system for the hearing impaired according to the present invention.

Referring to FIG. 1, the speech language training system for the hearing impaired according to this embodiment includes a storage unit 100, a display unit 200, a recognition unit 300, and a correction unit 400.

The storage unit 100 stores learning object information including learning characters, learning pronunciation organ shape, learning waveform, and learning frequency of the learning object.

The learning object is an object for which a hearing-impaired learner (1) trains language by speaking out loud.

Learning objects may be words composed of specific objects or titles such as 'school', 'teacher', 'sky', 'ah', 'ya', 'uh', 'yeo', 'ga', 'ga' It may be a single syllable such as , 'geo', or 'bran'. Additionally, it may be a sentence in which syllables and words are consecutive.

The learning text is a learning object written in Korean, and the learner (1) can read the learning text and recognize the learning object.

The learning pronunciation organ shape is an image showing the shape of the pronunciation organ when the above-mentioned learning characters are spoken. This may include the shape and position of the tongue, the shape of the lips or mouth, and the shape of the inside of the mouth.

The learner (1) can learn how to use the pronunciation organ necessary to properly sound the learning letter by imitating the shape of the pronunciation organ expressed as an image through the learning pronunciation organ shape and vocalizing it.

In addition, the learning pronunciation organ shape may further include image information about the pronunciation organ.

Language is composed of consonants and vowels combined, and each consonant and vowel is transmitted as voice through air vibration through the organic operation of the vocal and pronunciation organs.

Therefore, for accurate language learning, the learner (1) must learn how to move his or her own body by practicing not only the pronunciation organ but also how to use the vocal organ.

The learning waveform is a vibration waveform of the voice that propagates into the air when the above-mentioned learning text is read aloud. It has an amplitude that varies depending on the time axis.

The learning frequency is extracted from the frequency component of the above-described learning waveform, and can generally be obtained through Fourier transform.

The learning pronunciation organ shape, learning waveform, and learning frequency can be obtained when a normal person with normal hearing and language skills reads the learning text aloud.

It is desirable for people with normal hearing and language skills to have completed speech or pronunciation training or to be employed in a related industry.

For example, experts who have excellent Korean reading skills and can speak standard language, such as announcers and voice actors, are suitable.

The recognition unit 300 generates practice information comparable to the learning object information from the audio-visual expression 301 generated when the learner practices the language according to the learning object.

The recognition unit 300 generates practice information from the voice of the learner 1 or the shape of the learner's lips when the learner reads the learning text aloud.

Therefore, the recognition unit 300 may have an input means 320 including a voice recording device capable of recording the voice of the learner 1 and a video recording device for photographing the shape of the face or lips of the learner 1. .

The correction unit 400 generates correction information to minimize the difference between the learning object information and the practice information and displays it through the display unit 200.

The display unit 200 displays learning object information and correction information to the learner 1 (201).

At this time, it is desirable for the display unit 200 to clearly recognize the learning object to the learner by indicating the learning object using learning characters.

Accordingly, the learner 1 can receive quantitative and qualitative information as to whether the sound of the learning text read out loud by voice is similar to the sound of the learning text read out loud by the learning target information, that is, the sound of a normal person reading the learning text out loud.

This allows the learner to approach a learning object, that is, a normal person speaking aloud a learning letter, through repeated practice, and maximize the practice effect.

Figure 2 is a diagram showing a second embodiment of the speech language training system for the hearing impaired according to the present invention.

Referring to FIG. 2, in the speech language training system for the hearing impaired according to another embodiment of the present invention, the correction unit 400 derives a practice waveform and a practice frequency from the learner's voice and compares them with the learning waveform and learning frequency, respectively, to match the voices. It includes a voice analysis unit 410 that calculates the suitability as quantitative information, and the correction information includes the voice matching suitability.

At this time, the correction information may further include a comparison graph that compares the learning waveform and learning frequency with the practice waveform and practice frequency, respectively.

By comparing the learning waveform and the practice waveform, the learner (1) can check which point was wrong during the entire time he or she made the sound.

By comparing the learning frequency and the practice frequency, the learner (1) can find out which part of the frequency, i.e., high or low, is wrong.

In particular, high and low sounds are not displayed as Korean consonants and vowels.

However, if a normal person cannot control high and low sounds when communicating verbally with a hearing impaired person, the unnaturalness can be felt even more clearly than if the pronunciation of consonants and vowels is unclear.

Above all, by informing the learner of the suitability of voice matching, the learner (1) can check with quantitative information whether the situation is gradually improving or worsening during the repeated practice process.

Voice matching suitability is the statistical voice matching ratio (SVMR) for voices and basic words, and is calculated by comparing the envelope of each learning waveform and practice waveform, or the size of each frequency of the learning frequency and practice frequency. You may.

Alternatively, by comparing the movements of the learning pronunciation organ shape and the practice pronunciation organ shape extracted from the lip shape of the learner 1, the degree of consistency of the change in the lip shape can be used to calculate the suitability of voice matching.

Referring to FIG. 2, in the speech language training system for the hearing impaired according to this embodiment, the correction unit 400 includes a letter analysis unit ( 420), and at this time, the correction information includes letter matching suitability.

The above-mentioned voice matching suitability is quantitative information, and the high and low levels of the information are clear. The learner (1) can clearly know whether the voice he or she is producing is close to the learning waveform, learning frequency, or learning pronunciation organ shape.

However, the above-mentioned voice matching suitability is not an analysis of language, but an analysis method and result of voice, that is, sound waves.

Therefore, even though the learner (1) is training himself/herself in language, it is difficult to accurately judge the accuracy of pronunciation required for actual communication because he/she only receives data analyzed by sound waves of his/her own voice.

In other words, if the learning object is “Ah,” it is difficult for the learner (1) to distinguish whether the voice he or she produced is “Ah,” “Uh,” or “Wow.”

If this is calculated with the voice matching suitability calculated through simple sound wave analysis, various quantitative information can be provided, but the problem is that it is not clear what kind of sound, that is, what alphabet, the voice produced by the learner (1) sounds to others. There is.

In the speech language training system for the hearing impaired according to this embodiment, the learner (1) analyzes the voice he or she has produced as a sound wave itself to determine the suitability of voice matching and recognizes the voice he or she has produced as a language to obtain the correct language, that is, alphabet. You can simultaneously receive feedback on the letter matching suitability, which tells you whether you pronounced it as .

At this time, the correction unit 400 generates a practice pronunciation organ shape including the mouth shape through which the learner 1 pronounces the learning pronunciation organ, and the correction information includes a comparison image that compares the learning pronunciation organ shape and the practice pronunciation organ shape. .

In other words, the effect of language training can be maximized by providing the learner (1) with the shape and position of the pronunciation organ by dividing it into consonants and vowels and comparing the movement of the pronunciation organ of the learner with the movement of the pronunciation organ of a normal person.

Figure 3 is a diagram showing a third embodiment of the speech language training system for the hearing impaired according to the present invention.

In the speech language training system for the hearing impaired according to this embodiment, the correction unit 400 may further include a correction evaluation unit that generates at least one of voice matching trend information or letter matching trend information.

Voice matching trend information analyzes multiple voice matching suitability generated as the learner's language practice is repeated, and provides quantitative or qualitative information on whether the learner's language practice improves or worsens as the voice matching suitability increases or decreases. Display.

Letter matching trend information analyzes multiple letter matching suitability generated as the learner's language practice is repeated, and provides quantitative or qualitative information on whether the learner's language practice improves or worsens as the letter matching suitability increases or decreases. Display.

Accordingly, disabled people can learn how to control their own vocalization and pronunciation organs so that they can find the correct pronunciation and pronunciation method on their own.

Claims (7)

A storage unit that stores learning object information including learning characters, learning pronunciation organ shape, learning waveform, and learning frequency of the learning object;
a recognition unit that generates practice information comparable to the learning object information from the learner's language practice according to the learning object;
a correction unit that generates correction information to minimize the difference between the learning target information and the practice information; and
It includes a display unit that displays the learning object information and the correction information,
The correction department,
A voice analysis unit that derives a practice waveform and a practice frequency from the learner's voice and compares them with the learning waveform and the learning frequency, respectively, to calculate the suitability of mutual voice matching as quantitative information,
The correction information is,
It further includes a comparison graph that compares the voice matching suitability, the learning waveform and the learning frequency, and the practice waveform and the practice frequency, respectively,
The correction department,
By analyzing the plurality of voice matching suitability generated from the learner's repeated language practice, generating voice matching trend information indicating whether the learner's language practice improves or worsens as the voice matching suitability increases or decreases; , Analyzing a plurality of letter matching suitability generated from the learner's repeated language practice to generate letter matching trend information indicating whether the learner's language practice improves or worsens as the letter matching suitability increases or decreases. Calibration evaluation unit; and
It further includes a alphabet analysis unit that derives practice alphabets from the learner's voice and compares them with learning letters to calculate alphabet matching suitability,
The learner generates a practice pronunciation organ shape including a mouth shape for pronouncing the learning letter,
The correction information is,
A speech language training system for the hearing impaired including a comparison image comparing the letter matching suitability, the learning pronunciation organ shape, and the practice pronunciation organ shape. delete delete delete delete delete delete

Images