CN110349565B - Auxiliary pronunciation learning method and system for hearing-impaired people - Google Patents
Auxiliary pronunciation learning method and system for hearing-impaired people Download PDFInfo
- Publication number
- CN110349565B CN110349565B CN201910590429.3A CN201910590429A CN110349565B CN 110349565 B CN110349565 B CN 110349565B CN 201910590429 A CN201910590429 A CN 201910590429A CN 110349565 B CN110349565 B CN 110349565B
- Authority
- CN
- China
- Prior art keywords
- learner
- information
- pronunciation
- phoneme
- vibration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 208000032041 Hearing impaired Diseases 0.000 title claims abstract description 56
- 238000000034 method Methods 0.000 title claims abstract description 45
- 230000009471 action Effects 0.000 claims abstract description 39
- 238000012545 processing Methods 0.000 claims abstract description 27
- 238000006243 chemical reaction Methods 0.000 claims abstract description 21
- 238000004590 computer program Methods 0.000 claims description 20
- 230000009467 reduction Effects 0.000 claims description 9
- 238000010835 comparative analysis Methods 0.000 claims description 8
- 238000000605 extraction Methods 0.000 claims description 7
- 210000001847 jaw Anatomy 0.000 description 21
- 238000001228 spectrum Methods 0.000 description 21
- 238000010586 diagram Methods 0.000 description 19
- 241000209140 Triticum Species 0.000 description 7
- 235000021307 Triticum Nutrition 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 230000000694 effects Effects 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 210000001260 vocal cord Anatomy 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- 210000000988 bone and bone Anatomy 0.000 description 4
- 210000000867 larynx Anatomy 0.000 description 4
- 238000010183 spectrum analysis Methods 0.000 description 4
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 230000033001 locomotion Effects 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 210000000214 mouth Anatomy 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000035807 sensation Effects 0.000 description 2
- 238000012937 correction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 210000004373 mandible Anatomy 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 210000003437 trachea Anatomy 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B21/00—Teaching, or communicating with, the blind, deaf or mute
- G09B21/009—Teaching or communicating with deaf persons
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
- G10L2015/025—Phonemes, fenemes or fenones being the recognition units
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/225—Feedback of the input speech
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Electrically Operated Instructional Devices (AREA)
Abstract
The invention discloses an auxiliary pronunciation learning method for hearing-impaired people, which comprises the following steps: collecting pronunciation phoneme information sent by a pronunciation part of a learner through a learner pickup-feedback channel; processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and extracting learner characteristic information; matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information; converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information, and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel; the learner vibration information and the standard vibration information are respectively converted into vibration actions, and the pronunciation part of the learner is respectively vibrated, so that the learner can correct self-pronunciation, the efficiency of pronunciation learning is improved, and the accuracy of pronunciation learning is improved.
Description
Technical Field
The invention belongs to the technical field of voice processing, and particularly relates to a hearing-impaired person-oriented auxiliary pronunciation learning method and system.
Background
The hearing-impaired people generally describe the pronunciation method for the teacher to learn the pronunciation of Chinese language, so that the learner can perceive and learn by observing the lips of the teacher and touching the pronunciation part with fingers.
Because the hearing-impaired person does not have auditory feedback, the learner cannot sense the pronunciation of the learner, and the teacher needs to indirectly describe the pronunciation by gestures, characters, videos and the like to inform the learner of the learning effect; the teaching mode has long time consumption and low teaching efficiency, and the comprehension ability of the learner is limited by the expression mode of the teacher, so that the teaching effect is poor.
Disclosure of Invention
In view of this, the embodiment of the present invention provides an assistant pronunciation learning method and system for hearing-impaired people, which can solve the problems of low teaching efficiency and poor teaching effect.
The first aspect of the embodiment of the invention provides an auxiliary pronunciation learning method for hearing-impaired people, which comprises the following steps: collecting pronunciation phoneme information sent by a pronunciation part of a learner through a learner pickup-feedback channel; processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and extracting learner characteristic information; matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information; converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information, and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel; and respectively converting the learner vibration information and the standard vibration information into vibration actions, and respectively vibrating the pronunciation part of the learner.
In a first possible implementation manner of the first aspect of the embodiment of the present invention, the step of performing analog-to-digital conversion to process the pronunciation phoneme information into a phoneme digital signal and extract learner feature information includes: processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and converting the phoneme digital signal from a time domain to a frequency domain to obtain a phoneme spectrogram; and extracting learner characteristic information in the phoneme spectrogram, wherein the learner characteristic information comprises voice fundamental frequency information and phoneme characteristic peak information.
In a second possible implementation manner of the first aspect of the embodiment of the present invention, the number of the learner sound-pickup-feedback channels is multiple, the number of the learner pronunciation parts is multiple, and each of the learner sound-pickup-feedback channels corresponds to each of the learner pronunciation parts one by one.
With reference to the second possible implementation manner of the first aspect of the embodiment of the present invention, in a third possible implementation manner of the embodiment of the present invention, the step of converting the learner vibration information and the standard vibration information into vibration actions respectively to vibrate the pronunciation part of the learner respectively includes: converting a plurality of learner vibration information into learner vibration actions corresponding to the plurality of learner vibration information respectively, wherein each learner vibration action corresponds to each learner pronunciation part one to one; each learner vibration action vibrates the corresponding learner pronunciation part; converting the plurality of standard vibration information into standard vibration actions corresponding to the plurality of standard vibration information respectively, wherein each standard vibration action corresponds to each pronunciation part of the learner one by one; and each standard vibration action vibrates the corresponding pronunciation part of the learner respectively.
In a fourth possible implementation manner of the first aspect of the embodiment of the present invention, after the step of matching the learner feature information in a preset pronunciation phoneme set as standard phoneme information corresponding to the learner feature information, the method further includes: and displaying the comparative analysis result of the learner characteristic information and the standard phoneme information.
In a fifth possible implementation manner of the first aspect of the embodiment of the present invention, after the step of acquiring information of a phoneme of pronunciation uttered by a pronunciation part of a learner through a learner pickup-feedback channel, the method further includes: and performing noise reduction processing on the pronunciation phoneme information.
A second aspect of an embodiment of the present invention provides an auxiliary pronunciation learning system for hearing-impaired people, including: the acquisition module is used for acquiring pronunciation phoneme information sent by a pronunciation part of the learner through a learner pickup-feedback channel; the extraction module is used for processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion and extracting learner characteristic information; the matching module is used for matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information; the conversion module is used for converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel; and the vibration module is used for respectively converting the learner vibration information and the standard vibration information into vibration actions and respectively vibrating the pronunciation part of the learner.
In a first possible implementation manner of the second aspect of the embodiment of the present invention, the method further includes: and the display module is used for displaying the comparative analysis result of the learner characteristic information and the standard phoneme information.
A third aspect of the embodiments of the present invention provides an assistant pronunciation learning system for a hearing impaired person, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the steps of the assistant pronunciation learning method for a hearing impaired person as described in any one of the above items when executing the computer program.
A fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, where a computer program is stored, and the computer program, when executed by a processor, implements the steps of any one of the above-mentioned hearing-impaired person-oriented pronunciation-assisting learning methods.
Compared with the prior art, the embodiment of the invention has the following beneficial effects:
the embodiment of the invention collects the pronunciation phoneme information of the pronunciation part of the learner, converts the pronunciation phoneme information into vibration information and feeds the vibration information back to the learner pickup-feedback channel, feeds the vibration information converted from the standard phoneme information corresponding to the pronunciation phoneme information of the learner back to the learner pickup-feedback channel, and in addition, different pronunciation phoneme information corresponds to different vibration information; the learner can correct the self-pronunciation, the pronunciation learning efficiency is improved, the pronunciation learning accuracy is improved, and the teaching effect is further improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
Fig. 1 is a schematic flow chart illustrating a first embodiment of the auxiliary pronunciation method for hearing-impaired people according to the present invention;
fig. 2 is a flowchart illustrating a second embodiment of the auxiliary pronunciation method for hearing-impaired people according to the present invention;
fig. 3 is a schematic flow chart illustrating a third embodiment of the auxiliary pronunciation method for the hearing-impaired person according to the present invention;
FIG. 4 is a schematic diagram illustrating a feature map of different phonemes and a frequency band allocation of a speech acquisition unit according to an embodiment of the present invention;
FIG. 5 illustrates a sound collection vibration curve and spectrum of/ā/tone provided by an embodiment of the present invention;
FIG. 6 shows a graph of the spectrum analysis of/ā/tone on different channels provided by an embodiment of the present invention;
FIG. 7 illustrates sound collection vibration curves and spectra for/f/tone provided by embodiments of the present invention;
FIG. 8 is a graph showing spectral analysis of/f/tone in different channels according to an embodiment of the present invention;
FIG. 9 illustrates a sound collection vibration plot of a "storm" sound provided by an embodiment of the present invention;
FIG. 10 is a schematic structural diagram of a first embodiment of an assistant pronunciation learning system for hearing-impaired people according to the present invention;
FIG. 11 is a schematic structural diagram of a supplementary pronunciation learning system for hearing-impaired people according to a second embodiment of the present invention;
FIG. 12 is a schematic structural diagram of a third embodiment of the pronunciation-assisting learning system for hearing-impaired people according to the present invention;
FIG. 13 is a schematic structural diagram of a fourth embodiment of an assistant pronunciation learning system for hearing-impaired people according to the present invention;
FIG. 14 is a schematic structural diagram of a fifth embodiment of the pronunciation-assisting learning system for hearing-impaired people according to the present invention;
FIG. 15 is a diagram of an assistant pronunciation learning system for hearing impaired people according to an embodiment of the present invention;
fig. 16 is a schematic diagram of an assistant pronunciation learning system for hearing impaired people according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
In order to explain the technical means of the present invention, the following description will be given by way of specific examples.
The embodiment of the invention discloses a hearing-impaired person-oriented auxiliary pronunciation learning method and a system thereof.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a first embodiment of the assistant pronunciation learning method for hearing impaired people according to the present invention; specifically, the method comprises the following steps:
s101, collecting pronunciation phoneme information sent by a pronunciation part of a learner through a learner pickup-feedback channel;
learners are hearing-impaired persons who learn the pronunciation of phonemes, which are the smallest phonetic units divided according to the natural attributes of the speech, and are analyzed according to the pronunciation actions in syllables, and one action constitutes one phoneme. Phonemes are divided into two major categories, vowels and consonants. For example, the chinese syllable ā (o) has only one phoneme, the ai (i) has two phonemes, the d ā i (slow) has three phonemes, etc.
The learner's pickup-feedback channels are several, the learner's pronunciation parts are several, and optionally, each of the learner pickup-feedback channels corresponds to each of the learner's pronunciation parts one by one.
The multiple sound producing parts can be at least one of throat part, chin part, lip part and face part. Different learner pronunciation parts can be respectively and correspondingly provided with a learner pickup-feedback channel. Throat position can correspond and set up the larynx sound passageway, and the chin position can correspond and set up the jaw sound passageway, and the lip can correspond and set up the lip sound passageway, and the face can correspond and set up total passageway.
Further, the learner sound pick-up-feedback channel can be realized under the condition that the learner sound pick-up-feedback channels respectively do not correspond to the pronunciation parts of the learners one by one.
The learner pickup-feedback channels can be at least one of a laryngeal channel, a jaw sound channel, a lip sound channel and a total channel, and the total channel can assist the laryngeal channel, the jaw sound channel and the lip sound channel; the sound channel may be any two or more of a laryngeal channel, a jaw sound channel, and a lip sound channel.
The voice collecting device can be respectively arranged on different learner pickup-feedback channels to collect voices of different learner pronunciation parts, or the voice collecting device can be arranged on the face, such as a microphone to collect all voice information, so as to assist voice collection of other parts.
The sound pickup device can be arranged in the sound pickup-feedback channel of the learner to collect the information of the pronunciation phonemes sent by the pronunciation part of the learner; specifically, the method comprises the following steps: a throat sound channel can adopt a throat microphone such as a bone conduction sound pick-up to collect vocal cord vibration information and vowel information of the throat part; the jaw sound channel can collect vowel information and consonant information of the lower jaw part; burst sound information and frictional sound information of lips can be collected in the lip sound channel; it is also possible to collect all speech information of the face, e.g. the part near the lips, in the global channel by means of whole or global microphone. Wherein the sound pickup can be a full directional sound pickup, a bone conduction sound pickup or a wind noise perception sensor. Compared with the traditional recording equipment adopting a single-channel or double-channel pickup method, the method is easy to cause inaccurate pronunciation detail recognition. According to the embodiment of the invention, the pronunciation parts of a plurality of learners are identified, and pronunciation phoneme information is respectively collected at the proximity of the larynx, the lower jaw and the lip, so that accurate phoneme identification can be realized.
With reference to fig. 4, fig. 4 is a schematic diagram illustrating a feature diagram of different phonemes and a frequency band allocation diagram of a speech acquisition unit according to an embodiment of the present invention, where in fig. 4, 41 is a frequency region of a laryngeal channel; 42 is a jaw sound channel sampling frequency area; 43 is the lip sound channel sampling frequency region; 44 total microphone frequency region;
the frequency and loudness regions of the different phonemes are shown in fig. 4. The box is the frequency interval collected by the pickup-feedback channels of different learners. The vowels in the phonemes need vocal cords to vibrate for sounding and are low in frequency, so that the bone conduction larynx microphones can well collect sounds in the frequency range; the plosive and fricative frequencies in the phonemes are high and are emitted by the lips, so lip microphones can be used for acquisition; other pronunciation phonemes can be collected by the lower jaw microphone; the main wheat can collect all sound information.
The phonemes of human pronunciation have different frequency intervals, such as vowels of/ā/,/ō/,/yamin/,/ī/,/g/,/ǖ/etc., have lower sound frequencies and are voiced by trachea, so can be picked up by the throat channel pickup at the throat part, the jaw channel pickup at the chin part can also be picked up, but the signal intensity is not as strong as the throat channel, but the lip channel pickup at the lip part is weak because only high-frequency fricatives and wind noise are collected.
Consonants such as/m/,/n/,/ng/and the like have lower frequency intervals, and the lips are closed when being pronounced, so that the signal of the laryngeal channel is strongest, and the lip channel has almost no signal.
If the consonants are unvoiced such as/f/,/k/, etc., the vocal cords do not sound, so that the laryngeal channel has no signal, the jaw channel signal is weak, and the lip channel signal is strongest.
If the consonants and the plosives are/g/,/b/etc., the laryngeal channel, the jaw channel and the lip channel all have signals.
As an embodiment of the present invention, in conjunction with fig. 5, fig. 5 shows a sound collection vibration curve and a frequency spectrum of/ā/tone provided by the embodiment of the present invention, where 51 is a wholewheat vibration signal; 52 is a laryngeal channel vibration signal; 53 is jaw sound channel vibration signal; 54 is lip sound channel vibration signal; the spectrum of the whole wheat signal is 55; 56 is the laryngeal channel signal spectrum; 57 is the jaw tone channel signal spectrum; and 58 is the lip sound channel signal spectrum.
As can be seen from fig. 5, the throat channel records clear vocal cord vibration information, the jaw channel records vibration information of sound after oral resonance, and the lip channel signal is weak because/ā/sound is low in intensity in a high frequency region and wind noise is small.
FIG. 6 is a graph of the spectrum analysis of/ā/tone on different channels according to the embodiment of the present invention, wherein 61 is the whole wheat signal spectrum; 62 is the laryngeal channel signal spectrum; 63 is a jaw tone channel signal frequency spectrum; and 64 is the lip sound channel signal spectrum.
The characteristics of this pronunciation are further illustrated in fig. 6. Therefore, the signal strength of/ā/tone collected in the laryngeal channel, the jaw channel and the lip sound channel is approximately: strong-weak.
As another specific embodiment of the present invention, in conjunction with fig. 7, fig. 7 shows a sound collection vibration curve and a frequency spectrum of/f/sound provided by the embodiment of the present invention, wherein 71 is a whole wheat vibration signal; 72 is the laryngeal channel vibration signal; 73 is a jaw sound channel vibration signal; 74 is lip sound channel vibration signal; the spectrum of the whole wheat signal is 75; 76 is the laryngeal channel signal spectrum; 77 is the jaw tone channel signal frequency spectrum; and 78 is the lip sound channel signal frequency spectrum. As can be seen from fig. 7, since/f/is unvoiced consonant, vocal cords do not vibrate, so that the laryngeal channel has no vibration information, and the jaw channel also has almost no signal. The lip sound channel records the signal of/f/sound.
FIG. 8 is a graph of frequency spectrum analysis of the/f/tone in different channels according to the embodiment of the present invention, wherein 81 is the whole wheat signal frequency spectrum; 82 is the laryngeal channel signal spectrum; 83 is jaw tone channel signal frequency spectrum; 84 lip sound channel signal frequency spectrum. The characteristics of this pronunciation are further illustrated in fig. 8. Therefore, the signal strength of the/f/sound collected in the laryngeal sound channel, the jaw sound channel and the lip sound channel is approximate to: weak-strong.
As another embodiment of the present invention, in conjunction with fig. 9, fig. 9 is a graph of sound collection vibration of a "storm" sound provided by the embodiment of the present invention, where 91 is a throat channel vibration signal; 92 is jaw tone channel vibration signal; and 93 is a lip sound channel vibration signal.
Table 1 shows the recorded vibration intensity of three channels in a "storm" sound, representing the recorded vibration intensity of three channels under successive phonemes.
TABLE 1 vibration intensity recorded in "storm" sound by three channels
| Pronunciation phoneme | b- | -à- | ò | f- | -ē- | ng |
| Laryngeal channel | Weak → strong | High strength | High strength | Is free of | High strength | Middle strength |
| Jaw sound channel | Weak → strong | High strength | High strength | Weak (weak) | High strength | High strength |
| Lip sound channel | High strength | Weak (weak) | Weak (weak) | High strength | Weak (weak) | Is free of |
S102, processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and extracting learner characteristic information;
wherein the learner characteristic information comprises voice fundamental frequency information and phoneme characteristic peak information;
the pronunciation phoneme information may be converted into a phoneme digital signal using an analog/digital converter (a/D converter), wherein the pronunciation phoneme information is an analog signal collected by a microphone.
When the pronunciation phoneme information of a plurality of learner pronunciation parts is collected, analog-to-digital conversion processing is respectively carried out on each pronunciation phoneme information, and each learner characteristic information corresponding to each learner pronunciation part is respectively extracted.
Specifically, S102 includes, for example,
s1021, processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and converting the phoneme digital signal from a time domain to a frequency domain to obtain a phoneme spectrogram;
s1022, extracting learner feature information in the phoneme spectrogram, wherein the learner feature information comprises voice fundamental frequency information and phoneme feature peak information.
When the pronunciation phoneme information of a plurality of learner pronunciation parts is collected, respectively transforming each phoneme digital signal from a time domain to a frequency domain to obtain each phoneme spectrogram; and extracting common learner characteristic information from each phoneme spectrogram, wherein the learner characteristic information comprises voice fundamental frequency information and phoneme characteristic peaks.
S103, matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information;
the preset pronunciation phoneme set can be a teacher pronunciation phoneme set obtained by collecting pronunciation phonemes of a teacher; or the real-time pronunciation phoneme set obtained by collecting pronunciation phonemes sent in real time when the teacher guides the learner to learn, wherein the pronunciation phonemes sent by the teacher are the same pronunciation phonemes sent by the learner.
The preset pronunciation phoneme set can also be a standard pronunciation phoneme set established by collecting phonemes sent by pronunciators with different ages, different sexes and different timbres.
When the preset pronunciation phoneme set is the teacher pronunciation phoneme set, the pronunciation phoneme information of the learner can be collected at this time, and the corresponding standard phoneme information is matched in the teacher pronunciation phoneme set.
When the preset pronunciation phoneme set is a real-time pronunciation phoneme set, the pronunciation phoneme sent by the instructor is used as the standard phoneme information corresponding to the pronunciation phoneme information of the learner.
When the preset pronunciation phoneme set is the standard pronunciation phoneme set, the pronunciation phoneme information of the learner can be collected and matched with the standard pronunciation phoneme set, the participation of a teacher is not needed, teaching resources are saved, and the learner can independently learn and correct the pronunciation of the learner.
S104, converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information, and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel;
and converting the phoneme information into vibration information and feeding back the vibration information to the pronunciation part of the learner. Each phoneme information corresponds to vibration information of a specific different learner pickup-feedback channel combination.
And S105, respectively converting the learner vibration information and the standard vibration information into vibration actions, and respectively vibrating the pronunciation part of the learner.
The vibration action includes the position, vibration frequency, amplitude, time difference and the like of vibration; each piece of phoneme information corresponds to a specific vibration action.
Converting the learner vibration information and the standard vibration information into vibration actions respectively, vibrating the pronunciation part of the learner, and leading the learner to feel the vibration sensation caused by the phoneme sent by the learner and the vibration sensation caused by the standard phoneme; the learner can feel the difference between the self-uttered pronunciation phoneme and the standard phoneme.
The collected pronunciation phoneme information sent by the pronunciation part of the learner is finally converted into a vibration action, and the pronunciation part of the learner is vibrated, so that the learner can intuitively feel the source, vibration frequency and vibration strength of the sound and more intuitively feel the position and reason of the deviation between the phoneme sent by the learner and the standard phoneme.
If the vowel pronunciation is not accurate, the learner can feel that the feedback of the pronunciation phoneme and the standard phoneme is different in the throat; if the fricative pronunciation is not accurate, the learner will feel different feedback of his own pronunciation phoneme and the standard phoneme on the lip.
And because of the difference of physiological structures, the pronunciation tone of different learners may be different, the embodiment of the invention can ignore the pronunciation tone of different learners by converting the collected pronunciation phoneme information of learners into vibration information, unify the same phoneme information sent by different learners into the same vibration reference frequency, and can simplify the learning cost.
Specifically, the S105 includes, for example,
s1051, converting a plurality of learner vibration information into learner vibration actions corresponding to the plurality of learner vibration information respectively, wherein each learner vibration action corresponds to each learner pronunciation part one by one;
s1052, each of the learner 'S vibration motions vibrating the corresponding learner' S pronunciation part;
the learner can respectively set vibrating devices corresponding to throat part, lower jaw part and lip part, and can sense the pronunciation phoneme sent by the learner according to the vibration of the vibrating devices. Wherein vibrating device can be rotor vibrating motor, linear motor, bone conduction speaker, miniature speaker etc. can select for use different vibrating device according to being suitable for the scene.
S1053, converting the plurality of standard vibration information into a plurality of standard vibration actions corresponding to the standard vibration information respectively, wherein each standard vibration action corresponds to each pronunciation part of the learner one by one;
each of the canonical phoneme information corresponds to a specific canonical vibration information, and further corresponds to a specific canonical vibration action of a different learner pronunciation-feedback channel combination.
If the vocal cords vibrate most strongly when the learner pronounces the vowel, the vibration combination of the vowel can be set to be the most intense vibration sense of the laryngeal channel, and the vibration frequency is equal to the frequency of the vowel pronounced by the learner. If the learner only makes a sound by lips when making a fricative sound, the lip sound channel can make high-frequency vibration, and other channels do not vibrate; the vibration combination of the fricatives may be set to be the strongest in the lip sound channel and the frequency of the vibration is equal to the frequency of the fricatives uttered by the learner. Thereby converting the pronunciation of the learner into vibration information to be fed back to the learner.
The learner can feel the difference between the self pronunciation and the standard sound vibration feedback, so that the learner can exercise by himself and correct the self pronunciation. Until the learner's pronunciation is very close to the standard sound, the learner will perceive that the learner's pronunciation is correct if the learner's vibration feedback is the same as that of the standard sound.
And S1054, each standard vibration action vibrates the corresponding pronunciation part of the learner.
As yet another embodiment of the present invention, Table 2 shows the vibration feedback of a standard "storm" tone on different channels.
TABLE 2 vibration intensity and frequency at different channels for standard "storm" tones
The embodiment of the invention collects the pronunciation phoneme information of the pronunciation part of the learner, converts the pronunciation phoneme information into vibration information and feeds the vibration information back to the learner pickup-feedback channel, feeds the vibration information converted from the standard phoneme information corresponding to the pronunciation phoneme information of the learner back to the learner pickup-feedback channel, and in addition, different pronunciation phoneme information corresponds to different vibration information; the learner can correct the self-pronunciation, the pronunciation learning efficiency is improved, the pronunciation learning accuracy is improved, and the teaching effect is further improved.
The embodiment of the invention can also extract the learner characteristic information common to all the pronunciation phoneme information by collecting the pronunciation phoneme information of a plurality of learner pronunciation parts, and convert the pronunciation phoneme information into a vibration action to be fed back to the pickup-feedback channel of the learner, thereby increasing the accuracy rate of feedback.
With reference to fig. 2, fig. 2 is a flowchart illustrating a second embodiment of the method for assisting pronunciation learning for a hearing impaired person according to the present invention, after S103, further comprising,
s106, displaying the comparative analysis result of the learner characteristic information and the standard phoneme information.
The comparative analysis result can include pronunciation similarity, pronunciation improvement method, pronunciation anatomical map, pronunciation oral cavity diagram, indication of pronunciation deviation reasons such as wrong mouth shape position, pronunciation waveform comparison, etc.
The results of the comparative analysis may be displayed on a display; the learner can further improve and improve the learning efficiency through the information displayed by the display.
At the moment, the embodiment of the invention feeds back to the learner in an image and vibration mode, thereby improving the learning efficiency of the learner and improving the teaching effect.
With reference to fig. 3, fig. 3 is a flowchart illustrating a third embodiment of the method for assisting pronunciation learning for a hearing impaired person according to the present invention, after S101, further comprising,
s107, noise reduction processing is carried out on the pronunciation phoneme information.
And carrying out noise reduction processing on the acquired pronunciation phoneme information, and filtering out environmental noise or noise, thereby being beneficial to subsequent analog-to-digital conversion processing.
When pronunciation phoneme information sent by different pronunciation parts of a learner is collected, the audio range collected by the sound pick-up of each learner sound pick-up-feedback channel may be larger than a required audio interval, noise reduction processing can be performed on the collected pronunciation phoneme information respectively, unnecessary audio information can be filtered out through a digital filtering method, and effective information is extracted, wherein the effective information refers to the pronunciation phoneme information after the noise reduction processing.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
In an embodiment of the present invention, an assistant pronunciation learning system for hearing-impaired people is further provided, where the assistant pronunciation learning system for hearing-impaired people includes modules for executing the steps in the embodiment corresponding to fig. 1. Please refer to fig. 1 for the related description of the corresponding embodiment.
Fig. 10 is a schematic structural diagram of a first embodiment of the pronunciation-assisting learning system for hearing-impaired people according to the present invention. As shown in fig. 10, an assistant pronunciation learning system 2 for hearing impaired people according to this embodiment includes:
the acquisition module 21 is used for acquiring pronunciation phoneme information sent by a pronunciation part of the learner through a learner pickup-feedback channel;
the extraction module 22 is used for processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion and extracting learner characteristic information;
the matching module 23 is configured to match the learner feature information in a preset pronunciation phoneme set, where the matching is used as standard phoneme information corresponding to the learner feature information;
the conversion module 24 is configured to convert the learner characteristic information into learner vibration information, convert the standard phoneme information into standard vibration information, and respectively feed back the learner vibration information and the standard vibration information to the learner pickup-feedback channel;
and the vibration module 25 is used for respectively converting the learner vibration information and the standard vibration information into vibration actions and respectively vibrating the pronunciation part of the learner.
Fig. 11 is a schematic structural diagram of a supplementary pronunciation learning system for hearing-impaired people according to a second embodiment of the present invention. As shown in fig. 11, the extraction module 22 of this embodiment includes:
a spectrum module 221, configured to perform analog-to-digital conversion on the pronunciation phoneme information to obtain a phoneme digital signal, and transform the phoneme digital signal from a time domain to a frequency domain to obtain a phoneme spectrogram;
an extracting sub-module 222, configured to extract learner feature information in the phoneme spectrogram, where the learner feature information includes fundamental frequency information of speech and feature peak information of phonemes.
Fig. 12 is a schematic structural diagram of a third embodiment of the pronunciation-assisting learning system for hearing-impaired people according to the present invention. As shown in fig. 12, the vibration module 25 of this embodiment includes:
a learner converting sub-module 251, configured to convert a plurality of learner vibration information into learner vibration actions corresponding to the plurality of learner vibration information, respectively, where each learner vibration action corresponds to each learner pronunciation part one to one;
a learner vibrator module 252 for vibrating the corresponding learner pronunciation part by each learner vibration motion;
a standard phoneme conversion sub-module 253, configured to convert the plurality of standard vibration information into a plurality of standard vibration actions corresponding to the plurality of standard vibration information, where each standard vibration action corresponds to each pronunciation part of the learner one to one;
and a standard phoneme vibrator module 254 for vibrating the corresponding pronunciation part of the learner by each standard vibration motion.
Fig. 13 is a schematic structural diagram of a fourth embodiment of the pronunciation learning aid system for hearing impaired people according to the present invention. As shown in fig. 13, this embodiment further includes: the display module 26;
the display module 26 is configured to display a result of the comparative analysis between the learner characteristic information and the phoneme standard information.
Fig. 14 is a schematic structural diagram of a fifth embodiment of the pronunciation-assisting learning system for hearing-impaired people according to the present invention. As shown in fig. 14, this embodiment further includes: a noise reduction processing module 27;
the noise reduction processing module 27 is configured to perform noise reduction processing on the pronunciation phoneme information.
Fig. 15 is a schematic structural diagram of an assistant pronunciation learning system for hearing impaired people according to an embodiment of the present invention. Wherein 100 is a learner; 22 is an extraction module; 103 is a display; 104 is a larynx pick-up; 105 is a mandible pickup; 106 is a lip pickup; 107 is total wheat; 108 is a throat vibrating device; 109 is a lower jaw vibration device; 110 is a lip vibrating device. The embodiment of the invention can respectively reflect the information of vowels, consonants, plosives and fricatives of learners through different vibration characteristics, and the display 103 can display information such as pronunciation comprehensive accuracy, pronunciation improvement method, pronunciation anatomical map and the like.
Fig. 16 is a schematic diagram of an assistant pronunciation learning system for hearing impaired people according to an embodiment of the present invention. As shown in fig. 16, the system 6 for assisting pronunciation learning for hearing impaired people includes a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and operable on the processor 60, such as a program for implementing assisting pronunciation learning for hearing impaired people. The processor 60 executes the computer program 62 to implement the steps of the above-mentioned embodiments of the method for assisting pronunciation learning for hearing impaired people, such as S101 to S105 shown in fig. 1. Alternatively, the processor 60, when executing the computer program 62, implements the functions of the modules/units in the above-described device embodiments, such as the functions of the modules 21 to 25 shown in fig. 10.
Illustratively, the computer program 62 may be partitioned into one or more modules/units that are stored in the memory 61 and executed by the processor 60 to implement the present invention. The one or more modules/units may be a series of computer program instruction segments capable of performing specific functions, which are used to describe the execution process of the computer program 62 in the hearing impaired person-oriented pronunciation-assisting learning device 6. For example, the computer program 62 may be divided into an acquisition module, an extraction module, a matching module, a transformation module, and a vibration module (module in a virtual device), and each module has the following specific functions:
the acquisition module is used for acquiring pronunciation phoneme information sent by a pronunciation part of the learner through a learner pickup-feedback channel;
the extraction module is used for processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion and extracting learner characteristic information;
the matching module is used for matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information;
the conversion module is used for converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel;
and the vibration module is used for respectively converting the learner vibration information and the standard vibration information into vibration actions and respectively vibrating the pronunciation part of the learner.
The auxiliary pronunciation learning device 6 for hearing impaired people can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The hearing impaired person-oriented pronunciation-assisting learning device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will appreciate that fig. 16 is only an example of the hearing impaired person oriented auxiliary pronunciation learning apparatus 6, and does not constitute a limitation of the hearing impaired person oriented auxiliary pronunciation learning apparatus 6, and may include more or less components than those shown, or combine some components, or different components, for example, the hearing impaired person oriented auxiliary pronunciation learning apparatus 6 may further include an input/output device, a network access device, a bus, etc.
The Processor 60 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The memory 61 may be an internal storage unit of the auxiliary pronunciation learning device 6 for the hearing impaired, such as a hard disk or a memory of the auxiliary pronunciation learning device 6 for the hearing impaired. The memory 61 may also be an external storage device of the auxiliary pronunciation learning apparatus 6 for hearing impaired people, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are equipped on the terminal device. Further, the memory 61 may include both an internal storage unit and an external storage device of the auxiliary pronunciation learning apparatus 6 for the hearing impaired person. The memory 61 is used for storing the computer program and other programs and data required by the terminal device. The memory 61 may also be used to temporarily store data that has been output or is to be output.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
The auxiliary pronunciation learning system for the hearing-impaired person in the embodiment of the invention can be applied to the learning of pronunciation phonemes of the hearing-impaired person and can also be applied to the correction of pronunciation in the daily spoken language communication of a learner.
Collecting phoneme information sent by a communication object of a learner, converting the phoneme information into vibration information, feeding the vibration information back to a pickup-feedback channel of the learner, and vibrating a pronunciation part of the learner to enable the learner to feel and understand the speaking information of the communication object.
The auxiliary pronunciation learning system for hearing impaired people in the embodiment of the invention can also be applied to learning of other languages such as English, French and cantonese.
For example, in english, the pronunciation information is divided into vowels, consonants, etc. according to different pronunciation positions, the pronunciation information of different pronunciation parts is collected, converted into vibration information and fed back to the learner, so as to vibrate the pronunciation part of the learner, help the learner self-correct pronunciation, and improve learning efficiency.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implemented, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated modules/units, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. . Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may comprise any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, a software distribution medium, etc. It should be noted that the computer readable medium may contain content that is subject to appropriate increase or decrease as required by legislation and patent practice in jurisdictions, for example, in some jurisdictions, computer readable media does not include electrical carrier signals and telecommunications signals as is required by legislation and patent practice.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present invention, and are intended to be included within the scope of the present invention.
Claims (7)
1. An assistant pronunciation learning method for hearing-impaired people is characterized by comprising the following steps:
collecting pronunciation phoneme information sent by a pronunciation part of a learner through a learner pickup-feedback channel;
processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and extracting learner characteristic information;
matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information;
converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information, and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel;
respectively converting the learner vibration information and the standard vibration information into vibration actions, and respectively vibrating the pronunciation part of the learner;
the step of processing the pronunciation phoneme information into phoneme digital signals by the analog-to-digital conversion and extracting learner characteristic information comprises the following steps:
processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion, and converting the phoneme digital signal from a time domain to a frequency domain to obtain a phoneme spectrogram;
extracting learner characteristic information in the phoneme spectrogram, wherein the learner characteristic information comprises voice fundamental frequency information and phoneme characteristic peak information;
the learner's pickup-feedback channels are several, the learner's pronunciation parts are several, and each learner's pickup-feedback channel is respectively corresponding to each learner's pronunciation part one by one; the pickup-feedback channel of the learner is as follows: a laryngeal channel, a jaw tone channel, a lip tone channel and a main channel; the sounding part is as follows: throat, chin, lips and face;
the step of respectively converting the learner vibration information and the standard vibration information into vibration actions and respectively vibrating the pronunciation part of the learner comprises the following steps of:
converting a plurality of learner vibration information into learner vibration actions corresponding to the plurality of learner vibration information respectively, wherein each learner vibration action corresponds to each learner pronunciation part one to one;
each learner vibration action vibrates the corresponding learner pronunciation part;
converting the plurality of standard vibration information into standard vibration actions corresponding to the plurality of standard vibration information respectively, wherein each standard vibration action corresponds to each pronunciation part of the learner one by one;
and each standard vibration action vibrates the corresponding pronunciation part of the learner respectively.
2. The method for learning the auxiliary pronunciation as claimed in claim 1, wherein the step of matching the learner feature information with a predetermined pronunciation phoneme set as the standard phoneme information corresponding to the learner feature information further comprises:
and displaying the comparative analysis result of the learner characteristic information and the standard phoneme information.
3. The method for learning auxiliary pronunciation as claimed in claim 1, wherein the step of collecting pronunciation phoneme information uttered by the pronunciation part of the learner through the learner pick-up-feedback channel is followed by further comprising:
and performing noise reduction processing on the pronunciation phoneme information.
4. A hearing impaired person oriented pronunciation aid learning system for performing the method according to any one of claims 1 to 3, comprising:
the acquisition module is used for acquiring pronunciation phoneme information sent by a pronunciation part of the learner through a learner pickup-feedback channel;
the extraction module is used for processing the pronunciation phoneme information into a phoneme digital signal through analog-to-digital conversion and extracting learner characteristic information;
the matching module is used for matching the learner characteristic information in a preset pronunciation phoneme set to serve as standard phoneme information corresponding to the learner characteristic information;
the conversion module is used for converting the learner characteristic information into learner vibration information, converting the standard phoneme information into standard vibration information and respectively feeding back the learner vibration information and the standard vibration information to the learner pickup-feedback channel;
and the vibration module is used for respectively converting the learner vibration information and the standard vibration information into vibration actions and respectively vibrating the pronunciation part of the learner.
5. The system for assisting pronunciation learning of an auditory handicapped person according to claim 4, further comprising:
and the display module is used for displaying the comparative analysis result of the learner characteristic information and the standard phoneme information.
6. An assisted pronunciation learning device for hearing impaired persons comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method according to any one of claims 1 to 3 when executing the computer program.
7. Computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910590429.3A CN110349565B (en) | 2019-07-02 | 2019-07-02 | Auxiliary pronunciation learning method and system for hearing-impaired people |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910590429.3A CN110349565B (en) | 2019-07-02 | 2019-07-02 | Auxiliary pronunciation learning method and system for hearing-impaired people |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110349565A CN110349565A (en) | 2019-10-18 |
| CN110349565B true CN110349565B (en) | 2021-03-19 |
Family
ID=68177526
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910590429.3A Active CN110349565B (en) | 2019-07-02 | 2019-07-02 | Auxiliary pronunciation learning method and system for hearing-impaired people |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110349565B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853624A (en) * | 2019-11-29 | 2020-02-28 | 杭州南粟科技有限公司 | Speech rehabilitation training system |
| CN112885168B (en) * | 2021-01-21 | 2022-09-09 | 绍兴市人民医院 | Immersive speech feedback training system based on AI |
| CN114758647A (en) * | 2021-07-20 | 2022-07-15 | 无锡柠檬科技服务有限公司 | Language training method and system based on deep learning |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2272097A (en) * | 1992-10-30 | 1994-05-04 | Visible Sound Ltd | Improvements relating to teaching aids for the deaf. |
| US20040194610A1 (en) * | 2003-03-21 | 2004-10-07 | Monte Davis | Vocal pitch-training device |
| US20100235170A1 (en) * | 2009-03-12 | 2010-09-16 | Rothenberg Enterprises | Biofeedback system for correction of nasality |
| CN108320625A (en) * | 2018-04-17 | 2018-07-24 | 长春大学 | Vibrational feedback system towards speech rehabilitation and device |
| CN109598991A (en) * | 2019-01-11 | 2019-04-09 | 张翩 | A kind of pronunciation of English tutoring system, device and method |
-
2019
- 2019-07-02 CN CN201910590429.3A patent/CN110349565B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2272097A (en) * | 1992-10-30 | 1994-05-04 | Visible Sound Ltd | Improvements relating to teaching aids for the deaf. |
| US20040194610A1 (en) * | 2003-03-21 | 2004-10-07 | Monte Davis | Vocal pitch-training device |
| US20100235170A1 (en) * | 2009-03-12 | 2010-09-16 | Rothenberg Enterprises | Biofeedback system for correction of nasality |
| CN108320625A (en) * | 2018-04-17 | 2018-07-24 | 长春大学 | Vibrational feedback system towards speech rehabilitation and device |
| CN109598991A (en) * | 2019-01-11 | 2019-04-09 | 张翩 | A kind of pronunciation of English tutoring system, device and method |
Non-Patent Citations (3)
| Title |
|---|
| 《基于移动端和PC的双端听障康复训练系统研究》;张淑艳等;《无线互联科技》;20170930(第18期);第140-142页 * |
| 《基于舌部可视化模型的听障儿童发音训练》;史丽娟等;《光学精密工程》;20150228;第23卷(第2期);第582-588页 * |
| 《言语可视化在发音错误分析中的作用》;蔡莲红等;《NCMMSC 2015》;20151031;第1-5页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110349565A (en) | 2019-10-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Tran et al. | Improvement to a NAM-captured whisper-to-speech system | |
| US20200294509A1 (en) | Method and apparatus for establishing voiceprint model, computer device, and storage medium | |
| CN110349565B (en) | Auxiliary pronunciation learning method and system for hearing-impaired people | |
| CN106575500B (en) | Method and apparatus for synthesizing speech based on facial structure | |
| US5340316A (en) | Synthesis-based speech training system | |
| US20160314781A1 (en) | Computer-implemented method, computer system and computer program product for automatic transformation of myoelectric signals into audible speech | |
| Hansen et al. | On the issues of intra-speaker variability and realism in speech, speaker, and language recognition tasks | |
| Freitas et al. | An introduction to silent speech interfaces | |
| CN106448673B (en) | chinese electronic larynx speech conversion method | |
| CN110970036A (en) | Voiceprint recognition method and device, computer storage medium and electronic equipment | |
| Karpov | An automatic multimodal speech recognition system with audio and video information | |
| CN114121006A (en) | Image output method, device, equipment and storage medium of virtual character | |
| CN106303695A (en) | Audio translation multiple language characters processing method and system | |
| Rudzicz | Production knowledge in the recognition of dysarthric speech | |
| Meltzner et al. | Speech recognition for vocalized and subvocal modes of production using surface EMG signals from the neck and face. | |
| CN109767750B (en) | Voice radar and video-based voice synthesis method | |
| WO2017008075A1 (en) | Systems and methods for human speech training | |
| Mishra et al. | An Overview of Hindi Speech Recognition | |
| Wand | Advancing electromyographic continuous speech recognition: Signal preprocessing and modeling | |
| JP4381404B2 (en) | Speech synthesis system, speech synthesis method, speech synthesis program | |
| Valaki et al. | A hybrid HMM/ANN approach for automatic Gujarati speech recognition | |
| CN107203539B (en) | Speech evaluating device of complex word learning machine and evaluating and continuous speech imaging method thereof | |
| CN111724764B (en) | Method and device for synthesizing music | |
| Fuchs et al. | Turbulent sounds: An interdisciplinary guide | |
| Ravindran et al. | Cepstral and linear prediction techniques for improving intelligibility and audibility of impaired speech |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |