KR20140074449A

KR20140074449A - Apparatus and method for learning word by using native speaker's pronunciation data and word and image data

Info

Publication number: KR20140074449A
Application number: KR1020120142388A
Authority: KR
Inventors: 주홍찬
Original assignee: 주홍찬
Priority date: 2012-12-08
Filing date: 2012-12-08
Publication date: 2014-06-18

Abstract

The present invention relates to a word learning apparatus and method using image data and native pronunciation data
A storage unit for storing word data, image data of words, pronunciation images of native speakers, and sound waveform data; An input unit for word selection and word input; A display unit in which word data is displayed when learning of a word selected by the learner is started and image data related to the displayed word is displayed; An audio processing unit for receiving a learner's voice when a learner pronounces a word displayed at the time of word learning; A voice analysis module for analyzing the voice inputted through the audio processor and converting the voice into a waveform and comparing and analyzing the waveform of the native pronunciation data with respect to the inputted voice; A camera unit for collecting a mouth shape image when a learner pronounces a word; An image processing unit for normalizing the focus and size of the mouth-shaped image input from the camera unit; And a display unit for displaying a waveform image processed by the speech analysis module and the sound waveform processing unit and a stored original sound image of the native speaker, And controlling to display "good" when the similarity is 50% or more and "bad" when the similarity is less than 50%. The image processing apparatus according to claim 1, Provides a word learning device.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a word learning apparatus and method using word and image data and native pronunciation data,

The present invention relates to words and image data, and to a method and apparatus for generating vocabulary words and image data, and a method and apparatus for generating vocabulary words and vocabulary words (hereinafter referred to as " pronunciation data " More specifically, the word " interpreted in the language of the country in which the accent, the pronunciation symbol, and the word associated with the word are selected for learning " (Hereinafter referred to as " image data "), which can associate the word with an image or an animation or image And displays a lip shape reproduction picture of a native speaker's pronunciation and a lip shape reproduction picture of a learner's pronunciation, The display and, to a native pronunciation and vocabulary learning apparatus and method for comparing the similarity of the learner's pronunciation to words for effective learning to help pronunciation, using the image data and the data of the native pronunciation.

When learning a foreign language, it is very important to learn words that are the basis of meaning.

Therefore, foreign language vocabulary learning devices which record vocabulary words and meanings of vocabulary are organized.

In order to construct a foreign language sentence, learners memorize a lot of foreign language words which are the basis of meaning. However, existing foreign language word learning devices only list words, pronunciation symbols and word meaning, And memorization, it causes boredom, which reduces the interest of learning.

In addition, the existing foreign language word learning device can not know what words the learners are misreading when memorizing foreign language words, and does not provide a program to intensively learn words that are forgotten.

In addition, the existing foreign language word learning device does not provide a program that can help pronunciation correction, so it is possible to read a foreign language, but it is becoming difficult to speak foreign language.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to solve the above-mentioned problems and provide an image display apparatus and a display method thereof, which are capable of displaying word data to be learned and providing image data capable of reminding displayed words, And a word learning apparatus and method using assistant image data and native speaker pronunciation data.

It is also an object of the present invention to provide a method and apparatus for generating pronunciation data of native speakers,

The present invention provides a word learning apparatus and method using image data and native pronunciation data that can be corrected so that a learner sounds similar to a pronunciation of a native speaker in pronunciation of a word through comparison of data.

It is also an object of the present invention to provide a speech recognition apparatus,

The present invention provides a word learning apparatus and method using image data and native pronunciation data that enable a user to precisely check the position of a stress.

It is another object of the present invention to provide a word learning apparatus and a word learning apparatus using image data and native speaker pronunciation data for providing a wide range of language learning by providing various kinds of pronunciation data of native speakers, such as female, male, Method.

A word learning apparatus using image data and native speech data according to the present invention includes a storage unit storing word data, word image data, native speaker sound source, image and waveform data;

An input unit for word selection and word input;

A display unit for displaying word data and displaying image data of a displayed word when learning of the selected word starts;

When a learner pronounces a word displayed at the time of word learning,

An audio processor for receiving an input signal;

A voice analysis module and a voice waveform processor for analyzing the voice input through the audio processor and converting the voice into a waveform and comparing and analyzing the waveform of the native pronunciation data with respect to the inputted voice;

A camera unit for collecting a mouth shape image when a learner pronounces a word;

The learner's word input, which is input from the camera unit,

An image processing unit for normalizing the focus and size of data; And

The learner's pronunciation of a word standardized through the image processor,

Comparing the image with a word-pronounced image of a native speaker stored in the storage unit,

The waveform processed through the speech analysis module and the speech waveform processor and the pronunciation waveforms of the stored native speakers are displayed and the similarity is compared to control to display "good" in 50% or more similarity and "bad" in less than 50% And a control unit.

Also, when the control unit determines that the word < RTI ID = 0.0 >

And a wired / wireless network unit capable of displaying word data, image data, and pronunciation data of native speakers related to words matching words stored in the storage unit.

In addition, the control unit may select a word from the DMB English broadcast,

Image data and pronunciation data of native speakers related to a word corresponding to a word stored in the DMB module.

Further, the layer of the display unit is divided into Home, Dic and Option tabs

And the like.

In addition, the Home tab displays the words selected by the learner in sequence, and displays word-related word data related to displayed words, image data, faces, words when a native speaker pronounces words, mouth shapes when pronouncing words of native speakers, Shape, a voice waveform of a native speaker, and a voice waveform of a learner.

The Option tab is used to select whether to display image data of a word, to select an output time of a word once, to select whether or not video and audio reproduction is to be performed in video and audio reproduction in word pronunciation of a native speaker, A selection of whether or not to pronounce a pronunciation symbol, a selection of a pronunciation symbol, a selection of a pronunciation output of a country, and a selection of a pronunciation correction test mode.

In addition, in the word memorization test,

A Korean voice output, and an English voice output, and displays a question so that an answer can be selected in a multiple choice manner.

In addition, the controller may present a problem by pronunciation of a native speaker in a word memorization test, and allow the learner to input the word to confirm the correct answer.

A word learning method using image data and native pronunciation data according to the present invention includes: inputting a selection command of a word to be learned from a learner;

Displaying word data and image data according to the selected word;

Displaying an image in native pronunciation of the selected word;

When the learner's words are pronounced according to the displayed words,

A step to be performed;

When a word is pronounced by a native speaker

;

A step of displaying a comparison screen of a native speech waveform and a learner speech waveform; And

And displaying "bad" if the similarity degree between the native speech waveform and the learner speech waveform is "good" if the similarity is 50% or more and "bad" if the similarity is less than 50%.

According to the word learning method using the image data and the native pronunciation data of the present invention, word data and image data of a displayed word are provided together so that words can be more easily understood through visual information, By comparing the pronunciation data of the learner, it is possible to calibrate the learner so that the learner pronounces the word similar to the pronunciation of the native speaker. By comparing the loudspeakers, the intensity of the pronunciation and the location of the loudness can be checked in detail. There is an effect that the pronunciation data can be widely learned by providing various kinds of pronunciation data such as female, male, American pronunciation, English pronunciation, and phonetic pronunciation.

1 is a block diagram showing a configuration of a word learning apparatus using image data and native speech data according to an embodiment of the present invention.
2 shows an example of word selection for learning according to an embodiment of the present invention
drawing.
3 is a diagram showing an example of a word learning screen according to an embodiment of the present invention.
FIG. 4 is a diagram illustrating dictionary information of a word in a Dic tab according to an embodiment of the present invention
Fig.
FIG. 5 is a flowchart illustrating a method of selecting a word learning option in the Option tab according to an embodiment of the present invention.
Fig.
6 shows an example of a multiple-choice word memorizing method according to an embodiment of the present invention
Drawing.
FIG. 7 is a flowchart illustrating a method of inputting a word by listening to a pronunciation of a native speaker according to an embodiment of the present invention
Fig. 5 is a diagram showing an example of a word memorizing method of the expression; Fig.
FIG. 8 is a graph showing the results of multiple choice questions
A word that is not matched by the learner is checked and displayed as a list
Fig.
9 is a flowchart of a word learning method using image data and native pronunciation data according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, a word learning apparatus and method using image data and native pronunciation data of the present invention will be described with reference to the accompanying drawings.

1 is a block diagram showing a configuration of a word learning apparatus using image data and native pronunciation data according to an embodiment of the present invention.

Referring to FIG. 1, a word learning apparatus 100 using image data and native speaker pronunciation data includes a storage unit 104 storing word data, image data, pronunciations of native speakers, and speech waveform data, word selection and word input A display unit 114 for displaying word data and displaying image data of a displayed word when learning of a word selected by a learner is started; An audio processing unit 106 for receiving a voice, a voice analyzing unit 106 for analyzing the voice inputted through the audio processing unit 106, converting the voice into a waveform, comparing the waveform of the native pronunciation data with the similarity of the inputted voice, An analysis module unit 108 and a sound waveform processing unit 110, a camera unit 112 for collecting mouth-shaped images at the learner's pronunciation of a word, And an image processing unit (116) for normalizing the focus and size of the mouth image input from the image processing unit (12) The waveforms processed by the speech analysis module 108 and the speech waveform processor 110 and the pronunciation waveforms of the stored native speakers are displayed and the degree of similarity is compared with each other, The control unit 122 controls the display unit 122 to display the words stored in the storage unit 104 among the words displayed on the screen of the Internet web browser, A wired / wireless network unit 118 for displaying word data, image data, and pronunciation data of native speakers related to a matching word; And a DMB module 120 for displaying word data, image data, and pronunciation data of native speakers related to words that match the words stored in the storage unit 104.

Although not shown, the screen of the display unit 114 is divided into left and right

A screen of the Internet web browser or a DMB broadcast image is displayed on the left side,

A word matching the word stored in the storage unit 104 is displayed,

It is preferable that the word data, image data, and native pronunciation images stored in the storage unit 104 are reproduced when a word is selected.

This is because the learner can watch the screen of the Internet web browser or the DMB English broadcast

You can check the learning status of the word through the check box

There is an effect that can be reviewed.

The word learning apparatus 100 includes a computer terminal, a tablet PC, a language course,

And the input unit 102 includes a key pad or a touch screen module of the word learning apparatus 100 and a touch pan.

The audio processing unit 106 receives audio input and output and media output

Includes a microphone and speaker to make it work.

2 shows an example of word selection for learning according to an embodiment of the present invention

3 is a diagram showing an example of a learning screen of a word according to an embodiment of the present invention.

As shown in FIGS. 2 and 3, the learner first

Select a word.

As shown in FIG. 2, the learner selects the desired word

It is possible to select a word according to a predetermined level, and the method is not limited thereto.

When the learner has completed the selection of the words he /

3, the image data 11 related to the word, the face 12, the word data 13, and the mouth shape 14 when pronouncing the word of the native speaker are displayed on the Home tab 10, A face or mouth shape 15, a native speaker's voice waveform 16 and a learner's voice waveform 17 are displayed when a learner's word is pronounced, and a Next, Repeat, Slow button and a record and playback button are displayed at the bottom.

First, a word is displayed together with related image data and word data, and at the same time, a moving picture is reproduced when a word of a native speaker who pronounces the word is pronounced.

Thereafter, a pop-up message

The user can look up the camera provided in the word learning apparatus 100 and pronounce the corresponding word.

When voice recording and video recording are completed, a native speaker of the word

In the case of pronounciation, the pronunciation video with the enlarged mouth part and the learner's recorded pronunciation video are played.

The image processing unit 116 may display a mouth shape on a frame of the corresponding screen

It is desirable to display a requirement to the learner in the form of a pop-up menu so that the recorded image can be re-recorded if it can not be utilized as data.

In addition, the learner's pronunciation video reproduction is not the recording method but the present learner's

It is desirable that the learner is able to practice his or her mouth shape to be the same as the mouth shape of the native speaker while confirming the mouth shape of the learner directly through the screen.

At this time, the voice analysis module 108 analyzes the learner's voice,

Converted into a waveform through the speech waveform processor 110 based on the received data, and displayed so as to be compared with the native speech waveform.

FIG. 4 is a block diagram illustrating a dictionary of word information in a Dic tab according to an embodiment of the present invention.

As shown in Fig.

As shown in FIG. 4, when a word is being reproduced, the Dic tab 20

Click to see the dictionary meaning and example sentence of the word.

FIG. 5 is a diagram illustrating a word learning option line in Option tab according to an embodiment of the present invention.

Fig. 8 is a view showing an example of a tack list; Fig.

5, the Option tab 30 is used to select whether to display image data of a word, to select the output time of a word once, to reproduce video and audio in word pronunciation of a native speaker, A method of selecting whether or not to reproduce an image, selecting an accent representation, selecting whether to display a pronunciation symbol, selecting a gender of a voice output, selecting a pronunciation voice output for each country, and selecting a pronunciation correction test mode, So that the learner can adjust it in a way that is most appropriate for him / her.

FIG. 6 is a block diagram of a multiple word memorizing method according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating an example of a word memorizing method of a method of inputting a word by listening to a pronunciation of a native speaker according to an embodiment of the present invention.

As shown in Figs. 6 to 7,

Problems with text, text in English, output in Korean, or output in English

You can present a choice of answers by choosing a multiple choice, or you can present the problem with the pronunciation of the native speaker and let the learner enter the word to confirm the correct answer.

The way the learner enters the word is a multiple choice finger or a touch pen

You can select a number or enter a number using the keypad. If you type a word directly, you can write the word directly with your finger or touch pen, or you can type the word on the keypad.

FIG. 8 is a graph showing the multiple choice questions and the pronunciation of native speakers according to an embodiment of the present invention

Listen to words that are not matched by the learner,

Fig.

As shown in FIG. 8, when the user listens to the multiple choice questions and the pronunciation of native speakers,

A word which is not matched by the learner in the question of inputting a word is stored in the storage unit 104 so that it can be checked again in the list so that the learner can learn again,

9 is a flowchart of a word learning method using image data and native pronunciation data according to an embodiment of the present invention.

Referring to FIG. 9, the learners firstly learn through the input unit 102

(S200).

When the selection of the learning word through the input unit 102 is completed,

The image data and the word data of the selected word are displayed on the Home 10 of the display unit 114 and the image is displayed and reproduced in the native pronunciation of the word (S202).

Then, through the audio processing unit 106,

And acquires the face image during pronunciation of the learner through the camera unit 112 (S204).

A sound image of a native speaker stored in the storage unit 104 is reproduced,

So that the learner's normal-sounding image of focus and size is reproduced through the loudspeaker 116 (S206).

In the case of the learner's pronunciation,

The distance from the camera, the size of the face occupied by the image for each learner, and the like.

Accordingly, the image processing unit 116 recognizes the face shape of the learner

It is preferable to enlarge or reduce the size of the face in the frame so as to allow the learner's pronounced image to be displayed with the standardized focus and size.

At this time, when the learner pronounces the pronunciation,

Method can be applied.

Then, the learner's voice inputted through the audio processing unit 106

The speech analysis module 108 analyzes the data and converts the waveform into a waveform through the speech waveform processor 100 (S208).

The learner's voice waveform is the waveform of the native speech of the native speaker stored in the storage unit 104

It is displayed side-by-side so that they can be compared, and the learner can confirm where the native speaker pronounces strongly.

The waveform is a wave generated when the vibration shape of the sound is divided by time difference.

In the form of chapters, speech waveforms are an important factor in identifying accents and rhythms when pronouncing words.

When you pronounce English words, depending on which part you pronounce strongly

It is an essential part of the learner's ability to learn the pronunciation of a native speaker's pronunciation because it is different.

As described above, when a learner selects a desired word for learning,

The word data and the image data of the corresponding word are displayed and the moving picture is displayed when the native speaker is pronounced so that the learner can input the voice and the video in pronunciation of the word so that the learner can compare the voice and the video with the pronunciation of the native speaker, The waveform can be compared with the learner's voice waveform, thereby correcting the correct pronunciation of the word and acquiring an accurate accent.

In the above, the case where the learner learns English is taken as an example.

The present invention can also be applied to other languages such as Chinese, Japanese, German, French and so forth in addition to English.

100: language learning apparatus 102: input unit
104: storage unit 106: audio processing unit
108: Voice analysis module 110: Voice waveform processor
112: camera section 114: display section
116: image processing unit 118: wired / wireless network unit
120: DMB module unit 122:

Claims

A word learning apparatus using image data and native pronunciation data, the apparatus comprising: a storage unit for storing word data, word image data, native pronunciation images, and waveform data;
An input unit for word selection and word input;
A display unit for displaying word data and image data of the displayed word when learning of the selected word starts;
When a learner pronounces a word displayed in learning a word, the learner's voice is input
An audio processing unit for receiving the audio signal;
A voice input through the audio processing unit is analyzed and converted into a waveform, and a waveform for comparing and analyzing the waveform of the native pronunciation data with respect to the input voice
A speech analysis module and a speech waveform processor;
A camera unit for collecting a mouth shape image when a learner pronounces a word;
And the learner's pronunciation of the input image data
An image processing unit for normalizing the focus and size; And
A mouth shape image for word learner's pronunciation of a standardized learner through the image processing unit,
Comparing and displaying images of words of a native speaker stored in the storage unit,
The waveform processed through the speech analysis module and the speech waveform processor and the pronunciation waveforms of the stored native speakers are displayed and the similarity is compared to control to display "good" in 50% or more similarity and "bad" in less than 50% And a control unit for recognizing the pronunciation of the word.

The method according to claim 1,
The control unit stores the displayed words of the Internet web browser in the storage unit
And a wired / wireless network unit for displaying word data, image data, and pronunciation data of native speakers related to words matching the extracted words, and a word learning apparatus using image data and native pronunciation data.

The method according to claim 1,
Wherein the control unit determines whether words stored in the storage unit
And a DMB module for displaying word data, image data, and pronunciation data of native speakers related to a word corresponding to the word matching the word matching unit.

The method according to claim 1,
The layer of the display unit is divided into Home, Dic and Option tabs
A word learning apparatus using characteristic image data and native pronunciation data.

5. The method of claim 4,
In the Home tab, word selected by the learner is sequentially displayed, and word data and image data of the displayed word are displayed. When the word of the native speaker is pronounced, the face, word, Wherein the display means displays at least one of a mouth shape, a native speech waveform, and a learner's speech waveform.

5. The method of claim 4,
The option tab is used to select whether to display image data of a word, to select a word output time, to select video and audio reproduction in word pronunciation of a native speaker, Selecting at least one of pronunciation, pronunciation, pronunciation, and pronunciation; selecting a pronunciation symbol; selecting a pronunciation symbol; selecting a gender of a voice output; selecting a pronunciation voice output by a country; and selecting a pronunciation correction test mode. Learning device.

The method according to claim 1,
In the word memorization test, the control unit displays Korean text, English text,
Outputting and English speech output, and controlling the display so as to be able to select an answer in a multiple choice manner. The word learning apparatus using image data and native pronunciation data.

The method according to claim 1,
The control unit is configured to present a problem by pronunciation of a native speaker in a word memorization test,
And the correct answer is confirmed by allowing the user to input the word, and the word learning apparatus using the native pronunciation data.

A word learning method using image data and native pronunciation data, the method comprising: inputting a selection instruction of a word to be learned from a learner;
Displaying word data of the selected word and image data associated with the word; Displaying an image in native pronunciation of the displayed word; A step of inputting a voice and an image when a learner's word is pronounced according to the displayed word; A step of displaying a video when a word is pronounced by a native speaker and an image when a learner pronounces a word; And displaying a comparison screen of the native speech waveform and the learner speech waveform; And displaying the word " bad " when the similarity of the native speech waveform and the learner speech waveform is " good " if the similarity is 50% or more and less than 50%, respectively.