KR101877559B1 - Method for allowing user self-studying language by using mobile terminal, mobile terminal for executing the said method and record medium for storing application executing the said method - Google Patents

Abstract

The present invention provides a language self-learning service using a mobile terminal that enables a user to efficiently and easily perform pronunciation correction, tone correction, vocalization correction, and the like in language learning such as a foreign language by utilizing a recording function of a mobile terminal ≪ / RTI >
Specifically, the present invention provides a method of providing a language self-learning service using a mobile terminal, the method comprising: generating an original learning file for generating or storing voice information for language learning; A primary audio file playback step of outputting primary input audio information of the original learning file using the sound output unit of the mobile terminal; And a dubbing file generation step of receiving the secondary input voice information, which is voice information of the user, using the acoustic input unit of the mobile terminal, and generating and storing the secondary input voice information as a dubbing file.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method for providing a language self-learning service using a mobile terminal, a mobile terminal for executing the same, and a recording medium storing an application for executing the same. MEDIUM FOR STORING APPLICATION EXECUTING THE SAID METHOD}

The present invention provides a language self-learning service using a mobile terminal that enables a user to efficiently and easily perform pronunciation correction, tone correction, vocalization correction, and the like in language learning such as a foreign language by utilizing a recording function of a mobile terminal A mobile terminal for implementing the method, and a recording medium storing a program for implementing the method.

Modern age is a global era, and due to the arrival of this global age, language learning devices or language learning programs capable of studying foreign languages have been continuously developed in various forms.

In particular, the conventional language device has been developed as a cassette having a sound reproducing function, a CD player, a DVD player, an MP3 player, a smart phone, and the like. These linguistic devices have been developed not only for the sound reproduction function but also for the linguistic pronunciation of the learner due to the development of the recording function.

An example of such a language device according to the related art is a recording device disclosed in Korean Patent Laid-Open Publication No. 10-2009-0119598.

However, Korean Laid-Open Patent Publication No. 10-2009-0119598 and the like are constructed so that only the learner's voice information can be recorded separately after temporarily stopping or stopping the voice signal of the learning contents.

As a result, the apparatus for language according to the prior art has a time period for reproducing the audio signal of the learning contents (for example, conversation contents of foreigners, foreigner's voice information, etc.) , The learner himself or herself is made to be as similar or coincident with the speech signal of the learning content (that is, with a similar tone, with similar pronunciation, or at a similar rate of speech, for how much foreigner's conversation or foreigner's voice information is) It has been argued that it is not easy to recognize whether or not it is pronounced.

Accordingly, an object of the present invention is to solve the problems in the prior art.

More specifically, it is an object of the present invention to provide a mobile terminal that enables a user to efficiently, precisely, and easily perform pronunciation correction, tone correction, and utterance speed correction during language learning such as a foreign language, A service providing method and a recording medium storing the service providing method.

According to an aspect of the present invention, there is provided a method of providing a language self-learning service using a mobile terminal, the method comprising: generating an original learning file for generating language learning voice information; A primary audio file playback step of outputting primary input audio information of the original learning file using the sound output unit of the mobile terminal; And a dubbing file generation step of receiving the secondary input voice information, which is the voice information of the user, using the sound input unit of the mobile terminal and generating and storing the secondary input voice information as a dubbing file, and providing the self learning service using the mobile terminal can do.

Preferably, in the step of generating the original learning file, the primary input voice information, which is the language learning voice information to be learned by the user, is recorded using the sound input unit of the mobile terminal and is generated and stored as an original learning file Is executed.

Preferably, the original learning file generation step is performed in accordance with a user's input signal or automatically. And extracting and storing language learning audio information from the audio information content and the moving picture content stored in the memory unit of the mobile terminal.

Preferably, in the step of generating the original learning file, the language learning audio information or the language learning content that the user wants to learn is downloaded from the language self learning service providing server using the wireless communication unit of the mobile terminal, To be generated and stored.

Preferably, the dubbing file generating step includes: a user input signal receiving step of receiving, from the user, a user input signal for executing the dubbing file generating step after the first audio file reproducing step starts; A secondary input voice information receiving step of driving the acoustic input unit of the mobile terminal based on the user input signal to receive the secondary input voice information; A dubbing file generating step of superimposing the secondary input audio information on the primary input audio information of the original learning file to generate superposed audio information; And a dubbing file storing step of storing the superimposed audio information in a memory unit of the mobile terminal.

Preferably, the primary audio file reproducing step includes a user input signal receiving step of receiving a user input signal for executing the primary audio file reproducing step from the user, wherein the user input signal is received The first audio file reproducing step and the dubbing file generating step are executed simultaneously.

Preferably, the at least one of the original learning file generation step and the dubbing file generation step includes a noise removal step of removing ambient noise speech information from the input speech information.

Preferably, the method further includes a secondary audio file playback step of outputting superimposed audio information of the dubbing file using the sound output unit of the mobile terminal.

Preferably, the secondary audio file reproducing step includes: a user input signal receiving step of receiving from the user a user input signal for executing the secondary audio file reproducing step; And outputting audio information of a dubbing file to output superimposed audio information of the dubbing file based on the user input signal.

Preferably, a user input signal receiving step of receiving, from the user, a user input signal for executing the dubbing file reproducing step after the secondary audio file reproducing step; A third input voice information receiving step of driving the sound input unit of the mobile terminal based on the user input signal to receive third input voice information which is re-input voice information of the user; A dubbing file reproducing step of superimposing the third-order input audio information on the primary input audio information of the original learning file to generate additional superposed audio information; And an additional dubbing file storing step of storing the additional overlapping audio information in a memory unit of the mobile terminal.

Preferably, the dubbing file generation step may include a step of generating a first dubbing file in which the volume magnitude of the primary input voice information is larger than the volume magnitude of the secondary input voice information, Or to generate a second dubbing file in which the volume magnitude of the secondary input audio information is larger than the volume magnitude of the primary input audio information or to generate both the first dubbing file and the second dubbing file .

It is preferable that the secondary audio file reproducing step is performed to output the first dubbing file or to output the second dubbing file in accordance with a user input signal or automatically, And sequentially outputting the second dubbing file or sequentially outputting the second dubbing file and the first dubbing file.

Preferably, the method for providing a language self-learning service using the mobile terminal further includes a step of, after the dubbing file generation step, extracting and storing only the secondary input voice information from the superimposed voice information of the dubbing file, And an information extracting step.

Preferably, the secondary audio file reproducing step includes: an original learning file audio information output step of outputting primary input audio information which is audio information of the original learning file; A secondary input voice information output step of outputting the secondary input voice information; And a dubbing file audio information output step of outputting superimposed audio information which is audio information of the dubbing file.

Preferably, the script file generation step is executed after the start of the original learning file generation step, and extracts text information from the original learning file and generates and stores the text information as a script file; And a script file output step executed after the start of the primary audio file playback step and outputting text information of the script file using the display unit of the mobile terminal.

Preferably, the script file output step is performed so as to visually change the text information based on the reproduction time information of the primary input audio file after the start of the primary audio file reproduction step .

According to another embodiment of the present invention for solving the above-mentioned problems, the present invention can provide a mobile terminal that executes the above-described method for providing a language self-learning service using the mobile terminal.

According to another embodiment of the present invention, there is provided a recording medium storing an application for executing a method for providing a language self-learning service using the mobile terminal.

According to a solution to the above-mentioned problem, the present invention is configured to generate a dubbing file and reproduce the dubbing file, so that the learner simultaneously compares the linguistic voice information pronounced with the linguistic voice information of the original learning file, The user can accurately and easily correct pronunciation, tonality, and utterance speed during learning of his / her own language (e.g., a foreign language).

In addition, since the speech information for language learning can be directly recorded and generated, the language learning contents can be easily generated indefinitely, and the diversity of language learning contents can be significantly improved.

In addition, the present invention can extract only voice information from moving picture contents and facilitate language learning using foreign movies, foreign dramas, and the like.

In addition, the present invention is configured to download and use language learning audio information or language learning contents through a service providing server, so that a service providing server provider can sell various kinds of audio information or contents to generate revenue.

Further, the present invention can provide a user with the sound quality of the original learning file and / or the dubbing file with improved clear sound quality by including the noise eliminator, thereby improving the user's language learning convenience.

Further, the present invention is configured to generate a new dubbing file during playback of the dubbing file, so that a new dubbing file can be easily generated even if the user mistakenly pronounces it.

Further, the present invention can generate a dubbing file such that one of the original learning file audio information (i.e., primary input audio information) and the secondary input audio information in the dubbing file is relatively louder than the other, Selectively or sequentially, so that the user's self-pronunciations can be significantly improved and the user's learning effect on the language can be enhanced.

In addition, according to the present invention, only the secondary input voice information is extracted and stored separately, and the original learning file, the secondary input voice information, and the dubbing file can be selectively and / or sequentially reproduced, It is possible to make it possible to efficiently calibrate the pronunciation voice information of the user by repeatedly comparing the voice information of the original learning file.

In addition, since the present invention is configured to generate and output a script, a learner can easily perform a dubbing operation when creating a dubbing file, and since the language learning contents are simultaneously provided audibly and visually, The learning effect can be remarkably improved.

In addition, the present invention is configured to visually change the text information of the script in synchronization with the reproduction time of the original learning file after the start of the dubbing file generation step, thereby making it possible for the user to perform the dubbing operation more easily and accurately. As a result, it is possible to remarkably improve the user's language learning pronunciation learning effect and / or the language pronunciation correcting effect.

1 is a schematic block diagram of a mobile terminal according to an embodiment of the present invention.
2 is a schematic flow chart of a method for providing a language self-learning service according to an embodiment of the present invention.
FIG. 3 is a detailed flowchart of FIG. 2. FIG.
FIG. 4 is a schematic flow chart of an original learning file generation step according to a further embodiment.
FIG. 5A is a schematic view of a voice file reproduction output screen output to the display unit when the primary audio file reproduction step according to the present invention is executed. FIG. Fig. 3 is a schematic view of a dubbing output screen to be outputted; Fig.
6 is a schematic flow chart of a method for providing a language self-learning service according to a first additional embodiment.
7 is a detailed flowchart of Fig.
8 is a schematic flow chart of a method for providing a language self-learning service according to a second additional embodiment.
9 is a schematic flow chart of the script file generation step of FIG.
10 is a schematic flow chart of the script file output step of FIG.

Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. It should be noted that the drawings denoted by the same reference numerals in the drawings denote the same reference numerals whenever possible, in other drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. And certain features shown in the drawings are to be enlarged or reduced or simplified for ease of explanation, and the drawings and their components are not necessarily drawn to scale. However, those skilled in the art will readily understand these details.

1 is a schematic block diagram of a mobile terminal 1000 according to an embodiment of the present invention.

1, a mobile terminal 1000 according to an exemplary embodiment of the present invention includes a wireless communication unit 110, an input unit 120 including a touch-type input unit 121 and a sound input unit 122, An output unit 150 including an audio output unit 151 and a display unit 152, a memory unit 160 for storing various information and data, an interface unit 170, A learning service providing unit 140, and a control unit 130 for controlling the above-described components.

The mobile terminal 1000 may include a mobile phone, a smart phone, a laptop computer, a tablet PC, a personal digital assistant (PDA), a wearable device, and the like.

The wireless communication unit 110 is configured to enable wireless data communication with the mobile terminal 1000 and a language self-learning service providing server.

The input unit 120 includes a touch-type input unit 121 that receives a user's input signal as a user's signal, and an acoustic input unit 122 that receives a user's input signal as a user's voice signal.

The touch-type input unit 121 may include a touch screen, a keyboard, a physical button, and the like.

The sound input unit 122 includes a microphone, and the microphone can receive a voice signal of a user by a microphone and process the voice as electrical voice information (data). The processed voice information (data) can be converted into a form recognizable by the control unit 130 and transmitted. The microphone may be implemented with various noise reduction algorithms for eliminating noise generated in receiving an external sound signal. Of course, as shown in the figure, the language self-learning service provider 140 may include a separate noise eliminator 148.

The interface unit 170 serves as a path to all the external devices connected to the control unit 130 in terms of electrical or communication signals. The interface unit 170 receives data from an external device or supplies power to each component in the mobile terminal 1000 or transmits data in the mobile terminal 1000 to an external device. For example, a wired / wireless headset port, a charging port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / An input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The memory unit 160 may be a flash memory type, a hard disk type, a multimedia card micro-type, a card type memory (for example, SD Or XD memory), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read- Memory, a magnetic memory, a magnetic disk, an optical disk, and a solid state drive (SSD). In addition, the mobile terminal 1000 may operate in association with a web storage that performs a storage function of the memory unit 160 on the Internet.

The output unit 150 includes an audio output unit 151 and a display unit 152. The sound output unit 151 may include an external sound device connection terminal such as a speaker, an earphone, or a headphone.

The control unit 130 may be included in the mobile terminal 1000 such as the wireless communication unit 110, the input unit 120, the output unit 150, the memory unit 160, and the interface unit 170 Components and programs (control algorithms).

The language self-learning service provider 140 may send a language self-learning service application to the mobile terminal 1000 (or the mobile terminal 1000) via the wireless communication unit 110, The control unit 130 or the control unit 130 in the case of downloading and executing the control algorithm or program to the memory unit 160 of the control unit 130. [

The speech self-learning service providing unit 140 includes a recorder driving unit 141, a reproduction driving unit 142, a dubbing driving unit 143, an audio information filtering unit 144, a display driving unit 145, A text converting unit 146, a noise removing unit 148, and a voice information extracting unit 149. [

The recorder driving unit 141 activates the sound input unit 122 (e.g., a microphone) through the input unit 120 and the control unit 130 to execute the recording function. The recorder driving unit 141 is configured to drive an original learning file generation step (S1100) to be described later.

The playback drive unit 142 activates the audio output unit 151 (e.g., a speaker) through the input unit 120 and the control unit 130 to execute a play function for playing back the moving picture content or the audio information. The playback drive unit 142 is configured to drive a first audio file playback step (S1200) and a second audio file playback step (1400), which will be described later. The playback drive unit 142 may include an iterative playback unit for repeatedly playing back one audio information or file, and an interval repetition playback unit for repeatedly playing only a certain portion of one audio information.

The dubbing driver 143 activates the sound input unit 122 (for example, a microphone) through the input unit 120 and the controller 130 to perform a recording function. The dubbing driver 143 is configured to drive a dubbing file generating step (S1300) and a dubbing file reproducing step, which will be described later.

The voice information filter unit 144 removes the band corresponding to the primary input voice information from the dubbing file including the primary input voice information and the secondary input voice information in a superimposed manner to extract only the secondary input voice information do.

The display driver 145 displays the screen information, the UI information, and the icon-type input unit 120 related to the language self-learning service through the display unit 152 and the control unit 130 on the display unit 152 do.

The voice-to-text conversion unit 146 converts the language learning audio information (i.e., the recorded original learning file) received through the recorder driving unit 141 into text information and stores the text information in the memory unit 160. [ The voice-to-text conversion unit 146 extracts corresponding reproduction time information corresponding to the text information from the language learning voice information, and stores the extracted reproduction time information in the memory unit 160.

The noise removing unit 148 is configured to remove noise from the sound information and / or sound information received from the sound input unit 122 (i.e., microphone) through the input unit 120 and the control unit 130. [ The voice information filter unit 144 is configured to drive a noise removing step to be described later.

The voice information extracting unit 149 is configured to extract only voice information (for example, conversation contents in a foreign movie or drama) from the moving picture contents stored in the memory unit 160 of the mobile terminal 1000. [ The extracted voice information can be used as an original learning file.

Each component of the language self-learning service providing unit 140 will be described in more detail in a description of a language self-learning service providing method (S1000) using the mobile terminal 1000 according to the present invention described below do.

FIG. 2 is a schematic flow chart of a method for providing a language self-learning service (S1000) according to an embodiment of the present invention, FIG. 3 is a detailed flowchart of FIG. 2, 5A is a schematic flow chart of the generation step S1100 of FIG. 5A. FIG. 5A is a schematic flowchart of the generation of the audio file reproduction output screen 152a output to the display unit 152 when the primary audio file reproduction step S1200 according to the present invention is executed And FIG. 5B is a schematic view of a dubbing output screen 152b output to the display unit 152 when the dubbing file generation step (S1300) according to the present invention is executed.

2 and 3, a method S1000 for providing a language self-learning service according to an embodiment of the present invention is implemented using the components and control algorithms included in the mobile terminal 1000 described above An original learning file generation step (S1100); A primary audio file playback step (S1200); And a dubbing file generation step (S1300), and further includes a secondary audio file reproduction step (1400).

The control unit 130 may activate or deactivate the sound input unit 122 in conjunction with the recorder driving unit 141 to generate or store language learning audio information.

More specifically, according to the present embodiment, the original learning file generation step S1100 is a step of generating the original learning file using the sound input unit 122 of the mobile terminal 1000, Information is recorded and created and stored as an original learning file. Here, the primary input audio information may include a foreign language conversation content, a foreign language class content, a foreign language movie content, a foreign language music lyrics, a foreign language news content, and the like according to a user's selection, and does not limit the kind thereof.

That is, the controller 130 receives the first user input signal input through the touch-type input unit 121 (S1110). The first user input signal is a signal input by the user through the touch-type input unit 121 so that the user drives the recorder driving unit 141 and the sound input unit 122.

Preferably, after receiving the first user input signal in the first user input signal receiving step (S1110), the control unit determines that the remaining storage capacity of the memory unit is less than a predetermined capacity (for example, 100 Mbytes) The control unit may control the display unit and / or the sound output unit to output the capacity shortage notification information that allows the user to visually and / or audibly recognize the capacity shortage state of the memory unit.

Here, the capacity shortage notification information may be text information such as "There is not enough storage space, delete unnecessary files and try again."

Then, the control unit 130 receives primary input audio information (or language learning audio information) input through the audio input unit 122 (S1120). Here, the primary input audio information is a language learning audio information, Contents of a foreign language class, content of a foreign language movie, lyrics of a foreign language music, contents of a foreign language news, and the like.

Then, the controller 130 generates the primary input audio information as an original learning file (S1130), and stores the original learning file in the memory unit (S1140). And can be reproduced by the reproduction drive section 142. [

Preferably, during the execution of the primary input voice information receiving step (S1120), the control unit 130 activates the noise removing unit 148 to remove the surrounding noise voice information from the primary input voice information .

4, the original learning file generation step S1100 may be performed in accordance with a user's input signal or automatically stored in the memory unit 160 of the mobile terminal 1000 Extracting speech information for language learning from audio information contents and moving picture contents, and storing the extracted audio information.

In operation S1150, the control unit 130 searches the moving picture content and displays the moving picture content list on the display unit 152, among the key data and information stored in the memory unit 160.

In step S1160, the control unit 130 activates the playback driver 142 to play the selected moving image content according to the input signal of the user when the input signal of the user is received. At this time, the moving picture content may be output to the actual display unit 152, but only the audio information output step to be performed later may be performed without being output to the display unit 152. [

In step S1170, the control unit 130 activates the audio information extraction unit 149 to extract only audio information from the moving picture content played by the playback drive unit 142. [

Thereafter, the control unit 130 stores the extracted voice information in the memory unit 160 as an original learning file (S1180).

Although not shown in the drawing, in still another embodiment, the original learning file generation step S1100 may be performed by using the wireless communication unit 110 of the mobile terminal 1000, The language learning contents can be downloaded from the language self learning service providing server and generated and stored as an original learning file.

After the original learning file generation step S1100, the controller 130 activates the audio output unit 151 and the playback driving unit 142 to output the primary input audio information of the original learning file (S1200).

Specifically, the control unit 130 receives the second user input signal input through the touch-type input unit 121 (S1210). Here, the second user input signal is input to a user Signal.

The control unit 130 drives the playback driver 142 and the audio output unit 151 to output audio information of the original learning file (i.e., primary input audio information or audio information for language learning) (S1220) .

At this time, the control unit 130 can sequentially execute the dubbing file generation step (S1300).

According to a further embodiment, when the second user input signal is received, the control unit 130 controls the reproducing operation so that the first audio file reproducing step (S1200) and the subsequent dubbing file generating step (S1300) The sound output unit 151, the dubbing driving unit 143, and the sound input unit 122 may be activated at the same time.

Preferably, the control unit may output the audio file reproduction output screen 152a as shown in FIG. 5A to the display unit during the execution of the primary audio file reproduction step S1100.

The audio file reproduction output screen 152a includes a reproduction step control section 121a, a reproducible file list section 121b, and a dubbing execution section.

The playback step control unit 121a is a touch-type input unit for receiving an input signal of a user as well as an output screen, and includes a playback / pause control icon unit, a back-up icon unit, An icon section for specifying a repeated section, and an icon section for volume control.

The playable file list unit 121b is an output screen for a file list in a memory unit that can be reproduced by the language self-learning service providing unit 140 and is a touch-type input unit for receiving a user's input signal at the same time.

The dubbing execution unit includes a dubbing guidance information output unit 152a-1 and a dubbing execution icon 121c.

When the user's input signal is received through the dubbing execution icon 121c, the control unit controls the display unit to switch the audio file reproduction output screen 152a of FIG. 5A to the dubbing output screen 152b of FIG. 5B .

The dubbing output screen 152b is configured the same as the audio file reproduction output screen 152a except for the dubbing execution unit. The dubbing execution unit of the dubbing output screen 152b includes a dubbing execution time output unit 152b-1, a dubbing status information output unit 152b-2, a dubbing execution / pause control icon unit 121d, And an icon portion 121e.

The configuration of the above-described output screen is illustrative, and the present invention is of course not limited thereto.

The control unit 130 activates the sound input unit 122 and the dubbing driving unit 143 of the mobile terminal 1000 to reproduce the voice information of the user (i.e., the learner) The secondary input audio information is received and controlled to be generated and stored as a dubbing file (S1300). Here, the secondary input audio information is audio information that the user has pronounced according to the audio information of the original learning file.

Specifically, the control unit 130 receives a third user input signal for executing the dubbing file generation step (S1300) from the user (S1310) after the first audio file playback step (S1200) is started. For example, the control unit starts the dubbing file generation step (S1300) when receiving a user's input signal through the dubbing execution icon unit 121c in FIG. 5A.

Thereafter, the control unit 130 drives the sound input unit 122 of the mobile terminal 1000 to receive the secondary input voice information based on the third user input signal (S1320)

Thereafter, the control unit 130 superimposes the secondary input audio information temporally on the primary input audio information of the original learning file (or superimposes the reproduction time and the dubbing time), and generates superimposed audio information as a dubbing file (S1330).

Thereafter, the control unit 130 stores the dubbing file containing the overlapping audio information in the memory unit 160 of the mobile terminal 1000 (S1340).

Preferably, during the execution of the secondary input voice information receiving step (S1320), the control unit 130 activates the noise removing unit 148 to control to remove the surrounding noise voice information from the secondary input voice information .

The dubbing file generation step S1300 may be performed by the second user input signal inputted in the second user input signal receiving step S1210 of the first audio file reproducing step S1200 as described above, May be executed by the third user input signal input in the third user input signal receiving step S1310 of the file generating step S1300.

After the dubbing file generation step S1300, the control unit 130 activates the sound output unit 151 and the playback driving unit 142 of the mobile terminal 1000 and transmits the superimposed audio information of the dubbing file to the user (S1400).

Specifically, the control unit 130 receives a fourth user input signal for executing the second audio file playback step 1400 through the touch-type input unit 121 from the user (S1410).

Thereafter, the control unit 130 outputs the superimposed audio information of the dubbing file to the audio output unit 151 based on the fourth user input signal (S1420).

As a further embodiment, after the secondary audio file reproduction step 1400, the control unit 130 controls the dubbing drive unit 143, the memory unit 160 and the audio output The control unit 151 can be controlled.

Specifically, first, the control unit 130 receives a fifth user input signal for executing the dubbing file regeneration step (not shown) from the user.

Then, the control unit 130 controls the memory unit 160, the reproduction drive unit 142, and the sound output unit 151 to reproduce the audio information of the original recording file based on the fifth user input signal.

The control unit 130 drives the sound input unit 122 and the dubbing driving unit 143 of the mobile terminal 1000 based on the fifth user input signal to generate tertiary input voice information .

Thereafter, the controller 130 superimposes the third-order input audio information on the primary input audio information of the original learning file to generate additional superposed audio information.

Then, the control unit 130 stores the additional superposed voice information in the memory unit 160 of the mobile terminal 1000.

As a further embodiment of the dubbing file generation step S1300, the control unit 130 may control the volume of the primary input voice information and the secondary input voice, It is possible to control to generate a dubbing file by relatively adjusting the volume of the information.

Specifically, the controller 130 generates a first dubbing file in which the volume level of the primary input voice information is larger than the volume level of the secondary input voice information, or the volume level of the secondary input voice information It is possible to generate a second dubbing file that is larger than the volume level of the primary input audio information or to generate both the first dubbing file and the second dubbing file.

As an alternative embodiment, the dubbing file generation step (S1300) is executed as an initial setting to generate a first dubbing file in which the volume magnitude of the primary input voice information is smaller than the volume magnitude of the secondary input voice information, The size of the primary input audio information and the size of the secondary input audio information may be adjusted in accordance with the input signal of the user. That is, the dubbing file generation step (S1300) is basically performed so that the first dubbing file (i.e., a dubbing file in which the volume level of the primary input voice information is smaller than the volume level of the secondary input voice information) The second input audio information may be generated to generate a second dubbing file in which the volume magnitude of the secondary input voice information is smaller than the volume magnitude of the primary input voice information according to a user's input signal input through the input unit later, A third dubbing file having the same magnitude of the volume of the voice information and the volume magnitude of the secondary input voice information can be executed.

In a further embodiment of the dubbing file generation step S1300, as a further embodiment of the secondary audio file reproduction step 1400, the control unit 130 may generate a secondary audio file according to a user input signal (manually) , Outputting the first dubbing file through the sound output unit 151 or outputting the second dubbing file through the sound output unit 151 or outputting the first dubbing file and the second dubbing file sequentially And outputting the second dubbing file and the first dubbing file sequentially through the sound output unit 151. In this case,

FIG. 6 is a schematic flow chart of a method for providing a language self-learning service (S2000) according to a first additional embodiment, and FIG. 7 is a detailed flowchart of FIG.

The method for providing a language self-learning service (S2000) according to the present embodiment substantially includes the technical features included in the language self-learning service providing method (S1000) according to the embodiment described above, The detailed description of the contents overlapping with the above-mentioned contents will be omitted as much as possible.

According to the present embodiment, the present invention provides an original learning file generation step (S2100); A primary audio file playback step (S2200); A dubbing file creation step (S2300); Secondary input speech information extraction step (S2400); And a secondary audio file playback step (S2500).

Here, the original learning file generation step S2100, the primary audio file playback step SQ200, and the dubbing file generation step S2300 according to the present embodiment may include an original learning file generation step S1100, (S1200) and the dubbing file generation step (S1300).

According to the present embodiment, after the dubbing file generation step (S2300), the control unit 130 extracts and stores only the secondary input voice information in the superimposed voice information of the dubbing file S2400).

Specifically, first, the control unit 130 copies the dubbing file and stores the copy dubbing file in the memory unit.

Thereafter, the control unit 130 activates the playback driver and the audio voice information filtering unit 144 to control to remove the primary input audio information from the audio information of the copy dubbing file (S2410).

Thereafter, the control unit 130 generates secondary input audio information (learner audio information) remaining in the audio information of the copy dubbing file (S2420).

Then, the control unit 130 stores the secondary input voice information in the memory unit (S2430).

The secondary audio file reproducing step S2500 according to the present embodiment includes an original learning file audio information output step S2520 of outputting primary input audio information which is audio information of the original learning file; A secondary input voice information output step (S2530) for outputting the secondary input voice information; And a dubbing file audio information output step (S2540) for outputting superimposed audio information which is audio information of the dubbing file.

For example, the control unit 130 may control the reproduction drive unit and the sound output unit so as to be output in the order of the original learning file, the secondary input audio information, and the dubbing file. The output order of the original learning file, the secondary input audio information, and the dubbing file is merely an example, and can be appropriately adjusted by the user's choice.

FIG. 8 is a schematic flow chart of a method for providing a language self-learning service (S3000) according to a second additional embodiment, FIG. 9 is a schematic flow chart for a script file generation step (S3200) Is a schematic flow chart for the script file output step (S3400) of Fig.

The method 300000 for providing a language self-learning service according to the present embodiment substantially includes the technical features included in the language self-learning service providing method S1000 (S2000) according to the embodiment described above, For the purpose of avoiding redundant explanations, the detailed description of the contents overlapping with the above-mentioned contents will be omitted below.

According to the present embodiment, the present invention provides a method for generating an original learning file (S3100); A script file creation step (S3200); A primary audio file playback step (S3300); A script file output step (S3400); A dubbing file generation step S3500; And a secondary audio file playback step (S3600).

Here, the original learning file generation step S3100, the primary audio file playback step S3300, the dubbing file generation step S3500, and the secondary audio file reconstruction step S3600 according to the present embodiment may be performed in accordance with the above- The steps S1100 and S2100 of the original sound file creation step S1100 and the steps S1200 and S2200 of the first sound file reproduction step S1300 and S2300 of the second sound file generation step S1400, In addition, the method for providing a language self-learning service (S3000) according to the present embodiment further includes a script file generation step (S3200), a script file generation step (S3200) between the original sound file creation step S3100 and the first sound file playback step A car audio file reproducing step (S3300) and a script file outputting step (S3400) executed sequentially or simultaneously.

Specifically, the control unit 130 activates the sound input unit, the recorder driving unit, and the voice-to-text information conversion unit 146 to start text from the original learning file (step S300) Information is extracted and generated and stored as a script file (S3200).

That is, the controller 130 extracts and / or converts the text information from the speech information of the original learning file received through the sound input unit or imported from the memory unit, and stores the extracted text information in the memory unit (S3210).

In a further embodiment of S3210, the text information is text information corresponding to speech information of the original learning file, and is downloaded from the self-learning service providing server according to a user's input signal or automatically, and stored Or may be moved from the user's external memory device to the memory part and stored by the user's input signal. At this time, the text information includes corresponding reproduction time information corresponding to the audio information of the original play file.

The control unit 130 extracts corresponding reproduction time information corresponding to the audio information of the original play file at the same time or sequentially with the voice-text information conversion step and stores the information in the memory unit (S3220).

In step S3230, the controller 130 combines the text information stored in the memory unit and the corresponding reproduction time information corresponding to the text information, and creates a script file.

Thereafter, the control unit 130 stores the script file in the memory unit (S3240).

The control unit 130 outputs the text information of the script file using the display unit 152 of the mobile terminal in step S3400 after or after the start of the primary audio file playback step S3300.

Then, the script file outputting step S3400 may first output the text information of the script file after the start of the primary audio file reproducing step S3300 (S3410) (Step S3420) based on the reproduction time information of the text information (step S3430), and visually change the text information.

At this time, the text information may be output so that the color or size of the text changes according to the reproduction time information of the primary input voice file.

Hereinafter, a mobile terminal according to a further embodiment of a mobile terminal according to an embodiment of the present invention will be described.

A mobile terminal according to a further embodiment of the present invention includes a touch-type input unit 121 for inputting input data for operation control according to a user's selection; A recorder driving unit 141 for recording a sound source input to the microphone 122; A reproduction drive unit 142 for reproducing the recorded sound source through an acoustic output unit 151 such as a speaker; A sound information extracting unit 149 for recognizing the sound source to be reproduced and recognizing the user's sound input to the microphone 122 after the sound source is reproduced; A matching unit (i.e., a function of the control unit 130) for matching the sound source with the voice of the user recognized by the voice information extracting unit to generate matching data; A display unit 152 for visually displaying the matching data; And a control unit 130 for controlling the reproduction of the sound source according to the reproduction option set by the user.

At this time, the reproduction option includes a reproduction speed of a sound source, a reproduction unit, and a reproduction repeat section, and the reproduction unit may include a word, a semantic segment, a sentence, and a paragraph.

In the present embodiment, the voice information extracting section 149 recognizes the voice of the user input through the microphone, analyzes the frequency of the voice signal in the voice using a voice analysis process, and finds a formant And is configured to analyze speech.

The reproduction driver 142 records a sound source, that is, a user's voice or learning contents, and outputs the recorded contents according to the user's selection.

The playback drive unit 142 may record the moving image in cooperation with the camera and output the recorded content.

The matching unit is a function included in the control unit, and matches the voice of the user with the learning contents. Specifically, since the matching unit grasps the degree of matching between the user's voice and the sound source, and shows the difference, the user can effectively practice pronunciation for sound similar to a sound source (native speaker's pronunciation).

The memory unit 160 stores a content database, a user information database, a voice analysis process, text to speech (TSS), speech to text (STT), and the like.

The analog voice input to the microphone 122 by the voice analysis process is converted into a digital signal. This digital signal consists of numbers representing the amplitude of the signal in close proximity to the 1 / 10,000 second unit. The voice analysis process analyzes the voice by analyzing the frequency of the voice signal and finding a formant.

A formant is a spectrum in which the sound of a person is analyzed by a frequency, and a graph or the like shows a certain degree of energy at a certain frequency. For example, if anyone pronounces 'ah', it sounds 'ah' regardless of the nature of the sound. This is because the spectrum of 'ah' has the same spectrum distribution regardless of the sound of any person.

TSS is also called speech synthesis. It is a technology that automatically generates speech sound waves. The speech of a person selected by the model is recorded and divided into certain speech units. The signals are input to the synthesizer with a sign, It is a technique to articulate speech only by reassembling units. On the other hand, STT is a technique for recognizing the voice of a person to be input and characterizing the voice.

The control unit 130 controls the reproduction speed, the reproduction unit, and the reproduction repetition period of the sound source reproduced according to the setting of the user. Here, playback units can be words, semantic segments, sentences, and paragraphs. For example, when the set reproduction unit is a sentence, the sentence is divided into sentences one by one, so that the user can read along with the outputted sentence.

Accordingly, since the pronunciation similarity can be visually confirmed through the formant contrast of the sound source and the user voice at the time of learning, the present invention can grasp the degree of similarity between the user's pronunciation and the sound source (pronunciation of the native speaker) There is an effect that can be done.

Hereinafter, a method for providing a language self-learning service according to a further embodiment using a mobile terminal according to a further embodiment will be described.

A method for providing a language self-learning service according to a further embodiment includes: a reproducing step in which one of the recorded or recorded learning contents is reproduced by a user's selection; A recognition step of receiving and recognizing a user's voice after the reproduction; A matching data generation step of generating matching data by matching the voice recognized in the recognition step and the voice content; And a providing step of providing the matching data to a user.

In the service providing method according to the above-mentioned additional embodiment, the reproducing step in which one of the learning contents is reproduced may correspond to the 'first audio file reproducing step (S1200)' in FIG. 2 or 3 described above, Hereinafter, the recognition step of receiving and recognizing the voice of the user may correspond to the 'secondary input voice information receiving step (S1320)' in FIG. 2 or FIG.

In addition, in the service providing method according to the additional embodiment described above, the matching data generating step of generating matching data by matching the voice recognized in the recognizing step and the voice contents, and providing the matching data to the user, Or may be performed automatically after the 'dubbing file creation step (S1300)' in FIG. 2 or FIG. 3 as described above.

Here, the reproducing step may include: setting a reproduction speed by user input; Setting a playback unit by user input; And setting a repetitive playback interval by user input.

Preferably, the recognition step may be performed by analyzing the user's voice with a frequency so as to confirm how much energy is present at a certain frequency.

In the providing step, the matching data may be provided as at least one of a graph and a matching percentage.

In addition, the method of providing a language self-learning service according to a further embodiment may be executed to return to the regeneration step if the matching data is below a certain criterion.

More specifically, the method for providing a language self-learning service according to a further embodiment of the present invention provides a formant of the learning content to be reproduced to the display unit 152 when the learning content is reproduced. In addition, the text of the learning contents can be displayed in a part of the display unit 152 using the STT according to the setting of the user.

When the user listens to the learning contents reproduced according to the reproduction unit, a formant of the user voice input by the microphone 122 is provided to the display unit 152. [

Preferably, the user voice input through the microphone is analyzed by a voice analysis processor, and the voice formants generated according to the analysis are provided by being filtered through an FFT (Faster Fourier Transform). This is because the quality of the signal processing can be improved by removing the noise through the filtering using the noise removing unit 148. The filtered formants can be visualized and stored in a graph or the like.

Therefore, the user can check his or her pronunciation while looking at the frequency band that changes according to the pronunciation in real time.

For example, Korean fricative / s / /, and English fricative / s / are different frequencies of sound produced when pronouncing, because the tongue has a different point of connection that causes contact in the mouth. When the FFT is performed, the Korean fricative / ㅅ / has no sound of low frequency (0 ~ 3000Hz) and the largest volume is distributed around 6000Hz in the middle frequency range, whereas the English fricatives are / You can visually confirm through the formant that the volume is low and the volume of the high frequency band above 8000Hz is greater.

Thereafter, the controller drives the matching unit to match the recognized speech with the learning content to be reproduced, and generates matching data.

In order to generate matching data, a formant of the corresponding learning content stored in the memory unit 160 is provided to the matching unit when the learning content is reproduced. This formant is called the first formant. Also, a formant of the user voice inputted to the microphone is provided to the matching unit. This formant is called the second forme.

The matching unit matches the first formant and the second formant, and provides the generated matching data to the display unit 152.

The matching data is an evaluation for learning including a matching rate, and is provided as a score by quantifying or quantifying it by a percentage, a score or the like so that the user can easily check it. The learner can use the score to figure out how much his pronunciation has improved each time. This scoring has the advantage of motivating learners to learn.

Thereafter, the controller may determine whether the matching data is equal to or greater than a predetermined criterion, and if the matching rate is equal to or less than a predetermined criterion (for example, 90%), If the matching data is equal to or greater than a predetermined reference value, the next reproduction section is reproduced.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, And the like. Accordingly, such modifications are deemed to be within the scope of the present invention, and the scope of the present invention should be determined by the following claims.

1000: Mobile terminal
110:
120: Input unit
130:
140: Language Self-Study Service Offering
150:
160:
170:
S1000, S2000, S3000: Language Self-Learning Service Delivery Method

Claims (19)

A method for providing a language self-learning service using a mobile terminal,
An original learning file generation step of generating or storing speech learning speech information;
A primary audio file playback step of outputting primary input audio information of the original learning file using the sound output unit of the mobile terminal;
A dubbing file generation step of receiving the secondary input voice information, which is voice information of the user, using the acoustic input unit of the mobile terminal and generating and storing the secondary input voice information as a dubbing file; And
Extracting and storing only the secondary input audio information in the superimposed audio information of the dubbing file at the user's selection and extracting the superimposed audio information;
Wherein the dubbing file generating step generates superimposed audio information by superimposing the secondary input audio information on the primary input audio information of the original learning file and generates a superimposed audio information on the basis of the volume of the primary input audio information and the volume of the secondary input audio information The size is relatively adjusted to generate a dubbing file,
Wherein the audio file reproduction output screen includes a reproduction step control part, a reproducible file list part, and a dubbing execution part, wherein the audio file reproduction output screen comprises:
Wherein the playback step control section includes an icon section for playback / pause control, an icon section for fast forwarding, an icon section for fast forward, an icon section for repeated playback selection, an icon section for specifying a repeat section, and an icon for volume control, Wherein the list portion is an output screen for a file list in a playable memory unit of the language self-learning service providing unit, the dubbing execution unit includes a dubbing guide information output unit and a dubbing execution icon,
When the user input signal is received through the dubbing execution icon, the control unit controls the display unit to switch the audio file reproduction output screen to a dubbing output screen,
Wherein the dubbing output screen includes the playback step control unit, the playable file list unit, the dubbing execution time output unit, the dubbing status information output unit, the icon unit for dubbing execution / pause control, and the dubbing completion icon unit,
Further comprising a secondary audio file playback step of outputting superimposed audio information of the dubbing file using the sound output unit of the mobile terminal, wherein the secondary audio file playback step is a step of playing back the secondary audio file, A second learning input audio information output step of outputting the second input audio information and a dubbing file audio information output step of outputting superimposed audio information which is audio information of the dubbing file, Wherein the mobile terminal is a mobile terminal. The method according to claim 1,
The original learning file generation step includes:
Wherein the voice input unit is used to record and store the primary input voice information as voice learning voice information to be learned by the user using the voice input unit of the mobile terminal, Delivery method. The method according to claim 1,
The original learning file generation step may be performed in accordance with a user's input signal or automatically
And extracting and storing language learning audio information from the audio information content and the moving picture content stored in the memory unit of the mobile terminal. The method according to claim 1,
The original learning file generation step includes:
Characterized in that the mobile terminal is executed to download language learning audio information or language learning contents to be learned by a user from a language self learning service providing server using the wireless communication unit of the mobile terminal, A method for providing self - learning service using language. The method according to claim 1,
The dubbing file generation step includes:
A user input signal receiving step of receiving, from a user, a user input signal for executing the dubbing file generating step after the first audio file reproducing step starts;
A secondary input voice information receiving step of driving the acoustic input unit of the mobile terminal based on the user input signal to receive the secondary input voice information;
A dubbing file generating step of superimposing the secondary input audio information on the primary input audio information of the original learning file to generate superposed audio information; And
And storing the overlapping audio information in a memory unit of the mobile terminal. The method according to claim 1,
Wherein the primary audio file reproducing step comprises:
And a user input signal receiving step of receiving from the user a user input signal for executing the primary audio file reproducing step,
Wherein the first audio file reproducing step and the dubbing file generating step are simultaneously performed when the user input signal is received. The method according to claim 1,
At least one of the original learning file generation step and the dubbing file generation step comprises:
And a noise removing step of removing ambient noise speech information from the input speech information. delete The method according to claim 1,
Wherein the secondary audio file reproducing step comprises:
A user input signal receiving step of receiving a user input signal for executing the secondary audio file reproducing step from a user;
And outputting audio information of a dubbing file to output superimposed audio information of the dubbing file based on the user input signal. The method according to claim 1,
After the secondary audio file playback step,
A user input signal receiving step of receiving a user input signal for executing a dubbing file reproducing step from a user;
A third input voice information receiving step of driving the sound input unit of the mobile terminal based on the user input signal to receive third input voice information which is re-input voice information of the user;
A dubbing file reproducing step of superimposing the third-order input audio information on the primary input audio information of the original learning file to generate additional superposed audio information; And
And storing the additional audio information in a memory unit of the mobile terminal. The method of claim 1, further comprising: delete The method according to claim 1,
The dubbing file generation step includes:
So as to generate a first dubbing file in which the volume magnitude of the primary input voice information is larger than the volume magnitude of the secondary input voice information,
Or a second dubbing file in which the volume level of the secondary input voice information is larger than the volume level of the primary input voice information,
Or both the first dubbing file and the second dubbing file,
Wherein the mobile terminal is a mobile terminal. 13. The method of claim 12,
The secondary audio file reproducing step may be performed in accordance with a user input signal or automatically,
To output the first dubbing file,
Or to output the second dubbing file,
Or sequentially outputting the first dubbing file and the second dubbing file,
Or sequentially outputting the second dubbing file and the first dubbing file,
Wherein the mobile terminal is a mobile terminal. delete delete The method according to claim 1,
A script file generation step that is executed after the start of the original learning file generation step, extracts text information from the original learning file, and generates and stores the extracted text information as a script file; And
And outputting a text file of the script file, the script file being executed after the start of the primary audio file playback step and using the display unit of the mobile terminal. Service delivery method. 17. The method of claim 16,
The script file output step includes:
And after the start of the primary audio file reproduction step, the text information is visually changed based on reproduction time information of the primary input voice file. A method for providing a language self-learning service using a mobile terminal according to any one of claims 1 to 7 or 9 to 10 or 12 to 13 or 16 to 17 The mobile terminal. A method for providing a language self-learning service using a mobile terminal according to any one of claims 1 to 7 or 9 to 10 or 12 to 13 or 16 to 17 Readable recording medium on which an application is recorded.

Images