KR100383061B1 - A learning method using a digital audio with caption data - Google Patents

Abstract

An object of the present invention is to combine the digital audio and caption data inserted into the digital audio to create each digital data file and to selectively output such digital data file, foreign language learning or music learning using speeches, news or movies, etc. To provide a learning method using digital audio and its caption data to systematically perform.

In order to achieve the above object, the present invention typically divides one song into a singer voice, accompaniment, and lyrics caption in the case of the flexible learning, and in the first mode, creates and records a digital data file under conditions including all of them, and the second In the mode, the digital data file of the karaoke mode is created and recorded only by the accompaniment and lyrics caption. In the third mode, the digital data file is created by recording the digital data file by the music accompaniment. When the digital files created above are reproduced by the digital signal processing apparatus, the digital data files of the corresponding mode can be selectively reproduced according to the selection of the reproduction mode by the user, so that the user can effectively learn any music. It features.

Description

A learning method using a digital audio with caption data}

An object of the present invention is to combine the digital audio and caption data inserted into the digital audio to create each digital data file and to selectively output such digital data file, foreign language learning or music learning using speeches, news or movies, etc. To provide a learning method using digital audio and its caption data to systematically perform.

The original digital audio data only contains information about the voice. However, it is possible to store caption information that can be defined by the user in such digital audio data, and also can reproduce them simultaneously through a voice output device such as an earphone and a display device such as an LCD.

Digital audio data is composed of a minimum unit whose bit string is called a frame (or AAU: Audio Access Unit). These frame units include the audio portion of all DVD (Digital Versatile Disk) standards and MPEG standards in addition to the MP3 format.

Since the software for inputting caption data on the digital audio data can indicate the position at which the caption is displayed by the frame number, the software can be applied to all digital audio data having a bit string of data configured in units of frames.

A method of loading caption data into digital audio, and extracts and compares the waveform of the key word with the first word or the first syllable of the caption unit as the key word, and finds the information of playback time and stores the information in the digital audio data for each caption unit. A simplified caption designation algorithm has been previously filed by the applicant of the present invention to enable the editing and retrieval of various caption information from digital audio data and to optimize the caption display amount according to the screen display capability of the liquid crystal display device.

In the method of loading captions on the digital audio data, first, when the input mode of caption data is selected on the digital audio data, the size of the caption display screen, whether the word is cut off during the caption and whether the display screen is divided or not, is determined. Create a caption unit in the order of word, line, paragraph, page, display the list, select the caption unit to be synchronized through the frame data loading process, and play the frame data to determine whether the frame number is automatically assigned. If the frame number is automatically assigned, the sub-routine automatically assigns the caption unit frame number, and if not, the synchronous frame number is assigned to the caption unit in accordance with the playback contents, and the frame number is modified among the caption units. If no correction is necessary, syllable units and word units It includes a process of specifying a frame number by each unit.

Therefore, when loading caption data on digital audio data, the process of extracting and comparing the waveform of the key word using the first word or the first syllable as the key word of the caption data as a key word enables the user to quickly find the playback point information. By including the data in the data, it is possible to edit and load various caption information in the digital audio data, to optimize the caption display amount according to the screen display capability of the liquid crystal display device, and to simplify the caption designation algorithm.

On the other hand, the caption data is recorded on the digital audio data through the improved caption designation algorithm as described above, and the music information including the language learning information or the music song is stored in the memory device, and then repeatedly played. Can be obtained. In particular, in the case of language learning, the focus is on the cultivation of the listening ability. At the same time, the corresponding subtitles are displayed on the display device, so that the hearing and the vision can be used at the same time, thereby significantly improving the learning effect.

However, until now, audiovisual learning using captioned digital audio players had to be trained only for the simultaneous output of uniform audio and caption set by the program developer or program data provider.

An object of the present invention is to combine the caption data and the digital audio data loaded on the digital audio in the form of digital audio only, caption only, and digital audio caption, respectively, and store them in a memory. If you select the saved data of, audio and caption can be played selectively or simultaneously according to the selection condition, and the learning effect can be enhanced by setting the learning pattern of the desired difficulty when learning and memorizing language using language, song and speech. The present invention provides a learning method using digital audio and its caption data.

1 is a block diagram of a digital audio player device as an example of a hardware device to which the learning method of the present invention is applied.

2 is a flowchart illustrating an input / output process of digital audio for flexible learning and caption data thereof according to the present invention.

3 is a flowchart illustrating an input / output process of digital audio for language learning and caption data thereof using a foreign language speech based on the present invention.

4 is a flowchart illustrating an input / output process of digital audio for language learning and caption data using a scenario and sound track of a foreign language drama based on the present invention.

5A through 5C are output state diagrams of caption screens for each type of learning using foreign language drama films.

※ Explanation of symbols about main part of drawing ※

31: modem 32: communication interface

33: internal learning data memory 34: codec

35: amplifier 36: audio output device

37: LCD driver 38: LCD

39: DSP / CPU 40: switch unit

41: external learning data memory 42: PC

43: data server 44: connector

45: Romans 46: Ram

47: D / A converter 48: filter

In order to achieve the above object, the present invention provides a method of dividing a song into a singer voice, a performance, and a lyrics caption. In the first learning pattern storage mode, a digital data file is created and recorded under conditions including all of them. In the pattern storage mode, the digital data file of the karaoke mode is created and recorded only by the music performance and lyrics caption. In the third learning pattern storage mode, the digital data file is created and recorded only by the music accompaniment. After creating a separate digital data file, and subsequently reproduce the digital files created in the digital signal processing apparatus includes the process of repeating the learning of any song by playing the digital data file of the mode in accordance with the selection of the playback mode by the user Characterized in that.

In addition, the present invention divides a foreign language speech or news into the speaker's voice and the caption of the speech, in the first learning pattern storage mode to create and record a digital data file with the conditions including the audio and caption data, the second learning pattern storage In the mode, a digital data file is created and recorded under the condition that the speaker's voice and caption data and translation text data corresponding to the caption data are displayed.In the third learning pattern storage mode, only the speaker voice is created and recorded as a digital data file. The foreign language listening and learning method can improve the foreign language listening ability by changing the difficulty of language learning by selectively reproducing and outputting the digital data file of the corresponding mode according to the selection of the playback mode by the user of the digital signal processing apparatus. It is done.

In addition, the present invention distinguishes foreign language drama films and corresponding scenario captions of the drama films when the subject of study is set to foreign currency, and in the first learning pattern storage mode, audio and speaker indications and dialogue captions of all the speakers are integrated into a digital data file. In the second learning pattern storage mode, the learner can practice pronunciation on behalf of the speaker of the corresponding role by deleting and recording the voice for each speaker of the cast under the actual caption condition under the full caption condition as in the first learning pattern. To create and save a digital data file in user audio dubbing mode, the third learning pattern storage mode is to create and save a digital data file for each speaker so that the dialogue caption and audio output of the speaker are processed as blank and mute respectively. After that, any of the first to third learning patterns in the learning mode by the user It is characterized by a method of learning conversation using foreign currency, which enables to concentrate on learning and memorizing the dialogue of a particular speaker (actress) by selecting me and selecting the speaker. .

Hereinafter, the present invention will be described with reference to the accompanying drawings. The learning methods of the present invention can be equally applied to portable caption-type digital audio players and all computers. In addition, any form of signal processing device having a digital sound processor (DSP), an audio output device, and a display device may contain all the voices and corresponding caption data, such as foreign language learning, pop songs or speeches, according to the method of the present invention. The method of the present invention can be applied to learning.

1 is a schematic block diagram of a digital audio player system in which the method of the present invention will be exemplarily implemented.

As referred to herein, the digital audio player 50 includes a modem 31 for receiving caption digital learning data transmitted through a PSTN / ISDN network from a caption learning network server 43 of a wired switch center, and a PC 42. External learning data memory connected in parallel to a communication interface 32 for receiving data transmitted through the data bus from the data bus as a readable data from the internal device, and an internal learning data memory 33 storing learning voice and caption data. It is connected to 41 via a connector 44.

The modem 31, the communication interface 32, and the internal learning data memory 33 are connected to a DSP / CPU 39 having an I / O port, a ROM 45, and a RAM 46.

The DSP / CPU 39 has a switch unit 40 having keys such as PLAY, REW, FF, and STOP, and an LCD 38 for converting caption data into characters and displaying them, and in the DSP / CPU 39 The processed digital audio data signal is input to the amplifier 35 through the codec 34, the converter 47, and the filter 48, and configured to be output through the audio output device 36.

When downloading caption language learning data from the outside to such a digital audio player, if the data source is a modem communication mode connected to a database server 43 of a wired switch office, the CPU drives the modem 31 to perform DTMF dialing. You will be connected to the wired switch office server.

The digital data required by the digital audio player can be received from the PC 42 via the interface device 32 via the data bus.

The DSP / CPU unit 39 processes digital audio and caption data input from the modem 31 or the communication interface 32 and stores the digital audio and caption data in the internal learning data memory 33.

The switch unit 40 connected to the DSP / CPU unit 39 controls and selects various functions of the digital audio player. For example, when the play switch inside the switch unit 40 is turned on, the CPU 39 of the player system is turned on. ) Becomes the learning reproduction mode, and the selected caption digital file is retrieved from the internal learning data memory 33 and processed.

The digital audio data processed by the DSP / CPU unit 39 is converted into an analog signal through a codec 34, a converter 47, and a filter 48, amplified by an amplifier 35, and then a voice output device 36. Through the LCD driver 37, the caption data processed by the DSP / CPU 39 is output as a voice to the outside and displayed on the LCD 38 as text.

By using the audio-visual effect through the simultaneous voice-character output will be able to improve the learning effect, such as language.

2 is a flowchart illustrating an input / output process of digital audio for flexible learning and its caption data in the learning method of the present invention.

In this case, if there is a music selection, it is judged whether the digital data file creation mode for the flexible learning is performed, and if the digital file creation mode, that is, the data input mode for the learning, in the first learning pattern storage mode, one song is selected as a singer voice, performance and lyrics caption. Create and save digital data files under karaoke conditions that include them all. The storage is stored in the internal learning data memory 33.

Next, in the second learning pattern storage mode, the digital data file is created and stored only with the music accompaniment and the lyrics caption, and it is determined whether the song is composed of the voice of the duet singer. In this case, in the third learning pattern storage mode, the digital data file is created and stored in the internal learning data memory 33 only by the accompaniment in the third learning pattern storage mode.

However, if it is determined that the duet singer is judged after the execution of the second learning pattern storage mode, if the duet singer is flexible, the fourth learning pattern storage mode creates a digital data file using only the singer's voice and lyrics caption, and the b singer in the fifth learning pattern storage mode. Build a flexible learning data file by creating a digital data file with only the voice and lyrics captions.

The division and storage of the data related to the learning song can be recorded and stored as many times as desired through repetition of the whole process.

Then, as the user selects a desired playback mode, for example, various types of flexible panel cycle modes, the digital data file of the corresponding mode can be reproduced to interestingly perform lyrics learning and practice for a specific song.

FIG. 3 is a flowchart illustrating the input / output process of digital audio and caption data required for language listening learning using a speech or a foreign language news broadcast by a foreign speaker in the learning method of the present invention.

In this case, when the speech or news for language learning is selected, the language listening method using the speech or news is determined to determine whether the digital data file creation mode for learning is used. In a one-person briefing dialogue such as a speech or news, caption data of the speech or news is included in the voice data, and a digital data file is created and stored in the internal learning data memory 33.

Next, if the audio caption data of the speech or news is judged to be translated, and in the double caption mode according to the simultaneous translation condition, in the second learning pattern storage mode, one LCD screen is divided into two at the same time as the speaker's voice output, and the speech voice is partially above. The digital data file is created and stored in the internal learning data memory 33 to display the caption text for the display and the translation of the caption text on the remaining portion of the LCD.

In the third learning pattern storage mode, only the voice of the speaker is created and stored in the internal learning data memory 33 in the third learning pattern storage mode, unless a translation is required in the process of determining whether to translate the caption data such as speech.

In this state, digital audio and caption data of foreign language speeches are separately stored, and the user can select a playback mode according to his or her taste and level to perform listening learning using foreign language contents or foreign language news. do.

4 is a flowchart illustrating an input / output process of digital audio and caption data required for language listening learning using a foreign language drama film in the learning method of the present invention.

Here, the scenario of a foreign language drama to be studied and the language listening using a real-time sound track voice are judged as to whether or not to create a digital data file creation mode for learning. The internal learning data memory (33) is composed of all the voices of the speakers, the name of each speaker according to the voice, and the dialogue captions of the speakers in one digital data file. Store in In this case, the metabolism caption is displayed on the LCD as shown in FIG. 5A.

Next, in the second learning pattern storage mode, the speaker name caption and the dialogue caption for each speaker are output under the full sound condition, and the voice metabolism is muted and stored for the speaker's voice output in real time. The second learning pattern storage mode is recorded for every speaker.

This mode is a user audio dubbing mode so that learners can practice pronunciation on behalf of the speaker of the cast, so that they can check their pronunciation status to learn a language using foreign currency. To this end, through the microphone installed in the digital audio signal processing apparatus, the voices pronounced by the learner himself in the following format are fed back at a certain time so that the learner can listen to his / her voice.

In the third learning pattern storage mode, only the speaker name of the speaker is output at each dialogue point for each speaker, and sound track audio and dialogue captions are set to mute and blank sections, respectively. To create a digital data file and record it.

This allows learners to develop their ability to speak foreign languages through the form of participating in one of the films.

By giving a unique code for each speaker instead of the speaker's name and matching the speaker's name to the corresponding unique number, the speaker's metabolic caption can be included in the memory.

In this way, when the learning database using the dialogue and audio of the foreign language drama film is established, the caption data of the corresponding speaker is selectively selected when the speaker of the role to be learned is selected to select and learn the corresponding learning mode, that is, the foreign currency learning mode. Can be sorted and printed or deleted.

In this case, as shown in FIG. 5B, when the user himself / herself decides to cast a participant and participates in the play, the speaker mode selected by the user may replace his voice with a specific cast (speaker) in a foreign language drama scenario. Since it is fed back to one's ear through a microphone mounted on the audio signal processing device, it becomes possible to learn conversation by recognizing one's pronunciation and correcting wrong pronunciation or utterance.

In addition, as shown in FIG. 5C, the speaker name is always output even when executing a specific caption deletion mode of a particular speaker so that the learner can participate as if he or she is one of the scenarios of the drama. It is possible to increase the learning effect of conversation.

By using this function, it is possible to obtain a very high learning effect in the native listening practice and conversational learning through the repeated learning in the form of participation using foreign currency. In addition, according to the learner's choice can be deleted in the same way the specific caption and speech of the specific speaker can be effectively applied to the high-level language learning and metabolic memorization.

In the above description, the learning data of various patterns by genre are recorded and selectively called out in the digital data storage means installed in the digital audio player, for example, but the learning data processed in various forms is downloaded from the server or the PC. After receiving and storing, it is also possible to call up the desired recording mode and execute the learning according to the method of the present invention.

As described above, when the caption data is loaded on the digital audio data for learning, the caption data and the digital audio are separately produced in various forms to create and store the learning data in various patterns for one learning object. Afterwards, the user of the digital audio player with the caption function selects and plays the learning data of the desired step or pattern among the above learning patterns to improve the effect of listening and memorizing learning of language, song, and foreign language speech. It can bring a distinctive effect.

In addition, the learning data processed to enable the learning using the digital audio and caption data thereof according to the present invention may be stored in various types of recording media, for example, magnetic recording media such as CD ROM, DVD and hard disk, and flash memory. It is possible to store on a recording medium such as a device and supply it to the user.

In addition, the processed learning digital data that can be stored in various types of recording media as described above can be supplied to the user through the Internet or a communication network.

Claims (8)

A DSP / CPU having a program for controlling a digital audio player in accordance with a command input through a switch section, a learning data memory for storing digital data readable by the DSP / CPU, and a digital output from the DSP / CPU. D / A converter and filter for converting and converting data into analog signal, Codec for converting and outputting digital data of specific format output from DSP / CPU into analog signal, Analog signal output from the filter or codec In the learning method using a digital audio player comprising an amplifier for amplifying and outputting through a voice output device and an LCD driver for outputting an operation function or caption data through the LCD in response to an output signal from the DSP / CPU, Set a digital data file including all of the singer voice, accompaniment, and lyrics caption for one song in a first learning pattern storage mode, and set the digital data file including the accompaniment and lyrics caption of the song in a second learning pattern storage mode. The digital data file including only the accompaniment of the music to a third learning pattern storage mode, and the DSP according to a reproduction mode selection for the first, second or third learning pattern storage mode input through a switch unit. Learning method using digital audio and its caption data by reproducing the digital data file of the learning pattern storage mode selected in the CPU / CPU and outputting audio data to the audio output device and caption data through the LCD . A DSP / CPU having a program for controlling a digital audio player in accordance with a command input through a switch section, a learning data memory for storing digital data readable by the DSP / CPU, and a digital output from the DSP / CPU. D / A converter and filter for converting and converting data into analog signal, Codec for converting and outputting digital data of specific format output from DSP / CPU into analog signal, Analog signal output from the filter or codec In the learning method using a digital audio player comprising an amplifier for amplifying and outputting through a voice output device and an LCD driver for outputting an operation function or caption data through the LCD in response to an output signal from the DSP / CPU, The digital data file including both the voice and the caption thereof is set to the first learning pattern storage mode, the digital data file including only the voice is set to the second learning pattern storage mode, and the first or The digital data file of the learning pattern storage mode selected by the DSP / CPU is reproduced according to the reproduction mode selection for the second learning pattern storage mode. The audio data is output to the audio output device, and the caption data is output through the LCD. Learning method using digital audio and its caption data. The method of claim 2, The learning method uses the digital audio and its caption data, characterized in that the digital data file including only the caption is set to the third learning pattern storage mode and reproduced when the third learning pattern storage mode is selected by the switch unit. A DSP / CPU having a program for controlling a digital audio player in accordance with a command input through a switch section, a learning data memory for storing digital data readable by the DSP / CPU, and a digital output from the DSP / CPU. D / A converter and filter for converting and converting data into analog signal, Codec for converting and outputting digital data of specific format output from DSP / CPU into analog signal, Analog signal output from the filter or codec In the learning method using a digital audio player comprising an amplifier for amplifying and outputting through a voice output device and an LCD driver for outputting an operation function or caption data through the LCD in response to an output signal from the DSP / CPU, Set the digital data file including the speaker voice and the caption for the speech to the foreign language speech or news in the first learning pattern storage mode, and store the second learning pattern of the digital data file containing only the speaker speech of the foreign language speech or news. Mode, and reproduces the digital data file of the learning pattern storage mode selected by the DSP / CPU according to the reproduction mode selection for the first or second learning pattern storage mode inputted through the switch unit, so that the audio data is output to the audio output device. The raw caption data is output through the LCD to learn a foreign language. The method of using digital audio and caption data thereof. The method of claim 4, wherein The learning method sets a digital data file including only caption data of the speaker speech of the foreign language speech or news into a third learning pattern storage mode, and reproduces it when the third learning pattern storage mode is selected through a switch unit; Learning method using digital audio and its caption data. A DSP / CPU having a program for controlling a digital audio player in accordance with a command input through a switch section, a learning data memory for storing digital data readable by the DSP / CPU, and a digital output from the DSP / CPU. D / A converter and filter for converting and converting data into analog signal, Codec for converting and outputting digital data of specific format output from DSP / CPU into analog signal, Analog signal output from the filter or codec In the learning method using a digital audio player comprising an amplifier for amplifying and outputting through a voice output device and an LCD driver for outputting an operation function or caption data through the LCD in response to an output signal from the DSP / CPU, A digital data file containing all speaker names, the speaker's voice, and the dialogue caption corresponding to the speaker's voice in the movie is set to the first learning pattern storage mode, and the voice for each speaker of the cast is performed under the full caption conditions of the dialogue. Deletes and saves during the actual voice progression so that the learner can practice pronunciation on behalf of the speaker of the corresponding role, and sets the digital data file including the user audio dubbing mode as the second learning pattern storage mode, and the dialogue of the speaker. A digital data file is created for each speaker so that captions and voice outputs are processed as blanks and mutes, respectively, and set to a third learning pattern storage mode, and the first, second or third learning pattern storage mode or a speaker input through a switch unit. Replays the selected learn pattern storage mode or digital data file of the speaker according to the playback mode selection for the Audio data by the learning method using the digital audio and his caption data, characterized in that the caption data to the voice output device to output through the LCD learning a foreign language. The method of claim 6, wherein when the caption data of the speaker name and the dialogue is input, a unique code for each speaker is assigned instead of the speaker name and the speaker name is matched to the corresponding unique number. A method of learning using digital audio and its caption data, characterized by selectively deleting dialogue and audio output. A recording medium containing learning digital audio data and caption data prepared according to any one of claims 1, 2, 4 or 6.