KR20110066622A - Apparatus and method of interpreting an international conference based speech recognition - Google Patents

Abstract

PURPOSE: An international interpretation device and method thereof based on voice recognition are provided to supply text data or a synthetic voice which is interpreted in a native language to attendees. CONSTITUTION: A conference participant information registering unit(100) registers conference participant information including the language used by a conference participant. A voice recognition unit(200) registers a keyword according to the conference participant contents of presentations in advance. The voice recognition circuit outputs a voice recognition result of a keyword form. A language interpreting unit(300) performs conversion to a target language corresponding to a using language per conference participants.

Description

Apparatus and method of interpreting an international conference based speech recognition}

The present invention relates to an apparatus and method for interpreting speech based international conferences, and more particularly, to an apparatus and method for supporting smooth communication among participants of international conferences using different languages.

The present invention is derived from the research conducted as part of the IT growth engine technology development project of the Ministry of Knowledge Economy and the Ministry of Information and Communication Research and Development. [Task management number: 2006-S-036-04, Task name: Large-capacity interactive distribution for new growth engine industries] Development of processing voice interface technology].

In general international conferences, it is common to conduct a conference using an interpreter who can simultaneously interpret the language used by the presenter, or to hold a conference using English, which is a global language. In this case, if you need a simultaneous interpreter who can translate the language, or if you use English as the official language, your understanding will vary greatly depending on the English level of the meeting attendees.

SUMMARY OF THE INVENTION The present invention has been proposed in view of the above-mentioned conventional circumstances, and provides a speech recognition-based international conference interpretation apparatus and method for interpreting in various languages so that various meeting participants with different English understandings can understand key keywords mentioned in the meeting. The purpose is to provide.

In order to achieve the above object, a voice recognition-based international conference interpretation device according to a preferred embodiment of the present invention includes a conference participant who pre-registers conference participant information including a language of a conference participant participating in a multi-party conference. An information register; A speech recognition unit that registers a key word according to the presentation content of each meeting participant in advance and recognizes a voice accompanying the conference participant based on a pre-registered key word and outputs a result of speech recognition in the form of a key word; And a language translator configured to analyze the voice recognition result in the form of a key word and convert the result into a target language corresponding to a language used for each pre-registered conference participant.

The speech recognition unit may include a key word extracting unit extracting a key word from the presentation content for each meeting participant and storing the key word in a key word database; A voice receiver for receiving a voice accompanying the presentation of the conference participant; A preprocessor extracting a feature vector for the received voice; And a decoding unit for decoding the extracted feature vector and outputting a speech recognition result in the form of a keyword based on a keyword stored in a keyword database.

The speech recognition unit registers key words in advance by conference participation before the multi-party conference.

Key words registered in advance in the speech recognition unit include subjects, nouns, and verbs.

The speech recognition unit outputs a textual result based on a key word.

The language translator may include: a language analyzer configured to receive a voice recognition result from the voice recognizer and analyze a type of input language; A user language information analyzer for analyzing a language used by conference participants; And a converting unit converting the received speech recognition result into a target language for each conference participant by performing a mapping corresponding to an input language to an output language based on the band dictionary for each conference participant.

The language translation unit outputs the output in the form of synthesized sound or text.

Voice recognition-based international conference interpreter translates multi-party conferences between remote conference participants through a communication network

In the voice recognition-based international conference interpretation method according to a preferred embodiment of the present invention, the conference participant information registration unit registers conference participant information including the language of the conference participant participating in the multi-party conference in advance for each conference participant. step; A voice recognition step of registering a key word according to the presentation content of each meeting participant in advance, and recognizing a voice accompanying the presentation of the meeting participant based on the pre-registered key word and outputting a speech recognition result in the form of a key word; And a language translation step of analyzing, by the language translation unit, the voice recognition result in the form of a core word, converting the result into a target language corresponding to a language used for each conference participant registered in advance.

The speech recognition step may include: extracting a key word from a presentation content of each conference participant and storing the key word in a key word database; A voice reception step of receiving a voice accompanying the presentation of the conference participant; A preprocessing step of extracting a feature vector for the received speech; And decoding the extracted feature vector, and outputting a speech recognition result in the form of a keyword based on the keyword stored in the keyword database.

The speech recognition step registers key words in advance for each meeting participation before the multi-party meeting.

Key words registered in advance in the speech recognition phase include subjects, nouns, and verbs.

The speech recognition result by the speech recognition step is a textual result based on the key word.

The language translation step may include: a language analysis step of receiving a voice recognition result by the voice recognition step and analyzing a type of input language; Analyzing user language information for analyzing a language used by conference participants; And converting the received speech recognition result into a target language for each conference participant by performing mapping corresponding to an input language to an output language based on the band dictionary for each conference participant.

In the language translation step, an output in the form of synthesized sound or text is output.

According to the present invention, the presenter can freely speak his / her native language and provide attendees with text data or synthesized voices translated into their native languages, thereby facilitating the progress of the international conference and understanding of the participants. have.

Hereinafter, an apparatus and method for interpreting voice recognition based international conference according to an embodiment of the present invention will be described with reference to the accompanying drawings. Prior to the detailed description of the present invention, the terms or words used in the specification and claims described below should not be construed as being limited to the ordinary or dictionary meanings. Therefore, the embodiments described in the specification and the drawings shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent all of the technical idea of the present invention, various modifications that can be replaced at the time of the present application It should be understood that there may be equivalents and variations.

1 is a view for explaining a system to which the speech recognition based international conference interpretation apparatus according to an embodiment of the present invention is applied.

The device 15 of the embodiment of the present invention is connected to a communication network (eg, the Internet 14). A conference participant at a remote location connected to the Internet 14 may access a device 15 according to an embodiment of the present invention through his terminal 11, 12, or 13 and receive a voice recognition based international conference interpretation service. .

The terminals 11, 12, and 13 may be connected to the device 15 of the embodiment of the present invention via the Internet 14, and may be a terminal capable of inputting and outputting voice and outputting a screen of text.

The apparatus 15 of the embodiment of the present invention interprets the international conference based on voice recognition during the multilateral international conference. A more detailed description of the device 15 will be given later.

In the case of Fig. 1, the language to be used is registered to the meeting participants before the meeting starts.

Then, if a participant of any group of conference attendees speaks "Hello I am a strong point" using Korean via a terminal (e.g. 11), this voice data is transmitted via the Internet 14 to the device of the embodiment of the present invention. 15) (16, 17).

The device 15 of the embodiment of the present invention receives the "Hi. I'm a strong person. ”We perform voice recognition on the voice data.

Apparatus 15 of the embodiment of the present invention is recognized as a "hi low strength" as a result of speech recognition. The device 15 translates the recognized key word based recognition result into a target language and delivers the result (18, 19, 20, 21).

For example, if a language used in advance by a conference participant of the terminals 11 and 13 is English, the conference participant receives a service such as “Hello, I'm JeomJakang” in English (20, 21). If the conference attendee of the terminal 12 registers a language used in advance in Japanese, the conference participant receives a service such as “Konnichiha. Hakankanjijotojimasu” in Japanese (19).

At this time, the interpretation result displayed on the screen uses a keyword (key word) output method that outputs only the core word, not the sentence type spoken by the speaker. The reason for using the keyword output method is based on the keyword recognition and output which is important for the proceeding of the conference because the recognition success rate decreases in the case of dialogue continuous word recognition. Currently, the word recognition rate of AT & T Large Vocabulary Continues Speech Recognition (AT & T LVCSR) system, which is developed based on conversational phone speech, is 71.6%, indicating a poor recognition success rate.

The device 15 of the embodiment of the present invention described above can be usefully used even offline.

Figure 2 is a block diagram showing the configuration of the speech recognition based international conference interpretation apparatus according to an embodiment of the present invention.

Voice recognition-based international conference interpretation device according to an embodiment of the present invention, the meeting participant information registration unit 100, call control unit 102, voice recognition unit 200, and language translation unit 300.

The conference participant information registration unit 100 registers conference participant information including a language of the conference participant participating in the multi-party conference in advance for each conference participant. Here, the meeting participant information includes a language used, details of the meeting participant, ID and password, and the like.

The call control unit 102 performs call control necessary for the multiparty conference.

The voice recognition unit 200 registers key words (eg, including words, nouns, and verbs) according to the presentation content of each meeting participant in advance, and recognizes a voice accompanying the presentation of the meeting participant based on the pre-registered key words. Outputs speech recognition results in the form of key words.

The speech recognizer 200 includes a speech receiver 202, a preprocessor 204, a decoder 206, a keyword extractor 208, a keyword database 210, and an acoustic model database 212. The voice receiver 202 receives and buffers the voice accompanying the presentation of the conference participant. The preprocessor 204 removes noise mixed in the voice data from the voice receiver 202, and extracts a feature vector by extracting a start point and an end point corresponding to the voice interval from the data from which the noise is removed. The key word extraction unit 208 extracts a key word from the presentation content for each meeting participant input before the start of the multi-party conference and stores the key word in the key word database 210. The key word extraction unit 208 extracts a key word from the presentation content input from the conference participant information registration unit 100 or the call control unit 102. Of course, the key word may be extracted from the presentation content input through a separate microphone instead of the conference participant information registration unit 100 or the call control unit 102. In the following description of the present invention, it is assumed that the call control unit 102 has a microphone (not shown) for convenience, and the announcement content is input to the voice recognition unit 200 through the call control unit 102 and the key word extraction unit 208 is provided. Let's extract the keywords from. The acoustic model database 212 stores standard speech patterns for each voice generated based on hidden Markov. The decoding unit 206 decodes the feature vector extracted from the preprocessor 204, and generates a speech recognition result in a keyword form based on a keyword stored in the keyword database 210 and a standard pattern acoustic model of the acoustic model database 212. Outputs Preferably, the decoding unit 206 uses a feature pattern extracted from the preprocessing unit 204 and a standard pattern acoustic model generated based on a hidden Markov, a class-based finite state network (FSN), and a beater using a dictionary. Perform a decoding function that performs non-searching. As a result of the decoding, the decoding unit 206 outputs a speech recognition result in the form of a keyword (keyword).

The language translator 300 analyzes the voice recognition result in the form of a key word and converts the result into a target language corresponding to a language used for each conference participant registered in advance.

The language translator 300 includes a language analyzer 302, a user language information analyzer 304, a converter 308, an output 310, and a user information database 312.

The language analyzer 302 receives a speech recognition result from the speech recognizer 200 and analyzes what kind of language is input (eg, Korean, English, Japanese, Chinese, French, etc.).

The user language information analysis unit 304 analyzes what language is used by the conference participants based on the information stored in the user information database 312. In the user information database 312, user information (that is, conference participant information including a language, a conference participant ID, a password, etc.) registered in the conference participant information registration unit 100 is registered. Analytical operation is possible.

The conversion unit 308 converts the received speech recognition result into a target language for each conference participant by performing a mapping corresponding to an input language to an output language based on a built-in band dictionary database 306 for each conference participant.

The output unit 310 outputs the converted output in the target language for each conference participant by the synthesizer when the format of the output device does not have a screen for outputting the synthesized sound, and the output unit 310 has a screen for outputting. Output in text form.

3 is a flowchart illustrating a voice recognition based international conference interpretation method according to an embodiment of the present invention.

First, the participants participating in the multi-party conference before the voice recognition-based international conference interpretation service is first registered with the participants of the conference, including the language used, and the key words in the presentation during the conference (S10). That is, the conference participant information input through the conference participant information registration unit 100 is stored in the user information database 312 of the language translator 300. The presentation content during the conference, which is input in advance through the microphone (not shown) of the call controller 102, is input to the keyword extracting unit 208 of the speech recognition unit 200 and the keyword extracting process in the keyword extracting unit 208. Through the key words in the presentation is stored in the key word database (210). Here, the information stored in the user information database 312 and the keyword database 210 is classified and stored by country and conference participants or stored in other forms. In the present invention, a key word is extracted and stored in advance for key word-based speech recognition. This is to increase recognition success rate. That is, if the registration is made in advance, the recognition speed is fast and the recognition success rate is increased. For example, nouns and verbs are defined as key words. The speech recognition unit 200 increases the system speed and recognition success rate by constructing a grammar network by subdividing into classes according to regions.

After the registration process is completed, the call control unit 102 starts an international conference interpretation service by giving a call start command to all conference participants (S12).

If there is no call termination (“No” in S14), the presenter naturally starts to speak the words to be announced using the mother tongue (S16).

Accordingly, the voice receiver 202 of the voice recognition unit 200 buffers the input voice data, and the preprocessor 204 performs preprocessing on the buffered voice data. That is, the preprocessor 204 removes the noise mixed in the speech data, extracts the start point and the end point corresponding to the speech section from the data from which the noise is removed, and extracts the feature vector. Then, the decoding unit 206 is a standard pattern acoustic model, class-based finite state network (FSN), dictionary (i.e., keyword database) generated based on the feature vector extracted from the preprocessor 204 and the hidden Markov. 210) to perform a decoding function for performing a Viterbi search. The decoding unit 206 outputs a speech recognition result in the form of a keyword (keyword) as a result of decoding (S18).

When the speech recognition result in the form of a keyword is input to the language translator 400, the language analyzer 302 of the language translator 400 analyzes the speech recognition text to input the type of language (eg, Korean, English, Japanese, Chinese, etc.) to determine what is (S20).

In addition, the user language information analyzer 304 determines what language the conference attendees use based on the information stored in the user information database 312 (S22). This is because the languages used by the conference attendees are different so that the corresponding languages can be output in correspondence.

Thereafter, the conversion unit 308 converts the target language for each conference participant by performing the band dictionary mapping corresponding to the input language to the output language using the built-in band dictionary database 306 (S24 and S26).

When the conversion to the target language for each meeting participant is completed as described above, the output unit 310 outputs the converted result as a synthesized sound when the output device does not have a screen for output, and when the screen has the screen for output. The converted result is output in text form (S28).

This voice recognition based international conference interpretation service is terminated when there is a call termination in the call control unit 102 ("Yse" in S14).

On the other hand, the present invention is not limited only to the above-described embodiments and can be carried out by modifications and variations within the scope not departing from the gist of the present invention, the technical idea that such modifications and variations are also within the scope of the claims Must see

1 is a view for explaining a system to which the speech recognition based international conference interpretation apparatus according to an embodiment of the present invention is applied.

Figure 2 is a block diagram showing the configuration of the speech recognition based international conference interpretation apparatus according to an embodiment of the present invention.

3 is a flowchart illustrating a voice recognition based international conference interpretation method according to an embodiment of the present invention.

Claims (15)

A conference participant information register configured to pre-register conference participant information including a language of the conference participant participating in the multi-party conference for each conference participant; A voice recognition unit that registers a key word according to the presentation content of each conference participant in advance, and recognizes a voice accompanying the presentation of the conference participant based on the pre-registered key word and outputs a voice recognition result in the form of a key word; And And a language translator configured to analyze the result of speech recognition in the form of a key word and convert the result into a target language corresponding to a language used for each conference participant registered in advance. The method according to claim 1, The speech recognition unit, A key word extracting unit extracting a key word from the presentation content for each meeting participant and storing the key word in a key word database; A voice receiver configured to receive a voice accompanying the presentation of the conference participant; A preprocessor extracting a feature vector for the received voice; And And a decoding unit which decodes the extracted feature vector and outputs a result of speech recognition in the form of a key word based on the key word stored in the key word database. The method according to claim 1, The speech recognition unit, the speech recognition based international conference interpreter, characterized in that for registering the key word in advance before the multi-party conference. The method according to claim 1, The key word pre-registered in the speech recognition unit includes a subject, a noun, a verb. The method according to claim 1, The speech recognition unit, a speech recognition-based international conference interpreter, characterized in that for outputting the result of the text form based on the key word. The method according to claim 1, The language translation unit, A language analyzer configured to receive a speech recognition result from the speech recognizer and analyze a type of input language; A user language information analyzer for analyzing a language used by the conference participants; And A speech recognition-based international conference, comprising: a conversion unit converting the received speech recognition result into a target language for each conference participant by performing mapping corresponding to an input language to an output language based on a band dictionary for each conference participant; Interpreter desk. The method according to claim 1, And the language translator outputs an output in the form of synthesized sound or text. The method according to claim 1, The voice recognition-based international conference interpretation device is a voice recognition-based international conference interpretation device, characterized in that for translating the multi-party conference between the meeting participants of a remote location through a communication network. A meeting participant information registration step of registering, by the conference participant information registration unit, conference participant information including a language of the conference participant participating in the multi-party conference in advance for each conference participant; The speech recognition unit pre-registers a key word according to the presentation content for each conference participant, and recognizes a voice accompanying the conference participant based on the pre-registered key word and outputs a speech recognition result in the form of a key word. step; And And a language translation step of translating the speech recognition result of the core form into a target language corresponding to a language used for each conference participant registered in advance and outputting the language. Way. The method according to claim 9, The speech recognition step, A key word extraction step of extracting a key word from the presentation content for each meeting participant and storing the key word in a key word database; A voice receiving step of receiving a voice accompanying the presentation of the conference participant; A preprocessing step of extracting a feature vector for the received speech; And Decoding the extracted feature vector, and outputting a speech recognition result in the form of a key word based on a key word stored in the key word database. The method according to claim 9, The voice recognition step is a voice recognition-based international conference interpretation method, characterized in that for registering the key word in advance for each meeting participation before the multi-party meeting. The method according to claim 9, The key word pre-registered in the speech recognition step includes a subject, a noun and a verb. The method according to claim 9, The speech recognition result of the speech recognition step is a speech recognition based international conference interpretation method, characterized in that the result of the text form based on the key word. The method according to claim 9, The language translation step, A language analysis step of receiving a speech recognition result by the speech recognition step and analyzing a type of input language; Analyzing user language information for analyzing a language used by the conference participants; And And converting the received speech recognition result into a target language for each conference participant by performing a mapping corresponding to an input language to an output language based on a band dictionary for each conference participant. How to interpret. The method according to claim 9, In the language translation step, speech recognition-based international conference interpretation method characterized in that for outputting the output in the form of synthesized sound or text.

Images