KR20190029237A - Apparatus for interpreting and method thereof - Google Patents

Abstract

According to the present invention, disclosed is an interpretation apparatus performing two-way simultaneous interpretation as a portable server including a translation module performing two-way translation and included in the interpretation apparatus and a storage module storing a voice recognition DB and a translation DB and functioning as an input apparatus and/or an output apparatus for at least one speaker in a relationship of parties of simultaneous interpretation. According to the present invention, two-way simultaneous interpretation is possible in an environment in which the interpretation apparatus is not connected to the Internet by using a portable server.

Description

[0001] APPARATUS FOR INTERPRETING AND METHOD THEREOF [0002]

The present invention relates to an interpretation apparatus and a method thereof, and more particularly, to an interpretation apparatus and a method thereof capable of offline interpretation without connection to the Internet.

With the development of transportation and communication means, human and material exchanges between countries have become active. Despite the expansion of human and material exchanges, the heterogeneous language between countries is acting as a barrier in communication.

To reduce the inconvenience caused by heterogeneous languages, the conversion between characters in a heterogeneous language is called translation. And the spoken language translation system refers to the conversion between the voices of different languages, which is the interpretation of broadcast news.

Especially, the inconvenience between the speakers of different languages at various conferences between countries is solved through simultaneous interpretation. In particular, the automatic interpretation is to convert the voice of the first language into the voice of the second language in both directions.

Simultaneous interpreting, which was the exclusive use of past simultaneous interpreters, has been made automatic interpreting by machine due to the development of technology of speech recognition, automatic translation and voice synthesis.

The automatic interpretation means a process and a technique of converting the utterance of the first language into a second language through processes such as speech recognition and automatic translation, outputting it as a subtitle, or reproducing the voice through a speaker after synthesizing the voice.

1 is an exemplary view of a translation system 10 according to the prior art using a translation server.

Referring to FIG. 1, according to the related art, in the case of the interpretation device 11 of the user terminal type including the simultaneous interpretation system 10, a wired or wireless communication network 14 for speech recognition and translation, The server 12 and the database 130 connected to the radiotelephone network are used to provide an interpretation function for a communication device not connected to the network in an environment not connected to the network such as an airplane operating in the sky or a foreign local light spot I can not do it and I have problems.

In the case of using a single interpretation device of the user terminal type according to the related art, most of the conversationists are limited to only one of them and the other does not have a user terminal, It is inconvenient for both parties to use it as a tool for input / output.

2 is an illustration of an interpretation situation according to the prior art by one user terminal.

Referring to FIG. 2, between the two talkers, ① an utterance-② interpreter, ③ an utterance-④ interpreter are performed according to the order of numbers. In this case, there is a problem that both dialogue users have to input voice by alternately using one interpretation device and output it. Simultaneous conversation is impossible or difficult to handle, and a voice must be input to the interpreter at regular intervals.

Prior Art 1 and Korean Patent No. 10-1626109 (May 25, 2015) disclose techniques for interpreting devices and methods.

The prior art 1 includes a voice input unit, a control unit, a communication unit, a display unit, and a user input unit for receiving operational input on previously translated sentences from the user.

Prior Art 2 and Korean Patent No. 10-1747874 (Jun. 19, 2017) disclose an automatic interpretation system.

The prior art 2 relates to an automatic interpretation system for communicating with portable devices such as a PC or a mobile phone, a smart phone, a PDA, a laptop, or the like, or used for a direct automatic interpretation terminal, A wearable automatic interpretation input / output device for transmitting a microphone signal, a bone-cord microphone signal and a gesture signal of a speaking person through a network and outputting an interpretation result signal received through a network; And a voice recognition section for detecting voice data sections in the voice recognition microphone signal by using a bone-cord microphone signal or a gesture signal transmitted through the network from the wearable automatic interpretation input / output device and performing voice recognition and interpretation of voice data in the detected section And a server for transmitting an interpretation result signal to the wearable automatic interpretation input / output apparatus via a network.

Prior Art 3 and Korean Patent No. 10-1589433 (Jan. 22, 2016) disclose a simultaneous interpretation system.

In the simultaneous interpretation system described in the prior art 3, at least two or more headsets for inputting and outputting a voice and an interpreter voice language output from the headset are input, and an interpreter voice language in which the interpreter voice language is interpreted is designated as a designated headset The simultaneous interpretation system using short-distance communication between users via one portable terminal enables simultaneous interpretation, thereby enabling more efficient and free conversation.

The prior art 1 includes a communication unit that transmits a sentence to be translated into the first language to the translation server and receives a translated sentence of the second language from the translation server. Based on this, Able to know.

Also, the prior art 2 receives user voice signals provided from the voice microphones, a user's bone marrow signal provided from the bone marrow microphones, and a user behavior (gesture) detection signal provided from the motion sensor, And a communication module for transmitting the voice recognition result information transmitted from the first server to the second to nth servers or outputting the interpretation result information transmitted from the second to n servers Based on this, it can be seen that the server performs voice recognition and interpretation.

The control unit of the prior art 3 acquires the voice of the second language converted based on the voice of the first language by using an interpretation server for converting the voice of the first language into voice of the second language, It can be seen that the interpretation server is used.

The interpreter apparatus and method according to the embodiment of the present invention relate to a technique that enables simultaneous interpretation without using an interpretation server on a network in a situation where an internet environment such as an airplane in flight or a foreign local travel destination is not established, The present invention is intended to solve the above problems as a technique different from the prior art that has been examined.

It is an object of the present invention to provide an interpretation apparatus and a method thereof, which are created to solve the above-mentioned problems, using a portable server.

It is also an object of the present invention to provide an interpretation apparatus and an interpretation apparatus capable of interpreting in an environment where a remote network environment is not provided.

It is another object of the present invention to provide an interpretation apparatus and a method for providing an input device and / or an output device to a counterpart of a speaker who does not have a user terminal, thereby reducing inconvenience.

It is another object of the present invention to provide an interpretation apparatus and method which can quickly output a reliable result by using a portable server that stores data relating to speech recognition and translation in a relatively small capacity in a high capacity memory.

An interpretation device according to an embodiment of the present invention includes a translation module included in itself to perform bidirectional translation; And a storage module in which a speech recognition DB and a translation DB are stored, characterized in that it performs bidirectional simultaneous interpretation and functions as an input device and / or an output device for at least one of the speakers in the relationship of parties in simultaneous interpretation .

The interpretation apparatus may further include a speech recognition module for performing a bidirectional speech recognition using the recorded data and / or the recorded data received from the user terminal.

The speech recognition DB and the translation DB correspond to a database determined by reducing the recognition range in descending order of frequency of appearance according to the type of language. And an engine using the engine.

Further, the interpretation apparatus may further include an input button for determining the voice input of the speaking party and / or the other party.

Further, the interpretation apparatus may further include a communication unit for communicating with the user terminal by wire or wireless. When the user terminal functions as an input device and / or an output device for the one of the speakers, And / or an output device.

The interpretation apparatus may further include a control module for generating an acknowledgment message for confirming the completed operation to the user terminal.

The reproducing module may reproduce the TTS data transmitted from the user terminal and output the result to the speaker.

The interpretation apparatus may further include the user terminal having a control unit for controlling the portable server.

In addition, the control unit controls driving of the portable server, voice recording, transmission of recorded data, and transmission of acknowledgment messages.

Also, the user terminal may further include a storage unit, and the storage unit may store a server program installed to link the control unit and the control module, and the storage module may include a client installed to link the control unit and the control module, And stores the program.

In addition, the portable server may further include a power button for switching between Standby, Ready, and Run states in relation to the user terminal, and for switching between a Power On state and a Power Off state.

The portable server is characterized in that, in the standby state, the communication module is on, the control module is in a sleep state, and the control module is turned on in the ready state.

Further, the control module is characterized in that the tone color is determined using the sound spectrum of the speaking person's voice, and the speaking person of the different language speech is identified simultaneously according to the determined tone color.

Further, the control module distinguishes the kind of the heterogeneous language by using a scoring according to the translation result of the sample voice, with respect to the speech of the simultaneous uttered heterogeneous language.

The interpretation method according to an embodiment of the present invention is characterized in that a portable server functioning as an input device and / or an output device for at least one of the speakers in the relationship with the parties of the simultaneous interpretation, Recognizing speech using a speech recognition module and a speech recognition DB; And translating the recognized text using a translation module and a translation DB included in the translation module.

The recognizing of the voice may include receiving a voice of at least one of the speakers through a microphone, recording the voice through the voice recognition module, and performing voice recognition using the recorded data.

The interpretation method may further include a step of outputting a synthesized voice corresponding to the voice of the one of the speakers in accordance with the translation of the translation module through a reproduction module provided with a speaker.

In addition, the step of recognizing the voice may use an input button to determine a voice input time point of the user and / or the other party.

In addition, the interpretation method further includes a step of the wired or wireless communication between the portable server and the user terminal, and when the user terminal functions as an input device and / or an output device with respect to any one of the speakers, And the server functions as an input device and / or an output device for the other party.

The interpretation method may further include generating a message by the control module to transmit an acknowledgment message for confirming the completed operation to the user terminal.

The outputting of the synthesized voice may be performed by reproducing the TTS data transmitted from the user terminal and outputting the result to the speaker.

The interpretation method may further include a step of the user terminal having the control unit controlling the portable server.

In addition, the controlling step controls driving of the portable server, voice recording, transmission of recorded data to the portable server, and display of the confirmation message.

Also, the user terminal may further include a storage unit, and the storage unit may store a server program installed to link the control unit and the control module, and the storage unit may store a client program installed to link the control unit and the control module, Is stored.

In addition, the portable server may switch between a Standby, Ready, and Run states in relation to the user terminal, and switch between a Power On state and a Power Off state through a power button.

The portable server is characterized in that, in the standby state, the communication module is on, the control module is in a sleep state, and the control module is turned on in the ready state.

Further, the step of recognizing the voice may further include determining a tone color using the sound spectrum of the speaker's voice, and distinguishing the speakers of the simultaneous-uttered heterogeneous language voice according to the determined tone color.

The step of recognizing the speech may further include the step of discriminating the kind of the heterogeneous language speech using scoring according to the translation result of the sample speech with respect to the speech of the simultaneous speaking heterogeneous language do.

According to the present invention, bidirectional simultaneous interpretation is possible even in an environment that is not connected to the Internet by using a portable server.

In addition, a terminal capable of inputting and / or outputting audio related to interpretation can be provided to the other party who does not have the terminal.

In addition, reliable results can be promptly output by using a portable server including a high capacity memory storing a DB relating to speech recognition and translation.

BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is an illustration of an interpretation system according to prior art using a translation server.
2 is an illustration of an interpretation situation according to the prior art by one user terminal.
3 is a block diagram of an interpretation device according to an embodiment of the present invention.
4 is a block diagram of a storage module in accordance with an embodiment of the present invention.
5 is a front view of an interpretation device according to an embodiment of the present invention.
FIG. 6 is an exemplary diagram showing the transition of the state of an interpreter according to an embodiment of the present invention. FIG.
7 is an exemplary diagram illustrating communication between a portable server and a user terminal according to an embodiment of the present invention.
8 is a block diagram of a user terminal in accordance with an embodiment of the present invention.
9 is a flowchart of an interpretation method according to an embodiment of the present invention.
FIG. 10 is a flow chart of step S130 in the interpretation method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Like reference symbols in the drawings denote like elements. Furthermore, specific structural and functional descriptions for embodiments of the present invention are presented for the purpose of describing an embodiment of the present invention only, and, unless otherwise defined, all terms used herein, including technical or scientific terms Have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as ideal or overly formal in the sense of the art unless explicitly defined herein .

Hereinafter, an interpreter according to an embodiment of the present invention will be described.

3 is a block diagram of an interpretation device according to an embodiment of the present invention.

The interpretation apparatus 100 according to an embodiment of the present invention having the components shown in FIG. 3 corresponds to a portable server. In order to distinguish between the interpreter device 100 including the user terminal 200 and the interpreter device 100 including the user terminal 200 described below, the interpreter device 100, which does not include the user terminal 200, Is defined as the portable server (100).

3, the interpretation apparatus 100 according to an embodiment of the present invention includes a speech recognition module 110, a translation module 120, a reproduction module 130, a storage module 140, an input module 150, An output module 160, a power module 170, a communication module 180, and a control module 190. In addition, the interpretation device 100 may further include a user terminal 200. That is, the interpreting device 100 operates in a stand-alone mode, which is not coupled with the user terminal 200 in a wired / wireless manner, and performs the function thereof, or is coupled with the user terminal 200 in a wired / And perform its function. However, the TTS engine needs to be included internally for independent operation.

The interpretation device 100 is a portable server including a speech recognition DB and a storage module 140 in which a translation DB is stored, and performs interpretation for at least one speaker in the relationship of the parties of the simultaneous interpretation to the input device and / .

Here, the 'function as an input device and / or an output device' is a function that uses a source language and a target language, including simultaneous interpretation between two parties In the case of plural words, it is meant that a speaking person or a speaking person on one side and a speaking person or a speaker on both sides can input his / her voice and listen to the voice output of the translated side.

The voice recognition module 110 records voice of a speaking person and performs voice recognition using the recorded data. The voice recognition module 110 automatically recognizes the voice signal from the mouth of the speaker and converts the voice signal into a character string. Another name of the sound recognition module is ASR (Automatic Speech Recognition), Voice Recognition or STT (Speech-to-Text).

The speech recognition module 110 may be based on a probability statistic. That is, the voice recognition module 110 uses a probability model based on acoustic models (Acoustic Model, AM, and Language Model, LM) used in speech recognition. And the key algorithm, HMM (Hidden Markov Model), can also be based on probability statistics. The above models are examples and do not limit the present invention.

GMM (Gaussian Mixture Model) can be used as an acoustic model, and N-gram can be used as a language model. Furthermore, it is also desirable to use a Deep Neural Network (DNN), which is one of the Deep Learning architectures instead of the GMM. In order to improve speech recognition performance, a good speech model and a language model are set, and the set models can be learned by a deep learning algorithm. It is desirable that the learning DB necessary for learning includes a spoken and dialogue voice and a language DB.

The translation module 120 translates the output text into a character of a target language when the voice of a speaking person uttered by the source language recognized by the voice recognition module 110 is outputted as text. The interpretation apparatus 100 according to an embodiment of the present invention may include the translation module 120 in addition to the speech recognition module 110. [

The translation method performed by the translation module 120 includes at least one of a rule-based method, a corpus-based method, and a neural machine translation (NMT). Rules-based methods are divided into direct translation, indirect conversion, and intermediate language depending on the depth of analysis. A corpus-based method is an example-based method and a statistical-based method.

Stochastic Machine Translation (SMT) technology is a technique of translating sentences by learning model parameters from bilingual corpus through statistical analysis. When developing a grammar or semantic representation, we make a model for translation from the corpus of language pairs that we want to translate rather than by hand. Therefore, if we can secure only a corpus, we can expand the language relatively easily.

A disadvantage of statistical-based automatic translation technology is that large bilingual corpus is required and there is no common semantic representation linking multiple languages.

Neural Machine Translation (NMT) is a technique to overcome these drawbacks.

SMT is a method of translating sentences based on a statistical model after breaking a word or a group of words into phrases. The key is to model statistical translation rules based on massive learning data.

On the other hand, NMT translates sentences as a whole through artificial intelligence (AI). This is because the artificial neural network converts the sentence information into a vector (coordinate value) which means a specific point in the virtual space.

For example, the word 'person' is' [a, b, c, ... , x, z] '. Since the vector contains all the information such as words, phrases, and word order, it is possible to translate sentence units that understand the context. Artificial neural networks place sentences with similar meanings close to each other.

Higher order vectors are utilized in the NMT technique. The artificial neural network is learned by using the learning data consisting of the sentence of the source language and the sentence of the destination language, and the learned artificial neural network recognizes the sentence information as a vector.

The reproduction module 130 outputs the synthesized voice corresponding to the voice of the speaker according to the translation of the translation module 110. [ To this end, the user terminal 200 transmits the TTS data to the interpretation device 100. In this case, the user terminal 200 includes a speech synthesis module. The playback module 130 plays back the TTS data transmitted from the user terminal 200, and the result is output to the speaker.

On the other hand, when the interpreting device 100 includes a voice synthesizing module, the interpreting device 100 itself generates TTS data and reproduces it.

Speech synthesis is called text-to-speech (TTS) or voice synthesis. As a method of voice synthesis, a music composition method can be used. In the combination method, sentences are extracted from the score database according to the analysis and analysis results. The synthesized sounds of the various candidates are generated, and the most suitable one is adopted in consideration of the rhythm and smoothness. Further, the sound spectrum of the speaker's voice is used to determine the tone color of the speaker, and the synthesized sound is post-processed so as to match the tone color, so that a synthesized sound close to the tone of the original speaker can be output. In addition, the emotion of the speaker may be perceived, and the perceived emotion may be displayed in the synthesized sound.

4 is a block diagram of a storage module in accordance with an embodiment of the present invention.

4, the storage module 140 basically stores a voice recognition DB and a translation DB, and additionally stores a TTS engine. Further, in order to interoperate between the interpreter 100 and the user terminal 200, And a server program corresponding to the client program stored in the server 200.

The speech recognition DB 141 according to an embodiment of the present invention is a DB constructed by learning speech by various utterances using an algorithm of deep learning and reducing or enlarging the recognition range according to the frequency of uttered contents . That is, in order to recognize the contents of the utterance having a high frequency, it is necessary to increase the DB amount relatively and reduce the amount of DB to recognize the contents of the utterance with a low frequency.

In order to obtain a high-quality speech recognition rate, the larger the amount of the speech recognition DB 141 is, the more advantageous it is, but there is a problem of time delay and overload. According to the above method, it is possible to construct a low- Do.

Also, as with the above method, various translation examples can be learned by using the deep learning algorithm, the colloquial expression can be expanded according to the frequency of translation examples, have.

Therefore, the speech recognition module 110 and the translation module 120 according to the present invention can use a low-capacity speech recognition DB or a translation DB compared to the DB constructed without considering the frequency.

Here, the storage module 140 includes a volatile RAM, a nonvolatile ROM, and a flash memory, and stores various digital files according to the functions thereof.

The input module 150 includes a microphone 151, a voice input button 152, and a power button 153.

5 is a front view of an interpretation device 100 according to an embodiment of the present invention.

Referring to FIG. 5, a microphone 151 receives a voice of a speaker and converts the voice into an electrical signal. The electrical signal can then be digitally converted by the A / D converter. In FIG. 5, the positions of the speaker 161 and the microphone 151 are not limited to those shown in FIG. 5, but may be designed to be placed close to the lower USB jack.

The voice input button 152 can be used to determine the voice input of the speaker. The speech recognition module 110 should provide not only the transcription corresponding to the inputted speech but also the information about the sentence boundary. For this purpose, a voice input button 152 may be used.

 That is, the speaker starts speaking or touching the voice input button 152 or touches or touches the voice input button 152 at the beginning and end of the utterance, respectively, That is, the speech recognition module 110, as shown in FIG.

If you process text that is not delimited by sentence units, translation performance may be degraded. In order to prevent this, a voice input button 152 may be used to input voice in units of sentences in a previous step of voice recognition.

The voice input button 152 may share the function with a power button 153 and a button, which will be described later. In other words, on / off of the voice input is switched by touching using one button, and on / off of the power can be switched by long and short pressing operation. In this case, in order to operate the voice input button 152, it is preferable to incorporate a capacitor on the touch surface to detect whether or not the touch is performed.

Since the power button 153 is depressed for a short time or a long time by the depressing operation, power can be applied to the interpreter device 100 to start driving or terminate the driving. Then, the interpreting device 100 can be placed in a state of Ready and Standby after Power On. The interpreting device 100 can consume power differently according to the above-mentioned state. A detailed description thereof will be described later.

Referring again to FIG. 5, the output module 160 includes a speaker 161 and various LED indicators 162, 163, and 164. The LED indicator includes a battery status display LED 162, a power on / off status display LED, and a communication connection display LED with the user terminal 200.

The power module 170 includes a power source and a charge / discharge device for charging and / or discharging the power source. A battery can be used as a power source. Then, the battery is charged and discharged by the charge / discharge device, and the discharged electric power acts as an energy source for driving the interpretation device 100. [

Power supply to the interpretation apparatus 100 is performed by a power button. Hereinafter, the status change of the interpretation apparatus 100 using the power button will be described.

FIG. 6 is an exemplary diagram showing the transition of the state of an interpreter according to an embodiment of the present invention. FIG.

Referring to FIG. 6, the interpretation apparatus 100 as a portable server can be placed in a power on state, a ready state, a standby state, a run state, and an off state. Switch.

In the standby state, the communication module 180 is in an ON state and the control module 190 is in a sleep state. In the Ready state, the control module 190 is switched from the Off state to the On state.

When the power button is pressed in the power off state, the interpreting device 100 switches to the ready state. Then, after waiting for a predetermined time, the communication module 180 is turned on, and the interpreter device switches to the standby mode.

Next, when the control module 190 of the interpretation apparatus 100 and the control unit 270 of the user terminal 200 are interlocked, that is, turned on, the interpreting apparatus 100 switches to the Ready state.

Next, the interpreter enters the Run state, which performs the function according to the command in the ready state. After waiting for a predetermined time even in the ready state, the interpreting device 100 can switch to the standby state.

Most of the operations are in the Ready or Run state. When the power button is depressed in these states, the translating apparatus 100 switches to the power off state.

The communication module 180 communicates with the user terminal 200 in a wired or wireless manner. In this case, when the user terminal 200 functions as an input device and / or an output device for a speaking party on either side of the talker, the interpreting device 100 functions as an input device and / . That is, the other party can input his / her voice to the dedicated interpreting device 200, and can listen to the translated voice of the speaker by outputting.

7 is an exemplary diagram illustrating communication between a portable server 100 and a user terminal 200 according to an embodiment of the present invention.

Referring to FIG. 7, the interpretation device 100 and the user terminal 200 corresponding to the portable server are in the Bluetooth pairing state. According to the embodiment shown in FIG. 7, the source language can be set to Korean and the destination language can be set to English using the UI of the client program of the user terminal 200. [ Then, by touching the microphone button having the Korean and English names and making a speech, it is possible to alleviate the burden of the interpreting device 100 discriminating the language.

The control module 190 generates an acknowledgment message to confirm to the user terminal 200 that the operation performed by the interpreter device 100, for example, voice recognition or translation, has been completed. The generated confirmation message is transmitted to the user terminal 200 through the communication unit.

In general, the control module controls driving of the portable server as the interpretation apparatus 100, voice recording, transmission of recorded data, and transmission of acknowledgment messages.

There is no guarantee that one side's utterance ends and the other's utterance is between the utterances. Therefore, it is necessary to distinguish the different kinds of speech inputted at the same time in the simultaneous utterance. In order to solve this problem, the control module 190 determines characteristics of the tone color using the sound spectrum of the speaker's voice, and filters the voice of the simultaneous speaking heterogeneous language using the determined characteristics of the tone color using the filter. Accordingly, the speakers of the different language speech are distinguished, and the sounds of the different language can be separated from each other by filtering.

Furthermore, the control module 190 distinguishes which language the disparate languages correspond to, using scoring according to the translation result of the sample voice, for the voice of the simulated speech of the different languages .

Specifically, in the case where voices of English and Korean are input mixedly, the filtered speech signal according to the characteristics of the tone of the English speaker and the tone of the Korean speech is translated into English and Korean for one speech signal, We attempt to translate Korean and English into English and translate them into points, and adopt the translation with the highest score to decide which language is the language.

8 is a block diagram of a user terminal in accordance with an embodiment of the present invention.

8, a user terminal 200 according to an exemplary embodiment of the present invention includes a communication unit 210, a display 220, a storage unit 230, an input unit 240, an output unit 250, a power source unit 260 And a control unit 270. [0033]

Various embodiments of the user terminal 200 may include a cellular telephone, a smartphone having wireless communication capabilities, a personal digital assistant (PDA) having wireless communication capabilities, a wireless modem, a portable computer having wireless communication capabilities, a digital Gaming devices with wireless communication, music storage and playback appliances with wireless communication capabilities, Internet appliances capable of wireless Internet access and browsing, as well as portable devices incorporating such combinations of functions Units or terminals, but is not limited thereto.

The communication unit 210 may include a communication module corresponding to various communication networks of the communication network 14, for example, a Bluetooth module, a WiFi module, an Ethernet, a USB module, and a cellular wireless communication module. It is most preferable to include a wired communication unit such as a USB module and a short-range communication module such as a Bluetooth module, a Zigbee module, and an NFC module.

The display 220 corresponds to a device for displaying a screen composed of pixels such as an LCD display and an LED display.

The storage unit 230 stores a client program for linking the control unit 207 and the control module 109. The storage unit 230 includes a volatile RAM, a non-volatile ROM, and a flash memory, and stores various digital files according to the functions thereof. In particular, the storage unit 230 may store the TTS engine, not the portable server 100, but the user terminal.

The input unit 240 includes a keyboard, a touch screen, and a mouse for setting various parameters.

The output unit 250 includes a speaker, a headset, and an earset. Especially, the headset and earset formed integrally with the microphone have useful functionality in hands-free simultaneous interpretation.

The power supply unit 260 includes a power source and a charge / discharge device for charging and / or discharging the power source. A battery can be used as a power source. The battery is charged and discharged by the charge / discharge device, and the discharged power acts as an energy source for driving the user terminal 200.

The control unit 270 may be implemented as a CPU (Central Processing Unit) corresponding to a central processing unit in terms of hardware. Specifically, the control unit 270 may include a storage unit 230 for interlocking with the control module 190 of the interpretation apparatus 100, And the CPU that performs an operation in this state.

The control unit 270 may operate the control module 190 to perform various functions of the interpretation device 100. [

According to an embodiment of the present invention, the language setting to be performed by the interpreting device 100 and the user terminal 200 using the client program can be automatically performed. That is, the control unit 207 can automatically set Korean as the source language by referring to the setting language of the user terminal 200, including the control module 109.

In addition, when the user terminal 200 set in the heterogeneous language is within a certain distance, the source language and the object language can be automatically set by referring to the language setting of each user terminal.

In addition, the control unit 207 scales the degree of completeness of translation by using a sample of speech uttered through the interpreter 100, and automatically sets the language having the highest score as the arrival word.

Hereinafter, an interpretation method according to an embodiment of the present invention will be described.

9 is a flowchart of an interpretation method according to an embodiment of the present invention.

9, an interpretation method according to an embodiment of the present invention includes interpreting apparatuses corresponding to a portable server functioning as an input device and / or an output device with respect to at least one speaker in relation to parties in simultaneous interpretation, (100). ≪ / RTI >

The interpretation method may include: a step (S110) of the wired or wireless communication between the portable server and the user terminal; A step S120 of controlling the portable server by the user terminal 200 having the control unit; Recognizing a voice using a voice recognition module and a voice recognition DB included in the voice recognition module (S130); And translating the recognized text using a translation module and a translation DB included in the translation module (S140); (S150) of outputting a synthesized voice corresponding to a voice of one of the speakers in accordance with the translation of the translation module via the playback module; And a step S160 of causing the control module to generate a message and transmit it to the user terminal 200 in order to transmit a confirmation message to the user terminal 200 to confirm the operation completed by the translating device 100 do.

Here, the step of recognizing the voice (S130) may include receiving the voice of at least one of the speakers through a microphone, recording the voice through the voice recognition module, and performing voice recognition using the recorded data do.

Herein, the voice recognition step may include using a voice input button to determine a voice input time point of the user and / or the other party.

Further, when the user terminal functions as an input device and / or an output device with respect to any one of the speakers, the portable server functions as an input device and / or an output device with respect to the other user.

Here, the step of outputting the synthesized voice (S150) is characterized by reproducing the TTS data transmitted from the user terminal 200 and outputting the synthesized voice.

Here, the controlling step controls driving of the portable server 100, voice recording, transmission of recorded data, and transmission of the confirmation message.

Here, the step of recognizing the voice may further include determining a tone color using the sound spectrum of the voice, and distinguishing a speaker of a different language speech that has been simultaneously fired according to the determined tone color.

Here, the step of recognizing the speech includes the step of discriminating the kind of the heterogeneous language using scoring according to the translation result of the sample speech, with respect to the speech of the different language of the simultaneous utterance .

FIG. 10 is a flow chart of step S130 in the interpretation method according to an embodiment of the present invention.

Referring to FIG. 10, the step of recognizing speech comprises the steps of: determining a tone color using a sound spectrum of a speaker's voice; discriminating a speaker of a different language speech that has been simultaneously uttered according to the determined tone color; And discriminating the kind of the heterogeneous language using scoring according to the translation result of the sample voice, for the voice of the simultaneous uttered heterogeneous language.

It is preferable that sounds input to the interpreting device 100 and the user terminal 200, for example, voices of the speakers, are distinguished from each other.

However, there may be a case where the first speaker using Korean and the second speaker using English speak at the same time. In addition, the voice of the first speaker, which is input through the microphone of the user terminal 200 adjacent to the first speaker, is interpreted by the interpreter 200 corresponding to the portable server close to the second utterance due to the ringing of the space or the high decibel value, Lt; / RTI >

In such a case, a voice in which the voice of the first speaker and the voice of the second speaker are mixed is also input through the interpreter device 100 and inputted through the user terminal 200 as well. Therefore, the interpreting device 100 that needs to recognize a voice needs to distinguish and separate it from a mixture of English and Korean before recognizing the speech.

In order to separate mixed speech, it is necessary to distinguish which partial speech belongs to which one of the partial voices constituting the mixed speech. To solve this problem, it is necessary to distinguish the speakers. Furthermore, if we can distinguish the kind of language, we will be able to match the distinctive language with the distinctive speaker.

In general, an interpretation apparatus and method according to an embodiment of the present invention can perform interpretation using a portable server in a situation where a wireless communication environment is not provided, provide an input / output device for a counterpart, It is possible to improve the quality of speech recognition by inputting the arrival word separately, and in order to distinguish between the utterances of the simultaneous utterances of the different languages according to the tone of the utterance using the sound spectrum, To an interpretation apparatus and method for distinguishing the types of languages.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, I will understand the point. Accordingly, the technical scope of the present invention should be determined by the following claims.

11, 200: user terminal 12, 300: interpretation server
13: Database 14: Network
100: Interpretation device 110: Voice recognition module
120: translation module 130: reproduction module
140: storage module 141: voice recognition DB
142: translation DB 150: input module
151: microphone 152: voice recognition button
153: power button 160: output module
161: Speaker 162: Battery status display LED
163: Power On / Off display LED
164: Bluetooth display LED 170: Power module
200: user terminal 210:
220: display unit 230: storage unit
240: input unit 250: output unit
260: power supply unit 270:

Claims (30)

A translation module included within itself that performs bidirectional translation; And
A portable server including a voice recognition DB and a translation DB, and a storage module for storing the voice recognition DB and the translation DB, wherein the voice recognition DB and the translation DB are stored in the portable terminal, Interpreters. The method according to claim 1,
The interpretation apparatus comprises:
Further comprising: a voice recognition module for recording a voice of a speaking person and performing bidirectional voice recognition using the recording data and / or the recording data received at the user terminal. The method according to claim 1 or 2,
The speech recognition DB includes:
The database is constructed by learning the speech due to various utterances using the deep learning algorithm and reducing or enlarging the recognition range according to the frequency of utterance contents.
The translation DB,
It is a database constructed by learning various translation examples using the algorithm of deep learning, expanding colloquial expressions according to frequency of translation examples,
The speech recognition engine or translation engine,
Characterized in that a voice recognition DB or a translation DB of a low capacity as compared with a DB whose frequency is not taken into account is used. The method of claim 2,
The interpretation apparatus comprises:
Further comprising a reproduction module for outputting a synthesized voice corresponding to a voice of a speaker in accordance with the translation of the translation module. The method of claim 2,
The interpretation apparatus comprises:
Further comprising a voice input button for determining the voice input of the speaker and / or the other party. The method of claim 5,
The interpretation apparatus comprises:
Further comprising a communication module for communicating wired or wireless with the user terminal,
Wherein the user terminal functions as an input device and / or an output device for the other user when the user terminal functions as an input device and / or an output device for the one of the speakers. The method of claim 6,
The interpretation apparatus comprises:
Further comprising a control module for generating a confirmation message for confirming the completed operation to the user terminal so as to transmit the confirmation message to the user terminal. The method of claim 6,
Wherein the playback module comprises:
And reproduces the TTS data transmitted from the user terminal and outputs the result to the speaker. The method of claim 6,
The interpretation apparatus comprises:
Further comprising: a user terminal having a control unit for controlling the portable server. The method of claim 9,
The control module includes:
Wherein said control means controls driving of said portable server, voice recording, transmission of recorded data, and transmission of said acknowledgment message. The method of claim 10,
The user terminal further includes a storage unit,
Wherein the storage unit stores a client program for linking the control unit and the control module,
Wherein the storage module stores a server program for the client program. The method of claim 6,
The portable server includes:
Further comprising a power button for switching between Standby, Ready, and Run states in relation to the user terminal and switching between Power On state and Power Off state, In claim 12,
The portable server includes:
In the standby state, the communication module is in an on state, the control module is in a sleep state,
And in the Ready state, the control module is switched to the On state. The method of claim 7,
The control module includes:
Wherein a tone color is determined using a sound spectrum of a speaking person's voice and a speaking person of a simultaneous speaking different language speech is distinguished according to the determined tone color. 15. The method of claim 14,
The control module includes:
Characterized in distinguishing different kinds of languages by using scoring according to the translation result of the sample speech with respect to the speech of the simultaneous uttered heterogeneous language. A portable server functioning as an input device and / or an output device for at least one speaker in a relationship with the parties of the simultaneous interpretation,
Recognizing speech using a speech recognition module and a speech recognition DB included in the speech recognition module; And
And translating the recognized text using a translation module and a translation DB included in the translation module. 18. The method of claim 16,
The step of recognizing the speech comprises:
Receiving a voice of the at least one speaking person through a microphone,
Wherein the voice recognition module records the voice and performs voice recognition using the voice data. 18. The method of claim 17 or 18,
The speech recognition DB includes:
The database is constructed by learning the speech due to various utterances using the deep learning algorithm and reducing or enlarging the recognition range according to the frequency of utterance contents.
The translation DB,
It is a database constructed by learning various translation examples using the algorithm of deep learning, expanding colloquial expressions according to frequency of translation examples,
The speech recognition engine or translation engine,
Wherein a voice recognition DB or a translation DB of a lower capacity than the DB in which the frequency is not taken into account is used. 19. The method of claim 18,
The interpretation method comprises:
Further comprising the step of outputting a synthesized voice corresponding to the voice of one of said speakers in accordance with the translation of said translation module via said reproduction module. 19. The method of claim 18,
The step of recognizing the speech comprises:
Wherein a voice input button is used to determine a voice input point of the user and / or the other party. The method of claim 20,
The interpretation method comprises:
Further comprising the step of the wired or wireless communication between the portable server and the user terminal,
When the user terminal functions as an input device and / or an output device for any one of the speakers,
Wherein the portable server functions as an input device and / or output device for the other party. 23. The method of claim 21,
The interpretation method comprises:
Further comprising the step of the control module generating a message to cause the user terminal to transmit a confirmation message to confirm the completed operation. 23. The method of claim 21,
Wherein the step of outputting the synthesized voice comprises:
And reproduces the TTS data transmitted from the user terminal and outputs a synthesized voice. 23. The method of claim 21,
The interpretation method comprises:
Further comprising the step of the user terminal having the control unit controlling the portable server. 27. The method of claim 24,
Wherein the controlling comprises:
Wherein said control means controls the operation of said portable server, voice recording, transmission of recorded data, and transmission of said acknowledgment message. 26. The method of claim 25,
The user terminal further includes a storage unit,
Wherein the storage unit stores a client program for linking the control unit and the control module,
Wherein the storage module stores a server program for the client program. 23. The method of claim 21,
The portable server includes:
Wherein the control unit switches between Standby, Ready, and Run states in relation to the user terminal, and switches between a Power On state and a Power Off state through a power button. According to claim 25,
The portable server includes:
In the standby state, the communication module is in an on state, the control module is in a sleep state,
And in the Ready state, the control module is switched to the On state. 23. The method of claim 22,
The step of recognizing the speech comprises:
Determining a tone color using the sound spectrum of the speaker's voice and distinguishing the speakers of the different language speech which are simultaneously fired according to the determined tone color. 29. The method of claim 29,
The step of recognizing the speech comprises:
The method further comprises the step of discriminating the kind of the heterogeneous language speech using scoring according to the translation result of the sample speech with respect to the speech of the simultaneous uttered heterogeneous language.

Images