KR102158739B1 - System, device and method of automatic translation - Google Patents

Abstract

An automatic interpretation device using a two-channel voice signal according to the present invention includes a speaker, a communication module for transmitting and receiving data with an earphone device including first and second microphones, and a program for generating an interpretation result using the two-channel voice signal. And a processor for executing the stored memory and a program stored in the memory, wherein the processor transmits a first voice signal, which is a speaker's voice received through the first microphone, through the second microphone as the program is executed. Automatic interpretation by comparing the noise signal received through the noise signal and the second voice signal including the speaker's voice, and extracting all or selectively the speaker's voice included in the first and second voice signals based on the comparison result Perform.

Description

Automatic interpretation system, device and method {SYSTEM, DEVICE AND METHOD OF AUTOMATIC TRANSLATION}

The present invention relates to an automatic interpretation system, device, and method using a two-channel voice signal.

Due to the recent improvement in voice recognition technology, the automatic interpretation function using a smartphone is widely used, and many applications support multilingual services. However, the improved voice recognition performance and multi-language support raise expectations for automatic interpretation, but it does not provide smooth functions in situations where actual interpretation is required.

However, in the case of automatic interpretation using a smartphone, the following problems may occur.

First, users usually share one device to interpret two languages. In this case, because they speak alternately across devices, they stay at the question and answer level in the form of simply uttering short sentences rather than natural conversations.

Second, when different users use their own devices and use the same application, the inconvenience of alternately speaking by sharing devices may be reduced, but translations and synthesized sounds of the interpreter language are delivered to the other party unless the two devices are paired. There is a problem that does not work.

Third, when devices of different users are paired with each other and a hands-free earphone is used, a more natural conversation is possible, but when two users talk in close proximity, the crosstalk problem that each other's utterances are input to the microphone of the other's hands-free earphone Will occur.

The present invention is an automatic interpretation system, device, and method using a two-channel voice signal that blocks speech interference between speakers and enables strong voice recognition against external noise so that a plurality of users can have a natural level of conversation in an automatic interpretation situation. Want to provide.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, the automatic interpretation device using a two-channel voice signal according to the first aspect of the present invention is a communication module that transmits and receives data with an earphone device including a speaker, first and second microphones. And a memory storing a program for generating an interpretation result using the two-channel voice signal and a processor for executing the program stored in the memory. At this time, as the processor executes the program, the first voice signal, which is the voice of the speaker received through the first microphone, is received through the second microphone, and the second voice includes the noise signal and the voice of the speaker. An automatic interpretation is performed by comparing the voice signal and extracting all or selectively the speaker's voice included in the first and second voice signals based on the comparison result.

The first microphone is located inside the earbud of the earset device, and the second microphone is located outside the earbud, and may be located within a preset distance from the speaker's mouth.

The first voice signal and the second voice signal are converted into the one-channel signal in the earphone device and received through the communication module, and the processor decodes the one-channel signal to determine the first two-channel voice signal. And extraction as a second voice signal.

The processor detects a voice section and a pause section of the speaker from the first voice signal, detects a noise signal in a section corresponding to the detected idle section in the second voice signal, and detects the detected noise signal. The speaker's voice may be extracted based on the signal-to-noise ratio.

When the signal-to-noise ratio of the noise signal falls within a preset range, the processor weights the first and second voice signals, and extracts the speaker's voice from the weighted first and second voice signals. have.

When a plurality of noises are included in the noise signal, the processor may each generate an acoustic model corresponding to the type of noise and perform the automatic interpretation through the acoustic model.

When the signal-to-noise ratio of the noise signal exceeds a preset range, the processor may extract a speaker's voice included in the second voice signal and perform the automatic interpretation.

When the signal-to-noise ratio of the noise signal is less than a preset range, the processor may extract a speaker's voice included in the first voice signal and perform the automatic interpretation.

In addition, an automatic interpretation method using a two-channel voice signal in an automatic interpretation device according to a second aspect of the present invention includes the steps of: receiving first and second voice signals converted into a one-channel signal from an earphone device; Decoding the 1-channel signal into the first and second audio signals, which are 2-channel audio signals; Comparing the first and second voice signals; And extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result, and performing automatic interpretation for the extracted speaker's voices. In this case, the first voice signal includes the speaker's voice received through the first microphone of the earset device, and the second voice signal is a noise signal received through the second microphone of the earset device and the speaker’s voice. Includes.

The first microphone is located inside the earbud of the earset device, and the second microphone is located outside the earbud, and may be located within a preset distance from the speaker's mouth.

Comparing the first and second voice signals may include detecting the speaker's voice section and the idle section from the first voice signal, and in a section corresponding to the detected idle section from the second voice signal. It may include the step of detecting a noise signal of.

In the step of extracting all or selectively the speaker's voice included in the first and second voice signals based on the comparison result, the speaker's voice may be extracted based on the signal-to-noise ratio of the detected noise signal. .

The step of extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result may include determining whether the signal-to-noise ratio of the noise signal is within a preset range; When the determination result falls within the preset range, weighting the first and second voice signals and extracting the speaker's voice from the weighted first and second voice signals.

delete

In the automatic interpretation method according to the present invention, when a plurality of noises are included in the noise signal, each acoustic model corresponding to the type of noise may be generated.

The step of extracting all or selectively the speaker's voice included in the first and second voice signals based on the comparison result may include determining whether the signal-to-noise ratio of the noise signal is within a preset range, and the When the determination result exceeds the preset range, extracting a speaker's voice included in the second voice signal may be included.

The step of extracting all or selectively the speaker's voice included in the first and second voice signals based on the comparison result may include determining whether the signal-to-noise ratio of the noise signal is within a preset range, and the When the determination result is less than the preset range, extracting a speaker's voice included in the first voice signal may be included.

In addition, the automatic interpretation system using the two-channel voice signal according to the third aspect of the present invention receives the first voice signal, which is the speaker's voice, through a first microphone located inside the earbud, and receives the first voice signal from the outside of the earbud. An earphone device for receiving a noise signal and a second voice signal including the speaker's voice through a second microphone located, converting the first and second voice signals into a one-channel signal, and transmitting the one-channel signal And extracts the first and second voice signals, which are two-channel voice signals, and the speaker's voice included in the first and second voice signals based on the result of comparing the first and second voice signals. It includes an automatic interpretation device that extracts all or selectively and performs automatic interpretation.

The automatic interpretation device detects a voice section and a pause section of the speaker from the first voice signal, detects a noise signal in a section corresponding to the detected idle section from the second voice signal, and detects the detected noise The speaker's voice included in the first and second voice signals may be extracted based on the signal-to-noise ratio of the signal.

When the signal-to-noise ratio of the noise signal falls within a preset range, the automatic interpretation device weights the first and second voice signals and extracts the speaker's voice from the weighted first and second voice signals. Thus, the automatic interpretation can be performed.

When the signal-to-noise ratio of the noise signal exceeds a preset range, the automatic interpretation device extracts the speaker's voice included in the second voice signal to perform the automatic interpretation, and the signal-to-noise ratio of the noise signal is preset. If it is less than the range, the automatic interpretation may be performed by extracting the speaker's voice included in the first voice signal.

According to any one of the above-described problem solving means of the present invention, it is possible to solve the cross-talk problem in an automatic interpretation situation using a free interactive voice recognition function.

In addition, by using a two-channel voice signal, it is possible to provide an automatic interpretation service that guarantees strong voice recognition performance against external noise through the creation and application of an acoustic model specialized for the noise section and noise type.

1 is a view for explaining an automatic interpretation system according to an embodiment of the present invention.
2 is a diagram showing an embodiment of an earset device.
3 is a block diagram of an earset device.
4 is a diagram for explaining a 1-channel signal.
5 is a block diagram of an automatic interpretation device according to an embodiment of the present invention.
6 is a diagram showing a sample of an audio waveform restored to a 2-channel audio signal.
7 is a flowchart of an automatic interpretation method according to an embodiment of the present invention.
FIG. 8 is a flowchart for explaining contents of extracting a speaker's voice by applying all or selectively the first and second voice signals.

Hereinafter, exemplary embodiments of the present application will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present application. However, the present application may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts not related to the description are omitted in order to clearly describe the present application, and similar reference numerals are attached to similar parts throughout the specification.

When a part of the specification "includes" a certain component, it means that other components may be further included, rather than excluding other components unless otherwise stated.

The present invention relates to an automatic interpretation system (1), a device (200) and a method.

Most of the smartphone applications currently in use that provide automatic interpretation functions are used by two people with different languages with one device.

For example, in Korean and English, when a Korean speaker speaks, the translated English is output or uttered, whereas when a foreign speaker speaks English, the interpreted Korean is uttered.

In this case, rather than a natural conversation, it is a minimal form of communication, and the form of vocalization is bound to be unnatural. Therefore, in order to enable interpretation in the form of a natural conversation between two people, it is necessary to break away from the existing use case of holding a device in hand and looking at an interpretation app.

First, the user's hand must be free from a smartphone device that performs an interpreter function. To do that, you need an earphone to replace the device's microphone and speakers. In this way, when two different people communicate with each other while wearing the ear set, natural vocalization is possible.

However, in this case, a problem that must be solved from the viewpoint of speech recognition arises. That is, in the method of sharing one device, the microphone of the device is used, and two people participating in the interpretation conversation alternately utter the voice, so only the voice of the speaker who is speaking is input except for external noise.

However, when two different people communicate with each other wearing an earset, not only their own utterance but also the other's utterance is inputted to the earset microphone.

For natural interpretation conversations, each other comes close to each other within 1 meter. Recently, the earset's high microphone sensitivity allows it to respond to the voice of the other party in addition to its own voice.

For example, in the case of a Korean or English interpreter, the Korean's vocalization is input to the Korean recognizer through the earset worn by the person, and at the same time, input to the English recognizer through the earset worn by the other party. The opposite is also true. In this case, the voice recognizers of each other malfunction, thereby interfering with the interpretation conversation.

In order to solve such a fatal problem, a cross-talk prevention function is required.

To prevent crosstalk, there is language recognition technology as an existing method. In the case of this technology, only one language is input to the speech recognizer because the speech recognizer recognizes the language of the speaker and receives only a specific language as an input of the corresponding speech recognizer. However, language recognition technology has a problem that interpretation is impossible when a language recognition error occurs.

In addition, in consideration of such language recognition failure, a method of simultaneously operating recognizers for a plurality of preset languages is used. However, since it is difficult to preset a user's language, the interpretation function is limited.

In order to solve such a crosstalk problem, the automatic interpretation system 1, the device 200, and the method according to an embodiment of the present invention recognize voice only when the microphone 120 of the earset device 100 speaks. Through the speaker's speech recognition function that performs the function, it is possible to allow two or more users to have a natural level of conversation in an automatic interpretation situation.

Hereinafter, the automatic interpretation system 1 and the device 200 according to an embodiment of the present invention will be described with reference to FIGS. 1 to 6.

1 is a view for explaining an automatic interpretation system 1 according to an embodiment of the present invention.

An automatic interpretation system 1 according to an embodiment of the present invention includes an earphone device 100 and an automatic interpretation device 200.

The earset device 100 may be a hands-free earset device, and may be paired with the automatic interpretation device 200 through Bluetooth Low Energy (BLE), but the pairing method is not necessarily limited to Bluetooth.

When the automatic interpretation device 200 receives a two-channel audio signal wirelessly from the earphone device 100, the automatic interpretation device 200 decodes the corresponding signal to extract the speaker's voice, and then performs automatic interpretation.

At this time, the automatic interpretation device 200 is an intelligent terminal in which computer support functions such as Internet communication and information search are added to a portable terminal, and a mobile phone or a smart phone capable of installing and executing a number of applications (ie, applications) desired by the user It may be a (smart phone), a pad, a smart watch, a wearable terminal, and other mobile communication terminals.

2 is a diagram showing an embodiment of the ear set device 100. 3 is a block diagram of the earset device 100. 4 is a diagram for explaining a 1-channel signal.

Earset device 100 according to an embodiment of the present invention, as shown in Figure 2, the speaker and the first microphone 120 located inside the earbud 110, the second microphone 130 located outside the earbud 110 , Bluetooth chipset 140 and a battery 150.

In this case, the earset device 100 may be, for example, a Bluetooth neckband-type earset, but is not limited thereto, and any device including a speaker and a microphone may be used.

The neckband-type earset device 100 includes two earbuds 110 that are inserted into both ears in a form worn on the user's neck, and each of the two earbuds 110 is provided with a speaker. In addition, the at least one earbud 110 includes a first microphone 120 for receiving a first voice signal, which is a speaker's voice. Since the first microphone 120 is located inside the earbud 110, external noise is mostly shielded, and only the voice of a speaker wearing the earset device 100 can be received relatively cleanly.

Meanwhile, the sound quality of the first voice signal acquired through the first microphone 120 in the earbud 110 is significantly lowered compared to the signal acquired with a general microphone.

To this end, the earset device 100 in an embodiment of the present invention recognizes voice by using a two-channel voice signal through a second microphone 130 located outside the earbud 110 in addition to the first microphone 120. Can improve performance.

The second microphone 130 receives a noise signal and a second voice signal including a speaker's voice. In this case, the noise signal is a concept including external noise including the voice of the speaker wearing the earset device 100 and the voice of another speaker.

The second microphone 130 is located outside the earbud 110 and may be preferably located within a preset distance from the speaker's mouth, which is a suitable position for acquiring the speaker's voice. In this case, the second microphone 130 may be positioned on the main body of the neckband in the case of the neckband type, and may be positioned on the wire in the case of a general wire type earset device.

Meanwhile, the first voice signal acquired by the first microphone 120 may be used for authenticating a user wearing the earset device 100, and the second microphone 130 is a general micro second voice signal. This is an external signal acquired by a general microphone.

At this time, the first voice signal for authenticating the user is a signal that is acquired as it is attached to the speaker's body and used for a special purpose, such as a signal resonating in the user's ear or a signal detected by the trembling of the vocal cords in the neck. it means.

The first audio signal can be clearly distinguished from the signal by his or her utterance even if the speaker does not respond to sounds other than the speaker himself or responds. In the case of using the first voice signal, even when two or more speakers speak together, only the user's voice can be distinguished, and in particular, the start and end of the user's speech can be accurately selected.

In other words, since it is possible to accurately distinguish the case where a speaker other than the self is speaking, it is possible to be free from interference problems caused by the voice of the other party. In addition, as described later, since the end point detection (EPD), which is the start and end of the speaker's speech, and the idle section can be accurately distinguished, only the external noise signal generated in the idle section can be extracted, and the extracted noise signal It is also possible to use an acoustic model specialized for the noise by discriminating the type of noise.

Referring to FIG. 3, the Bluetooth chipsets CSR8670 and 140 of the earset device 100 according to an embodiment of the present invention include an audio interface module 141, a female processor (STM32L, 142), a DSP module 143, and a Bluetooth device. Module 144.

The audio interface module 141 receives first and second audio signals, which are two-channel audio signals acquired by the first and second microphones 120 and 130. The two-channel audio signal is output to the I2S digital audio terminal through the audio interface module 141.

Meanwhile, all existing Bluetooth chipsets allow two microphone inputs, but only one channel (mono signal) through the hands-free profile (HFP) protocol through the DSP module 143 ).

Accordingly, the earset device 100 according to an embodiment of the present invention additionally uses the arm processor 142 using a serial port profile (SPP) protocol for data communication instead of the existing hands-free profile, A 2-channel audio signal, which is a second audio signal, may be converted into a 1-channel signal.

The 1-channel signal converted by the arm processor 142 may be configured to alternately in byte units as shown in FIG. 4. That is, the first voice signal may be assigned as '0' byte and the second voice signal may be assigned as '1' byte to constitute a 1-channel signal. In this case, a voice coding scheme may be applied to the converted 1-channel signal in order to prevent interference from surrounding Bluetooth signals and improve a transmission rate.

The 1-channel signal configured as described above is input through the UART terminal and may be transmitted to the automatic interpretation device 200 through the Bluetooth module 144.

On the other hand, the detailed hardware specifications of the above-described earset device 100 are only an example, and any one is sufficient if the above-described functions can be performed through a combination of configurations having various specifications.

Next, a description will be given of the automatic interpretation device 200 according to an embodiment of the present invention.

5 is a block diagram of an automatic interpretation device 200 according to an embodiment of the present invention.

The automatic interpretation device 200 according to an embodiment of the present invention includes a communication module 210, a memory 220, and a processor 230.

The communication module 210 transmits and receives data to and from the earphone device 100 and the automatic interpretation device 200 of another speaker. The communication module 210 may include both a wired communication module and a wireless communication module. The wired communication module may be implemented as a telephone line communication device, a cable groove (MoCA), an Ethernet, IEEE1294, an integrated wired home network, and an RS-485 control device. In addition, the wireless communication module may be implemented with WLAN (wireless LAN), Bluetooth, HDR WPAN, UWB, ZigBee, Impulse Radio, 60GHz WPAN, Binary-CDMA, wireless USB technology, and wireless HDMI technology.

The memory 220 stores a program for generating an interpretation result using a two-channel audio signal. Here, the memory 220 collectively refers to a nonvolatile storage device and a volatile storage device that continuously maintains stored information even when power is not supplied.

For example, the memory 220 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD), and a micro SD card. Magnetic computer storage devices such as NAND flash memory, hard disk drives (HDD), such as cards, and optical disc drives such as CD-ROM, DVD-ROM, etc. I can.

The processor 230 executes a program stored in the memory 220. Specifically, when receiving a 1-channel signal from the earphone device 100 through the communication module 210, the processor 230 converts the 1-channel signal into the first and second audio signals, which are the original 2-channel audio signals through a decoding process. Restore.

And, based on the result of comparing the first and second voice signals, all or selectively extracting all or selectively the speaker's voices included in the first and second voice signals, and performing automatic interpretation.

6 is a diagram showing a sample of an audio waveform restored to a 2-channel audio signal.

Among the two-channel audio signals, the first audio signal is a signal obtained from the first microphone 120 located in the earbud 110 of the earset device 100, and the second audio signal is a second audio signal located outside the earset device 100. 2 This is a signal obtained from the microphone 130.

As described above, the second voice signal includes not only the voice of the speaker but also the voice and noise of the surrounding speaker due to the characteristic of the second microphone 130 located outside.

On the other hand, since the first voice signal is a signal obtained by the first microphone 120 located inside the earbud 110, the level of the signal is small, but relatively independent from external noise, so that only the user's own voice is acquired. .

6 is a situation where'speaker A'and'speaker B'simultaneously utter each other in a room where music is played, and the two-channel audio signal acquired from the ear set device of'speaker A'is automatically interpreted by the'speaker A'. ).

In the second audio signal, the voices of “Speaker A” and “Speaker B” and the sound of music are all mixed, so that each section cannot be distinguished, whereas the first audio signal only records the utterances of “Speaker A”. In addition, the first voice signal can clearly distinguish between the pause sections (B) between the voice sections (A, A'of'Speaker A').

The processor 230 detects the speaker's voice section (A, A') and the pause section (B) from the first voice signal, and a second voice signal corresponding to the detected pause section (B) of the first voice signal. The noise signal at is detected, and the speaker's voice can be extracted based on the signal-to-noise ratio (SNR) of the detected noise signal.

Due to this characteristic, the first voice signal can be used for recognizing the speaker's utterance. When using the second voice signal corresponding to the idle period of the first voice signal, it is difficult to determine the intensity and type of noise in the current environment. It becomes possible.

After classifying the speaker's voice section and the idle section using the first voice signal, the processor 230 uses this to calculate the signal-to-noise ratio of the detected noise signal of the second voice signal according to Equation 1 below.

[Equation 1]

Here, S _rms and N _rms mean the root of the mean of the speech and noise signals, respectively.

On the other hand, in order to calculate the SNR from a single signal, it is necessary to accurately identify the noise section of the signal. In the case of a single signal, since voice and noise are mixed, it is often difficult to obtain an accurate voice and noise section.

However, in the case of an embodiment of the present invention, since a two-channel voice signal is used, an accurate voice section and a noise section can be distinguished, and accordingly, an accurate SNR can be calculated to determine the noise state of the signal.

The calculated SNR value can be classified into three ambient noise states. If the SNR value falls within the preset range (e.g. 5 to 15 dB), it is a noise voice, if it is less than the preset range (e.g. 0 to 5 dB), it is a strong noise voice, and exceeds the preset range (e.g. 15-20dB) can be distinguished by clear voice.

According to the classified method, the processor 230 may apply the weighted sum method according to Equation 2 for the first voice signal and the second voice signal to be used as an input value for final voice recognition.

[Equation 2]

Here, α is a weighting factor and has a value of 0≦α≦1. S _m1 is the first audio signal and s _m2 is the second audio signal.

The processor 230 sets the weight factor α to 0 when the signal-to-noise ratio of the noise signal exceeds a preset range, that is, is classified as a clear speech. Accordingly, the processor 230 may extract the speaker's voice included in the second voice signal and use it as an input value for final voice recognition.

Also, the processor 230 sets the weight factor α to 1 when the signal-to-noise ratio of the noise signal is less than a preset range, that is, classified as a strong noise speech. Accordingly, the processor 230 may extract the speaker's voice included in the first voice signal and use it as an input value for final voice recognition.

Since this first audio signal is acquired from the first microphone 120 included in the earbud 110, the sound quality is significantly lowered compared to the second audio signal acquired by the second microphone 130, which is an external microphone. Independent use of voice recognition may result in poor voice recognition performance.

Nevertheless, in a strong noise environment such as 0~5dB, voice recognition through the second voice signal is almost impossible. On the other hand, if the first voice signal is used, voice recognition is possible even if performance is slightly degraded. There is an advantage of securing a certain voice recognition performance.

In addition, when the signal-to-noise ratio of the noise signal falls within a preset range, the processor 230 may use the weight factor α as a weighted sum of the two signals in consideration of the SNR value excluding 0 and 1. In this case, the weight factor α may be set to be proportional to the SNR value in a preset range.

In order to simply weight the two voice signals and use them as an input value for final voice recognition, the samples of the two signals must be exactly matched in the voice section and the non-voice section. Since the two-channel audio signals according to an embodiment of the present invention are generated to match each other in units of samples, even when the two signals are weighted, the signals are exactly the same.

Through this feature, the processor 230 may analyze a section of the second voice signal corresponding to the idle section of the first voice signal and extract the speaker's voice from the weighted first and second voice signals.

In this case, when the noise signal includes one or more noises, the processor 230 may each generate an acoustic model corresponding to the type of noise.

In general, in speech recognition, speech recognition performance is optimized when the acoustic environment of the input signal and the acoustic environment of the voice recognition model match. Theoretically, if an acoustic model specialized for a plurality of acoustic environments is constructed and selected according to the acoustic environment of the input signal, a voice recognition system optimized for the acoustic environment becomes possible.

However, in order to know the acoustic environment of the input signal in advance, it is necessary to prioritize the discrimination of the correct voice section and the non-voice section, and if an error occurs in the discrimination, the voice recognition performance can be further degraded.

In addition, since the acoustic model cannot be changed during the speech recognition process once it is determined, only one acoustic model is used in the existing speech recognition system.

In contrast, the automatic interpretation device 200 according to an embodiment of the present invention improves the accuracy of the non-speech section of the input signal through the analysis of the two-channel voice signal so that the acoustic environment can be grasped, thus solving the problems as in the prior art. In addition, it has the characteristic that it is possible to select an acoustic model specialized for an acoustic environment.

For reference, the components shown in FIG. 5 according to an embodiment of the present invention may be implemented in software or in hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), and perform predetermined roles. can do.

However,'elements' are not meant to be limited to software or hardware, and each element may be configured to be in an addressable storage medium or configured to reproduce one or more processors.

Thus, as an example, components are components such as software components, object-oriented software components, class components and task components, processes, functions, properties, procedures, and sub- Includes routines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables.

Components and functions provided within the components can be combined into a smaller number of components or further separated into additional components.

Hereinafter, an automatic interpretation method using a 2-channel voice signal in the automatic interpretation device 200 according to an embodiment of the present invention will be described with reference to FIGS. 7 and 8.

7 is a flowchart of an automatic interpretation method according to an embodiment of the present invention. FIG. 8 is a flowchart for explaining contents of extracting a speaker's voice by applying all or selectively the first and second voice signals.

In the automatic interpretation method according to an embodiment of the present invention, first, when first and second audio signals converted into 1-channel signals are received from the earphone device 100 (S110), the 1-channel signal is converted into a 2-channel audio signal. It is decoded into the first and second voice signals (S120).

At this time, the first voice signal includes the speaker's voice received through the first microphone 120 located inside the earbud 110 of the earset device 100, and the second voice signal is external to the earbud 110 It includes a noise signal received through the second microphone 120 located at and the speaker's voice.

Next, the first and second voice signals are compared (S130), and all or selectively, all or all of the speaker's voices included in the first and second voice signals are extracted based on the comparison result (S140).

At this time, in the step of comparing the first and second voice signals, the speaker's voice section and the idle section are detected from the first voice signal, and a noise signal in the section of the second voice signal corresponding to the detected idle section is detected. .

Thereafter, the signal-to-noise ratio of the detected noise signal is calculated (S141) to extract the speaker's voice. Specifically, when it is determined whether or not the signal-to-noise ratio of the noise signal is included within a preset range (S142), the first and second voice signals are weighted (S144), and from the weighted first and second voice signals The speaker's voice can be extracted (S146).

On the other hand, if the signal-to-noise ratio of the noise signal exceeds a preset range, the speaker's voice included in the second voice signal is extracted (S143, S146), and if the signal-to-noise ratio of the noise signal is less than the preset range, the first voice signal The speaker's voice included in is extracted (S145, S146).

Next, automatic interpretation is performed on the extracted speaker's voice (S150).

In the above description, steps S110 to S150 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as necessary, or the order between steps may be changed. In addition, even if other contents are omitted, the contents already described with respect to the automatic interpretation system 1 and the device 200 in FIGS. 1 to 6 are also applied to the automatic interpretation method of FIGS. 7 and 8.

Although the methods and systems of the present invention have been described in connection with specific embodiments, some or all of their components or operations may be implemented using a computer system having a general-purpose hardware architecture.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

1: Automatic interpretation system
100: earset device
110: earbuds
120: first microphone
130: second microphone
140: Bluetooth chipset
150: battery
200: automatic interpretation device
210: communication module
220: memory
230: processor

Claims (20)

In the automatic interpretation device using a two-channel audio signal,
A communication module that transmits and receives data to and from an earphone device including a speaker, first and second microphones, the first microphone is located inside the earbud of the earset device, the second microphone is located outside the earbud, and the speaker Is located within a preset distance from the mouth of
A memory storing a program for generating an interpretation result using the two-channel audio signal, and
Including a processor for executing a program stored in the memory,
As the processor executes the program, the first voice signal, which is the voice of the speaker received through the first microphone, is received through the second microphone, and the second voice signal including the noise signal and the voice of the speaker And, based on the comparison result, automatic interpretation is performed by extracting all or selectively the speaker's voices included in the first and second voice signals,
The processor detects a voice section and a pause section of the speaker from the first voice signal, detects a noise signal in a section corresponding to the detected idle section in the second voice signal, and detects the detected noise signal. An automatic interpretation device for extracting the speaker's speech from at least one of the first speech signal and the second speech signal based on a signal-to-noise ratio. delete The method of claim 1,
The first audio signal and the second audio signal are converted into a 1-channel signal in the earphone device and received through the communication module,
Wherein the processor decodes the one-channel signal and extracts the two-channel voice signals into the first and second voice signals. delete The method of claim 1,
When the signal-to-noise ratio of the noise signal falls within a preset range, the processor weights the first and second voice signals, and extracts the speaker's voice from the weighted first and second voice signals. Automatic interpretation device. The method of claim 5,
When the noise signal includes a plurality of noises, the processor generates each acoustic model corresponding to the type of noise, and performs the automatic interpretation through the acoustic model. The method of claim 1,
Wherein the processor extracts the speaker's voice included in the second voice signal and performs the automatic interpretation when the signal-to-noise ratio of the noise signal exceeds a preset range. The method of claim 1,
Wherein the processor extracts the speaker's voice included in the first voice signal and performs the automatic interpretation when the signal-to-noise ratio of the noise signal is less than a preset range. In the automatic interpretation method using a 2-channel audio signal in an automatic interpretation device,
Receiving first and second audio signals converted into 1-channel signals from the earphone device;
Decoding the 1-channel signal into the first and second audio signals, which are 2-channel audio signals;
Comparing the first and second voice signals;
Extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result, and
Including the step of performing automatic interpretation for the extracted speaker's voice,
The first voice signal includes a speaker's voice received through a first microphone inside the earbud of the earbud device, and the second voice signal is a noise signal received through a second microphone outside the earbud and the Contains the speaker's voice,
The extracting comprises detecting a voice section and a pause section of the speaker from the first voice signal, detecting a noise signal in a section corresponding to the detected idle section from the second voice signal, and the detected And extracting the speaker's voice from at least one of the first voice signal and the second voice signal based on the signal-to-noise ratio of the noise signal. delete delete delete The method of claim 9,
Extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result,
Determining whether the signal-to-noise ratio of the noise signal is within a preset range;
If the determination result falls within the preset range, weighting the first and second voice signals and
And extracting the speaker's voice from the weighted first and second voice signals. The method of claim 13,
When the noise signal contains a plurality of noises, the automatic interpretation method further comprising generating an acoustic model corresponding to the type of the noise. The method of claim 9,
Extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result,
Determining whether the signal-to-noise ratio of the noise signal is within a preset range, and
And extracting a speaker's voice included in the second voice signal when the determination result exceeds the preset range. The method of claim 9,
Extracting all or selectively the speaker's voices included in the first and second voice signals based on the comparison result,
Determining whether the signal-to-noise ratio of the noise signal is within a preset range, and
And extracting a speaker's voice included in the first voice signal when the determination result is less than the preset range. In the automatic interpretation system using a two-channel audio signal,
The first voice signal, which is the speaker's voice, is received through a first microphone located inside the earbud, and a second voice including a noise signal and the speaker's voice through a second microphone located outside the earbud An earphone device for receiving a signal, converting the first and second audio signals into a 1-channel signal and transmitting the same; And
The first and second voice signals, which are two-channel voice signals, are extracted by receiving the one-channel signal, and included in the first and second voice signals based on a result of comparing the first and second voice signals. Including an automatic interpretation device for performing automatic interpretation by extracting all or selectively the voice of the speaker,
The automatic interpretation device detects a voice section and a pause section of the speaker from the first voice signal, detects a noise signal in a section corresponding to the detected idle section from the second voice signal, and the detected An automatic interpretation system for extracting the speaker's voice from at least one of the first voice signal and the second voice signal based on a signal-to-noise ratio of a noise signal. delete The method of claim 17,
When the signal-to-noise ratio of the noise signal falls within a preset range, the automatic interpretation device weights the first and second voice signals and extracts the speaker's voice from the weighted first and second voice signals. Automatic interpretation system. The method of claim 17,
When the signal-to-noise ratio of the noise signal exceeds a preset range, the automatic interpretation device extracts the speaker's voice included in the second voice signal and performs the automatic interpretation,
When the signal-to-noise ratio of the noise signal is less than a preset range, the automatic interpretation system is performed by extracting the speaker's voice included in the first voice signal.

Images