KR102628500B1 - Apparatus for face-to-face recording and method for using the same - Google Patents

Abstract

The present invention relates to a face-to-face recording terminal device and a face-to-face recording method using the same. The face-to-face recording terminal device according to an embodiment of the present invention is provided between two speakers, faces the first speaker, and receives a first microphone signal. a first microphone array that generates; a second microphone array facing toward the second speaker and generating a second microphone signal; and a speaker that outputs a preset audio signal, from the first microphone signal or the second microphone signal, a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker. It may include a multi-channel signal processor that separates an audio signal and a third audio signal corresponding to the audio signal output from the speaker.

Description

Face-to-face recording terminal device and face-to-face recording method using the same {Apparatus for face-to-face recording and method for using the same}

The present invention relates to a face-to-face recording terminal device that can separate the voices of each speaker during recording and a face-to-face recording method using the same.

Recently, when selling financial products, incomplete sales, such as selling risky financial products without explaining all the information, have become a problem. As a way to prevent this, a method has been proposed to record the consultation details between the counselor and the customer and then convert them into text and monitor them. However, in this case, for accurate monitoring, it is necessary to separate multiple speakers included in the recording. .

Conventionally, in order to separate multiple speakers, a method of training a speaker model in advance and registering the voice of a specific speaker was used. Specifically, feature extraction methods such as Cepstrum, LPC (Linear Predictive Coding), MFCC (Mel-Frequency Cepstrum Coefficients), and PLP (Perceptual Linear Predictive Analysis) are used to extract voice features of a specific speaker, Speakers were distinguished by comparing them.

However, in the case of the conventional method, speakers must be registered in advance and speakers separated based on the voice characteristics of the registered speakers, so not only is it impossible to pre-register and learn the voices of customers that change every day, but it is also necessary to keep the counselor and customer connected. There are difficulties in separating and processing in real time.

Korean Patent Publication No. 10-1750338

The purpose of the present invention is to provide a face-to-face recording terminal device that can accurately separate the voices of each speaker included in the recording and the audio signal played from the speaker, and a face-to-face recording method using the same.

The present invention is a face-to-face recording terminal device that can accurately separate the voices of each speaker included in the recording and the audio signal played from the speaker and generate STT (Speech-to-Text) data for determining incomplete sales. The purpose is to provide a face-to-face recording method using this.

The present invention is intended to provide a face-to-face recording terminal device that can specify the voices of each speaker included in a recording by adjusting the characteristic parameters of the microphone signal along with beam forming, and a face-to-face recording method using the same.

The purpose of the present invention is to provide a face-to-face recording terminal device that can effectively remove noise contained in a recording and a face-to-face recording method using the same.

A face-to-face recording terminal device according to an embodiment of the present invention is provided between two speakers, and includes a first microphone array facing the first speaker and generating a first microphone signal; a second microphone array facing toward the second speaker and generating a second microphone signal; and a speaker that outputs a preset audio signal, from the first microphone signal or the second microphone signal, a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker. It may include a multi-channel signal processor that separates an audio signal and a third audio signal corresponding to the audio signal output from the speaker.

Here, the multi-channel signal processing unit applies DOA (Direction of Arrival) masking to the first microphone signal based on the setting angle set corresponding to the position of the first speaker, and the DOA-masked first speaker By performing preprocessing on the microphone signal, the first voice signal can be extracted from the first microphone signal.

Here, the multi-channel signal processing unit preprocesses the first microphone signal using at least one of Dynamic Range Control (DRC), Audio Equalizer (EQ), Noise Reduction (NR), and Automatic Gain Control (AGC). It can be done.

Here, the multi-channel signal processing unit uses the DRC to attenuate signals whose size is less than a threshold among the DOA-masked first microphone signals and amplify signals whose size is greater than the threshold to separate the first voice signal from noise. can do.

Here, the multi-channel signal processing unit may limit the input gain of the received first microphone signal or the second microphone signal to less than a threshold value.

Here, the multi-channel signal processing unit may apply VAD (Voice Activity Detection) to the first microphone signal or the second microphone signal to remove non-voice noise included in the first microphone signal.

Here, the first microphone array and the second microphone array are provided in an internal sealing part including a sound insulation member, and can block the audio signal output from the speaker from flowing into the face-to-face recording terminal device.

Here, the multi-channel signal processor may apply VAD to the first microphone signal to distinguish the audio signal output from the speaker and remove it from the first microphone signal.

Here, the multi-channel signal processing unit may remove the audio signal from the second microphone signal by applying Acoustic Echo Cancellation (AEC), which uses the audio signal as an echo reference signal.

Here, the multi-channel signal processing unit can transmit the echo reference to the server to perform Speech-to-Text (STT).

The face-to-face recording method according to an embodiment of the present invention relates to a face-to-face recording method using a face-to-face recording terminal device including a first microphone array, a second microphone array, and a speaker, wherein the first microphone array is directed toward the first speaker. generating a first microphone signal, a second microphone array facing toward the second speaker generating a second microphone signal, and the speaker outputting a preset audio signal; and from the first microphone signal or the second microphone signal, a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker, and an audio signal output from the speaker. It may include the step of separating the third voice signal.

Additionally, the means for solving the above problems do not enumerate all the features of the present invention. The various features of the present invention and the resulting advantages and effects can be understood in more detail by referring to the specific embodiments below.

According to the face-to-face recording terminal device and the face-to-face recording method using the same according to an embodiment of the present invention, it is possible to accurately separate the voices of each speaker included in the recording and the audio signal played from the speaker.

According to the face-to-face recording terminal device and the face-to-face recording method using the same according to an embodiment of the present invention, it is possible to effectively remove noise included in the recording.

According to the face-to-face recording terminal device and the face-to-face recording method using the same according to an embodiment of the present invention, the voices of each speaker included in the recording are accurately separated from the audio signal played from the speaker, and STT is used to determine whether or not there is an incomplete sale. (Speech-to-Text) data can be generated, through which it is possible to monitor for incomplete sales.

Figure 1 is a schematic diagram showing a face-to-face recording system according to an embodiment of the present invention.
Figure 2 is a block diagram showing a face-to-face recording terminal device according to an embodiment of the present invention.
Figure 3 is a schematic diagram showing a face-to-face recording terminal device according to an embodiment of the present invention.
Figure 4 is an exemplary diagram showing channel separation performance when each speaker speaks using a face-to-face recording terminal device according to an embodiment of the present invention.
Figure 5 is an example diagram showing channel separation performance when a speaker and a speaker speak simultaneously using a face-to-face recording terminal device according to an embodiment of the present invention.
Figure 6 is an exemplary diagram showing channel separation performance when speakers simultaneously speak using a face-to-face recording terminal device according to an embodiment of the present invention.
Figure 7 is a flowchart showing a face-to-face recording method using a face-to-face recording terminal device according to an embodiment of the present invention.

Hereinafter, with reference to the attached drawings, preferred embodiments will be described in detail so that those skilled in the art can easily practice the present invention. However, when describing preferred embodiments of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. In addition, the same symbols are used throughout the drawings for parts that perform similar functions and actions.

In addition, throughout the specification, when a part is said to be 'connected' to another part, this does not only mean 'directly connected', but also 'indirectly connected' with another element in between. Includes. In addition, 'including' a certain component means that other components may be further included rather than excluding other components, unless specifically stated to the contrary. Additionally, terms such as “unit” and “module” used in the specification refer to a unit that processes at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software.

Figure 1 is a schematic diagram showing a face-to-face recording system according to an embodiment of the present invention.

Referring to Figure 1, a face-to-face recording system according to an embodiment of the present invention may include a face-to-face recording terminal device 100 and a server 200.

Hereinafter, a face-to-face recording system according to an embodiment of the present invention will be described with reference to FIG. 1.

The face-to-face recording terminal device 100 can be provided between two speakers 1 and 2, and records the conversation between the speakers 1 and 2 using the microphone array 110 and 120, or records a preset audio signal, etc. It can be output through the speaker 130. Here, the face-to-face recording terminal device 100 can be used during consultations between customers and counselors at banks, securities companies, insurance companies, etc., and the recorded consultation details can be used to prevent incomplete sales of financial products. In other words, the face-to-face recording terminal device 100 can output essential information for the complete sale of a financial product and provide it to the customer through the speaker 130, so that the customer can confirm that these information has been informed. A recording of the consultation may be provided.

The server 200 can receive recordings from the face-to-face recording terminal device 100 and can perform STT (Speech-To-Text) on the recordings. In other words, the server 200 can automatically verify whether there is an incomplete sale of a financial product by monitoring whether there is a complete sale based on the text generated through STT.

However, when the server 200 converts a recording into text to check for incomplete sales, there is a problem in that it is difficult to perform an accurate STT if the voices of each speaker in the recording are not separated. In particular, when information on financial products is output through the speaker 130, the content output from the speaker 130 may also be input to each microphone array 110 and 120, making it more difficult to distinguish between speakers. there is.

On the other hand, according to the face-to-face recording terminal device 100 according to an embodiment of the present invention, accurate speaker separation and noise removal can be performed through beam forming and preprocessing for speaker separation, and thus high quality. It is possible to provide recordings for STT recognition. In addition, the face-to-face recording terminal device 100 according to an embodiment of the present invention records only the voices of the counselor and the customer, the terms and conditions explanation TTS (Text-to-Speech) output from the speaker 130, and various surrounding noises, etc. It can provide a noise canceling function that removes all noise, and it is possible to distinguish the voices of the agent and the customer in real time while recording. Hereinafter, a face-to-face recording terminal device 100 according to an embodiment of the present invention will be described.

Figures 2 and 3 are block diagrams showing a face-to-face recording terminal device according to an embodiment of the present invention. Referring to Figures 2 and 3, the face-to-face recording terminal device 100 according to an embodiment of the present invention includes a first microphone array 110, a second microphone array 120, a speaker 130, and a multi-channel signal. It may include a processing unit 140.

The first microphone array 110 may be installed toward the first speaker 1, and may receive a voice spoken by the first speaker 1 and generate a first microphone signal. Here, a plurality of microphones may be provided in the first microphone array 110 for beam forming, and depending on the embodiment, as shown in FIG. 2, a left first microphone 110a and a right first microphone ( 110b) two microphones may be included. However, it is not limited to this, and the number and arrangement of microphones included in the first microphone array 110 can be changed in various ways depending on the embodiment. Here, the first speaker 1 may be a counselor selling financial products, etc.

The second microphone array 120 may be installed toward the second speaker 2, and may receive a voice spoken by the second speaker 2 and generate a second microphone signal. Here, a plurality of microphones may be provided in the second microphone array 120 for beam forming, and depending on the embodiment, as shown in FIG. 2, a left second microphone 120a and a right second microphone ( 120b) two microphones may be included. However, it is not limited to this, and the number and arrangement of microphones included in the second microphone array 120 can be changed in various ways depending on the embodiment. Here, the second speaker 2 may be a customer purchasing a financial product, etc.

Here, when the first microphone array 110 and the second microphone array 120 are arranged as shown in FIG. 2, the face-to-face recording terminal device 100 is used to connect the first speaker 1 and the second speaker 2. It can be placed on the side to perform face-to-face recording.

The speaker 130 can output a preset audio signal. The speaker 130 can be installed on the side of the second speaker 2 corresponding to the customer, and the audio signal output from the speaker 130 contains information essential for complete sales, such as an explanation of the terms and conditions for the financial product the counselor wants to sell. Matters may be included. Here, the audio signal may be a TTS signal that converts text, such as a description of the terms and conditions of a financial product, into voice.

Meanwhile, depending on the embodiment, the first microphone array 110 and the second microphone array 120 may each be provided within an internal sealing portion (not shown). A sound insulation member may be included in the internal sealing part (not shown), through which the audio signal output from the speaker 130 passes through the inside of the face-to-face recording terminal device 100 and the first microphone array 110 and the second microphone. Inflow into the array 120 can be blocked.

The multi-channel signal processor 140 receives a first voice signal corresponding to the voice of the first speaker 1 and a second voice signal corresponding to the voice of the second speaker 2 from the first microphone signal or the second microphone signal. and a third voice signal corresponding to the audio signal output from the speaker 130 can be separated. That is, by using the phase or size difference measured in the plurality of first microphone arrays 110a and 110b, the angle of incidence (DOA: Direction of Arrival) of each sound signal input to the first microphone array 110 can be obtained. The multi-channel signal processing unit 140 can be implemented to receive only sound signals incident at a preset angle of incidence corresponding to the position of the first speaker 1. At this time, assuming the location of the first speaker 1, the DOA of the first microphone array 110 can be designed in advance, and the DOA of the second microphone array 120 can be set in the same manner.

Thereafter, the multi-channel signal processing unit 140 may utilize each DOA to separate the first and second voice signals corresponding to the voices of the first speaker 1 and the second speaker 2. . That is, based on the respective set angles (α1, α2) and set angle ranges (β1, β2) set corresponding to the positions of the first speaker (1) and the second speaker (2), the first microphone signal and the second speaker (2) DOA masking can be applied to the microphone signal, and through this, among the various sound signals input to the first microphone array 110 and the second microphone array 120, the set angle (α1, α2) and the set angle range ( Except for the acoustic signals incident within β1, β2), the remaining acoustic signals can be treated as noise.

After performing DOA masking, the multi-channel signal processing unit 140 can perform preprocessing on the DOA-masked first microphone signal, and through this, extract the first voice signal from the first microphone signal. At this time, the multi-channel signal processing unit 140 performs pre-processing on the first microphone signal using Dynamic Range Control (DRC), Audio Equalizer (EQ), Noise Reduction (NR), and Automatic Gain control (AGC). You can. That is, since it may be difficult to extract only the voice of the first speaker 1 using DOA masking alone, the first voice signal can be extracted by performing additional preprocessing for speaker separation on the first microphone signal. Here, preprocessing can be performed in the same way on the second microphone signal. Through this, each of the first microphone array 110 and the second microphone array 120 collects only the voices of the first speaker 1 and the second speaker 2 speaking right in front, respectively, and produces the first voice signal and It is possible to generate a second voice signal. In other words, the phenomenon where the voices of the customer and the agent are mixed and not separated can be minimized.

Here, the multi-channel signal processing unit 140 can set parameters for optimal speaker separation preprocessing.

First, the multi-channel signal processing unit 140 can set the setting angle and setting angle range for DOA masking. At this time, the setting can be made with reference to the reverberation effect of the indoor environment where the face-to-face recording terminal device 100 is installed. The reverberation effect may vary depending on the indoor environment, and the speaker separation effect due to DOA masking may appear differently depending on the reverberation effect. Therefore, after measuring the reverberation effect for each indoor environment, the setting angle and setting angle range for DOA masking can be appropriately set. Depending on the embodiment, the reverberation effect may be measured using RT (Reverberation time) 60.

Afterwards, the multi-channel signal processing unit 140 can perform settings for DRC. When DOA masking is applied to the first microphone signal, the first speaker's voice signal is generally input relatively loudly and the remaining noise occurring around is input small. Therefore, in cases where speaker separation is not sufficient through DOA alone, DRC can be used to attenuate signals whose size is less than the threshold and amplify signals whose size is greater than the threshold among the DOA-masked first microphone signals. Through this, the first voice signal and noise can be sufficiently separated (for example, 10 dB or more). Here, the amplified signal can be compensated by reducing the output gain at the final stage of preprocessing.

Meanwhile, problems such as noise amplification may occur when applying DRC due to indoor reverberation effects, actual people's sitting positions, and people's movements when speaking. In this case, it is possible to increase the speaker separation effect by minimizing the attack time of DRC.

Additionally, if sufficient speaker separation is not performed even when adjusting the setting angle and setting angle range of the DOA or the threshold or operation start time of the DRC, the input gain of the first microphone signal or the second microphone signal is adjusted. ) can be adjusted. In other words, if the size of the input gain is limited to less than the threshold, the size of the noise input through DOA masking is reduced, making it possible to improve separation performance through DOA masking. However, the size of the input gain can be limited to at least the minimum input size that can be recognized by the automatic speech recognition (ASR) engine.

Meanwhile, depending on the embodiment, while a customer and a counselor are conducting a consultation, there may be cases where voices or noises from a consultation at a neighboring counter, voices or noises from other customers waiting in the hall, etc. may flow in. In order to remove such noise, the multi-channel signal processing unit 140 can distinguish each noise into speech noise and non-speech noise and remove them.

Specifically, the multi-channel signal processing unit 140 may apply Voice Activity Detection (VAD) to the first microphone signal. VAD is an algorithm that distinguishes voice signals and non-voice signals based on DNN (Deep Neural Network). Using VAD, it is possible to detect voice included in the first microphone signal. Accordingly, the multi-channel signal processing unit 140 can apply VAD to detect non-speech noise included in the first microphone signal and remove the detected non-speech noise from the first microphone signal. Voice noise can be removed by applying DRC or performing preprocessing to control EQ, gain, etc. for the first microphone signal.

According to another embodiment, audio signals reproduced from the speaker 130 may flow into the first microphone 110 and the second microphone 120, respectively. That is, when a TTS audio signal for explanation of terms and conditions, etc. is output from the speaker 130, the corresponding audio signal is included in the first microphone signal and the second microphone signal, so that each of the first voice signal and the second voice signal can be accurately It can be difficult to separate. In this case, the multi-channel signal processing unit 140 may apply different algorithms to the first microphone array 110 and the second microphone array 120 to remove the audio signal.

Specifically, in the case of the first microphone array 110, the audio signal can be primarily removed from the first microphone signal using DOA masking. Thereafter, the audio signal included in the first microphone signal can be distinguished by applying VAD, and the differentiated audio signal can be removed from the first microphone signal. Additionally, preprocessing such as DRC may be performed to generate a first voice signal from which the audio signal has been removed.

In the case of the second microphone array 120, audio signals can be removed using Acoustic Echo Cancellation (AEC). The audio signal output from the speaker 130 may be stored in the face-to-face recording terminal device 100 or may be provided from an external server 200, etc. Accordingly, the multi-channel signal processing unit 140 can extract the audio signal to be output from the speaker 130 and apply AEC by setting it as an echo reference. Thereafter, the multi-channel signal processing unit 140 removes the echo reference from the second microphone signal received from the second microphone array 120 using AEC, thereby removing the voice of the second speaker 2 included in the second microphone signal. Noise from audio signals other than those can be removed.

Additionally, the multi-channel signal processing unit 140 may transmit the first voice signal, the second voice signal, and the echo reference to the server 200, and the server 200 may transmit the first voice signal using a voice recognition engine (not shown). STT can be performed on the signal, the second voice signal, and the echo reference. In other words, a transcript can be created so that it can be confirmed that the contents of the consultation between the customer and the counselor and the guidance output through the speaker 130 were actually heard and explained to the customer. Afterwards, the server 200 can monitor whether there is an incomplete sale based on each recording. At this time, the multi-channel signal processing unit 140 can perform speaker separation in a simple manner instead of the existing complex speaker separation (source separation) algorithm, so each first voice signal, second voice signal, and echo reference are analyzed in real time. It is possible to provide it to the server 200, and the server 200 can monitor incomplete sales in real time based on this.

Meanwhile, depending on the embodiment, it is possible to pre-register one of the plurality of speakers (1, 2) through deep learning-based feature extraction and perform DOA-based speaker separation for the remaining speakers. . For example, in the case of a bank, since the agent is fixed and the customer changes, the voice characteristics of the agent are registered to accurately distinguish the agent's voice signal, and for the customer, voice signals are extracted based on DOA to provide more accurate results. It is also possible to perform voice separation.

Figures 4 to 6 are exemplary diagrams showing channel separation performance using a face-to-face recording terminal device according to an embodiment of the present invention.

Referring to Figure 4, when using a face-to-face recording terminal device according to an embodiment of the present invention, it can be seen that channel separation performance is implemented with a 23dB difference when an employee (first speaker) and a customer (second speaker) each speak. You can. Figure 5 shows a case where the customer and the speaker speak at the same time. It can be seen that even when the customer and the speaker speak at the same time, a channel separation performance of 42dB is implemented. In addition, Figure 6 shows a case where a customer and an employee speak simultaneously, and it is possible to implement a channel separation performance of at least 17 dB even when a customer and an employee speak simultaneously. Therefore, it can be confirmed that the voices of customers and employees can be clearly separated even when speaking simultaneously.

Figure 7 is a flowchart showing a face-to-face recording method using a face-to-face recording terminal device according to an embodiment of the present invention. Here, the face-to-face recording terminal device may include a first microphone, a second microphone, and a speaker.

Referring to Figure 7, the first microphone array facing the first speaker generates the first microphone signal, the second microphone array facing the second speaker generates the second microphone signal, and the speaker generates the preset audio signal. can be output (S10). Here, the first microphone array and the second microphone array may each include a plurality of microphones, and the speaker transmits TTS as an audio signal, including information essential for complete sales, such as an explanation of the terms and conditions for the financial product the agent wants to sell. Can be printed. Here, depending on the embodiment, the first microphone array and the second microphone array may each be provided within the inner sealing portion. A sound insulating member may be included in the inner sealing part, and the sound insulating member can block the audio signal output from the speaker from flowing into the first microphone array and the second microphone array through the inside of the face-to-face recording terminal device.

Thereafter, the face-to-face recording terminal device receives a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker, and an audio signal output from the speaker from the first microphone signal or the second microphone signal. The third voice signal corresponding to can be separated (S20). In other words, the face-to-face recording terminal device can apply DOA masking to the first microphone signal and the second microphone signal based on the preset angle and angle range corresponding to the positions of each speaker, and through this, the first microphone signal Among the various sound signals input to the array and the second microphone array, the remaining sound signals except for the sound signals incident within the set angle range can be processed as noise. Additionally, after performing DOA masking, preprocessing can be performed on the DOA masked first microphone signal, and through this, the first voice signal can be extracted from the first microphone signal. At this time, the face-to-face recording terminal device may perform preprocessing on the first microphone signal using Dynamic Range Control (DRC), Audio Equalizer (EQ), Noise Reduction (NR), and Automatic Gain Control (AGC). Here, preprocessing can be performed in the same way on the second microphone signal.

Meanwhile, depending on the embodiment, while a customer and a counselor are conducting a consultation, there may be cases where voices or noises from a consultation at a neighboring counter, voices or noises from other customers waiting in the hall, etc. may flow in. In order to remove such noise, each noise can be removed by distinguishing it into speech noise and non-speech noise. Specifically, the face-to-face recording terminal device may apply VAD (Voice Activity Detection) to the first microphone signal. Using VAD, it is possible to detect voice included in the first microphone signal. Therefore, by applying VAD, non-speech noise included in the first microphone signal can be detected and the detected non-speech noises can be removed from the first microphone signal. Additionally, in the case of voice noise, it can be removed by applying DRC to remove voice noise, etc., or by performing preprocessing to control EQ, gain, etc. for the first microphone signal.

According to another embodiment, audio signals reproduced from a speaker may flow into the first microphone array and the second microphone array, respectively. In this case, the face-to-face recording terminal device can apply different algorithms to the first microphone array and the second microphone array to remove the audio signal.

Specifically, in the case of the first microphone array, the audio signal can be primarily removed from the first microphone signal using DOA masking. Thereafter, the audio signal included in the first microphone signal can be distinguished by applying VAD, and the differentiated audio signal can be removed from the first microphone signal. Additionally, preprocessing such as DRC may be performed to generate a first voice signal from which the audio signal has been removed.

In the case of the second microphone array, the audio signal can be removed using AEC (Acoustic Echo Cancellation). In other words, the face-to-face recording terminal device can extract the audio signal to be output from the speaker and apply AEC by setting it as an echo reference. In this case, by removing the echo reference from the second microphone signal received from the second microphone array using AEC, noise such as an audio signal other than the second speaker's voice included in the second microphone signal can be removed.

Additionally, the face-to-face recording terminal device can transmit the first voice signal, the second voice signal, and the echo reference to the server, and the server can perform STT on the first voice signal, the second voice signal, and the echo reference. In other words, a transcript can be created so that it can be confirmed that the contents of the consultation between the customer and the counselor and the guidance output through the speaker were actually heard and explained to the customer. Afterwards, the server can monitor incomplete sales based on each transcript.

The above-described present invention can be implemented as computer-readable code on a program-recorded medium. A computer-readable medium may continuously store a computer-executable program or temporarily store it for execution or download. In addition, the medium may be a variety of recording or storage means in the form of a single or several pieces of hardware combined. It is not limited to a medium directly connected to a computer system and may be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, And there may be something configured to store program instructions, including ROM, RAM, flash memory, etc. Additionally, examples of other media include recording or storage media managed by app stores that distribute applications, sites or servers that supply or distribute various other software, etc. Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

The present invention is not limited to the above-described embodiments and attached drawings. For those skilled in the art to which the present invention pertains, it will be clear that components according to the present invention can be replaced, modified, and changed without departing from the technical spirit of the present invention.

100: Face-to-face recording terminal device 110: First microphone array
120: second microphone array 130: speaker
140: Multi-channel signal processing unit

Claims (11)

In a face-to-face recording terminal device provided between two speakers,
a first microphone array facing toward the first speaker and generating a first microphone signal;
a second microphone array facing toward the second speaker and generating a second microphone signal; and
Facing the second speaker, it includes a speaker that outputs a preset audio signal,
From the first microphone signal or the second microphone signal, a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker, and an audio signal corresponding to the audio signal output from the speaker. It includes a multi-channel signal processing unit that separates the third voice signal,
The multi-channel signal processing unit
Transmit the third voice signal to the server to perform STT (Speech-to-Text),
The multi-channel signal processing unit
By applying different algorithms to the first microphone signal and the second microphone signal, the audio signal included in the first microphone signal and the second microphone signal is removed,
Apply VAD (Voice Activity Detection) to the first microphone signal to distinguish the audio signal output from the speaker and remove it from the first microphone signal,
A face-to-face recording terminal device that removes the audio signal from the second microphone signal by applying AEC (Acoustic Echo Cancellation) that uses the audio signal as an echo reference signal.
The method of claim 1, wherein the multi-channel signal processing unit
Direction of Arrival (DOA) masking is applied to the first microphone signal based on the setting angle set corresponding to the position of the first speaker, and preprocessing is performed on the DOA-masked first microphone signal. , A face-to-face recording terminal device, characterized in that extracting the first voice signal from the first microphone signal.
The method of claim 2, wherein the multi-channel signal processing unit
Face-to-face recording, characterized in that preprocessing is performed on the first microphone signal using at least one of DRC (Dynamic Range Control), EQ (Audio Equalizer), NR (Noise Reduction), and AGC (Automatic Gain control). Terminal device.
The method of claim 3, wherein the multi-channel signal processing unit
A face-to-face recording terminal that uses the DRC to attenuate signals whose size is less than a threshold among the DOA-masked first microphone signals and amplify signals whose size is greater than the threshold to separate the first voice signal from noise. Device.
The method of claim 3, wherein the multi-channel signal processing unit
A face-to-face recording terminal device characterized in that the input gain of the received first microphone signal or the second microphone signal is limited to less than a threshold value.
The method of claim 1, wherein the multi-channel signal processing unit
A face-to-face recording terminal device, characterized in that non-voice noise included in the first microphone signal is removed by applying VAD (Voice Activity Detection) to the first microphone signal or the second microphone signal.
The method of claim 1, wherein the first microphone array and the second microphone array are
A face-to-face recording terminal device provided in an internal sealing portion including a sound insulation member to block audio signals output from the speaker from flowing into the inside of the face-to-face recording terminal device.
delete delete The method of claim 1, wherein the multi-channel signal processing unit
A face-to-face recording terminal device characterized in that it transmits the echo reference to a server to perform Speech-to-Text (STT).
In a face-to-face recording method using a face-to-face recording terminal device including a first microphone array, a second microphone array, and a speaker,
A first microphone array facing the first speaker generates a first microphone signal, a second microphone array facing the second speaker generates a second microphone signal, and a speaker facing the second speaker generates a preset audio signal. outputting a signal; and
From the first microphone signal or the second microphone signal, a first voice signal corresponding to the voice of the first speaker, a second voice signal corresponding to the voice of the second speaker, and an audio signal output from the speaker Including the step of separating the third voice signal,
The separation step is
Transmitting the third voice signal to the server to perform STT (Speech-to-Text),
The separation step is
By applying different algorithms to the first microphone signal and the second microphone signal, the audio signal included in the first microphone signal and the second microphone signal is removed,
Apply VAD (Voice Activity Detection) to the first microphone signal to distinguish the audio signal output from the speaker and remove it from the first microphone signal,
A face-to-face recording method in which the audio signal is removed from the second microphone signal by applying AEC (Acoustic Echo Cancellation), which uses the audio signal as an echo reference signal.

Images