JP7035686B2 - Remote calling devices, remote calling programs, and remote calling methods - Google Patents

Description

The present invention relates to a remote call device, a remote call program, and a remote call method, and can be applied to, for example, a call start processing used in a video conference system, a conference call system, or the like.

In recent years, there have been increasing opportunities to make calls and communicate with remote locations such as video conferences and telework using remote call systems such as video conference systems and telephone conference systems.

In a remote call system, in order to make a call with a remote call partner, enter the contact information such as the call partner's phone number with an input device (for example, mouse, keyboard, remote control, etc.) connected to the remote call system. , Select and connect. Or, in recent years, with the spread of mobile terminals (for example, smartphones and tablet PCs), remote calling systems operate on mobile terminals, and contacts are entered on the screen of the mobile terminal using the keyboard displayed on the screen to connect. In many cases, you can connect to a remote location by touching the contact displayed on the touch panel display of the mobile terminal to select and connect, or by touching the image of the other party displayed on the screen to connect.

Further, Patent Document 1 proposes a communication support robot system that incorporates a remote communication system into a robot and supports communication between a close relative and a single elderly person.

The communication support robot system described in Patent Document 1 is connected to a call partner and starts a call by touching an image of a close relative or an elderly person displayed on a touch panel display.

Japanese Unexamined Patent Publication No. 2015-184597

However, in the communication support robot system described in Patent Document 1, the contact of the other party is input and connected by the input device in the same manner as the conventional remote call system to start and end the call, and the contact displayed on the touch panel display. Touch to connect and start a call. Connecting by the conventional connection method is different from the actual face-to-face call, so the sense of presence (the feeling of having a face-to-face conversation) is very low.

Also, using the voice recognition system installed in the communication support robot, for example, the name of the other party to connect to or a command to start a conversation (for example, "Hello", "Excuse me", etc.), etc. Even if the calling voice (hereinafter referred to as "calling voice") is spoken, the word is input to the voice recognition system, the connection destination is determined from the voice recognition result, and the connection can be started, the calling voice is still used. , Since the call partner is determined from the result of voice recognition after being input to the voice recognition process and connected to the call partner, the call voice is not transmitted to the call partner. For this reason, the call partner is suddenly connected, and the call partner feels a sense of discomfort or anxiety, and the sense of presence does not improve.

Furthermore, when the monitor on which the image of the other party of the remote call system is displayed becomes a large screen, the microphone is often installed near the monitor so as not to interfere with the use. In addition, since the user talks in a position where the entire screen can be seen, the distance between the user and the microphone is increased. Therefore, when there are people around and talking, or when the sound of the air conditioner or air conditioner is loud, the noise of the usage environment is large, and it becomes difficult to hear the user's voice and it is not possible to talk.

Therefore, even if the usage environment is noisy, you can make a call voice, or even if you are talking to the other party, you can connect to the other party with the call voice and start a call, just like when you make an actual face-to-face call. However, to the other party, the call is sent to the other party and then the call is started, and the voice to disconnect such as a command to end the conversation (for example, "Goodbye", "Excuse me", etc.) (hereinafter , Called "disconnect voice"), a remote call device, a remote call program, and a remote call method are desired, in which the call is terminated and the call is disconnected.

The first aspect of the present invention is (1) a signal processing unit that performs predetermined signal processing using input signals of a plurality of microphones that collect audio, and (2) a processing signal output from the signal processing unit. Whether or not the processed signal is a connection command voice using the results of the audio buffer unit held for a certain period, (3) the voice recognition unit that performs voice recognition for the processed signal, and (4) the voice recognition unit. A command determination unit that determines whether or not the connection destination is determined when the processing signal is determined to be the connection command voice by the command determination unit, and (6) usually the signal processing unit . The processing signal output from is output to the network communication unit that transmits to the remote call partner, and if the process signal is determined to be the connection command voice by the command determination unit , after connecting to the call partner terminal, After the processing signal held in the audio buffer section is output to the network communication section and the processing signal held in the audio buffer section is output, the processing signal output from the signal processing section is output again . It is characterized by having an output switching unit that switches to output to the network communication unit .

The second remote call program of the present invention comprises a computer, (1) a signal processing unit that performs predetermined signal processing using input signals of a plurality of microphones that pick up voice, and (2) the above signal processing unit. Using the results of the audio buffer unit that holds the processed signal output from the above for a certain period of time, (3) the voice recognition unit that performs voice recognition for the processed signal, and (4) the voice recognition unit, the processing signal. However, a command determination unit that determines whether or not the connection command voice is used, and (5) a connection determination unit that determines the connection destination when the processing signal is determined to be the connection command voice by the command determination unit, and (6). ) Normally, the processing signal output from the signal processing unit is output to the network communication unit that transmits to the remote call partner, and when the command determination unit determines that the processing signal is the connection command voice, a call is made. After connecting to the destination terminal, the processing signal held in the audio buffer section is output to the network communication section, the processing signal held in the audio buffer section is output, and then output from the signal processing section . It is characterized in that it functions as an output switching unit that switches the processed signal to be output to the network communication unit again .

A third aspect of the present invention is a remote communication method used for a remote communication device, which includes a signal processing unit, an audio buffer unit, a voice recognition unit, a command determination unit, a connection determination unit, and an output switching unit (1). ) The signal processing unit performs predetermined signal processing using the input signals of a plurality of microphones that collect audio, and (2) the audio buffer unit constants the processing signal output from the signal processing unit. The period is held, (3) the voice recognition unit performs voice recognition for the processing signal, and (4) the command determination unit uses the result of the voice recognition unit, and the processing signal is a connection command. It is determined whether or not it is voice, (5) the connection determination unit determines the connection destination when the processing signal is determined to be the connection command voice by the command determination unit, and (6) the output switching unit determines. Normally, the processing signal output from the signal processing unit is output to the network communication unit that transmits the remote communication partner, and when the command determination unit determines that the processing signal is the connection command voice, a call is made. After connecting to the destination terminal, the processing signal held in the audio buffer section is output to the network communication section, the processing signal held in the audio buffer section is output, and then output from the signal processing section . It is characterized in that the processed signal is switched to be output to the network communication unit again .

According to the present invention, when the user connects to the other party even if he / she is away from the microphone, he / she connects with the name of the other party to connect to and the words that start the conversation, which is the same as when making an actual face-to-face call. By starting and ending the connection with the words that end the call, the state where the conversation starts and the state where the conversation ends can be reproduced, and both sides can feel a high sense of presence.

It is a block diagram which shows the structure of the remote communication apparatus which concerns on 1st Embodiment. It is explanatory drawing which shows an example of the command list part which concerns on 1st Embodiment. This is an example of the device arrangement and the user's position in the room of one of the bases where the remote communication device according to the first embodiment is used. It is a block diagram which shows the structure of the remote communication apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the structure of the remote communication apparatus which concerns on 3rd Embodiment.

(A) First Embodiment In the following, an embodiment of the remote call device, the remote call program, and the remote call method of the present invention will be described in detail with reference to the drawings.

The first embodiment illustrates, for example, a case where the above-mentioned remote communication device, remote communication program, and remote communication method of the present invention are applied to a microphone input unit of a video conference system, a telephone conference system, or the like.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram showing a configuration of a remote communication device 100 according to the first embodiment.

The remote communication device 100 of the first embodiment of the present invention may be constructed as a dedicated board, for example, or is realized by writing a remote communication program to a DSP (digital signal processor). It may be realized by a CPU and software (remote call program) executed by the CPU, but functionally, it can be represented by FIG.

In FIG. 1, the remote communication device 100 according to the first embodiment of the present invention includes a microphone array 101, a microphone amplifier 102, an AD converter 103, a call processing unit 104, a NW communication unit 105, a network 106, and a DA converter 107. It has a speaker amplifier 108, a speaker 109, a video camera 120, and a monitor 121.

The microphone array 101 is a plurality of microphones that receive human voices and sounds.

The microphone amplifier 102 amplifies a plurality of input signals received by the microphone array 101.

The AD converter 103 converts a plurality of input signals amplified by the microphone amplifier 102 from an analog signal to a digital signal. Hereinafter, the signal converted by the AD converter 103 will be referred to as a “microphone array input signal”.

The call processing unit 104 processes a plurality of input microphone input signals into signals, and outputs the processed signals to the output terminal. At the same time, the call processing unit 104 stores the signal processed signal in the audio buffer. Further, the call processing unit 104 recognizes the signal processed signal by voice, and when the voice recognition result matches one of the commands in the command list unit, the connection determination result and the sound signal stored in the audio buffer are displayed for a certain period of time. It outputs, and when the output is completed for a certain period of time, the signal processed signal is output again.

The NW communication unit 105 performs connection processing with the network 106 based on the connection determination result output from the call processing unit 104. After the connection, the remote communication device 100 exchanges voice with another NW communication unit (another remote communication device) connected to the network 106 via the NW communication unit 105.

The DA converter 107 converts voice from the network 106 (sound signal transmitted via the NW communication unit 105) from a digital signal to an analog signal.

The speaker amplifier 108 amplifies the analog signal.

The speaker 109 is a speaker that converts an electric signal into vibration of air and outputs it as sound.

Next, the detailed configuration of the call processing unit 104 will be described.

The call processing unit 104 includes an input terminal 110, a signal processing unit 111, an audio buffer unit 112, a voice recognition unit 113, a command list unit 114, a command determination unit 115, an output switching unit 116, an output terminal 117, a connection determination unit 118, and It has a connection determination result output terminal 119.

The input terminal 110 is an interface for inputting a microphone array input signal to the calling processing unit 104.

The signal processing unit 111 processes the microphone array input signal and outputs the processed signal.

The audio buffer unit 112 is a buffer that holds the processing signal for a certain period of time.

The voice recognition unit 113 recognizes the processed signal by voice and outputs the result of the voice recognition.

The command list unit 114 is a list in which commands are held. The command list unit 114 holds, for example, a list of commands in a text file as shown in FIG. Note that FIG. 2 is an example, and various values (formats) can be applied to the content and format of the data to be retained.

The command determination unit 115 determines whether or not the voice recognition result exists in the command list of the command list unit 114, and outputs the determination result.

The output switching unit 116 determines the sound signal to be output from the command determination result, and outputs the sound signal.

The output terminal 117 is an interface that outputs the sound signal of the call processing unit 104.

The connection determination unit 118 determines the connection destination with the network 106 (for example, the videophone of the other party) based on the voice recognition result by the voice recognition unit 113 and the command determination result based on the command determination unit 115.

The connection determination result output terminal 119 outputs the connection determination result to the NW communication unit 105.

The video camera 120 is a photographing device installed at its own base (a base where the remote communication device 100 is installed). The image captured by the video camera 120 is transmitted to the network 106 by the NW communication unit 105.

The monitor 121 is a video output device. The image output by the monitor 121 is, for example, an image taken by a video camera installed at the base of the other party, and this image (encoded data) is received by the NW communication unit 105 via the network 106. After decoding (decoding), it is input to the monitor 121.

(A-2) Operation of First Embodiment The operation of the remote communication device 100 according to the first embodiment of the present invention will be described in detail.

FIG. 3 is an example of the device arrangement and the user's position in the room of one of the bases using the remote communication device according to the first embodiment. It is assumed that the same remote communication device 100 is installed at the base of the other party. Room 151 is, for example, a conference room, and the height may be such that the monitor 121 can be easily installed and has a sufficient height (for example, the height of the monitor 121 + several m, or 2 m or more). The size (area) of 151 is such that a monitor 121, a microphone array 101, a speaker 109, etc. can be easily installed and has a sufficient margin, or a sufficient area for a user 152 to talk (for example, laterally). It suffices if there is a number of vertical meters).

First, when the operation of the remote communication device 100 starts, the monitor 121 displays the image captured by the camera 120 of the remote communication device 100 at the other party's base on the monitor 121 of the remote communication device 100 at the other party's base. To. On the monitor 121 of the other party's base, the image taken by the video camera 120 of the own base is displayed on the monitor of the other party's base. At this time, the audio is not connected, and only the image taken by the camera 120 is displayed at both bases, so that the state of each base can be confirmed.

When the user 152 approaches the monitor 121 and makes a call with a person who has moved to the entire image of the other party's base in order to see the image of the other party's base displayed on the monitor 121, the user 152 causes the user 152 to make a call. Speak a call and start a conversation.

A sound signal such as a sound emitted by the user 152 or an analog sound signal superimposed with an environmental sound is input to each microphone of the microphone array 101.

The analog sound signal input to the microphone array 101 is amplified by the microphone amplifier 102, converted from the analog signal to the digital signal by the AD converter 103, and the microphone input signal x (m,) is connected to the input terminal 110 of the call processing unit 104. It is input as n). In the microphone input signal x (m, n), m is a parameter for identifying each microphone in the microphone array 101, and n is a parameter indicating a time series of the input signal.

When a signal starts to be input to the input terminal 110 of the call processing unit 104, first, the microphone input signal x (m, n) is input to the signal processing unit 111.

The signal processing unit 111 performs microphone array processing on the input signal, and performs directivity processing and sound source separation processing for separating sound sources. As a method of directivity processing, for example, there is a method of processing a delay sum array processing which is a conventional microphone array processing according to the following equations (1) and (2).

In the above equation (1), x'(n) is the microphone array processing signal, M is the number of microphones, τ _m is the delay amount, d in equation (2) is the microphone spacing, θ is the directivity angle, and c is the sound velocity. (For example, when directivity is formed in the front direction of the microphone array, θ = 0 and τ _m = 0).

As the signal processing calculation means, various methods can be widely applied. For example, a conventional microphone array process other than the delay sum array process or two sets of microphone arrays may be used in a specific area. Microphone array processing that can collect sound may also be used. The signal processing unit 111 outputs the calculated microphone array processing signal x'(n) to the audio buffer unit 112, the voice recognition unit 113, and the output switching unit 116.

When the remote communication device 100 is in operation, the output switching unit 116 outputs a silent signal as an output signal y (n) to the output terminal 117 as shown in the following equation (3).
y (n) = 0 ... (3)

Further, the call processing unit 104 simultaneously holds the microphone array processing signal x'(n) at the write position write_index of the audio buffer buffer (n) of the audio buffer unit 112 according to the following equation (4). After holding, the call processing unit 104 advances (increments) the writing position write_index as shown in the following equation (5).

BUFFER_SIZE in the above equation (5) is the buffer length of the audio buffer of the audio buffer unit 112.

Further, the call processing unit 104 simultaneously performs voice recognition of the microphone array processing signal x'(n) by the voice recognition unit 113, and outputs the voice recognition result to the command determination unit 115.

The command determination unit 115 compares the result of voice recognition with the command list held in the command list unit 114 (for example, the command list in FIG. 2), and the "person name" in the command list and the "connection command" in the command list. Continues to determine whether or not voice recognition has been performed (for example, "Hello Mr. XX"). Then, the command determination unit 115 outputs the determination result to the output switching unit 116, and outputs the determination result and the voice recognition result to the connection determination unit 118 (for example, when the "person name" and the "connection command" are continuously voice-recognized. The judgment result is 1, otherwise it is 0, etc.).

The connection determination unit 118 makes a connection determination based on the voice recognition result by the voice recognition unit 113 and the command determination result based on the command determination unit 115, and outputs the connection determination result to the NW communication unit 105. For example, if the determination result is 1 and the command determination unit 115 outputs the voice recognition result "Hello Mr. XX", the connection determination unit 118 is near the remote communication device 100 of the other party's base. If Mr. XX is present, the signal connected to the remote communication device 100 at the other party's base is output to the connection determination result output terminal 119. For the determination of whether or not Mr. XX can enter near the remote communication device 100 of the base, for example, the information in which the person near the terminal is registered in advance is used.

The NW communication unit 105 performs connection processing with the network 106 based on the connection determination result output via the connection determination result output terminal 119, and after the connection, the remote communication device exchanges voice via the NW communication unit 105. Is done.

On the other hand, the output switching unit 116 continuously outputs a silent signal to the output terminal 117 when the command determination unit 115 does not continuously recognize the voice recognition result of the voice recognition unit 113 by the "person name" and the "connection command". When the "person name" and the "connection command" are continuously voice-recognized by the command determination unit 115, the read position read_index of the audio buffer unit 112 is calculated according to the following equation (6).

The LEN of the above equation (6) is a length for reproducing the processing signal held in the audio buffer unit 112. As the method for determining LEN, various methods can be widely applied, and for example, there is a method in which the length is the same as the buffer size of the audio buffer unit 112 (LEN = BUFFER_SIZE). Alternatively, a method may be used in which the microphone input signal held in the audio buffer unit 112 is subjected to audio section processing, the length of the sound held in the buffer is obtained, and the length is set to LEN.

Then, as shown in the following equation (7), the output switching unit 116 uses the sound signal held in the audio buffer unit 112 as the output signal y (n) at the output terminal 117 for a fixed time (for example, the time length of LEN). Minutes) Output and advance the read position read_index as shown in the following equation (8) (increment).

The NW communication unit 105 transmits the output signal y (n) output from the output terminal 117 to the NW communication unit of the other party connected by the network 106.

When the output switching unit 116 outputs the sound signal held in the audio buffer unit 112 for a certain period of time, the output switching unit 116 outputs the microphone array processing signal x'(n) to the output signal y (n) as shown in the following equation (9). Is output to the output terminal 117.
y (n) = x'(n) ... (9)

The NW communication unit 105 transmits the output signal y (n) output via the output terminal 117 to the NW communication unit of the other party continuously connected by the network 106.

On the other hand, the voice of the other party transmitted from the network 106 is converted from a digital signal to an analog signal by the DA converter 107 via the NW communication unit 105, amplified by the speaker amplifier 108, and output by the speaker 109.

After a while, when the call is terminated, the user 152 utters a disconnect voice to end the conversation.

A sound signal such as a sound emitted by the user 152 or an analog sound signal superimposed with an environmental sound is input to each microphone of the microphone array 101.

The analog sound signal input to the microphone array 101 is amplified by the microphone amplifier 102, converted from an analog signal to a digital signal by the AD converter 103, and is converted into a digital signal by the AD converter 103, and the microphone input signal x (m, It is input as n) and is input to the signal processing unit 111.

The signal processing unit 111 performs microphone array processing on the input signal, performs directional processing and sound source separation processing for separating sound sources, and recognizes the calculated microphone array processing signal x'(n) as the audio buffer unit 112 and voice recognition. Output to unit 113 and output switching unit 116.

As shown in the equation (9), the output switching unit 116 outputs the microphone array processing signal x'(n) to the output terminal 117 as the output signal y (n).

Further, the call processing unit 104 simultaneously holds the microphone array processing signal x'(n) at the write position write_index of the audio buffer buffer (n) of the audio buffer unit 112 according to the equation (4). After holding, the call processing unit 104 advances (increments) the writing position write_index as shown in the equation (5).

Further, the call processing unit 104 simultaneously performs voice recognition of the microphone array processing signal x'(n) by the voice recognition unit 113, and outputs the voice recognition result to the command determination unit 115.

The command determination unit 115 compares the result of voice recognition with the command list held in the command list unit 114, and determines whether or not the result of voice recognition exists in the list of "disconnect commands". Then, the command determination unit 115 outputs the determination result to the output switching unit 116 and the connection determination unit 118 (for example, only when the "disconnect command" in the command list is voice-recognized (for example, "goodbye"). , If the "disconnect command" is voice-recognized during a call, the judgment result is 2, otherwise it is 0, etc.). The connection determination unit 118 makes a disconnection determination based on the voice recognition result by the voice recognition unit 113 and the command determination result based on the command determination unit 115, and connects the signal to disconnect from the NW communication unit on the other side to the NW communication unit 105. Output to the determination result output terminal 119.

The NW communication unit 105 performs disconnection processing with the other party's NW communication unit based on the connection determination result output via the connection determination result output terminal 119.

On the other hand, if the command determination unit 115 determines that the voice recognition result of the voice recognition unit 113 does not exist in the disconnect command list of the command list unit 114 after the output switching unit 116 is connected to the other party, When the microphone array processing signal is continuously output to the output terminal 117 and the command determination unit 115 determines that the result of the voice recognition of the voice recognition unit 113 exists in the disconnect command list of the command list unit 114, the equation (3) is used. As shown in the above, a silent signal is output to the output terminal 117 as an output signal y (n).

(A-3) Effect of First Embodiment As described above, according to the first embodiment, the remote communication device 100 performs signal processing for emphasizing voice by using a plurality of microphones, and signal processing. The generated signal is once held in the audio buffer, voice recognition is performed for the signal processed at the same time, it is determined whether the result of the voice recognition is the call voice, and if it is the call voice, it is connected to the other party. By outputting the call voice held in the buffer after that, the conversation is started after the call voice is transmitted to the other party, and after the conversation is started, voice recognition is performed for the signal processed signal, and the voice is recognized. It is determined whether or not the result of voice recognition is a disconnected voice, and in the case of a cut voice, a state close to a face-to-face conversation can be reproduced by disconnecting, and the conversation can be started with a high sense of presence.

Further, the remote communication device 100 of the first embodiment uses a plurality of microphones (microphone arrays) for picking up the call voice even in an environment where the usage environment is noisy, and performs signal processing for emphasizing the voice. Therefore, the voice recognition unit 113 can perform accurate voice recognition and a call.

(B) Second Embodiment Next, a second embodiment of the remote call device, the remote call program, and the remote call method of the present invention will be described in detail with reference to the drawings.

The second embodiment exemplifies the case where the sound output method of the remote communication device of the present invention is different from the first embodiment.

(B-1) Configuration of the Second Embodiment FIG. 4 is a block diagram showing the configuration of the remote communication device 200 according to the second embodiment.

The remote communication device 200 of the second embodiment is different from the remote communication device 100 of the first embodiment in that the distance estimation unit 202, the amplifier control unit 203, and the digital amplifier unit 204 are components. The other components are the same as or correspond to the components of the remote communication device 100 of FIG. 1 according to the first embodiment. In FIG. 4, the same or corresponding components of the remote communication device 100 according to the first embodiment are designated by the same reference numerals.

Further, since the detailed description of the same or corresponding component as that of the first embodiment is duplicated, it is omitted here.

In FIG. 4, the remote communication device 200 according to the second embodiment of the present invention includes a microphone array 101, a microphone amplifier 102, an AD converter 103, a call processing unit 201, a NW communication unit 105, a network 106, and a DA converter 107. It has a speaker amplifier 108, a speaker 109, a video camera 120, and a monitor 121.

Further, the call processing unit 201 includes an input terminal 110, a distance estimation unit 202, an amplifier control unit 203, a digital amplifier unit 204, a signal processing unit 111, an audio buffer unit 112, a voice recognition unit 113, a command list unit 114, and a command determination unit. It has 115, an output switching unit 116, an output terminal 117, a connection determination unit 118, and a connection determination result output terminal 119.

The distance estimation unit 202 estimates position information such as the direction, position, and distance of the user from a plurality of microphone input signals, and outputs the estimated position information.

The amplifier control unit 203 calculates and outputs an amplifier value for amplifying a plurality of microphone input signals from the estimated position information.

The digital amplifier unit 204 amplifies a plurality of microphone input signals based on the amplifier value output from the amplifier control unit 203.

(B-2) Operation of the Second Embodiment The basic operation of the voice processing in the remote communication device 200 according to the second embodiment is the same as the voice processing described in the first embodiment.

Hereinafter, the processing operations in the distance estimation unit 202, the amplifier control unit 203, and the digital amplifier unit 204, which are different from the first embodiment, will be described in detail.

The distance estimation unit 202 estimates position information such as the direction and position of the user from a plurality of microphone input signals. Various methods can be widely applied to the method of estimating the user's position information. For example, a cross-correlation function is calculated using two microphone input signals out of a plurality of microphone input signals, and a cross-correlation function is used. The user's direction may be estimated from. In addition, four microphone input signals out of a plurality of microphone input signals are extracted, two microphone input signals are used from the two microphone input signals to calculate two cross-correlation functions, and two directions and each microphone are calculated from the two cross-correlation functions. The position of the user may be calculated from the position of. The distance estimation unit 202 outputs the estimated position information to the amplifier control unit 203.

The amplifier control unit 203 determines the value of the digital amplifier from the position estimation information output from the distance estimation unit 202. Various methods can be widely applied to the method of determining the amplifier value, and for example, there is a method of determining according to the distance as in the following equation (10).

The AMP of the above equation (10) is an amplifier value, and k is a distance (unit: meters) from the microphone array. Further, the AMP value may be continuously determined by the distance from the microphone array. The amplifier control unit 203 outputs the determined amplifier value to the digital amplifier unit 204.

As shown in the following equation (11), the digital amplifier unit 204 multiplies the amplifier value output from the amplifier control unit 203 with the microphone input signal.
x'' (m, n) = AMP · x (m, n) ... (11)

The digital amplifier unit 204 outputs the multiplied and amplified microphone input signal x ″ (m, n) to the signal processing unit 111.

The signal processing unit 111 performs the above-mentioned microphone array processing using the amplified microphone input signal x ″ (m, n), and performs directivity processing and sound source separation processing for separating sound sources.

(B-3) Effect of Second Embodiment As described above, according to the second embodiment, the remote communication device 200 has a microphone input signal even when the user calls at a place away from the monitor 121. By estimating the distance, position, and direction of the user from the above, controlling the digital amplifier, and setting the volume to an appropriate level, the voice can be transmitted at an appropriate volume.

(C) Third Embodiment Next, a third embodiment of the voice processing device, the voice processing program, and the voice processing method of the present invention will be described in detail with reference to the drawings.

The third embodiment exemplifies the case where the distance estimation method of the remote communication device of the present invention is different from the second embodiment.

(C-1) Configuration of Third Embodiment FIG. 5 is a block diagram showing a configuration of a remote communication device according to a third embodiment.

The second embodiment of the remote communication device 300 of the third embodiment is that the input of the distance estimation unit 202 of the remote communication device 200 configuration of the second embodiment is not a plurality of microphone input signals but a distance sensor 302. It is different from the remote communication device 200 of the form. In FIG. 5, the same or corresponding components of the remote communication device 200 according to the second embodiment are designated by the same reference numerals.

Further, since the detailed description of the components that are the same as or correspond to the second embodiment is duplicated, they will be omitted here.

In FIG. 5, the remote communication device 300 according to the third embodiment of the present invention includes a microphone array 101, a microphone amplifier 102, an AD converter 103, a call processing unit 301, a NW communication unit 105, a network 106, and a DA converter 107. It has a speaker amplifier 108, a speaker 109, a video camera 120, and a monitor 121.

Further, the call processing unit 301 includes an input terminal 110, a distance sensor 302, a distance sensor input terminal 303, a distance estimation unit 304, an amplifier control unit 203, a digital amplifier unit 204, a signal processing unit 111, an audio buffer unit 112, and a voice recognition unit. It has 113, a command list unit 114, a command determination unit 115, an output switching unit 116, an output terminal 117, a connection determination unit 118, and a connection determination result output terminal 119.

The distance sensor 302 is a sensor that detects the distance, position, and direction of the user 152.

The distance sensor input terminal 303 is an interface for calling the sensor input signal of the distance sensor 302 and inputting it to the processing unit 401.

The distance estimation unit 304 estimates the position information of the user 152 from the input signal of the distance sensor input via the distance sensor input terminal 303, and outputs the estimated position information.

(C-2) Operation of the Third Embodiment The basic operation of the voice processing in the remote communication device 300 according to the third embodiment is the call processing described in the first embodiment and the second embodiment. Is similar to.

Hereinafter, the processing operations in the distance sensor 302, the distance sensor input terminal 303, and the distance estimation unit 304, which are different from the first embodiment and the second embodiment, will be described in detail.

The distance sensor 302 uses the sensor to estimate position information such as the direction and position of the user. As the sensor, various sensors can be widely applied, and for example, an infrared sensor or an ultrasonic sensor may be used. The distance estimation unit 304 starts processing when the sensor input signal is input to the distance sensor input terminal 303.

When a signal starts to be input to the distance sensor input terminal 403, the distance estimation unit 304 estimates the distance, direction, and position from the user 152 to the microphone array 101 based on the input signal of the distance sensor.

The amplifier control unit 203 determines the value of the digital amplifier from the position estimation information output from the distance estimation unit 202. Various methods can be widely applied to the method of determining the amplifier value, and for example, there is a method of determining according to the distance as in the above equation (10). Further, the AMP value may be continuously determined by the distance from the microphone array. The amplifier control unit 203 outputs the determined amplifier value to the digital amplifier unit 204.

(C-3) Effect of Third Embodiment As described above, according to the third embodiment, by providing the distance sensor, the distance of the user is accurately measured and the appropriate amplifier value is amplified. By doing so, the microphone input signal does not become small depending on the position where the user talks, and the voice can be transmitted at an appropriate volume by setting the appropriate volume.

(D) Other Embodiments Although various modified embodiments have been described in each of the above-described embodiments, the present invention can also be applied to the following modified embodiments.

(D-1) The remote communication device described in each of the above-described embodiments may be mounted on a device that starts a call by a command by voice input when starting a call in a conference call, for example. ..

(D-2) The call processing unit and the NW communication unit of the remote communication device described in each of the above-described embodiments may be processed by a processing device (for example, a server) on the network.

(D-3) In the remote communication device described in each of the above-described embodiments, the microphone array 101 is arranged in front of the monitor 121 as shown in FIG. 2, but this is the example in which the microphone array 101 is arranged. Not limited to. For example, the microphone array 101 may be arranged on the upper part or the side surface of the monitor 121. Further, when the remote communication device includes a projector and a screen, a screen for forming an image of a projected image from the projector may be used as an alternative to the monitor 121. This screen may be an ordinary screen or a screen that transmits sound. In the case of a screen that transmits sound, the microphone array 101 may be arranged behind the screen.

100 ... remote communication device, 101 ... microphone array, 102 ... microphone amplifier, 103 ... AD converter, 104 ... call processing unit, 105 ... NW communication unit, 106 ... network, 107 ... DA converter, 108 ... speaker amplifier, 109 ... Speaker, 110 ... Input terminal, 111 ... Signal call processing unit, 112 ... Audio buffer unit, 113 ... Voice recognition unit, 114 ... Command list unit, 115 ... Command determination unit, 116 ... Output switching unit, 117 ... Output terminal, 118 ... Connection determination unit, 119 ... Connection determination result output terminal, 120 ... Video camera, 121 ... Monitor, 151 ... Room, 152 ... User, 200 ... Remote communication device, 201 ... Call processing unit, 202 ... Distance estimation unit, 203 ... Amplifier control unit, 204 ... Digital amplifier unit, 300 ... Remote communication device, 301 ... Call processing unit, 302 ... Distance sensor, 303 ... Distance sensor input terminal, 304 ... Distance estimation unit, 401 ... Call processing unit, 402 ... Distance estimation unit, 403 ... Distance sensor input terminal.

Claims (4)

A signal processing unit that performs predetermined signal processing using the input signals of multiple microphones that collect audio, and
An audio buffer unit that holds the processed signal output from the signal processing unit for a certain period of time,
A voice recognition unit that recognizes voice for the above processed signal,
Using the result of the voice recognition unit, a command determination unit that determines whether or not the processing signal is a connection command voice, and
When the processing signal is determined to be the connection command voice by the command determination unit, the connection determination unit that determines the connection destination and
Normally, the processing signal output from the signal processing unit is output to the network communication unit that transmits it to the remote call partner, and when the command determination unit determines that the processing signal is the connection command voice, the call partner After connecting to the destination terminal, the processing signal held in the audio buffer section is output to the network communication section, the processing signal held in the audio buffer section is output, and then the signal processing section outputs the processing signal . A remote communication device including an output switching unit that switches the processed signal to be output to the network communication unit again . A distance estimation unit that estimates the distance to the user who uses the remote communication device, and
A microphone amplifier control unit that determines the amplification value of the input signals of the plurality of microphones according to the distance estimated by the distance estimation unit, and
The remote communication device according to claim 1, further comprising a digital amplifier unit that amplifies the signal by multiplying the input signals of the plurality of microphones by the amplification value determined by the microphone amplifier control unit. Computer,
A signal processing unit that performs predetermined signal processing using the input signals of multiple microphones that collect audio, and
An audio buffer unit that holds the processed signal output from the signal processing unit for a certain period of time,
A voice recognition unit that recognizes voice for the above processed signal,
Using the result of the voice recognition unit, a command determination unit that determines whether or not the processing signal is a connection command voice, and
When the processing signal is determined to be the connection command voice by the command determination unit, the connection determination unit that determines the connection destination and
Normally, the processing signal output from the signal processing unit is output to the network communication unit that transmits it to the remote call partner, and when the command determination unit determines that the processing signal is the connection command voice, the call partner After connecting to the destination terminal, the processing signal held in the audio buffer section is output to the network communication section, the processing signal held in the audio buffer section is output, and then the signal processing section outputs the processing signal . A remote call program characterized by functioning as an output switching unit that switches the processed signal to be output to the network communication unit again . A remote call method used for remote call devices.
It has a signal processing unit, an audio buffer unit, a voice recognition unit, a command determination unit, a connection determination unit, and an output switching unit.
The signal processing unit performs predetermined signal processing using the input signals of a plurality of microphones that collect sound, and performs predetermined signal processing.
The audio buffer unit holds the processed signal output from the signal processing unit for a certain period of time.
The voice recognition unit performs voice recognition on the processed signal and performs voice recognition.
The command determination unit determines whether or not the processing signal is a connection command voice by using the result of the voice recognition unit.
The connection determination unit determines the connection destination when the processing signal is determined to be the connection command voice by the command determination unit.
The output switching unit normally outputs the processing signal output from the signal processing unit to the network communication unit that transmits the processing signal to the remote call partner, and the command determination unit determines that the processing signal is the connection command voice. In this case, after connecting to the other party's terminal, the processing signal held in the audio buffer section is output to the network communication section, and the processing signal held in the audio buffer section is output. A remote communication method characterized in that the processing signal output from the signal processing unit is switched to be output to the network communication unit again .

Images