JP4867579B2 - Remote conference equipment - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reproduce correctly the acoustic field information on sound source by outputting it to a speaker array, while suppressing the consumption of transmission line resource, in a teleconference device for reproducing the received sound and its acoustic field. <P>SOLUTION: A teleconference device 1 includes a function as a transmitter unit 1A and a receiver unit 1B. The transmitter unit 1A performs transmission 20 by transmitting an audio signal 27B formed from a sound collection signal of a microphone array Mi (i=1-N) and position information 27A. The position information forms a plurality of sound collection beams directed to a particular direction and selects the sound collection beam of the maximum sound volume. In the receiver unit 1B, a parameter calculation unit 32 sets a virtual sound source according to the data of received signals 30 and the parameter calculation unit 32 sets a delay parameter. A virtual sound source generation signal processing unit 33 forms a sound emission beam according to the parameters and outputs it to a speaker SPi. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a remote conference apparatus that transmits voice together with position information and reproduces the received voice and its sound field.

  Conventionally, there has been proposed a remote conference apparatus that receives a voice on the transmission side and reproduces a sound field of the voice on the transmission side (see Patent Documents 1 and 2).

  Patent Document 1 describes the following apparatus. The voice of the speaker is picked up by the microphone, the speaker position information is formed by the speaker information obtained from the microphone, and the speaker position information is multiplexed and transmitted together with the voice information. Further, the position of the speaker to be ringed is switched on the receiving side based on the speaker position information sent, and the voice and position of the speaker are reproduced on the receiving side.

Also, Patent Document 2 discloses a method for creating stereoscopic audio information that reproduces a sound field of a transmission source by transmitting audio information received by a plurality of microphone arrays and outputting it to the same number of speaker arrays. Has been.
JP-A-9-261351 Japanese Patent Laid-Open No. 2-114799

  However, in Patent Document 1, it is difficult to perform accurate sound image localization because the method of localizing sound to be reproduced is based on only the sound volume balance of a plurality of speakers.

  In Patent Document 2, since sound field information received by a plurality of microphone arrays is output as it is, natural sound can be output, but a signal having the same number of channels as the number of array speakers is transmitted to a remote place. Therefore, there is a drawback that a lot of transmission line resources are consumed.

  Accordingly, the present invention provides an apparatus for reproducing a sound field of a transmission destination by outputting a sound signal to the speaker array based on the received sound source position information, thereby suppressing the consumption of transmission line resources and the sound field of the transmission destination. The purpose is to reproduce accurately.

In the present invention, means for solving the above-described problems are configured as follows.

(1) The present invention
A microphone array in which a plurality of microphones that collect sound and output as sound collection signals are arranged in an array;
A sound collection beam forming unit that respectively forms a sound collection beam directed to a plurality of sound collection areas by delay-adding the sound collection signals output from the plurality of microphones;
Position information detection means for detecting, as position information, a sound collection area corresponding to a sound collection beam indicating the maximum volume among the plurality of sound collection beams;
Transmitting means for transmitting the output of the sound pickup signal of the microphone and the position information;
An array speaker in which a plurality of speakers are arranged in an array; and
A receiving unit for receiving an audio signal and position information from the outside;
A signal processing unit that processes the received audio signal so as to form a sound emission beam having a position determined based on the received position information as a virtual sound source position and supplies the sound output beam to the plurality of speakers;
Equipped with a,
The position information detecting means detects a sound collecting area of the sound collecting beam having the maximum sound volume, and then obtains an acquisition sound collecting beam directed to a plurality of subdivided sound collecting areas obtained by further subdividing the sound collecting area. The position information is detected based on subdivided sound collection areas corresponding to the plurality of acquisition sound collection beams that are formed and are selected in descending order of the volume of the acquisition sound collection beam .

The device of the present invention has a function for one device for connecting two or more remote conference devices to each other via a communication line or a network. That is, a receiving unit and a transmitting unit are provided.
In the transmission side unit, the sound source position information detection means detects the position of the sound source, and the microphone picks up the sound signal of the sound source and converts this sound into an electric signal.

  The sound source position information detection means forms a sound collection beam directed to a plurality of positions from the sound collection signals of the plurality of microphones, and determines the position of the sound collection area of the sound collection beam from the maximum volume of the plurality of sound collection beams. Detect as speaker location information. The transmission unit transmits the audio signal obtained by the array microphone and the position information.

In the receiving unit, the receiving unit receives the sound signal of the sound source and the position information of the sound source from the counterpart device. The signal processing unit processes the received audio signal so as to form a sound emission beam having a virtual sound source position at a position determined by the received sound source position information behind the speaker unit. Supply to the unit. For example, the coordinates of the virtual sound source can be on a coordinate axis plane with the center of the apparatus as the origin and the back direction as the Y axis.
Although one of the two remote conference apparatuses has been described above, the same applies to sound signal collection / sound emission in the opposite direction.

  With these configurations of the present invention, a realistic positional relationship as if the conference participants are interacting with a virtual sound source in the conference room on the opposite device side provided on the opposite side across the remote conference device You can have a conversation. Further, since the position information of the sound source is received together with the sound signal of the sound source, it is possible to eliminate the disadvantage of consuming a lot of transmission line resources as in Patent Document 2. Further, unlike Patent Document 2, it is not necessary to limit the number of speakers to the same number on the transmission side and the reception side.

  Note that the output of the collected sound signal of the microphone transmitted by the transmitting means may be any sound collected signal of the microphone, and the sound source synthesizes any one or a plurality of microphone sound signals (for example, It may be a simple addition), may output a sound collection beam obtained by delay-adding sound collection signals from a plurality of microphones, or may synthesize a plurality of sound collection beams.

  In this configuration, the position information detection means forms the acquisition sound collection beam directed to the subdivided sound collection area, and selects a sound volume of the sound collection beam having a large volume. it can. Further, in this configuration, instead of forming and selecting a sound collecting beam for each fine area from the beginning, selection is made by narrowing down to two stages. Therefore, since the number of sound collecting beams to be formed can be reduced, the hardware used for the position information detecting means can be simplified.

(3) The present invention
The position information detecting means is arranged between subdivided sound collection areas corresponding to a plurality of acquisition sound collection beams selected in descending order of the volume of the acquisition sound collection beam according to the strength of the selected acquisition sound collection beam. And detecting the position information by proportional distribution,
A signal adding unit is provided that combines the outputs of the selected plurality of acquisition sound collection beams by the proportional distribution and transmits the output to the transmission unit as the sound collection signal output of the microphone .

  In this configuration, the position information detecting means detects the position information between the selected subdivided sound collection areas by proportional distribution corresponding to the strength of the selected acquisition sound collection beam. The intermediate portion can be interpolated, and more accurate position information can be obtained. Further, since the signal processing unit synthesizes the outputs of the selected plurality of acquisition sound collecting beams based on the proportional distribution, it is possible to interpolate an intermediate portion of the sound collection areas of the respective sound collection beams. Further, by always interpolating in a balanced manner in this way, the sound collecting beams can be switched smoothly even when the sound source moves.

  According to the present invention, it is possible to interact with a conference participant in a realistic positional relationship as if interacting with a virtual sound source in a conference room on the other device side provided on the opposite side across the remote conference device. Can do.

  A remote conference apparatus used for voice communication according to the present embodiment will be described with reference to FIG. 1A and 1B are an external view (A) and a schematic functional diagram (B) of a remote conference device used for voice communication according to the present embodiment.

  First, the configuration of the remote conference apparatus 1 will be described with reference to FIG. As shown in FIG. 1 (A), the remote conference device 1 has an appearance of a speaker array composed of a plurality of speakers SPi (i = 1 to N, N is an integer) and a plurality of microphones Mi (i = 1 to N). ), And is installed on the desk 101, for example. Connect to two remote conferencing devices installed at remote locations via a communication line, and perform voice communication with each other.

  The plurality of speakers SPi (i = 1 to N) constituting the speaker array are connected to an audio processing system that can handle an audio signal independently, and can output audio independently, and a microphone The plurality of microphones Mi (i = 1 to N) constituting the array can independently convert the audio output into digital data. The description of the coordinate systems X, Y, and Z will be described later.

  Next, an outline of functions of the remote conference apparatus 1 will be described with reference to FIG. As shown in the diagram on the left side of FIG. 1B, when transmitting a voice, the transmitting speaker 102 </ b> A speaks toward the remote conference device 1. Then, in the transmission-side unit 1A, a plurality of microphones Mi (i = 1 to N) constituting the microphone array each collects sound, and the position of the speaker 102A on the transmission side is based on the collected sound. Is detected, and the collected sound signal and position information indicating the position are transmitted. This position information is calculated from the strength of the pattern by forming a plurality of patterns of sound collection beams synthesized (ie, delay-added) by setting a delay for each collected electrical signal. The coordinate system of this position information can be given by an XYZ axis coordinate system as shown in FIGS. 1 (A) and 1 (B), for example. Specifically, in the XYZ axis coordinate system, the vertical direction with respect to the sound receiving surface of the teleconference device 1 is the Z axis, the left and right direction is the X axis, and the depth direction perpendicular to the sound receiving surface is the Y axis direction. . Further, instead of the X-axis and Z-axis directions, it is also possible to give the polar coordinates of the distance R from the center of the sound receiving surface and the angle θ.

  As shown in the diagram on the right side of FIG. 1B, on the reception side of the remote conference device 1, the reception side unit 1B determines the position of the virtual sound source VS based on the transmitted audio signal and position information. Based on this, the delay amount to the signal output to each speaker SPi is calculated, and a delay is added to the speech spoken on the transmission side unit 1A side and output to the speaker SPi. Thereby, the sound field of the voice spoken on the transmission side unit 1A side is accurately reproduced, and the sound field on the transmission side is transmitted to the listener 102B on the reception side. Here, since it is natural to set the virtual sound source VS at a position facing the listener 102B on the receiving side talking to the receiving unit 1B, the X-axis coordinates in the left and right directions as viewed from the listener 102B on the receiving side and The Y-axis coordinate in the depth direction is set so that the value actually obtained by the transmission-side unit 1A and the value of the coordinate for setting the virtual position by the reception-side unit 1B are reversed. The same applies when using the polar coordinates described above.

In the above description of FIG. 1, the transmission side unit 1 </ b> A and the reception side unit 1 </ b> B have been described using different symbols in order to conceptually distinguish them. Bidirectional communication is possible, and the listener 102B on the receiving side can also transmit the voice to the speaker 102A on the transmitting side through the microphone array of the receiving unit 1B. That is, each remote conference apparatus is integrally provided with the transmission side unit 1A and the reception side unit 1B of FIG.
In the present embodiment, human voice is given as an example of the voice transmitted by the remote conference device 1, but the target of the device 1 of the present embodiment is not limited to the human voice. For example, music may be used. The same applies hereinafter.

  Here, referring to FIG. 5A, a method for searching for the position of the speaker performed by the transmitting unit 1A will be briefly described (details of FIG. 5 will be described later). As shown in FIG. 5A, the detection beam forming unit (corresponding to the detection beam forming unit 21 shown in FIG. 2 described later) in the teleconference apparatus 1 follows the delay pattern of the transmitting speaker 102A. A sound collection beam directed to a plurality of sound collection areas 111 to 114 assumed as positions is formed.

  This sound collecting beam is formed by superimposing digital audio signals by adjusting a delay amount difference due to a distance difference from each of the microphones Mi (i = 1 to N) so as to match the specific position. As a result, the voices at the specific position are input while being strengthened with each other, and the voices coming from other than the specific position are canceled out, and have directivity to the voice at the specific position. .

  If these sound collection areas 111 to 114 and the position of the speaker 102A on the transmission side match, the sound collection beam having the highest volume can be obtained from these sound collection beams. Therefore, the beam forming unit for detection inside the remote conference apparatus 1 simultaneously forms four sound collecting beams directed to the sound collecting areas 111 to 114 and searches for a direction in which the sound volume is high. In addition, an acquisition beam forming unit (corresponding to the acquisition beam forming unit 22 of FIG. 2 described later and having a circuit configuration similar to that of the detection beam forming unit) inside the teleconference apparatus 1 was obtained in this way. Subdivision areas 131 to 134 for further dividing the direction in which the volume is high (area 114 in FIG. 5) are set. In addition, the acquisition beam forming unit forms a sound collection beam directed to the subdivided areas 131 to 134, respectively. The remote conference apparatus 1 selects a sound collecting beam having a large volume from the sound collecting beams directed to the subdivided areas 131 to 134. Thus, by searching in two steps, the position of the speaker 102A can be searched accurately and quickly.

  The beam forming unit is always operated, or the teleconferencing device 1 detects the position at a predetermined interval, for example, every 0.5 seconds, and the beam position calculating unit 24 then detects the position. Get instructions and get up and running.

Next, the internal configuration of the remote conference apparatus 1 will be described with reference to FIG. FIG. 2 is a block diagram showing this internal configuration.
First, the configuration of the transmission side unit 1A will be described with reference to FIG. The transmission side unit 1A includes a microphone Mi and an A / D converter 29 in order to digitally process the collected sound. Also, a sound collecting beam directed to a plurality of positions in front of the microphone assumed to be present is formed so that the position where the speaker is speaking can be estimated and position information can be output as the control signal 241. A detection beam forming unit 21, a BPF 23 (bandpass filter), and a beam position calculation unit 24. Further, the vicinity of the position detected by the beam position calculation unit 24 is searched in more detail, and the acquisition beam forming unit 22, the signal selection unit 25, and the signal addition are performed so that a sound collecting beam directed to the position can be formed. Part 26 is provided. Further, a multiplexing unit 28 is provided for multiplexing the position information 27A and the collected sound beam. Each configuration will be described below.

The microphones Mi (i = 1 to N) in FIG. 2A are N microphones as shown in FIG. 1, and these microphones Mi convert the voice of the speaker 102A on the transmission side into a voice signal.
The A / D converter 29 can be composed of an IC for A / D conversion, converts an audio signal acquired by the microphone Mi into a digital audio signal, and sends the digital audio signal to the detection beam forming unit 21 and the acquisition beam forming unit 22. .

  The detection beam forming unit 21 simultaneously forms a plurality of sound collecting beams directed to an area assumed to have a speaker. Therefore, for example, a ring buffer RAM for performing a delay process or an equivalent thereof is provided to adjust the delay amount of the audio signal collected by each microphone Mi. In order to perform this delay processing, a ROM storing a program for controlling writing and reading of the ring buffer RAM and data for delay control is provided. In addition, the beam forming unit 21 includes a calculation unit for executing this program. The remote conference device 1 is provided with a DSP that operates the signal processing of the remote conference device 1, and the calculation unit of the beam forming unit for detection 21 is configured as one function thereof.

Therefore, as described above, the detection beam forming unit 21 in FIG. 2A sets a sound collection area of the sound collection beam assuming a plurality of positions of the speaker in front of the transmission side unit 1A, and the position A sound collecting beam directed to the head is formed.

  In the following description, the latter sound collection beam is referred to as an acquisition beam in order to distinguish between the sound collection beam formed by the detection beam forming unit 21 and the sound collection beam formed by the acquisition beam forming unit 22.

The detection beam forming unit 21 in FIG. 2A forms a sound collecting beam directed to a sound collecting area (corresponding to 111 to 114 in FIG. 5A) in order to search for a rough range.
The acquisition beam forming unit 22 has the same configuration as the detection beam forming unit 21. However, the acquisition beam forming unit 22, based on the calculation result of the beam position calculation unit 24 that analyzes the collected sound beam of the detection beam forming unit 21, collects the sound collection area (sound collection in FIG. The subdivided sound collection areas 131 to 134) in the area 114 are set. The acquisition beam forming unit 22 forms acquisition beams directed to the subdivided sound collection areas 131 to 134 and outputs them to the signal selection unit 25. In this way, the detection beam forming unit 21 and the acquisition beam forming unit 22 form the sound collection beam and the acquisition beam finely step by step and search for the position of the speaker 102A on the transmission side. Compared to the detailed search, the hardware configuration can be simplified, the calculation speed can be increased, and the response to the change in the position of the transmitting speaker 102A can be increased.
The BPF 23 in FIG. 2A performs a convolution operation on the collected sound beams MB1 to MB4 of the detection beam forming unit 21 with a filter that cuts a band other than the voice band necessary for detection of the speaker 102A on the transmission side, The sound collection beams MB′1 to MB′4 are output. As a result, the amount of calculation can be reduced, and the search performed by the beam position calculation unit 24 can be speeded up. In the BPF 23, even if such a band is cut, there is no problem because the detection beam forming unit 21 sets the rough sound collection areas 111 to 114.

  The beam position calculation unit 24 selects the pattern with the highest volume from the collected sound beams MB′1 to MB′4 output from the BPF 23. Thereby, the position of the speaker 102A on the transmission side can be specified. The beam position calculation unit 24 outputs the specified position information to the acquisition beam forming unit 22 as a control signal 241. Specifically, an ID code is provided in advance for a pattern of a plurality of positions where it is assumed that there is a speaker 102A on the transmission side, and the beam position calculation unit 24 selects an ID code at a position where the volume of the collected sound beam is high. Output to the acquisition beam forming unit 22.

  The calculation of the sound volume in the beam position calculation unit 24 can be substituted by performing FFT conversion on digital time-series audio data and adding the squares of gains for each of a plurality of specific frequencies. Further, the beam position calculation unit 24 operates every predetermined interval, for example, every 0.5 seconds. Thereby, the beam position calculation unit 24 can detect the movement of the speaker 102 </ b> A as the control signal 241.

2A receives the control signal 241 from the beam position calculation unit 24, and the sound collection beam having the loudest volume among the sound collection beams synthesized by the detection beam formation unit 21. The acquisition beams directed to a plurality of (four in this embodiment) sound collection areas that further subdivide the sound collection area corresponding to 1 are formed.
In addition, the signal selection unit 25 selects two acquisition beams (corresponding to the subdivided sound collection areas 133 and 134 in FIG. 5) of the acquisition beams output from the acquisition beam forming unit 22, and selects these. The position proportionally distributed according to the volume of the acquisition beam is determined as the sound source position 102A, and this is output to the multiplexing unit 28 as position information 27A.

  The signal adder 26 in FIG. 2A synthesizes the two acquisition beams output from the signal selector 25 in proportion to each other according to the volume, and synthesizes the synthesized digital audio signal 27B to the multiplexer 28. send. Thus, by always synthesizing two or more acquisition beams, it is possible to smoothly move the voice even when the speaker moves or is switched.

  The multiplexing unit 28 in FIG. 2A multiplexes the position information 27A generated by the signal selection unit 25 and the audio signal 27B generated by the signal addition unit 26, and performs transmission 20 to the reception side unit 1B.

  Next, the internal configuration of the receiving unit 1B will be described with reference to FIG. The reception side unit 1B includes a reception unit 31, a parameter calculation unit 32, a virtual sound source generation signal processing unit 33, a DAC 34i (i = 1 to N), and an AMP 35i (i = 1 to N). Speakers SPi (i = 1 to N) are connected. Each configuration will be described below.

The reception unit 31 in FIG. 2B receives the reception signal 30 for the position information 27A and the audio signal 27B from the transmission-side unit 1A. The audio signal 27B is sent to the virtual sound source generation signal processing unit 33, and the position information 27A is sent to the parameter calculation unit 32.
The parameter calculation unit 32 sets a virtual sound source VS (see FIG. 1B) based on the position information 27A, and a distance 36i (i = 1 to N) from the position of the virtual sound source VS to the position of each speaker SPi. ). Then, the delay amount parameter is set by dividing the distance 36i by the speed of sound.

The virtual sound source generation signal processing unit 33, the DAC 34i (i = 1 to N), and the AMP 35i (i = 1 to N) in FIG. 2B will be described.
Based on the parameters set by the parameter calculation unit 32, the virtual sound source generation signal processing unit 33 converts the audio received by the reception unit 31, that is, the audio signal 27B generated by the multiplexing unit 28 of the transmission side unit 1A, into the speaker SPi. Signal processing is performed for each output system (i = 1 to N) and output to the DAC 34i (i = 1 to N). Although the audio signal 27B input to the receiving unit 31 is one system, the audio signal 27B is subjected to delay processing for each output system of the speaker SPi based on the parameter (delay amount setting) of the parameter calculation unit 32. As a result, a sound emitting beam with the virtual sound source VS as the focal point can be output from the speaker SPi, and sound image localization as if there is a virtual sound source in front of the receiving side unit 1B can be realized.

The DAC 34i converts the audio digital data obtained by the virtual sound source generation signal processing unit 33 into an analog signal and outputs the analog signal.
The AMP 35i (i = 1 to N) amplifies the output of the DAC 34i (i = 1 to N), respectively, and outputs the amplified output to the speaker SPi (i = 1 to N).

Here, a supplementary explanation will be given for the search refinement by the signal selection unit 25 of the transmission-side unit 1A. For example, the signal selection unit 25 may be performed only when the acquisition beam forming unit 22 determines that the speaker is stationary for a specific time. This is because there is a case where there is no time for performing this narrowing search due to the problem of processing capability. In this case, the generation of the audio signal 27B is processed as follows (A) to (B).
(A) When the position of the sound source stays in one of the roughly set sound collection areas 111 to 114 for a specific time, the acquisition beam forming unit 22 detects that the control signal 241 is constant for a predetermined period, A plurality of acquisition beams directed to subdivision sound collection areas (corresponding to 131 to 134 in FIG. 5A) further inside the sound collection area selected by the beam position calculation unit 24 are formed.
(B) When the position of the sound source moves between the sound collection areas 111 to 114 set roughly, the acquisition beam forming unit 22 detects the fluctuation of the control signal 241 and sends it to the acquisition beam forming unit 22. The acquisition beam directed to the sound collection areas 111 to 114 in which a plurality of coarse ranges similar to the detection beam forming unit 21 are set is formed. The acquisition beam in this case is directed to a sound collection area in a rough range in the same manner as the sound collection beam.

In either of these cases (A) and (B), the signal selection unit 25 selects two of the plurality of acquisition beams output from the acquisition beam forming unit 22 and outputs a signal with a high volume. The signal adding unit 26 synthesizes these acquisition beams by proportionally distributing them according to the volume.
In addition, the signal adding unit 26 in FIG. 2A does not simply synthesize two high-volume sounds when the sound source moves greatly, but fades out the sound before switching, and the sound after switching. May be faded in to crossfade the sound. When the sound source moves greatly as described above, the sound can be naturally connected. During the time for performing this crossfade, the transmitting unit 1A and the receiving unit 1B transmit and receive the interpolation positions of the sound collection areas 111 to 114, and the crossfading start and end times. The parameter calculation unit 32 also moves gradually within the crossfade time for the position VS of the virtual sound source, and sets parameters based on this movement. This makes the movement of the sound image natural.

  In addition, when the position of the sound source moves between the sound collection areas 111 to 114 set roughly, a plurality of rough ranges similar to those of the detection beam forming unit 21 are set in the acquisition beam forming unit 22, but in this case, The position information 27 </ b> A output from the signal selection unit 25 is equivalent to the control information 241 that is position information output from the beam position calculation unit 24. Therefore, instead of the position information 27A output by the signal selection unit 25, the control information 241 obtained by the beam position calculation unit 24 may be output to the multiplexing unit 28 as 27C (see FIG. 2A). .

The parameter setting method in the parameter calculation unit 32 of FIG. 2 will be specifically described with reference to FIG. FIG. 3 is a conceptual diagram regarding this setting method.
First, as shown in FIG. 3A, the distance 36i between the virtual sound source VS and each speaker SPi of the receiving unit 1B is calculated. Next, as shown in FIG. 3B, parameters of the parameter calculation unit 32 are set. For example, a value obtained by dividing the distance 361 by the speed of sound V for SP1.

  As described above with reference to FIGS. 2 and 3, the reception-side unit 1B can transmit the audio signal 27B of the transmission-side unit 1A without transmitting the audio system for the number N of transmission source microphones Mi. A sound field can be accurately reproduced on the speaker SPi (i = 1 to N) based only on the position information 27A.

<Example of sound collection by the transmitting device>
Hereinafter, with reference to FIG. 4 and FIG. 5, the sound collection example for the detection beam forming unit 21 described with reference to FIG.

  With reference to FIG. 4, a description will be given of a method in which the detection beam forming unit 21 forms a sound collection beam directed to the sound collection areas 111 to 114. FIG. 4 shows virtual calculation tables 2111 to 2114 in order to explain a method of adding a delay to the inputs of X1 to XN. The sound collecting beams MB1 to MB4 to be formed are synthesized by adding a predetermined delay to the inputs of X1 to XN. In the ROM (not shown) of the detection beam forming unit 21, delay pattern data 2121 to 2124 are recorded corresponding to the sound collection areas 111 to 114 so that the sound collection beam is directed to the sound collection area. . The detection beam forming unit 21 sets the delay amount to the delay Dji (j = 1 to 4, i = 1 to N) based on the delay pattern data 2121 to 2124. The microphone input synthesizers 2141 to 2144 add the signals obtained by passing the inputs of the delays X1 to XN through Dj1 to DjN for each of the calculation tables 2111 to 2114, and output the collected sound beams MB1 to MB4.

  In the above description, the calculation tables 2111 to 2114 are shown for each of the sound collecting beams MB1 to MB4 for the sake of explanation. However, in terms of mounting, it is easy to configure D11 to D41 together with a single ring buffer memory or the like. is there. X1 is input to this ring buffer, and output taps D11 to D41 are provided. Similarly, D1i to D4i (i = 2 to N) each constitute a ring buffer for inputting one Xi, and output taps D1i to D4i are provided. Further, in terms of mounting, it is preferable to prepare delay pattern data 2121 to 2124 for each input Xi (i = 1 to N) instead of for each of the sound collecting beams MB1 to MB4.

  Next, a method for detecting a sound source position will be described with reference to FIG. The detection beam forming unit 21 simultaneously forms sound collection beams directed to the plurality of sound collection areas 111 to 114. The beam position calculation unit 24 compares the volume levels of the sound collection beams and monitors whether there is a speaker in any one of the sound collection areas.

  In FIG. 5 (A), the sound signals picked up by the microphones Mi (i = 1 to N) are synthesized by delaying only an appropriate delay time so as to be directed to the sound collection areas 111 to 114. Hereinafter, a method of setting a delay time set for each microphone (that is, a delay time for delaying sound collected by each microphone) will be described. Here, a sound collection beam that converges in the sound collection area 111 will be described as an example. As shown in FIG. 5A, a distance 121i (i = 1 to N) from the center of the sound collection area 111 to the microphone Mi (i = 1 to N) is calculated. Dividing this distance by the speed of sound (340 m / sec) calculates the propagation time of the sound wave from the sound collection area 111 to each microphone. Among the calculated propagation times, the longest one is used as a reference time, and the difference between the reference time and the propagation time to each microphone is set as a delay time set for the microphone. This delay time is set in the detection beam forming unit 21.

  By delaying and synthesizing the sound signal collected by each microphone with the delay time set in this way, the sound propagated from the sound collection area 111 to each microphone can be synthesized with the same phase. The sound in the sound area 111 can be output with a high gain. On the other hand, since the sound propagated from other areas is synthesized with the phase shifted, the amplitude is canceled and the gain becomes low.

  The sound collection beam that converges in the sound collection area 111 has been described above, but the same applies to the sound collection beams that converge in the sound collection areas 112, 113, and 114.

Next, the calculation method of the beam position calculation unit 24 illustrated in FIG. 2A will be described more specifically with reference to FIG. In the following description, it is assumed that the speaker on the transmission side is present at the position 102A (in the subdivided sound collection area 133 in the sound collection area 114) shown in FIG. Further, it is assumed that the waveform of the sound source generation signal has a shape indicated by reference numeral 240.
First, the detection beam forming unit 21 sets the sound collection areas 111 to 114 and searches for the position of the speaker 102A on the transmission side. If a speaker is present at 102A in FIG. 5A, the volume of the sound collection beam MB′4 directed to the sound collection area 114 is the highest. On the other hand, the sound collection beam directed to the sound collection areas 113, 112, and 111 moving away from the sound collection area 114 has a small volume as shown in the waveform 243 → the waveform 242 → the waveform 241. Therefore, the beam position calculation unit 24 selects the sound collection area (114 in FIG. 5) corresponding to the sound collection beam MB′1 to MB′4 shown in FIG.
Similarly to the beam position calculation unit 24, the signal selection unit 25 selects and outputs the acquisition beam output with the highest volume. However, the signal selection unit 25 selects two acquisition beams in order from the one with the largest volume among the subdivided sound collection areas 131 to 134 set by the acquisition beam forming unit 22. The acquisition beam is a beam corresponding to the subdivided sound collection areas 133 and 134 in FIG.

  With the configuration described above, it is possible to form a sound collection beam MBj directed in the direction of the sound collection area 11j as shown in FIG. The acquisition beam forming unit 22 shown in FIG. 2A can also have the same configuration.

<Specific usage and sound field reproduction>
A specific usage pattern of the apparatus of the present embodiment and an example of sound field reproduction will be described with reference to FIG. FIG. 6 is a diagram showing this specific usage pattern. As described above with reference to FIG. 1, the remote conference device 1 has a function of one unit for connecting to each other via a communication line or a network, and two or more remote conferences equivalent to this are provided. The devices are connected to each other via a network.

  Since the two transmission side units 1A and the reception side unit 1B shown in FIGS. 6A and 6B transmit and receive the position information together with the audio signal, the same number of speakers SPi or the microphone Mi are not required. Although the sound field of the speaker 102A on the transmission side can be transmitted, the following description will be made assuming that the remote conference device 1 having the same configuration is used. For the sake of simplicity, in the following description, two-dimensional coordinates on the XY plane are used, and the positions of the centers of the transmission side unit 1A and the reception side unit 1B are the coordinates of the origin as shown in FIG.

  As shown in FIG. 6A, it is assumed that the X and Y coordinates of the transmitting speaker 102A are (X1, -Y1). In this embodiment, when viewed from the speaker, the right side of the transmitting unit 1A and the receiving unit 1B is in the X plus direction (that is, the left direction in FIG. 6A and the right side in FIG. 6B). The right direction is the X plus direction). The transmission-side unit 1A of the remote conference apparatus 1 analyzes the voice signal emitted from the transmission-side speaker 102A and obtained by the microphone Mi to obtain the position information of the transmission-side speaker 102A. This position information can be obtained by analyzing the audio signal of the microphone Mi (i = 1 to N) as described with reference to FIGS. In the following description of FIG. 6, as a result of obtaining this position information, it is assumed that the transmitting speaker 102A is at a positive position on the X axis of the transmitting speaker 102A as shown in FIG. An explanation will be given.

  As shown in FIG. 6B, from the viewpoint of the listener 102B on the reception side, it is natural that the speaker 102A on the transmission side is conversing face to face. The value of the X coordinate at the position where the alternative virtual sound source VS should be set is -X1 to the left of the origin, and the value of the Y coordinate is Y1 because it is behind the depth side. As a result, the coordinates of the position where the virtual sound source VS should be set are (−X1, Y1). Then, as described above with reference to FIG. 3B, the parameter calculation unit 32 in FIG. 2B performs, for each speaker SPi (i = 1 to N), based on the position of the virtual sound source VS. Calculate the delay and volume parameters. Then, as shown in FIG. 2, this is set in the virtual sound source generation signal processing unit 33, and the speaker SPi (i = 1 to N) via the DAC 34i (i = 1 to N) and AMP 35i (i = 1 to N). ) To output as audio.

It supplements about description of the above embodiment.
In the above description, for ease of explanation, the transmission side unit 1A has been described as performing transmission. However, the transmission side units 1A and 1B do not perform only transmission or only reception. The remote conference apparatus 1 has a function of performing two-way communication. The voice spoken by the listener 102B on the receiving side and the position information of the listener 102B on the receiving side are acquired using the receiving unit 1B. To the transmitting unit 1A. Its function is the same as described above.

  Further, in the above description, the transmitting side speaker 102A is one and described as one sound source, but a plurality of speakers may be present. In this case, a plurality of pieces of position information are prepared, and the sound volume at each position is detected and transmitted. The receiving unit performs the parameter setting as shown in FIG. 3 described above for each speaker, and adds the outputs of the speakers SPi (i = 1 to N) calculated based on the parameters for each unit.

  In FIGS. 2 and 5, the number of sound collecting beams, acquisition beams, and sound collecting areas that are output simultaneously is four, but any of these may be more or less. The number of sound collecting beams and acquisition beams formed simultaneously is not limited to four. Further, if the sound collecting beams are output not in the same time but in a time division manner and the sound volumes are compared, not only two sound collecting beams and acquisition beams but one is sufficient.

  In addition, when assuming a case where there are a plurality of speakers, when narrowing down such a staged position, at least in the first stage, instead of narrowing down the speaker position to one, a plurality of candidates May be further searched for each neighborhood.

  In the above description, the number of speakers SPi and microphones Mi (i = 1 to N) is the same N, but it is not necessarily the same. In the transmission path, unlike the case of Patent Document 2, transmission is performed using the position information of the sound source, so even if these numbers are different, the sound field of the transmission source can be reproduced.

  In this embodiment, the detection beam forming unit 21 and the acquisition beam forming unit 22 set the sound collection beam and the acquisition beam in two stages, and search for the speaker 102A on the transmission side. You may search by setting the area to narrow down.

  Even without providing the acquisition beam forming unit 22, the position information 27C can be detected and output only by the beam position calculating unit 24. It is possible to detect which of the sound collection areas 111 to 114 includes the sound source 102A by comparing the collected sound beams even with the beam position calculation unit 24 alone. Further, the beam position calculation unit 24 can simply detect the acquisition beam having the maximum sound volume and can detect the position information 27C corresponding to the acquisition beam, without necessarily obtaining by proportional distribution.

  Furthermore, in the above description, the transmission side unit 1A in FIG. 2 transmits the sound collection beam or the acquisition beam as the audio signal 27B, but may simply transmit the output of any of the microphones M1 to MN. . Moreover, you may transmit what added any or all of microphone M1-MN. In addition, any audio signal may be used as long as the collected sound signals output from the microphones M1 to MN are used. Even in this case, the receiving-side unit 1B can accurately reproduce the sound field of the transmission destination while suppressing the consumption of transmission line resources based on the position information 27A or 27C. However, the noise resistance performance of the remote conference apparatus 1 is improved by transmitting the sound collection beam or acquisition beam toward the speaker 102A.

It is the external view of the remote conference apparatus used for the voice communication of this embodiment, and a schematic function diagram. It is a block diagram showing the internal structure of the remote conference apparatus used for the voice communication of this embodiment. 5 is a conceptual diagram of a parameter setting method in a parameter calculation unit 32. FIG. The internal block diagram of the beam forming part for a detection of the teleconference apparatus used for the audio | voice communication of this embodiment is shown. It is explanatory drawing for pinpointing a speaker's position. It is a figure explaining a specific usage pattern and a sound field reproduction example.

Explanation of symbols

1-teleconference device, 1A-transmission side unit, 1B-reception side unit,
101-desk, 102A-speaker on transmitting side, 102B-listener on receiving side,
111-114-sound collection area, 12ji (j = 1-4, i = 1-N)-distance,
131-134-subdivided sound collection area, SPi (i = 1-N)-speaker,
Mi (i = 1 to N) -microphone, MBj (j = 1 to 4) -sound collecting beam,
VS-virtual sound source, 20-transmission, 21-beam forming unit for detection,
211j (j = 1 to 4) —calculation table,
212j (j = 1 to 4) -delay pattern data,
Dji (j = 1 to 4, i = 1 to N) -delay,
214j (j = 1 to 4)-microphone input synthesis unit,
22-Beam acquisition unit for acquisition, 23-BPF, 24-Beam position calculation unit,
241-control signal, 25-signal selection unit, 26-signal addition unit,
27A-position information, 27B-speech signal, 28-multiplexer, 29-A / D converter,
30-received signal, 31-receiver, 32-parameter calculator,
33-virtual sound source generation signal processing unit, 34i (i = 1 to N) -DAC,
35i (i = 1 to N) -AMP, 36i (i = 1 to N) -distance

Claims (2)

A microphone array in which a plurality of microphones that collect sound and output as sound collection signals are arranged in an array;
A sound collection beam forming unit that respectively forms a sound collection beam directed to a plurality of sound collection areas by delay-adding the sound collection signals output from the plurality of microphones;
Position information detection means for detecting, as position information, a sound collection area corresponding to a sound collection beam indicating the maximum volume among the plurality of sound collection beams;
Transmitting means for transmitting the output of the sound pickup signal of the microphone and the position information;
An array speaker in which a plurality of speakers are arranged in an array; and
A receiving unit for receiving an audio signal and position information from the outside;
A signal processing unit that processes the received audio signal so as to form a sound emission beam having a position determined based on the received position information as a virtual sound source position and supplies the sound output beam to the plurality of speakers;
Equipped with a,
The position information detecting means detects a sound collecting area of the sound collecting beam having the maximum sound volume, and then obtains an acquisition sound collecting beam directed to a plurality of subdivided sound collecting areas obtained by further subdividing the sound collecting area. A teleconferencing device that detects the position information based on subdivided sound collection areas corresponding to a plurality of sound collection beams for acquisition that are formed and selected in descending order of volume of the sound collection beam for acquisition . The position information detecting means is arranged between subdivided sound collection areas corresponding to a plurality of acquisition sound collection beams selected in descending order of the volume of the acquisition sound collection beam according to the strength of the selected acquisition sound collection beam. And detecting the position information by proportional distribution,
The output of the previous SL multiple selected acquisition sound beam synthesized by the proportional distribution, teleconference according to claim 1, further comprising a signal adding means for transmitting to said transmission means as an output of the sound collection signal of the microphone apparatus.

Images