JP2011077649A - Sound collecting device, gain control method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound collecting device, a gain control method, and a program. <P>SOLUTION: The sound collecting device includes: a signal acquisition part which forms directivity to acquire a signal for every orientation direction; a recording part in which the orientation direction and a signal threshold are recorded by being associated with each other; a voice detection part which detects a voice signal from the signal acquired by the signal acquisition part; a determination part which determines whether or not a signal level of the voice signal detected by the voice detection part exceeds a signal threshold corresponding to the orientation direction in which the voice signal is acquired; and a gain control part which performs gain adjustment of the voice signal exceeding the signal threshold to a first signal level based on a determination result by the determination part, wherein the gain control part does not perform the gain adjustment of a voice signal below the signal threshold, or performs the gain adjustment of the voice signal below the signal threshold to a second signal level lower than the first signal level. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a sound collection device, a gain control method, and a program.

  Recently, research on telecommunications systems for users to talk with other users in remote locations has been actively conducted. In this remote communication system, a user's voice is collected by a sound collection device including a microphone and the sound collected by the sound collection device is transmitted to another user existing in a remote place.

  Here, when the microphone is omnidirectional, not only the user's voice but also noise and reverberation are collected, so that it becomes difficult to hear the user's voice. In order to solve this problem, there has been proposed a sound collecting device that forms directivity in a plurality of directions using a plurality of microphones and limits the sound collecting direction to a direction in which sound exists among the plurality of directional directions. For example, in Patent Document 1, directivity is formed using a microphone array, a maximum direction is detected from a directivity direction in which a signal level is equal to or higher than a predetermined level, and a directivity aim is aligned with the detected direction. A method for picking up sound is disclosed.

  In addition, when performing a video conference between multiple points or a conference call in which three or more users can talk simultaneously, since the microphone state and the distance between the microphone and the user differ from place to place, an audio signal obtained by sound collection The signal level also varies from place to place. With respect to this problem, Patent Document 2 discloses an automatic gain control method and an automatic gain control apparatus that adaptively control the signal level of an audio signal.

  Further, Patent Document 3 discloses a detection unit that detects a sound source direction, a sound collection unit that collects sound emitted from the detected sound source direction, and an output level that is constant every time the directional direction is switched. An apparatus including gain adjusting means for automatically adjusting gain is disclosed.

  The automatic gain control is generally control for amplifying a low level signal and limiting an excessive level signal to an appropriate level. With regard to the application of this automatic gain control, Patent Document 2 discloses that automatic gain control is applied to a speech section in which speech is detected so that low-level noise that is not speech is not amplified.

JP 2003-304589 A JP-A-8-250944 Japanese Patent Laid-Open No. 9-140000

  Here, the voice uttered by the user includes a voice as an utterance to be clearly communicated to other users and a low-volume voice such as a tweet, singing or chatting with a neighboring user. When the signal level of speech as speech is low, it is desirable to amplify the volume to a level that is easy for a remote user to hear, but it is not appropriate to amplify other speech such as singing or chatting by automatic gain control. However, the conventional apparatus has a problem that not only the voice as a speech but also other voices are amplified to a predetermined level.

  Therefore, the present invention has been made in view of the above problems, and the object of the present invention is to provide a new and improved sound collection device capable of performing gain control according to the type of sound, A gain control method and program are provided.

  In order to solve the above-described problem, according to an aspect of the present invention, a signal acquisition unit that forms directivity and acquires a signal for each direction, and a recording unit that records the direction and signal threshold in association with each other A voice detection unit that detects a voice signal from the signal acquired by the signal acquisition unit, and a signal in which the signal level of the voice signal detected by the voice detection unit corresponds to the pointing direction in which the voice signal is acquired A determination unit configured to determine whether or not the threshold value is exceeded; and a gain control unit configured to adjust a gain of an audio signal exceeding the signal threshold value to a first signal level based on a determination result by the determination unit. The sound collection device is provided in which the unit does not adjust the gain of the audio signal below the signal threshold, or adjusts the gain to a second signal level lower than the first signal level.

  The sound collection device further includes a learning unit that learns a signal threshold recorded in the recording unit in association with a directivity direction based on a speaker's prior utterance, and the learning unit is based on the preliminary utterance. You may learn the signal threshold value corresponding to the directivity direction from which the audio | voice signal was acquired based on the signal level of the said audio | voice signal acquired by the said prior speech.

  The sound collection device includes a position detection unit that detects a distance between the speaker and the sound collection device based on utterance by the speaker, and a signal threshold value corresponding to a directivity direction in which an audio signal is acquired based on the utterance And a learning unit that sets the value according to the distance between the speaker detected by the position detection unit and the sound collection device.

  The learning unit may set the signal threshold to a smaller level as the distance between the speaker detected by the position detection unit and the sound collection device is longer.

  The sound collection device further includes a plurality of sets of microphone arrays arranged separately from each other, and the signal acquisition unit acquires a signal for each directivity direction from each of the sound collection signals by the plurality of sets of microphone arrays, The position detection unit is configured to detect the sound source and the speaker based on a directivity direction in which an audio signal based on the utterance is acquired from each of the sound pickup signals by the plurality of sets of microphone arrays and an arrangement interval of the plurality of sets of microphone arrays. You may detect the distance with a sound apparatus.

  In order to solve the above problem, according to another aspect of the present invention, a step of associating and recording a directivity direction and a signal threshold value, a step of forming a directivity and acquiring a signal for each directivity direction, Detecting an audio signal from the acquired signal, and determining whether a signal level of the detected acquired audio signal is higher than a signal threshold corresponding to a directivity direction in which the audio signal is acquired. And adjusting the gain of an audio signal that exceeds the signal threshold to a first signal level, and no gain adjustment is performed for the audio signal that is below the signal threshold, or the first The gain may be adjusted to a second signal level lower than the signal level.

  In order to solve the above problem, according to another aspect of the present invention, a computer is configured to associate a directivity direction with a signal threshold value, a signal acquisition unit that forms a directivity and acquires a signal for each directivity direction. A recording unit that is recorded, a voice detection unit that detects a voice signal from the signal acquired by the signal acquisition unit, and a signal level of the voice signal detected by the voice detection unit is acquired from the voice signal. A determination unit configured to determine whether or not a signal threshold value corresponding to a directivity direction is exceeded; and based on a determination result by the determination unit, an audio signal that exceeds the signal threshold value is gain-adjusted to a first signal level, and the signal threshold value There is provided a program for functioning as a gain control unit that does not adjust the gain of an audio signal lower than or equal to a second signal level lower than the first signal level.

  As described above, according to the present invention, it is possible to perform gain control according to the type of sound.

It is explanatory drawing which showed the structure of the telecommunication system by embodiment of this invention. It is a functional block diagram showing the configuration of the remote communication device according to the first embodiment. It is explanatory drawing which showed the principle of the beam former. It is explanatory drawing which showed arrangement | positioning of a microphone. It is explanatory drawing which showed the specific structure of the directivity formation part. It is explanatory drawing which showed the specific example of the content recorded on a speaker characteristic recording part. It is explanatory drawing which showed the specific example of the gain adjustment by a gain control part. It is explanatory drawing which showed the specific example of the gain adjustment by a gain control part. It is the flowchart which showed operation | movement of the remote communication apparatus by 1st Embodiment. It is the functional block diagram which showed the structure of the remote communication apparatus by 2nd Embodiment. It is explanatory drawing which showed the positional relationship of a sound source position (user position) and a microphone array. It is the flowchart which showed the operation | movement of the remote communication apparatus by 2nd Embodiment.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  In the present specification and drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numeral. For example, a plurality of configurations having substantially the same functional configuration are differentiated as necessary, such as the remote communication devices 10A and 10B. However, when it is not necessary to particularly distinguish each of a plurality of constituent elements having substantially the same functional configuration, only the same reference numerals are given. For example, when it is not necessary to distinguish between the remote communication devices 10A and 10B, they are simply referred to as the remote communication device 10.

(Telecommunication system)
First, a telecommunications system 1 according to an embodiment of the present invention will be described with reference to FIG. FIG. 1 is an explanatory diagram showing the configuration of a telecommunications system 1 according to an embodiment of the present invention. As shown in FIG. 1, the remote communication system 1 according to the embodiment of the present invention includes remote communication devices 10 </ b> A and 10 </ b> B and a communication network 12.

  The remote communication device 10 </ b> A and the remote communication device 10 </ b> B are remotely arranged in different buildings or different rooms and are connected via the communication network 12. The communication network 12 is a wired or wireless transmission path for information transmitted from a device connected to the communication network 12. For example, the communication network 12 may include a public line network such as the Internet, a telephone line network, a satellite communication network, various local area networks (LANs) including the Ethernet (registered trademark), a wide area network (WAN), and the like. . Further, the communication network 12 may include a dedicated line network such as an IP-VPN (Internet Protocol-Virtual Private Network).

  The remote communication device 10 has functions as a sound collection device and an image pickup device, and transmits the collected sound and the picked-up video to another remote communication device 10 via the communication network 12. Further, the remote communication device 10 receives audio and video from another remote communication device 10 via the communication network 12, outputs the received audio from a speaker or the like, and displays the video on a display.

  As a result, a user who uses the remote communication device 10 can communicate with a user in a remote location using audio and video. Specifically, in the example shown in FIG. 1, the users A to C can communicate with the users D and E who use the remote communication device 10B by using the remote communication device 10A.

  As described above, the remote communication device 10 can process both video and audio. However, in the remote communication device 10 according to the present embodiment, voice gain control is an important point. Will be described with emphasis on voice gain control.

(Organizing the background)
Recently, research on video conferencing apparatuses that collect voices uttered by users and automatically amplify the collected voices to a certain level and transmit them has been conducted. Here, the voice uttered by the user includes a voice as a speech to be clearly communicated to other users and a low-volume voice such as a tweet, a talk, a chat with a neighboring user, and a consultation. When the signal level of speech as speech is low, it is desirable to amplify the volume so that it is easy for a remote user to hear. However, it is not appropriate to amplify other speech such as singing or chatting by automatic gain control.

  On the other hand, voices such as singing and chatting are also useful information for knowing the atmosphere and situation of a remote place, so they may be removed or suppressed as unnecessary information in the same way as noise other than voice. It's not a good idea.

  Therefore, the remote communication device 10 according to the present embodiment has been created against the background of the above circumstances. According to the remote communication device 10 according to the present embodiment, it is possible to listen to speech as an utterance at an appropriate volume and to extract other speech without being suppressed. Hereinafter, the remote communication device 10 will be described in detail.

(First embodiment)
FIG. 2 is a functional block diagram showing the configuration of the remote communication device 10 according to the first embodiment. As shown in FIG. 2, the remote communication device 10 includes a microphone array 100 including microphones 100-1 to 100-m, a directivity forming unit 101, a voice detection unit 102, a speech level learning unit 103, and a speech. A person characteristic recording unit 104, a speech determination unit 105, a gain control unit 106, and a communication unit 107 are provided. Note that each functional block shown in FIG. 2 need not be provided in one container, and may be provided separately in different containers.

  The microphones 100-1 to 100-m and the directivity forming unit 101 function as a signal acquisition unit that forms directivity and acquires a sound signal emitted from the directivity direction for each directivity direction. Specifically, the microphones 100-1 to 100-m and the directivity forming unit 101 form directivity by performing signal processing based on the principle of the beam former.

  Note that the directivity forming method is not limited to this example. For example, i microphones having directivity are arranged so that the directivity of each microphone is directed in the center direction of each arc obtained by equally dividing the circumference into i. By doing so, the directivity may be formed. However, since this forming method has many restrictions such as the necessity of microphones for the necessary direction, in this embodiment, the directivity is formed by performing signal processing based on the principle of the beam former. According to the principle of this beam former, directivity can be formed in an arbitrary direction by using omnidirectional microphones arranged at two or more different positions. The directivity forming method according to this embodiment will be described below together with the principle of the beam former.

  FIG. 3 is an explanatory diagram showing the principle of the beam former. In the example shown in FIG. 3, the two microphones M1 and M2 are installed with a distance L apart. In this case, the sound wave coming from the θ direction first reaches the microphone M1 close to the sound source, and then travels a distance d to reach the microphone M2. Here, the distance d is expressed as the following Equation 1.

Therefore, the sound reception signal x ₂ (t) of the microphone M2 is the time required for the sound wave to travel the distance d with respect to the sound reception signal x ₁ (t) of the microphone M1, as shown in Equation 2 below. This signal is delayed by τ.

From Equation 2, the delay unit 22 gives a delay of time τ to x ₁ (t), and the adder 24 adds the delayed x ₁ (t) and x ₂ (t) as shown in Equation 3 below. For example, in-phase signals are added together, so that the signal component from the θ direction is emphasized. When the microphone M2 is closer to the sound source, a delay is given to the sound reception signal x ₂ (t) of the microphone M2, contrary to the above.

Signal processing for forming such directivity in the time domain can be similarly performed in the frequency domain. The Fourier transform result of the signal whose time axis is delayed by τ is ^obtained by multiplying the result of Fourier transform of the signal before the delay by e ^−jωτ . Therefore, when the Fourier transform results of b (t), x ₁ (t), and x ₂ (t) are B (ω), X ₁ (ω), and X ₂ (ω), the above Equation 3 is It is expressed as follows on the area.

  When forming a beam former in the time domain, the delay time τ can be selected only in units of sampling periods in digital processing, but the delay time τ can be arbitrarily selected in the frequency domain, so the delay time τ is changed. Thus, it is possible to arbitrarily determine the directivity direction (enhancement direction).

  The principle of the beam former has been described above with reference to FIG. Next, with reference to FIGS. 4 and 5, the microphones 100-1 to 100-m and the directivity forming unit 101 that form directivity in the frequency domain based on the principle of the beam former will be described in detail.

FIG. 4 is an explanatory diagram showing the arrangement of the microphones 100-1 to 100-m. As shown in FIG. 4, the microphones 100-1 to 100-m are linearly arranged at equal intervals (distance L intervals). The received sound signals x ₁ (n) to x _m (n) from the microphones 100-1 to 100 -m are converted into digital signals by an AD conversion unit (not shown), and then transmitted to the directivity forming unit 101. Entered.

In the example shown in FIG. 4, the arrival of sound waves to the microphones 100-2, 100-3,..., 100-m corresponds to τ, 2τ, ..., (m-1) τ, delayed. For this reason, the sound signals x ₁ (n), x ₂ (n),... Received by the microphones 100-1, 100-2,. It becomes. Furthermore, by adding the received sound signals in phase, only the signal coming from the θ direction can be emphasized. If the number of microphones 100 is increased, the number of received sound signals to be added is also increased, so that the gain with respect to the pointing direction is improved. The directivity forming unit 101 according to the present embodiment realizes the above principle in the frequency domain.

  FIG. 5 is an explanatory diagram showing a specific configuration of the directivity forming unit 101. As illustrated in FIG. 5, the directivity forming unit 101 includes an FFT (Fast Fourier Transform) 301-1 to 301-m, a delay control unit 302, multiplication units 303-1 to 303-m, and an addition unit 304. And an inverse FFT 305.

The FFTs 301-1 to 301 -m receive the received sound signals x ₁ (n) to x _m (n) converted into digital signals, and perform fast Fourier transform to receive the received sound signals x ₁ (n) to x _m (n) is converted into a frequency domain spectrum X ₁ (ω) to X _m (ω).

The spectra X ₁ (ω) to X _m (ω) are supplied to the multipliers 303-1 to 303-m, and the multipliers 303-1 to 303-m multiply the delay coefficients. This delay coefficient is given to each of the multipliers 303-1 to 303-m by the delay controller 302 in accordance with the direction of directivity to be formed.

Specifically, as shown in FIG. 4, when the arrival time difference of the sound wave generated by the distance difference d expressed by Lsin θ is τ, the delay control unit 302 sends e to the multiplication units 303-1 to 303-m. ^{−jω (m−1) τ} , e− ^{jω (m−2) τ} ,..., E− ^jωτ , 1 are given as delay coefficients. In the case where directivity is formed in the downward direction in FIG. 4, the microphone 100-m is closest to the sound source, so that the delay control unit 302 has the largest delay coefficient in the multiplication unit 303-m, contrary to the above. give.

The adder 304 adds the multiplication results of the spectrums X ₁ (ω) to X _m (ω) and the delay coefficients by the multipliers 303-1 to 303-m, and collects the sound collection signal spectrum B ₁ (ω) for each directivity direction. ~ Bj (ω) is acquired. That is, in the present embodiment, it is assumed that sound collected signals from j directions are acquired. For example, a sound collected signal from the center direction of each arc obtained by equally dividing the circumference into j can be acquired. it can.

The inverse FFT 305 performs inverse Fourier transform on the collected sound signal spectrums B ₁ (ω) to Bj (ω) obtained by the adding unit 304, and acquires sound collected signals b _{0 to} b _j in the time domain. Thereby, the subsequent processing is performed in the time domain. Note that the directivity forming unit 101 may not be provided with the inverse FFT 305, and the collected sound signal spectra B ₁ (ω) to Bj (ω) may be output from the directivity forming unit 101. In this case, the subsequent processing is performed in the frequency domain.

As shown in FIG. 2, the sound detection unit 102 receives the sound collection signals b _{1 to} b _j for each directivity direction acquired by the directivity forming unit 101. Then, the voice detection unit 102 detects the presence or absence of a voice signal with respect to the collected sound signals b _{1 to} b _j corresponding to all directivity directions, and detects the detected voice signal and the signal level (for example, average amplitude) of the voice signal. Value) and direction information are output to the speech determination unit 105.

  Note that the detection method of the audio signal by the audio detection unit 102 is not particularly limited. For example, the voice detection unit 102 determines that voice input has started when the signal level of the collected sound signal is equal to or higher than a predetermined level for a predetermined time or more, and then the signal level of the collected sound signal is equal to or higher than a predetermined level for a predetermined time or longer It may be determined that the voice input is finished when the value is lower than.

  The speech level learning unit 103 learns a speech level (signal threshold value) for determining whether or not a speech signal is a speech based on a user's prior speech for each pointing direction, and sends the direction to the speaker characteristic recording unit 104. And the speech level are recorded in association with each other.

  Specifically, prior to the start of the conference, the remote communication device 10 is set to the speech level registration mode, and each user speaks a prescribed voice sequentially toward the microphone 100. When the user utters, a sound collection signal for each directional direction is supplied to the sound detection unit 102 via the microphone 100 and the directivity forming unit 101, and the sound detection unit 102 detects the user's sound signal from the sound collection signal. Then, the voice detection unit 102 supplies the signal level of the detected voice signal to the speech level learning unit 103 together with the directivity direction in which the voice signal is detected.

  The speech level learning unit 103 averages the signal level of the voice signal for each user by the speech time, and records the speech level corresponding to the average result in the speaker characteristic recording unit 104 in association with the direction.

  FIG. 6 is an explanatory diagram showing a specific example of the contents recorded in the speaker characteristic recording unit 104. As shown in FIG. 6, the speech level is recorded in association with each direction based on the prior speech by the user. In addition, a predetermined initial value is recorded in association with a direction in which no user voice is detected as shown in “direction 3”. The speech level may be the average result of the signal level of the audio signal by the speech level learning unit 103, or may be a value lower than the average result such as 70% of the average result in order to have a margin. Good.

  The speech determination unit (determination unit) 105 is supplied with the audio signal detected by the audio detection unit 102, the signal level of the audio signal, and direction information in the conference operation mode. The speech determination unit 105 reads the speech level corresponding to the direction indicated by the supplied direction information from the speaker characteristic recording unit 104, and determines whether the signal level of the audio signal exceeds the read speech level. Here, the voice signal determined to have reached the speech level is treated as speech, and the other speech is treated as monologue or chat.

  The gain control unit 106 adjusts the gain of the audio signal based on the determination result by the speech determination unit 105. Specifically, the gain control unit 106 adjusts the gain of the voice signal determined by the speech determination unit 105 to exceed the speech level so that the signal level becomes the speech transmission level (first signal level).

  On the other hand, the gain control unit 106 adjusts the gain of the audio signal determined by the speech determination unit 105 to be lower than the speech level to another speech transmission level (second signal level) lower than the speech transmission level. Alternatively, the gain control unit 106 may output the audio signal determined to be below the speech level by the speech determination unit 105 without adjusting the gain. Specific examples of such gain adjustment by the gain control unit 106 are shown in FIGS.

  7 and 8 are explanatory diagrams showing specific examples of gain adjustment by the gain control unit 106. FIG. As shown in the left diagram of FIG. 7, when the signal level of the audio signal detected from “direction 1” exceeds “speech level 1”, the gain control unit 106 is illustrated in the right diagram of FIG. 7. Thus, the signal level of the audio signal is amplified to the speech transmission level.

  On the other hand, as shown in the left diagram of FIG. 8, when the signal level of the audio signal detected from “direction 1” is lower than “speech level 1”, the gain control unit 106 displays in the right diagram of FIG. 8. As shown, the signal level of the audio signal is amplified to another audio transmission level lower than the speech transmission level.

  The audio signal gain-adjusted by the gain adjusting unit 106 as described above is transmitted to another remote communication device 10 by the communication unit 107. The communication unit 107 receives an audio signal transmitted from another remote communication device 10, and the remote communication device 10 has a configuration for outputting the audio signal received by the communication unit 107.

  The configuration of the remote communication device 10 according to the first embodiment has been described above. Next, the operation of the remote communication device 10 according to the first embodiment will be described with reference to FIG.

  FIG. 9 is a flowchart showing the operation of the remote communication device 10 according to the first embodiment. As shown in FIG. 9, first, the remote communication device 10 is set to the speech level registration mode, and each user utters in advance (S404). Then, the voice detection unit 102 detects the voice signal based on the prior utterance, and the utterance level learning unit 103 learns the utterance level for each pointing direction based on the signal level of the voice signal detected by the voice detection unit 102 ( S408).

  Subsequently, when a voice signal is detected by the voice detection unit 102 in the conference operation mode (S412), the speech determination unit 105 determines that the signal level of the voice signal detected by the voice detection unit 102 corresponds to the detection direction. It is determined whether or not the speech level is exceeded (S416).

  When it is determined that the signal level of the audio signal is higher than the speech level, the gain control unit 106 adjusts the gain of the signal level of the audio signal to the speech transmission level (S420). On the other hand, when it is determined that the signal level of the audio signal is lower than the speech level, the gain control unit 106 adjusts the signal level of the audio signal to another audio transmission level or does not perform gain adjustment (S424). .

  Subsequently, the communication unit 107 transmits the audio signal after the gain adjustment by the gain control unit 106 to another remote communication device 10 (S428). Thereafter, the processing from S412 is repeated.

(Effects of the first embodiment)
As described above, the remote communication device 10 according to the first embodiment includes the voice detection unit 102 that detects a voice signal from sound-collected signals from a plurality of directional directions, and the utterance level for each directional direction based on prior utterances by the user. The speech level learning unit 103 for learning the speech, the speech determination unit 105 for determining whether or not the speech signal detected by the speech detection unit 102 exceeds the speech level, and the speech signal exceeding the speech level is amplified to the speech transmission level. The gain control unit 106 adjusts the gain of another audio signal to another audio transmission level lower than the speech transmission level.

  Therefore, according to the remote communication device 10 according to the first embodiment, a user who listens to an appropriate volume level is not affected by the individual difference of the loudness of each user or the user position. It is possible to listen at. Furthermore, according to the remote communication device 10 according to the first embodiment, it is possible to convey a sound with a smaller volume than a speech while preventing a sound with a small volume other than a speech from being amplified to the same level as the speech. As a result, for example, speech such as monologue and chat is transmitted without being removed, so that the listening user can know the atmosphere and situation of the remote place from the speech such as monologue and chat.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. In the first embodiment, an example has been described in which the remote communication device 10 is set to the speech level registration mode prior to the start of the conference in order to learn the speech level. On the other hand, the remote communication device 10 ′ according to the second embodiment sequentially learns during the conference operation without providing a special operation mode when learning the speech level.

  Here, since the speech level of each user is not learned at the start of the conference, an initial reference value is required. However, since the speech level of each user varies greatly depending on the positional relationship between the remote communication device 10 'and each user, it is not optimal to give a uniform reference value.

  Therefore, in the second embodiment, the distance between each user and the remote communication device 10 'is detected, and a value corresponding to the detected distance is set as the initial value of the speech level. Hereinafter, the remote communication device 10 ′ according to the second embodiment for realizing such a function will be described in detail.

  FIG. 10 is a functional block diagram showing the configuration of the remote communication device 10 ′ according to the second embodiment. As shown in FIG. 10, the remote communication device 10 ′ according to the second embodiment includes voice direction detection units 620 and 621, a sound source position detection unit 607, a speech level learning unit 608, and a speaker characteristic recording unit 609. And a gain control unit 610 and a communication unit 611.

  The voice direction detection unit 620 includes a microphone array 601, a directivity forming unit 603, and a voice detection unit 605. The voice direction detection unit 621 includes a microphone array 602, a directivity formation unit 604, and a voice detection unit 606. Prepare.

  Similarly to the first embodiment, the microphone array 601 and the microphone array 602 are configured by a plurality of microphones arranged linearly, and are arranged with a known distance D therebetween.

  The directivity forming unit 603 acquires a sound collection signal for each directivity direction from a sound reception signal by the microphone array 601, and the sound detection unit 605 detects a sound signal from the sound collection signal for each directivity direction. Similarly, the directivity forming unit 604 acquires a sound collection signal for each directivity direction from the sound reception signal by the microphone array 602, and the sound detection unit 606 detects the sound signal from the sound collection signal for each directivity direction. Since the directivity formation method and the audio signal detection method have been described in the first embodiment, a detailed description thereof will be omitted.

  The sound source position detection unit 607 is based on the direction in which the sound signal is detected by the sound detection unit 605, the direction in which the sound is detected by the sound detection unit 606, and the distance D between the microphone array 601 and the microphone array 602. Is detected. Hereinafter, a specific description will be given with reference to FIG.

  FIG. 11 is an explanatory diagram showing the positional relationship between the sound source position (user position) and the microphone array 601 and microphone array 602. The sound source direction θ1 viewed from the microphone array 601 is detected by the sound detection unit 605, and the sound source direction θ2 viewed from the microphone array 602 is detected by the sound detection unit 606. The distance D between the microphone array 601 and the microphone array 602 is known. Here, the sound source direction θ 1, the sound source direction θ 2, and the distance D correspond to a triangular sandwiched side of a triangle having the microphone array 601, the microphone array 602, and the sound source as vertices.

  Therefore, the sound source position detection unit 607 can specify the sound source position according to the principle of triangulation based on the sound source direction θ1, the sound source direction θ2, and the distance D. For example, the sound source position detection unit 607 may specify the sound source position by substituting the sound source direction θ1, the sound source direction θ2, and the distance D into a predetermined mathematical expression. Alternatively, the sound source position may be specified based on a table created in advance showing the relationship between the sound source direction θ1, the sound source direction θ2, and the sound source position.

  Furthermore, the sound source position detection unit 607 calculates the distance between the sound source position and the remote communication device 10 ′ from the specified sound source position, and supplies it to the speech level learning unit 608. Note that the distance between the sound source position and the remote communication device 10 ′ may be the distance between the sound source position and the microphone array 601.

  The speech level learning unit 608 calculates the direction and the sound source position detection unit 607 when the direction in which the audio signal is detected by the audio detection unit 605 is the first direction, that is, the direction of a new user. The speech level corresponding to the distance is set in the speaker characteristic recording unit 609 in association with the speech level.

  Specifically, as the distance between the user and the remote communication device 10 ′ increases, the signal level of the audio signal in the remote communication device 10 is attenuated even if the user speaks at the same volume. Therefore, the speech level learning unit 608 sets the speech level to a smaller level as the distance calculated by the sound source position detection unit 607 is longer. More specifically, since the signal level of the sound wave is inversely proportional to the square of the travel distance, the speech level learning unit 608 sets the speech level so that it is inversely proportional to the square of the distance calculated by the sound source position detection unit 607. May be.

  In addition, when a speech signal is detected from a direction in which the speech level is already set, the speech level learning unit 608 sequentially updates the speech level according to the signal level of the speech signal. For example, the speech level learning unit 608 may raise the speech level by a level proportional to the difference between the signal level of the speech signal and the speech level when the signal level of the speech signal exceeds the speech level.

  In the present embodiment, the speech level learning unit 608 includes a function as a speech determination unit. That is, the speech level learning unit 608 determines whether or not the signal level of the audio signal detected by the audio detection unit 605 is higher than the speech level corresponding to the detection direction of the audio signal.

  Similarly to the gain control unit 106 of the first embodiment, the gain control unit 610 adjusts the gain of the audio signal determined by the speech level learning unit 608 to exceed the speech level so that the signal level becomes the speech transmission level. To do. On the other hand, the gain control unit 610 adjusts the gain of the audio signal determined to be below the speech level by the speech level learning unit 608 to another audio transmission level lower than the speech transmission level. Alternatively, the gain control unit 610 may output the speech signal determined to be below the speech level by the speech level learning unit 608 without adjusting the gain. The audio signal whose gain has been adjusted by the gain adjusting unit 106 as described above is transmitted to the other remote communication device 10 ′ by the communication unit 107.

  The configuration of the remote communication device 10 ′ according to the second embodiment has been described above. Next, the operation of the remote communication device 10 'according to the second embodiment will be described with reference to FIG.

  FIG. 12 is a flowchart showing the operation of the remote communication device 10 'according to the second embodiment. As shown in FIG. 12, when the audio signal is detected by the audio detection unit 605 and the audio detection unit 606 (S504), the sound source position detection unit 607 detects the audio signal by the audio detection unit 605 and the audio detection unit 606. Based on the determined direction, the sound source position is detected (S508).

  Also, the speech level learning unit 608, when the direction in which the audio signal is detected by the audio detection unit 605 is the first direction (S512), the sound source position detected by the sound source position detection unit 607, the remote communication device 10 ′, The speech level is set according to the distance (S516).

  On the other hand, when a speech signal is detected from a direction in which the speech level is already set, the speech level learning unit 608 determines whether the signal level of the speech signal exceeds the speech level (S520). If it is determined that the signal level of the audio signal exceeds the speech level, the gain control unit 610 adjusts the gain of the signal level of the audio signal to the speech transmission level (S524). If it is determined that the signal level of the audio signal is lower than the speech level, the gain control unit 610 adjusts the gain of the signal level of the audio signal to another audio transmission level lower than the speech transmission level (S528).

  Subsequently, the communication unit 611 transmits the audio signal after the gain adjustment by the gain control unit 610 to another remote communication device 10 '(S532). Thereafter, the processing from S504 is repeated.

(Effect of 2nd Embodiment)
As described above, according to the second embodiment, it is possible to learn and set a speech level during a conference operation without providing an operation mode such as a speech level learning mode in the remote communication device 10 ′. More specifically, it is possible to detect the distance between each user and the remote communication device 10 ′ and set a value corresponding to the detected distance as the initial value of the speech level.

  Although the preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

  For example, each step in the processing of the remote communication devices 10 and 10 ′ in this specification does not necessarily have to be processed in time series in the order described as the flowchart. For example, each step in the processing of the remote communication devices 10 and 10 ′ may be processed in an order different from the order described as the flowchart, or may be processed in parallel.

  It is also possible to create a computer program for causing hardware such as CPU, ROM and RAM incorporated in the remote communication devices 10 and 10 'to perform the same functions as the components of the remote communication devices 10 and 10' described above. It is. A storage medium storing the computer program is also provided.

10, 10 'Remote communication device 12 Communication network 100, 601, 602 Microphone array 101, 603, 604 Directivity forming unit 102, 605, 606 Voice detection unit 103, 608 Speech level learning unit 104, 609 Speaker characteristic recording unit 105 Statement determination unit 106, 610 Gain control unit 107, 611 Communication unit 301-1, 301-m FFT
302 Delay control unit 303-1, 303-m Multiplication unit 304 Addition unit 305 Inverse FFT
607 Sound source position detection unit

Claims (7)

A signal acquisition unit that forms directivity and acquires a signal for each direction;
A recording unit in which the pointing direction and the signal threshold are recorded in association with each other;
An audio detection unit for detecting an audio signal from the signal acquired by the signal acquisition unit;
A determination unit that determines whether a signal level of the audio signal detected by the audio detection unit exceeds a signal threshold value corresponding to a directivity direction in which the audio signal is acquired;
A gain control unit that adjusts a gain of an audio signal that exceeds the signal threshold to a first signal level based on a determination result by the determination unit;
With
The sound collection device, wherein the gain control unit does not perform gain adjustment of an audio signal that is lower than the signal threshold value, or performs gain adjustment to a second signal level lower than the first signal level. The sound collecting device includes:
A learning unit that learns a signal threshold value recorded in association with a directivity direction on the recording unit based on a speaker's prior utterance,
The learning unit learns a signal threshold corresponding to a directivity direction in which an audio signal is acquired based on the prior utterance based on a signal level of the audio signal acquired by the prior utterance. The sound collecting device described in 1. The sound collecting device includes:
A position detection unit that detects a distance between the speaker and the sound collection device based on the utterance by the speaker;
A learning unit that sets a signal threshold corresponding to a directivity direction in which an audio signal is acquired based on the utterance according to a distance between the speaker detected by the position detection unit and the sound collection device;
The sound collection device according to claim 1, further comprising:   The sound collection device according to claim 3, wherein the learning unit sets a signal threshold value to a smaller level as the distance between the speaker detected by the position detection unit and the sound collection device is longer. The sound collecting device includes:
Further comprising a plurality of sets of microphone arrays spaced apart,
The signal acquisition unit acquires a signal for each directivity direction from each of the collected sound signals by the plurality of microphone arrays,
The position detection unit is configured to determine whether the voice signal based on the utterance is acquired from each of the collected signals from the plurality of sets of microphone arrays and the speaker and the position based on the arrangement interval of the plurality of sets of microphone arrays. The sound collection device according to claim 3, wherein a distance from the sound collection device is detected. Recording the pointing direction and the signal threshold in association with each other;
Forming a directivity and acquiring a signal for each directivity direction;
Detecting an audio signal from the acquired signal;
Determining whether the signal level of the detected audio signal is above a signal threshold corresponding to the direction of orientation from which the audio signal was acquired;
Adjusting a gain of an audio signal that exceeds the signal threshold to a first signal level;
Including
A gain adjustment method in which gain adjustment is not performed for an audio signal that is less than the signal threshold, or the gain is adjusted to a second signal level lower than the first signal level. Computer
A signal acquisition unit that forms directivity and acquires a signal for each direction;
A recording unit in which the pointing direction and the signal threshold are recorded in association with each other;
An audio detection unit for detecting an audio signal from the signal acquired by the signal acquisition unit;
A determination unit that determines whether or not a signal level of the audio signal detected by the audio detection unit exceeds a signal threshold value corresponding to a directivity direction in which the audio signal is acquired;
Based on the determination result by the determination unit, the audio signal that exceeds the signal threshold is gain-adjusted to the first signal level, and the gain adjustment of the audio signal that is less than the signal threshold is not performed, or from the first signal level A gain controller for adjusting the gain to a low second signal level;
Program to function as

Images