JP6409378B2 - Voice communication apparatus and program - Google Patents

Description

  The present invention relates to a voice communication technique for transferring voice between users.

  A technology for suppressing noise components from an acoustic signal collected by a sound collection device in a voice communication device that transmits and receives an acoustic signal indicating speech such as a conversation sound uttered by a user via a communication network between a plurality of users. Traditionally provided. For example, Patent Document 1 discloses a technique for removing a noise component spectrum (estimated noise spectrum) estimated from an acoustic signal from the acoustic signal. In the technique of Patent Document 1, a final estimated noise spectrum is generated from an estimated noise spectrum according to the presence / absence of a speech component in an acoustic signal and an estimated noise spectrum estimated regardless of the presence / absence of the speech component. Remove.

JP 2010-102204 A

  By using an advanced noise suppression technique such as that disclosed in Patent Document 1, it is possible to suppress a noise component included in an acoustic signal with very high accuracy and transmit it to the voice communication device of the other party. However, if noise components including sound (hereinafter referred to as “environmental sound”) existing around the speaker are removed with an excessively high accuracy, there is a problem that the environmental sound around the speaker cannot be transmitted to the other party. . In view of the above circumstances, an object is to appropriately transmit environmental sounds around the user to the other party.

  In order to solve the above-described problems, a voice communication device according to the present invention includes a receiving unit that receives a received signal transmitted from a communication device of a communication partner, and a sound corresponding to the received signal received by the receiving unit. A sound emission unit that emits sound, a sound collection unit that collects sound including target sound and environmental sound, and generates a sound collection signal; and the target sound component of the sound collection signal generated by the sound collection unit A first signal processing unit that generates a first acoustic signal that emphasizes the sound, and a second signal processing unit that generates a second acoustic signal that emphasizes the environmental sound component of the collected sound signal generated by the sound collecting unit; A control unit that controls the level of the second acoustic signal according to the level of the received signal received by the reception unit; and a transmission unit that transmits the first acoustic signal and the second acoustic signal. . In the above configuration, the first acoustic signal in which the target sound component is emphasized and the second acoustic signal in which the environmental sound component is enhanced among the collected sound signals generated by the sound collecting unit are transmitted to the communication device of the other party. Therefore, it is possible to appropriately transmit environmental sounds around the user to the other party. Note that the target sound is the sound that is the target of sound collection, and specifically, the sound produced by the user of the voice communication device. On the other hand, the environmental sound is sound other than the target sound, and is radiated from the sound that is present around the user of the voice communication device (such as crowded noises in the crowds and operating sounds of air conditioning equipment) and from the sound emission unit. And feedback sound collected by the sound collection unit. The feedback sound is, for example, the voice of the call partner transmitted from the communication device of the call partner and radiated from the sound emitting unit.

  By the way, since the other party's voice included in the received signal is included in the environmental sound as a feedback sound that reaches the sound collecting unit from the sound emitting unit, for example, the level of the second acoustic signal is set regardless of the level of the received signal. In the maintained configuration, the voice of the other party's user circulates between the user's voice communication apparatus and the other party's voice communication apparatus, which may result in howling. In view of the above circumstances, the control unit according to a preferred aspect of the present invention controls the level of the second acoustic signal so that the level of the second acoustic signal decreases as the level of the received signal increases. . In the above aspect, since the level of the second acoustic signal is controlled so that the level of the second acoustic signal decreases as the level of the received signal increases, the level of the second acoustic signal is maintained regardless of the level of the received signal. There is an advantage that howling can be effectively prevented as compared with the configuration.

  In a preferred aspect of the present invention, the control unit controls the level of the second acoustic signal so that the level of the second acoustic signal decreases when the level of the received signal exceeds a threshold value. In the above aspect, since the level of the second acoustic signal is controlled so that the level of the second acoustic signal decreases when the level of the received signal exceeds the threshold value, the second acoustic signal does not depend on the level of the received signal. Compared to a configuration in which the level is linked to the level of the received signal, there is an advantage that environmental sound is appropriately transmitted.

  In a preferred aspect of the present invention, the control unit divides the received signal into a voice section and an insertion section other than the voice section, and the level of the second acoustic signal relative to the level of the received signal in the insertion section. The level of the second acoustic signal is controlled such that the fluctuation of the second acoustic signal is reduced as compared with the voice interval. In the above aspect, the received signal is divided into a voice section and an insertion section, and the second acoustic signal is reduced in the insertion section so that the fluctuation in the level of the second acoustic signal with respect to the level of the received signal is reduced compared to the voice section. Level is controlled. Therefore, there is an advantage that the possibility that the user of the other party of the call perceives the uncomfortable feeling due to the fluctuation of the environmental sound in the insertion section is reduced. Note that the voice section is a section in which the voice of the other party is dominant in the received signal, and the insertion section is a section other than the voice section (for example, a speaker's utterance between adjacent voice sections). (Interrupted section).

  In a preferred aspect of the present invention, when the time length of the insertion section is less than a threshold value, the control unit causes a change in the level of the second acoustic signal relative to the level of the received signal in the insertion section. While the level of the second acoustic signal is controlled so as to be reduced compared to the second acoustic signal, when the time length of the insertion section exceeds the threshold, the second acoustic signal with respect to the level of the received signal in the insertion section The level of the second acoustic signal is controlled so that the fluctuation of the signal level is equivalent to that of the voice section. In the above aspect, when the time length of the insertion section exceeds the threshold (a situation in which it is estimated that a series of utterances of the other party has been completed), the level of the second acoustic signal corresponding to the level of the received signal is the voice section. When the level is lower than the threshold, the level of the second acoustic signal is controlled so that the control of the level of the second acoustic signal according to the level of the received signal is suppressed as compared with the voice interval. . Therefore, when the time length of the insertion section is less than the threshold value, it is possible to reduce the possibility that the user perceives a sense of incongruity due to the fluctuation of the environmental sound. On the other hand, when the time length of the insertion section exceeds the threshold, it is possible to appropriately transmit the environmental sound on the user side to the other party. According to the above aspect, there is an advantage that it is possible to transmit an environmental sound of an appropriate level according to the state of speech of the other party to the other party.

  The voice communication apparatus according to the present invention is realized by hardware (electronic circuit) such as DSP (Digital Signal Processor) dedicated to processing related to voice communication, and general-purpose computation such as CPU (Central Processing Unit). This is also realized by cooperation between the processing device and the program. A program according to the present invention includes a receiving unit that receives a received signal transmitted from a communication device of a communication partner, a sound emitting unit that emits sound according to the received signal received by the receiving unit, a target sound, and an environmental sound. And a first signal processing for generating a first acoustic signal in which the target sound component is emphasized among the collected signals generated by the sound collecting unit. A second signal processing unit that generates a second acoustic signal that emphasizes the environmental sound component of the collected sound signal generated by the sound collecting unit, and the level of the received signal received by the receiving unit. The computer is caused to function as a control unit that controls the level of the second acoustic signal and a transmission unit that transmits the first acoustic signal and the second acoustic signal. The program of the present invention can be provided in a form stored in a computer-readable recording medium and installed in the computer. The recording medium is, for example, a non-transitory recording medium, and an optical recording medium (optical disk) such as a CD-ROM is a good example, but a known arbitrary one such as a semiconductor recording medium or a magnetic recording medium This type of recording medium can be included. For example, the program of the present invention can be provided in the form of distribution via a communication network and installed in a computer. The present invention is also specified as an operation method (voice communication method) of the voice communication device according to each of the above aspects.

It is a figure which shows the structure of the communication system which concerns on 1st Embodiment. It is a block diagram of a voice communication apparatus. It is explanatory drawing of the specific form of a voice communication apparatus. It is a block diagram of a 1st signal processing part. It is a block diagram of an articulation processing part. It is a block diagram of a control part. It is explanatory drawing of the relationship between the level of a received signal, and an adjustment value. It is explanatory drawing of the change of the adjustment value according to the level of a received signal. It is a block diagram of a control part concerning a 2nd embodiment. It is explanatory drawing of the relationship between the level and adjustment value in an audio | voice area and an insertion area. It is explanatory drawing of the relationship between the level of a received signal and adjustment value in 3rd Embodiment. It is a block diagram of the audio | voice communication apparatus which concerns on 4th Embodiment. It is a block diagram of the audio | voice communication apparatus which concerns on a modification.

<First Embodiment>
FIG. 1 is a diagram showing a configuration of a communication system using a voice communication apparatus according to the first embodiment of the present invention. As illustrated in FIG. 1, the communication system 100 includes a communication network 200 and a plurality of voice communication devices D (D1, D2). Each of the plurality of voice communication devices D is, for example, a communication terminal carried by a user, and performs a voice call via the communication network 200 with another voice communication device D. A communication network (for example, a mobile communication network) 200 includes a large number of relay devices including a base station and an exchange station. In FIG. 1, only two voice communication apparatuses D (D1, D2) communicating with each other are illustrated for convenience. In the following description, it is assumed that the user U1 and the user U2 make a call using the voice communication device D1 used by the user U1 and the voice communication device D2 used by the user U2. Further, although the configuration and the operation are illustrated with a focus on the voice communication device D1 for convenience, the configuration and the operation of the voice communication device D2 are the same.

  FIG. 2 is a block diagram of the voice communication device D1. The voice communication device D1 transmits an acoustic signal (hereinafter referred to as “transmission signal”) ST representing the surrounding sound such as voice uttered by the user U1 to the communication network 200 and the voice of the user U2 who is the other party. Is a device that receives an acoustic signal (hereinafter referred to as “received signal”) SR representing the sound including sound from the communication network 200 and emits sound corresponding to the received signal SR, and includes an acoustic processing unit 10, a sound collecting unit 20, and a communication unit. 30, a control unit 40, and a sound emitting unit 50. Each element (for example, the acoustic processing unit 10 and the control unit 40) illustrated in FIG. 2 is realized by, for example, an arithmetic processing unit (CPU) executing programs stored in various recording media. A configuration in which each function of the acoustic processing unit 10 is distributed over a plurality of integrated circuits or a configuration in which a dedicated electronic circuit (DSP) realizes each function may be employed. An A / D converter that converts an acoustic signal into a digital signal and a D / A converter that converts an acoustic signal into an analog signal are not shown for convenience.

  The sound collection unit 20 is an acoustic device that collects ambient sounds and generates a sound collection signal M (MA1, MA2, MB), and a plurality of sound collection devices 22 (22A1,. 22A2, 22B). The sound emitting unit 50 is an acoustic device (for example, a speaker or an earphone) that emits sound corresponding to the received signal SR received from the voice communication device D2 of the user U2 via the communication network 200.

  A mixed sound of the target sound and the environmental sound arrives at the sound collection unit 20. The target sound is the sound that is the target of sound collection, and specifically is the sound produced by the user U1. The environmental sound is sound other than the target sound, such as sound existing around the user U1 (for example, crowded noise in the crowd, operation sound of the air conditioning equipment, etc.) and the sound collecting device 22 radiated from the sound emitting unit 50. And the return sound collected. The feedback sound is, for example, the utterance sound of the user U2 transmitted from the voice communication device D2.

  FIG. 3 is an external view of the voice communication device D according to the first embodiment. In FIG. 3, a glasses-type wearable terminal is illustrated as the voice communication device D1. The voice communication device D1 is an electronic device including a main body portion 60 positioned in front of both eyes of the user U1, and support portions 62 and support portions 64 installed on both sides of the main body portion 60. The sound collecting device 22A1 is installed on the base end side (main body 60 side) of the support portion 62 attached to the left ear of the user U1, and on the base end side of the support portion 64 attached to the right ear of the user U1. A sound collecting device 22A2 is installed. That is, the sound collecting device 22A1 and the sound collecting device 22A2 are separated from each other with a distance d1. The sound collection device 22B is installed on the front end side of the support portion 64 (on the opposite side to the sound collection device 22A2 and behind the user U1).

  The sound collecting device 22A (22A1, 22A2) is an omnidirectional microphone arranged for collecting a target sound. The sound collection device 22A generates a sound collection signal MA (MA1, MA2) representing the waveform of the surrounding sound (mixed sound of the target sound and the environmental sound). On the other hand, the sound collecting device 22B is an omnidirectional microphone arranged for collecting environmental sound, and generates a sound collecting signal MB representing an acoustic waveform containing the environmental sound predominantly compared to the target sound. To do. As described above, in the configuration in which the sound collection unit 20 and the sound emission unit 50 are arranged on the support unit 62 and the support unit 64, the user U1 can make a loud call (hands-free call).

  The sound processing unit 10 in FIG. 2 generates a transmission signal ST according to the sound collection signal M (MA1, MA2, MB) generated by the sound collection unit 20. The communication unit 30 includes a transmission unit 32 and a reception unit 34, and is a communication device (antenna and modulation / demodulation circuit) that communicates with the voice communication device D2 via the communication network 200. The transmission unit 32 transmits the transmission signal ST to the communication network 200 with the voice communication device D2 as the transmission destination. On the other hand, the receiving unit 34 receives the transmission signal ST transmitted from the voice communication device D2 from the communication network 200 as the reception signal SR. As described above, sound corresponding to the reception signal SR received by the receiving unit 34 is radiated from the sound emitting unit 50.

  As illustrated in FIG. 2, the acoustic processing unit 10 according to the first embodiment includes a first signal processing unit 11, a second signal processing unit 12, and an addition unit 13. The first signal processing unit 11 emphasizes the target sound component (suppresses the environmental sound component) in the collected sound signal MA generated by the sound collecting unit 20 (sound collecting device 22A1, sound collecting device 22A2). Is generated.

  FIG. 4 is a block diagram of the first signal processing unit 11. As illustrated in FIG. 4, the first signal processing unit 11 includes a dereverberation unit 111, a directivity control unit 112, a dereverberation suppression unit 113, a noise suppression unit 114, a band enhancement unit 115, and an intensity adjustment unit 116. Is done. The reverberation suppressing unit 111 is an adaptive filter that uses the received signal SR supplied from the receiving unit 34 to the sound emitting unit 50 and the collected sound signals MA (MA1, MA2) generated by the sound collecting devices 22 (22A1, 22A2). By processing, the estimated echo component E superimposed on the collected sound signal MA (MA1, MA2) is estimated, and the estimated echo component E is suppressed from the collected sound signal MA (MA1, MA2), thereby generating the acoustic signal X1 (X1a, X1b). ) Is generated. The estimated echo component E is an acoustic component obtained by estimating the feedback sound coming from the sound emitting unit 50 to the sound collecting unit 20.

  The directivity control unit 112 controls the directivity direction of the sound collection device 22A (22A1, 22A2). Specifically, the directivity control unit 112 performs control so that a beam of sound collection (an area where sound collection sensitivity is high) is directed toward the mouth of the user U1 by, for example, a known beam forming process (for example, delay addition type beam formation). Then, the acoustic signal X2 in which the target sound component is emphasized in the acoustic signal X1 (X1a, X1b) is generated. The reverberation suppression unit 113 generates the acoustic signal X3 by suppressing the estimated echo component E from the acoustic signal X2. The reason why the estimated echo component E is suppressed by both the reverberation suppressing unit 111 and the reverberation suppressing unit 113 is that the estimated echo component E cannot be sufficiently suppressed by only one suppression by the reverberation suppressing unit 111. The noise suppression unit 114 generates the acoustic signal X4 by suppressing noise components (acoustic components other than the target sound component) from the acoustic signal X3. For noise component suppression, known noise suppression processing such as spectral subtraction can be arbitrarily employed. The band emphasizing unit 115 controls (equalizes) the frequency characteristics of the acoustic signal X4 so that the acoustic component in the frequency band including the target sound component (voiced sound) in the acoustic signal X4 is enhanced compared to the other bands. To generate an acoustic signal X5. The intensity adjusting unit 116 generates the first acoustic signal S1 by adjusting the dynamic range of the level of the acoustic signal X5 for each frequency band (Dynamic Range Control).

  The second signal processing unit 12 in FIG. 2 generates a second acoustic signal S2 that emphasizes the environmental sound component (suppresses the target sound component) in the collected sound signal MB generated by the sound collecting unit 20 (sound collecting device 22B). To do. As illustrated in FIG. 2, the second signal processing unit 12 includes an articulation processing unit 120 and an adjustment unit 130.

  FIG. 5 is a block diagram of the articulation processing unit 120. The articulation processing unit 120 of the first embodiment is an element that generates an environmental sound signal SE in which the environmental sound component is emphasized in the collected sound signal MB, and as illustrated in FIG. 5, a noise suppression unit 121 and a band enhancement unit. 122 and an intensity adjusting unit 123.

  The noise suppression unit 121 generates the acoustic signal Y1 by suppressing a noise component (for example, hiss noise) specific to the device constituting the sound emission unit 50 in the sound collection signal MB generated by the sound collection device 22B. The band emphasizing unit 122 controls (equalizing) the frequency characteristics of the acoustic signal Y1 so that the acoustic component in the frequency band including the environmental sound component in the acoustic signal Y1 is enhanced compared to the other bands. Y2 is generated. The intensity adjusting unit 123 generates the environmental sound signal SE by adjusting the dynamic range of the level of the acoustic signal Y2 for each frequency band.

  2 adjusts the environmental sound signal SE generated by the articulation processing unit 120 according to the adjustment value G to generate the second acoustic signal S2. Specifically, a multiplier that multiplies the environmental sound signal SE by the adjustment value G can be suitably employed as the adjustment unit 130. As understood from the above description, the level of the second acoustic signal S2 is adjusted according to the adjustment value (gain) G.

  2 adds the first acoustic signal S1 generated by the first signal processing unit 11 and the second acoustic signal S2 generated by the second signal processing unit 12 to generate the transmission signal ST. . The transmission signal ST after the addition by the adder 13 is transmitted from the transmitter 32 to the voice communication device D2 of the user U2 via the communication network 200.

  The control unit 40 variably controls the adjustment value G applied to the adjustment by the adjustment unit 130. The control unit 40 according to the first embodiment controls the adjustment value G according to the level of the received signal SR received by the receiving unit 34 and supplied to the sound emitting unit 50. As understood from the above description, the control unit 40 functions as an element for controlling the level of the second acoustic signal S2 in accordance with the level of the received signal SR.

  FIG. 6 is a block diagram of the control unit 40. As illustrated in FIG. 6, the control unit 40 includes a level calculation unit 42 and an adjustment value setting unit 44. The level calculator 42 calculates the level LE of the received signal SR. For calculating the level LE of the received signal SR, a known technique can be arbitrarily adopted. For example, the level LE can be calculated by smoothing the power of the received signal SR in the direction of the time axis.

  The adjustment value setting unit 44 sets an adjustment value G corresponding to the level LE of the reception signal SR calculated by the level calculation unit 42. The adjustment value setting unit 44 of the first embodiment uses an adjustment value table (not shown) that defines the relationship between the level LE of the received signal SR and the adjustment value G for setting the adjustment value G. Specifically, the adjustment value setting unit 44 acquires the adjustment value G corresponding to the level LE of the reception signal SR from the adjustment value table. As illustrated in FIG. 7, in the first embodiment, the adjustment value G is set in the control unit 40 so that the adjustment value G decreases as the level LE of the received signal SR increases. Therefore, the level of the second acoustic signal S2 decreases as the level LE of the received signal SR increases (the utterance volume of the user U2 who is the other party is higher). In the above description, the configuration using the adjustment value table is exemplified. However, a configuration in which the adjustment value G is calculated by a predetermined calculation using the level LE of the received signal SR may be employed.

  FIG. 8 illustrates a specific example of the fluctuation of the adjustment value G according to the level LE of the reception signal SR. When the level LE of the received signal SR is sufficiently small (t1 to t2), the adjustment value G is maintained at the maximum value (for example, 1). When the level LE of the received signal SR increases with the start of utterance by the user U2, the adjustment value G decreases with time (t2 to t3) in conjunction with the level LE. When the series of utterances by the user U2 approaches the end and the level LE of the received signal SR decreases, the adjustment value G increases with time in conjunction with the level LE (t3 to t4).

  As can be understood from the above example, the level of the second acoustic signal S2 including the environmental sound component changes every moment according to the presence or absence of the voice of the user U2 who is the other party (the level LE of the received signal SR). . Specifically, in a state where the voice of the user U2 is low (for example, when the user U2 is silent and listens to the voice of the user U1), the environmental sound component present around the user U1 is predominantly included. The transmission signal ST obtained by adding the second acoustic signal S2 to the first acoustic signal S1 of the utterance sound (target sound component) of the user U1 is transmitted to the voice communication device D2 of the user U2 who is the other party. On the other hand, in a state where the voice of the user U2 is predominantly contained in the feedback sound that reaches the sound pickup unit 20 from the sound emitting unit 50 (a state in which the user U2 utters), it is transmitted to the voice communication device D2 of the user U2. Among the transmitted signals ST, the level of the second acoustic signal S2 containing the feedback sound is lowered, and the transmitted signal ST having a small environmental sound component and a large amount of the utterance sound component of the user U1 is transmitted.

  As described above, in the first embodiment, the first sound signal S1 in which the target sound component is emphasized and the environmental sound component in the sound collection signal MA collected by the sound collection device 22A (22A1, 22A2) is emphasized. Both the second acoustic signal S2 is transmitted to the voice communication device D2. Therefore, in comparison with a configuration in which only the first acoustic signal S1 with the target sound component emphasized is transmitted to the voice communication device D2, the user U2 of the voice communication device D2 can not only hear the user U1 but also the user. There is an advantage that environmental sounds around U1 can be heard. Moreover, in the first embodiment, since the level of the second acoustic signal S2 is adjusted independently of the first acoustic signal S1, the environmental sound around the user U1 is transmitted to the user U2 at an appropriate level. Is possible.

  By the way, as a configuration for generating a transmission signal ST in which an environmental sound of an appropriate level is added to the voice of the user U1 (a configuration for transmitting the environmental sound to the user U2), for example, the second signal of the first embodiment. A configuration in which the processing unit 12 is omitted and the ambient sound component of the collected sound signal MA is not completely removed by the first signal processing unit 11 but is left in the transmission signal ST at an appropriate level (hereinafter, “ Also referred to as “proportional 1”). However, in the configuration of the proportional 1 in which the target sound component and the environmental sound component are processed together in a single system, waveform distortion inevitably occurs in the target sound component in the process of suppressing the environmental sound component. There may be a problem that the sound quality is degraded (for example, the auditory clarity of the target sound component is degraded). In contrast to the contrast 1, in the first embodiment, the enhancement of the target sound component by the first signal processing unit 11 and the enhancement of the environmental sound component by the second signal processing unit 12 are performed separately from each other. Then, the transmission signal ST including the processed first acoustic signal S1 and second acoustic signal S2 is transmitted. With the above configuration, the first signal processing unit 11 sufficiently emphasizes the target sound component while maintaining the sound quality (particularly clarity) by the acoustic processing optimized for emphasizing the target sound component, while the second signal processing unit 12, it is possible to sufficiently enhance the environmental sound component while maintaining the sound quality by the acoustic processing optimized for enhancing the environmental sound component. Therefore, there is an advantage that the transmission signal ST in which each of the target sound component and the environmental sound component is maintained at high sound quality can be transmitted to the user U2. In particular, since the clarity of each of the target sound component and the environmental sound component is maintained, the user U2 can clearly perceive the situation where the user U1 generates the target sound in the presence of the environmental sound. It is possible to generate a transmission signal ST with a sense of presence.

  From the viewpoint of adding the second acoustic signal S2 having a dominant environmental sound component to the first acoustic signal S1 having a dominant target sound component, an element that controls the level of the second acoustic signal S2 (the control unit 40 and A configuration (hereinafter referred to as “proportional 2”) in which the adjustment unit 130) is omitted and the environmental sound signal SE generated by the articulation processing unit 120 is added to the first acoustic signal S1 as the second acoustic signal S2 may be assumed. However, since the voice of the user U2 included in the received signal SR is included in the environmental sound as a feedback sound that reaches the sound collecting unit 20 from the sound emitting unit 50, the second is irrespective of the level LE of the received signal SR. In the configuration of contrast 2 in which the level of the acoustic signal S2 is maintained, the voice of the user U2 circulates between the voice communication device D1 and the voice communication device D2, which may cause howling as a result. In contrast to contrast 2, in the first embodiment, the level of the second acoustic signal S2 (the level of the environmental sound component added to the first acoustic signal S1) is controlled in accordance with the level LE of the received signal SR. The Specifically, the higher the level LE of the received signal SR, the lower the level of the second acoustic signal S2. That is, as described above, the second acoustic signal S2 is suppressed to a low level within the period when the user U2 speaks. Therefore, according to the first embodiment, there is an advantage that howling caused by the voice uttered by the user U2 can be effectively prevented as compared with the comparative example 2.

  In addition, as understood from the above description, in the sound that the user U2 listens to, the volume of the environmental sound around the user U1 increases and decreases every moment, so that the user U2 who listens to the sound feels a sense of incongruity. The possibility of perception is also assumed. However, considering the general tendency that the user is not conscious of the incoming sound during his / her utterance (not intensively listening to the utterances of other people during his / her utterance), the user U2 The possibility that the user U2 perceives a sense of incongruity due to the change in the volume of the environmental sound of the user U1 in the sound to be listened to is not a special problem.

Second Embodiment
A second embodiment of the present invention will be described below. In the first embodiment, the level of the second acoustic signal S2 is controlled according to the level LE of the received signal SR. In the above configuration, if the environmental sound fluctuates in a section where the voice of the user U2 is interrupted between successive utterances by the user U2, the user U2 may perceive a sense of discomfort. In consideration of the above circumstances, in the second embodiment, in the received signal SR, a section where the voice of the user U2 is dominant (hereinafter referred to as “voice section”) and a section other than the voice section (hereinafter referred to as “insertion section”). And the adjustment value G is set differently. In addition, about the element which an effect | action and function are the same as that of 1st Embodiment in each form illustrated below, the reference | standard referred by description of 1st Embodiment is diverted, and each detailed description is abbreviate | omitted suitably.

  FIG. 9 is a block diagram of the control unit 40 of the second embodiment, and FIG. 10 is an explanatory diagram of setting of the adjustment value G in the second embodiment. As illustrated in FIG. 9, the adjustment value setting unit 44 of the second embodiment includes a section detection unit 46. As illustrated in FIG. 10, the section detector 46 divides the received signal SR into a voice section TV (TV1, TV2) and an insertion section TD (TD1, TD2) on the time axis. The insertion section TD is a section where, for example, the voice of the user U2 is interrupted between successive utterances by the user U2. The section detector 46 divides the received signal SR into a voice section TV and an insertion section TD by determining the presence / absence of voice for each frame obtained by dividing the received signal SR on the time axis, for example. A known voice detection technique is arbitrarily employed for discrimination between the voice section TV and the insertion section TD.

  FIG. 10 is an explanatory diagram of the adjustment value G of the voice section TV and the insertion section TD. In FIG. 10, among the voice sections TV exemplified on the time axis, a voice section TV (TV1, TV2) and an insertion section TD (TD1, TD2) are shown for convenience. The adjustment value setting unit 44 of the second embodiment sets the adjustment value G individually for the voice section TV and the insertion section TD. The setting of the adjustment value G in each section (voice section TV, insertion section TD) will be described in detail below.

  Within the voice section TV, the adjustment value setting unit 44 sets the adjustment value G according to the level LE of the received signal SR, as in the first embodiment. Specifically, the adjustment value setting unit 44 sets the adjustment value G so that the level of the second acoustic signal S2 decreases as the level of the received signal SR increases.

  On the other hand, in the insertion section TD, the adjustment value G is set according to the state of speech of the user of the voice communication device D2. Specifically, as exemplified below, the adjustment value setting unit 44 sets the adjustment value G according to the comparison result between the time length of the insertion interval TD and the threshold value T0.

(A). When the length of the insertion section TD is less than the threshold value T0 (TD <T0)
When the time length of the insertion section TD is less than the threshold value T0 (when the interval between adjacent voice sections TV is short), it is estimated that the user U2 of the voice communication device D2 is in the middle of speaking. If the level of the environmental sound component (second acoustic signal S2) fluctuates in the same manner as in the voice interval TV during successive utterances, there is a possibility that the user U2 perceives a sense of discomfort. In consideration of the above tendency, the adjustment value setting unit 44 adjusts the adjustment value G so that the fluctuation of the level of the second acoustic signal S2 according to the level LE of the reception signal SR is suppressed as compared with the voice interval TV. Set. Specifically, as illustrated in the insertion interval TD1 in FIG. 10, the fluctuation of the adjustment value G with respect to the level LE of the received signal SR (the rate of change of the adjustment value G with respect to the level LE) is adjusted with respect to the level LE in the voice interval TV. The adjustment value G is set so as to be gradual compared with the fluctuation of the value G. A configuration in which the adjustment value G is maintained at a constant value in the insertion interval TD whose time length is less than the threshold T0 (a configuration in which the rate of change of the adjustment value G with respect to the level LE is set to zero) may be employed.

(B). When the length of the insertion section TD exceeds the threshold value T0 (TD> T0)
When the time length of the insertion section TD exceeds the threshold value T0 (when a sufficient time length has passed without the user U2 uttering from the end point of the immediately preceding voice section TV), the series of utterances of the user U2 ends. (For example, the user U2 is changing to the user U1 and the user U2 is listening to the voice of the user U1 of the voice communication device D1). The sound is preferably transmitted to the user U2 at an appropriate level. Considering the above tendency, the adjustment value setting unit 44 sets the adjustment value G so that the fluctuation of the level of the second acoustic signal S2 corresponding to the level LE of the received signal SR becomes equal to that of the voice section TV1. Specifically, as illustrated in the insertion interval TD2 in FIG. 10, the control of the adjustment value G similar to that in the audio interval TV is started when the elapsed time from the end point of the immediately previous audio interval TV2 exceeds the threshold T0. The A configuration in which the adjustment value G is increased to the maximum value according to the level LE of the received signal SR when the elapsed time from the end point of the voice section TV2 exceeds the threshold value T0 is also conceivable. However, in the configuration in which the adjustment value G increases rapidly, the user U2 may perceive a sense of discomfort. Therefore, as illustrated in FIG. 10, the adjustment value G is gradually increased so that the environmental sound around the user U1 is transmitted to the user U2 at an appropriate level.

  The adjustment unit 130 controls the level of the second acoustic signal S2 by multiplying the environmental sound signal SE by the adjustment value G set for each of the voice interval TV and the insertion interval TD. Since the subsequent processing is the same as in the first embodiment, detailed description thereof is omitted.

  In the second embodiment, the same effect as in the first embodiment is realized. Further, in the second embodiment, in the insertion section TD (insertion section TD whose time length is less than the threshold value T0) in the reception signal SR, fluctuations in the level of the second acoustic signal S2 are suppressed compared to the voice section TV. In this way, the level of the second acoustic signal S2 is controlled. Therefore, there is an advantage that the possibility that the user U2 perceives a sense of incongruity due to the fluctuation of the environmental sound in the insertion section TD is reduced.

  On the other hand, in the insertion interval TD in which the time length exceeds the threshold value T0 (a situation in which a series of utterances by the user U2 is estimated to have ended), the control of the level of the second acoustic signal S2 according to the level LE of the received signal SR is performed. The control returns to the same control as in the voice section TV. Therefore, there is an advantage that the environmental sound on the user U1 side can be appropriately transmitted to the user U2 after the end of the utterance of the user U2. As understood from the above description, in the second embodiment, it is possible to transmit an environmental sound of an appropriate level according to the state of the utterance of the user U2 to the user U2.

<Third Embodiment>
In the first embodiment, the adjustment value G is linearly changed with respect to the level LE of the received signal SR, but the mode of change of the adjustment value G with respect to the level LE is not limited to the above example. For example, in a range where the level LE of the received signal SR is so low that howling due to the received signal SR does not occur, the feedback to the user U2 of the voice communication device D2 is more effective than the suppression of howling by reducing the level of the second acoustic signal S2. Prioritize environmental sound transmission. Considering the above circumstances, in the third embodiment, when the level LE of the received signal SR exceeds a predetermined threshold value L0, the adjustment value G is reduced so that the adjustment value G is reduced as the level LE increases. Is set.

  FIG. 11 is an explanatory diagram of the relationship (adjustment value table) between the level LE of the received signal SR and the adjustment value G in the third embodiment. As illustrated in FIG. 11, when the level LE of the received signal SR is lower than the threshold value L0 (LE <L0), the adjustment value G is maintained at a predetermined value (for example, the maximum value 1) independent of the level LE. The On the other hand, when the level LE of the received signal SR exceeds the threshold value L0 (LE> L0), the adjustment value G is set so that the adjustment value G decreases as the level LE of the received signal SR increases. Specifically, an area exceeding the threshold L0 in the range of the level LE is divided into a plurality of ranges, and the rate of change of the adjustment value G with respect to the level LE is set individually for each range. FIG. 11 illustrates a case where the rate of change (gradient) of the adjustment value G with respect to the level LE increases as the level LE increases.

  In the third embodiment, the same effect as in the first embodiment is realized. In the third embodiment, the adjustment value G is reduced only when the level LE of the received signal SR exceeds the threshold value L0 (so that the level of the second acoustic signal S2 is lowered), that is, The adjustment value G is set so that the degree of change of the adjustment value G differs according to the level LE of the received signal SR. Therefore, there is an advantage that the environmental sound is appropriately transmitted as compared with the configuration in which the level of the second acoustic signal S2 is linked to the level LE of the received signal SR regardless of the level LE of the received signal SR. is there.

<Fourth embodiment>
FIG. 12 is a block diagram of the voice communication device D1 of the fourth embodiment. In the first embodiment, the sound collecting device 22B for collecting the environmental sound is installed separately from the sound collecting device 22A for collecting the target sound. In the fourth embodiment, as illustrated in FIG. 12, a common sound collecting device 22A2 is also used for collecting the target sound and the environmental sound. The sound collection signal MA2 generated by the sound collection device 22A2 is supplied to both the first signal processing unit 11 and the second signal processing unit 12, and the second signal processing unit 12 emphasizes the environmental sound component of the sound collection signal MA2. As a result, the second acoustic signal S2 is generated. In the above configuration, it is not necessary to separately provide a sound collecting device for collecting the target sound and a sound collecting device for collecting the environmental sound, so that the device configuration of the voice communication device D1 is simplified. There is. Note that the sound collecting device 22A1 may be configured by a directional microphone, and the directional direction may be the mouth of the user of the voice communication device D1. Further, the sound collecting device 22A2 that is used for collecting the target sound and the environmental sound may be constituted by a non-directional microphone.

<Modification>
Each of the above-described embodiments can be variously modified. Specific modifications are exemplified below. Two or more modes arbitrarily selected from the following examples can be appropriately combined.

(1) The components of the first signal processing unit 11 and the second signal processing unit 12 illustrated in the above-described embodiments are arbitrary, and the components illustrated in FIGS. 4 and 5 can be omitted as appropriate. For example, a configuration in which the directing control unit 112 is omitted from the first signal processing unit 11 can be suitably employed. In the configuration including the directivity control unit 112 as in each of the above-described embodiments, a plurality of sound collecting devices 22 (22A1, 22A2) are necessary for the beam forming process for controlling the directivity direction. In the omitted configuration (configuration in which beam forming processing is not executed), for example, as illustrated in FIG. 13, the sound collection unit 20 may be configured by one sound collection device 22A1. The collected sound signal MA1 collected from the sound collecting device 22A1 is supplied to the first signal processing unit 11 and the second signal processing unit 12. As understood from the above description, the sound collection unit 20 of each of the above-described forms is comprehensively expressed as an element that collects sound including the target sound and the environmental sound and generates the sound collection signal M, and collects sound. The number of sound collecting devices 22 constituting the sound unit 20 and the presence or absence of directivity are not questioned.

(2) In each of the above-described embodiments, the transmission signal ST corresponding to the addition result of the first acoustic signal S1 corresponding to the target sound component and the second acoustic signal S2 corresponding to the environmental sound component is used as the voice communication device of the communication partner. Although the configuration of transmitting to D2 has been illustrated, the first acoustic signal S1 and the second acoustic signal S2 can be transmitted separately from each other and added by the voice communication device D2. That is, the adding unit 13 of the voice communication device D1 can be omitted. As understood from the above description, the transmission unit 32 of each of the above-described forms is comprehensively expressed as an element that transmits the first acoustic signal S1 and the second acoustic signal S2, and the first acoustic signal S1 and the second acoustic signal S2 It includes both a configuration for transmitting an addition signal (transmission signal ST) with the acoustic signal S2 and a configuration for transmitting each of the first acoustic signal S1 and the second acoustic signal S2.

(3) As the communication network 200, a broadband IP (Internet Protocol) network or a public wireless LAN (WiFi) can be suitably employed. Since the frequency component corresponding to the uttered sound (target sound component) is relatively low-band and the frequency component corresponding to environmental sound is relatively high-band, a broadband communication system compliant with high-speed data communication standards is available. Is preferred.

(4) In each of the above-described embodiments, the glasses-type wearable terminal is exemplified as the voice communication device D. However, if the electronic device is capable of voice communication and can be carried by the user, the voice communication device D may be used. Is optional. For example, a known communication terminal such as a mobile phone or a smartphone can be arbitrarily used as the voice communication device D.

(5) In the above-described embodiments, the configuration in which the environmental sound around the user is transmitted to the other party along with the target sound is exemplified. However, in reality, it may be assumed that the other party does not want to know his / her whereabouts. The Therefore, the user arbitrarily selects an operation mode in which the environmental sound is added to the target sound in the same manner as the above-described embodiments and an operation mode in which the environmental sound is not added (an operation mode in which the operation of the second signal processing unit 12 is invalidated). It is also possible to adopt a configuration that can be selected.

(6) In each of the above-described embodiments, the adjustment value G is linearly changed with respect to the level LE of the received signal SR. However, the mode of change of the adjustment value G with respect to the level LE is not limited to the above examples. For example, a configuration in which the adjustment value G is nonlinearly changed with respect to the level LE of the received signal SR or a configuration in which the adjustment value G is defined in a curve with respect to the level LE of the received signal SR can be employed.

D: Voice communication device, 10: Acoustic processing unit, 11: First signal processing unit, 12: Second signal processing unit, 20: Sound collection unit, 22A1, 22A2, 22B ... Sound collection device, 30 …… Communication unit, 32 …… Transmitting unit, 34 …… Reception unit, 120 …… Articulation processing unit, 130 …… Adjustment unit, 40 …… Control unit, 42 …… Level calculation unit, 44 …… Adjustment value setting Department, 200 ... communication network.

Claims (5)

A receiving unit for receiving a reception signal transmitted from a communication device of a communication partner;
A sound emitting unit that emits sound according to the received signal received by the receiving unit;
A sound collection unit that collects sound including target sound and environmental sound and generates a sound collection signal;
A first signal processing unit for generating a first acoustic signal by the first signal processing to emphasize the target sound components to the environmental sound component of the collected sound signal the sound pickup unit has generated,
Emphasizing the environmental sound component with respect to the target sound components of the sound collection signals the sound pickup unit has generated, the second signal to generate a second audio signal by a different second signal processing from the first signal processing A processing unit;
A control unit for controlling the level of the second acoustic signal according to the level of the received signal received by the receiving unit ;
A voice communication device comprising: a transmission unit that transmits the first acoustic signal and the second acoustic signal. The voice communication apparatus according to claim 1, wherein the control unit controls the level of the second acoustic signal so that the level of the second acoustic signal decreases as the level of the received signal increases. The control unit divides the received signal into a voice section and an insertion section other than the voice section, and a fluctuation in the level of the second acoustic signal with respect to the level of the received signal is compared with the voice section in the insertion section. The voice communication device according to claim 1, wherein the level of the second acoustic signal is controlled so as to be reduced. When the time length of the insertion section is less than a threshold, the control unit is configured to reduce a variation in the level of the second acoustic signal with respect to the level of the received signal in the insertion section as compared with the voice section. When the time length of the insertion section exceeds the threshold value, the fluctuation of the level of the second acoustic signal with respect to the level of the received signal is detected in the insertion section. The voice communication apparatus according to claim 3, wherein the level of the second acoustic signal is controlled to be equal to the section. A receiving unit for receiving a reception signal transmitted from a communication device of a communication partner;
A sound emitting unit that emits sound according to the received signal received by the receiving unit;
A sound collection unit that collects sound including target sound and environmental sound to generate a sound collection signal,
The first signal processing unit for generating a first acoustic signal by the first signal processing to emphasize the target sound components to the environmental sound component of the sound pickup unit is the sound pickup signal generated,
Emphasizing the environmental sound component with respect to the target sound components of the sound collection signals the sound pickup unit has generated, the second signal to generate a second audio signal by a different second signal processing from the first signal processing Processing section,
A control unit for controlling the level of the second acoustic signal in accordance with the level of the received signal received by the receiving unit; and
A program that causes a computer to function as a transmission unit that transmits the first acoustic signal and the second acoustic signal.

Images