JP2017123554A - Speech apparatus and audio signal correction program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent an occurrence of echo by learning erroneous transfer characteristics during a call.SOLUTION: The speech apparatus includes: a receiver; a first microphone; a second microphone; an echo suppression unit, and a transfer characteristic learning unit. The receiver outputs an audio signal as an air conduction sound. The first microphone picks up the air conduction sound and the second microphone picks up the bone conduction sound. The echo suppression unit suppresses the echo component included in the air-conduction sound signal input from the first microphone on the basis of the transfer characteristic of the sound from the receiver to the first microphone. When the input level of the bone conduction sound signal input from the second microphone is equal to or less than a predetermined threshold value, the transfer characteristic learning unit learns the transfer characteristic on the basis of the received speech signal and the air conduction sound signal outputted from the receiver.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a call device and an audio signal correction program.

  A communication device such as a mobile phone terminal includes a receiver that outputs (radiates) an audio signal as air conduction sound, and a microphone that collects air conduction sound that propagates around the communication device (hereinafter referred to as a “microphone”). In a state in which a call connection with another call device is established, the call device outputs an audio signal (received signal) received from the other call device by the receiver and an air conduction sound signal picked up by the microphone. (Transmission signal) is transmitted to another communication device. For this reason, the transmission signal in one of the communication devices may include sound output from the receiver of the communication device. When a voice signal including voice output from the receiver is transmitted from one call device to the other call device, echo is generated and the call quality deteriorates.

  As a technique for preventing deterioration of call quality due to echo, a technique for suppressing an echo component included in a voice signal input from a microphone based on a transfer characteristic of air conduction sound from the receiver to the microphone is known. The echo component is a component that causes an echo to occur. When suppressing the echo component, a pseudo echo signal is generated from the transmission coefficient indicating the transmission characteristic of the sound from the receiver to the microphone and the received signal, and a residual signal is generated based on the pseudo echo signal and the transmitted signal. (For example, see Patent Document 1).

  Further, when suppressing the echo component based on the transfer characteristic, the echo component included in the transmission signal is learned by learning the transfer characteristic based on the reception signal output from the receiver and the transmission signal input from the microphone. A technique for appropriately suppressing the above is known.

  In addition, when the voice signal input from the microphone contains the voice of the user of the telephone device, learning of the transfer characteristic is prevented to prevent erroneous transfer characteristic learning and the echo component suppression performance is reduced. A technique for preventing this is known (see, for example, Patent Document 2).

JP 2013-081163 A Japanese Patent Laid-Open No. 04-342317

  In a call device that collects air conduction sound with a microphone, the voice signal (air conduction sound) emitted by the user is low, and the sound signal input from the microphone is not processed even though the user is speaking. The contained user's voice may not be detected. For this reason, when a voice signal input from a microphone includes a user's voice, a call device that does not learn a transfer characteristic learns an incorrect transfer characteristic based on the voice signal including the user's voice. The echo component suppression performance may be reduced.

  In one aspect, an object of the present invention is to prevent generation of echoes and deterioration of transmission sound quality due to learning of erroneous transfer characteristics during a call.

  The telephone device according to one aspect includes a receiver, a first microphone, a second microphone, an echo suppression unit, and a transfer characteristic learning unit. The receiver outputs an audio signal as an air conduction sound. The first microphone picks up air conduction sound, and the second microphone picks up bone conduction sound. The echo suppression unit suppresses an echo component included in the air conduction sound signal input from the first microphone based on the transmission characteristic of the sound from the receiver to the first microphone. The transfer characteristic learning unit learns transfer characteristics based on the reception signal and the air conduction sound signal output from the receiver when the input level of the bone conduction sound signal input from the second microphone is equal to or lower than a predetermined threshold. To do.

  According to the above-described aspect, it is possible to prevent the occurrence of echoes and the deterioration of the transmission sound quality due to learning of erroneous transfer characteristics during a call.

It is a figure which shows the structure of the telephone apparatus which concerns on 1st Embodiment. It is a figure which shows the functional structure of the transmission signal processing part in the telephone apparatus which concerns on 1st Embodiment. It is a flowchart explaining the echo suppression process which concerns on 1st Embodiment. It is a flowchart explaining the content of a user voice detection process. It is a flowchart explaining the content of the process which learns a transfer characteristic. It is a flowchart explaining the content of the process which suppresses an echo component. It is a figure which shows the functional structure of the transmission signal process part in the telephone apparatus which concerns on 2nd Embodiment. It is a flowchart (the 1) explaining the echo suppression process which concerns on 2nd Embodiment. It is a flowchart (the 2) explaining the echo suppression process which concerns on 2nd Embodiment. It is a flowchart explaining the content of the process which correct | amends a bone-conduction sound signal. It is a flowchart explaining the example of the calculation method of a bone-conduction sound correction characteristic. It is a figure which shows the functional structure of the transmission signal process part in the telephone apparatus which concerns on 3rd Embodiment. It is a flowchart (the 1) explaining the echo suppression process which concerns on 3rd Embodiment. It is a flowchart (the 2) explaining the echo suppression process which concerns on 3rd Embodiment. It is a graph explaining the calculation method of the reliability of a bone-conduction sound signal. It is a graph explaining the calculation method of the update coefficient of a transfer characteristic. It is a figure which shows the functional structure of the principal part in the telephone apparatus which concerns on 4th Embodiment. It is a graph explaining the calculation method of the reliability of the bone-conduction sound signal in 4th Embodiment. It is a figure which shows the hardware constitutions of a computer.

[First Embodiment]
FIG. 1 is a diagram illustrating a configuration of a communication device according to the first embodiment.

  The call device according to the present embodiment is a mobile communication device such as a mobile phone terminal that can make a call. As shown in FIG. 1, the communication device 1 of the present embodiment includes an RF transmission / reception unit 2, an antenna 3, a baseband processing unit 4, an audio signal processing unit 5, a receiver 8, and a first microphone 9. And a second microphone 10. Further, the communication device 1 includes a D / A converter 11, A / D converters 12A and 12B, and amplifiers 13A, 13B, and 13C. In addition, the communication device 1 includes an input operation unit, a display unit, and the like (not shown).

  The RF transmitter / receiver 2 demodulates a signal received by the antenna 3 and modulates a signal to be transmitted to another communication device.

  The baseband processing unit 4 performs baseband processing on the signal demodulated by the RF transmission / reception unit 2 and the signal modulated by the RF transmission / reception unit 2. The baseband processing unit 4 includes an A / D converter that converts the analog signal demodulated by the RF transmission / reception unit 2 into a digital signal, and a D / A converter that converts the digital signal modulated by the RF transmission / reception unit 2 into an analog signal. including.

  The audio signal processing unit 5 performs predetermined processing on the audio signal. The voice signal processing unit 5 includes a reception signal processing unit 6 that performs processing on a voice signal received from another telephone device, and a transmission signal processing unit 7 that performs processing on a voice signal transmitted to the other telephone device. .

  The receiver 8 outputs (radiates) the voice signal processed by the reception signal processing unit 6 to the outside of the communication device 1 as an air conduction sound. The audio signal processed by the reception signal processing unit 6 is converted from a digital signal to an analog signal by the D / A converter 11, amplified by the amplifier 13 </ b> A, and then output from the receiver 8. In the present specification, the receiver 8 is used. However, the receiver 8 is not limited to this, and any device that can output (radiate) an audio signal as an air conduction sound to the outside of the communication device 1 is acceptable.

  The first microphone 9 collects sound (air conduction sound) that propagates in the air and arrives at the communication device 1. The air-conducted sound signal collected by the first microphone 9 is amplified by the amplifier 13B, converted from an analog signal to a digital signal by the A / D converter 12A, and then input to the transmission signal processing unit 7. Hereinafter, the audio signal input from the first microphone 9 to the transmission signal processing unit 7 is also referred to as an air conduction sound signal.

  The second microphone 10 collects sound (for example, bone conduction sound) transmitted to the call device 1 through a solid body that is in contact with the call device 1. The bone-conducted sound signal collected by the second microphone 10 is amplified by the amplifier 13C, converted from an analog signal to a digital signal by the A / D converter 12B, and then input to the transmission signal processing unit 7. Hereinafter, the audio signal input from the second microphone 10 to the transmission signal processing unit 7 is also referred to as a bone conduction sound signal.

  When the call connection with the other call device is established, the call device 1 of the present embodiment is inputted from the first microphone 9 together with the process of outputting the audio signal received from the other call device from the receiver 8. A process of transmitting the air conduction sound signal to another communication device is performed. At this time, the transmission signal processing unit 7 performs a process of transmitting the air conduction sound signal input from the first microphone 9 to another communication device. The transmission signal processing unit 7 performs a process of suppressing an echo component included in the air conduction sound signal as one of the processes for the air conduction sound signal. The echo component included in the air conduction sound signal is a sound component that causes an echo when listening to the sound output from the receiver of the communication apparatus 1 or another communication apparatus. As shown in FIG. 1, the echo is generated when a part 14 of the sound output from the receiver 8 is collected by the first microphone 9.

  FIG. 2 is a diagram illustrating a functional configuration of a transmission signal processing unit in the communication device according to the first embodiment.

  As shown in FIG. 2, the transmission signal processing unit 7 includes a user voice detection unit 701, a transfer characteristic learning unit 702, an echo suppression unit 703, and a storage unit 711.

  The user voice detection unit 701 detects voice uttered by the user of the communication device 1 included in the bone conduction signal input from the second microphone 10. The user of the call device 1 is a person who makes a call while listening to the sound (air conduction sound) output from the receiver 8 of the call device 1. Hereinafter, the voice uttered by the user is also referred to as user voice. Note that the user voice detection unit 701 according to the present embodiment determines that the user voice is detected when the user voice is detected from the bone conduction signal.

  The transfer characteristic learning unit 702 learns the transfer characteristic of sound propagating from the receiver 8 to the first microphone 9. The transfer characteristic learning unit 702 according to the present embodiment learns transfer characteristics using the received signal and the air conduction sound signal input from the first microphone 9. Also, in the transmission signal processing unit 7 according to the present embodiment, when the user voice is not detected by the user voice detecting unit 701, the transfer characteristic learning unit 702 learns the transfer characteristic.

  The echo suppression unit 703 suppresses an echo component included in the air conduction sound signal based on the transfer characteristic. The echo suppression unit 703 applies a transfer characteristic to the received signal to estimate an echo component included in the air conduction sound signal, and removes the estimated echo component from the air conduction sound signal. When the user voice is not detected by the user voice detection unit 701, the echo suppression unit 703 suppresses the echo component of the air conduction sound signal based on the transfer characteristic learned by the transfer characteristic learning unit 702. On the other hand, when user voice is detected by the user voice detection unit 701, the echo suppression unit 703 suppresses the echo component of the air conduction sound signal based on the transfer characteristics stored in the storage unit 711.

  The storage unit 711 stores the initial value of the transfer characteristic and the transfer characteristic learned by the transfer characteristic learning unit 702.

  When the call connection between the communication device 1 of the present embodiment and another communication device is established, the transmission signal processing unit 7 of the audio signal processing unit 5 is sequentially input air conduction sound signal, bone conduction sound signal, And echo suppression processing as shown in FIG. 3 is performed based on the received signal.

FIG. 3 is a flowchart for explaining echo suppression processing according to the first embodiment.
As shown in FIG. 3, the transmission signal processing unit 7 first initializes a variable t for identifying a frame, which is a processing unit of the audio signal, to 1 (step S1).

  Next, the transmission signal processing unit 7 inputs the air conduction sound signal, the bone conduction sound signal, and the reception signal of the frame t (step S2). In step S2, the transmission signal processing unit 7 inputs, for example, the bone conduction sound signal and the reception signal to the user voice detection unit 701, and inputs the air conduction sound signal to the transfer characteristic learning unit 702 and the echo suppression unit 703. .

  Next, the transmission signal processing unit 7 performs user voice detection processing (step S3) using the bone conduction sound signal. The process of step S3 is performed by the user voice detection unit 701. The user voice detection unit 701 performs processing for detecting voice from the bone conduction sound signal of the frame t (processing target frame) input in step S2. Further, the user voice detection unit 701 determines whether or not a user voice has been detected based on the result of the voice detection process (step S4). When the user voice is detected from the bone conduction signal, the user voice detection unit 701 determines that the user voice has been detected (step S4; Yes). When the user voice is detected (step S4; Yes), the user voice detection unit 701 performs a process (step S6) of suppressing the echo component of the air conduction sound signal based on the transfer characteristic in the echo suppression unit 703. Make it.

  On the other hand, when the user voice is not detected (step S4; No), the user voice detection unit 701 causes the transfer characteristic learning unit 702 to learn the transfer characteristic using the received signal and the air conduction sound signal ( Step S5) is performed. In step S5, the transfer characteristic learning unit 702 acquires the air conduction sound signal and the reception signal of the frame t, and based on the current transfer characteristic and the air conduction sound signal and the reception signal of the frame t, the air characteristic sound of the frame t. Learn (calculate) transfer characteristics to be applied to the sound guide signal. Here, the current transfer characteristic is an initial value of the transfer characteristic or a transfer characteristic used in the echo suppression process for the air conduction sound signal one frame before (frame t−1). The transfer characteristic learning unit 702 reads the current transfer characteristic from the storage unit 711. When the transfer characteristic is learned in step S <b> 5, the transfer characteristic learning unit 702 transmits the learned transfer characteristic to the echo suppression unit 703 and stores it in the storage unit 711.

  When the process of step S5 in the transfer characteristic learning unit 702 is completed, the echo suppression unit 703 then suppresses the echo component of the air conduction sound signal based on the transfer characteristic (step S6). When the transfer characteristic learning unit 702 learns the transfer characteristic in the process for the air conduction sound signal of the frame t, the echo suppression unit 703 suppresses the echo component of the air conduction sound signal of the frame t based on the learned transfer characteristic. . On the other hand, when the transfer characteristic learning unit 702 has not learned the transfer characteristic, the echo suppression unit 703 acquires the air conduction sound signal of the frame t and echoes the initial value of the transfer characteristic or the air conduction sound signal one frame before. The echo component of the air conduction sound signal is suppressed based on the transfer characteristic used in the suppression process. At this time, the echo suppression unit 703 reads the transfer characteristic from the storage unit 711. The echo suppression unit 703 outputs the air conduction sound signal in which the echo component is suppressed to the baseband processing unit 4.

  When the suppression of the echo component with respect to the air conduction sound signal of the frame t is completed, the transmission signal processing unit 7 determines whether or not the frame t in which the echo component is suppressed is the final frame (step S7). When the frame t in which the echo component is suppressed is not the final frame (step S7; No), the transmission signal processing unit 7 updates the variable t to t + 1 (step S8), and performs the processing of steps S2 to S6 for the subsequent frame. I do. On the other hand, when the frame t in which the echo component is suppressed is the final frame (step S7; Yes), the transmission signal processing unit 7 ends the echo suppression process.

FIG. 4 is a flowchart for explaining the contents of the user voice detection process.
As described above, the user voice detection process (step S3) in the echo suppression process of the present embodiment is performed by the user voice detection unit 701 using the bone conduction sound signal. As shown in FIG. 4, the user voice detection unit 701 first calculates the frame power Pd (unit: dB) of the bone conduction sound signal of the frame t (step S301). In step S301, the user voice detection unit 701 calculates a frame power Pd based on a frequency spectrum (power spectrum) obtained by frequency analysis with respect to the bone conduction sound signal of the frame t. The frequency analysis for the bone conduction sound signal is performed by the user voice detection unit 701 or a frequency analysis unit not shown in FIG.

  Next, the user sound detection unit 701 calculates a difference value ΔP = Pb−Pf between the frame power Pd of the bone conduction sound signal calculated in step S301 and the frame power Pf (unit: dB) when there is no user sound. Is calculated (step S302). Here, the frame power Pf when there is no user voice is an average value of the frame power when the voice signal (bone conduction signal) input from the second microphone 10 does not include the user voice. The frame power Pf is obtained in advance by collecting a sound that does not include user voice by the second microphone 10.

  Next, the user voice detection unit 701 determines whether or not the frame power difference value ΔP calculated in step S302 is larger than the determination threshold value THb (step S303). When the user speaks with the telephone device 1 in contact with the head, the bone conduction sound signal input from the second microphone 10 includes the user voice. Therefore, the frame power Pd when the user sound is included in the bone conduction sound signal is larger than the frame power Pf when there is no user sound. Therefore, the determination threshold value THp is an arbitrary positive value (for example, 6 dB).

  When ΔP> THb (step S303; Yes), the user voice detection unit 701 determines that the process result is “user voice detected” (step S304), and ends the user voice detection process (return). On the other hand, if ΔP ≦ THb (step S303; No), the user voice detection unit 701 determines that the process result is “no user voice detected” (step S305), and ends the user voice detection process.

  Note that the user voice detection process of FIG. 4 is merely an example, and the process of detecting the user voice from the bone conduction signal determines whether the input level of the bone conduction signal is a level including the user voice. If it can be determined, the user voice may be detected by another method.

  When the user voice detection process (steps S301 to S305) is completed, the user voice detection unit 701 determines whether or not a user voice has been detected (step S4). The determination in step S4 is performed to determine whether or not to learn transfer characteristics. In the communication apparatus 1 of this embodiment, when a user voice is not detected (step S4; No), a process of learning transfer characteristics (step S5) is performed. That is, the communication device 1 performs processing for learning transfer characteristics when no user voice is detected from the bone conduction sound signal. The transfer characteristic learning unit 702 performs the process of learning the transfer characteristic. For this reason, when the user voice is not detected (step S4; No), the user voice detection unit 701 transmits the input received signal to the transfer characteristic learning unit 702. The transfer characteristic learning unit 702 according to the present embodiment performs processing as illustrated in FIG.

FIG. 5 is a flowchart for explaining the contents of the process of learning the transfer characteristics.
As shown in FIG. 5, the transfer characteristic learning unit 702 first obtains the frequency spectrum of the air conduction sound signal and the reception signal of the frame t (step S501). The frequency spectra of the air conduction sound signal and the reception signal are calculated by performing frequency analysis in the transfer characteristic learning unit 702, for example. The frequency spectrum of the air conduction sound signal and the reception signal may be calculated by performing frequency analysis of the air conduction sound signal and the reception signal in a frequency analysis unit not shown in FIG.

  Next, the transfer characteristic learning unit 702 initializes a variable i for identifying the frequency band of the frequency spectrum to i = 0 (step S502).

  Next, the transfer characteristic learning unit 702 reads the current transfer characteristic EH (i, t−1) from the storage unit 711 (step S503). The transfer characteristic EH (i, t-1) is a transfer characteristic applied to the amplitude spectrum of the frequency band i in the process of suppressing the echo component of the air conduction sound signal of the frame t-1. When the variable t is an initial value (for example, t = 1), the transfer characteristic learning unit 702 reads the initial value of the transfer characteristic as the current transfer characteristic EH (i, 0).

  Next, the transfer characteristic learning unit 702 is based on the amplitude spectrum A (i, t) of the air conduction sound signal, the amplitude spectrum Ref (i, t) of the received signal, and the current transfer characteristic EH (i, t−1). Thus, the transfer characteristic EH (i, t) is learned (calculated) (step S504). The transfer characteristic EH (i, t) is a transfer characteristic applied to the amplitude spectrum A (i, t) of the air conduction sound signal. In step S504, the transfer characteristic learning unit 702 calculates the transfer characteristic EH (i, t) using the equation (1).

  Α in equation (1) is a transfer characteristic update coefficient, and is a constant of 0 <α <1 (for example, α = 0.99).

  Next, the transfer characteristic learning unit 702 determines whether or not processing has been performed for all frequency bands (step S505). When there is an unprocessed frequency band (step S505; No), the transfer characteristic learning unit 702 updates the variable i to i + 1 (step S506), and performs the processing of steps S503 and S504. When processing has been performed for all frequency bands (step S505; Yes), the transfer characteristic learning unit 702 ends the process of learning transfer characteristics to be applied to the air conduction sound signal of the frame t. At this time, the transfer characteristic learning unit 702 stores the learned transfer characteristic EH (i, t) in the storage unit 711, for example. Also, the transfer characteristic learning unit 702 transmits, for example, the amplitude spectra of the air conduction sound signal and the reception signal and the learned (calculated) transfer characteristic to the echo suppression unit 703. In this case, the echo suppression unit 703 performs a process (step S6) of suppressing the echo component of the air conduction sound signal of the frame t based on the transfer characteristic EH (i, t) learned by the transfer characteristic learning unit 702. That is, when the user voice is not detected by the user voice detection unit 701, the echo suppression unit 703 calculates the echo component of the air conduction sound signal of the frame t based on the learned transfer characteristic EH (i, t). Repress. On the other hand, when the user voice is detected by the user voice detection unit 701, the echo suppression unit 703 uses the echo component included in the air conduction sound signal of the frame t based on the transfer characteristic EH (i, t−1). Repress. That is, when user voice is detected by the user voice detection unit 701, the echo suppression unit 703 includes EH (i, t) = EH (i, t−1) and is included in the air conduction sound signal of the frame t. Suppresses the echo component.

FIG. 6 is a flowchart for explaining the content of the processing for suppressing the echo component.
As shown in FIG. 6, the echo suppression unit 703 first acquires the frequency spectrum of the air conduction sound signal and the reception signal of the frame t and the transfer characteristics (step S601). When the transfer characteristic EH (i, t) applied to the air conduction sound signal of the frame t is learned, the echo suppression unit 703 acquires the transfer characteristic EH (i, t) from the transfer characteristic learning unit 702. On the other hand, when the transfer characteristic EH (i, t) applied to the air conduction sound signal of the frame t is not learned, the echo suppression unit 703 reads the transfer characteristic EH (i, t−1) from the storage unit 711. It is assumed that the transfer characteristic EH (i, t).

  Next, the echo suppression unit 703 initializes a variable i for identifying a frequency band to i = 0 (step S602).

  Next, the echo suppression unit 703 calculates an echo estimation signal Eest (i, t) based on the amplitude spectrum Ref (i, t) of the received signal and the transfer characteristic EH (i, t) (step S603). . The echo estimation signal Eest (i, t) represents an estimated value of an echo component included in the amplitude spectrum of the frequency band i in the air conduction sound signal. The echo suppression unit 703 calculates an echo estimation signal Eest (i, t) using the following equation (2).

    Eest (i, t) = Ref (i, t) × EH (i, t) (2)

  Ref (i, t) and EH (i, t) in Expression (2) are transfer characteristics applied to the amplitude spectrum of the frequency band i and the air conduction sound signal of the frequency band i in the received signal, respectively.

  Next, the echo suppression unit 703 calculates an amplitude spectrum in which the echo component is suppressed based on the amplitude spectrum A (i) of the frequency band i in the air conduction sound signal and the echo estimation signal Eest (i, t) ( Step S604). The echo suppression unit 703 calculates an amplitude spectrum Amod (i, t) in which the echo component is suppressed using the following equation (3).

    Amod (i, t) = A (i, t) −Eest (i, t) (3)

  Next, the echo suppression unit 703 determines whether or not processing has been performed for all frequency bands (step S605). If there is an unprocessed frequency band (step S605; No), the echo suppression unit 703 updates the variable i to i + 1 (step S606), and performs the processes of steps S603 and S604. When processing is performed for all frequency bands (step S605; Yes), the echo suppression unit 703 converts the frequency spectrum of the air conduction sound signal in which the echo component is suppressed into a signal in the time domain (step S607). ). After that, the echo suppressing unit 703 ends the process of outputting the air conduction sound signal converted into the time domain signal and suppressing the echo component.

  The air conduction sound signal input from the first microphone 9 includes not only the voice uttered by the user in a state where the telephone conversation device 1 is in contact with the head, but also noise that propagates around the user. Therefore, when the process for detecting the user voice using the air conduction sound signal is performed, the user voice is included even though the user voice is included because the user voice is low or the noise is high. May not be detected. As described above, when the detection of the user voice included in the audio signal (frame) to be processed fails, there is a possibility that the echo component cannot be appropriately suppressed based on an appropriate transfer characteristic.

  On the other hand, in the echo suppression process of the present embodiment, the user voice is detected using the bone conduction sound signal input from the second microphone 10 as described above. The bone conduction sound collected by the second microphone 10 is a sound that propagates through the user's skull, skin tissue, or the like when the user (speaker) speaks, in other words, a sound that propagates through the solid. Therefore, the bone conduction sound signal input from the second microphone 10 has very few components such as noise (air conduction sound) propagating around the user. Therefore, by detecting the user voice using the bone conduction sound signal input from the second microphone 10, it is possible to suppress a decrease in the detection accuracy of the user voice. That is, according to the present embodiment, it is possible to reduce erroneous determination when determining whether to learn transfer characteristics based on the presence or absence of user voice. Therefore, according to the present embodiment, the echo component suppression performance is reduced by learning an erroneous transfer characteristic, in other words, the generation of echo and the deterioration of the transmission sound quality by using an incorrect (inappropriate) transfer characteristic. Etc. can be prevented.

[Second Embodiment]
FIG. 7 is a diagram illustrating a functional configuration of a transmission signal processing unit in the communication device according to the second embodiment.

  The call device 1 according to the present embodiment is a mobile communication device capable of making a call such as a mobile phone terminal, and the functional configuration thereof is the same as that of the call device 1 according to the first embodiment.

  As shown in FIG. 7, the transmission signal processing unit 7 in the communication device 1 of the present embodiment includes a user voice detection unit 701, a bone conduction correction unit 704, a transfer characteristic learning unit 702, and echo suppression. Unit 703, first storage unit 711, and second storage unit 712.

  The user voice detection unit 701 detects voice uttered by the user of the communication device 1 included in the bone conduction signal input from the second microphone 10. In the present embodiment, it is determined that the user voice is detected only when the user voice is detected from the bone conduction sound signal.

  The bone conduction sound correction unit 704 corrects the bone conduction sound signal based on the bone conduction sound correction characteristic stored in the second storage unit 712. In the transmission signal processing unit 7 of the present embodiment, when the user voice is detected by the user voice detection unit 701, the bone conduction sound correction unit 704 corrects the bone conduction signal. The bone conduction sound correction characteristic is a characteristic representing a correspondence relationship between the user voice in the bone conduction sound signal and the user voice in the air conduction sound signal. That is, in the transmission signal processing unit 7 according to the present embodiment, the bone conduction sound correction unit 704 corrects the bone conduction sound signal, thereby estimating the user voice included in the air conduction sound signal.

  The transfer characteristic learning unit 702 learns the transfer characteristic of sound propagating from the receiver 8 to the first microphone 9. The transfer characteristic learning unit 702 according to the present embodiment learns the transfer characteristic using the received signal and the air conduction sound signal when the user voice is not detected, and receives the received signal when the user voice is detected. The transfer characteristic is learned using the air conduction sound signal and the corrected bone conduction sound signal.

  The echo suppression unit 703 suppresses an echo component included in the air conduction sound signal based on the transfer characteristic learned by the transfer characteristic learning unit 702. The echo suppression unit 703 applies a transfer characteristic to the received signal to estimate an echo component included in the air conduction sound signal, and removes the estimated echo component from the air conduction sound signal.

  The first storage unit 711 stores the initial value of the transfer characteristic and the transfer characteristic learned by the transfer characteristic learning unit 702. The second storage unit 712 stores bone conduction sound correction characteristics.

  When the call connection between the communication device 1 of the present embodiment and another communication device is established, the transmission signal processing unit 7 of the audio signal processing unit 5 is sequentially input air conduction sound signal, bone conduction sound signal, Based on the received signal, echo suppression processing as shown in FIGS. 8A and 8B is performed.

  FIG. 8A is a flowchart (part 1) for explaining echo suppression processing according to the second embodiment. FIG. 8B is a flowchart (part 2) for explaining echo suppression processing according to the second embodiment.

  As shown in FIG. 8A, the transmission signal processing unit 7 first initializes a variable t for identifying a frame, which is a processing unit of an audio signal, to 1 (step S1).

  Next, the transmission signal processing unit 7 inputs the air conduction sound signal, the bone conduction sound signal, and the reception signal of the frame t (step S2). In step S <b> 2, for example, the transmission signal processing unit 7 inputs the bone conduction sound signal and the reception signal to the user voice detection unit 701, and inputs the air conduction sound signal to the transfer characteristic learning unit 702.

  Next, the transmission signal processing unit 7 performs user voice detection processing (step S3) using the bone conduction sound signal. The process of step S3 is performed by the user voice detection unit 701. The user voice detection unit 701 performs a process of detecting user voice from the bone conduction sound signal of the frame t (processing target frame) input in step S2. The user voice detection unit 701 performs, for example, the processes of steps S301 to S305 described in the first embodiment as the process of step S3 (see FIG. 4). Further, the user voice detection unit 701 determines whether user voice has been detected based on the result of the user voice detection process (step S4). The user voice detection unit 701 determines that the user voice has been detected when the user voice is detected from the bone conduction sound signal in step S3. When the user voice is not detected (step S4; No), the user voice detection unit 701 causes the transfer characteristic learning unit 702 to learn the transfer characteristic using the received signal and the air conduction sound signal (step S5). ). In this case, the transfer characteristic learning unit 702 performs, for example, the processes of steps S501 to S506 described in the first embodiment as the process of step S5 (see FIG. 5).

  On the other hand, when the user voice is detected (step S4; Yes), the user voice detection unit 701 causes the bone conduction sound correction unit 704 to correct the bone conduction sound signal (step S11). The bone conduction sound correction unit 704 corrects each amplitude spectrum by applying the bone conduction sound correction characteristic read from the second storage unit 712 to the amplitude spectrum of each frequency band in the frequency spectrum of the bone conduction signal. When the bone conduction sound signal is corrected, the bone conduction sound correction unit 704 causes the transfer characteristic learning unit 702 to learn the transfer characteristic using the received signal, the air conduction sound signal, and the corrected bone conduction signal (step S12). To do. In this case, the transfer characteristic learning unit 702 performs, for example, the same processes as steps S501 to S506 described in the first embodiment as the process of step S12. However, in step S12, the transfer characteristic learning unit 702 obtains the frequency spectrum (amplitude spectrum) of the received signal, the air conduction sound signal, and the corrected bone conduction sound signal in frame t as a process corresponding to step S501. I do. In step S12, the transfer characteristic learning unit 702 performs the processing corresponding to step S504, the amplitude spectrum of the reception signal, the air conduction sound signal, and the corrected bone conduction signal in frame t, and the transfer characteristic EH (i, t -1) and a process of calculating the transfer characteristic EH (i, t).

  After completing the learning of the transfer characteristic in step S5 or S12, the transmission signal processing unit 7 then suppresses the echo component of the air conduction sound signal based on the transfer characteristic (step S6). The echo suppression unit 703 performs the process in step S6. The echo suppression unit 703 performs, for example, the processes of steps S601 to S607 described in the first embodiment as the process of step S6 (see FIG. 6).

  When the echo suppression unit 703 finishes the process of step S6, the transmission signal processing unit 7 determines whether or not the frame t in which the echo component is suppressed is the final frame (step S7). When the frame t in which the echo component is suppressed is not the final frame (step S7; No), the transmission signal processing unit 7 updates the variable t to t + 1 (step S8), and steps S2 to S6 and S11 for the subsequent frames. , And S12. On the other hand, when the frame t in which the echo component is suppressed is the final frame (step S7; Yes), the transmission signal processing unit 7 ends the echo suppression process.

  As described above, in the echo suppression processing according to the present embodiment, when the user voice is not detected, the transfer characteristic is learned using the air conduction sound signal, and when the user voice is detected, the air conduction sound is detected. The transfer characteristic is learned using the signal and the bone conduction sound signal. Further, when learning the transfer characteristic using the air conduction sound signal and the bone conduction sound signal, the bone conduction sound correction unit 704 corrects the bone conduction sound signal based on the bone conduction sound correction characteristic (step S11). )I do. The bone conduction sound correction unit 704 performs the process shown in FIG. 9 as the process of step S11.

FIG. 9 is a flowchart for explaining the content of the process for correcting the bone conduction sound signal.
As shown in FIG. 9, the bone conduction sound correction unit 704 first acquires the frequency spectrum of the bone conduction sound signal of the frame t (step S1101). The bone conduction sound correction unit 704 acquires the frequency spectrum of the bone conduction sound signal from the user voice detection unit 701 or the frequency analysis unit not shown in FIG.

  Next, the bone conduction sound correcting unit 704 initializes a variable i for identifying a frequency band to i = 0 (step S1102).

  Next, the bone conduction sound correction unit 704 generates a bone conduction sound signal based on the amplitude spectrum B (i, t) of the frequency band i in the bone conduction signal and the bone conduction sound correction characteristic coef (i, t). The amplitude spectrum of the frequency band at is corrected (step S1103). In step S1103, the bone conduction sound correction unit 704 calculates an amplitude spectrum correction value Bmod (i, t) using the equation (4).

    Bmod (i, t) = B (i, t) × coef (i, t) (4)

  Next, the bone conduction sound correction unit 704 determines whether or not processing has been performed for all frequency bands (step S1104). When there is an unprocessed frequency band (step S1104; No), the bone conduction sound correction unit 704 updates the variable i to i + 1 (step S1105), and performs the process of step S1103. When processing is performed for all frequency bands (step S1104; Yes), the bone conduction sound correcting unit 704 outputs the corrected bone conduction signal (amplitude spectrum Bmod (i, t) of each frequency band). The process is transmitted to the transfer characteristic learning unit 702, and the process of correcting the bone conduction sound signal is terminated.

  When the bone conduction sound correction unit 704 corrects the bone conduction sound signal in this way, the transfer characteristic learning unit 702 learns the transmission characteristic using the received signal, the air conduction sound signal, and the corrected bone conduction signal. To do. In this case, the transfer characteristic learning unit 702 learns (calculates) the transfer characteristic applied to the air conduction sound signal of the frame t using the following formula (5).

  A (i, t) in Expression (5) is an amplitude spectrum of the frequency band i in the air conduction sound signal of the current frame (frame t). Ref (i, t) in Expression (5) is an amplitude spectrum of the frequency band i in the received signal of the current frame. EH (i, t−1) in Expression (5) is a transfer characteristic applied to the amplitude spectrum of the frequency band i in the air conduction sound signal one frame before (frame t−1). Α in Expression (5) is a transfer characteristic update coefficient, and in the present embodiment, a constant of 0 <α <1 (for example, α = 0.99). That is, in the transfer characteristic learning method using equation (5), the transfer characteristic is determined based on the received signal of frame t and the audio signal obtained by subtracting the bone conduction sound signal corrected from the air conduction sound signal of frame t. learn.

  Note that the bone conduction sound correction characteristic used in the process of correcting the bone conduction sound signal (step S11) is calculated in advance using an information processing device different from the communication device 1, for example, by the method shown in FIG. Keep it.

FIG. 10 is a flowchart for explaining an example of a method for calculating the bone conduction sound correction characteristic.
As illustrated in FIG. 10, the information processing apparatus that calculates the bone conduction sound correction characteristic first acquires a sample of the air conduction sound signal and the frequency spectrum of the bone conduction sound signal (step S21). In step S <b> 21, the information processing apparatus acquires an air conduction sound signal input from the first microphone 9 and a bone conduction sound signal input from the second microphone 10 of the communication device 1 for each of a plurality of frames.

  Next, the information processing apparatus initializes a variable i for identifying a frequency band to i = 0 (step S22).

  Next, the information processing apparatus calculates an average amplitude spectrum Fa (i) of the air conduction sound signal and an average amplitude spectrum Fb (i) of the bone conduction sound signal (step S23). The average amplitude spectrum Fa (i) of the air conduction sound signal is calculated based on the amplitude spectrum of the frequency band i in the air conduction sound signals for a plurality of frames. Similarly, the average amplitude spectrum Fb (i) of the bone conduction sound signal is calculated based on the amplitude spectrum of the frequency band i in the bone conduction sound signals for a plurality of frames.

  Next, the information processing apparatus uses the average amplitude spectrum Fa (i) of the air conduction sound signal in the frequency band i and the average amplitude spectrum Fb (i) of the bone conduction sound signal to the amplitude spectrum in the frequency band i. A bone conduction sound correction characteristic coef (i) is calculated (step S24). The bone conduction sound correction characteristic coef (i) is calculated by, for example, the following formula (6).

  Next, the information processing apparatus determines whether or not processing has been performed for all frequency bands (step S25). When there is an unprocessed frequency band (step S25; No), the information processing apparatus updates the variable i to i + 1 (step S26), and performs the processes of steps S23 and S24. When the processing is performed on all the frequency bands (step S25; Yes), the information processing apparatus ends the bone conduction sound correction characteristic calculation processing. The set of bone conduction sound correction characteristics coef (i) of each frequency band i obtained in this way is stored in the second storage unit 712 of the communication device 1.

  The bone conduction sound correction characteristic coef (i) calculated by the equation (6) has an average amplitude spectrum Fa (of the frequency band i in the air conduction sound signal to the average amplitude spectrum Fb (i) of the frequency band i in the bone conduction signal. i). Therefore, correcting the bone conduction sound signal by the equation (4) using the bone conduction sound correction characteristic coef (i) can be said to estimate the user voice included in the air conduction sound signal from the bone conduction sound signal. . That is, by correcting the bone conduction sound signal using Expression (4), the transmission signal processing unit 7 can estimate the user's voice included in the air conduction sound signal. Therefore, the transfer characteristic learning unit 702 can estimate an air conduction sound signal that does not include the user voice based on the air conduction sound signal and the corrected bone conduction sound signal. Therefore, even when the air conduction sound signal input from the first microphone 9 includes the user sound, the transfer characteristic learning unit 702 is equivalent to the case where the user sound is not included based on the estimated air conduction sound signal. It is possible to learn transfer characteristics with a certain degree of reliability. That is, according to the echo suppression processing according to the present embodiment, when the user voice included in the air conduction sound signal input from the first microphone 9 is not correctly detected, an erroneous transfer characteristic is learned. Therefore, it is possible to prevent a reduction in echo component suppression performance.

  Further, in the echo suppression processing according to the present embodiment, even when the user's voice is included in the air conduction sound signal input from the first microphone 9, the user voice is removed from the air conduction sound signal to obtain the transfer characteristic. It is possible to learn (estimate). Therefore, according to the echo suppression processing of the present embodiment, even when the transfer characteristic changes during the user's utterance, it is possible to learn the appropriate transfer characteristic and suppress the echo component. Therefore, even during the user's utterance, it is possible to prevent the occurrence of echoes and the deterioration of the transmission sound quality due to using an incorrect (inappropriate) transfer characteristic.

[Third Embodiment]
FIG. 11 is a diagram illustrating a functional configuration of a transmission signal processing unit in the communication device according to the third embodiment.

  The communication device 1 according to the present embodiment is a mobile communication device capable of making a call such as a mobile phone terminal, and the functional configuration thereof is the same as that of the call device 1 according to the first embodiment.

  As shown in FIG. 11, the transmission signal processing unit 7 in the communication device 1 of the present embodiment includes a user voice detection unit 701, a reliability calculation unit 705, a bone conduction sound correction unit 704, and a transfer characteristic. A learning unit 702 and an echo suppression unit 703 are included. The transmission signal processing unit 7 includes a first storage unit 711 and a second storage unit 712.

  The user voice detection unit 701 detects voice uttered by the user of the communication device 1 included in the bone conduction signal input from the second microphone 10. Also in this embodiment, the user voice detection unit 701 determines that the user voice has been detected when the user voice is detected from the bone conduction sound signal.

  The reliability calculation unit 705 calculates the reliability of the bone conduction sound signal input from the second microphone 10. Here, the reliability of the bone conduction sound signal is a value representing the reliability that the input bone conduction signal is an audio signal reflecting the voice (bone conduction sound) uttered by the user. The reliability calculation unit 705 of the present embodiment calculates the reliability of the bone conduction sound signal based on the bone conduction sound signal and the air conduction sound signal. In the transmission signal processing unit 7 of the present embodiment, when the user voice is detected by the user voice detection unit 701, the reliability calculation unit 705 calculates the reliability of the bone conduction sound signal. The reliability of the bone conduction sound signal calculated by the reliability calculation unit 705 is used to determine whether or not to use the corrected bone conduction signal for learning of transfer characteristics.

  The bone conduction sound correction unit 704 corrects the bone conduction sound signal based on the bone conduction sound correction characteristic stored in the second storage unit 712. In the transmission signal processing unit 7 of the present embodiment, when the user voice is detected by the user voice detection unit 701 and the reliability of the bone conduction signal is larger than the threshold value, the bone conduction sound correction unit 704 generates a bone. Correct the sound guide signal.

  The transfer characteristic learning unit 702 learns the transfer characteristic of sound propagating from the receiver 8 to the first microphone 9. The transfer characteristic learning unit 702 according to the present embodiment detects the user signal and the voice when the user voice is not detected, and when the user voice is detected but the reliability of the bone conduction sound signal is equal to or less than the threshold. The transfer characteristic is learned using the sound guide signal. In addition, the transfer characteristic learning unit 702 detects the user voice and, when the reliability of the bone conduction sound signal is larger than the threshold value, uses the reception signal, the air conduction sound signal, and the corrected bone conduction sound signal. Learn transfer characteristics.

  The echo suppression unit 703 suppresses an echo component included in the air conduction sound signal based on the transfer characteristic learned by the transfer characteristic learning unit 702. The echo suppression unit 703 applies a transfer characteristic to the received signal to estimate an echo component included in the air conduction sound signal, and removes the estimated echo component from the air conduction sound signal.

  The first storage unit 711 stores the initial value of the transfer characteristic and the transfer characteristic learned by the transfer characteristic learning unit 702. The second storage unit 712 stores bone conduction sound correction characteristics.

  When the call connection between the communication device 1 of the present embodiment and another communication device is established, the transmission signal processing unit 7 of the audio signal processing unit 5 is sequentially input air conduction sound signal, bone conduction sound signal, Based on the received signal, echo suppression processing as shown in FIGS. 12A and 12B is performed.

  FIG. 12A is a flowchart (part 1) for explaining echo suppression processing according to the third embodiment. FIG. 12B is a flowchart (part 2) illustrating the echo suppression processing according to the third embodiment.

  As shown in FIG. 12A, the transmission signal processing unit 7 first initializes a variable t for identifying a frame, which is a processing unit of the audio signal, to 1 (step S1).

  Next, the transmission signal processing unit 7 inputs the air conduction sound signal, the bone conduction sound signal, and the reception signal of the frame t (step S2). In step S <b> 2, the transmission signal processing unit 7 inputs a bone conduction sound signal and a reception signal to the user voice detection unit 701, for example. In addition, the transmission signal processing unit 7 inputs an air conduction sound signal to the transfer characteristic learning unit 702, for example. Further, the transmission signal processing unit 7 inputs the bone conduction sound signal and the air conduction sound signal to the reliability calculation unit 705.

  Next, the transmission signal processing unit 7 performs user voice detection processing (step S3) using the bone conduction sound signal. The process of step S3 is performed by the user voice detection unit 701. The user voice detection unit 701 performs a process of detecting user voice from the bone conduction sound signal of the frame t (processing target frame) input in step S2. The user voice detection unit 701 performs, for example, the processes of steps S301 to S305 described in the first embodiment as the process of step S3 (see FIG. 4). Further, the user voice detection unit 701 determines whether user voice has been detected based on the result of the user voice detection process (step S4). The user voice detection unit 701 determines that the user voice has been detected when the user voice is detected from the bone conduction sound signal. When the user voice is not detected (step S4; No), the user voice detection unit 701 causes the transfer characteristic learning unit 702 to learn the transfer characteristic using the received signal and the air conduction sound signal (step S5). ). In this case, the transfer characteristic learning unit 702 performs, for example, the processes of steps S501 to S506 described in the first embodiment as the process of step S5 (see FIG. 5).

  On the other hand, when the user voice is detected (step S4; Yes), the user voice detection unit 701 causes the reliability calculation unit 705 to calculate the reliability of the bone conduction sound signal (step S9). For example, the reliability calculation unit 705 calculates the reliability of the bone conduction sound signal based on the correlation coefficient between the bone conduction sound signal and the air conduction sound signal. In addition, the reliability calculation unit 705 notifies the user voice detection unit 701 of the reliability of the calculated bone conduction sound signal. Upon receiving the reliability of the bone conduction sound signal, the user voice detection unit 701 determines whether the received reliability is greater than the threshold value TH (step S10). When the reliability of the bone conduction sound signal is equal to or less than the threshold value TH (step S10; No), the user voice detection unit 701 learns the transfer characteristic using the reception signal and the air conduction sound signal in the transfer characteristic learning unit 702. The process (step S5) is performed.

  When the reliability of the bone conduction sound signal is larger than the threshold value TH (step S10; Yes), the user sound detection unit 701 causes the bone conduction sound correction unit 704 to correct the bone conduction sound signal (step S11). The bone conduction sound correction unit 704 performs, for example, the processes of steps S1101 to S1105 described in the second embodiment as the process of step S11 (see FIG. 9).

  When the bone conduction sound signal is corrected, the bone conduction sound correction unit 704 causes the transfer characteristic learning unit 702 to learn the transfer characteristic using the received signal, the air conduction sound signal, and the corrected bone conduction signal (step S12). To do. In this case, the transfer characteristic learning unit 702 performs, for example, the same processes as steps S501 to S506 described in the first embodiment as the process of step S12. However, in step S12, the transfer characteristic learning unit 702 obtains the frequency spectrum (amplitude spectrum) of the received signal, the air conduction sound signal, and the corrected bone conduction sound signal in frame t as a process corresponding to step S501. I do. In step S12, the transfer characteristic learning unit 702 performs the processing corresponding to step S504, the amplitude spectrum of the reception signal, the air conduction sound signal, and the corrected bone conduction signal in frame t, and the transfer characteristic EH (i, t -1) and a process of calculating the transfer characteristic EH (i, t).

  When the transmission characteristic learning in step S5 or S12 is completed, the transmission signal processing unit 7 next suppresses the echo component of the air conduction sound signal based on the transmission characteristic as shown in FIG. 12B (step S6). . The echo suppression unit 703 performs the process in step S6. The echo suppression unit 703 performs, for example, the processes of steps S601 to S607 described in the first embodiment as the process of step S6 (see FIG. 6).

  When the echo suppression unit 703 finishes the process of step S6, the transmission signal processing unit 7 determines whether or not the frame t in which the echo component is suppressed is the final frame (step S7). When the frame t in which the echo component is suppressed is not the final frame (step S7; No), the transmission signal processing unit 7 updates the variable t to t + 1 (step S8), and steps S2 to S6, and The process of S9-S12 is performed. On the other hand, when the frame t in which the echo component is suppressed is the final frame (step S7; Yes), the transmission signal processing unit 7 ends the echo suppression process.

  As described above, in the echo suppression processing according to the present embodiment, whether or not learning of transfer characteristics using a bone conduction sound signal is performed based on the reliability of the bone conduction signal when a user voice is detected. judge. That is, even if the user voice is detected from the bone conduction signal, if the reliability of the bone conduction signal is low, the transfer characteristic is learned without using the bone conduction signal. Therefore, it is possible to prevent a decrease in the reliability of the transfer characteristic due to learning of the transfer characteristic using the bone conduction sound signal with low reliability. Therefore, when the user voice is detected from the bone conduction sound signal, the echo component can be more appropriately suppressed based on the highly reliable transfer characteristic.

FIG. 13 is a graph illustrating a method for calculating the reliability of the bone conduction sound signal.
In the echo suppression processing of this embodiment, as described above, it is determined whether or not to perform transfer characteristic learning using the bone conduction sound signal based on the reliability of the bone conduction sound signal. The reliability of the bone conduction sound signal is calculated based on the correlation coefficient between the bone conduction sound signal and the air conduction sound signal. The correlation coefficient corr between the bone conduction sound signal and the air conduction sound signal is calculated using the following equation (7).

N in Equation (7) is the number of samples in the frame, and N = 160 in the case of 8 kHz sampling. In the equation (7), sa _j and sb _j are the j-th sample in the air conduction sound signal and the j-th sample in the bone conduction sound signal, respectively.

  When calculating the reliability of the bone conduction sound signal based on the correlation coefficient corr, for example, it is calculated based on the correspondence between the correlation coefficient corr and the reliability R as shown in FIG. That is, the reliability R of the bone conduction sound signal is calculated using the following formula (8).

  The first correlation threshold corrL and the second correlation threshold corrH in Expression (8) are arbitrary values that satisfy 0 <corrL <corrH <1, for example, corrL = 0.2 and corrH = 0.7.

  In order to collect the bone conduction sound with the second microphone 10, the user needs to speak while the communication device 1 is in contact with the user's head. At this time, if the contact state between the user's head and the communication device 1 is unstable, there is a difference between the voice uttered by the user and the user voice included in the bone conduction sound signal. Therefore, it is difficult to correctly estimate the user voice included in the air conduction sound signal. Therefore, in the echo suppression processing according to the present embodiment, it is determined based on the reliability R of the bone conduction sound signal whether or not the bone conduction sound signal is used for learning transfer characteristics. When it is determined that the reliability of the bone conduction sound signal is low and it is difficult to correctly estimate the user voice included in the air conduction sound signal, the transmission signal processing unit 7 receives the reception signal and the air conduction sound signal. Learning transfer characteristics using only That is, even if the transmission signal processing unit 7 detects the user voice from the bone conduction signal, if the reliability R of the bone conduction signal is low, the transmission signal processing unit 7 learns the transfer characteristics without using the bone conduction signal. . Therefore, according to the present embodiment, it is possible to prevent learning an erroneous transfer characteristic by using a bone conduction sound signal with low reliability (in other words, a bone conduction sound signal that does not appropriately reflect the user voice). It becomes possible.

  When the user voice is detected and the reliability R of the bone conduction sound signal is low (step S10; No), for example, when the user voice is detected in the echo suppression processing according to the first embodiment. The echo component may be suppressed without learning the transfer characteristics.

  Further, when the reliability R of the bone conduction sound signal is calculated as in the present embodiment, instead of determining whether or not the bone conduction sound signal is used for learning of the transfer characteristics based on the reliability R, the bone conduction sound is determined. Depending on the signal reliability R, the value of the transfer characteristic update coefficient α in the equation (5) may be changed.

FIG. 14 is a graph illustrating a method for calculating the transfer coefficient update coefficient.
In Expression (5) used for transfer characteristic learning (calculation) using the bone conduction sound signal, the amplitude spectrum Bmod of the bone conduction sound signal in the calculated transmission characteristic EH (i, t) as the update coefficient α decreases. The contribution of (i, t) increases. Therefore, for example, when the reliability R of the bone conduction sound signal is low, the update coefficient α is determined so that the contribution degree of the bone conduction signal of the frame t in the calculated transfer characteristic EH (i, t) is small.

  When changing the update coefficient α of the transfer characteristic according to the reliability R of the bone conduction sound signal, the value of the update coefficient α is based on, for example, the correspondence between the reliability R and the update coefficient α as shown in FIG. To change. That is, when the update coefficient α of the transfer characteristic is changed according to the reliability R of the bone conduction sound signal, the update coefficient α is calculated using the following formula (9).

  The minimum value αmin of the update coefficient α is an arbitrary value of 0 <αmin <1, for example, αmin = 0.95. The first determination threshold RL and the second determination threshold RH are arbitrary values that satisfy 0 <RL <RH <1, for example, RL = 0.2 and RH = 0.7.

  When the update coefficient α is changed according to the reliability R of the bone conduction sound signal, for example, in the process of learning transfer characteristics using the bone conduction sound signal of FIG. 12A (step S12), the update coefficient is expressed by Expression (9). α is calculated (determined).

  As described above, when the reliability R of the bone conduction sound signal is low by changing the update coefficient α according to the reliability R so that the update coefficient α is increased when the reliability R of the bone conduction sound signal is low. It is possible to reduce the contribution degree of the bone conduction sound signal to the transfer characteristic calculated by Expression (5). Therefore, it is possible to prevent learning (calculation) of erroneous transfer characteristics using a bone conduction sound signal with low reliability (in other words, a bone conduction signal that does not properly reflect the user's voice). .

  Further, when the update coefficient α is determined with reference to the correspondence relationship between the reliability R of 0 ≦ R ≦ 1 and the update coefficient α as shown in FIG. 14, the update coefficient α becomes 1 when R <RL. The contribution degree of the bone conduction sound signal to the transfer characteristic calculated by Expression (5) is zero. Therefore, when determining the update coefficient α with reference to the correspondence relationship between the reliability R of 0 ≦ R ≦ 1 and the update coefficient α as shown in FIG. 14, for example, the determination in step S10 in FIG. 12A can be omitted. It is.

  When the transfer characteristic update coefficient α is changed in accordance with the reliability R of the bone conduction sound signal in the echo suppression processing of FIGS. 12A and 12B, the correspondence relationship between the reliability R and the update coefficient α is, for example, the reliability Only a range in which the degree R is greater than the threshold value TH (TH <R ≦ 1) may be prepared.

[Fourth Embodiment]
In the present embodiment, a communication device that calculates the reliability R of the bone conduction sound signal based on the pressure (pressing load) that the communication device 1 receives from the user's head will be described.

FIG. 15 is a diagram illustrating a functional configuration of a main part in the communication device according to the fourth embodiment.
As shown in FIG. 15, the communication device 1 of this embodiment includes a receiver 8, a first microphone 9, a second microphone 10, a transmission signal processing unit 7, and a pressure sensor 15. Note that the communication device 1 according to the present embodiment includes an RF transmission / reception unit 2, an antenna 3, a baseband processing unit 4, a received signal processing unit 8 and the like not shown in FIG.

  The pressure sensor 15 is used to detect the pressure that the communication device 1 receives from the user's head during a call. Therefore, the pressure sensor 15 is mounted on the call device 1 in a manner capable of detecting the pressure applied to the area where the user's head is in contact with the user's head in the plane facing the user's head during the call in the call device 1. The

  The transmission signal processing unit 7 includes a user voice detection unit 701, a reliability calculation unit 706, a bone conduction sound correction unit 704, a transfer characteristic learning unit 702, an echo suppression unit 703, and a first storage unit 711. And a second storage unit 712.

  The user voice detection unit 701 detects voice uttered by the user of the communication device 1 included in the bone conduction signal input from the second microphone 10. Also in this embodiment, the user voice detection unit 701 determines that the user voice has been detected when the user voice is detected from the bone conduction sound signal.

  The reliability calculation unit 706 calculates the reliability of the bone conduction sound signal input from the second microphone 10. The reliability calculation unit 706 of the present embodiment calculates the reliability of the bone conduction sound signal based on the pressure detection result by the pressure sensor 15. In the transmission signal processing unit 7 of the present embodiment, when the user voice is detected by the user voice detection unit 701, the reliability calculation unit 706 calculates the reliability of the bone conduction sound signal. The reliability of the bone conduction sound signal calculated by the reliability calculation unit 706 is used to determine whether or not the corrected bone conduction signal is used for learning transfer characteristics.

  The bone conduction sound correction unit 704 corrects the bone conduction sound signal based on the bone conduction sound correction characteristic stored in the second storage unit 712. In the transmission signal processing unit 7 of the present embodiment, when the user voice is detected by the user voice detection unit 701 and the reliability of the bone conduction signal is larger than the threshold value, the bone conduction sound correction unit 704 generates a bone. Correct the sound guide signal.

  The transfer characteristic learning unit 702 learns the transfer characteristic of sound propagating from the receiver 8 to the first microphone 9. The transfer characteristic learning unit 702 according to the present embodiment detects the user signal and the voice when the user voice is not detected, and when the user voice is detected but the reliability of the bone conduction sound signal is equal to or less than the threshold. The transfer characteristic is learned using only the sound guide signal. In addition, the transfer characteristic learning unit 702 detects the user voice and uses the received signal, the air conduction sound signal, and the corrected bone conduction sound signal when the reliability of the bone conduction sound signal is larger than the threshold value. To learn transfer characteristics.

  The echo suppression unit 703 suppresses an echo component included in the air conduction sound signal based on the transfer characteristic learned by the transfer characteristic learning unit 702. The echo suppression unit 703 applies a transfer characteristic to the received signal to estimate an echo component included in the air conduction sound signal, and removes the estimated echo component from the air conduction sound signal.

  The first storage unit 711 stores the initial value of the transfer characteristic and the transfer characteristic learned by the transfer characteristic learning unit 702. The second storage unit 712 stores bone conduction sound correction characteristics.

  When the call connection between the call device 1 of the present embodiment and another call device is established, the transmission signal processing unit 7 of the audio signal processing unit 5 performs the echo suppression processing shown in FIGS. 12A and 12B. In the echo suppression process according to the present embodiment, the reliability calculation unit 706 performs a process (step S9) of calculating the reliability R of the bone conduction sound signal. The reliability calculation unit 706 calculates the reliability R of the bone conduction sound signal based on the pressure detected by the pressure sensor 15 (in other words, the pressure received by the communication device 1 from the user's head).

  FIG. 16 is a graph for explaining a calculation method of the reliability of the bone conduction sound signal in the fourth embodiment.

  When calculating the reliability of the bone conduction sound signal based on the pressure detected by the pressure sensor 15, for example, it is calculated based on the correspondence between the pressure P and the reliability R as shown in FIG. That is, when calculating the reliability R of the bone conduction sound signal based on the pressure P detected by the pressure sensor 15, the reliability R is calculated using the following equation (10).

  The first pressure threshold value PL and the second pressure threshold value PH are arbitrary values satisfying 0 <PL <PH, for example, PL = 0.2 kPa and PH = 1.2 kPa.

  The greater the force with which the user presses the communication device 1 against the head, the greater the pressure P detected by the pressure sensor 15. In addition, when the user presses the communication device 1 against the head with a force larger than a predetermined pressing force, the bone conduction sound is correctly transmitted from the user's head to the communication device 1 (second microphone 10). The On the other hand, when the force with which the user presses the communication device 1 against the head is small, the bone conduction sound transmitted from the user's head to the communication device 1 (second microphone 10) becomes unstable. Therefore, in the echo suppression processing according to the present embodiment, transmission is performed only when the reliability R of the bone conduction sound signal calculated based on the pressure P received by the communication device 1 from the user's head is greater than the threshold value TH. A bone conduction sound signal corrected for characteristic learning is used. Thereby, it is possible to prevent learning (calculating) an erroneous transfer characteristic using a bone conduction sound signal that does not appropriately reflect the user's voice.

  The reliability R of the bone conduction sound signal is based on the first reliability Rcorr calculated based on the correlation coefficient between the bone conduction sound signal and the air conduction sound signal, and the pressure P detected by the pressure sensor 15. You may calculate by following formula (11) using the calculated 2nd reliability Rp.

    R = β × Rcorr + (1−β) × Rp (11)

  The first reliability Rcorr and the second reliability Rp in Expression (11) are calculated using, for example, Expression (8) and Expression (10), respectively. Further, β in the equation (11) is a weighting factor. The weighting coefficient β is an arbitrary value satisfying 0 ≦ β ≦ 1, for example, β = 0.5.

  The reliability (reliability) of the reliability R is increased by determining (calculating) the reliability R of the bone conduction sound signal based on a plurality of reliability calculated using different information as in Expression (11). It becomes possible. Therefore, it becomes possible to more effectively prevent learning (calculation) of erroneous transfer characteristics using a bone conduction sound signal that does not properly reflect the user voice.

  Note that the functional configuration of the transmission signal processing unit 7 (calling device 1) shown in the first to fourth embodiments is merely an example, and the echo suppression processing described in each embodiment can be executed. It may be configured as follows.

  Also, the flowcharts shown in FIGS. 3 to 6, 8A and 8B, 9, 10, 12 </ b> A and 12 </ b> B are only examples, and the processing content and processing procedure can be changed as appropriate.

  Moreover, the call device 1 according to the first to fourth embodiments can be realized by using, for example, a computer and a program executed by the computer. Hereinafter, the communication device 1 realized using a computer and a program will be described with reference to FIG.

FIG. 17 is a diagram illustrating a hardware configuration of a computer.
As shown in FIG. 17, the computer 20 includes a processor 2001, a main storage device 2002, an auxiliary storage device 2003, an input device 2004, a display device 2005, an interface device 2006, a communication control device 2007, and a storage medium. A driving device 2008. These elements 2001 to 2008 in the computer 20 are connected to each other by a bus 2010 so that data can be exchanged between the elements.

  The processor 2001 is an arithmetic processing unit such as a central processing unit (CPU), and controls the overall operation of the computer 20 by executing various programs including an operating system.

  The main storage device 2002 includes a read only memory (ROM) and a random access memory (RAM) not shown. In the ROM of the main storage device 2002, for example, a predetermined basic control program read by the processor 2001 when the computer 20 is started is recorded in advance. The RAM of the main storage device 2002 is used as a working storage area as necessary when the processor 2001 executes various programs. The RAM of the main storage device 2002 can be used for storing, for example, transfer characteristics and bone conduction sound correction characteristics.

  The auxiliary storage device 2003 is a storage device having a larger capacity than the main storage device 2002 such as a hard disk drive (HDD) or a solid state drive (SSD). The auxiliary storage device 2003 can store various programs executed by the processor 2001, various data, and the like. The auxiliary storage device 2003 can be used for storing, for example, a calling program including the processes shown in FIGS. Further, the auxiliary storage device 2003 can be used for storing, for example, transmission characteristics and bone conduction sound correction characteristics.

  The input device 2004 is, for example, a keyboard device or a touch panel device. When an operator (user) of the computer 20 performs an operation such as pressing the input device 2004, the input device 2004 transmits input information associated with the operation content to the processor 2001.

  The display device 2005 is a liquid crystal display, for example. The display device 2005 displays various text screens, images, and the like according to display data transmitted from the processor 2001 or the like.

  The interface device 2006 is a device that connects the computer 20 to other electronic devices, and includes a Universal Serial Bus (USB) standard connector and the like. Devices that can be connected to the computer 20 by the interface device 2006 include a receiver 8, a first microphone 9, a second microphone 10, and the like.

  The communication control device 2007 is a device that controls various communications between the computer 20 and other communication devices via the network 21 such as a telephone network or the Internet. Control of communication performed by the communication control apparatus 2007 includes control of a call (transmission and reception of audio signals) between the computer 20 and another call apparatus 22 via the network 21.

  The storage medium drive device 2008 reads programs and data recorded in a portable storage medium (not shown) and writes data stored in the auxiliary storage device 2003 to the portable storage medium. As the portable storage medium, for example, a flash memory equipped with a USB standard connector can be used. Further, as a portable storage medium, an optical disc such as a Compact Disk (CD), a Digital Versatile Disc (DVD), and a Blu-ray Disc (Blu-ray is a registered trademark) can be used.

  In the computer 20, the processor 2001 reads out the program including the processes of FIGS. 3 to 6 from the auxiliary storage device 2003 or the like, and suppresses the echo component of the air conduction sound signal input from the first microphone 9. An audio signal is transmitted to and received from 22.

  Note that the computer 20 used as the communication device 1 does not have to include all the components shown in FIG. 17, and some components can be omitted depending on the application and conditions. For example, it is possible to omit the interface device 2006 and connect the receiver 8, the first microphone 9, the second microphone 10 and the like directly to the printed circuit board.

The following additional notes are further disclosed with respect to the embodiments including the examples described above.
(Appendix 1)
A receiver that outputs an audio signal as air conduction sound;
A first microphone that collects the air conduction sound;
A second microphone for picking up bone conduction sound;
An echo suppression unit that suppresses an echo component included in an air conduction sound signal input from the first microphone based on a transmission characteristic of sound from the receiver to the first microphone;
Transmission that learns the transfer characteristic based on the received signal to be output from the receiver and the air conduction sound signal when the input level of the bone conduction sound signal inputted from the second microphone is equal to or lower than a predetermined threshold value A characteristic learning unit;
A call device comprising:
(Appendix 2)
The transfer characteristic learning unit, when an input level of the bone conduction sound signal exceeds the threshold, the received signal, and a voice signal obtained by subtracting the bone conduction signal from the air conduction signal; Learning the transfer characteristics based on
The telephone call device according to supplementary note 1, wherein:
(Appendix 3)
A bone conduction sound correction unit that corrects the bone conduction sound signal based on the bone conduction sound correction characteristic calculated in advance based on the air conduction sound signal and the bone conduction sound signal, and the bone conduction sound signal; Prepared,
When the input level of the bone conduction sound signal exceeds the threshold, the transfer characteristic learning unit corrects the bone conduction sound signal corrected by the bone conduction sound correction unit from the received signal and the air conduction sound signal. Learning transfer characteristics based on the audio signal obtained by subtracting
The telephone call device according to Supplementary Note 2, wherein
(Appendix 4)
A reliability for calculating a reliability of the bone conduction sound signal reflecting a sound emitted from the sound source of the bone conduction sound when an input level of the bone conduction sound signal exceeds the threshold; A calculation unit,
The transfer characteristic learning unit is configured to subtract the bone conduction sound signal from the received signal and the air conduction sound signal when the reliability of the bone conduction sound signal exceeds a predetermined threshold. Learn transfer characteristics based on
The telephone call device according to Supplementary Note 2, wherein
(Appendix 5)
The reliability calculation unit calculates the reliability of the bone conduction sound signal based on a correlation coefficient between the bone conduction sound signal and the air conduction sound signal.
The telephone call device according to supplementary note 4, characterized in that:
(Appendix 6)
The communication device further includes a pressure sensor provided in the communication device in a mode of detecting pressure received from the bone conduction sound source,
The reliability calculation unit calculates the reliability of the bone conduction sound signal based on the pressure detected by the pressure sensor.
The telephone call device according to supplementary note 4, characterized in that:
(Appendix 7)
The transfer characteristic learning unit, when an input level of the bone conduction sound signal exceeds the threshold value and the speech signal includes voice, the bone conduction sound from the reception signal and the air conduction sound signal. Learning transfer characteristics based on the audio signal obtained by subtracting the signal,
The telephone call device according to Supplementary Note 2, wherein
(Appendix 8)
Obtaining an air conduction sound signal from a first microphone that collects the air conduction sound and obtaining a bone conduction sound signal from a second microphone that collects the bone conduction sound;
Transfer characteristics of sound from the receiver to the first microphone based on the reception signal output from the receiver and the air conduction sound signal when the input level of the bone conduction sound signal is equal to or lower than a predetermined threshold value To learn and
Correcting the air conduction sound signal input from the first microphone based on the transfer characteristic;
An audio signal correction program that causes a computer to execute processing.
(Appendix 9)
When the input level of the bone conduction sound signal exceeds the threshold, the transfer characteristic is determined based on the received signal and an audio signal obtained by subtracting the bone conduction signal from the air conduction sound signal. Learning, causing the computer to perform further processing,
The audio signal correction program according to appendix 8, wherein
(Appendix 10)
The process of learning the transfer characteristic when the input level of the bone conduction sound signal exceeds the threshold,
Based on the bone conduction sound correction characteristic calculated based on the air conduction sound signal and the bone conduction sound signal in advance, and the bone conduction sound signal, the bone conduction sound signal is corrected,
Learning a transfer characteristic based on the received signal and an audio signal obtained by subtracting the bone conduction sound signal corrected from the air conduction sound signal,
The audio signal correction program according to Supplementary Note 9, wherein
(Appendix 11)
The process of learning the transfer characteristic when the input level of the bone conduction sound signal exceeds the threshold,
Calculating the reliability that the bone conduction sound signal is an audio signal reflecting the sound emitted by the bone conduction sound source;
When the reliability of the bone conduction sound signal exceeds a predetermined threshold value, transfer characteristics are determined based on the received signal and an audio signal obtained by subtracting the bone conduction signal from the air conduction sound signal. Learn, process,
The audio signal correction program according to Supplementary Note 9, wherein
(Appendix 12)
Calculating a reliability of the bone conduction sound signal based on a correlation coefficient between the bone conduction sound signal and the air conduction sound signal, causing the computer to execute a process;
The audio signal correction program according to appendix 10, wherein the program is corrected.
(Appendix 13)
The process of calculating the reliability of the bone conduction sound signal,
Obtaining the pressure received by the case where the second microphone is installed from the bone conduction sound source;
Calculating the reliability of the bone conduction sound signal based on the acquired pressure, including processing,
The audio signal correction program according to appendix 10, wherein the program is corrected.
(Appendix 14)
When the input level of the bone conduction sound signal exceeds the threshold, the transfer characteristic is determined based on the received signal and an audio signal obtained by subtracting the bone conduction signal from the air conduction sound signal. Learning, causing the computer to perform further processing,
The audio signal correction program according to Supplementary Note 9, wherein
(Appendix 15)
The process of correcting the air conduction sound signal is as follows:
Based on the transfer characteristics and the received signal, the speech component output from the receiver included in the air conduction sound signal is estimated,
Subtracting the estimated audio component from the air conduction sound signal, including processing.
The audio signal correction program according to appendix 8, wherein

DESCRIPTION OF SYMBOLS 1 Call apparatus 2 RF transmission / reception part 3 Antenna 4 Baseband process part 5 Audio | voice signal process part 6 Received signal process part 7 Transmission signal process part 701 User voice detection part 702 Transfer characteristic learning part 703 Echo suppression part 704 Bone conduction correction Units 705 and 706 reliability calculation unit 711 (first) storage unit 712 second storage unit 8 receiver 9 first microphone 10 second microphone 15 pressure sensor 20 computer 2001 processor 2002 main storage device 2003 auxiliary storage device 2004 Input device 2005 Display device 2006 Interface device 2007 Communication control device 2008 Storage medium drive device 21 Network 22 Communication device

Claims (7)

A receiver that outputs an audio signal as air conduction sound;
A first microphone that collects the air conduction sound;
A second microphone for picking up bone conduction sound;
An echo suppression unit that suppresses an echo component included in an air conduction sound signal input from the first microphone based on a transmission characteristic of sound from the receiver to the first microphone;
Transmission that learns the transfer characteristic based on the received signal to be output from the receiver and the air conduction sound signal when the input level of the bone conduction sound signal inputted from the second microphone is equal to or lower than a predetermined threshold value A characteristic learning unit;
A call device comprising: The transfer characteristic learning unit, when an input level of the bone conduction sound signal exceeds the threshold, the received signal, and a voice signal obtained by subtracting the bone conduction signal from the air conduction signal; Learn transfer characteristics based on
The call device according to claim 1. A bone conduction sound correction unit that corrects the bone conduction sound signal based on the bone conduction sound correction characteristic calculated in advance based on the air conduction sound signal and the bone conduction sound signal, and the bone conduction sound signal; Prepared,
When the input level of the bone conduction sound signal exceeds the threshold, the transfer characteristic learning unit corrects the bone conduction sound signal corrected by the bone conduction sound correction unit from the received signal and the air conduction sound signal. Learning transfer characteristics based on the audio signal obtained by subtracting
The communication device according to claim 2. A reliability for calculating a reliability of the bone conduction sound signal reflecting a sound emitted from the sound source of the bone conduction sound when an input level of the bone conduction sound signal exceeds the threshold; A calculation unit,
The transfer characteristic learning unit is configured to subtract the bone conduction sound signal from the received signal and the air conduction sound signal when the reliability of the bone conduction sound signal exceeds a predetermined threshold. Learn transfer characteristics based on
The communication device according to claim 2. The reliability calculation unit calculates the reliability of the bone conduction sound signal based on a correlation coefficient between the bone conduction sound signal and the air conduction sound signal.
The call device according to claim 4, wherein: The communication device further includes a pressure sensor provided in the communication device in a mode of detecting pressure received from the bone conduction sound source,
The reliability calculation unit calculates the reliability of the bone conduction sound signal based on the pressure detected by the pressure sensor.
The call device according to claim 4, wherein: Obtaining an air conduction sound signal from a first microphone that collects the air conduction sound and obtaining a bone conduction sound signal from a second microphone that collects the bone conduction sound;
Transfer characteristics of sound from the receiver to the first microphone based on the reception signal output from the receiver and the air conduction sound signal when the input level of the bone conduction sound signal is equal to or lower than a predetermined threshold value To learn and
Correcting the air conduction sound signal input from the first microphone based on the transfer characteristic;
An audio signal correction program that causes a computer to execute processing.

Images