JP6079179B2 - Hands-free call device - Google Patents

Description

  The present invention relates to a hands-free call device.

  In a hands-free communication device, a device that reduces the influence of echoes generated by the sound output from a speaker around a microphone has been proposed.

  For example, in the technique described in Patent Document 1, based on the signal level of the transmission audio signal (or reception audio signal), the total gain of the transmission audio signal and the reception audio signal is set to “1” or less so that each audio signal Amplify or reduce the signal level. For example, when a transmission audio signal is input, control is performed to amplify the signal level of the transmission audio signal and reduce the signal level of the reception audio signal.

  A known echo cancellation technique for removing echoes is generally configured using an adaptive filter that learns the transfer function of an echo path from a received voice signal and a transmitted voice signal (for example, Patent Document 2). That is, for the transmission voice signal, a pseudo echo is generated from the reception voice signal by an adaptive filter, and the echo component is subtracted from the transmission voice signal.

  However, when learning the transfer function, there is a problem that the speech uttered by the user becomes a disturbance and the estimation accuracy of the transfer function is lowered. Therefore, in Patent Document 2, when the movement of the user's mouth is detected by the camera, and the movement of the user's mouth is detected, it is determined that the user is speaking, and updating of the adaptive filter is stopped. To do. This improves the accuracy of the pseudo echo.

JP-A-9-162786 JP 2000-295338 A

  However, in the technique described in Patent Document 1, when a transmission voice signal is input, if the reception voice signal has a signal level corresponding to a low voice of about the noise level, the signal level of the reception voice signal is reduced. As a result, it is feared that the received voice output from the speaker cannot be heard. Conversely, when the received voice signal is input and the transmitted voice signal has a signal level corresponding to a low voice of about the noise level, the other party cannot hear the user's voice.

  Further, as in the technique described in Patent Document 2, it is possible to make the echo difficult to hear by increasing the accuracy of the pseudo echo, but it is not always possible to completely remove the echo. For example, in a situation where the transfer function changes dynamically, such as when the user is driving, the transfer function corresponding to the actual echo path changes while the update of the adaptive filter is stopped. And since a difference will arise between the transfer function which the adaptive filter learned, and an actual transfer function, the precision of a pseudo echo falls. As a result, annoying echoes are transmitted to the other party.

  The present invention has been made in view of the above circumstances. The purpose of the present invention is to reduce the signal level of the received audio signal when the received audio signal is received from the other party during the user's speech. Even if the signal level corresponds to, the user can listen to the received voice from the other party in the received voice signal, and even if the user's uttered voice is low voice, the user can talk without reducing the signal level. An object of the present invention is to provide a hands-free communication device that can transmit to the other party and reduce the influence of echo transmitted from the user side to the other party when the user is not speaking.

In order to achieve the object, the present invention includes a camera (1) that captures an image of a user's face,
A camera image processing unit (2) for detecting movement of the user's mouth from image data captured by the camera;
A hands-free call device (100) comprising: a signal level adjustment unit (3) for adjusting a signal level of a transmission voice signal transmitted from a user to a call partner by adjusting an amplification factor by a continuous or stepwise change. Because
The signal level adjustment unit includes:
The amplification factor is configured to be set to a value located between 0 and 1, and when the movement of the mouth is not detected by the camera image processing unit, the amplification factor is set to a value smaller than 1 .
When mouth movement is detected by the camera image processing unit, the amplification factor of the transmission audio signal is set to a value of 1 or more.

  According to such a configuration, the signal level of the transmission audio signal is reduced when the movement of the user's mouth is not detected. Therefore, in a situation where the transfer function of the echo path is dynamically changed, even if an echo is input to the microphone when the user's mouth is not moving, the influence of the echo can be reduced without being affected by the environmental change.

  Furthermore, since the signal level of the transmission voice signal is adjusted and the signal level of the reception voice signal is not reduced, even when the reception voice signal from the communication partner is received, the signal level is low and the user utters. It is possible to listen to the received voice from the other party.

  In addition, even if there is an input of a received voice signal from the other party, if the movement of the user's mouth is detected by the camera image processing unit, the amplification factor of the signal level adjustment unit is set to 1 or more. Even if the uttered voice is quiet, it can be communicated to the other party.

It is a block diagram showing the structure of the hands-free call device in this embodiment. It is a flowchart which shows the flow of control of this embodiment. It is a flowchart showing the flow of the amplification factor adjustment process in the control of this embodiment. It is a conceptual diagram explaining the action | operation of this embodiment. It is a conceptual diagram explaining the action | operation of this embodiment. It is a conceptual diagram explaining the action | operation of this embodiment. It is a block diagram showing the modification of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of the hands-free call device 100 of the present invention. 1 includes a signal processing unit 1 and a camera 6, and the signal processing unit 1 is connected to a speaker 4, a microphone 5, a camera 6, and a telephone interface unit 7. Note that the telephone interface unit 7 and the hands-free call device 100 may be connected by wireless such as Bluetooth (registered trademark) in addition to wired communication. A telephone (not shown) outputs a received voice signal received via a network such as a public line to the telephone interface unit 7, and the telephone interface unit 7 outputs the received voice signal to the signal processing unit 1. Also, the transmission voice signal input to the telephone from the telephone interface unit 7 is transmitted to the call partner via the network. Here, the received voice signal includes an echo component and noise other than the echo component in addition to the received voice from the other party. In addition to the uttered voice uttered by the user, the transmitted voice signal also includes an echo component and noise other than the echo component.

  The signal processing unit 1 includes a camera image processing unit 2, a signal level adjustment unit 3, and an echo cancellation unit 10. The microphone 5 is a small omni-directional microphone, collects sounds uttered by the user (uttered speech S) and ambient sounds such as noise, converts them into electrical signals, and outputs them to the signal processing unit 1 To do. The speaker 4 converts the electrical reception audio signal input from the signal processing unit 1 into sound and outputs the sound.

  The signal processing unit 1 includes an input unit that receives a signal from the microphone 5 and an output unit that outputs a signal to the telephone interface unit 7 and an amplifier that amplifies the signal level of the received audio signal or the transmitted audio signal. Although it is provided, illustration is omitted. Similar amplifiers (not shown) are also provided in the input unit for inputting signals from the telephone interface unit 7 and the output unit for outputting to the speaker 4. The amplification factor of these amplifiers is fixed.

  Here, the process of generating an echo and the operation of the echo canceling unit 10 will be described with reference to FIG. First, the received audio signal received via the telephone interface unit 7 is converted into sound by the speaker 4 and emitted. In general, an echo component and a noise component are superimposed on the received voice signal in addition to the voice (received voice R) uttered by the other party, but for the sake of explanation here, the echo component and noise in the received voice signal are superimposed. The components are ignored and the received voice signal and the received voice R are the same. In addition, it demonstrates using the same symbol R, without distinguishing the electrical signal corresponding to a received voice, and a received voice. Similarly, the uttered voice S and the echo component r will be described with the same symbols.

  Part of the sound emitted from the speaker 4 (in this case, the received voice R is ignored because the echo component and the noise component are ignored) goes around the microphone 5 through the echo path and is collected as the echo component r. Since the user's speech S and the echo component r are input to the microphone 5, the transmission sound signal input from the microphone 5 to the signal processing unit 1 is S + r. This echo component r becomes annoying noise and is transmitted to the other party.

  In addition, the transmission voice signal S (strictly S + r) transmitted from the user to the other party is also echoed from the speaker 4 to the microphone 5 on the other party side. At this time, the received voice signal input from the telephone interface unit 7 to the signal processing unit 1 is R + s in which the echo component s is superimposed on the received voice R.

  The echo canceling unit 10 performs a process of removing these echo components r and s using the received voice signal (≈received voice R) and the transmitted voice signal (≈uttered voice S). More specifically, the echo canceling unit 10 includes a DSP (Digital Signal Processor) 11 that generates a pseudo echo signal, a reception-side adder 12, and a transmission-side adder 13. As shown in FIG. 1, the received voice R (strictly speaking, the voice obtained by removing the echo component r from the received voice signal by the echo cancellation unit 10) output to the speaker 4 is also input to the DSP 11. The DSP 11 generates a pseudo echo signal r ′ based on the received voice R. Here, the pseudo echo signal r ′ may be generated using an adaptive filter that sequentially learns the transfer function of the echo path as in the prior art. Then, by subtracting the pseudo echo signal r ′ generated by the DSP 11 from the audio signal S + r input from the microphone 5 by the adder 13, the transmission audio signal to be transmitted to the other party is S + r−r ′.

  Here, if the pseudo echo signal r ′ generated from the received voice R completely matches r, the echo is completely removed, and the user's uttered voice S is transmitted to the other party. However, even if a known echo cancellation technique is used, it is difficult to generate a complete pseudo echo signal (r = r ′), and generally r≈r ′. Even if r = r ′ can be realized temporarily, the echo path and its transfer function fluctuate when the user changes posture or the environment around the speaker 4 and the microphone 5 changes. While the function is being learned, r ≠ r ′ and a residual echo is generated. Furthermore, a more complicated circuit is required to realize the echo cancellation unit 10 with higher accuracy.

  The echo component s due to the uttered voice S can be similarly reduced by a known echo cancellation technique. That is, as shown in FIG. 1, the transmission voice signal (≈ utterance voice S) from which the echo component r has been removed by the echo cancellation unit 10 is input to the DSP 11, and the DSP 11 is a pseudo echo signal s predicted from the utterance voice S. Generate '. The adder 13 subtracts the pseudo echo signal s ′ generated by the DSP 11 from the received audio signal R + s input from the telephone interface unit 7, so that the signal output to the speaker 4 becomes R + s−s ′. . Again, since s = s ′ is not satisfied, a residual echo is generated.

  The camera 6 sequentially captures the user's face and sequentially outputs the captured image data to the camera image processing unit 2. The number (frame rate) of images of the user's face per second is set to a value that is sufficient to detect the movement of the mouth when the user speaks and can be processed by the camera image processing unit 2 described later. To do. That is, when the frame rate is small, the interval at which image data is captured becomes long, and there is a possibility that the timing for detecting mouth movements may be delayed with respect to the user's speech. Alternatively, the detection accuracy may be reduced due to a decrease in continuity of image data between frames. On the other hand, if the frame rate is too large, the processing load on the image processing unit 2 described later increases. As an example, in this embodiment, it is 30 frames per second.

  The camera image processing unit 2 detects the movement of the user's mouth from the image data input from the camera 6. As a method for detecting the movement of the mouth, as an example, as in the technique described in Patent Document 2, the motion vector or luminance value between feature points detected in a predetermined range (mouth area) centered on the mouth A difference or the like may be used. As a method of specifying the mouth region, for example, the position of the user's pupil is detected, the mouth range is statistically predicted from the position of the pupil, and the pattern matching method is used to specify the range.

  As a mouth motion detection method based on motion vectors, the identified mouth area is divided into a plurality of blocks, a motion vector is obtained for each block in the area, and a cumulative value of the magnitudes of the motion vectors is obtained. Good. Then, it is determined whether or not the cumulative value of the magnitudes of the motion vectors is larger than a predetermined threshold. If the cumulative value is larger than the predetermined threshold, it is assumed that the movement of the mouth is detected.

  Even when using an inter-frame difference in luminance value instead of a motion vector, the inter-frame difference in luminance value is obtained for each block within the specified mouth area, and it is determined whether it is greater than a predetermined threshold. If it is larger, the movement of the mouth is detected if it is larger than the predetermined threshold.

  As described above, when the movement of the user's mouth is detected, the camera image processing unit 2 outputs to the signal level adjustment unit 3 an ON signal that instructs to set the amplification factor from 0 to 1. On the other hand, when it is determined that there is no movement of the user's mouth continuously in a predetermined number of frames (for example, 3 frames), an OFF signal that instructs to set the amplification factor from 1 to 0 is signal level adjusted. Output to part 3.

  The signal level adjustment unit 3 is configured to be able to change the ratio (hereinafter referred to as amplification factor) of the output value with respect to the input value of the signal level (for example, voltage value) of the transmission audio signal. Based on this amplification factor, the transmission audio signal is adjusted. The signal level of the signal is adjusted and output to the telephone interface unit 7. The state where the amplification factor is 0 is not limited to strict 0, but also includes a state where the signal level is reduced to a level that cannot be heard by the other party. Further, the amplification factor and the signal level may be completely set to 0 by opening a circuit that transmits the transmission audio signal.

  Further, the amplification factor of the signal level adjustment unit 3 is sequentially changed based on the signal input from the camera image processing unit 2. In this embodiment, when an ON signal is input from the camera image processing unit 2, the amplification factor is set to 1, and the signal level of the transmission audio signal input to the signal level adjustment unit 3 is output without being changed. To do. On the other hand, when an OFF signal is input from the camera image processing unit 2, the gain is set to 0 and the signal level of the transmission audio signal is set to 0. That is, the transmission audio signal input to the signal level adjustment unit 3 while the OFF signal is input from the camera image processing unit 2 is discarded. For this reason, only when the movement of the mouth is detected by the camera image processing unit 2, the transmission voice signal is transmitted to the call partner via the telephone interface unit 7 or the like.

  Here, the operation in this embodiment will be described with reference to the flowchart of FIG. FIG. 2A shows processing when a transmission voice signal is transmitted from the user to the call partner, and FIG. 2B shows processing when a reception voice signal is received from the partner side.

  The flowchart of FIG. 2A is sequentially executed by the signal processing unit 1, and starts from step S12.

  In step S <b> 12, echo cancellation processing is performed by the echo canceling unit 10 on the transmission voice signal (uttered speech S + echo component r) input from the microphone 5 to the signal processing unit 1. Here, the echo cancellation processing to be performed uses a known technique as described above. The transmission audio signal (S + r−r ′) from which the echo component has been removed by the echo cancellation processing is output to the signal level adjustment unit 3, and the process proceeds to step S14.

  In step S14, the signal level adjusting unit 3 adjusts the signal level of the transmission audio signal to 0 or 1 based on an instruction from the camera image processing unit 2, and the process proceeds to step S16. This amplification factor is sequentially adjusted according to a flowchart of amplification factor adjustment processing shown in FIG. The transmission audio signal whose signal level is adjusted by the signal level adjustment unit 3 is output to the telephone interface unit 7 and also to the DSP 11. The transmission voice signal input to the telephone interface unit 7 is transmitted from the telephone to the other party via the network in step S18.

  Further, the flowchart of FIG. 2B is sequentially executed by the signal processing unit 1, and starts from step S22. In step S22, the echo cancellation unit 10 performs a known echo cancellation process on the received voice signal (received voice R + echo component s) using the transmitted voice signal, and the process proceeds to step S24.

  In step S24, the audio signal (R + s−s ′) from which the echo has been removed by the echo cancellation processing in the previous step S22 is output to the DSP 11 and also output to the speaker 4. In step S26, the speaker 4 converts the received voice signal input from the signal processing unit 1 into sound and outputs the sound.

  The flowchart shown in FIG. 3 shows the flow of the amplification factor adjustment process, and this process is sequentially executed as the call starts. In this amplification factor adjustment process, first, in step S160, as described above, the presence / absence of the movement of the user's mouth is detected from the image data captured by the camera 6. If mouth movement is detected, step S162 is set to YES and the process proceeds to step S164. On the other hand, when the movement of the mouth is not detected, step S162 is NO and the process proceeds to step S166. In step S164, an ON signal is output from the camera image processing unit 2 to the signal level adjustment unit 3 so as to set the amplification factor to 1. In step S166, the camera image processing unit 2 outputs an OFF signal to the signal level adjustment unit 3 so as to set the amplification factor to zero.

  Hereinafter, in each situation classified according to the presence / absence of the movement of the user's mouth and the utterance state and the utterance state of the other party, the respective operations of the above-described embodiment, the comparison technique A, and the comparison technique B are described. Will be described with reference to FIG.

  Here, the comparison technique A is to adjust the amplification factor of the reception voice signal and the transmission voice signal according to the signal level of the transmission voice signal from the user and the signal level of the reception voice signal from the other party. Note that Patent Document 1 does not exemplify control for determining the respective amplification factors in consideration of the signal levels of both the reception audio signal and the transmission audio signal. This comparison technique A is a method for convenient determination of processing when both a transmission audio signal and a reception audio signal are input in Patent Document 1, and compares the signal levels of the transmission audio signal and the reception audio signal. The voice with the higher signal level is determined as the voice signal to be transmitted. That is, in the comparison technique A, the signal having the smaller signal level is reduced, and control is performed so that the sum of the gains of the reception audio signal and the transmission audio signal becomes 1. Further, when the signal levels of the transmission audio signal and the reception audio signal are the same, the respective gains are adjusted to be 0.5.

  In comparison technique A, the effect of adjusting the amplification factor according to the signal levels of the transmission audio signal and the reception audio signal and the effect of adjusting the amplification factor by the movement of the mouth as in the present embodiment. The purpose is to verify the difference.

  In comparison technique B, only known echo cancellation technique as described in Patent Document 2 is applied to the signal processing unit 1b as an example. The purpose of the comparison technique B is to compare a situation where no echo is generated in the known echo cancellation technique and a situation where no echo is generated in the present embodiment.

  In any of the present embodiment, the comparison technique A, and the comparison technique B, the signal processing unit (1, 1a, 1b) is input to an input / output unit that is connected to an external device such as the speaker 4 or the microphone 5. An amplifier for amplifying the signal level of the signal (or output signal) by a certain amount is provided. However, particularly in the present embodiment and the comparison technique A, the element of interest is not an amplifier whose amplification factor is fixed, but an amplifier whose amplification factor is adjusted according to the situation (that is, a signal level adjustment unit). It is.

  Next, each parameter in Table 1 will be described. First, the presence / absence of the “mouth movement” column indicates whether the camera image processing unit 2 has detected the movement of the user's mouth. Further, in the “utterance” column, “x” indicates a state in which no speech is performed, “Δ” indicates a state in which a speech is made but the noise level is low, and “◯” indicates a state in which speech is performed at a normal voice level (ie, volume). In the “microphone” column, the signal level of the transmission audio signal is set to 1. In the “SP” (SP = speaker) column, the signal level of the reception audio signal is set to 1 for passing through the signal as it is, and 0. 5. The state that does not pass is set to 0. That is, it shows how the amplification factor for each signal is set in the assumed situation.

  ○ in the “call” column indicates that at least one of the transmission audio signal and the reception audio signal is reduced from an audible signal level to an inaudible signal level (= 0) by adjusting the amplification factor. X, and other than that. In the “Echo” column, if the pseudo echo signal does not completely remove the echo component and the pseudo echo signal can deviate from the echo, there is a situation where the echo is transmitted to the other party or the user. And

In comparison technique A, the effect of adjusting the amplification factor according to each signal level of the transmission audio signal and the reception audio signal, and the effect of adjusting the amplification factor by the movement of the mouth as in this embodiment. Since this is a comparison target for verifying the difference, the echo column is omitted in the comparison technique A. Further, since the comparison technique B is a comparison between a situation in which no echo is generated by a known echo cancellation technique and a situation in which no echo is generated by the present embodiment, other than the echo column is omitted in the comparison technique B.

  Thereafter, when this embodiment is first compared with the comparative technique A, there is a difference in whether or not a call can be made by adjusting the amplification factor. 6 and 8 will be described. Next, when this embodiment is compared with the comparative technique B, the difference in the effects of echoes occurs. 11 and 12 will be described.

  First, no. The situation 6 is a situation in which the movement of the user's mouth is detected by the camera image processing unit 2, the user speaks with a low noise level, and the other party speaks with a normal voice level. . The operation of this embodiment and the comparison technique A in such a situation will be described with reference to FIG. In addition, No. In 6 and 8, in order to examine whether or not a call can be made by adjusting the amplification factor, the comparison technique B corresponding to the conventional echo cancellation technique is not mentioned.

  In the comparison technique A, since the signal level of the reception audio signal R is larger than the signal level of the transmission audio signal S and the difference is large, it is determined that the reception audio signal R should be transmitted. In this case, as shown in FIG. 4, the amplification factor of the signal level adjustment unit 3R on the reception side is 1, and the amplification factor of the signal level adjustment unit 3S on the transmission side is 0. For this reason, the user's utterance voice included in the transmission voice signal is not transmitted to the other party. Strictly speaking, in this comparison technique A, there is a case where the amplification factor does not become zero, but it is set to 0 for convenience because the listener cannot hear by further attenuating the low-noise signal at the noise level.

  On the other hand, in the present embodiment, since the movement of the user's mouth is detected by the camera image processing unit 2, the amplification factor on the utterance side is 1. Since the amplification factor is 1, the transmission voice signal is not reduced, and the user's voice can be transmitted to the other party.

  Next, no. 8 is a situation where the user's mouth movement is detected by the camera image processing unit 2 and the user is speaking at a normal voice level and the other party is speaking at a noise level. is there. The operation of this embodiment and the comparison technique A in such a situation will be described with reference to FIG.

  In the comparison technique A, since the signal level of the transmission audio signal S is smaller than the signal level of the reception audio signal R and the difference is large, it is determined that the transmission audio signal S should be transmitted. In this case, as shown in FIG. 5, the amplification factor of the signal level adjustment unit 3S on the transmission side is 1, and the amplification factor of the signal level adjustment unit 3R on the reception side is 0. For this reason, the signal level of the received voice signal becomes 0, and the received voice from the other party in the received voice signal is not transmitted to the user. On the other hand, in this embodiment, since the amplification factor on the receiving side is not reduced, the received voice from the other party can be transmitted to the user regardless of the voice level.

  As described above, by adjusting the amplification factor on the transmission side according to the presence / absence of movement of the mouth of the user instead of the signal levels of the transmission audio signal and the reception audio signal, the reception audio signal and the audio output signal are Even if they are input at the same time and there is a difference in their signal levels, mutual voices can be transmitted to the other party. That is, the present embodiment is configured to adjust the signal level of the transmission audio signal and does not reduce the signal level of the reception audio signal. Therefore, when receiving audio from the communication partner, the signal level of the reception audio signal is low and the user Even if there is an utterance, it is possible to listen to the received voice from the other party. Even if a received voice signal is input, if the movement of the user's mouth is detected by the camera image processing unit, the amplification factor of the signal level adjusting unit is set to 1 or more, so that the uttered voice is a low voice. Even so, you can tell the other party.

  However, if only the signal level adjustment unit 3 and the camera image processing unit 2 have an amplification factor of 1, there is a possibility that an echo may be heard by the other party. Therefore, as in the present embodiment, in addition to the signal level adjustment unit 3 that adjusts the amplification factor, a configuration that suppresses the influence of echoes using an existing echo cancellation technique is more preferable. Further, a configuration for reducing the influence of noise using a known noise cancellation technique may be combined.

  No. described next. 11 and 12, in a situation where the movement of the user's mouth is not detected, the user is not speaking and the other party is speaking (No. 11 is a low-noise utterance, No. 12 is Normal speech level). The operation of the present embodiment and the comparison technique B in such a situation will be described with reference to FIG. Here, the purpose of comparing the present embodiment and the comparison technique B with respect to the influence of echoes is not described, and therefore the comparison technique A is not mentioned.

  In the comparative technique B, if the pseudo echo signal generated by the echo cancel unit 10a is slightly deviated from the actual echo component due to the change of the echo path and its transfer function, the echo is transmitted to the other party. Is concerned.

  On the other hand, in this embodiment, since the movement of the user's mouth is not detected, the amplification factor is zero. Since the amplification factor is 0, the signal level of the transmission voice signal is 0, and the echo is not heard by the other party regardless of the accuracy (or presence / absence) of the pseudo echo signal.

  Further, in the case of the comparison technique B, when noise (noise other than echo) generated around the microphone 5 is input to the microphone 5, the noise is not removed by the echo canceling unit 10 and transmitted to the other party. End up. On the other hand, it is also possible to reduce noise using a known noise cancellation technique, but as with the echo cancellation technique, if the pseudo noise signal for canceling the noise has shifted from the actual noise, The other party can hear you. However, when the movement of the user's mouth is not detected as in this embodiment, the amplification factor is set to 0, so that noise can not be transmitted to the other party during that time.

  According to the above configuration, since the amplification factor on the receiving side is not adjusted in this embodiment, both voices can be transmitted simultaneously even when the received voice from the other party is small and the user's spoken voice is large. Even when the voice of the other party is loud and the voice of the user is low, when the movement of the user's mouth is detected by the camera image processing unit 2, the amplification factor on the transmission side is set to 1, so both voices are It becomes possible to convey.

  Furthermore, when it is determined that there is no movement of the user's mouth, the signal level of the transmission audio signal becomes 0 because the amplification factor becomes 0, so that even if echo or noise is input to the microphone 5 The possibility of being transmitted to the other party can be reduced.

(Modification)
Furthermore, the present invention may be configured (modified example) as shown in FIG. FIG. 7 is a diagram corresponding to FIG. 1 and is a block diagram showing a configuration of a hands-free call device according to a modification. Note that a description overlapping that in FIG. 1 is omitted.

  The buffer unit 8 accumulates the transmission audio signal from which the echo component r has been removed or reduced by the echo cancel unit 10 for a certain period of time (hereinafter referred to as a delay time), and then outputs it to the signal level adjustment unit 3. Here, the delay time is set to such an extent that it does not give a sense of incongruity to the user and the other party (for example, several tens of ms). This delay time takes into account the time required from the moment when the user speaks to the time when the movement of the mouth accompanying the speech is detected by the camera image processing unit 2 and the amplification factor of the signal level adjusting unit 3 is set to 1. It may be what you did.

  In order to detect the movement of the mouth, it is necessary to analyze the image captured by the camera 6 with the camera image processing unit 2, so that the detection process of the mouth movement is not in time at the beginning of the user's talk. The amplification factor was 0, and there was a possibility that the beginning of the talk would be lost.

  However, by adopting a configuration including the buffer unit 8 as in this modification, the time taken for the speech level to reach the signal level adjustment unit 3 after the uttered voice is acquired by the microphone 5 is accumulated in the buffer unit 8. Delay by time. And the part missing from the uttered voice can be reduced by the amount delayed by the buffer unit 8, and the possibility of giving a strange feeling to the other party can be reduced. Also, if the camera image processing unit 2 can detect the movement of the mouth and set the amplification factor to 1 between the start of the talk and the output from the buffer unit 8, the talk start part is also missing. You can tell the other party without having to.

  In the present embodiment, the control for setting the amplification factor from 1 to 0 is performed when the camera image processing unit 2 determines that there is no movement of the mouth for a predetermined number of frames. Absent. The amplification factor may be changed from 1 to 0 after a predetermined time (for example, about several seconds) from the time when it is determined that there is no movement of the mouth for a predetermined number of frames.

  By doing so, for example, when the user thinks about the content of the speech and speaks in an intermittent manner, the amplification rate changes between 0 and 1 in a short time, and accordingly, the beginning of the speech is The possibility of being missing can be reduced.

  In the above configuration, the binary values of the amplification factors 0 and 1 of the signal level adjusting unit 3 are used, but the present invention is not limited to this. A configuration may be adopted in which the amplification factor when the movement of the mouth of the user is not detected is set to a value smaller than 1, and the amplification factor when the movement of the mouth is detected is set to a value of 1 or more. When the movement of the mouth of the user is not detected, the amplification factor is set to a value smaller than 1. When the movement of the mouth is not detected, the signal level of the transmission voice signal is attenuated and transmitted to the other party. . In this case, there is a possibility that the noise and echo input to the microphone 5 are transmitted to the other party, but the influence of the noise and echo can be reduced by the amount that the signal level is attenuated by the signal level adjusting unit 3. it can.

  Further, the amplification factor of the signal level adjusting unit 3 may be changed from 0 to 1 (or from 1 to 0) continuously or stepwise instead of stepwise. Furthermore, it is good also as a structure which takes the value between 0 and 1, such as 0.4, 0.5, 0.6 between 0 and 1.

  Further, when the face image data of the other party can be acquired, such as a videophone, a processing unit that performs the same processing as the camera image processing unit of the present invention is applied to the other party's face image data, and the processing is performed. It is good also as a structure which adjusts the signal level of a received audio | voice signal based on the detection result of a part. With such a configuration, when the movement of the other party's mouth is not detected, the signal level on the receiving side is set to 0, so that the echo or noise from the other party is transmitted to the user. Can be suppressed.

  When the camera image processing unit 2 detects that the user is about to start speaking from the movement of the mouth and the feature points around the mouth and the movement pattern of the mouth immediately before speaking, the signal level adjustment unit 3 Alternatively, an ON signal may be output.

  It should be noted that the present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims, and the embodiments obtained by appropriately combining the respective technical means disclosed are also included in the present invention. It is included in the technical scope of the invention.

  DESCRIPTION OF SYMBOLS 100 ... Hands-free call apparatus, 1 ... Signal processing part, 2 ... Camera image processing part, 3 ... Signal level adjustment part, 4 ... Speaker, 5 ... Microphone, 6 ... Camera, 8 ... Buffer part, 10 ... Echo cancellation part, R: Received voice (signal corresponding to received voice), S: Spoken voice (signal corresponding to spoken voice)

Claims (6)

A camera (1) that captures the user's face;
A camera image processing unit (2) for detecting movement of the user's mouth from image data captured by the camera;
A hands-free call device (100) comprising: a signal level adjustment unit (3) for adjusting a signal level of a transmission voice signal transmitted from a user to a call partner by adjusting an amplification factor by a continuous or stepwise change. Because
The signal level adjustment unit includes:
The amplification factor is configured to be set to a value located between 0 and 1, and when the movement of the mouth is not detected by the camera image processing unit, the amplification factor is set to a value smaller than 1.
A hands-free call device characterized in that when a movement of the mouth is detected by the camera image processing unit, an amplification factor of a transmission voice signal is set to a value of 1 or more. In claim 1,
The signal level adjustment unit sets an amplification factor to 0 when mouth movement is not detected by the camera image processing unit, and transmits audio signal when mouth movement is detected by the camera image processing unit. A hands-free call device characterized in that the amplification factor is set to 1. In claim 1 or 2,
The hands-free call device is a hands-free call device in which a signal corresponding to the sound collected by the microphone (5) is input as a transmission voice signal,
A hands-free call device comprising a buffer unit (8) for storing a transmission voice signal for a certain period of time between the microphone and the signal level adjustment unit and outputting the signal to the signal level adjustment unit. In any one of Claims 1-3,
The hands-free communication device sets the amplification factor of the signal level adjustment unit to 0 when mouth movement is not detected by the camera image processing unit for a certain period of time. In any one of Claims 1-4,
The hands-free call device generates a pseudo echo signal estimated from the received voice signal based on the received voice signal and an echo component of the received voice signal mixed in the transmitted voice signal, and the pseudo echo signal A hands-free communication device comprising: an echo cancel unit (10) for canceling an echo component by subtracting the signal from the transmission voice signal. In any one of Claim 1 to 5,
The hands-free call device generates a pseudo echo signal estimated from the transmission voice signal based on the transmission voice signal and an echo component of the transmission voice signal mixed in the reception voice signal, and the pseudo echo signal A hands-free communication device comprising an echo canceling unit (10) for canceling an echo component by subtracting the signal from the received voice signal .

Images