JP2006215206A - Speech processor and control method therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for properly controlling a masking signal output so that inputted speech of a user and outputted speech from equipment cannot be heard by a third party and do not give displeasure to the third party, relating to the equipment with a speech input and/or speech output function. <P>SOLUTION: Measurement is carried out at least about either a position of the third party or noise in there, and according to a result of the measurement, a masking signal for masking the speech of the user inputted to an input means is determined to the third party. After that, according to an operation state of the input means (S501, S502), the output of the determined masking signal is controlled (S503, S504). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an audio processing apparatus and a control method thereof in consideration of user privacy protection.

  Up to now, voice recognition technology has been developed mainly for improving the performance of voice recognition, and has already been put into practical use in car navigation systems, voice response systems (Interactive Voice Response: IVR) and the like. However, on the other hand, voice recognition technology is hardly permeated for office equipment such as personal computers, copiers and facsimile machines, and home appliances such as televisions and telephones. The reason for this is that input / setting using other input means such as a keyboard, remote control, buttons, etc., is possible without using voice, but these devices are becoming increasingly multifunctional and complex year by year. In particular, it is considered that input and setting for devices other than personal computers that do not have a keyboard generally become more complicated. That is, in the future, a user interface that is more intuitive and easy to understand will be required. Voice is expected as one modality in a multimodal user interface combined with an existing user interface or one of user interfaces satisfying such requirements. As described above, the user interface using sound is intuitive and is considered to be a useful interface that can be input and set more quickly than when using the existing input means if correctly recognized.

  However, as a factor that hinders the spread of user interfaces using such voices, even if voice recognition performance close to 100% is obtained, when a third party exists in the vicinity, the voice is spoken toward the device. You may be embarrassed to hear the voice you are listening to or do not want to hear the content. Also, particularly in office situations, for a third party in the vicinity of the user of the device, the voice uttered toward the device is verbal, so the user's utterance is worrisome. Therefore, it may be impossible to concentrate on business.

  The situation as described above can occur not only in the voice uttered by the user but also in the voice output from the device. In other words, it is convenient to be able to confirm the settings entered with the remote control and buttons by voice output, but the third party does not want to hear the contents, and for the third party, Similarly, there is a problem that linguistic information is a concern.

  As described above, in a device using a user interface using sound, it may be necessary to provide a mechanism in which a user's input sound and a device's output sound are not heard by surrounding third parties. For the purpose of not leaking such voice to the outside, it is possible in principle to cancel the waveform by generating a voice waveform in reverse phase with respect to voice input or voice output, It is extremely difficult to cancel a sound with respect to an arbitrary direction and distance of a plurality of points, and this method cannot be used in practice. Therefore, in practice, it is necessary to deal with any of the following methods: (1) devise equipment, (2) impose a burden on the user, and (3) impose a burden on a third party.

  For example, the equipment of (1) may be a sound barrier around the equipment, but it is not possible to install such equipment in offices and homes because of space and cost. Is. On the other hand, as a method of forcing the burden on the user in (2), a method of using an earphone or a bone conduction speaker when confirming the output of a device in which voice input is performed by whispering can be considered. Forcing this burden detracts from the advantages of the voice interface. Also, (3) as a method of imposing a burden on a third party, by outputting another masking signal from the speaker at the time of voice input or voice output, the voice of the user or the voice output of the device is heard from the third party. There is a way to prevent it. This method is much cheaper than the method (1) in terms of cost, and if it is masked appropriately, the user can speak normally. There is a feature that the burden on the person does not occur. However, even if this method cannot control the output of the masking signal properly, the voice of the user or the sound output of the device is heard by a third party (the masking signal level is low, the duration of the masking signal The problem is that the third person is uncomfortable (or the masking signal level is too high, or the duration of the masking signal is too long).

  As a method for generating a masking signal for a user's utterance, Japanese Patent Laid-Open No. 9-305196 (Patent Document 1) discloses a musical sound signal mainly composed of a formant frequency component contained in a vowel of speech as a voice signal of the speaker. A method of radiating with an average power greater than the average level is disclosed.

JP-A-9-305196

  However, the technique disclosed in Patent Document 1 has the following problems.

  First, in Patent Document 1, a musical sound signal radiated as a masking signal is controlled so that its average power is greater than the average level of the voice of the speaker. However, the average level of sound observed from the microphone varies depending on the gain of the microphone, and the average power of the masking signal radiated from the speaker also varies depending on the gain of the speaker. That is, the average level of the sound captured from the microphone and the average power of the masking signal radiated from the speaker do not match the level of the signal level heard by a third party. In other words, there is no guarantee that appropriate masking is performed by this control method.

  In Patent Document 1, it is assumed that a position where a third party exists is behind the user. Therefore, only one speaker that outputs a masking signal is prepared. However, since it is difficult in the first place to presume the location of a third party in an actual office or home, a single speaker is not always sufficient.

  Moreover, in patent document 1, the surrounding noise level is not taken into consideration, and a masking signal is uniformly output regardless of a noisy environment or a quiet environment. Then, if there are many similar devices in the vicinity or the surrounding noise level is high, it will be very noisy, so even if the user's input voice is masked, There is a problem that it causes a lot of discomfort.

  Further, Patent Document 1 only describes that the timing of outputting the masking signal is that the user outputs a musical sound when standing at a predetermined position in front of the voice input device. There is no description of the timing of termination. In addition, according to this method, there is a problem in that an unnecessary masking signal is output because a musical sound is continuously output regardless of whether or not sound is input. Furthermore, there is no suggestion of changing the level of the masking signal in accordance with the content of the voice input.

  In addition, Patent Document 1 does not mention masking for audio output at all.

  The present invention has been made to solve at least one of the problems described above, and in a device having a voice input and / or voice output function, the user's input voice and the output voice of the equipment are heard from a nearby third party. It is an object of the present invention to provide a technique for appropriately controlling the output of a masking signal so as not to cause discomfort to a third party.

  In order to solve the above-described problem, for example, a speech processing apparatus of the present invention has the following configuration. That is, a speech processing apparatus for processing speech information received from an input means for inputting speech uttered by a user, wherein a measurement means for measuring a surrounding environment, and a third result based on a measurement result by the measurement means Determining means for determining a masking signal for masking a user's voice input to the input means for a person, and a masking signal determined by the masking signal determining means based on an operating state of the input means The control means which controls the output of this is provided.

  According to the present invention, the output of the masking signal is appropriately controlled so that the input voice of the user and the output voice of the device are not heard by a nearby third party and the third party is not uncomfortable. .

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a hardware configuration of a speech processing apparatus to which the present invention is applied.

  A CPU 101 controls the entire apparatus, a ROM 102 stores various parameters and a boot program, and a RAM 103 provides a work area to the CPU 101 and functions as a main storage device. Reference numeral 104 denotes an external storage device such as a hard disk, CD-ROM, DVD-ROM, or memory card. Here, a recording / playback program 111 for recording / playback, a voice recognition program 112 for performing voice recognition processing, and a voice A speech synthesis program 112 for performing synthesis, a masking signal 114, and a control program 110 for comprehensively processing these programs and data can be held. When the external storage device 104 is a hard disk, various programs installed from a CD-ROM or the like are stored. These programs stored in the external storage device 104 are loaded into the RAM 103 and executed by the CPU 101. Of course, these programs and data are not stored in the external storage device 104 but may be stored in the ROM 102 in advance.

  Reference numeral 105 denotes a microphone that captures a voice uttered by a user. At the time of capturing the sound, the sound signal collected by the microphone 105 is amplified by the microphone amplifier 105a. Reference numeral 106 denotes a sound output speaker, which outputs recorded sound, synthesized sound, and the like to the user via the first speaker amplifier 106a. Reference numeral 107 denotes a masking signal output speaker, which outputs a masking signal to a third party existing around the voice input / output device through the second speaker amplifier 107a.

  Reference numeral 108 denotes an auxiliary input / output device such as a display, a button, a numeric keypad, a touch panel, a mouse, a keyboard, a microphone, a video camera, an infrared sensor, and the like. It performs imaging with a third-party video camera in the vicinity, sensing with an infrared sensor, and displaying a menu on the display.

  Reference numeral 109 denotes a bus for connecting the above-described units.

  Hereinafter, as the first embodiment, a case will be described in which the above-described voice processing device is caused to function as a voice input device. Further, as the second embodiment, a case where the above-described sound processing device is caused to function as a sound output device will be described, and a modification thereof will be described in the third embodiment. Furthermore, as a fourth embodiment, a case will be described in which the voice processing device is caused to function as a voice interaction device by cooperating a voice input function and a voice output function. In the following description, a microphone is used as the user's voice input device and a speaker is used as the voice output device. However, the present invention is not limited thereto, and for example, a handset having a voice input / output function may be used. Good.

(First embodiment)
In the first embodiment, a case where a voice processing device functions as a voice input device will be described. The voice input function realized here is used for recording or voice recognition, for example.

  FIG. 2 is a block diagram illustrating a functional configuration of the voice input device according to the present embodiment.

  Reference numeral 205 denotes a voice input unit that inputs a user's voice. Reference numeral 203 denotes a voice input status management unit that manages the operating state of the voice input unit 205, and monitors events that are issued by triggering operations related to voice input. Reference numeral 201 denotes a surrounding environment measurement unit that measures the noise level around the voice input device 105, the presence / absence of a third party existing around the voice input / output device, and the position of the third party. Reference numeral 202 denotes a masking signal determination unit that determines a masking signal for masking the input voice to a third party based on information from the voice input state management unit 203 and the surrounding environment measurement unit 201. Reference numeral 204 denotes a masking signal control unit that controls the output of the masking signal determined by the masking signal determination unit 202.

  Next, an example of voice input processing in the present embodiment will be described using the flowcharts of FIGS. In the present embodiment, the processing of the voice input unit 205 and the masking signal control unit 204 will be described as event-driven processing. The target events are a “voice input start” event indicating that voice capturing has started, and a “voice input end” event indicating that voice capturing has ended.

  FIG. 3 is a flowchart showing a processing flow of the surrounding environment measurement unit 201 and the masking signal determination unit 202.

  First, in step S <b> 301, noise around the apparatus is captured using the microphone 105. In this case, instead of the microphone 105, a microphone prepared for measuring the surrounding environment may be used as the auxiliary input device 108 to capture noise around the device. As the timing of capturing, the system may automatically capture at a predetermined interval, or may capture according to some event related to the user's operation. Or you may make it take in noise according to the operation instruction prepared only for taking in noise.

Next, in step S302, the level of the masking signal is determined. Various methods can be considered for determining this level. As one of the most preferable methods, masking is performed using the average logarithmic power P _E of the ambient noise signal x _E (t) captured in step S301. The amplitude of the signal x _M (t) can be changed by the following equation.

x ′ _M (t) = f (P _E ) · x _M (t) (1)

Here, f (•) is a function for controlling the increase / decrease in the amplification of the masking signal x _M (t) with respect to the average logarithmic power P _E of the surrounding noise signal x _E (t), and any function can be used. Is possible. For example, what is shown in FIG. 6 may be used. The function f shown in the figure is a control that reduces the masking signal when the average log power of the surrounding noise signal is large, and conversely increases the masking signal when the surrounding noise signal is small. is there. This function includes the average logarithmic power P _M of the masking signal x _M (t), the mounting position and direction of the microphone 105, the microphone gain of the microphone amplifier 105a, the mounting position and direction of the masking signal output speaker 107, and the second The amplifier gain of the speaker amplifier 107a, the directional characteristics of the speaker (for example, whether it is a directional speaker or an omnidirectional speaker), the microphone gain and mounting position and direction of the microphone for measuring the surrounding environment, and a position where a third party is assumed to exist It is necessary to design in consideration of the direction.

  In addition, when there are a plurality of masking signals, it is possible to change the masking signal to be output according to the situation of the surrounding environment. Specifically, it is possible to select the masking signal closest to the spectrum obtained by subtracting the surrounding noise spectrum from the average speech spectrum. Any masking signal may be used, but when this voice input device is used for voice recognition, the voice recognition performance is not easily deteriorated, and it is comfortable for third parties and users in the vicinity. Things are desirable.

  FIG. 4 is a flowchart showing a processing flow of the voice input unit 205 and the voice input status management unit 203.

  First, in step S401, event standby is performed. Next, when an event is detected, the process proceeds to step S402, and the type of the detected event is determined. If a “voice input start” event indicating that voice capture has started is detected, the process proceeds to step S403. If a “voice input end” event indicating that voice capture has been completed is detected, step S403 is performed. The process proceeds to S404.

  In step S403, capturing of the user's voice from the microphone 105 is started, and the process returns to step S401. Note that it is assumed that the audio is still captured after the process returns to step S401. On the other hand, in step S404, the capturing of the user's voice from the microphone 105 is terminated, and the process is terminated.

  Here, each event of “speech input start” and “speech input end” may be given by the user, such as a user pressing a button, taking a handset, or placing the handset. It may occur as an event of state transition in the voice interval detection method.

  FIG. 5 is a flowchart showing a processing flow of the masking signal control unit 204.

  Here, in the process shown in FIG. 4, a masking signal for preventing a third party from hearing the voice uttered by the user is output from the masking signal output speaker 107. First, in step S501, event waiting is performed. Next, when an event is detected, the process proceeds to step S502, and the type of the detected event is determined. If a “voice input start” event indicating that voice capture has started is detected, the process proceeds to step S503. If a “voice input end” event indicating that voice capture has been completed is detected, step S503 is performed. The process proceeds to S504.

  In step S503, output of the masking signal determined in step S302 is started, and the process returns to step S501. Note that it is assumed that the masking signal continues to be output after the processing returns to step S501. In step S504, the masking signal output ends and the process ends.

  In the above processing example, the output level of the masking signal is determined in step S301 and step S302 before the user's voice is uttered. However, since the utterance level of the user's voice is not known in advance, It is necessary to determine the level of the masking signal by predicting the voice level of the person. However, when the utterance level of the actual user is lower than the prediction, the masking signal is output at an unnecessarily large level. On the other hand, if the actual user's utterance level is higher than predicted, a sufficient masking signal is not output, and there is a possibility that the content of the user's utterance will be heard by a third party. . In order to alleviate this problem, the level of the masking signal may be determined dynamically according to the input level of the user's utterance.

  FIG. 7 is a flowchart showing a processing flow for dynamically determining the masking signal level using the voice input level.

Steps S701 to S704 are the same as steps S401 to S404, respectively, and thus description thereof is omitted. In step S705, after the voice capture is started, the signal level of the captured voice is measured in a predetermined time unit, and the level of the masking signal is adaptively compared with the predicted speech level of the user. change. Specifically, the logarithmic power of the predicted utterance level of the user is P _S , and the logarithmic power of the utterance level of the user at the time t (logarithmic power of a predetermined short time interval) measured in step S705 is P ′. _Assuming that _S , x ′ _M (t) obtained by Expression (1) may be changed by the following expression.

x ″ _M (t) = g (P ′ _S / P _S ) · x ′ _M (t) (2)

Here, g (•) is a function for obtaining the degree of amplification of the masking signal x ′ _M (t) with respect to the ratio of the logarithmic power. For example, a function as shown in FIG. 20 may be used. This function is to increase the masking signal when P ′ _S / P _S > 1, that is, when the logarithmic power measured in step S705 is larger than the expected speech level, and to decrease the masking signal when it is small. .

  FIG. 8 is a diagram illustrating an example of an external configuration of the voice input device according to the present embodiment.

  Reference numeral 801 denotes a main body of the voice input device that houses the hardware shown in FIG. Reference numeral 803 denotes a masking signal output speaker for outputting a masking signal to a third party, which corresponds to the masking signal output speaker 107 of FIG. As in this example, when there is only one masking signal speaker or when the direction in which a third party exists is not known in advance, it is desirable to use a non-directional speaker. Reference numeral 802 denotes a microphone for measuring the user's voice input and surrounding noise environment, and corresponds to the microphone 105 in FIG. The microphone 802 is preferably installed at a position closer to the user than the masking signal output speaker 803 so that sound can be appropriately input by a signal output from the masking signal speaker 803.

  FIG. 9 shows a modification of FIG. 8, which has a masking signal output function for a predetermined direction.

  Reference numeral 901 denotes a main body of the voice input device that houses the hardware shown in FIG. Reference numeral 902 denotes a microphone for inputting a user's voice and corresponds to the microphone 105 in FIG. Reference numerals 906, 907, and 908 denote masking signal output speakers for outputting a masking signal to a third party, and correspond to the masking signal output speaker 107 of FIG. Reference numerals 903, 904, and 905 denote microphones for measuring the noise level around the apparatus at three points. As described above, since a plurality of microphones for measuring the surrounding environment and speakers for outputting a masking signal are provided, masking is performed according to individual noise levels, directions, and positions measured by the microphones 903, 904, and 905. The level of each masking signal output from the signal output speakers 906, 907, and 908 can be changed. Specifically, for example, it is possible to perform control such that a large masking signal is output in a direction where the noise level is small and a small masking signal is output in a direction where the noise level is large.

  Also, as in this example, when a plurality of microphones for measuring the surrounding environment and a plurality of masking signal speakers are provided, it is desirable to use directional microphones and speakers. The microphone 902 is positioned closer to the user than the masking signal output speakers 906, 907, and 908 so that voice input can be appropriately performed by signals output from the masking signal speakers 906, 907, and 908. It is desirable to install in.

  FIG. 10 is a flowchart illustrating an example of a processing flow of the surrounding environment measurement unit 201 and the masking signal determination unit 202.

  In step S1001, at least one of the presence / absence of a third party around the apparatus, the direction of the third party, and the position of the third party is measured. There are various measurement methods, but an infrared sensor may be used as a suitable method. The position and number of infrared sensors are set according to the measurement accuracy and the number, direction, and position assumed by a third party. As another method for replacing the infrared sensor, there is a method using a video camera. In this case, whether or not there is a third party by taking a picture of the surroundings of the device with a video camera, making a person determination using this image, or using a difference image from the image when there is no third party It becomes possible to measure the direction of the third party and the position of the third party. In addition, any method such as a wireless IC tag or a non-contact IC card may be used as long as a third party existing around the apparatus can be detected.

  Next, in step S1002, the level of the masking signal is determined according to the presence status of the third party. The presence of a third party is the presence or absence of the third party, the number of third parties, the distance from the device to the third party, and the direction of the third party with respect to the device. At this time, as shown in FIG. 9, when there are a plurality of masking signal output speakers, the level of the masking signal is determined for each speaker according to the presence of a third party existing in the direction of each speaker. To do. In addition, the masking signal can be determined and controlled in consideration of both the noise situation around this apparatus and the presence of third parties. For example, a case where there are four microphones (directions A, B, C, and D) in the same direction each for a noise measurement microphone, a third party position measurement infrared sensor, and a masking signal output speaker will be described. To do. Now, A has a high noise level and no third party, B has a low noise level and no third party, C has a high noise level and has a third party, and D has a low noise level. Suppose you have a third party. In this case, the masking signal level can be appropriately output by setting the level of the masking signal output from each speaker as A <B <C <D.

  As is clear from the above description, according to the present embodiment, in the voice input device, the masking signal is used so that the voice input by the user is not heard by a nearby third party and the third party is not uncomfortable. Can be appropriately controlled.

(Second Embodiment)
In the second embodiment, a case will be described in which the voice processing apparatus in FIG. 1 is functioned as a voice output apparatus. The voice output here includes not only the reproduction of the recorded voice but also the output of the voice synthesized by the voice synthesis process.

  FIG. 11 is a block diagram illustrating a functional configuration of the audio output device according to the present embodiment.

  Reference numeral 1101 denotes an audio output unit that outputs audio. Reference numeral 1103 denotes an audio output status management unit that manages the operation state of the audio output unit 1101. Here, an event issued with an operation related to audio output as a trigger is monitored. Reference numeral 1102 denotes a masking signal determination unit that determines a masking signal for masking output sound to a third party based on information from the sound output state management unit 1103. A masking signal control unit 1104 controls the output of the masking signal determined by the masking signal determination unit 1102.

  Next, an example of audio output processing in the present embodiment will be described using the flowcharts of FIGS. In the present embodiment, the processing of the audio output unit 1101 and the masking signal control unit 1104 will be described as event-driven processing. The target events are an “audio output start” event indicating that audio output has started, and an “audio output end” event indicating that audio output has ended.

  FIG. 12 is a flowchart showing a processing flow of the audio output unit 1101 and the audio output status management unit 1103.

  First, in step S1201, event waiting is performed. If an event is detected, the process advances to step S1202 to determine the type of event detected. If an “audio output start” event indicating that audio output has started is detected, the process proceeds to step S1203. If an “audio output end” event indicating that audio output has ended is detected, step S1203 is performed. The process proceeds to S1204.

  In step S1203, audio output is started from the audio output speaker 106, and the process returns to step S1201. It is assumed that audio output continues after the process returns to step S1201. On the other hand, in step S1204, the audio output from the audio output speaker 106 is terminated, and the process ends.

  Here, each event of “sound output start” and “sound output end” is generated by the sound output device as a predetermined timing even if it is given by the user such as a user pressing a button. Also good.

  FIG. 13 is a flowchart showing a processing flow of the masking signal control unit 1104.

  Here, a masking signal for preventing the output sound from being heard by a third party in the process shown in FIG. First, in step S1301, event waiting is performed. Next, if an event is detected, the process advances to step S1302 to determine the type of the detected event. If an “audio output start” event indicating that audio output has started is detected, the process proceeds to step S1303. If an “audio output end” event indicating that audio output has ended is detected, step S1303 is performed. The process proceeds to S1304.

  In step S1303, the masking signal output speaker 107 starts outputting a predetermined masking signal, and the process returns to step S1301. Note that the output of the masking signal continues even after the processing returns to step S1301. In step S1304, the output of the masking signal is terminated and the process is terminated.

Here, the level of the masking signal includes the average logarithmic power P _M of the masking signal x _M (t), the amplifier gain of the first speaker amplifier 106a, the mounting position and direction of the audio output speaker 106, and the second speaker amplifier 107a. Considering amplifier gain, masking signal output speaker 107 mounting position and direction, speaker directivity (for example, directional speaker or omnidirectional speaker), position and direction in which a third party is assumed to exist, etc. Need to design.

  Any masking signal may be used, but a masking signal that does not make it difficult to hear the output sound to the user, or that is comfortable to a third party or a user in the vicinity is desirable.

  FIG. 14 is a diagram illustrating an example of an external configuration of the audio output device according to the present embodiment.

  Reference numeral 1401 denotes a main body of the audio output apparatus that accommodates the hardware shown in FIG. Reference numeral 1403 denotes a masking signal output speaker for outputting a masking signal to a third party, which corresponds to the masking signal output speaker 107 of FIG. As in this example, when there is only one masking signal speaker or when the direction in which a third party exists is not known in advance, it is desirable to use a non-directional speaker. Reference numeral 1402 denotes an audio output speaker for outputting audio to the user, and corresponds to the audio output speaker 106 of FIG. The audio output speaker 1402 is placed closer to the user's ear than the masking signal output speaker 1403 so that the output sound is not difficult for the user to hear due to the signal output from the masking signal speaker 1403. It is desirable to do.

  As is clear from the above description, according to the present embodiment, in the audio output device, the output of the masking signal is performed so that the output audio is not heard by a nearby third party and the third party is not uncomfortable. It becomes possible to control appropriately.

(Third embodiment)
In the second embodiment described above, the generation and output of the masking signal for the output sound is performed only based on the event issued by the operation related to the sound output. However, the second embodiment is related to the sound input function. As described, the surrounding environment can be measured, and this information can also be used to generate and output a masking signal.

  FIG. 15 is a block diagram illustrating a functional configuration of the audio output device according to the present embodiment. This is a configuration in which a surrounding environment measurement unit 1505 is added to the configuration of FIG. 11 according to the second embodiment. The surrounding environment measurement unit 1505 measures the noise level, noise direction, noise position, presence / absence of a third party existing around the apparatus, and the direction and position of the third party around the sound output device. Since other components are the same as those in FIG. 11, the same reference numerals are assigned and description thereof is omitted.

  FIG. 16 is a diagram illustrating an example of an external configuration of the audio output device according to the present embodiment.

  Reference numeral 1601 denotes a main body of the audio output apparatus that accommodates the hardware shown in FIG. Reference numeral 1602 denotes an audio output speaker for outputting audio to the user, and corresponds to the audio output speaker 106 of FIG. Reference numerals 1606, 1607, and 1608 denote masking signal output speakers for outputting a masking signal to a third party, and correspond to the masking signal output speaker 107 of FIG.

  Reference numerals 1603, 1604, and 1605 denote microphones for measuring the noise level around the apparatus at three points. As described above, since a plurality of microphones for measuring the surrounding environment and speakers for outputting masking signals are provided, a masking signal output speaker is used according to the individual noise levels measured by the microphones 1603, 1604, and 1605. It becomes possible to change the level of each masking signal output from 1606, 1607, and 1608.

  In addition, as in this example, when a plurality of microphones for measuring the surrounding environment and speakers for masking signals are provided, it is desirable to use directional microphones and speakers. Further, the masking signal output speakers 1606, 1607, and 1608 are connected to the speaker 1602 so that the sound output from the masking signal 1606, 1607, and 1608 does not become difficult for the user to hear. It is desirable to install it at a position away from the user.

  In addition to measuring noise, the measurement of the surrounding environment should be done for people other than users, such as the presence or absence of third parties around the device, the direction and position of third parties, or a combination of these. You can also. This case can be realized by a method similar to the flowchart of FIG. 10 described in the first embodiment related to the voice input function.

  As is clear from the above description, according to the present embodiment, in the audio output device, the output sound is not heard by the surrounding third party by considering the measurement result of the surrounding environment, and the third party It is possible to appropriately control the output of the masking signal so as not to be uncomfortable.

(Fourth embodiment)
In the fourth embodiment, a case will be described in which the voice processing apparatus of FIG. 1 is made to function as a voice dialogue apparatus by cooperating a voice input function and a voice output function.

  FIG. 17 is a block diagram showing a functional configuration of the voice interactive apparatus according to the present embodiment.

  Reference numeral 1705 denotes a voice recognition unit that recognizes a user's input voice. Reference numeral 1703 denotes a voice input status management unit that manages the operating state of the voice recognition unit 1705, where events relating to voice recognition are monitored. Reference numeral 1701 denotes a surrounding environment measurement unit that measures a noise level, a noise direction, a noise position around the apparatus, the presence / absence of a third party existing around the apparatus, and a direction and position of the third party. Reference numeral 1702 denotes a masking signal determination unit that determines a masking signal for masking input sound or output sound to a third party based on information from the sound input state management unit 1703 and the surrounding environment measurement unit 1701. A masking signal control unit 1704 controls the output of the masking signal determined by the masking signal determination unit 1702. Reference numeral 1709 denotes a voice output unit that outputs synthesized voice. Reference numeral 1707 denotes an audio output status management unit that manages the operating state of the audio output unit 1707, where events relating to audio synthesis are monitored. A recognition result interpretation unit 1706 interprets the result of the voice recognition unit 1705. Reference numeral 1708 denotes a dialogue management unit that manages the state of dialogue with the user. Reference numeral 1710 denotes a response generation unit that generates contents to be output to the user according to the conversation state.

  FIG. 18 is a diagram illustrating an example of an external configuration of the voice interaction apparatus according to the present embodiment.

  Reference numeral 1801 denotes a main body of the voice interactive apparatus that accommodates the hardware shown in FIG. Reference numeral 1804 denotes a masking signal output speaker for outputting a masking signal to a third party, which corresponds to the masking signal output speaker 107 of FIG. As in this example, when there is only one masking signal speaker or when the direction in which a third party exists is not known in advance, it is desirable to use a non-directional speaker. Reference numeral 1802 denotes a microphone for measuring the voice input of the user and the surrounding noise environment, and corresponds to the microphone 105 in FIG. The microphone 1802 is preferably installed at a position closer to the user than the masking signal output speaker 1804 so that voice input can be appropriately performed by the masking signal output from the masking signal speaker 1804.

  Reference numeral 1803 denotes an audio output speaker for outputting audio to the user, which corresponds to the audio output speaker 106 of FIG. In order not to make it difficult for the user to hear the output sound due to the masking signal output from the masking signal speaker 1804, the sound output speaker 1803 is closer to the user's ear than the masking signal output speaker 1804. It is desirable to install.

  FIG. 19 shows an example of the timing of masking signal output in the voice interactive apparatus of this embodiment. The abscissa represents the elapsed time, and the ordinate represents the response (System) of the voice interactive device, the user's utterance (User), and the output of the masking signal (Mask) in order from the top. In this example, the system first outputs a voice saying “Please enter your name” from time t1 to time t3, and then the user says “Taro Yamada” from time t4 to time t5. Further, after that, a scene is shown in which the system performs a sound output of “Taro Yamada” from time t8 to time t9.

  At this time, the masking signal is output at the following timing. First, since it is considered that the voice output at the time t1 to t3 in the first system, “Please enter your name”, may be heard by a third party, no masking signal for this voice output is output. . Next, it is necessary to output a masking signal for the user's “Taro Yamada” utterance. At this time, since the user may start speaking without waiting for the end of the system sound output, the masking signal is output from the time t2 earlier than the end time t3 of the system sound output as shown in the figure. It is preferable to start. The end time of the masking signal is set to a time t6 obtained by adding a slight margin to the end time t5 of the utterance “Taro Yamada” of User.

  Next, the system's voice output “Taro Yamada-san” is also considered to be information that is not wanted to be heard by a third party, so a slight margin is taken into consideration for the output start time t8 and end time t9. The masking signal is output between time t7 and time t10.

  Needless to say, the output timing of the masking signal is not limited to this example. In addition, when a user is using the device or when the device is turned on, always output a masking signal and input / output At the timing, the level of the masking signal may be increased. In addition, the level of the masking signal may be changed according to the level of privacy when the user is asked to speak. Needless to say, the position and number of microphones for measuring the surrounding environment and the position and number of speakers for outputting a masking signal are not limited to those shown in FIG.

  Further, as described above, information relating to a third party may be measured using an infrared sensor, a video camera, or the like as the surrounding environment, and a masking signal may be determined and controlled using the measured information.

  As is clear from the above description, according to the voice interaction apparatus of the present embodiment, the input voice of the user and the output voice of the device are not heard by a nearby third party, and the third party is not uncomfortable. In addition, the masking signal output can be appropriately controlled.

(Other embodiments)
As mentioned above, although embodiment of this invention was explained in full detail, this invention may be applied to the system comprised from several apparatuses, and may be applied to the apparatus which consists of one apparatus.

  In the present invention, a software program that realizes the functions of the above-described embodiments is directly or remotely supplied to a system or apparatus, and the computer of the system or apparatus reads and executes the supplied program code. Is also achieved. In that case, as long as it has the function of a program, the form does not need to be a program.

  Therefore, in order to realize the functional processing of the present invention with a computer, the program code itself installed in the computer and the storage medium storing the program also constitute the present invention. In other words, the claims of the present invention include the computer program itself for realizing the functional processing of the present invention and a storage medium storing the program.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a storage medium for supplying the program, for example, flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R).

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a storage medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the hardware constitutions of the audio | voice processing apparatus with which this invention is applied. It is a block diagram which shows the function structure of the audio | voice input apparatus which concerns on 1st Embodiment. It is a flowchart which shows an example of the processing flow of the surrounding environment measurement part and masking signal determination part which concern on 1st Embodiment. It is a flowchart which shows an example of the processing flow of the audio | voice input part and audio | voice input status management part which concern on 1st Embodiment. It is a flowchart which shows an example of the processing flow of the masking signal control part which concerns on 1st Embodiment. It is a figure which shows an example of the function used when determining the level of the masking signal in 1st Embodiment. It is a flowchart which shows an example of the processing flow which determines a masking signal level dynamically using the audio | voice input level in 1st Embodiment. It is a figure which shows an example of the external appearance structure of the audio | voice input apparatus in 1st Embodiment. It is a figure which shows another example of the external appearance structure of the audio | voice input apparatus in 1st Embodiment. It is a flowchart which shows an example of the processing flow of the surrounding environment measurement part and masking signal determination part in 1st Embodiment. It is a block diagram which shows the function structure of the audio | voice output apparatus which concerns on 2nd Embodiment. It is a flowchart which shows an example of the processing flow of the audio | voice output part and audio | voice output condition management part in 2nd Embodiment. It is a flowchart which shows an example of the processing flow of the masking signal control part in 2nd Embodiment. It is a figure which shows an example of the external appearance structure of the audio | voice output apparatus in 2nd Embodiment. It is a block diagram which shows the function structure of the audio | voice output apparatus which concerns on 3rd Embodiment. It is a figure which shows an example of the external appearance structure of the audio | voice output apparatus in 3rd Embodiment. It is a block diagram which shows the function structure of the voice interactive apparatus which concerns on 4th Embodiment. It is a figure which shows an example of an external appearance structure of the voice interactive apparatus in 4th Embodiment. It is a figure which shows an example of the timing of the masking signal output in the voice interactive apparatus of 4th Embodiment. It is a figure which shows an example of the function used when determining the level of the masking signal in 1st Embodiment.

Claims (15)

A voice processing device for processing voice information received from an input means for inputting voice uttered by a user,
Measuring means for measuring the surrounding environment;
Determining means for determining a masking signal for masking the voice uttered by the user to a third party based on the measurement result by the measuring means;
And a control means for controlling the output of the masking signal determined by the masking signal determination means based on the operating state of the input means.   The speech processing apparatus according to claim 1, wherein the measurement unit measures at least one of a position of a third party and noise.   The speech processing apparatus according to claim 1, wherein the operation state of the input unit is determined based on a start event and an end event of speech input.   The audio processing apparatus according to claim 1, wherein the measuring unit includes an infrared sensor or a video camera for measuring a position of a third party. An audio processing device comprising output means for outputting audio,
Measuring means for measuring the surrounding environment;
Determining means for determining a masking signal for masking the sound output from the output means to a third party based on the measurement result by the measuring means;
And a control means for controlling the output of the masking signal determined by the masking signal determination means on the basis of the operating state of the output means.   6. The speech processing apparatus according to claim 5, wherein the measuring unit measures at least one of a third party position and noise.   The sound processing apparatus according to claim 5, wherein the operation state of the output unit is determined based on a start event and an end event of sound output.   The audio processing apparatus according to claim 5, wherein the measurement unit includes an infrared sensor or a video camera for measuring a position of a third party. A speech processing apparatus comprising: reception means for receiving voice information from input means for inputting voice uttered by a user; and output means for outputting voice,
Measuring means for measuring the surrounding environment;
A decision to mask a user's voice input to the input means for a third party and to determine a masking signal for masking the voice output from the output means based on a measurement result by the measuring means Means,
And a control means for controlling the output of the masking signal determined by the masking signal determination means on the basis of the operating states of the input means and the output means. A control method of a voice processing device for processing voice information received from an input means for inputting voice uttered by a user,
A measurement step for measuring the surrounding environment;
A determination step of determining a masking signal for masking the voice uttered by the user to a third party based on the measurement result of the measurement step;
And a control step of controlling the output of the masking signal determined by the masking signal determination step based on the operating state of the input means.   The method according to claim 10, wherein the measuring step measures at least one of a third party position and noise. A method for controlling a speech processing apparatus comprising output means for outputting speech,
A measurement step for measuring the surrounding environment;
A determination step of determining a masking signal for masking the sound output from the output means to a third party based on the measurement result of the measurement step;
And a control step of controlling the output of the masking signal determined by the masking signal determination step based on the operating state of the output means.   The method according to claim 12, wherein the measurement step measures at least one of a position of a third party and noise. A control method for a voice processing device comprising: a receiving means for receiving voice information from an input means for inputting voice uttered by a user; and an output means for outputting voice,
A measurement step for measuring the surrounding environment;
A decision for masking a user's voice input to the input means for a third party and determining a masking signal for masking the voice output from the output means based on the measurement result of the measurement step. Steps,
And a control step for controlling the output of the masking signal determined by the masking signal determination step based on the operating states of the input means and the output means.   The program for implement | achieving the control method of the audio processing apparatus in any one of Claim 10 to 14 with a computer.

Images