JP6194740B2 - Audio processing apparatus, audio processing method, and program - Google Patents

Description

  The present invention relates to a voice processing device, a voice processing method, and a program.

  In recent years, it has been considered to provide information that assists a person in accordance with the behavior and state when the person is performing a normal activity. In such an application, it is bothersome to take out the terminal every time information is provided or to turn the line of sight toward the terminal. Therefore, a hands-free and eyes-free user interface that does not occupy human hands and eyes is considered necessary.

  In an acoustic technique that can be used for such an interface, a method of changing a sound localization position for providing information is known. For example, a method of changing a sound image localization direction by changing sound pressure levels and sound generation times of a plurality of speakers, a method of delaying left and right channel signals, and the like are known (see, for example, Patent Documents 1 to 3). . In addition, when a conversation is detected, a method of moving a localization position of a sound image of content audio to an arbitrary position, a method of specifying a viewpoint of a virtual image, and a method of changing sound according to the specified viewpoint are known (for example, And Patent Documents 4 to 5). Furthermore, a method for detecting a conversation is also known (see, for example, Patent Document 6 and Non-Patent Document 1).

JP 2001-112083 A JP-A-8-237790 International Publication Number WO00 / 45619 JP 2011-97268 A Japanese Patent Laid-Open No. 10-137445 JP 2007-17620 A

Proceedings of the Acoustical Society of Japan “Study of a robust speech recognition decoder using VAD reliability”, Tsubasa Onishi, Dickson Paul, Koji Iwano, Sadayoshi Furui, p. 49-50 (September 2009)

  In acoustic technology used for a user interface that does not occupy human hands or eyes, it may be desirable to adjust the volume balance between the sound of the information to be provided and the surrounding environmental sound in accordance with environmental changes and user conditions. . However, since the volume of the environmental sound changes every moment, it is difficult to manually adjust the volume balance. In addition, in the conventional acoustic technology that changes the localization position of the acoustic sound as described above, the relevance with the situation in the real world may be lost, and the volume of the sound of the information to be provided and the environmental sound There is a problem that the balance cannot be adjusted.

  According to one aspect, an object of the present invention is to make it possible to control the volume balance between the environmental sound and the sound of information to be provided in response to changes in the surrounding sound environment.

  An audio processing apparatus according to one aspect includes a sound collection unit, an audio acquisition unit, a superposition ratio calculation unit, a superposition processing unit, and an output unit. The sound collection unit collects environmental sounds. The voice acquisition unit acquires an information sound of information to be provided. The superimposition ratio calculating unit calculates a difference between a first representative value of the time series data of the sound pressure level of the information sound and a second representative value of the time series data of the sound pressure level of the environmental sound, and a first predetermined value. A superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound so as to compensate for a difference from the value is calculated. The superimposition processing unit performs a process of superimposing the information sound and the environmental sound based on the superposition ratio. The output unit outputs an audio signal that has been subjected to the superimposing process.

  According to the sound processing device, the sound processing method, and the program according to the embodiment, the volume balance between the environmental sound and the sound of the information to be provided can be controlled in accordance with the change in the surrounding sound environment.

It is a figure which shows an example of the hardware constitutions of the speech processing system by 1st Embodiment. It is a block diagram which shows an example of the function of the speech processing unit by 1st Embodiment. It is a figure explaining the method of calculating the representative value of the sound pressure level by 1st Embodiment. It is a figure explaining the method of calculating the representative value of the sound pressure level by 1st Embodiment. It is a figure explaining the method of calculating the representative value of the sound pressure level by 1st Embodiment. It is a flowchart which shows operation | movement of the speech processing system by 1st Embodiment. It is a figure which shows notionally an example of the utilization condition of the speech processing system by 2nd Embodiment. It is a block diagram which shows an example of a structure of the speech processing system by 2nd Embodiment. It is a block diagram which shows an example of the function of the speech processing unit by 2nd Embodiment. It is a figure explaining the gaze state by 2nd Embodiment. It is a block diagram which shows an example of the function for detecting the gaze state by 2nd Embodiment. It is a figure which shows an example of the infrared information by 2nd Embodiment. It is a figure which shows an example of the attention object information by 2nd Embodiment. It is a figure which shows an example of the front list | wrist by 2nd Embodiment. It is a flowchart which shows the main operation | movement of the speech processing system by 2nd Embodiment. It is a flowchart which shows the gaze detection process by the audio | voice processing system of 2nd Embodiment. It is a block diagram which shows an example of the function of the speech processing unit by 3rd Embodiment. It is a flowchart which shows operation | movement of the speech processing system by 3rd Embodiment. It is a flowchart which shows operation | movement of the speech processing system by 3rd Embodiment. It is a figure which shows an example of the hardware constitutions of the speech processing system by a modification. It is a figure which shows an example of the hardware constitutions of a standard computer.

(First embodiment)
Hereinafter, the speech processing system 1 according to the first embodiment will be described with reference to the drawings. FIG. 1 is a diagram illustrating an example of a hardware configuration of the voice processing system 1. The sound processing system 1 superimposes the sound of information to be provided (hereinafter referred to as “information sound”) and the environmental sound that is the surrounding sound with a balance that is automatically adjusted in response to changes in the surrounding sound environment. It is a system that provides. As shown in FIG. 1, the voice processing system 1 includes a voice processing device 2 and a microphone device 30.

  The sound processing device 2 is a device that acquires environmental sound and outputs a sound signal in which information sound is superimposed on the environmental sound. The arithmetic processing device 3, the storage unit 5, the input unit 23, the display unit 25, and the sound input / output unit. 15. The voice processing device 2 can be, for example, a multifunctional mobile phone, a tablet computer, a music playback device, or the like.

  The arithmetic processing device 3 is a processor that controls the operation of the audio processing device 2. For example, the arithmetic processing device 3 reads and executes a control program stored in advance in the storage unit 5 to perform processing for controlling the operation of the audio processing device 2.

  The storage unit 5 is, for example, a semiconductor memory or the like, and includes a read only access memory (ROM) 7 and a random access memory (RAM) 9. The storage unit 5 stores, for example, a control program for controlling the operation of the sound processing device 2, various information necessary for the operation of the sound processing device 2, calculation results, and the like.

  The input unit 23 is a device for inputting information, and is, for example, a touch panel or a keyboard. The display unit 25 is a device that displays information, such as a liquid crystal display device. The voice input / output unit 15 outputs a voice signal to a voice output device such as a speaker or an earphone connected to the voice processing device 2, or receives a voice signal from a voice acquisition device such as a microphone connected to the voice processing device 2. It is a device that accepts input.

  The microphone device 30 includes an earphone 32 and a microphone 34, and is connected to the audio processing device 2 by wire or wireless to exchange audio. The microphone device 30 is, for example, a binaural microphone device. The earphone 32 is a device that can be worn on the ear and outputs the sound signal generated by the sound processing device 2 as sound. The microphone 34 is preferably formed integrally with the earphone 32, and is a device that collects environmental sound and outputs it as a sound signal to the sound processing device 2.

  FIG. 2 is a block diagram illustrating an example of functions of the audio processing device 2. As shown in FIG. 2, the voice processing device 2 includes a sound collection unit 41, a voice acquisition unit 43, a superposition ratio calculation unit 45, a superposition processing unit 47, and an output unit 49. The sound collection unit 41 acquires an audio signal of environmental sound input from the microphone device 30, for example. The voice acquisition unit 43 acquires, for example, the information sound provided to the user by the voice processing device 2 via the microphone device 30 from the storage unit 5 or the like. The superimposition ratio calculation unit 45 sets the environmental sound so that the difference between the sound pressure level of the environmental sound acquired by the sound collection unit 41 and the sound pressure level of the information sound acquired by the sound acquisition unit 43 becomes a predetermined value. The superposition ratio of the sound pressure when superimposing the sound and the information sound is calculated. The superimposition processing unit 47 generates an audio signal in which the environmental sound and the information sound are superimposed according to the superposition ratio calculated by the audio acquisition unit 43. The output unit 49 outputs the audio signal generated by the superimposition processing unit 47 to, for example, the microphone device 30.

  Here, the operation of the superposition ratio calculation unit 45 according to the first embodiment will be further described with reference to FIGS. 3 to 5. 3 to 5 are diagrams illustrating a method for calculating a representative value of the sound pressure level based on the audio signal. 3 to 5 are diagrams illustrating an example of a method for calculating the representative value of the sound pressure level by different methods. 3 to 5, the horizontal axis indicates time, and the vertical axis indicates sound pressure level.

  FIG. 3 shows an example of an audio signal under constant white noise. Here, the sound pressure level may be the maximum sound pressure level at the same time of the signals of the plurality of channels when the sound signal of the sound includes signals of the plurality of channels.

  In FIG. 3, the moving average, the weighted average, and the median value are represented with respect to the original sound pressure level. The moving average is an average of time-series data of sound pressure levels within a fixed time in the past of a calculation target time of the moving average. The weighted average is an average calculated by adding a predetermined weight (coefficient) to time-series data of sound pressure levels within a fixed time in the past at the time at which the weighted average is calculated. In this example, the sound pressure level at a time closer to the calculation target time is calculated by increasing the weight. The median is the median when the time-series data of sound pressure levels within a certain time in the past of the median calculation target time are arranged in order of magnitude.

  In the example of FIG. 3, the original sound pressure level has a large variation in the sound pressure level with respect to the time, but the moving average, the weighted average, and the median are the sound pressure levels in which the variation is alleviated. In a situation such as under constant white noise, the change of the value is suppressed at any representative value, but the moving average is less changed and more stable than the other.

  FIG. 4 shows an example in which the sound pressure level has increased after a certain time. In this way, when the environment changes midway and the sound pressure level transitions, the follow-up is faster in the order of the weighted average, moving average, and median with respect to the change. In other words, it can be said that the weighted average can follow the increase in the sound pressure level faster than other calculation methods. In order to reflect this tendency in the representative value, it is considered preferable to employ a weighted average as the representative value.

  FIG. 5 shows an example in which there is a sudden change in the sound pressure level at a certain time. At this time, the median, moving average, and weighted average are not affected by the sudden change in the sound pressure level. In the situation where a loud sound suddenly enters like this, the median value hardly changes, but it can be seen that the value is greatly changed by dragging others. A sudden change is a transient phenomenon. Therefore, it is considered that a value that is not affected by a sudden change is preferable as the representative value, and in this case, it is preferable to adopt the median value.

As described above, there are a plurality of methods for calculating the representative value of the sound pressure level. In addition, the effective scenes differ depending on the representative value calculation method. For example, different representative values may have the following characteristics.
Moving average: Effective under white noise or periodic noise environment Weighted average: Effective when quick response to environmental changes, etc. Median: Effective under sudden loud noise

  Therefore, it is preferable to calculate the representative value by an effective method according to the state of fluctuation of the sound pressure level depending on the time. In particular, for example, when situations such as those shown in FIGS. 4 and 5 are predicted in advance, it is possible to set a representative value calculation method suitable for each situation.

  In the present embodiment, the sound processing device 2 includes the representative value Lb of the time-series data of the sound pressure level of the environmental sound collected by the sound collecting unit 41 and the sound pressure level of the information sound provided to the user. The representative value Lc of the time series data is calculated by a set calculation method.

  Hereinafter, the operation of the speech processing system 1 according to the first embodiment will be further described with reference to FIG. FIG. 6 is a flowchart showing the operation of the voice processing system 1. The processing by the sound processing system 1 is processing performed by the arithmetic processing device 3 reading and executing a control program stored in advance, but here, for convenience, it is assumed that each function shown in FIG. 2 performs processing. To do.

  As shown in FIG. 6, the superimposition ratio calculation unit 45 first determines a target value X of the sound pressure level difference (S61). Here, the sound pressure level difference is a representative value Lb of the sound pressure level of the environmental sound (hereinafter referred to as the environmental representative value Lb) and a representative value Lc of the sound pressure level of the information sound (hereinafter referred to as the information representative value Lc). Is the difference. The voice processing device 2 sets the target value X to a desired value, and controls so that the difference between the environment representative value Lb and the information representative value Lc always matches the target value X. Thus, when the environmental sound and the information sound are superimposed, for example, the information sound can be easily heard in a noisy environment, or the spoken voice can be heard with priority over the information sound.

  The sound collection unit 41 acquires environmental sound information from the microphone device 30 (S62). The sound acquisition unit 43 stores the current sound pressure level value of the information sound in a buffer (not shown). The sound collection unit 41 stores the current level value of the environmental sound in a buffer (not shown) (S63). Furthermore, the voice acquisition unit 43 calculates an information representative value Lc. The sound collection unit 41 calculates the environmental representative value Lb (S64).

  Here, the time T for determining the section for obtaining each representative value is an adjustment parameter. As an example, when the moving average is used as the representative value, considering that the superposition balance does not change suddenly due to the sudden generation of environmental sound, the time T can be determined as follows.

Since the discrimination range of the sound pressure level is 0.5 (dB) to 1.0 (dB), when the moving average value of the sound pressure level of the environmental sound is L, the sound pressure level difference is A. When a sudden sound of (dB) continues for k seconds, the minimum time for which the change of the moving average value is 0.5 (dB) or less is obtained.
|| {L (T−k) + (L + A) k} /T−L||≦0.5 (Expression 1)

The following formula 2 is obtained from the formula 1.
|| kA / T || ≦ 0.5 (Formula 2)

  For example, if k = 1 (second) and A = 30 (dB), then T = 60 (second). That is, by setting the time T = 60 (seconds), even if a sound 30 dB higher than the average environmental sound pressure level is generated for 1 second, the average value of the environmental sound pressure level is 0. Only 5 (dB) is increased, and as a result, the superposition ratio S hardly changes.

  Subsequently, the superposition ratio calculation unit 45 obtains a difference Y between the information representative value Lc and the environment representative value Lb (S65). The superimposition ratio calculation unit 45 obtains a difference Z between the target value X and the difference Y (S66). Further, the superposition ratio calculation unit 45 obtains the superposition ratio S so that the decibel expression of the distribution ratio of the sound pressure of the information sound and the environmental sound matches the difference Z (S67).

Here, the representative value of the time series data for the nearest T seconds of the sound pressure level of the environmental sound measured by the microphone device 30 is the environment representative value Lb, and the representative value of the time series data for the nearest T seconds of the sound pressure level of the information sound. When the value is the information representative value Lc, the superposition ratio S of the sound pressures can be obtained from the following equation 3.
X- (Lc-Lb) = 20 log ((1-S) / S) (Formula 3)

  This expression 3 shows the idea of comparing the target sound pressure level difference X with the current average sound pressure level difference (Lb−Lc) and compensating for the missing amount by adjusting the superposition ratio S. .

  The superimposition processing unit 47 superimposes the environmental sound and the information sound by the superposition ratio S calculated as described above based on the environmental sound acquired from the microphone device 30 and the time series data of the information sound acquired by the sound acquisition unit 43. Then, the output sound pressure is determined (S68). The output unit 49 reproduces sound by the microphone device 30 by outputting a sound signal corresponding to the obtained output sound pressure (step 70).

Here, if the sound pressure of the sound pressure at the moment of the environmental sound that p _b, information sound and p _c, the output sound pressure po is calculated by Equation 4 below.
_{po = Sp b + (1-} S) p c ··· ( Equation 4)

  Here, it is considered that the information sound, which is the content provided to the user, is adjusted so that the sound pressure level difference can be heard comfortably. For example, since the sound pressure level of a normal conversation is 60 (dB) and the sound pressure level of a conference room is 40 (dB), the target information sound is X = 20 (dB) higher than the environmental sound. Set the value X.

  In the audio processing device 2, for example, when there is no end instruction from the input unit 23 or the like (S70: NO), the process is repeated from S62, and when there is an end instruction (S70: YES), the process ends.

  As described above, according to the sound processing system 1 according to the first embodiment, the sound collection unit 41 collects environmental sounds, and the sound acquisition unit 43 acquires information sounds. The superimposition ratio calculation unit 45 calculates the environment representative value Lb and the information representative value Lc by a predetermined calculation method. The superimposition ratio calculation unit 45 calculates the superposition ratio S so that the difference between the environment representative value Lb and the information representative value Lc becomes a predetermined target value X. The superimposition processing unit 47 synthesizes speech based on the calculated superposition ratio S. The output unit 49 outputs the synthesized voice.

  As described above, in the sound processing system 1 according to the first embodiment, the sound pressure level difference between the environmental sound and the information sound can be automatically controlled so as to be a predetermined target value. Therefore, when a device that closes the ear, such as a canal-type earphone, is used, it is possible to prevent problems such as lack of attention to the surroundings and inability to talk due to the inability to hear environmental sounds around the user. As described above, the environmental sound and the information sound are superimposed and reproduced by the microphone device 30, so that the user can hear the environmental sound while maintaining a sense of direction and presence.

  Since it is possible to automatically adjust the superposition ratio of the environmental sound and the information sound, it is possible to realize a sound volume balance suitable for the situation. That is, by setting the target value X of the difference between the information representative value and the environmental representative value to a desired value in advance, the volume of the environmental sound is too high to concentrate on the information sound, or conversely, the volume of the environmental sound is low. It is prevented that it is too low and the attention to the surroundings is insufficient. Moreover, since the superimposition ratio can be adjusted automatically, the volume balance suitable for the surrounding sound environment that changes from moment to moment can be automatically maintained without the user manually adjusting the volume balance. Increases nature. By adjusting both the volume of the information sound and the volume of the environmental sound, it is possible to perform control such that the environmental sound is given priority.

(Second Embodiment)
Next, the speech processing system 100 will be described according to the second embodiment. FIG. 7 is a diagram conceptually illustrating an example of a usage status of the voice processing system 100. In the second embodiment, the same configurations and operations as those of the first embodiment are denoted by the same reference numerals, and redundant description is omitted. The voice processing system 100 includes a head device 130, a voice processing device 20, and an infrared generator 125.

  Similar to the voice processing system 1, the voice processing system 100 is a system that superimposes and outputs environmental sounds and information sounds. In the voice processing system 1, the microphone device 30 is connected to the voice processing apparatus 2, but in the voice processing system 100, the head device 130 is connected to the voice processing apparatus 20. The voice processing system 100 includes an infrared generator 125, and the voice processor 20 can measure its own position using infrared rays.

  The speech processing system 100 according to the second embodiment is assumed to be used in an area where there is a gaze target object that the user 110 is expected to gaze, such as the poster 111 and the poster 113. Therefore, it is preferable that the infrared ray generator 125 irradiates a region such as the front of the poster 111 or the front of the poster 113, for example. At this time, the infrared ray generator 125 may be provided at a location corresponding to the upper side of the user 110. As a result, the speech processing apparatus 20 detects, for example, a position such as an area in front of the poster 111 as its own position.

  FIG. 8 is a block diagram illustrating an example of the configuration of the voice processing system 100. As shown in FIG. 8, in the voice processing system 100, the voice processing apparatus 20 is connected to the head device 130 by wire or wirelessly. The hardware configuration of the audio processing device 20 can be the same as that of the audio processing device 2 according to the first embodiment. The voice processing device 20 acquires the position information of the infrared ray generator 125 from the infrared position information Data Base (DB) 143. The infrared position information DB 143 may be stored in the storage unit 5 of the voice processing device 20 in advance, or may be acquired, for example, via an information processing device that can be connected to the voice processing device 20 via a communication network. May be.

  The head device 130 includes an earphone 32, a microphone 34, a microcomputer 135, an acceleration sensor 137, a gyro sensor 139, and an infrared light receiving unit 141. As shown in FIG. 7, the head device 130 can listen to the sound while the user is wearing the head, such as headphones. The head device 130 picks up the environmental sound 121, the environmental sound 123, and the like with the microphone 34.

  The acceleration sensor 137 detects the acceleration of the head device 130. The acceleration sensor 137 may be a three-dimensional acceleration sensor, for example. The gyro sensor 139 measures the tilt of the head device 130. The infrared light receiving unit 141 receives infrared light from the infrared generator 125.

  The microcomputer 135 is an integrated circuit that functions as an information processing apparatus that can execute a program that performs predetermined processing. For example, the microcomputer 135 divides and outputs the audio signal input from the audio processing device 20 to the left and right earphones 32. Further, the microcomputer 135 outputs the voice acquired by the microphone 34 to the voice processing device 20. Further, the microcomputer 135 outputs the acceleration detected by the acceleration sensor 137, the angle detected by the gyro sensor 139, the identification information of the infrared light emitting device from which the infrared light received by the infrared light receiving unit 141 is emitted, and the like to the voice processing device 20. . At this time, the microcomputer 135 may perform a predetermined process such as analyzing the identification information of the infrared ray generator 125 from the detection result of the infrared ray receiver 141.

  FIG. 9 is a block diagram illustrating an example of functions of the audio processing device 20. As shown in FIG. 9, the sound processing device 20 includes a sound collection unit 41, a sound acquisition unit 43, a superposition ratio calculation unit 45, a superimposition processing unit 47, and an output unit 49, similar to the sound processing device 2. . The audio processing device 20 further includes a stereophonic sound processing unit 151, a state measurement unit 153, a state detection unit 155, and a position / orientation estimation unit 157.

  The state measurement unit 153 acquires detection results from, for example, the acceleration sensor 137, the gyro sensor 139, and the infrared light receiving unit 141 of the head device 130. The detection result is, for example, information corresponding to the acceleration of the head device 130 obtained from the acceleration sensor 137, the angle of the head device 130 obtained from the gyro sensor 139, and the position of the head device 130 obtained from the infrared light receiving unit 141. It is.

  The position / orientation estimation unit 157 estimates the position and orientation of the user from the detection result acquired by the state measurement unit 153. The position and orientation of the user includes the position of the user 110 obtained as the position of the head device 130, the front range 173 of the user 110 obtained as the direction of the head device 130, and the like. The position of the head device 130 is calculated, for example, by adding a change in position obtained by integrating the acceleration obtained from the acceleration sensor 137 to the position information obtained from the detection result of the infrared light receiving unit 141. At this time, for example, the position / orientation estimation unit 157 refers to the identification information of the infrared ray generator 125 acquired from the head device 130 in the infrared position information DB 143 and acquires corresponding position information. Details of the infrared position information DB 143 will be described later. The direction of the head device 130 is calculated by integrating angle information obtained from the gyro sensor 139, for example.

  For example, the stereophonic sound processing unit 151 performs stereophonic sound processing such as changing the number of channels or adjusting the reproduction time and frequency characteristics of the left and right sound with respect to the sound from the sound acquisition unit 43 and the sound collection unit 41. Do. For this processing, for example, the conventional acoustic processing described in any of Patent Documents 1 to 5 can be used. By such processing, for example, the virtual generation position of the information sound 115 and the information sound 117 can be set at a desired position. Therefore, for example, the information sound 115 can be set at the position of the poster 111 and the information sound 117 can be set at the position of the poster 113.

  The state detection unit 155 detects the user state from the information measured by the state measurement unit 153. For example, whether or not the user is walking is detected based on the acceleration obtained from the acceleration sensor 137. As this detection method, a method generally used in a pedometer or the like can be used. As another example, the state detection unit 155 may detect whether or not a conversation is being performed around the user 110 from information measured by the microphone 34. Whether or not a conversation is being performed can be detected, for example, by a method described in Patent Document 6, Non-Patent Document 1, and the like. The state detection unit 155 outputs the detected user state to the superposition ratio calculation unit 45.

  In the superimposition ratio calculation unit 45, the information sound processed by the stereophonic sound processing unit 151 and the environmental sound collected by the sound collection unit 41 according to the state of the user detected by the state detection unit 155. The superposition ratio S is calculated. For the calculation of the superposition ratio S, the same method as in the first embodiment can be applied.

  In the present embodiment, the gaze state is further detected as one of the user states. FIG. 10 is a diagram for explaining a gaze state. As shown in FIG. 10, the gaze state refers to, for example, a gaze range 171 based on a front range 173 estimated by the user 110 for a predetermined time or more, such as an object that outputs the information sound 115. The state that is judged to be in.

  Hereinafter, gaze state detection will be described with reference to FIGS. 11 to 14. FIG. 11 is a block diagram illustrating an example of a function for detecting a gaze state. As illustrated in FIG. 11, the voice processing device 20 includes a head measurement unit 163 as the state measurement unit 153, and includes a target position acquisition unit 161 and a gaze state detection unit 165 as the state detection unit 155.

  12 to 14 are diagrams illustrating examples of data structures of various data used for detecting the gaze state. 12 is a diagram illustrating an example of the infrared information 175, FIG. 13 is a diagram illustrating an example of the gaze target information 180, and FIG. 14 is a diagram illustrating an example of the front list 185.

  As shown in FIG. 12, the infrared information 175 is the contents of the infrared position information DB 143 described above, and includes an infrared identification (ID) 177 and position information 178. The infrared ID 177 is identification information of the infrared generator 125. The position information 178 is information indicating a position where the head device 130 exists when the infrared ray output from the infrared ray generator 125 corresponding to the infrared ID 177 is detected.

  As shown in FIG. 13, the gaze target information 180 has a gaze target ID 182 and position information 183. The gaze target ID 182 is identification information of an object that may be a gaze target of the user 110. In the example of FIG. 7, for example, the poster 111, the poster 113, and the like. The position information 183 is information indicating the position of the gaze target corresponding to the gaze target ID 182.

  As shown in FIG. 14, the front list 185 has a gaze candidate ID 187 and a detection time 188. The gaze candidate ID 187 is identification information of a gaze target that is determined to be present in the gaze range 171 of the user 110 and is estimated that the user 110 is gazing. The detection time 188 is a time when it is detected that the gaze candidate ID 187 is included in the gaze range 171.

  Returning to FIG. 11, the head measurement unit 163 acquires information on acceleration, angle, and infrared light reception from the head device 130. The position and orientation estimation unit 157 estimates the position and orientation based on the information from the head measurement unit 163. The position and orientation are, for example, the position of the user 110 and the gaze range 171 of the user 110. The position / orientation estimation unit 157 acquires the position of the user 110 by searching the infrared ID 177 in the infrared information 175 for the identification information of the infrared generator 125 acquired by the head measurement unit 163 and acquiring the corresponding position information 178. To do. The position / orientation estimation unit 157 estimates, for example, the orientation of the head device 130 based on the acceleration and angle acquired from the head device 130, calculates the front range 173 of the user 110, and estimates the gaze range 171. The angle range of the front range 173 can be determined in advance.

  The target position acquisition unit 161 acquires the position information 183 of the object from the gaze target information 180. The gaze state detection unit 165 determines whether there is position information 183 included in the gaze range 171 estimated by the position / orientation estimation unit 157. When there is position information 183 included in the gaze range 171, the gaze state detection unit 165 displays the gaze target ID 182 corresponding to the position information 183 and the detected time, the gaze candidate ID 187 in the front list 185, and the detection time. Store as 188. When the gaze state detection unit 165 detects that an object with the same gaze candidate ID 187 within the gaze range 171 for a certain time or more, the user 110 is determined to be in the gaze state.

  Subsequently, the operation of the speech processing system 100 according to the present embodiment will be described with reference to FIGS. 15 and 16. FIG. 15 is a flowchart showing main operations of the voice processing system 100. FIG. 16 is a flowchart showing gaze detection processing by the audio processing system 100. The processing by the voice processing system 100 is processing performed by the arithmetic processing device 3 reading and executing a pre-stored control program. Here, for convenience, each function illustrated in FIG. 9 or FIG. 11 performs processing. It will be described as being performed. Detailed description of the same processing as in the first embodiment is omitted.

As shown in FIG. 15, first, the superimposition ratio calculation unit 45 determines a target value X of the sound pressure level difference according to the user state (S191). In the present embodiment, the state detection unit 155 detects the user and the surrounding state. Here, as described above, for example, the following states can be detected.
State a) State in which conversation is performed around: Hereinafter, the target value at this time is Xa, and this state is referred to as a conversation state.
State b) State in which the user 110 is walking: Hereinafter, the target value at this time is Xb, and this state is referred to as a walking state.
State c) State in which the user 110 is gazing at the gaze target: Hereinafter, the target value at this time is Xc, and this state is referred to as the gaze state.
State d) State in which states a) to c) are not detected: Hereinafter, the target value at this time is Xd, and this state is referred to as a normal state.

At this time, as the magnitudes of the targets Xa to Xd, it can be considered that the magnitude relationship of the following Expression 5 is set depending on the state.
Xa <Xb <Xd <Xc (Formula 5)

  These target values may be stored in advance in the storage unit 5, for example, and the target values may be changed when each state is detected by the state detection unit 155. For example, X = Xd can be set as an initial value.

  The sound collection unit 41 acquires environmental sound information from the head device 130 (S192). The stereophonic sound processing unit 151 performs stereophonic sound processing such as adjusting the reproduction time and frequency characteristics of the left and right sounds with respect to the sound from the sound acquisition unit 43, and calculates the information sound to be output to the superposition ratio calculation unit 45. (S193). At this time, the stereophonic sound processing generates information sound virtually generated from a desired position according to the position and orientation estimated by the position and orientation estimation unit 157, the user state detected by the state detection unit 155, and the like. It is good also as processing to do. One of the user states considered at this time is a gaze state. Details of the gaze state detection process will be described later.

  The sound acquisition unit 43 stores the current sound pressure level value of the information sound in a buffer (not shown). The sound collection unit 41 stores the current level value of the environmental sound in a buffer (not shown) (S194). Furthermore, the voice acquisition unit 43 calculates an information representative value Lc. The sound collection unit 41 calculates the environmental representative value Lb (S195). Here, it is preferable that the time T for determining the section for obtaining each representative value is determined in the same manner as in the first embodiment.

  Subsequently, the superposition ratio calculation unit 45 obtains a difference Y between the information representative value Lc and the environment representative value Lb (S196). The superimposition ratio calculation unit 45 obtains a difference Z between the target value X and the difference Y (S197). Further, the superposition ratio calculation unit 45 obtains the superposition ratio S so that the decibel expression of the distribution ratio of the sound pressure of the information sound and the environmental sound matches the difference Z (S198). The superposition ratio S is calculated by the same method as the calculation method in the first embodiment.

  The superimposition processing unit 47 superimposes the environmental sound and the information sound by the superposition ratio S calculated based on the time series data of the environmental sound acquired by the head device 130 and the information sound acquired by the sound acquisition unit 43, and outputs The sound pressure is determined (S199). The output unit 49 reproduces sound by the head device 130, for example, by outputting a sound signal with the output sound pressure obtained by the above-described Expression 4 (step 200).

  In the audio processing device 20, for example, when there is no end instruction from the input unit 23 or the like (S201: NO), the process is repeated from S191, and when there is an end instruction (S201: YES), the process ends.

  Next, gaze state detection processing will be described with reference to FIG. As illustrated in FIG. 16, the target position acquisition unit 161 acquires position information of a gaze target candidate from, for example, the gaze target information 180 (S231). The head measurement unit 163 acquires a detection result from the head device 130. The position and orientation estimation unit 157 estimates the head position and orientation of the user 110 as the gaze range 171 based on the detection result of the head measurement unit 163 (S232).

  The gaze state detection unit 165 detects a gaze target candidate whose position information 183 is in the gaze range 171 by comparing the gaze range 171 and the gaze target information 180 (S233). When none of the gaze target candidates are within the gaze range 171 (S233: NO), the gaze state detection unit 165 deletes the gaze candidate ID 187 and the detection time 188 from the front list 185 (S234).

  If the gaze target candidate is in the gaze range 171 in S233 (S233: YES), the gaze state detection unit 165 determines whether or not the gaze target candidate is already included in the front list 185 (S235). ). If not included (S235: NO), the gaze state detection unit 165 stores the current time and identification information corresponding to the gaze target candidate in the front list 185 as the detection time 188 and the gaze candidate ID 187. If the gaze target candidate is already included in the front list 185 (S235: YES), the time recorded in the front list 185 is compared with the current time. 110 is determined to be a gaze state (step 237), and the process returns to the process of S193 in FIG.

  As described above, according to the sound processing system 100 according to the second embodiment, the sound collection unit 41 collects environmental sounds, and the sound acquisition unit 43 acquires information sounds. The state detection unit 155 detects, for example, a conversation state, a walking state, and a gaze state based on the information detected by the head device 130. The superimposition ratio calculation unit 45 calculates the superposition ratio S so that the difference between the environment representative value Lb and the information representative value Lc becomes target values Xa to Xd determined in advance according to the detected state. The superimposition ratio calculation unit 45 calculates the environment representative value Lb and the information representative value Lc by a predetermined calculation method.

  The superimposition ratio calculation unit 45 calculates the superposition ratio S so that the difference between the environment representative value Lb and the information representative value Lc becomes a predetermined target value X. The superimposition processing unit 47 synthesizes speech based on the calculated superposition ratio S. The output unit 49 outputs the synthesized voice. In the gaze state, it is preferable that the stereophonic sound processing unit 151 performs the sound processing so that the information sound is generated from the object estimated to be gazing.

  As described above, in the sound processing system 100 according to the first embodiment, the sound pressure level difference between the environmental sound and the information sound can be automatically controlled so as to have a predetermined target value. Therefore, when a device that closes the ear, such as a canal-type earphone, is used, it is possible to prevent problems such as lack of attention to the surroundings and inability to talk due to the inability to hear environmental sounds around the user. At this time, the environmental sound and the information sound are superimposed and reproduced by the head device 130, so that the user can hear the environmental sound while maintaining the sense of direction and the presence.

  Further, for example, when a conversation state is detected, the environmental sound can be increased compared to the information sound, and the user can actively listen to the environmental sound and enable conversation. When the walking state is detected, safety can be considered by making the environmental sound larger than the normal state while being smaller than the conversation state. When the gaze state is detected, the information sound can be made louder than the normal state to actively provide information.

  In this way, the superimposition ratio can be adjusted automatically, so that the user can automatically maintain a volume balance suitable for the surrounding sound environment that changes from moment to moment without manually adjusting the volume balance. Increase convenience. By adjusting both the volume of the information sound and the volume of the environmental sound, it is possible to perform control such that the environmental sound is given priority.

  Furthermore, the sound information output from the earphone 32 worn by the person is processed by the stereophonic sound processing unit 151 in accordance with the position and orientation of the user estimated by the position and orientation estimation unit 157, and from any position and direction around the person The virtual sound source position can be set so that it can be heard. Thus, by detecting the position / posture of a person's head, it becomes possible to adjust in real time so that the sound source position is fixed in the surrounding environment. As a result, it is possible to realize a voice-arranged reality (AR) that makes a person feel as if there is a sound source in a real-world environment.

  By using this voice AR, it is possible to provide hands-free and eyes-free information. As an application example of the user interface using the voice AR, as shown in FIG. 7, in a venue such as an exhibition, acoustic processing corresponding to the position information of each exhibit around the user can be performed. For example, it is possible to perform processing such that a voice is output from the exhibit to the explanation voice regarding the exhibit. By such processing, it may be possible to provide guidance that makes it easy for the user to search for an interesting exhibition.

(Third embodiment)
Hereinafter, a voice processing system according to the third embodiment will be described. In the third embodiment, configurations and operations similar to those in the first or second embodiment are denoted by the same reference numerals, and redundant description is omitted.

  The voice processing system according to the third embodiment can have the same hardware configuration as the voice processing system 240 according to the second embodiment. The speech processing system according to the third embodiment is an example in which the speech processing system 100 includes a speech processing device 250 instead of the speech processing device 20 and has a function of switching a representative value calculation method.

  FIG. 17 is a block diagram illustrating an example of functions of the audio processing device 250. As illustrated in FIG. 17, the sound processing device 250 is similar to the sound processing device 20 in that the sound collection unit 41, the sound acquisition unit 43, the superposition ratio calculation unit 45, the superposition processing unit 47, the output unit 49, and the stereophonic sound processing unit. 151, a state measurement unit 153, a state detection unit 155, and a position / orientation estimation unit 157. The voice processing device 250 further includes a representative value switching unit 251.

  The representative value switching unit 251 switches the calculation method of the representative value of the time series data of the sound pressure level. Specifically, the moving average, the weighted average, and the median value described in the first embodiment may be adopted depending on the situation where each representative value is effective.

In the first embodiment, the situation where each representative value is valid has been described as follows.
Moving average: Effective under white noise or periodic noise environment Weighted average: Effective when quick response to environmental changes, etc. Median: Effective under sudden loud noise

Specifically, the following situations can be considered as examples of situations where each representative value is valid.
Moving average: When the sound of air conditioning or fans becomes a certain amount of noise, such as in a data center, etc. Weighted average: When the noise level changes intermittently, such as at a construction site, etc. Median: Loud door opening / closing sound in the office If the impact sound is loud in sports

  In the first and second embodiments, it is assumed that the calculation method of the representative value is always used in advance, but in this embodiment, the representative value switching unit 251 Analyzes the distribution of environmental sound and automatically switches the representative value calculation method to be used.

  18 and 19 are flowcharts showing the operation of the voice processing system according to the third embodiment. The processing by the speech processing system according to the third embodiment is processing performed by the arithmetic processing device 3 reading and executing a pre-stored control program. Here, for convenience, each function shown in FIG. Will be described as performing processing. Further, detailed description of the same processing as in the first or second embodiment is omitted.

  As shown in FIG. 18, the superimposition ratio calculation unit 45 first determines a target value X of the sound pressure level difference according to the user state (S281). In the present embodiment, as in the second embodiment, the state detection unit 155 detects the user and the surrounding state. Here, as described above, it is preferable to switch the target values Xa to Xd. These target values are preferably stored in the storage unit 5 in advance, for example, and when each state is detected by the state detection unit 155, the target value is preferably changed.

  The sound collection unit 41 acquires environmental sound information from the head device 130 (S282). The stereophonic sound processing unit 151 adjusts the reproduction time and frequency characteristics of the left and right sounds in accordance with the position and orientation of the head obtained by the head device 130 with respect to the sound from the sound acquisition unit 43. Processing is performed to calculate the information sound to be output to the superposition ratio calculation unit 45 (S283). At this time, the stereophonic sound processing generates information sound virtually generated from a desired position according to the position and orientation estimated by the position and orientation estimation unit 157 and the user state detected by the state detection unit 155. It is good also as processing. As one of the user states considered at this time, the gaze detection described in the second embodiment may be performed. The sound acquisition unit 43 stores the current sound pressure level value of the information sound in a buffer (not shown). The sound collection unit 41 stores the current level value of the environmental sound in a buffer (not shown) (S284).

  Here, before calculating the superimposition balance at each time, the representative value switching unit 251 analyzes the time-series data of the sound pressure level of the environmental sound for the past certain time acquired by the microphone 34, and the data distribution is normal. It is determined whether or not the distribution is close (S285). As a determination method, a test method such as Jack-Bella test using skewness or kurtosis of time series data is used. When it is determined that the distribution is close to the normal distribution (S285: YES), the representative value switching unit 251 uses a moving average for the representative value (S286). If it is not determined that the distribution is close to the normal distribution (S285: NO), the representative value switching unit 251 uses the median value as the representative value (S287).

  The voice acquisition unit 43 calculates the information representative value Lc and the environment representative value Lb based on the calculation method set by the representative value switching unit 251 (S288). Here, it is preferable that the time T for determining the section for obtaining each representative value is determined in the same manner as in the first embodiment.

  As shown in FIG. 19, the superposition ratio calculation unit 45 obtains a difference Y between the information representative value Lc and the environment representative value Lb (S289). The superimposition ratio calculation unit 45 obtains a difference Z between the target value X and the difference Y (S290). Further, the superposition ratio calculation unit 45 obtains the superposition ratio S so that the decibel expression of the distribution ratio of the information sound and the environmental sound matches the difference Z (S291). The superposition ratio S of the sound pressures can be obtained from the above equation 3.

  The superimposition processing unit 47 superimposes the environmental sound and the information sound with the superposition ratio S calculated based on the time series data of the environmental sound acquired by the head device 130 and the information sound acquired by the sound acquisition unit 43, The output sound pressure is determined (S292). The output unit 49 reproduces sound by the head device 130, for example, by outputting a sound signal with the output sound pressure obtained by the above-described Expression 4 (step 293).

  In the audio processing device 250, for example, when there is no end instruction from the input unit 23 or the like (S294: NO), the process is repeated from S281, and when there is an end instruction (S294: YES), the process ends.

  As described above, according to the sound processing system according to the third embodiment, the sound collection unit 41 collects environmental sounds, and the sound acquisition unit 43 acquires information sounds. The state detection unit 155 detects, for example, a conversation state, a walking state, and a gaze state based on the information detected by the head device 130. The superimposition ratio calculation unit 45 sets any one of the target values Xa to Xd as the target value according to the state detected by the state detection unit 155.

  In the present embodiment, the representative value switching unit 251 analyzes time series data of past environmental sounds, and uses a moving average as a representative value when the distribution of the time series data is close to a normal distribution. At this time, the environmental representative value Lb is obtained by a moving average. For example, the information representative value Lc may be obtained by a predetermined method.

  The superimposition ratio calculation unit 45 calculates the superposition ratio S so that the difference between the environment representative value Lb and the information representative value Lc becomes a predetermined target value X. The superimposition processing unit 47 synthesizes speech based on the calculated superposition ratio S. The output unit 49 outputs the synthesized voice. At this time, when the target value X is in the gaze state, the stereophonic sound processing unit 151 can perform acoustic processing so that the information sound is generated from the object estimated to be gaze.

  As described above, in the sound processing system according to the third embodiment, in addition to the effects exhibited by the sound processing system 100 according to the second embodiment, a method suitable for the surrounding sound environment is switched to a representative sound pressure level. The value can be calculated. Therefore, for example, the median value is usually used so that it is not greatly affected by sudden sound, but when there is almost no sudden sound and the noise moves to an environment close to white noise, it automatically switches to a more stable moving average, etc. , More flexible response is possible. Moreover, while responding to an environment that changes from moment to moment, there is an effect that the balance is not greatly changed in response to sudden changes in environmental sounds.

(Modification)
Hereinafter, the voice processing system 240 according to the modification will be described. The modification is, for example, a modification of the sound processing system according to the first to third embodiments. The voice processing system 240 includes a voice processing device 242 and a microphone device 30. The voice processing device 242 is an example in which some of the functions of the head device 130 of the voice processing system 100 are added to the voice processing device 2 of the voice processing system 1. In the present modification, the same configurations and operations as those in the first to third embodiments are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 20 is a diagram illustrating an example of a hardware configuration of the voice processing system 240. The sound processing device 242 is a device that outputs information sound superimposed on the environmental sound, and similarly to the sound processing devices 2 and 20 according to the first to third embodiments, the arithmetic processing device 3, the storage unit 5, An input unit 23, a display unit 25, and a voice input / output unit 15 are provided.

  The voice processing device 242 further includes a communication unit 11, an antenna 13, an acceleration sensor 245, and a gyro sensor 247. The communication unit 11 performs transmission / reception processing of information with the outside of the audio processing device 242. The antenna 13 transmits and receives electromagnetic waves wirelessly. The acceleration sensor 245 detects the acceleration of the voice processing device 242. The acceleration sensor 245 can be, for example, a three-dimensional acceleration sensor. The gyro sensor 247 detects the angle of the sound processing device 242. The functional configuration of the voice processing device 242 can be the same as that of the voice processing device 20 according to the second embodiment or the voice processing device 250 according to the third embodiment.

  In the present modification, the walking state of the user 110 can be detected based on the detection results of the acceleration sensor 245 and the gyro sensor 247 as in the case of the voice processing device 20. Further, the conversation state can be detected in the same manner as the voice processing device 20 or the voice processing device 250 based on the result of the sound picked up by the microphone 34 of the microphone device 30. Furthermore, it is possible to acquire its own position using the Global Positioning System (GPS) via the communication unit 11 and the antenna 13. Furthermore, by using the detection results obtained by the acceleration sensor 245 and the gyro sensor 247, the gaze state can be determined as in the second or third embodiment.

  Therefore, similar to the sound processing system according to the second or third embodiment, the sound processing system 240 can superimpose and output the environmental sound and the information sound with a superposition ratio suitable for the state of the user 110. It is.

  As described above, according to the audio processing system 240 according to the modification, the same effects as those of the audio processing device 20 or the audio processing device 250 according to the second or third embodiment can be obtained. Furthermore, since this configuration eliminates the need for the infrared generator 125, the voice processing system 240 can be used wherever GPS is available.

  Here, an example of a computer that is commonly applied to cause a computer to perform the operations of the sound processing methods according to the first to third embodiments and the modified examples will be described. FIG. 21 is a block diagram illustrating an example of a hardware configuration of a standard computer. As shown in FIG. 21, a computer 300 includes a central processing unit (CPU) 302, a memory 304, an input device 306, an output device 308, an external storage device 312, a medium driving device 314, a network connection device, and the like via a bus 310. It is connected.

  The CPU 302 is an arithmetic processing unit that controls the operation of the entire computer 300. The memory 304 is a storage unit for storing in advance a program for controlling the operation of the computer 300 or using it as a work area when necessary when executing the program. The memory 304 is, for example, a RAM or a ROM. The input device 306 is a device that, when operated by a computer user, acquires various information input from the user associated with the operation content and sends the acquired input information to the CPU 302. Keyboard device, mouse device, etc. The output device 308 is a device that outputs a processing result by the computer 300, and includes a display device and the like. For example, the display device displays text and images according to display data sent by the CPU 302.

  The external storage device 312 is a storage device such as a hard disk, and stores various control programs executed by the CPU 302, acquired data, and the like. The medium driving device 314 is a device for writing to and reading from the portable recording medium 316. The CPU 302 can perform various control processes by reading and executing a predetermined control program recorded on the portable recording medium 316 via the medium driving device 314. The portable recording medium 316 is, for example, a Compact Disc (CD) -ROM, a Digital Versatile Disc (DVD), a Universal Serial Bus (USB) memory, or the like. The network connection device 318 is an interface device that manages transmission / reception of various data performed between the outside by wired or wireless. A bus 310 is a communication path for connecting the above devices and the like to exchange data.

  A program that causes a computer to execute the sound processing methods according to the first to third embodiments and the modifications is stored in, for example, the external storage device 312. The CPU 302 reads a program from the external storage device 312 and causes the computer 300 to perform an audio processing operation. At this time, first, a control program for causing the CPU 302 to perform voice processing is created and stored in the external storage device 312. Then, a predetermined instruction is given from the input device 306 to the CPU 302 so that the control program is read from the external storage device 312 and executed. The program may be stored in the portable recording medium 316.

  The present invention is not limited to the embodiments described above, and various configurations or embodiments can be adopted without departing from the gist of the present invention. For example, the user state detected by the state detection unit 155 is not limited to the above four states (a conversation state, a walking state, a gaze state, and a normal state). Moreover, it is good also as a speech processing system which can detect only some of four states. The user state detection method is not limited to the above. If a similar state can be detected, another method can be adopted. For example, a geomagnetic sensor may be installed in the head device 130, and the posture of the head device 130 may be estimated based on the detection result of the geomagnetic sensor.

  The information sound is stored in advance in the audio processing device 2 or the like, but can be modified such as being acquired from another information processing device that can communicate with the audio processing device 2 or the like. The infrared information 175, the gaze target information 180, and the like may be acquired from another information processing apparatus. In addition, it is possible to modify the audio processing device that can be carried by the user 110, such as only reproducing the sound, acquiring the environmental sound, and the user state, and performing other processing using another information processing device.

Regarding the above embodiment, the following additional notes are disclosed.
(Appendix 1)
A sound collection unit that collects environmental sounds;
An audio acquisition unit for acquiring information sounds of information to be provided;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. A superposition ratio calculating unit that calculates a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
A superimposition processor that performs a process of superimposing the information sound and the environmental sound based on the superposition ratio;
An output unit that outputs an audio signal subjected to the superimposing process;
A speech processing apparatus comprising:
(Appendix 2)
The sound processing according to appendix 1, wherein the first representative value and the second representative value are any one of a moving average, a weighted average, and a median value of time series data of each sound pressure level. apparatus.
(Appendix 3)
It further has a state detection unit for detecting the user's state,
The state detection unit detects whether or not conversation is included in the environmental sound,
When the state detection unit detects the conversation, the superposition ratio calculation unit replaces the first predetermined value with the superposition ratio based on a second predetermined value smaller than the first predetermined value. The speech processing apparatus according to Supplementary Note 1 or Supplementary Note 2, wherein:
(Appendix 4)
It further has a state detection unit for detecting the user's state,
The state detection unit detects whether the user is in a walking state,
When the state detection unit detects the walking state, the superposition ratio calculation unit is smaller than the first predetermined value instead of the first predetermined value, and the state detection unit detects the environmental sound. The speech processing apparatus according to appendix 1 or appendix 2, wherein the superposition ratio is calculated based on a third predetermined value that is larger than a second predetermined value when it is detected that a conversation is included in .
(Appendix 5)
A state detection unit for detecting a user's state, and a target position acquisition unit for acquiring a position of a target related to the information sound,
Further comprising
The state detection unit detects whether the user is gazing at the position of the object,
When the state detection unit detects the state of gazing at the position of the object, the superposition ratio calculation unit calculates the superposition ratio based on a fourth predetermined value that is larger than the first predetermined value. The speech processing apparatus according to Supplementary Note 1 or Supplementary Note 2, wherein the speech processing device is calculated.
(Appendix 6)
When it is determined that the distribution of the environmental sound for a certain period of time in the past is close to a normal distribution, the first representative value and the second representative value are moving averages of time series data of respective sound pressure levels. If not, the speech processing apparatus according to any one of appendix 1 to appendix 5, further comprising a representative value switching unit that sets a median value.
(Appendix 7)
The audio processor
Picks up environmental sounds,
Get the information sound of the information you provide,
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
And a voice processing method.
(Appendix 8)
The speech processing method according to appendix 9, wherein the first representative value and the second representative value are a moving average, a weighted average, or a median value of time series data of each sound pressure level.
(Appendix 9)
Supplementary note 7 or Supplementary note 8, wherein when the conversation is detected, the superposition ratio is calculated based on a second predetermined value smaller than the first predetermined value instead of the first predetermined value. The voice processing method described in 1.
(Appendix 10)
When the walking state is detected, instead of the first predetermined value, the state detection unit is smaller than the first predetermined value, and the state detection unit detects that the environmental sound includes conversation. 9. The speech processing method according to appendix 7 or appendix 8, wherein the superposition ratio is calculated based on a third predetermined value larger than the second predetermined value.
(Appendix 11)
Obtaining the position of the object associated with the information sound;
Detecting whether the user is gazing at the position of the object;
Supplementary note 7 or Supplementary note 8, wherein when the state in which the position of the object is being watched is detected, the superposition ratio is calculated based on a fourth predetermined value that is larger than the first predetermined value. The voice processing apparatus according to 1.
(Appendix 12)
When it is determined that the distribution of the environmental sound for a certain period of time in the past is close to a normal distribution, the first representative value and the second representative value are moving averages of time series data of respective sound pressure levels. If not, the speech processing apparatus according to appendix 7 or appendix 8, further comprising a median value.
(Appendix 13)
Picks up environmental sounds,
Get the information sound of the information you provide,
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
A program that causes a computer to execute processing.
(Appendix 14)
The program according to appendix 13, wherein the first representative value and the second representative value are a moving average, a weighted average, or a median value of time series data of each sound pressure level.
(Appendix 15)
Supplementary note 13 or Supplementary note 14, wherein when the conversation is detected, the superposition ratio is calculated based on a second predetermined value smaller than the first predetermined value instead of the first predetermined value. The program described in.

DESCRIPTION OF SYMBOLS 1 Voice processing system 2 Voice processing apparatus 3 Arithmetic processing apparatus 5 Memory | storage part 7 ROM
9 RAM
11 Communication Unit 13 Antenna 15 Audio Input / Output Unit 23 Input Unit 25 Display Unit 30 Microphone Device 32 Earphone 34 Microphone

Claims (11)

A sound collection unit that collects environmental sounds;
An audio acquisition unit for acquiring information sounds of information to be provided;
A state detection unit for detecting whether or not the environmental sound includes a conversation;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. The superimposition ratio indicating the ratio of the sound pressure of the environmental sound to the sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound is calculated, and the state detection unit includes the conversation in the environmental sound. A superposition ratio calculating unit that calculates the superposition ratio based on a second predetermined value smaller than the first predetermined value instead of the first predetermined value ,
A superimposition processor that performs a process of superimposing the information sound and the environmental sound based on the superposition ratio;
An output unit that outputs an audio signal subjected to the superimposing process;
A speech processing apparatus comprising: A sound collection unit that collects environmental sounds;
An audio acquisition unit for acquiring information sounds of information to be provided;
A state detection unit that detects a state in which conversation is included in the environmental sound and detects whether or not the user is in a walking state;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. When the superposition ratio indicating the ratio of the sound pressure of the environmental sound to the sound pressure of the superimposition sound obtained by superimposing the information sound and the environmental sound is calculated, and the state detection unit detects the walking state Is, instead of the first predetermined value, smaller than the first predetermined value, and a second predetermined value when the state detection unit detects a state in which conversation is included in the environmental sound A superposition ratio calculation unit that calculates the superposition ratio based on a third predetermined value that is greater than
A superimposition processor that performs a process of superimposing the information sound and the environmental sound based on the superposition ratio;
An output unit that outputs an audio signal subjected to the superimposing process;
A speech processing apparatus comprising: A sound collection unit that collects environmental sounds;
An audio acquisition unit for acquiring information sounds of information to be provided;
A target position acquisition unit that acquires a position of a target object related to the information sound;
A state detection unit for detecting whether the user is gazing at the position of the object ;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. The superimposition ratio indicating the ratio of the sound pressure of the environmental sound to the sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound is calculated, and the state detection unit determines the position of the object. A superimposition ratio calculating unit that calculates the superposition ratio based on a fourth predetermined value that is larger than the first predetermined value instead of the first predetermined value when detecting a state in which ,
A superimposition processor that performs a process of superimposing the information sound and the environmental sound based on the superposition ratio;
An output unit that outputs an audio signal subjected to the superimposing process;
A speech processing apparatus comprising: 4. The first representative value and the second representative value are any one of a moving average, a weighted average, and a median value of time series data of each sound pressure level. The speech processing device according to any one of the above. When it is determined that the distribution of the environmental sound for a certain period of time in the past is close to a normal distribution, the first representative value and the second representative value are moving averages of time series data of respective sound pressure levels. The speech processing apparatus according to any one of claims 1 to 4, further comprising a representative value switching unit that sets a median value in a case that is not the case . The audio processor
Picks up environmental sounds,
Get the information sound of the information you provide,
Detecting whether the environmental sound includes a conversation;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
When a state in which conversation is included in the environmental sound is detected, the superposition ratio is calculated based on a second predetermined value smaller than the first predetermined value instead of the first predetermined value. ,
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
And a voice processing method. The audio processor
Picks up environmental sounds,
Get the information sound of the information you provide,
Detect a state in which conversation is included in the environmental sound,
Detect if the user is walking,
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
When the walking state is detected, instead of the first predetermined value, a second state is detected when a state is detected that is smaller than the first predetermined value and the environmental sound includes conversation. Calculating the superposition ratio based on a third predetermined value greater than the predetermined value of
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
And a voice processing method. The audio processor
Picks up environmental sounds,
Get the information sound of the information you provide,
Obtaining the position of the object associated with the information sound;
Detecting whether the user is gazing at the position of the object;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
If the user detects a state of gazing at the position of the object, the superposition ratio is based on a fourth predetermined value that is larger than the first predetermined value instead of the first predetermined value. To calculate
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
And a voice processing method. Picks up environmental sounds,
Get the information sound of the information you provide,
Detecting whether the environmental sound includes a conversation;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
When a state in which conversation is included in the environmental sound is detected, the superposition ratio is calculated based on a second predetermined value smaller than the first predetermined value instead of the first predetermined value. ,
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
A program that causes a computer to execute processing. Picks up environmental sounds,
Get the information sound of the information you provide,
Detect a state in which conversation is included in the environmental sound,
Detect if the user is walking,
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
When the walking state is detected, instead of the first predetermined value, a second state is detected when a state is detected that is smaller than the first predetermined value and the environmental sound includes conversation. Calculating the superposition ratio based on a third predetermined value greater than the predetermined value of
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
A program that causes a computer to execute processing. Picks up environmental sounds,
Get the information sound of the information you provide,
Obtaining the position of the object associated with the information sound;
Detecting whether the user is gazing at the position of the object;
The difference between the first representative value of the time series data of the sound pressure level of the information sound and the second representative value of the time series data of the sound pressure level of the environmental sound is compensated for the difference between the first predetermined value. Calculating a superposition ratio indicating a ratio of a sound pressure of the environmental sound to a sound pressure of the superposed sound obtained by superimposing the information sound and the environmental sound,
If the user detects a state of gazing at the position of the object, the superposition ratio is based on a fourth predetermined value that is larger than the first predetermined value instead of the first predetermined value. To calculate
Superimposing the information sound and the environmental sound based on the superposition ratio;
Outputting an audio signal subjected to the superimposing process;
A program that causes a computer to execute processing.