JP5339849B2 - Speech intelligibility improving method and speech intelligibility improving system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a voice articulation improving method and a voice articulation improving system capable of improving an effect by voice articulation improving processing by accurately measuring noise power even if there is a delay from a time when a voice data RG are generated to a time when the voice is detected by a microphone. <P>SOLUTION: A voice data stream which is outputted by a voice data generating section is multiplied by a predetermined gain and inputted to a speaker, and a voice signal detected by the microphone is taken at a predetermined length unit. An output start time and an output end time of the voice data stream are monitored, and a taking start time and an end time of the voice data stream of the predetermined length which is taken by the microphone are monitored. It is determined whether or not the voice data stream of the predetermined length which is taken by the microphone is the voice data stream in a noise section by using each of the time, and the noise power is detected by using the voice data stream of the noise section. When it is not the noise section, voice power is detected based on the voice data stream, the gain which is multiplied with the voice data stream is determined by using the noise power and the voice power. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a speech intelligibility improving method and a speech intelligibility improving system, and in particular, a section where no guidance voice signal is output is defined as a noise section, and guidance is performed using noise power in the noise section and voice power in a non-noise section. The present invention relates to a speech intelligibility improving method and a speech intelligibility improving system for controlling a gain of an audio signal.

  There is an in-vehicle speech intelligibility improvement system that makes it possible to clearly hear voices (navigation guide voices, news and mail reading voices) output from speakers even under noisy conditions. For example, in an in-vehicle navigation device, sound such as route guidance is output from a speaker to a vehicle interior, but when noise such as engine sound and road noise is high during traveling, it is difficult to hear the speaker output sound due to a masking effect. Therefore, the loudspeaker output sound can be clearly heard even under noise by performing loudness compensation on the sound data according to the power of the sound data to be output and the power of the noise to increase the gain of the entire sound band.

  FIG. 7 is a configuration diagram of a conventional speech intelligibility improvement system (Patent Document 1). According to the speech intelligibility improving system of FIG. 7 (refer to Patent Document 1 for detailed operation), the identification voice 71 simulates the guidance voice signal SG at the installation position of the microphone 72, and the subtractor 73 from the output of the microphone 72. The noise signal SN is extracted by subtracting the signal. The loudness compensation gain calculation unit 74 calculates the gain Gopt based on the guidance voice signal and the noise signal and inputs the gain Gopt to the RG correction unit (RouteGuidance voice correction unit) 75. At this time, the identification processing in the identification filter 76 is performed using the adaptive filter 77, and the adaptive algorithm unit 78 therein can be realized using various adaptive algorithms, and one of the typical ones is as follows. Although it is the LMS algorithm, the filter coefficient may be updated using a Fast-LMS algorithm (LMS algorithm in the frequency domain) or the like.

  However, if an estimation error occurs in the power of the audio signal, the error in the noise estimation power calculated by subtraction by the subtracter 73 becomes opposite to the sign of the error in the estimation power of the audio signal, and the difference width increases and the gain increases. Cannot be determined correctly. In addition, the conventional speech intelligibility improving system has a problem that an expensive DSP is required due to an excessive amount of calculation.

  Therefore, in the system that controls the gain of the audio signal based on the audio power and noise power of the audio signal, it is detected whether or not the audio power is above the set level, that is, whether or not the guidance audio is being output. Measure and save the noise power when the voice power is lower than the set level (when no guidance voice is output), and save the noise power when the voice power is higher than the set level (when the guidance voice is output) Has been proposed to control the gain of the audio signal based on the audio power and the estimated noise power.

FIG. 8 is a block diagram of the proposed speech intelligibility improving system.
The guidance voice generation unit 81 of the navigation device generates a guidance voice signal when approaching an intersection, for example. The sound driver 82 performs sound quality control on the guidance voice signal, amplifies it, and outputs it. The RG correction unit 83 multiplies the audio signal output from the sound driver 82 by the gain g determined by the correction value calculation unit 84, which will be described later, corrects the volume, and inputs it to the DAC 85. The DAC 85 converts the input audio signal into an analog signal. And input to the speaker 86. The speaker 86 outputs an input audio signal. The microphone 87 detects the synthesized sound of the guidance voice a and the surrounding noise n (engine sound, road noise, etc.), converts it into digital data by the ADC 88, and inputs it to the power calculator 89b via the audibility correction filter 89a. The power calculation unit 89b calculates the power by performing the square calculation of the amplitude of the input microphone detection signal, and inputs the power to the switching unit 89c.

  The switching unit 89c uses the power calculated by the power calculation unit 89b as noise via the fixed contact A in a section where no guide voice is output, that is, when the power of the audio signal (audio power) is smaller than the set value. In the section in which the guide voice is output, which is input to the power averaging unit 84b, that is, when the voice power is larger than the set value, the power calculated by the power calculation unit 89b is output to the B contact side and any unit Also do not enter. The noise power averaging unit 84b regards the power output from the power calculation unit 89b as noise power in a section where no guide voice is output, and the latest N (N: constant) powers output from the power calculation unit 89b. And the moving average value is stored as noise power in the power storage unit 84c. As a result, when the guide voice is output, the latest noise power in the section where the previous guide voice is not output is stored in the power storage unit 84c. The noise power during the output of the guide voice is regarded as the noise power stored in the power storage unit 84c, and the noise power stored in the power storage unit 84c is input to the loudness compensation gain calculation unit 84a.

  In parallel with the above, the voice signal output from the guidance voice generator 81 is input to the voice power calculator 89f via the audibility correction filter 89e. The audio power calculation unit 89f calculates the audio power by performing the square calculation of the amplitude of the input audio signal, and inputs the audio power to the determination unit 89g and the audio power averaging unit 89h. The determination unit 89g compares the input sound power with the set level, determines that the guide sound is not output when the sound power is lower than the set level, and when the sound power is higher than the set level. It determines with it being the area where the guide sound is output. Then, the determination unit 89g controls the switch 89c to input the power calculated by the power calculation unit 89b to the noise power averaging unit 84b in the section where the guide voice is not output, and outputs the guide voice. Then, it does not input to any unit. The audio power averaging unit 89h calculates an average value of M (M: constant) audio powers output from the audio power calculation unit 89f and inputs the average value to the variable gain unit 84d. The variable gain unit 84d sets the gain G that is set. Is multiplied by the average voice power and input to the loudness compensation gain calculator 84a. Note that the gain G set by the variable gain unit 84d is set by separately obtaining the gain G by assuming that the propagation characteristic from the input terminal of the speaker 86 to the microphone output terminal can be approximated only by the gain. To do. The loudness compensation control unit 89a makes the guidance voice clear regardless of the noise level based on the voice power input from the variable gain unit 84d and the noise power input from the power storage unit 84c in the section where the guide voice is output. The audible gain g is determined by the human loudness characteristic and input to the correction unit 83. The RG correction unit 83 receives the gain g, multiplies the guidance voice signal by the gain g, and outputs the result. Note that the loudness compensation control unit 84a does not perform the gain g determination control in a section where no guide voice is output.

  FIG. 9 shows an example in which the speech intelligibility improvement system of FIG. 8 is realized by a multi-process general-purpose CPU 91 and a DSP (Digital Signal Processor) 80, and the same parts as those in FIG. The DSP 80 executes the functions of the RG correction unit 83, the correction value calculation unit 84, and the noise separation unit 89 in FIG. 8, and the multi-process general-purpose CPU 91 performs guidance voice data creation processing of the guidance voice generation unit 81 in FIG. An audio reproduction process (VOICE application) 91d such as a process of transferring audio data to the sound driver 82 is performed. The general-purpose CPU 91 executes a plurality of applications such as a navigation process 91a, an in-vehicle audio process 91b, and a car phone process 91c in addition to the sound reproduction process, and preferentially executes a process with a high priority. It has become.

In the voice reproduction process, the RG generation unit 81b reads the guidance voice data encoded from the guidance voice data storage unit 81a, decodes it, and inputs it to the RG playback unit 81c. The RG playback unit 81c receives the input voice data. Is temporarily stored, and the sound data RG is input to the sound driver 82 as appropriate. The sound driver 82 performs predetermined processing on the sound data and inputs the sound data to the RG correction unit 83 via the sound driver 82. The RG correction unit 83 calculates the correction value calculated by the correction value calculation unit 84 to the input sound data RG. (Gain) is multiplied to perform volume correction, and the corrected audio signal RG ′ is converted into an analog signal and input to the speaker 86. The speaker 86 outputs the input audio signal RG ′, and the microphone 87 detects the audio signal and ambient noise, and inputs detection data (MIC data) to the noise separation unit 89 via the ADC 88. The noise separation unit 89 calculates the guidance voice power level and the noise power level using the MIC data and the guidance voice data RG, and inputs them to the correction value calculation unit 84. The correction value calculation unit 84 calculates a gain based on the input guidance voice power level and the noise power level, and inputs the gain to the RG correction unit 83. The RG correction unit 83 uses the input correction value as voice data. Multiply by and output. As a result, the gain of the audio signal has no significant error, and the amount of calculation can be greatly reduced.
Note that it has also been proposed to perform speech intelligibility improvement processing on a general-purpose CPU using only a multi-process general-purpose CPU 91 without using a DSP.
Japanese Patent Laid-Open No. 11-166835

  However, in the proposed technique, in a multi-process in which a general-purpose CPU processes a plurality of applications, each application has a priority, and the processing of the speech intelligibility improvement system is not always performed. There is a time delay until the RG sound generated by the RG generation unit and output from the speaker by the microphone is detected by the microphone and input to the noise separation unit. This will be specifically described with reference to FIG. FIG. 10 shows a case where the general-purpose CPU 91 executes the processing of the DSP 80 of FIG. 9. The GAE unit (correction unit 83, correction value calculation unit 84, noise separation unit 89), sound driver 82, RG reproduction unit The arrangement of 81c and the like is changed. The processing of the RG playback unit 81c on the general-purpose CPU 91 has a low priority, and the audio data input from the RG correction unit 83 cannot always be output immediately to the sound driver 82, but stays in the built-in buffer and causes a delay. In other parts, delay occurs. As a result, the sound driver 82 inputs the microphone detection voice data to the noise separation unit 89 after a considerable time has elapsed after the RG generation unit 81 b inputs the voice data RG to the noise separation unit 89. When such a delay occurs, accurate noise power cannot be measured, and the effect of the speech intelligibility improvement processing is reduced. FIG. 11 is a time chart for explaining that the noise power cannot be measured accurately. The output start time RGtime-S of the audio data RG, the output end time of the audio data RG is RGtime-E, and the start of capturing the audio signal by the microphone. Is MICtime-S, and the end time of audio signal capture by the microphone is MICtime-E. Times before RGtime-S and after time RGtime-E are noise intervals, and times RGtime-S to RGtime-E are non-noise intervals. There is no measurement error of noise power in the first noise section because no sound is output in period A, but since the sound is output in period B in the subsequent noise section, the sound is also detected as noise and accurate. Noise power cannot be measured.

As described above, an object of the present invention is to enable accurate measurement of noise power even when there is a delay between the generation of audio data RG and the detection of audio by a microphone.
Another object of the present invention is to improve the effect of the speech intelligibility improving process.

The present invention is a speech intelligibility improving method and a speech intelligibility improving system.
-Speech intelligibility improvement method The speech intelligibility improvement method according to the present invention is a multi-process in which a speech data sequence output by an audio data generation unit is multiplied by a predetermined gain and output to a speaker side, and also by a microphone. A first step of converting the detected audio signal into digital data and taking it in a predetermined length unit, monitoring an output start time and an output end time of the audio data string to the speaker side, and taking a predetermined length taken from the microphone A second step of monitoring the acquisition start time and the acquisition end time of the audio data sequence, and determining whether the audio data sequence of a predetermined length acquired from the microphone using each time is an audio data sequence of a noise section A third step, and a fourth step of detecting noise power using the voice data string of the noise section, Sometimes, a fifth step of detecting voice power based on the voice data string output from the voice data generating unit, and a sixth step of determining a gain to be multiplied by the voice data string using the noise power and the voice power. It is equipped with.

-Voice intelligibility improvement system The audio intelligibility improvement system of this invention is the audio | voice data generation part which produces | generates an audio | voice data sequence, and the audio | voice which receives the said audio | voice data sequence, converts audio | voice data into analog data, and outputs it to a speaker A signal output unit, a sound data capturing unit that converts the sound signal detected by the microphone into digital data and captures the data in units of a predetermined length, and an input start time and an input end time of the sound data string to the sound signal output unit; A time monitoring unit for monitoring a start time for capturing a predetermined length of audio data string and a time for capturing a predetermined length of audio data string in the sound data capturing unit, and a time monitoring unit for capturing the sound data using the respective times. A noise section determination unit for determining whether a predetermined length of the voice data string is a voice data string of a noise section; and the voice data capturing unit A noise power detection unit that detects noise power using the captured audio data sequence of the noise section; a correction value calculation unit that calculates a gain to be multiplied to the output audio data sequence using the noise power; A correction unit that multiplies the calculated gain by an audio data sequence generated by the audio data generation unit and inputs the result to the audio signal output unit, and when the audio data generation unit is not in the noise section, An audio power detection unit that detects audio power based on the audio data sequence to be output; and the correction value calculation unit compares the noise power with the audio power to determine a gain for multiplying the audio data sequence. .

  Another speech intelligibility improvement system according to the present invention includes a multi-process process for each of the speech data generation unit, the time monitoring unit, the noise section determination unit, the noise power detection unit, the correction value calculation unit, and the correction unit. It is realized by the CPU as one process.

  Another speech intelligibility improvement system according to the present invention includes a process in which each process of the voice data generation unit and the voice data output unit that outputs an input voice data string to another CPU is performed as one multi-process. The processing of the time monitoring unit, the noise section determination unit, the noise power detection unit, and the audio signal output unit is realized by another CPU as one multi-process.

  According to the present invention, the input start time and the input end time to the speaker side of the audio data sequence, and the acquisition start time and the acquisition end time of the audio data sequence of a predetermined length acquired from the microphone are monitored, and the respective times are monitored. Since it is determined whether or not the audio data string of a predetermined length captured from the microphone is an audio data string in the noise section, the voice is not detected as noise, and accurate noise power can be measured. .

  Further, according to the present invention, the gain for multiplying the audio data string to be input to the speaker is determined using the noise power of the noise section correctly, so that the speech intelligibility improvement process can be performed accurately and the noise level is reduced. However, the speaker output sound can be heard clearly.

Outline of the Present Invention FIG. 1 is a schematic explanatory view of the present invention, and FIG. 2 is an explanatory view showing whether the microphone detection data is sound data of a noise section. In FIG. 1, 10 is a general purpose CPU, 11 is an RG data storage unit, 12 is an RG generation unit, 13 is a GAE (Guidance Articulation Enhancement) unit, 14 is an RG playback unit, 15 is a sound driver, and 16 is D / A converter, 17 is a speaker, 18 is a microphone, 19 is an A / D converter, RG is an audio data sequence stored in the RG data storage unit, RG ′ is a correction value obtained by the GAE unit 13 MIC is a signal including the voice captured by the microphone 18 and ambient noise.

  The RG generation unit 12 reads out and generates the audio data string RG from the RG data storage unit 11, and inputs it to the noise separation unit 13b and the RG correction unit 13d. The RG correcting unit 13d multiplies the audio data sequence RG by the correction value calculated by the correction value calculating unit 13c, and inputs the audio data sequence RG 'whose volume has been corrected to the RG reproducing unit 14. The RG playback unit 14 temporarily stores the audio data sequence RG ′ and, when assigned with a process for data transfer, inputs the audio data sequence RG ′ to the sound driver 15 and inputs the input time to the input start time RGtime-S ( 2) and input to the noise section determination unit 13a. The sound driver 15 inputs the audio data string RG ′ to the D / A converter 16, and the input end time RGtime-E (see FIG. 2) when the audio data string disappears and the input ends is passed through the RG reproducing unit 14. And input to the noise section determination unit 13a. The D / A converter 16 inputs the analog-converted audio data sequence RG ′ to the speaker 17, and the speaker 17 outputs audio corresponding to the input audio data sequence RG ′. The microphone 18 takes in the MIC data (the voice of the output voice data string RG ′ and ambient noise), converts it into a digital signal by the A / D converter 19 and inputs it to the sound driver 15. The sound driver 15 stores the input MIC data in a buffer (storing unit) having a predetermined capacity (not shown), takes the time when the buffer is full as an end time MICtime-E (see FIG. 2), and determines a noise section determination unit. In 13a, the capture end time MICtime-E is input. The sound driver 15 inputs MIC data having a predetermined capacity to the noise section determination unit 13a and the noise separation unit 13b. The noise section determination unit 13a obtains the MIC data capturing start time MICtime-S (see FIG. 2) by subtracting the time MICbuftime corresponding to the buffer capacity from the capturing end time. The noise section determination unit 13a determines whether the captured end time MICtime-E is the input start time RGtime in order to determine whether the captured MIC data of a predetermined size (for example, S3 in FIG. 2) is noise section data (noise). It is determined whether it is older than -S or the input end time RGtime-E is older than the capture start time MICtime-S. If "YES" is determined, it is determined that the MIC data is data of a noise section, and the noise separation unit 13b calculates noise power using the voice data string MIC. Next, the correction value calculation unit 13c calculates a correction value for multiplying the audio data sequence RG using the noise power and the power of the audio data sequence, and the RG correction unit 13d multiplies the audio data sequence RG by the correction value, The corrected audio data string RG ′ is input to the speaker 17 via the RG reproducing unit 14, the sound driver 15, and the D / A converter 16. Further, if the determination is “NO”, the noise section determination unit 13a determines that the MIC data is not noise section data, and does not calculate noise power using the MIC data.

  From the above, noise power can be measured accurately even if there is a delay between the generation of the audio data sequence and the detection of the audio by the microphone, so that the effect of the audio intelligibility improvement process can be improved. it can.

Embodiment FIG. 3 is a configuration diagram of a speech intelligibility improvement system according to a first embodiment of the present invention.
During normal times, the RG generation unit 12 of the navigation device, for example, reads an audio data sequence from an RG data storage unit (not shown) when approaching an intersection, and generates an audio data sequence RG of guidance voice. The RG correction unit 13d inputs to the RG playback unit 14 an audio data sequence RG ′ whose volume has been corrected by multiplying the input audio data sequence RG by the correction value g calculated by the loudness compensation gain calculation unit 21 described later. The RG playback unit 14 stores the input audio data string in a built-in buffer, and reads the audio data string RG 'from the buffer by FIFO (first in first out) when the CPU permits the transfer to the sound driver. Input to the driver 15a. Further, the RG playback unit 14 measures the time when the input to the sound driver 15a is started (input start time RGtime-S) and notifies the noise section determination unit 13a.

  The sound driver 15a inputs the audio data string RG ′ input from the RG reproducing unit 14 to the D / A converter 16, and the input is completed when the input of all the audio data strings to the D / A converter 16 is completed. The time (input end time RGtime-E) is measured and notified to the noise section determination unit 13a via the RG playback unit 14. The D / A converter 16 converts the input audio data string RG ′ into analog data and inputs the analog data to the speaker 17. The microphone 18 collects the audio signal output from the speaker and ambient noise and inputs the collected sound signal to the A / D converter 19. The A / D converter 19 converts the input audio signal into digital data and inputs the digital data to the sound driver 15b as MIC data. The sound driver 15b stores the input MIC data in a built-in buffer 15c having a predetermined capacity. When the buffer 15c is full, the stored data of a predetermined size is input to the GAE unit 13, and the input The measured time (capture end time MICtime-E) is measured and input to the noise section determination unit 13a. Thereafter, the sound driver 15b starts storing the next MIC data in the buffer 15c, and inputs the stored data to the GAE unit 13 every time it is full, and inputs the capture end time MICtime-E to the noise section determining unit 13a. To do.

  The noise section determination unit 13a calculates the start time of MIC data import (capture start time MICtime-S) using the capture end time MICtime-E and the time (MICbuftime) according to the capacity of the buffer 15c (MICtime). -S = MICtime-E-MICbuftime), and using each time RGtime-S, RGtime-E, MICtime-S, and MICtime-E, it is determined whether the audio data taken from the buffer is noise section data (see FIG. 2). ). That is, the noise section determination unit 13a regards the period from RGtime-E to RGtime-S as the noise section and the period from RGtime-S to RGtime-E as the non-noise section, and inputs the MIC data capture end time MICtime-E. It is determined whether it is older than the start time RGtime-S or the input end time RGtime-E is older than the MIC data capture start time MICtime-S. If “YES”, the MIC data is data in the noise section. In the case of “NO”, it is determined that the data is in a non-noise section, and a switching signal for a noise section or a non-noise section is input to the switching unit 22. Of the MIC data S0 to S8 in FIG. 2, S2 to S4 are noise section data. The switching unit 22 is fixed to the contact point A, and is switched to the contact point B according to the signal of the non-noise section input by the noise section determination unit 13a. The sound driver 15b inputs the MIC data (noise signal) to the power calculation unit 24 through the fixed contact A and the audibility correction filter 23, and the power calculation unit 24 performs the square calculation of the amplitude of the input MIC data to obtain the noise power. Calculate

  The noise power averaging unit 25 obtains a moving average value of the latest N (N: constant) powers output from the power calculation unit 24 in the noise section, and uses the moving average value as noise power to the power storage unit 26. save. As a result, when the audio signal is output, the latest noise power in the immediately preceding noise section is stored in the power storage unit 26. In the present invention, the noise power in the non-noise section is regarded as the noise power stored in the power storage unit 26, and the noise power stored in the power storage unit 26 is input to the loudness compensation gain calculation unit 21.

  In parallel with the above, the audio data string RG output from the RG generation unit 12 is input to the audio power calculation unit 28 via the audibility correction filter 27. The audio power calculation unit 28 calculates the audio power by performing the square calculation of the amplitude of the input audio data string RG, inputs the audio power to the audio power averaging unit 29, and the audio power averaging unit 29 An average value of M (M: constant) audio powers input from the calculation unit 28 is calculated and input to the variable gain unit 31. The variable gain unit 31 multiplies the average audio power by the gain G and outputs the result. Note that the gain G set in the variable gain unit 31 assumes that the propagation characteristic from the input terminal of the speaker 17 to the microphone output terminal can be approximated only by the gain, and the characteristic identification unit 30 identifies and sets the gain G in advance. To do.

  The loudness compensation gain calculation unit 21 calculates a gain g at which the audio signal can be clearly heard regardless of the noise level based on the audio power input from the variable gain unit 31 and the noise power input from the power storage unit 26 in the non-noise section. The gain is determined by human loudness characteristics and input to the RG correction unit 13d. The RG correction unit 13d receives the gain g, multiplies the audio data string RG by the gain g, and outputs the result. The loudness compensation gain calculation unit 21 does not perform gain g determination control in the non-noise section.

  As described above, according to the present invention, since the MIC data S1 and S5 including the guidance voice shown in FIG. 2 are not used as the noise section data, the noise power in the noise section can be accurately measured and stored.

  FIG. 4 is a processing flow of the RG playback unit 14 and the sound driver 15a, and FIG. 5 is a processing flow of the noise section determination unit 13a, the correction value calculation unit 13c, and the sound driver 15b. Hereinafter, the noise section determination processing of the speech intelligibility improvement system will be described along these processing flows. However, the RG generation unit 12 reads the audio data string RG from the RG data storage unit 11 and inputs it to the RG correction unit 13d. The RG correction unit 13d multiplies the gain g calculated by the correction value calculation unit 13c, and uses the audio signal. It is assumed that a certain audio data string RG ′ is input to the RG reproducing unit 14.

  The RG playback unit 14 measures the input start time RGtime-S that is permitted by the CPU and starts to input the audio data string RG ′ to the sound driver 15a (step S401), and inputs the input start time RGtime-S to the noise section determination unit 13a. Is input (step S402).

  Next, the RG playback unit 14 passes the audio data string RG to the sound driver 15a (step S403), and the sound driver 15a performs a predetermined process on the received audio data string RG 'and inputs it to the D / A converter 16. The D / A converter 16 converts the digital audio data string RG ′ into an analog signal and inputs it to the speaker 17, and the speaker 17 outputs the audio signal (step S404).

  When the output of all the input audio data strings is completed, the sound driver 15a measures the time as the input end time RGtime-E (step S405), and inputs the input playback time RGtime-E to the RG playback unit 14. (Step S406), the RG reproducing unit 14 notifies the noise section determining unit 13a (Step S407). From the above, the noise section determination unit 13a regards the period from RGtime-E to RGtime-S as the noise section and the period from RGtime-S to RGtime-E as the non-noise section, and follows the processing flow of FIG. It is determined whether the data is noise section data. That is, it is determined whether the MIC data capture end time MICtime-E is older than the input start time RGtime-S or the input end time RGtime-E is older than the MIC data capture start time MICtime-S. Is determined that the MIC data is data in a noise section, and if “NO”, it is determined as a non-noise section.

  Hereinafter, the process of determining whether or not the MIC data captured from the microphone 18 is noise section data will be described with reference to FIG.

  The sound driver 15b sequentially stores the MIC data detected by the microphone 18 in the buffer 15c (step S501), determines whether the buffer 15c is full (step S502), and proceeds to step S503 if the buffer 15c is full. If not full, the processing of S501 to S502 is repeated.

  If it is determined in step S502 that the buffer 15c is full, the sound driver 15b measures the full time (capture end time) MICtime-E (step S503), and uses the capture end time MICtime-E as the noise interval. The MIC data input to the determination unit 13a and the MIC data stored in the buffer 15c are input to the GAE unit 13 (step S504).

  Next, the noise section determination unit 13a calculates the acquisition start time MICtime-S at which the acquisition starts by subtracting the time MICbuftime corresponding to the capacity of the buffer 15c from MICtime-E (step S505), and each time RGtime-S, RGtime- It is determined whether the audio data captured from the buffer 15c is noise section data using E, MICtime-S, and MICtime-E (step S506). That is, in step S506, the noise section determination unit 13a determines whether the capture end time MICtime-E is older than the input start time RGtime-S or whether the input end time RGtime-E is older than the capture start time MICtime-S. If “YES” is determined, the process proceeds to step S507. If “NO” is determined, the process proceeds to step S510.

  If “YES” is determined in step S506, the MIC data fetched from the buffer 15c is regarded as data in the noise section, and the noise separation unit 13b calculates the noise power using the MIC data and calculates the noise power. It inputs into the correction value calculation part 13c (step S507).

  Thereafter, the correction value calculation unit 13c calculates a correction value g by which the audio data string RG is multiplied using the input noise power, and inputs the calculated correction value g to the RG correction unit 13d (step S508). The RG correction unit 13d multiplies the input correction value g by the audio data string RG to correct the volume (step S509). Thereafter, the above process is repeated to update the correction value and determine the noise section.

  If “NO” is determined in step S506, it is determined that the voice data taken from the buffer is not noise section data, and the noise power is not calculated (step S510). Thereafter, the above process is repeated to update the correction value and determine the noise section. In this embodiment, the acquisition start time MICtime-S of the MIC data is calculated by subtracting the time (MICbuftime) corresponding to the buffer capacity from the acquisition end time MICtime-E. However, the present invention is not limited to this. The time (MICbuftime) corresponding to the buffer capacity is added to the capture start time MICtime-S to calculate the time when the MIC data capture is completed (capture end time MICtime-E) (MICtime-E = MICtime-S + MICbuftime) You may do it.

  As described above, according to the present embodiment, the input start time and input end time of the audio data string and the acquisition start time and the acquisition end time of the audio data string of a predetermined length acquired from the microphone are monitored, and the respective times are used. Therefore, since it is determined whether the audio data string of a predetermined length captured from the microphone is an audio data string in the noise section, the audio is not detected as noise (for example, the MIC data S1 and S5 in FIG. So that the exact noise power can be measured.

  In addition, according to the present embodiment, the gain for multiplying the audio data string input to the speaker by using the noise power of the noise section correctly is determined, so that the speech intelligibility improvement processing can be performed accurately, The speaker output sound can be heard clearly even under noise.

Modified Example FIG. 6 is a block diagram of a modified example of the present invention, and the same parts as those in FIG. The difference is that the voice clarity improvement processing by one general-purpose CPU is changed to the voice clarity improvement processing by another different general-purpose CPU.

  Another general-purpose CPU 40 includes a noise section determination unit 13a, a noise separation unit 13b, a correction value calculation unit 13c, and an RG correction unit 13d, and 13a to 13d are the same as the GAE unit 13 of the above embodiment. Process to improve speech intelligibility.

As described above, according to this modification, it is possible to obtain the same effects as in the above-described embodiment, no longer detect voice and noise, accurately measure noise power, and accurately perform speech intelligibility improvement processing. This can be performed, and the speaker output sound can be clearly heard even under noise.
In addition, according to this modification, the first general-purpose CPU that performs application processing and the second general-purpose CPU that performs GAE processing are used. Therefore, the first general-purpose CPU and the second general-purpose CPU are separated. It becomes possible, and the application and GAE can be detached.

It is a schematic explanatory drawing of this invention. It is explanatory drawing which shows whether microphone detection data is the audio | voice data of a noise area. It is a block diagram of the speech intelligibility improvement system of 1st Example of this invention. It is a processing flow of the RG playback unit 12 and the sound driver 15a. It is a processing flow of the noise area determination part 13a, the correction value calculation part 13c, and the sound driver 15b. It is a block diagram of the modification of this invention. This is a conventional speech intelligibility improvement system. It is a block diagram of the proposed speech intelligibility improvement system. It is an example which implement | achieves the audio | voice intelligibility improvement system of FIG. 8 with multi-process general purpose CPU and DSP (Digital Signal Processor). It is a figure explaining the cause which a delay produces on general purpose CPU. It is a time chart explaining that noise power cannot be measured correctly.

Explanation of symbols

10 General-purpose CPU
10a to 10d Application processed on general-purpose CPU 11 RG data storage unit 12 RG generation unit 13 Speech intelligibility improvement system 13a Noise section determination unit 13b Noise separation unit 13c Correction value calculation unit 13d RG correction unit 14 RG reproduction unit 15 Sound driver 15c buffer 23 auditory correction filter 24 power calculation unit 25 noise power averaging unit 26 power storage unit 27 auditory correction filter 28 audio power calculation unit 29 audio power averaging unit 30 characteristic identification unit 31 variable gain unit 40 another general-purpose CPU

Claims (18)

In the speech intelligibility improving method for controlling the gain of the audio signal using the noise power in the noise interval and the audio power in the non-noise interval as a noise interval in the interval where the audio signal is not output,
As a multi-process, the audio data sequence output from the audio data generation unit is multiplied by the gain and output to the speaker side, and the audio signal detected by the microphone is converted into digital data and captured in units of a predetermined length. First step,
A second step of monitoring an input start time and an input end time to the speaker side of the audio data sequence, and monitoring an acquisition start time and an acquisition end time of a predetermined length of audio data sequence acquired from the microphone;
A third step of determining whether or not the audio data string of a predetermined length captured from the microphone using each time is an audio data string of a noise section;
A fourth step of detecting noise power using the audio data string of the noise section;
A method for improving speech intelligibility, comprising: A fifth step of detecting voice power based on a voice data sequence output by the voice data generator when not in the noise section;
A sixth step of determining a gain by which the sound data string is multiplied using the noise power and the sound power;
The method for improving speech intelligibility according to claim 1. When the audio data string output by the audio data generation unit is input to the speaker via a sound driver, the second step includes:
Storing the time when the input of the audio data string to the sound driver is started as the input start time;
Storing the time when the input of the last audio data string to the speaker by the sound driver is ended as the input end time;
The speech intelligibility improving method according to claim 1 or 2, further comprising: The second step includes
Storing the audio data sequence captured from the microphone in a buffer of a predetermined capacity and storing the capture start time;
Outputting the entire audio data sequence stored when the buffer is full and the capture start time;
The speech intelligibility improving method according to claim 1 or 2, further comprising: The third step is a step of obtaining the capture end time by adding a certain time according to the capacity of the buffer to the output capture start time,
The speech intelligibility improving method according to claim 4, further comprising: The second step includes
Storing the audio data sequence captured from the microphone in a buffer having a predetermined capacity and storing the time as an end time when the buffer is full;
Outputting all stored audio data and the capture end time;
The speech intelligibility improving method according to claim 1 or 2, further comprising: The third step is a step of subtracting a certain time according to the capacity of the buffer from the output capture end time to obtain the capture start time;
The method for improving speech intelligibility according to claim 6. When the latest input start time is RGtime-S, the latest input end time is RGtime-E, the latest capture start time is MICtime-S, and the latest capture end time is MICtime-E.
Comparing the input start time RGtime-S and the capture end time MICtime-E;
Comparing the input end time RGtime-E and the capture start time MICtime-S;
If the capture end time MICtime-E is older than the latest input start time RGtime-S, or the latest input end time RGtime-E is older than the latest capture start time MICtime-S Determining that the audio data string of the predetermined length is an audio data string of the noise section;
The speech intelligibility improving method according to claim 1 or 2, characterized by comprising: In a speech intelligibility improving system that controls a gain of a speech signal using a noise power in a noise zone and a voice power in a non-noise zone as a noise zone where a voice signal is not output.
An audio data generator for generating an audio data sequence;
An audio signal output unit that converts the input audio data into analog data and outputs the analog data; and
An audio data capturing unit that converts an audio signal detected by a microphone into digital data and captures the data in units of a predetermined length;
Monitor the input start time and input end time of the audio data sequence to the audio signal output unit, the acquisition start time of the audio data sequence of a predetermined length and the acquisition end time of the audio data sequence of a predetermined length in the audio data acquisition unit A time monitoring unit to
A noise section determination unit that determines whether the predetermined length of the voice data sequence captured by the voice data capturing unit using each time is a noise section voice data sequence;
A noise power detection unit for detecting noise power using a voice data string of the noise section captured by the voice data capturing unit;
Using the noise power, a correction value calculation unit for calculating a gain to be multiplied with the output audio data string;
A correction unit that multiplies the calculated gain by the audio data sequence generated by the audio data generation unit and inputs the multiplication to the audio signal output unit;
A speech intelligibility improvement system characterized by comprising: A voice power detector that detects voice power based on a voice data sequence output by the voice data generator when not in the noise section;
The correction value calculating unit determines a gain by which the audio data string is multiplied using the noise power and the audio power;
The speech intelligibility improvement system according to claim 9. The time monitoring unit finishes inputting the last audio data sequence to the audio signal output unit, with the time when the audio data sequence output from the correction unit started to be input to the audio signal output unit as the input start time. Storing the time as the input end time;
The speech intelligibility improvement system according to claim 9 or 10. Furthermore, the audio data storage unit of the predetermined length that stores the audio data sequence captured by the audio data capturing unit,
The time monitoring unit outputs to the noise section determination unit the entire audio data sequence stored when the audio data storage unit is full and the start time of capturing the audio data sequence of the predetermined length.
The speech intelligibility improvement system according to claim 9 or 10. The noise section determination unit obtains the capture end time by adding a certain time according to the capacity of the audio data storage unit to the capture start time.
The speech intelligibility improvement system according to claim 12. Furthermore, the audio data storage unit of the predetermined length that stores the audio data sequence captured by the audio data capturing unit,
The time monitoring unit outputs the entire audio data sequence stored when the audio data storage unit is full and the capture end time to the noise section determination unit.
The speech intelligibility improvement system according to claim 9 or 10. The noise section determination unit obtains the capture start time by subtracting a certain time according to the capacity of the audio data storage unit from the capture end time.
The speech intelligibility improvement system according to claim 14. When the latest input start time is RGtime-S, the latest input end time is RGtime-E, the latest capture start time is MICtime-S, and the latest capture end time is MICtime-S.
The noise section determination unit compares the input start time RGtime-S and the capture end time MICtime-E, compares the latest input end time RGtime-E and the latest capture start time MICtime-S, and If the capture end time MICtime-E is older than the latest input start time RGtime-S, or the latest input end time RGtime-E is older than the latest capture start time MICtime-S , Determining that the audio data string of the predetermined length is an audio data string of the noise section,
The speech intelligibility improvement system according to claim 9 or 10.   Each process of the voice data generation unit, the time monitoring unit, the noise section determination unit, the noise power detection unit, the correction value calculation unit, and the correction unit is realized by the CPU as one multi-process. The speech intelligibility improvement system according to claim 9.   Each process of the audio data generation unit and the audio data output unit that outputs the input audio data string to another CPU is realized as a multi-process by the CPU, and a time monitoring unit and a noise section determination unit 10. The speech intelligibility improvement system according to claim 9, wherein each process of the noise power detection unit and the audio signal output unit is realized by another CPU as one multi-process.

Images