JP5169297B2 - Sound processing apparatus and program - Google Patents

Description

  The present invention relates to a technique for determining the type of sound.

Conventionally, a technique for distinguishing sound such as recorded sound by a sound collecting device (hereinafter referred to as “input sound”) into a voice section and a non-voice section has been proposed. For example, Patent Document 1 discloses a technique for detecting a sound based on the intensity of a component belonging to a predetermined frequency band in an input sound.
JP 2000-132177 A

  However, in the technique of Patent Document 1, it is difficult to detect an unvoiced sound having an acoustic characteristic similar to that of non-voice with high accuracy. Therefore, even in a section where voice (voiced sound and unvoiced sound) is actually continuing, the section of unvoiced sound may be erroneously determined as non-speech and the section determined as speech may be interrupted. In view of the above circumstances, an object of the present invention is to discriminate unvoiced sound with high accuracy.

In order to solve the above problems, a sound processing apparatus according to the first aspect of the present invention includes a modulation spectrum specifying unit that specifies a modulation spectrum for each unit section of an input sound, and a modulation frequency of the modulation spectrum is a predetermined frequency. First index calculation means for calculating a first index value corresponding to the intensity of the component belonging to the range (for example, the index calculation unit 42 in FIG. 2) and a second index value corresponding to the number of zero crossings of the input sound for each unit section Based on the magnitude of the first index value and the first threshold value, the second index calculation means (for example, the index calculation unit 44 in FIG. 2) for calculating the input sound and whether or not the input sound of each unit section is speech Based on the second index value , the voice determination means, and whether or not the input sound of each unit section is an unvoiced sound, based on the magnitude of the second threshold value and the first index value different from the first threshold value Unvoiced sound determination means.

  In the above configuration, since it is determined whether or not the input sound of each unit section is an unvoiced sound based on the intensity of the component whose modulation frequency falls within a predetermined range in the modulation spectrum, the frequency spectrum of the input sound is used. It is possible to discriminate the unvoiced sound with higher accuracy than the technology to do. Further, since the second index value corresponding to the number of zero crossings of the input sound is used for the determination of the unvoiced sound by the unvoiced sound determination means, the unvoiced sound can be determined with higher accuracy than the configuration using only the first index value. . For example, a voiced sound tends to have a smaller number of zero crossings than an unvoiced sound. Therefore, the unvoiced sound can be distinguished from a voiced sound with high accuracy by using the second index value.

  The sound processing apparatus according to a preferred aspect of the present invention is a third index calculation means for calculating a third index value corresponding to the flatness of the frequency spectrum of the input sound for each unit section (for example, the index calculation unit 46 in FIG. 2). The unvoiced sound determination means determines whether the input sound of each unit section is an unvoiced sound based on the first index value, the second index value, and the third index value. In the above aspect, since the third index value corresponding to the flatness of the frequency spectrum of the input sound is used for the determination of the unvoiced sound by the unvoiced sound determination means, only the first index value and the second index value are used. It is possible to discriminate unvoiced sound with high accuracy by comparison. For example, since sounds such as voiced sounds and environmental sounds (for example, push tones) tend to have lower frequency spectrum flatness than unvoiced sounds, the third index value is used to convert unvoiced sounds into voiced sounds and environmental sounds. And can be distinguished with high accuracy.

The sound processing apparatus according to the second aspect of the present invention includes a modulation spectrum specifying unit that specifies a modulation spectrum for each unit section of an input sound, and a modulation frequency corresponding to the intensity of a component that belongs to a predetermined range in the modulation spectrum. First index calculation means for calculating the first index value, third index calculation means for calculating a third index value corresponding to the flatness of the frequency spectrum of the input sound for each unit section, and input sound in each unit section The sound determination means for determining whether or not the sound is based on the magnitude of the first index value and the first threshold, and whether or not the input sound of each unit section is an unvoiced sound are different from the first threshold. An unvoiced sound determining means for determining based on the magnitude of the second threshold value and the first index value and the third index value is provided.

  In the second aspect, since it is determined whether or not the input sound of each unit section is an unvoiced sound based on the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum, the frequency spectrum of the input sound is It is possible to discriminate unvoiced sound with higher accuracy than the technology used. Further, since the third index value corresponding to the flatness of the frequency spectrum of the input sound is used for the determination of the unvoiced sound by the unvoiced sound determination means, the unvoiced sound is discriminated with higher accuracy than the configuration using only the first index value. Is done. For example, since sounds such as voiced sounds and environmental sounds (for example, push tones) tend to have lower frequency spectrum flatness than unvoiced sounds, the third index value is used to convert unvoiced sounds into voiced sounds and environmental sounds. And can be distinguished with high accuracy.

  Note that the unvoiced sound is sound that does not accompany vocal cord vibration (voice other than voiced sound). Unvoiced consonants and unvoiced voiced sounds (vowels) correspond to unvoiced sounds. Devoicing is a phenomenon in which a voiced sound that should be originally uttered with vocal cord vibration is uttered without any vocal cord vibration under some conditions.

  Further, “determining whether or not the input sound is an unvoiced sound based on the index value” means that the determination result of whether or not the input sound is an unvoiced sound changes depending on the magnitude of the index value. means. The specific method of determination by the unvoiced sound determination means is appropriately selected according to the definition of each index value as exemplified below.

  In the sound processing apparatus according to the first and second aspects, the first index value is defined such that the first index value decreases as the intensity of a component having a modulation frequency within a predetermined range in the modulation spectrum increases. In this case, for example, the content of determination by the unvoiced sound determination unit is selected such that the smaller the first index value, the higher the possibility that the input sound is determined to be unvoiced sound. For example, when an index value other than the first index value satisfies a condition for determining the input sound as an unvoiced sound, the unvoiced sound determination means determines that the first index value has a predetermined threshold value (for example, the threshold value T1B in FIG. 9). The input sound is determined to be an unvoiced sound if the input sound is lower, and the input sound is determined not to be an unvoiced sound if the first index value exceeds the threshold value. On the other hand, when the first index value is defined such that the first index value increases as the intensity of a component having a modulation frequency within a predetermined range in the modulation spectrum increases, for example, the larger the first index value, The content of the determination by the unvoiced sound determination means is selected so that the possibility that the input sound is determined to be an unvoiced sound increases. For example, when an index value other than the first index value satisfies a condition for determining an input sound as an unvoiced sound, the unvoiced sound determination means determines that the input sound is an unvoiced sound when the first index value exceeds a predetermined threshold value. It determines, and when a 1st index value is less than the said threshold value, it determines with an input sound not being an unvoiced sound.

  In the sound processing device according to the first aspect, when the second index value is defined such that the second index value increases as the number of zero crossings of the input sound increases, for example, the larger the second index value, The content of the determination by the unvoiced sound determination means is selected so that the possibility that the input sound is determined to be an unvoiced sound increases. For example, when an index value other than the second index value satisfies a condition for determining the input sound as an unvoiced sound, the unvoiced sound determination means determines that the second index value has a predetermined threshold (for example, the threshold T2 in FIG. 9). When it exceeds, the input sound is determined as an unvoiced sound, and when the second index value is below the threshold, it is determined that the input sound is not an unvoiced sound. On the other hand, when the second index value is defined so that the second index value decreases as the number of zero crossings of the input sound increases (for example, when the reciprocal of the number of zero crossings is calculated as the second index value), for example, The content of the determination by the unvoiced sound determination means is selected so that the possibility that the input sound is determined to be unvoiced increases as the second index value decreases. For example, when an index value other than the second index value satisfies a condition for determining an input sound as an unvoiced sound, the unvoiced sound determination means determines that the input sound is an unvoiced sound when the second index value is lower than a predetermined threshold value. It determines, and when a 2nd parameter | index value exceeds the said threshold value, it determines with an input sound not being an unvoiced sound.

  In the sound processing device according to the second aspect, when the third index value is defined such that the third index value decreases as the flatness of the frequency spectrum of the input sound increases, for example, the smaller the third index value, The content of the determination by the unvoiced sound determination means is selected so that the possibility that the input sound is determined to be an unvoiced sound increases. For example, when an index value other than the third index value satisfies a condition for determining an input sound as an unvoiced sound, the unvoiced sound determination means determines that the third index value has a predetermined threshold (for example, a threshold T3 in FIG. 9). The input sound is determined to be an unvoiced sound when lower than the threshold, and the input sound is determined not to be an unvoiced sound when the third index value exceeds the threshold value. On the other hand, when the third index value is defined such that the third index value increases as the flatness of the frequency spectrum of the input sound increases, for example, the input sound is determined to be unvoiced as the third index value increases. The content of determination by the unvoiced sound determination means is selected so that the possibility increases. For example, when an index value other than the third index value satisfies a condition for determining the input sound as an unvoiced sound, the unvoiced sound determination means determines that the input sound is an unvoiced sound when the third index value exceeds a predetermined threshold value. When the third index value is below the threshold, it is determined that the input sound is not an unvoiced sound.

  In the configuration in which the first index value is calculated so that the first index value decreases as the intensity of the component whose modulation frequency falls within the predetermined range in the modulation spectrum increases, the speech determination means has the first index value as the first index value. An input sound in a unit section that falls below a threshold value (for example, threshold value T1A in FIG. 6) is determined as speech, and the unvoiced sound determination means uses unvoiced sound as an input sound in a unit section in which the first index value falls below a second threshold value that is greater than the first threshold value. Is determined. Further, in the configuration in which the first index value is calculated so that the first index value increases as the intensity of the component whose modulation frequency belongs to the predetermined range in the modulation spectrum is higher, the sound determination means has the first index value The input sound in the unit section exceeding the first threshold is determined as speech, and the unvoiced sound determining means determines the input sound in the unit section in which the first index value exceeds the second threshold smaller than the first threshold as unvoiced sound. According to the above aspects, it is possible to distinguish voiced and unvoiced sounds with high accuracy.

  In the specific example of the sound processing device according to the first aspect and the second aspect, the sound that performs different processing on the input sound of the unit section determined by the unvoiced sound determination unit as the unvoiced sound and the input sound of the other unit sections Processing means are installed. In another specific example, sound processing means for performing different processing on the input sound of the unit section determined by the sound determination means as the sound and the input sound of the unit section determined by the unvoiced sound determination means as the unvoiced sound is installed. Is done. In the above configuration, it is possible to generate sound having a desired characteristic by executing appropriate processing according to the type of input sound (voice / non-voice or voiced / unvoiced sound).

The sound processing apparatus according to all of the above aspects is realized by hardware (electronic circuit) such as a DSP (Digital Signal Processor) dedicated to processing of input sound, or a general purpose such as a CPU (Central Processing Unit). This is also realized by cooperation between the arithmetic processing unit and the program. The program according to the first aspect includes a modulation spectrum specifying process for specifying a modulation spectrum for each unit section of the input sound, and a first index value corresponding to the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum. A first index calculation process for calculating; a second index calculation process for calculating a second index value corresponding to the number of zero crossings of the input sound for each unit section; and whether or not the input sound of each unit section is a voice. The second threshold value and the first index, which are different from the first threshold value, are determined based on the magnitude of the first index value and the first threshold value, and whether or not the input sound of each unit section is an unvoiced sound. The computer performs unvoiced sound determination processing that is determined based on the magnitude of the value and the second index value. The program according to the second aspect includes a modulation spectrum specifying process for specifying a modulation spectrum for each unit section of the input sound, and a first index value corresponding to the intensity of a component whose modulation frequency is within a predetermined range of the modulation spectrum. A first index calculation process for calculating, a third index calculation process for calculating a third index value corresponding to the flatness of the frequency spectrum of the input sound for each unit section, and whether or not the input sound of each unit section is speech The second threshold value and the second threshold value that are different from the first threshold value are voice determination processing that determines whether or not the input sound of each unit section is an unvoiced sound , based on the magnitude of the first index value and the first threshold value. The computer performs unvoiced sound determination processing that is determined based on the magnitude of the 1 index value and the third index value. According to the program of this invention, the effect | action and effect similar to the sound processing apparatus which concern on each above aspect are show | played. The program of the present invention is provided to a user in a form stored in a computer-readable recording medium and installed in the computer, or provided from a server device in a form of distribution via a communication network and installed in the computer. Is done.

  FIG. 1 is a block diagram of a remote conference system according to an embodiment of the present invention. The remote conference system 100 is a system in which a plurality of users U (conference participants) exchange voices with each other in geographically separated spaces R1 and R2. In each space R (R1, R2), a sound collecting device 12, a sound processing device 14, a sound processing device 16, and a sound emitting device 18 are installed.

  The sound collection device 12 is a device (microphone) that generates an acoustic signal SIN representing the waveform of the input sound VIN existing in the space R. Each sound processing device 14 in the space R1 and the space R2 generates an output signal SOUT from the acoustic signal SIN and transmits the output signal SOUT to the other sound processing device 16 in the space R1 and the space R2. The sound processing device 16 amplifies the output signal SOUT and outputs it to the sound emitting device 18. The sound emitting device 18 is a device (speaker) that emits sound waves according to the amplified output signal SOUT supplied from the sound processing device 16. With the above configuration, the utterance sound of each user U in the space R1 is output from the sound emitting device 18 in the space R2, and the utterance sound of each user U in the space R2 is output from the sound emitting device 18 in the space R1. Is output.

  FIG. 2 is a block diagram showing a configuration of the sound processing device 14 installed in each of the space R1 and the space R2. As shown in FIG. 2, the sound processing device 14 includes a control device 22 and a storage device 24. The control device 22 is an arithmetic processing device that functions as each element in FIG. 2 by executing a program. 2 are also realized by an electronic circuit such as a DSP. The storage device 24 stores a program executed by the control device 22 and various data used by the control device 22. A known storage medium such as a semiconductor storage device or a magnetic storage device is arbitrarily used as the storage device 24.

  The control device 22 determines the type of the input sound VIN for each of a plurality of sections (hereinafter referred to as “unit sections”) obtained by dividing the acoustic signal SIN (input sound VIN) supplied from the sound collection device 12 along the time axis (hereinafter referred to as “unit section”) A function of discriminating voiced / unvoiced sound / non-speech) and a function of generating an output signal SOUT by executing processing according to the discrimination result on the acoustic signal SIN are realized. A voiced sound is a sound (speech sound) accompanied by vibration of a vocal cord. Unvoiced sound is sound that does not involve vocal cord vibration. On the other hand, non-voice is sound (noise) other than voice. Various background noises (for example, operating sounds of air conditioning equipment) and various environmental sounds (for example, ringtones of mobile phones and door opening / closing sounds) correspond to non-speech.

  The frequency analysis unit 32 in FIG. 2 performs a frequency analysis including a Fourier transform (for example, FFT (Fast Fourier Transform)) on the acoustic signal SIN, thereby a plurality of frames obtained by dividing the acoustic signal SIN along the time axis. A frequency spectrum (logarithmic spectrum) S0 is calculated for each of. Each frame is set to a sufficiently short time length compared to the unit interval.

  The modulation spectrum specifying unit 34 specifies the modulation spectrum MS of the acoustic signal SIN (input sound VIN). The modulation spectrum MS corresponds to a result obtained by performing a Fourier transform on a temporal variation (hereinafter referred to as “time locus”) of a component belonging to a specific frequency band in the frequency spectrum S0 of the acoustic signal SIN.

  FIG. 3 is a block diagram of the modulation spectrum specifying unit 34. FIG. 4 is a conceptual diagram for explaining processing by the modulation spectrum specifying unit 34. Part (A) of FIG. 4 shows a spectrogram SP in which the frequency spectrum S0 specified for each frame by the frequency analysis unit 32 is arranged in time series.

  As shown in FIG. 3, the modulation spectrum specifying unit 34 includes a component extracting unit 342 and a frequency analyzing unit 344. As shown in part (A) and part (B) of FIG. 4, the component extraction unit 342 extracts the time trajectory ST of the intensity (energy) of the component belonging to the specific frequency band ω from the spectrogram SP. More specifically, the component extraction unit 342 calculates the intensity of the component belonging to the frequency band ω in the frequency spectrum S0 of each frame, and arranges the intensity of the frequency spectrum S0 in time series for a plurality of frames. Generate ST. In the frequency band ω, the frequency characteristic (modulation spectrum MS) of the time trajectory ST when the input sound VIN is speech and the frequency characteristic of the time trajectory ST when the input sound VIN is non-speech are significantly different. Selected experimentally or statistically. For example, the frequency band ω is selected in a range from 10 Hz (more preferably 50 Hz) to 800 Hz. A configuration in which the component extraction unit 342 extracts the time series of the intensity of one frequency component in each frequency spectrum S0 as the time locus ST is also employed.

  As shown in part (B) and part (C) of FIG. 4, the frequency analysis unit 344 in FIG. 3 performs a Fourier transform on the time trajectory ST, thereby dividing the time trajectory ST along the time axis. The modulation spectrum MS is calculated for each of the plurality of unit intervals TU. The unit section TU is a period of a predetermined time length (for example, about 1 second) composed of a plurality of frames. In the present embodiment, a configuration in which the unit sections TU do not overlap is illustrated for convenience, but a configuration in which the adjacent unit sections TU partially overlap is also employed.

  FIG. 5 shows modulation spectra MS of plural kinds of sounds (voiced / unvoiced / non-voice). Part (A) of FIG. 5 is a modulation spectrum MS of voiced sound (speech uttered as “Aiueo”). Part (B) of FIG. 5 is a modulation spectrum MS of sound rich in unvoiced sound. More specifically, part (B) of FIG. 5 is a modulation spectrum MS of a voice when “sashisoseso” is uttered so that “sa” and “shi” are almost silent. Further, part (C) and part (D) of FIG. 5 are non-voice modulation spectrum MS. More specifically, part (C) of FIG. 5 is a modulation spectrum MS of white noise (background noise), and part (D) of FIG. 5 is a modulation spectrum MS of a push tone (environmental sound) of the telephone.

  In the modulation spectrum MS of a normal human speech sound (that is, speech), as shown in part (C) of FIG. 4, the intensity is maximum at a modulation frequency of about 4 Hz corresponding to the frequency at which the syllable is switched during speech. In many cases. More specifically, in the modulation spectrum MS of voice (voiced sound and unvoiced sound) (part (A) and part (B) in FIG. 5), the intensity tends to be high within a range where the modulation frequency is below 10 Hz. On the other hand, in many non-speech modulation spectra MS (part (C) and part (D) in FIG. 5), the intensity of components in the range where the modulation frequency exceeds 10 Hz tends to be high.

  In consideration of the above difference in characteristics, in the present embodiment, the intensity of a component whose modulation frequency belongs to a predetermined range (hereinafter referred to as “determination target range”) A in the modulation spectrum MS specified by the modulation spectrum specifying unit 34 is calculated. Used to determine the type of input sound VIN. The determination target range A is set to a range in which the difference in intensity of the modulation spectrum MS is significant between voice and non-voice. For example, a range of 10 Hz or less (more preferably, a range of 2 Hz to 8 Hz) is appropriate as the determination target range A.

2 calculates an index value D1 corresponding to the intensity (energy) of a component belonging to the determination target range A for the modulation spectrum MS specified by the modulation spectrum specifying unit 34 for each unit section TU. More specifically, the index calculation unit 42 firstly calculates the intensity of the component whose modulation frequency belongs to the determination target range A in the modulation spectrum MS (for example, the added value or average of the intensity at each modulation frequency in the determination target range A). Value) L1 and the intensity of the modulation spectrum MS over the entire range of the modulation frequency (addition value or average value of the intensity at all modulation frequencies) L2. Second, the index calculation unit 42 calculates the index value D1 based on the following arithmetic expression (A) including the relative ratio (L1 / L2) between the intensity L1 and the intensity L2.
D1 = 1- (L1 / L2) (A)
As understood from the content of the arithmetic expression (A), the index value increases as the intensity L1 of the component within the determination target range A in the modulation spectrum MS increases (that is, the possibility that the input sound VIN is a voice). D1 is a small numerical value. Therefore, the index value D1 is an index for determining whether the input sound VIN is voice or non-voice. Further, since the determination target range A includes abundant frequency components at which syllables are switched during utterance, the index value D1 determines whether or not the input sound VIN includes a rhythm peculiar to speech (speech rhythm). It is also grasped as an index for

  2 calculates an index value D2 corresponding to the number of zero crossings of the input sound VIN (acoustic signal SIN) for each unit interval TU. The number of zero crossings (the number of zero crossings) is the number of times the sign of the intensity of the acoustic signal SIN is inverted (the number of times the intensity of the acoustic signal SIN changes across zero). The index calculation unit 44 calculates, for example, the total value of the number of zero crossings in the unit section TU or the average value of the number of zero crossings for each predetermined time in the unit section TU as the index value D2. Therefore, the larger the number of zero crossings of the acoustic signal SIN, the larger the index value D2.

  The index calculation unit 46 in FIG. 2 calculates an index value D3 corresponding to the flatness of the shape of the frequency spectrum S0 specified by the frequency analysis unit 32 for each unit interval TU. For example, first, the index calculation unit 46 divides the average frequency spectrum S0 over a plurality of frames in the unit section TU into a plurality of frequency bands, and the intensity (average value of energy) of the components in each frequency band. Is calculated. Secondly, the index calculation unit 46 calculates the relative ratio between the maximum value Emax and the minimum value Emin among the intensities of the plurality of frequency bands as the index value D3 (D3 = Emax / Emin) for each unit section TU. Therefore, the higher the flatness of the frequency spectrum S0 (that is, the smaller the difference between the maximum value Emax and the minimum value Emin), the smaller the index value D3. The index value D3 may be calculated by averaging the relative ratio between the maximum value Emax and the minimum value Emin in the frequency spectrum S0 of each frame for a plurality of frames in the unit interval TU.

  The determination unit 50 in FIG. 2 determines the type (voiced / unvoiced / non-voice) of the input sound VIN based on the index value D1, the index value D2, and the index value D3. The determination unit 50 according to this embodiment includes a voice determination unit 52 and an unvoiced sound determination unit 54. The voice determination unit 52 determines whether or not the input sound VIN of each unit section TU is voiced (particularly voiced) based on the index value D1. FIG. 6 is a flowchart showing a specific operation of the voice determination unit 52. The process of FIG. 6 is executed every time the index value D1 is calculated for one unit section TU.

  The voice determination unit 52 determines whether or not the index value D1 is below the threshold value T1A (step SA1). The threshold value T1A is set experimentally or statistically so that the voiced sound index value D1 is lower than the threshold value T1A and the non-speech index value D1 is higher than the threshold value T1A. If the result of step SA1 is negative, the voice determination unit 52 determines that the input sound VIN of the unit interval TU that is the target of the current process is not a voiced sound (step SA2). On the other hand, if the result of step SA1 is affirmative, the sound determination unit 52 determines that the input sound VIN of the current unit section TU is a voiced sound (step SA3). In step SA3, the voice determination unit 52 generates identification data d that designates a voiced sound and outputs the identification data d to the sound processing unit 60.

  Incidentally, as shown in FIG. 5, the characteristics of the modulation spectrum MS are different between voiced sound (part (A)) and unvoiced sound (part (B)). More specifically, the modulation spectrum MS of the unvoiced sound tends to have a maximum intensity at the modulation frequency on the high frequency side as compared with the modulation spectrum MS of the voiced sound. That is, for the unvoiced sound, since the intensity L1 of the component within the determination target range A of the modulation spectrum MS is lower than that of the voiced sound, the index value D1 calculated from the modulation spectrum MS of the unvoiced sound is the index value D1 of the voiced sound. In many cases, it becomes a large numerical value. Therefore, the unvoiced sound is not classified into speech only by comparing the index value D1 and the threshold value T1A in step SA1 in FIG. Therefore, the unvoiced sound determination unit 54 determines whether or not the input sound VIN of each unit section TU is an unvoiced sound.

  FIG. 7 shows a time waveform of the acoustic signal SIN. FIG. 8 shows the frequency spectrum S0 of the acoustic signal SIN. The part (A) in each of FIGS. 7 and 8 is a characteristic of voiced sound in which the modulation spectrum MS is illustrated in the part (A) of FIG. Similarly, part (B) in each of FIGS. 7 and 8 is a characteristic of unvoiced sound (part (B) in FIG. 5). Further, the part (C) in each of FIGS. 7 and 8 is a characteristic of white noise in which the modulation spectrum MS is illustrated in the part (C) of FIG. 5, and the part (D) in each of FIGS. FIG. 5D shows the characteristics of a push tone whose modulation spectrum MS is illustrated in part (D) of FIG.

  As can be understood from the comparison of part (A) to part (D) in FIG. 7, unvoiced sound (part (B)) and non-speech (part (C) and part (D)) Compared with)), the number of zero crossings per unit time is larger. Therefore, the index value D2 calculated for the unit interval TU of unvoiced sound or non-speech (white noise or push tone) is larger than the index value D2 of the unit interval TU of voiced sound.

  Further, as understood from the comparison of the parts (A) to (D) in FIG. 8, the unvoiced sound of the part (B) and the white noise of the part (C) are the voiced sound and the part (D ), The shape of the frequency spectrum S0 is flat (that is, there is little difference in intensity). Therefore, the index value D3 calculated for the unvoiced sound or white noise unit interval TU is smaller than the index value D3 of the voiced sound or push tone unit interval TU.

Further, as can be understood from the comparison of the parts (A) to (D) in FIG. 5, the voiced sound of the part (A) and the unvoiced sound (ie, the voice) of the part (B) are the white noise of the part (C). Compared with the push tone (that is, non-speech) of the portion (D), the intensity within the determination target range A is high in the modulation spectrum MS. Therefore, the index value D1 calculated for the unvoiced or voiced unit interval TU is smaller than the index value D1 of the non-speech unit interval TU.

  In consideration of the relationship between each index value D (D1 to D3) and the type of input sound VIN, the unvoiced sound determination unit 54 indicates whether or not the input sound VIN of each unit section TU is an unvoiced sound. The determination is made based on the value D1 or the index value D3. FIG. 9 is a flowchart showing a specific operation of the unvoiced sound determination unit 54. The process of FIG. 9 is executed each time the index value D1 to the index value D3 are calculated for one unit section TU.

  The unvoiced sound determination unit 54 determines whether or not the index value D2 calculated by the index calculation unit 44 exceeds the threshold value T2 (step SB1). The threshold value T2 is selected experimentally or statistically so that the unvoiced and non-speech index value D2 exceeds the threshold value T2 and the voiced sound index value D2 falls below the threshold value T2. If the result of step SB1 is negative, the unvoiced sound determination unit 54 determines that the input sound VIN of the current unit interval TU is not an unvoiced sound (step SB2).

  If the result of step SB1 is affirmative, the unvoiced sound determination unit 54 determines whether or not the index value D3 calculated by the index calculation unit 46 is below the threshold T3 (step SB3). The threshold value T3 is experimentally or statistically selected so that the index value D3 of unvoiced sound and background noise (white noise) is lower than the threshold value T3, and the index value D3 of voiced sound and environmental sound (push tone) is higher than the threshold value T3. The If the result of step SB3 is negative, the unvoiced sound determination unit 54 determines that the input sound VIN of the current unit section TU is not an unvoiced sound (step SB2).

  If the result of step SB3 is affirmative, the unvoiced sound determination unit 54 determines whether or not the index value D1 calculated by the index calculation unit 42 is below the threshold value T1B (step SB4). The threshold value T1B is selected experimentally or statistically so that the unvoiced and voiced sound index value D1 is lower than the threshold value T1B and the non-voice index value D1 is higher than the threshold value T1B. As described above with reference to FIG. 5, the modulation frequency having the maximum intensity in the modulation spectrum MS of unvoiced sound is on the high frequency side compared to the modulation spectrum MS of voiced sound. Therefore, the threshold value T1B set so as to exceed the index value D1 of both the unvoiced sound and the voiced sound is a larger numerical value than the threshold value T1A used by the voice determination unit 52 to discriminate the voiced sound.

  If the result of step SB4 is affirmative, the unvoiced sound determination unit 54 determines that the input sound VIN of the current unit section TU is an unvoiced sound (step SB5). In step SB5, the unvoiced sound determination unit 54 generates identification data d designating the unvoiced sound and outputs the identification data d to the sound processing unit 60. On the other hand, if the result of step SB4 is negative, the unvoiced sound determination unit 54 determines that the input sound VIN of the current unit section TU is not an unvoiced sound (step SB2). The unit section TU is classified as non-speech, which is determined that the voice determination unit 52 is not a voiced sound (step SA2) and the unvoiced sound determination unit 54 is determined not to be a voiceless sound (step SB2). That is, the identification data d designating non-speech is output from the determination unit 50 to the sound processing unit 60.

  The sound processing unit 60 in FIG. 2 performs processing on the acoustic signal SIN in the unit section TU according to the result of the determination section 50 (the sound determination section 52 and the unvoiced sound determination section 54) determining the unit section TU. Generates an output signal SOUT. Specific processing of the sound processing unit 60 will be described in detail below.

  First, the sound processing unit 60 separates the acoustic signal SIN of the unit interval TU determined by the voice determination unit 52 as a voiced sound and the acoustic signal SIN of the unit interval TU determined by the unvoiced sound determination unit 54 as an unvoiced sound. Execute the process. For example, the sound processing unit 60 suppresses the high-frequency component of the acoustic signal SIN for the unvoiced sound unit interval TU by the low-pass filter process, but does not perform the filter process for the voiced sound unit interval TU.

  Secondly, the sound processing unit 60 performs separate processing on the acoustic signal SIN in the unit section TU of voice (voiced sound or unvoiced sound) and the acoustic signal SIN in the non-sound unit section TU. For example, the sound processing unit 60 outputs the acoustic signal SIN as the output signal SOUT for the unit interval TU determined by the voice determination unit 52 as a voiced sound and the unit interval TU determined by the unvoiced sound determination unit 54 as an unvoiced sound. For the unit section TU determined to be speech, the output signal SOUT with the volume set to zero is output (that is, the acoustic signal SIN is not output). Therefore, in each of the space R1 and the space R2, the non-speech of the input sound VIN in the other space R is removed, and only the sound that the user originally needs to listen to passes through the sound processing device 16. And emitted from the sound emitting device 18. Furthermore, since the high frequency component of the unvoiced sound is suppressed, a sound that is easy for the user to listen to is emitted from the sound emitting device 18.

  In the configuration in which only the unit interval TU determined by the speech determination unit 52 as a voiced sound is output as the output signal SOUT, the unvoiced sound is processed as non-speech, so the sound radiated according to the output signal SOUT is a unit of unvoiced sound. Breaks off in section TU. In this embodiment, not only the voice determination unit 52 determines the unit interval TU of voiced sound, but also the unvoiced sound determination unit 54 determines the unit interval TU of unvoiced sound based on the index value D1 to the index value D3. Both voiced and unvoiced sounds of VIN can be emitted as output signal SOUT. Therefore, there is an advantage that the section corresponding to the unvoiced sound of the input sound VIN among the sounds radiated from the output signal SOUT is prevented.

  Further, since the voice (voiced sound and unvoiced sound) and non-voice are distinguished from each other based on the intensity L1 of the component within the determination target range A of the modulation spectrum MS (whether or not there is an utterance rhythm), the frequency spectrum of the input sound VIN It is possible to discriminate between speech and non-speech with high accuracy compared to the technique of Patent Document 1 using When the volume of non-voice is high, the intensity of the modulation spectrum MS is high over the entire band of the modulation frequency. Therefore, in a configuration in which speech and non-speech are distinguished based only on the intensity L1 within the determination target range A of the modulation spectrum MS, non-speech with a high volume may be erroneously determined as speech. In this embodiment, since the relative ratio between the intensity L1 within the determination target range A and the intensity L2 over the entire range of the modulation frequency is used for the determination by the determination unit 50, even if the volume of non-speech is high There is an advantage that non-voice can be accurately determined.

<Modification>
Various modifications are added to the above embodiment. An example of a specific modification is as follows. Two or more aspects may be arbitrarily selected from the following examples and combined.

(1) Modification 1
In the above embodiment, the index value D1 corresponding to the modulation spectrum MS, the index value D2 corresponding to the number of zero crossings, and the index value D3 corresponding to the flatness of the frequency spectrum S0 are used for discrimination of the input sound VIN. A configuration is also adopted in which the input sound VIN is discriminated based on one of the value D2 and the index value D3 and the index value D1. For example, in an aspect in which the index value D1 and the index value D2 are used, the index calculation unit 46 in FIG. 2 and step SB3 in FIG. 9 are omitted. Further, in an aspect in which the index value D1 and the index value D3 are used, the index calculation unit 44 in FIG. 2 and step SB1 in FIG. 9 are omitted. However, according to the configuration using three types of index values (D1 to D3) as in the configuration of FIG. 2, the input sound VIN is discriminated more accurately than the configuration using only two types of index values D. There is an advantage that you can.

(2) Modification 2
In the above embodiment, since the threshold value T1A is set so as to exceed the voiced sound index value D1 and lower than the unvoiced sound index value D1, the sound determination unit 52 is a means for determining the voiced sound from unvoiced sound or non-voiced sound. Function. However, by setting the threshold value T1A so as to exceed the index value D1 of both voiced and unvoiced sounds (that is, voice), the voice determination unit 52 is caused to function as means for discriminating voice (voiced and unvoiced sounds) from non-voice. May be. Even if the threshold value T1A is set so as to exceed the index value D1 for both voiced and unvoiced sounds, there is a possibility that the unit section TU of unvoiced sounds may be misjudged as non-speech. The unvoiced sound determination unit 54 is preferably used in order to classify the sound (ie, exclude from non-speech).

(3) Modification 3
In the above embodiment, the unvoiced sound determination unit 54 has performed the process of FIG. 9 (unvoiced sound determination) for all unit intervals TU of the input sound VIN, but the sound determination unit 52 has determined that the sound is not a voice (voiced sound). The process of FIG. 9 may be executed only for the unit section TU. In this modification, since the processing of the unvoiced sound determination unit 54 is omitted for the unit interval TU that the sound determination unit 52 determines to be sound, the processing load by the determination unit 50 (unvoiced sound determination unit 54) is reduced. There is.

(4) Modification 4
The definition of each index value D (D1, D2, D3) is changed as appropriate. Therefore, the relationship between the magnitude of each index value D (D1, D2, D3) and the type of the input sound VIN is arbitrary. For example, in the above embodiment, the index value D1 is defined such that the index value D1 decreases as the intensity L1 in the determination target range A in the modulation spectrum MS increases (that is, the input sound VIN decreases as the index value D1 decreases). A configuration in which the possibility of being determined to be speech is exemplified), but a configuration in which the index value D1 is defined such that the index value D1 increases as the intensity L1 in the determination target range A increases (that is, the index value D1 is large). A configuration in which the possibility that the input sound VIN is determined to be a voice is increased. In the configuration in which the index value D1 increases as the intensity L1 increases, the speech determination unit 52 determines that the input sound VIN of the unit section TU in which the index value D1 exceeds the threshold T1A is a voiced sound (steps SA1 and SA2), and the unvoiced sound The determination unit 54 determines that the input sound VIN in the unit section TU in which the index value D1 exceeds the threshold value T1B is an unvoiced sound (steps SB4 and SB5). The threshold value T1A is set to a smaller numerical value than the threshold value T1B.

  Further, a configuration in which the index value D2 is defined so that the index value D2 decreases as the number of zero crossings of the acoustic signal SIN increases (for example, a configuration in which the reciprocal of the number of zero crossings is the index value D2), or the flatness of the frequency spectrum S0. A configuration in which the index value D3 is defined so that the index value D3 increases as the value of R is higher is also suitable. Alternatively, the variance of the frequency spectrum S0 may be calculated as the index value D3. That is, the index calculator 46 calculates an index value D3 by averaging the squares of the difference values between the intensities (energy) of each frequency band obtained by dividing the frequency spectrum S0 and the average value of the intensity over the entire band over all frequency bands. Calculated as The index value D3 calculated by the above method becomes smaller as the flatness of the frequency spectrum S0 is higher.

(5) Modification 5
In the above embodiment, the modulation spectrum MS is specified by performing Fourier transform on the time trajectory ST of the component belonging to the frequency band ω in the frequency spectrum S0, but the time trajectory of the cepstrum of the acoustic signal SIN (input sound VIN). A configuration is also adopted in which the modulation spectrum MS is specified by performing a Fourier transform on. More specifically, the component extraction unit 342 of the modulation spectrum specifying unit 34 extracts a time trajectory ST of a component whose quefrency is within a specific range from the cepstrum of each frame of the acoustic signal SIN, and the frequency analysis unit 344 The modulation spectrum MS of each unit section TU is calculated by performing Fourier transform for each unit section TU on the time trajectory ST of the cepstrum.

(6) Modification 6
In the above embodiment, the index value D1 is used for the determination by the sound determination unit 52. However, a known technique is arbitrarily adopted as a method for determining whether or not the input sound VIN is sound (voiced sound). For example, the voice determination unit 52 detects the pitch (fundamental frequency) of the acoustic signal SIN, determines that a unit interval TU in which a clear pitch is detected is a voice, and determines a unit interval TU in which no pitch is detected as a non-voice. Such a configuration is also suitable. However, in the configuration of FIG. 2, since the index value D1 used in the unvoiced sound determination unit 54 is also used in the voice determination unit 52, an index value (for example, pitch) different from the index value D1 is determined by the voice determination unit 52. There is an advantage that the load of calculating the index value is reduced as compared with the configuration used for the determination. For example, in the sound processing device 14 that detects only the unit section TU of unvoiced sound from the input sound VIN, the sound determination unit 52 is omitted.

(7) Modification 7
In the above embodiment, the identification data d and the output signal SOUT are generated by the sound processing device 14 in the space R that picks up the input sound VIN, but the position where the identification data d is generated (the position where the input sound VIN is classified). ) And the position for generating the output signal SOUT are appropriately changed. For example, in the configuration in which the sound processing device 14 outputs the acoustic signal SIN generated by the sound collecting device 12 and the identification data d generated by the determination unit 50, the output signal SOUT is generated from the acoustic signal SIN and the identification data d. A sound processing unit 60 is installed in the sound processing device 16 on the receiving side. Further, in the configuration in which the sound processing device 14 transmits the acoustic signal SIN generated by the sound collecting device 12, the same elements as those in FIG. 2 are installed in the sound processing device 16 on the receiving side. However, the remote conference system 100 is merely an example of the application of the present invention. Therefore, transmission / reception of the output signal SOUT and the acoustic signal SIN is not essential in the present invention.

(8) Modification 8
In the above embodiment, the configuration in which the sound processing unit 60 does not output the acoustic signal SIN of the unit section TU determined to be non-speech (set the volume of the output signal SOUT to zero) is exemplified. The content of the process is changed as appropriate. For example, a configuration in which the sound processing unit 60 outputs, as the output signal SOUT, a signal obtained by reducing the volume of the acoustic signal SIN for the unit section TU determined as non-speech is also suitable. In addition, a configuration in which the output signal SOUT is generated by giving a separate acoustic effect to the acoustic signal SIN for a unit section TU of voice (voiced sound or unvoiced sound) and a unit section TU of non-voice, or a unit of voiced sound A configuration in which separate acoustic effects are applied to the acoustic signal SIN for the section TU and the unit section TU of unvoiced sound is also employed. Furthermore, in a configuration in which speaker recognition (speaker identification or speaker authentication) or voice recognition is performed at the output destination (sound processing device 16) of the output signal SOUT, the sound processing unit 60 is, for example, voiced or unvoiced sound. For the determined unit section TU, the feature quantity used for speech recognition and speaker recognition is extracted from the acoustic signal SIN and output as the output signal SOUT, while for the unit section TU determined to be non-speech, the feature quantity is used. Stop extracting.

It is a block diagram of the remote conference system which concerns on embodiment of this invention. It is a block diagram of the sound processing apparatus of FIG. It is a block diagram of the modulation spectrum specific | specification part of FIG. It is a conceptual diagram which shows the procedure of the process by the modulation spectrum specific | specification part of FIG. It is a modulation spectrum of multiple types of sound. It is a flowchart which shows operation | movement of the audio | voice determination part of FIG. It is a time waveform of multiple types of sound. It is a frequency spectrum of multiple types of sound. It is a flowchart which shows the operation | movement of the unvoiced sound determination part of FIG.

Explanation of symbols

100 …… Remote conference system, 12 …… Sound collecting device, 14 …… Sound processing device, 16 …… Sound processing device, 18 …… Sound emitting device, 22 …… Control device, 24 …… Storage device, 32 …… Frequency analysis unit 34... Modulation spectrum specifying unit 42... Index calculation unit 44 .. index calculation unit 46... Index calculation unit 50 .. determination unit 52. Judgment unit, 60 ... sound processing unit, VIN ... input sound, SIN ... acoustic signal, SOUT ... output signal, d ... identification data, MS ... modulation spectrum, D1, D2, D3 ... index value, TU: Unit section.

Claims (8)

Modulation spectrum specifying means for specifying the modulation spectrum for each unit section of the input sound;
First index calculating means for calculating a first index value according to the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum;
Second index calculation means for calculating a second index value corresponding to the number of zero crossings of the input sound for each unit section;
Voice determining means for determining whether or not the input sound of each unit section is voice based on the magnitude of the first index value and the first threshold;
An unvoiced sound that determines whether or not the input sound of each unit section is an unvoiced sound based on the magnitude of a second threshold value different from the first threshold value and the first index value and the second index value A sound processing apparatus comprising: a determination unit; Modulation spectrum specifying means for specifying the modulation spectrum for each unit section of the input sound;
First index calculating means for calculating a first index value according to the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum;
Third index calculating means for calculating a third index value corresponding to the flatness of the frequency spectrum of the input sound for each unit section;
Voice determining means for determining whether or not the input sound of each unit section is voice based on the magnitude of the first index value and the first threshold;
An unvoiced sound that determines whether or not the input sound of each unit section is an unvoiced sound based on the magnitude of a second threshold value different from the first threshold value and the first index value and the third index value A sound processing apparatus comprising: a determination unit; The first index calculation means calculates the first index value so that the first index value decreases as the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum is higher,
The voice determination unit determines that the input sound of the unit section in which the first index value is lower than the first threshold is a voice,
The unvoiced sound determination means determines that an input sound of a unit section in which the first index value falls below a second threshold value that is greater than the first threshold value is an unvoiced sound.
The sound processing apparatus according to claim 1 or 2 . The first index calculation means calculates the first index value so that the first index value increases as the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum is higher,
The voice determination unit determines that the input sound of the unit section in which the first index value exceeds the first threshold is a voice,
The unvoiced sound determining means determines an input sound of a unit section in which the first index value exceeds a second threshold value smaller than the first threshold value as an unvoiced sound.
The sound processing apparatus according to claim 1 or 2 . The unvoiced sound determining means determines whether or not it is an unvoiced sound only for a unit section determined by the sound determining means not to be a sound.
The sound processing apparatus according to any one of claims 1 to 4. A sound processing unit that performs low-pass filter processing on a unit section determined by the unvoiced sound determination unit as unvoiced sound, and that does not perform the low-pass filter process on a unit section determined by the sound determination unit as speech
The sound processing apparatus according to claim 1, comprising: A modulation spectrum specifying process for specifying a modulation spectrum for each unit section of the input sound;
A first index calculation process for calculating a first index value according to the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum;
A second index calculation process for calculating a second index value corresponding to the number of zero crossings of the input sound for each unit section;
A sound determination process for determining whether or not the input sound of each unit section is a sound based on the magnitude of the first index value and the first threshold;
An unvoiced sound that determines whether or not the input sound of each unit section is an unvoiced sound based on the magnitude of a second threshold value different from the first threshold value and the first index value and the second index value A program that causes a computer to execute judgment processing. A modulation spectrum specifying process for specifying a modulation spectrum for each unit section of the input sound;
A first index calculation process for calculating a first index value according to the intensity of a component whose modulation frequency belongs to a predetermined range in the modulation spectrum;
A third index calculation process for calculating a third index value corresponding to the flatness of the frequency spectrum of the input sound for each unit section;
A sound determination process for determining whether or not the input sound of each unit section is a sound based on the magnitude of the first index value and the first threshold;
An unvoiced sound that determines whether or not the input sound of each unit section is an unvoiced sound based on the magnitude of a second threshold value different from the first threshold value and the first index value and the third index value A program that causes a computer to execute judgment processing.

Images