JP2009069425A - Music detection device, speech detection device and sound field control device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately determine whether music or speech is included in a sound signal, and to perform proper scene determination for a listener. <P>SOLUTION: In a music detection device 1 of the invention, a music scale spectrum calculation section 11 calculates a spectrum of each frequency of an equal temperament scale from the sound signal, for each frame in which a predetermined period of the sound signal is expressed, and an autocorrelation coefficient calculation section 12 calculates autocorrelation value of the spectrum. A variance calculation section 16 digitizes a size of variation of a maximum of the autocorrelation value which is detected by a coefficient maximum detection section 13, and the maximum of the autocorrelation value in a succeeding plurality of frames. When the size of variance is smaller than a threshold which is determined beforehand, a music/non-music determination section 17 determines that the sound signal is music. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention provides a music detection apparatus, a sound detection apparatus, and a sound field control that are provided in a television receiver or the like and determine whether a scene of a program being broadcast is a scene including music or a scene including sound. It relates to the device.

  With the advancement of audio playback technology, users can stay at home by playing music at high volume with HiFi (High Fidelity) audio in a dedicated listening room and surround playback with a multi-channel home theater system. In the same way, you can enjoy the natural reverberation and realism of a concert hall or movie theater.

  On the other hand, when viewing content such as television broadcasts, viewers often view at a low volume in a living room or kitchen. Even when watching TV at such a low volume, there is a demand for a sense of reality and ease of listening to audio.

  Therefore, although it is necessary to control the sound field for the content being broadcast or being played back, in the case of digital broadcasting that is currently becoming the mainstream of television broadcasting, SI (Service Information) information transmitted along with the broadcast wave, or Sound field control corresponding to the genre of a program (that is, sound field control common to each program) is performed using EPG (Electronic Program Guide) information generated based on SI information. Conceivable.

  However, since one program is composed of a plurality of scenes such as a voice-only scene, a music-only scene, or a scene including both voice and music, the program genre is based only on SI information and EPG information. In sound field control according to the above, appropriate sound field control is performed in some scenes, but appropriate sound field control is not performed in other scenes.

  Therefore, in order to perform appropriate sound field control throughout a program, it is necessary to perform sound field control for each scene instead of sound field control for each program. For example, in the case of a music scene, a sense of reality is increased by performing sound field control that emphasizes sound pressure in low and high frequency bands. In the case of a sound scene, sound (human voice) can be easily heard by performing sound field control so as to emphasize sound pressure in the middle frequency band.

  Therefore, it is necessary to determine whether the current scene is a scene including music or a scene including audio.

  As a technique for detecting a music scene or a voice scene, for example, an audio band signal voice / music discriminating device described in Patent Document 1, a voice / music discriminating device described in Patent Document 2, and a music described in Patent Document 3 A detection apparatus, a music detection method and a recording / playback apparatus, an audio information classification apparatus described in Patent Document 4, and the like have been proposed.

  Patent Document 1 has a configuration in which low- and high-frequency sound pressures are detected and determined as music when the detected sound pressure is strong, or in a configuration where sound is determined when the received signal is monaural. An audio / music discrimination device is disclosed.

  In the audio-music discrimination device disclosed in Patent Document 2, the sound power is calculated for each frame, and whether each frame is sound or silence is determined based on the calculated power value. When the number of frames is larger than a predetermined threshold, it is determined as music.

  In the music detection device disclosed in Patent Document 3, the total power of each channel of 2-channel audio and the difference in power between the channels are calculated, and the total power in each channel and the difference in power between each channel are calculated. The music section is calculated based on the comparison result between the ratio and a predetermined threshold.

  In the audio information classification device disclosed in Patent Document 4, in this audio information classification device, only the sound section is extracted using the frequency data for each unit time, and 1 second is extracted from the extracted sound section. Calculate the energy change rate for each time, extract the voice interval according to the magnitude of the energy change rate, further determine the average band energy ratio from the frequency data per unit time, and extract the music interval from the average band energy ratio . Each of the techniques disclosed in Patent Documents 1 to 4 discriminates music or voice based on frequency spectrum power and energy information.

  Other techniques for detecting a music scene include a music detection circuit described in Patent Document 5, an audio signal input device using the circuit, and a video classification method and apparatus described in Patent Document 6. These techniques detect music scenes by paying attention to the harmonic structure of the frequency spectrum.

  The music detection circuit disclosed in Patent Document 5 filters an input signal by a plurality of bandpass filters, and determines whether or not the output signal is repeated with periodicity for each output signal from each bandpass filter. When it is determined and repeated with periodicity, it is determined that the input / output signal is music.

  The video classification device disclosed in Patent Document 6 analyzes the frequency of sound information included in input video information, and the strength of edges in the time direction at a constant frequency of a spectrogram in which the obtained spectra are arranged in the time direction. A music detection unit for detecting music from

As another technique for detecting an audio scene, there is an acoustic section detection method and apparatus disclosed in Patent Document 7. In the acoustic section detection method described in Patent Document 7, frequency conversion is performed for each frame, a correlation value between predetermined frequency bands within the same frame or between adjacent frames is calculated, and an identifier and minimum frequency band that takes the maximum value are calculated. Calculate the band number indicating the difference from the frequency band identifier that takes the value, correct the band number based on the distribution of the band number, and calculate the weighted band number that is the maximum value of the corrected correction band number Then, an acoustic feature amount is calculated by multiplying the correlation value by the weighted band number. Then, the speech section is determined based on the correlation value of the calculated acoustic feature quantity in the same frame or the correlation value between different frames.
JP 5-88695 (published on April 9, 1993) JP 6-4088 (published January 14, 1994) JP 2006-301134 (released on November 2, 2006) JP 10-247093 (published September 14, 1998) Japanese Patent Laid-Open No. 5-289963 (published on November 5, 1993) JP 10-187182 (released July 14, 1998) International Publication No. WO2004 / 111996 (International Publication on Dec. 23, 2004)

  However, with the configuration described in Patent Documents 1 to 4, that is, the configuration for determining based only on the power and energy information of the frequency spectrum, it is difficult to accurately determine the music scene and the audio scene.

  Further, in the configuration described in Patent Document 5, periodic noise may be erroneously determined as music. Further, in the case of the configuration described in Patent Document 6, since the edge strength is similar to the start of sound production, the sound production start is erroneously determined as music, and the accuracy of music detection is reduced. Is low.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a music detection device and a sound detection device for accurately discriminating music scenes, sound scenes, and the like based on various sound characteristics. And providing a sound field control device.

  In order to solve the above problems, a music detection apparatus according to the present invention includes a spectrum calculation unit that calculates a frequency spectrum for each frame representing a predetermined time of the acoustic signal, and an autocorrelation of the frequency spectrum. An autocorrelation value calculating means for calculating a value, a quantifying means for quantifying the magnitude of the variation of the maximum value of the autocorrelation value in a plurality of consecutive frames, and the magnitude of the variation from a predetermined threshold value In the case of being small, it is characterized by comprising a music determination means for determining the acoustic signal as music.

  According to said structure, in the music detection apparatus which concerns on this invention, a spectrum calculation means calculates a frequency spectrum for every flame | frame showing the predetermined time of this acoustic signal from an acoustic signal. Here, the spectrum calculating means may calculate a frequency spectrum on a linear frequency axis, that is, a frequency spectrum for every fixed frequency interval, or a frequency spectrum for each scale (for example, average temperament or pure temperament). It may be calculated and is not particularly limited.

  Further, according to the above configuration, the autocorrelation value calculating means calculates the autocorrelation value of the spectrum for each frame. Here, the autocorrelation value is a value representing the autocorrelation of the spectrum, and for example, N data representing the spectrum is represented by sp (i) (i = 0, 1,..., N−1). Then, it is represented by R1 (x) (x = 1, 2,..., M) shown in Equation 1. In the example shown in Equation 1, M autocorrelation values are calculated.

As the autocorrelation value, a value obtained by normalizing the autocorrelation function shown in Equation 1 or a value multiplied by a constant may be used, and is not particularly limited as long as the autocorrelation can be evaluated.

  Here, L is the number of product-sum operations when the autocorrelation value is calculated using Equation 1, and for example, if the number of spectrum data is 136 (that is, N = 136), about 68 is sufficient for L. It is. If the upper limit of x (that is, M) is also set to 68, the number of product-sum operations for calculating R1 (x) can be made equal for all x.

  For example, the autocorrelation value is multiplied by the spectrum data string sp (i) and the spectrum data string sp (i + x) obtained by shifting the data string sp (i) by x as in the autocorrelation function of Equation 1. It is represented by the sum of the combined values. When the peak value of the spectrum of the data string sp (i) has periodicity, the values of the data strings to be multiplied with each other peak when x is one period (that is, the frequency interval at which the peak value is taken). Since the values are equal to each other, the value of the autocorrelation function R1 (x) shown in Equation 1 is large.

  By the way, in the case where the acoustic signal is a signal representing a sound including overtones (for example, a violin sound), the frequency spectrum of the acoustic signal exhibits a peak value at a frequency that is an integral multiple of the frequency of the fundamental tone.

  Therefore, for example, when the spectrum calculation unit calculates the spectrum data for each fixed frequency interval (that is, on the linear frequency axis), the data string sp (i) includes the peak value data at the fixed data interval. Become.

  Therefore, in this case, in the case where the acoustic signal is a signal that represents a sound including overtones (for example, a violin sound), the autocorrelation function value R1 (x) is the maximum when x is the data interval of the peak value. Value.

  Similarly, when the spectrum calculation unit calculates a frequency spectrum for each scale (for example, equal temperament, pure temperament, etc.), the data string sp (i) similarly has peak value data at a constant data interval. .

  Therefore, also in this case, when the acoustic signal is a signal representing a sound including overtones (for example, a violin sound), the autocorrelation function value R1 (x) is obtained when x is a data interval of a peak value. The maximum value.

  Further, according to the above configuration, the digitizing means digitizes the magnitude of the variation of the maximum value of the autocorrelation value in a plurality of consecutive frames. The value obtained by quantifying the magnitude of variation includes, for example, variance, standard deviation, or the difference between the maximum value and the minimum value, and is not particularly limited.

  And according to said structure, a music determination means determines the said acoustic signal as music, when the magnitude | size of the said dispersion | variation is smaller than a predetermined threshold value. In the case of the sound of a musical instrument containing a harmonic component, the sound including the harmonic component continues for a certain period of time. That is, the spectrum waveform indicating the peak value in the harmonic component continues for a certain time (that is, in a plurality of frames). In that case, the maximum value of the autocorrelation value of the spectrum is also a value having a certain width in a plurality of frames.

  Therefore, if the variation of the maximum value of the autocorrelation value is sufficiently small, the harmonic component of the instrument is continued. Therefore, the music determination means determines whether or not the variation is sufficiently small by comparing the variation of the maximum value of the autocorrelation value with a predetermined threshold value.

  Thereby, the music detection apparatus according to the present invention can detect the sound of a musical instrument including a harmonic component, that is, the musical tone of a stringed instrument such as a violin or a wind instrument such as a trumpet.

  In the music detection apparatus according to the present invention, it is preferable that the spectrum calculating means calculates a spectrum of each frequency corresponding to a musical scale.

According to said structure, a spectrum calculation means calculates the spectrum of each frequency corresponding to musical scales, such as an equal temperament and a pure temperament. For example, the spectrum calculating means calculates a spectrum of each frequency of the average temperament scale for each frame representing a predetermined time of the acoustic signal from the acoustic signal. Here, the average tempered scale is a scale obtained by allocating one octave by a geometric progression. For example, in the case of twelve average temperament scales, one octave that is an interval at which the frequency is doubled is divided into twelve by a geometric sequence, and the ratio of the frequencies of adjacent sounds is the twelfth root of 2. That is, each frequency fn of the sound constituting the 12 equal temperament scale, if the frequency of the fundamental tone and _{f 0,} is represented by ^{_{fn = f 0 × 2 n /}} 12. The average scale is not limited to 12 average scales. Further, the frequency of the fundamental tone may be arbitrary and is not particularly limited. The spectrum calculating means calculates a spectrum corresponding to each frequency of the average tempered scale. Therefore, the calculated number of spectra is the same for each octave. For example, in the case of 12 average temperament scales, 12 spectra are calculated for each octave.

In the music detection apparatus according to the present invention, as described above, the autocorrelation value calculation means calculates the autocorrelation value of the spectrum for each frame. As described above, the autocorrelation value is a value representing the autocorrelation of the spectrum. For example, N data representing the spectrum of each frequency corresponding to the average temperament scale is represented by sp (i) (i = 0, 1). ,..., N−1), it is represented by R1 (x) (x = 1, 2,..., M) shown in Equation 1. In the example shown in Equation 1, M autocorrelation values are calculated.
Here, L is the number of product-sum operations when the autocorrelation value is calculated using Equation 1, and for example, if the number of spectrum data is 136 (that is, N = 136), about 68 is sufficient for L. It is. If the upper limit of x (that is, M) is also set to 68, the number of product-sum operations for calculating R1 (x) can be made equal for all x.

  Incidentally, as described above, when the acoustic signal is a signal representing a sound including overtones (for example, a violin sound), the frequency spectrum of the acoustic signal exhibits a peak value at a frequency that is an integral multiple of the frequency of the fundamental tone. Further, in that case, the frequency spectrum of the acoustic signal also shows a peak value at octave intervals from the fundamental frequency.

  Further, since the average temperament scale is distributed by the same number of frequencies for each octave, the spectrum data sequence sp (i) described above becomes spectrum data having a frequency separated by an octave at regular intervals. For example, in the case of 12 average temperament scales, spectrum data having a frequency one octave apart at twelve data intervals such as sp (0), sp (12), sp (24),. Since the frequency spectrum of the acoustic signal including overtones shows a peak value in the octave interval from the fundamental frequency, the octave interval, that is, a constant data interval (12 average temperament) is also included in the spectrum data string sp (i). In the case of a musical scale, the peak value data is obtained at 12 data intervals).

  Therefore, when the acoustic signal is a signal representing a sound including overtones (for example, a violin sound), the autocorrelation function value R1 (x) is such that x is an octave interval (or an integral multiple interval thereof). Sometimes maximum.

  As a result, the number of data in the spectrum data sequence sp (i) used for calculating the autocorrelation function value R1 (x) is reduced, and the amount of calculation can be reduced. Therefore, it is possible to detect the sound of a musical instrument containing a harmonic component, that is, the musical tone of a stringed instrument such as a violin or a wind instrument such as a trumpet at high speed.

  In the music detection apparatus according to the present invention, the autocorrelation value calculation means uses sp (i) (i = 0, 1,..., N−1) that is N pieces of data representing the spectrum, The M autocorrelation values are preferably calculated as the values of the autocorrelation function R1 (x) (x = 1, 2,..., M).

  In the music detection apparatus according to the present invention, it is preferable that the numerical means calculates the variance of the maximum value to numerically express the variation.

  In the music detection apparatus according to the present invention, in order to solve the above-described problem, a spectrum power calculation unit that calculates the spectrum power of each frequency corresponding to the scale for each frame representing a predetermined time of the sound signal from the sound signal. And a scale identification number for identifying each frequency is assigned to each frequency of the scale, and for each frame, a maximum scale for detecting a maximum scale identification number that maximizes the spectrum power among the scale identification numbers. An identification number detecting means; a numerical means for quantifying the magnitude of the variation of the maximum scale identification number in a plurality of consecutive frames; and the acoustic signal when the magnitude of the variation is larger than a predetermined threshold. And a music determination means for determining the music.

  According to the above configuration, in the music detection device according to the present invention, the spectrum power calculation means includes each of the sound signals corresponding to a scale such as an equal temperament or a pure temperament for each frame representing a predetermined time of the sound signal. Calculate frequency spectrum power. For example, the spectrum power calculation means calculates each frequency spectrum power of the average temperament scale. Here, the average tempered scale is a scale obtained by allocating one octave by a geometric progression. The average scale is not limited to the 12 average scale. Further, the frequency of the fundamental tone may be arbitrary and is not particularly limited. In this case, the spectrum calculating means calculates the spectrum power corresponding to each frequency of the average scale. Therefore, the calculated number of spectra is the same for each octave. For example, in the case of 12 average temperament scales, 12 spectral powers are calculated for each octave.

  Further, according to the above configuration, a scale identification number for identifying each frequency is assigned to each frequency corresponding to the scale, and the maximum scale identification number detection means includes the scale identification number for each frame. A maximum scale identification number that maximizes the spectrum power is detected. Here, the scale identification number is a continuous number assigned in ascending or descending order of each frequency corresponding to the scale, and the interval between adjacent numbers is equal. That is, the scale identification number is a number that represents the order of the pitch (or pitch) of each sound in the average tempered scale. In addition, the sound having the frequency having the maximum value of the spectrum power is the strongest sound among the sounds included in one frame. That is, the maximum scale identification number is an identification number representing the strongest sound among the sounds included in one frame.

  Further, according to the above configuration, the digitizing means digitizes the magnitude of variation in the maximum scale identification number in a plurality of consecutive frames. The value obtained by quantifying the magnitude of variation includes, for example, variance, standard deviation, or the difference between the maximum value and the minimum value, and is not particularly limited.

  And according to said structure, a music determination means determines the said sound signal as music, when the magnitude | size of the said dispersion | variation is larger than the predetermined threshold value. Music is expressed by a combination of high, low, high, short, and timbre of sounds. The change in the level of the sound means that the strongest sound included in one frame changes in a plurality of frames. That is, the maximum value scale identification number varies in a plurality of frames.

  Therefore, if the variation of the maximum value scale identification number is sufficiently large, the acoustic signal represents music. Therefore, the music determination means determines whether the variation is sufficiently large by comparing the variation of the maximum value scale identification number with a predetermined threshold value.

  Thereby, the music detection apparatus according to the present invention can detect music in which the presence or absence of a note, that is, the pitch of a sound changes.

  In the music detection apparatus according to the present invention, it is preferable that the numerical means calculates the variance of the maximum value and numerically expresses the variation.

  In the music detection device according to the present invention, in order to solve the above-described problem, a spectrum power having a frequency equal to or lower than a first threshold predetermined for each frame or a frequency lower than the first threshold is added from an acoustic signal. Low-frequency spectrum power calculating means for calculating the low-frequency spectrum power, and frame interval detection means for detecting a frame interval at which the autocorrelation value of the low-frequency spectrum power in a predetermined number of consecutive frames is maximum. High frequency spectrum power calculating means for calculating a high frequency spectrum power by adding a spectrum power of a frequency equal to or higher than a first threshold value or a frequency higher than the first threshold value for each frame from the acoustic signal; A ratio of the low-frequency spectrum power to the high-frequency spectrum power is equal to or greater than a predetermined second threshold; and If within a range in which the frame interval is predetermined, it is characterized in that it comprises a determining music determination means with the music the acoustic signal.

  According to the above configuration, in the music detection device according to the present invention, the low-frequency spectrum power calculation means uses a frequency equal to or lower than a first threshold value predetermined for each frame or a frequency less than the first threshold value from an acoustic signal. Are added to calculate the low-frequency spectrum power.

  Further, according to the above configuration, in the music detection device according to the present invention, the frame interval detection means includes a frame interval at which the autocorrelation value of the low frequency spectrum power is maximized in a predetermined number of consecutive frames. Is detected. Here, the autocorrelation value is a value representing the autocorrelation of the spectrum power in a predetermined number of consecutive frames, and for example, data representing each spectrum power of N frames is represented by spp (i) (i = 0, 1,..., N−1), it is represented by R2 (x) (x = 1, 2,..., M) shown in Equation 2. In the example shown in Equation 2, M autocorrelation values are calculated.

As the autocorrelation value, a value obtained by normalizing the autocorrelation function shown in Formula 2 or a value multiplied by a constant may be used, and there is no particular limitation as long as the autocorrelation can be evaluated.

  Here, L is the number of product-sum operations when the autocorrelation value is calculated using Equation 2. For example, if the number of data representing spectrum power is 128 (ie, N = 128), L is 64. The degree is sufficient. If the upper limit of x (that is, M) is also 64, the number of product-sum operations for calculating R2 (x) can be made equal for all x.

  The autocorrelation value is, for example, a spectral power data sequence spp (i) of a plurality of frames and a spectral power data sequence spp (i + x) obtained by shifting the data sequence spp (i) by x as in the autocorrelation function of Equation 2. ) And multiplied by the value. Then, when there is periodicity in the fluctuation of the spectral power of a plurality of frames represented by the data string spp (i), when x is one period (that is, a frame interval or a time interval that takes a peak value), Since the values of the data strings to be multiplied are mutually peak values, the value of the autocorrelation function R2 (x) shown in Equation 1 is large.

  By the way, when the acoustic signal is a signal representing a sound having a rhythm in a low frequency (for example, drum or drum sound), the temporal change in the low frequency spectrum power of the acoustic signal has a periodicity and a constant frame. The peak value is shown in the (time) interval.

  Therefore, when the acoustic signal is a signal representing a sound having a rhythm in a low frequency range, the autocorrelation function value R2 (x) is maximum when x is a frame interval equal to the period of time change of the low frequency spectrum power. Value.

  Moreover, according to said structure, in the music detection apparatus which concerns on this invention, a high frequency spectrum power calculation means is larger than a 1st threshold value or a frequency more than a 1st threshold value for every said frame from the said acoustic signal. The spectral power of the frequency is added to calculate the high frequency spectral power.

  And according to said structure, in the music detection apparatus which concerns on this invention, a music determination means is more than the 2nd predetermined threshold value with the ratio of the said low frequency spectrum power with respect to the said high frequency spectrum power, and When the frame interval is within a predetermined range, the acoustic signal is determined as music.

  In the case of the sound of a musical instrument having a rhythm in the low frequency range, the period of time change of the low frequency spectrum power is within a certain time range. For example, in the case of a drum or a drum, the cycle often forms a rhythm between 0.2 seconds and 1.5 seconds.

  Further, even in the case of a human voice, even a sound having a rhythm in the low frequency range includes almost no low frequency spectrum, and the low frequency spectrum power is very small. On the other hand, in the case of sounds such as drums, the spectrum power in the low band is large. Therefore, the sound of an instrument (such as a drum) having a rhythm in the low range has a relatively high spectrum power in the low range compared to a human voice. That is, in the case of the sound of an instrument having a rhythm in the low frequency range, the ratio of the low frequency spectrum power to the high frequency spectrum power is large. In other words, the ratio of the low frequency spectrum power to the sum of the spectral powers of all the bands becomes large.

  Therefore, when the ratio of the low frequency spectrum power to the high frequency spectrum power is large compared to a predetermined threshold and the frame interval is within a predetermined range, that is, the low frequency spectrum power When the period of time change is within a certain time range, the acoustic signal can be determined as music.

  Thereby, the music detection device according to the present invention can detect music having a period, that is, a rhythm in the sound in the low frequency region.

  In the music detection apparatus according to the present invention, the frame interval detection means uses spp (i) (i = 0, 1,..., N−1) that is N pieces of data representing the low frequency spectrum power. Thus, it is preferable to calculate the M autocorrelation values as the values of the autocorrelation function R2 (x) (x = 1, 2,..., M).

  In the speech detection apparatus according to the present invention, in order to solve the above-mentioned problem, the fundamental frequency extracting means for extracting the fundamental frequency for each frame from the acoustic signal, and the fundamental frequency in a predetermined number of consecutive frames. The fundamental frequency change detecting means for detecting a change in the frequency and the fundamental frequency change detecting means, the basic frequency is changing monotonously, or changing from monotone change to constant frequency, or from constant frequency to monotone It is detected that there is a change to the change, and the fundamental frequency is changed within a predetermined frequency range, and the width of the change in the fundamental frequency is smaller than the predetermined frequency width. And a sound determination means for determining the sound signal as sound.

  According to said structure, a fundamental frequency extraction means extracts a fundamental frequency for every flame | frame from an acoustic signal. Examples of the method for extracting the fundamental frequency include a cepstrum method and an instantaneous frequency method, and are not particularly limited.

  According to the above configuration, the sound determination means is configured such that the fundamental frequency changes monotonously (that is, monotonically increases or monotonically increases), or changes from monotonic change to a constant frequency (that is, monotonic). Is changing from increasing to constant frequency, or changing from monotonic decreasing to constant frequency), or changing from constant frequency to monotonic changing (ie, changing from constant frequency to monotonic increasing, or changing from constant frequency to monotonic decreasing) Is detected, and the fundamental frequency changes within a predetermined frequency range, and the range of change in the fundamental frequency is smaller than the predetermined frequency width, the acoustic signal is Judged as voice.

  When the change of the fundamental frequency is changing monotonously, it may represent a phrase component of a human voice. In addition, when the change in the fundamental frequency changes from a monotone change to a constant frequency, or when the change in the fundamental frequency changes from a constant frequency to a monotone change, it represents an accent component of a human voice. there is a possibility.

  The fundamental frequency band of the human voice is generally between about 100 Hz and 400 Hz. More specifically, the band of the fundamental frequency of the male voice is about 150 Hz ± 50 Hz, and the band of the fundamental frequency of the female voice is about 250 Hz ± 50 Hz. Moreover, the band of the fundamental frequency of the child is 50 Hz higher than that of women, and is about 300 Hz ± 50 Hz. Further, in the case of a phrase component or accent component of a human voice, the width of change in the fundamental frequency is about 120 Hz.

  In other words, if the basic frequency is changing monotonously, changing from monotonic change to constant frequency, or changing from constant frequency to monotone change, the maximum and minimum values of the basic frequency are If it is not within the predetermined range, it can be determined that it is not voice. In addition, when the basic frequency is changing monotonously, changing from monotonic change to constant frequency, or changing from constant frequency to monotone change, the maximum and minimum values of the basic frequency Even when the difference is larger than a predetermined value, it can be determined that the sound is not voice.

  Therefore, when the fundamental frequency is changing monotonously, changing from monotonic change to constant frequency, or changing from constant frequency to monotone change, the change of the basic frequency is predetermined. In the case of a change within the frequency range, that is, when the maximum value and the minimum value of the basic frequency are within a predetermined range, and the width of the change in the basic frequency is smaller than the predetermined frequency width. In other words, if the difference between the maximum value and the minimum value of the fundamental frequency is smaller than a predetermined value, the speech determination means can determine that it is a phrase component or an accent component. Moreover, if the predetermined frequency range is set in accordance with male voice, female voice, and child voice, male voice, female voice, and child voice can be distinguished.

  As a result, the voice detection device according to the present invention can accurately detect a human voice and can detect both a male voice and a female voice, and can also detect a female voice or a child voice. Can be detected to some extent.

  In the voice detection device according to the present invention, the voice determination unit is configured to change the frequency when the frequency change is within a range of about 100 Hz to about 400 Hz, and the frequency change width is smaller than about 120 Hz. It is preferable to determine the acoustic signal as sound.

  The sound field control device according to the present invention includes the number of times that the acoustic signal is determined to be music within a predetermined period by the music detecting device, and the acoustic signal within the predetermined period by the voice detecting device. Is characterized in that the state of sound field control is switched according to the number of times that is determined to be sound.

  According to said structure, the sound field control apparatus which concerns on this invention is the sound field control resulting from the determination result determined by the said music detection apparatus, and the misjudgment of the determination result determined by the said audio | voice detection apparatus. Can be prevented. Here, the sound field control state includes, for example, a sound field control for a music scene, a sound field control for a sound scene, and a sound field control state for a scene including both music and sound. As a result, switching of the sound field control state can be performed at an appropriate number of times, so that the structure of switching the sound field control state is realized only at the subjective time interval that the listener recognizes as one scene. it can.

  The sound field control device according to the present invention is characterized in that the condition for switching the sound field control is changed according to the controlled state.

  According to the above configuration, the sound field control device according to the present invention can set a determination condition that gives superiority to the current state of sound field control, and content whose scene changes frequently. Even in this case, excessive scene switching can be prevented.

  The music detection apparatus according to the present invention includes a spectrum calculation unit that calculates a frequency spectrum for each frame representing a predetermined time of the acoustic signal, and an autocorrelation value calculation unit that calculates an autocorrelation value of the frequency spectrum. And numerical means for digitizing the magnitude of the variation of the maximum value of the autocorrelation value in a plurality of consecutive frames, and if the variation is smaller than a predetermined threshold, the acoustic signal is music Music determination means for determining.

  Therefore, the music detection device according to the present invention can detect the sound of a musical instrument including a harmonic component, that is, the musical tone of a stringed instrument such as a violin or a wind instrument such as a trumpet.

  In the music detection device according to the present invention, spectrum power calculating means for calculating the spectrum power of each frequency corresponding to the scale for each frame representing a predetermined time of the sound signal from the sound signal, and for each frequency of the scale, A scale identification number for identifying each frequency is assigned, and for each frame, a maximum scale identification number detecting means for detecting a maximum scale identification number having the maximum spectrum power among the scale identification numbers, and a plurality of continuous scale identification numbers. A digitizing means for digitizing the magnitude of the variation of the maximum scale identification number in the frame; a music judging means for judging that the acoustic signal is music when the magnitude of the variation is larger than a predetermined threshold; It has.

  Therefore, the music detection device according to the present invention can detect music in which the presence or absence of a note, that is, the pitch of a sound changes.

  In the music detection device according to the present invention, the low frequency spectrum power is calculated by adding the spectrum power of the frequency below the first threshold or the frequency less than the first threshold predetermined for each frame from the acoustic signal. Band spectrum power calculation means, frame interval detection means for detecting a frame interval at which the autocorrelation value of the low band spectrum power in a predetermined number of consecutive frames is maximum, and the acoustic signal for each frame. A high-frequency spectrum power calculating means for calculating a high-frequency spectrum power by adding a spectrum power of a frequency equal to or higher than the first threshold value or a frequency greater than the first threshold value, and the low-frequency spectrum power relative to the high-frequency spectrum power Is equal to or greater than a predetermined second threshold, and the frame interval is predetermined. When in the range, and a, and determines music determination means with the music the acoustic signal.

  Therefore, the music detection apparatus according to the present invention can detect music having a period, that is, a rhythm in a sound in a low frequency region.

  In the speech detection device according to the present invention, a fundamental frequency extracting means for extracting a fundamental frequency for each frame from an acoustic signal, and a fundamental frequency change detection for detecting a change in the fundamental frequency in a predetermined number of consecutive frames. And the fundamental frequency change detecting means detect that the fundamental frequency is changing monotonously, changing from monotonic change to constant frequency, or changing from constant frequency to monotone change. And the acoustic signal is determined to be speech when the fundamental frequency changes within a predetermined frequency range and the width of the fundamental frequency change is smaller than the predetermined frequency width. And a voice determination means for performing the above.

  Therefore, the voice detection device according to the present invention can accurately detect a human voice, and can also detect both a male voice, a female voice, and a child voice.

(Music detection device 1)
An embodiment of the music detection apparatus 1 according to the present invention will be described below with reference to FIGS.

  FIG. 1 is a block diagram showing a configuration of a music detection apparatus 1 according to the present invention. The music detection apparatus 1 according to the present invention includes a frame dividing unit 5, a windowing unit 6, a spectrum conversion unit 7, and a music detection unit 10.

  The music detection unit 10 includes a scale spectrum calculation unit (spectrum calculation unit) 11, an autocorrelation coefficient calculation unit (autocorrelation value calculation unit) 12, a coefficient maximum value detection unit 13, a coefficient maximum value storage unit 14, and a coefficient maximum value comparison. A unit 15, a variance calculation unit (numericalization unit) 16, and a music / non-music determination unit (music determination unit) 17 are provided.

  The music detection device 1 is mounted on a television receiver or the like, and detects a music scene included in a broadcast program based on an acoustic signal included in the broadcast signal. Here, a music scene is a scene that includes music. In addition to scenes that consist only of music in music programs, there are scenes in which music flows in the background of audio (such as human speech). included. Note that the music detection device 1 can detect a music scene included in a program being reproduced based on an acoustic signal even when the recorded program is reproduced by a recording / reproduction device or the like. Not. In the present embodiment, the music detection apparatus 1 receives an acoustic signal digitally encoded by PCM (Pulse Code Modulation).

  Hereinafter, a music detection process in the music detection apparatus 1 shown in FIG. 1 will be described.

  The frame dividing unit 5 divides the input acoustic signal into frames and outputs the result to the windowing unit 6. In the present embodiment, the frame dividing unit 5 divides the frame into 1024 samples per frame. When the sampling frequency of the acoustic signal is 44.1 kHz, the time per frame is 23 ms (= (1 ÷ 44100) × 1024).

  The windowing unit 6 multiplies the acoustic signal divided into frames by a window function such as a Hanning window and outputs the result to the spectrum conversion unit 7. By applying a window function in the windowing unit 6, it is possible to reduce an error in frequency analysis for an acoustic signal divided into frames.

  The spectrum conversion unit 7 performs FFT (Fast Fourier Transform) on the acoustic signal output from the windowing unit 6, converts the time domain acoustic signal into frequency domain data, that is, a spectrum, and calculates a scale spectrum. To the unit 11. In the spectrum conversion unit 7, FFT is performed in units of frames. In the present embodiment, as described above, 1024 samples are included in one frame, and the spectrum conversion unit 7 performs 1024-point FFT.

  Based on the spectrum output from the spectrum conversion unit 7, the scale spectrum calculation unit 11 calculates a spectrum (hereinafter referred to as a scale spectrum) corresponding to each frequency of the 12 average temperament scales.

Here, the average scale is a scale obtained by allocating 1 octave by a geometric sequence, and the 12 average scale is a scale obtained by dividing 1 octave by 12 by a geometric sequence. . An octave represents the interval between a sound and a sound that is twice as high as that sound. That is, the frequency of a sound one octave away from a certain sound is doubled. Accordingly, in the 12 average temperament scale, one octave whose frequency is doubled is divided into 12 by the geometric sequence, so the frequency ratio of adjacent sounds is the 12th root of 2. In other words, each frequency fn of the sound constituting the 12 equal temperament scale, if the frequency of the fundamental tone and _{f 0,} is represented by ^{_{fn = f 0 × 2 n /}} 12.

  In the present embodiment, the scale spectrum calculation unit 11 calculates the spectrum of each frequency of the 12 average temperament scale as the scale spectrum. FIG. 2 is a diagram showing the relationship between 12 average temperament scales and frequency. The example shown in FIG. 2 is a table showing each frequency of 12 average temperament scales when the frequency is 440 kHz with reference to the sound of octave 4 (A). The scale numbers of 0 to 126 are assigned to the frequencies of the 12 average scale ratios in FIG. 2 in ascending order of frequency. Each frequency of the 12 average temperament scales can be identified by this scale number. “C, C #, D, D #, E, F, F #, G, G #, A, A #, B” are codes for distinguishing 12 sounds in one octave. Shows codes corresponding to the respective frequencies.

  The processing of the scale spectrum calculation unit 11 will be described more specifically. The scale spectrum calculation unit 11 calculates the absolute value of the spectrum corresponding to each frequency shown in FIG. That is, the absolute value of the spectrum corresponding to each frequency shown in FIG. 2 is calculated by linear interpolation using the absolute value of the spectrum for each fixed frequency interval output from the spectrum conversion unit 7. For example, according to FIG. 2, when the frequency corresponding to the scale number 9 is 13.75 Hz, but the spectrum from the spectrum conversion unit 7 does not include the spectrum of 13.75 Hz, the scale spectrum calculation unit 11 Of the spectrum from the conversion unit 7, the absolute value of the spectrum corresponding to 13.75 Hz is calculated by linear interpolation from the absolute values of the two spectra having a frequency close to 13.75 Hz. In this way, the scale spectrum calculation unit 11 calculates the scale spectrum corresponding to all the frequencies of the scale numbers 0 to 136. Then, the scale spectrum calculation unit 11 outputs the calculated scale spectrum to the autocorrelation coefficient calculation unit 12.

  The autocorrelation coefficient calculation unit 12 calculates the autocorrelation coefficient R1 (x) of the scale spectrum output from the scale spectrum calculation unit 11. That is, the autocorrelation coefficient calculation unit 12 calculates the autocorrelation coefficient R1 (x) of the scale spectrum for each frame.

As described above, in the present embodiment, the scale spectrum calculation unit 11 calculates the spectrum corresponding to each frequency shown in FIG. 2, that is, the scale spectrum corresponding to the scale numbers 0 to 136. In the equation for calculating the autocorrelation coefficient R1 (x), sp (i) (i corresponds to a scale number) represents a scale spectrum. Here, in this embodiment, in Equation 3, L = 68 and i is changed from 0 to 67. X represents the interval of the scale for calculating the autocorrelation coefficient, and x is varied from 1 to 68 to calculate the autocorrelation coefficient R1 (x) for each x. Then, autocorrelation coefficient calculation unit 12 outputs autocorrelation coefficient R1 (x) to coefficient maximum value detection unit 13.

  The coefficient maximum value detection unit 13 detects the maximum value from the autocorrelation coefficients R1 (1) to R1 (68) output from the autocorrelation coefficient calculation unit 12. That is, the coefficient maximum value detection unit 13 detects the maximum value of the autocorrelation coefficient of the scale spectrum in each frame (hereinafter referred to as the maximum autocorrelation coefficient). Then, the coefficient maximum value detection unit 13 outputs the maximum autocorrelation coefficient to the coefficient maximum value storage unit 14 and the coefficient maximum value comparison unit 15.

  The coefficient maximum value storage unit 14 stores the maximum autocorrelation coefficient in each frame output from the coefficient maximum value detection unit 13. That is, the coefficient maximum value storage unit 14 stores the maximum value of the autocorrelation coefficient of the scale spectrum for all frames as history data.

  The coefficient maximum value comparison unit 15 determines whether the maximum autocorrelation coefficient output from the coefficient maximum value detection unit 13, that is, the maximum autocorrelation coefficient of the current frame is a minute signal. More specifically, the coefficient maximum value comparison unit 15 compares the maximum autocorrelation coefficient of the current frame with a preset threshold value. When the maximum autocorrelation coefficient of the current frame is larger than the threshold, the coefficient maximum value comparison unit 15 determines that the signal is not a minute signal, and the variance calculation unit 16 determines the maximum autocorrelation coefficient of the current frame. Output.

  On the other hand, when it is determined that the maximum autocorrelation coefficient of the current frame is a minute signal, the maximum autocorrelation coefficient of the current frame is not output to the variance calculation unit 16. In this case, it is not determined whether the current frame is a music scene.

  Further, the coefficient maximum value comparison unit 15 extracts the maximum autocorrelation coefficient for the past frame from the coefficient maximum value storage unit 14, and the maximum autocorrelation coefficient of the extracted past frame is a minute signal in the same manner as the current frame. If it is not a minute signal, the maximum autocorrelation coefficient of the past frame to be determined is output to the variance calculation unit 16. On the other hand, when the maximum autocorrelation coefficient of the extracted past frame is a minute signal, the maximum autocorrelation coefficient of the determination target past frame is not output to the variance calculation unit 16.

  In the present embodiment, the coefficient maximum value comparison unit 15 sequentially extracts the power maximum values of past frames from the coefficient maximum value storage unit 14 in order of time closest to the current frame, and determines whether or not the signal is a minute signal. The process of outputting the maximum autocorrelation coefficient of the past frame to be determined to the variance calculation unit 16 based on the determination result is repeated. This process is repeated until the maximum autocorrelation coefficients of the four past frames are output to the variance calculation unit 16. Finally, the variance output unit 16 outputs the maximum autocorrelation coefficient to the variance calculation unit 16 for a total of five frames including the current frame and the four past frames.

  The variance calculation unit 16 calculates the variance for the maximum autocorrelation coefficients of the five frames output from the coefficient maximum value comparison unit 15 using the formula shown in Equation 4, and outputs the variance to the music / non-music determination unit 17. .

Here, Rx _i (i = 1 to 5) is the maximum autocorrelation coefficient of each of the five frames, and <Rx> is an average of the maximum autocorrelation coefficients of the five frames. N = 5.

  The music / non-music determination unit 27 determines a music scene (a scene in which music is included in an acoustic signal) when the variance output from the variance calculation unit 26 is smaller than a preset threshold value. That is, music is detected.

  The coefficient maximum value comparison unit 15 determines whether or not the maximum autocorrelation coefficient is a minute signal and outputs only the maximum autocorrelation coefficient that is not a minute signal to the variance calculation unit 16. The variance calculated by the variance calculation unit 16 is highly reliable as an index representing the variation of the maximum autocorrelation coefficient. However, it is not always necessary to use a configuration in which only the maximum autocorrelation coefficient that is not a minute signal is output to the variance calculation unit 16 by the minute signal determination in the coefficient maximum value comparison unit 15, and there is no particular limitation.

  FIG. 3 is a diagram showing a frequency spectrum of a trumpet, (a) is a diagram showing a frequency spectrum at a certain time, and (b) is a frequency spectrum 23 ms after the time showing the frequency spectrum of (a). FIG. FIG. 3A shows an example of a frequency spectrum when a sound of 880 Hz is blown with a trumpet, and the spectrum shows a peak in the vicinity of a frequency that is an integral multiple of the blown sound (ie, overtone). In addition, as shown in FIG. 3B, overtones continue even after 23 ms from the time showing the frequency spectrum of FIG.

  4A and 4B are diagrams showing the frequency spectrum of the koto, where FIG. 4A is a diagram showing the frequency spectrum at a certain time, and FIG. 4B is the frequency spectrum 23 ms after the time showing the frequency spectrum of FIG. FIG. FIG. 4A shows an example of a frequency spectrum when a sound with a koto is played, and the spectrum shows a peak in the vicinity of a frequency that is an integral multiple of the played sound (ie, a harmonic). In addition, as shown in FIG. 4B, overtones continue even after 23 ms from the time showing the frequency spectrum of FIG.

  As shown in FIGS. 3 and 4, each of the sounds of the trumpet and the iron koto includes a unique harmonic component, and the sound including the harmonic component continues for a certain period of time. In addition to the trumpet and iron koto shown in FIG. 3 and FIG. 4, the musical tone of a stringed instrument such as a violin includes a harmonic component.

  In the music detection apparatus 1 according to the present invention, the autocorrelation coefficient calculation unit 12 calculates the autocorrelation coefficient (autocorrelation value) of the frequency spectrum, and the coefficient maximum value detection unit 13 calculates the autocorrelation coefficient in each frame. The maximum value is calculated, and the variance calculation unit 16 calculates the variance of the maximum value (a size of variation) between a plurality of frames. If the variance is smaller than a predetermined threshold, the variance is determined as music.

  Since the frequency spectrum of the musical tone shows a peak in the harmonic component as described above, the autocorrelation coefficient becomes maximum when the data sequence of the original frequency spectrum is shifted by the frequency interval at which the harmonic component appears.

  In the case of a musical instrument sound, the same frequency spectrum continues for a certain period of time. Therefore, in the case of a musical sound, the maximum value of the autocorrelation coefficient of the frequency spectrum shows a substantially constant value over several frames. That is, the variation in the maximum value of the autocorrelation coefficient of the frequency spectrum between consecutive frames is small.

  In the music detection apparatus 1 according to the present invention, the scale spectrum calculation unit 11 calculates a scale spectrum corresponding to each frequency of 12 average temperament scales, and the autocorrelation coefficient calculation unit 12 calculates the autocorrelation coefficient of the scale spectrum. calculate. If the sound includes harmonic components, peak values indicating harmonic components appear at regular intervals in the scale spectrum.

  Therefore, in the case of a musical tone, the maximum value of the autocorrelation coefficient of the scale spectrum shows a substantially constant value over several frames, and the variation is small. Therefore, when the variance of the maximum value of the autocorrelation coefficient of the scale spectrum among a plurality of frames is smaller than a predetermined threshold, it can be determined that the music is music.

  In the present embodiment, whether the music is based on the scale spectrum calculated by the scale spectrum calculation unit 11, that is, using the variance in a plurality of frames for the maximum value of the autocorrelation coefficient of the scale spectrum. Although determination of whether or not is performed, a configuration of calculating based on the frequency spectrum output from the spectrum conversion unit 7 may be used. That is, the variance of a plurality of frames with respect to the maximum value of the autocorrelation coefficient of the frequency spectrum output from the spectrum converter 7 is calculated, and when the variance is smaller than a predetermined threshold, it is determined to be music. There may be, and it does not specifically limit.

  In the present embodiment, whether or not the music scene is determined based on the variance of the maximum value of the autocorrelation coefficient of the scale spectrum for five frames, but is used to calculate the variance. The number of frames may be 5 frames or more, and is not particularly limited.

(Music detection device 2)
An embodiment of the music detection apparatus 1 according to the present invention will be described below with reference to FIGS.

  FIG. 5 is a block diagram showing the configuration of the music detection device 2 according to the present invention. The music detection device 2 according to the present invention includes a frame dividing unit 5, a windowing unit 6, a spectrum conversion unit 7, and a music detection unit 20.

  The music detection unit 20 includes a scale spectrum calculation unit 21, a spectrum power calculation unit (spectrum power calculation unit) 22, a power maximum value detection unit (maximum scale identification number detection unit) 23, a power maximum value storage unit 24, and a power maximum value comparison. A unit 25, a variance calculation unit (numericalization unit) 26, and a music / non-music determination unit (music determination unit) 27.

  Similar to the music detection device 1, the music detection device 2 is mounted on a television receiver or the like, and detects a music scene included in a broadcast program based on an acoustic signal included in the broadcast signal. In the present embodiment, similarly to the music detection device 1, an audio signal digitally encoded by PCM (Pulse Code Modulation) is input to the music detection device 2.

  Hereinafter, a music detection process in the music detection apparatus 2 shown in FIG. 5 will be described.

  The processing contents of the frame division unit 5, the windowing unit 6, and the spectrum conversion unit 7 in the music detection device 2 are the same as those of the music detection device 1 and will not be described.

  The scale spectrum calculation unit 21 is a spectrum (referred to as scale spectrum) corresponding to each frequency of the 12 average temperament scales shown in FIG. ) Is generated. The scale spectrum calculation unit 21 performs the same processing as the scale spectrum calculation unit 11 in the music detection device 1, and thus detailed description thereof is omitted.

  The spectrum power calculation unit 22 calculates the spectrum power for each scale from the scale spectrum (that is, the value of the square of the spectrum; hereinafter referred to as the scale spectrum power) and outputs it to the power maximum value detection unit 23.

  The power maximum value detector 23 detects the maximum value of the scale spectrum power. Then, the power maximum value detection unit 23 sets the maximum value of the scale spectrum power (hereinafter referred to as the power maximum value) and the scale number corresponding to the power maximum value (hereinafter referred to as the maximum value scale number) as the power maximum. The data is output to the value storage unit 24 and the power maximum value comparison unit 25. Note that the scale number is the scale number shown in FIG. The scale number corresponds to the scale identification number in the claims.

  The power maximum value storage unit 24 stores the power maximum value and maximum value scale number of each frame output from the power maximum value detection unit 23. That is, the power maximum value storage unit 24 stores the power maximum value and the maximum value scale number as history data for all frames.

  The power maximum value comparison unit 25 determines whether or not the power maximum value output from the power maximum value detection unit 23, that is, the power maximum value of the current frame is a minute signal. More specifically, the power maximum value comparison unit 25 compares the power maximum value of the current frame with a preset threshold value. When the power maximum value of the current frame is larger than the threshold value, the power maximum value comparison unit 25 determines that the signal is not a minute signal, and outputs the maximum value scale number of the current frame to the variance calculation unit 26.

  On the other hand, when it is determined that the power maximum value of the current frame is a minute signal, the maximum scale number of the current frame is not output to the variance calculation unit 16. In this case, it is not determined whether the current frame is a music scene.

  Further, the power maximum value comparison unit 25 extracts the power maximum value and the power maximum scale for the past frame from the power maximum value storage unit 24, and the extracted power maximum value of the past frame is a minute signal in the same manner as the current frame. If it is not a minute signal, the maximum power scale of the past frame to be determined is output to the variance calculation unit 26. On the other hand, when the power maximum value of the extracted past frame is a minute signal, the maximum power scale of the past frame to be determined is not output to the variance calculation unit 26.

  In the present embodiment, the power maximum value comparison unit 25 sequentially extracts the power maximum values of past frames from the power maximum value storage unit 24 in order of time closest to the current frame, and determines whether or not the signal is a minute signal. Based on the determination result, the process of outputting the maximum power scale of the past frame to be determined to the variance calculating unit 26 is repeated. This process is repeated until the maximum power scales of the four past frames are output to the variance calculation unit 26. Finally, the maximum value scale number is output to the variance calculation unit 26 for the total 5 frames including the current frame and the four past frames.

  The variance calculation unit 26 calculates the variance for the maximum scale numbers of the five frames output from the power maximum value comparison unit 26 using the formula shown in Equation 5, and outputs the variance to the music / non-music determination unit 27.

Here, x _i (i = 1 to 5) is the maximum scale number of each of the five frames, and <x> is the average of the maximum scale numbers of the five frames. N = 5.

  The music / non-music determination unit 27 determines a music scene when the variance output from the variance calculation unit 26 is greater than a preset threshold.

  FIG. 6 is a diagram illustrating an example of the relationship between the frame and the maximum power scale. FIG. 6 is a graph showing the maximum scale number for each frame of the song “Drive” by the artist “Ketsumeishi”. As shown in FIG. 6, the maximum scale numbers vary around 40. As in the example shown in FIG. 6, music is usually composed of various sounds, and therefore there is variation in musical scale. In the music detection device 2, the music / non-music determination unit 27 can quantitatively evaluate the variation in the scale using the variance calculated by the variance calculation unit 26. Therefore, when the variance as an index of the scale variation is larger than a predetermined threshold, it can be determined that the music.

  In this embodiment, it is determined whether or not a music scene is based on the variance of the maximum power scale for five frames. However, the number of frames used to calculate the variance is five or more. There is no particular limitation.

  In the present embodiment, the scale spectrum calculation unit 21 is configured to calculate a spectrum corresponding to each frequency of the 12 average temperament shown in FIG. 2, but the scale spectrum calculation unit 21 has a configuration other than the 12 average temperament. There may be a configuration in which a spectrum corresponding to a scale of equal temperament or pure temperament is calculated as a scale spectrum, and there is no particular limitation.

(Music detection device 3)
An embodiment of the music detection device 3 according to the present invention will be described below with reference to FIGS.

  FIG. 7 is a block diagram showing the configuration of the music detection device 3 according to the present invention. The music detection device 3 according to the present invention includes a frame division unit 5, a windowing unit 6, a spectrum conversion unit 7, and a music detection unit 30.

  The music detection unit 30 includes an ultra low frequency spectrum power calculation unit (low frequency spectrum power calculation unit) 31, an ultra low frequency spectrum power storage unit 32, an ultra low frequency spectrum power autocorrelation coefficient calculation unit 33, and a coefficient maximum value determination unit. (Frame interval detection means) 34, high frequency spectrum power calculation unit (high frequency spectrum power calculation means) 35, ultra low frequency / high frequency power ratio calculation unit 36, and music / non-music determination unit (music determination unit) 37 I have.

  Similar to the music detection device 1, the music detection device 3 is mounted on a television receiver or the like, and detects a music scene included in a program being broadcast based on an acoustic signal included in the broadcast signal. In the present embodiment, similarly to the music detection device 1, an acoustic signal digitally encoded by PCM (Pulse Code Modulation) is input to the music detection device 3.

  Below, the music detection process in the music detection apparatus 3 shown in FIG. 7 will be described.

  The processing contents of the frame division unit 5, the windowing unit 6, and the spectrum conversion unit 7 in the music detection device 3 are the same as those of the music detection device 1, and the description thereof is omitted.

  The ultra-low frequency spectrum power calculation unit 31 has a spectrum power of 100 Hz (predetermined first threshold value) or less based on the spectrum for each frame received from the spectrum conversion unit 7 (hereinafter referred to as an input spectrum). The sum is calculated and output to the ultra low frequency spectrum power storage unit 32 and the high frequency spectrum power calculation unit 33. That is, the ultra-low frequency spectrum power calculation unit 31 extracts a spectrum of 100 Hz or less from the input spectrum, and calculates the sum of values obtained by squaring the extracted spectrum (hereinafter referred to as the ultra-low frequency spectrum power total). To do. That is, the total ultra-low frequency spectrum power is the total of the spectrum power for the ultra-low frequency spectrum of 100 Hz or less for each frame. In this embodiment, the total ultra-low frequency spectrum power is calculated as the sum of spectrum powers of 100 Hz or less, but may be the sum of spectrum powers less than 100 Hz. Further, the threshold value is not limited to 100 Hz.

  The ultra low frequency spectrum power storage unit 32 stores the total ultra low frequency spectrum power of 100 Hz or less output from the ultra low frequency spectrum power calculation unit 31. That is, the ultra-low frequency spectrum power storage unit 32 stores the ultra-low frequency spectrum power total as history data for all frames.

  Further, the ultra-low frequency spectrum power autocorrelation coefficient calculating unit 33 calculates the total ultra-low frequency spectrum power output from the ultra-low frequency spectrum power calculating unit 31, that is, the ultra-low frequency spectrum power total of the current frame, The autocorrelation coefficient of the low-frequency spectrum power between consecutive frames is calculated using the total ultra-low-frequency spectrum power of the past frame extracted from the high-frequency spectrum power storage unit 32. In the present embodiment, the autocorrelation coefficient R2 (x) shown in Equation 6 is calculated for a total of 128 frames including the current frame and the past 127 frames.

In the above formula for calculating the autocorrelation coefficient R2 (x), spp (i) represents the total ultra-low frequency spectrum power of each frame. Here, i represents a number for identifying a frame (hereinafter referred to as a frame identification number), and is an integer from 1 to 128. Frame identification numbers are assigned to each frame in time series in the order of 1 to 128. That is, spp (1) is the spectrum power of the past frame, and spp (128) is the spectrum power of the current frame. In the present embodiment, in Equation 6, L = 64 and i is changed from 0 to 63. Also, x represents the frame interval for calculating the autocorrelation coefficient, and x is varied from 1 to 64 to calculate the autocorrelation coefficient R2 (x) for each x. Then, the ultra low frequency spectrum power autocorrelation coefficient calculation unit 33 outputs the calculated 64 autocorrelation coefficients R2 (x) (x is an integer of 1 to 64) to the coefficient maximum value detection unit 34.

  The coefficient maximum value detection unit 34 detects the maximum value of R2 (1) to R2 (64) output from the ultra low frequency spectrum power autocorrelation coefficient calculation unit 33, and the autocorrelation coefficient R2 (x) is the maximum. The frame interval x indicating the value (hereinafter referred to as the maximum value frame interval) is output to the high-frequency spectrum power calculation unit 35.

  The high frequency spectrum power calculation unit 35 receives the input spectrum for each frame from the coefficient maximum value detection unit 34 together with the maximum value frame interval. That is, in the present embodiment, the input spectrum output from the spectrum conversion unit 7 includes the ultra low frequency spectrum power calculation unit 31, the ultra low frequency spectrum power autocorrelation coefficient calculation unit 33, and the coefficient maximum value determination unit 34. And input to the high-frequency spectrum calculator 35.

  Then, the high-frequency spectrum power calculation unit 35 has a spectrum power of 100 (predetermined first threshold) Hz or higher based on the spectrum for each frame received from the coefficient maximum value detection unit 34, that is, the input spectrum. The sum is calculated and output to the ultra low frequency / high frequency power ratio calculation unit 36. That is, the high frequency spectrum power calculation unit 35 extracts a spectrum of 100 Hz or more from the input spectrum, and calculates a sum of values obtained by squaring the extracted spectrum (hereinafter referred to as high frequency spectrum power total). That is, the total high-frequency spectrum power is the total spectral power for a high-frequency spectrum of 100 Hz or more for each frame. In the present embodiment, the total high-frequency spectral power is calculated as the total of spectral power of 100 Hz or more, but may be the total of spectral power greater than 100 Hz. Further, the threshold value is not limited to 100 Hz.

  In the present embodiment, the high frequency spectrum power calculation unit 35 outputs the total ultra low frequency spectrum power to the ultra low frequency / high frequency power ratio calculation unit 36 together with the high frequency spectrum power total. . That is, in the present embodiment, the total ultra-low frequency spectrum power calculated by the ultra-low frequency spectrum power calculation unit 31 is the same as the ultra-low frequency spectrum power autocorrelation coefficient calculation unit 33, the coefficient maximum value determination unit 34, The signal is input to the ultra low frequency / high frequency power ratio calculating unit 36 through the band spectrum calculating unit 35. The high frequency spectrum power calculation unit 35 also outputs the maximum value frame interval to the ultra low frequency / high frequency power ratio calculation unit 36.

  The ultra low frequency / high frequency power ratio calculation unit 36 is a ratio of the total high frequency spectrum power received from the high frequency spectrum power calculation unit 35 to the total ultra low frequency spectrum power (hereinafter, the ultra low frequency / high frequency power ratio). Is calculated and output to the music / non-music determination unit 37. More specifically, the ultra low frequency / high frequency power ratio is calculated by calculating the total ultra low frequency spectrum power / the high frequency spectrum power. Note that, as the ultra low frequency / high frequency power ratio, the total ultra low frequency spectrum power / (total ultra low frequency spectrum power + total high frequency spectrum power) may be calculated, and is not particularly limited. Further, the ultra low frequency / high frequency power ratio calculation unit 36 also outputs the maximum value frame interval to the music / non-music determination unit 37.

  Whether the music / non-music determination unit 37 has the ultra-low / high frequency power ratio output from the ultra-low / high frequency power ratio calculation unit 36 equal to or greater than a predetermined threshold value (for example, 0.0003). Determine whether or not. Further, the music / non-music determination unit 37 determines whether the maximum value frame interval is 10 frames or more and 64 frames or less (that is, 0.23 s to 1.5 s).

  Then, the music / non-music determination unit 37 determines that, as a result of the above two determinations, the maximum value frame interval is 10 frames or more and 64 frames or less, and the ultra low frequency / high frequency power ratio is 0.0003 or more. It is determined as a music scene.

  FIG. 8 is a diagram showing the frequency spectrum of the drum. The frequency spectrum of the drum shown in FIG. 8 does not include overtone components, unlike the frequency spectrum of the trumpet shown in FIG. 3 or the frequency spectrum of the iron koto shown in FIG. Therefore, a music scene of an instrument that does not include a harmonic component such as a drum, that is, is not a musical sound may not be detected by the music detection device 1.

  FIG. 9 is a diagram showing a time transition of the total spectral power of the drum of 100 Hz or less. The vertical axis indicates the total of spectrum power of 100 Hz or less when the least significant bit of 16-bit PCM is 1. The horizontal axis indicates the frame number when a certain time is set as the frame number 1. As shown in FIG. 9, the time transition of the spectral power of 100 Hz or less of the drum has periodicity. That is, a spectrum power peak of 100 Hz or less appears repeatedly in a certain period. In the music detection device 3, the ultralow frequency spectrum power autocorrelation coefficient calculation unit 33 calculates an autocorrelation coefficient of spectrum power of 100 Hz or less between a plurality of frames, and the coefficient maximum value determination unit 34 calculates the autocorrelation coefficient. Is detected as the maximum frame interval (that is, the maximum value frame interval). Here, in FIG. 9, since the spectral power of 100 Hz or less between a plurality of frames shows a peak in a constant cycle as described above, the data sequence of the original spectral power of 100 Hz or less is converted into a constant cycle in which the peak appears. The autocorrelation coefficient becomes the maximum value when shifted by the frame interval of minutes. That is, the maximum value frame interval at which the autocorrelation coefficient is maximum detected by the coefficient maximum value determination unit 34 is a period in which a peak of spectrum power of 100 Hz or less appears. Further, the period of the above peak as shown in FIG. 9 is within a certain time range in the case of an instrument such as a drum. Therefore, it is determined whether or not this period (that is, the maximum value frame interval) is within a predetermined range (for example, 10 frames or more and 64 frames or less, corresponding to a predetermined range in the claims). If it is not within the predetermined range, it can be determined that it is not music.

  By the way, although human speech has almost no component of 100 Hz or less, the component of 100 Hz or less, which is slightly contained, shows periodicity of spectral power. Therefore, further determination conditions are necessary so that human speech is not erroneously determined as music such as drums. Here, the component contained in the sound of the drum or the like is different from the human voice, and depends on the low range. Therefore, when the proportion of the ultra low frequency component of 100 Hz or less is very small, It can be determined that it is not music. Therefore, the ultra-low frequency / high frequency power ratio calculated by the ultra-low frequency / high frequency power ratio calculation unit 36 is a predetermined threshold value (for example, 0.0003, which is the second threshold value in the claims). It can be determined that it is not music if it is equal to or less than a predetermined threshold value, that is, if the proportion of the ultra low frequency component is very small.

  As a result, the music / non-music determination unit 37 sets the ultra low frequency / high frequency power ratio output from the ultra low frequency / high frequency power ratio calculation unit 36 to a predetermined threshold value (for example, 0.0003) or more. When the maximum value frame interval is within a predetermined range (for example, 10 frames or more and 64 frames or less), it can be determined that the music is stored.

(Voice detection device 4)
An embodiment of the voice detection device 4 according to the present invention will be described with reference to FIGS. 10 to 13 as follows.

  FIG. 10 is a block diagram showing the configuration of the voice detection device 4 according to the present invention. The voice detection device 4 according to the present invention includes a frame dividing unit 5, a windowing unit 6, a spectrum conversion unit 7, and a voice detection unit 40.

  The voice detection unit 40 includes a logarithmic spectrum calculation unit 41, a cepstrum calculation unit 42, a fundamental frequency extraction unit (basic frequency extraction means) 43, a fundamental frequency storage unit 44, a low-pass filter unit 45, and a phrase component analysis unit 46 (basic frequency change detection). Means), an accent component analysis unit 47 (basic frequency change detection unit), and a music / non-music determination unit (speech determination unit) 48.

  Similar to the music detection device 1, the audio detection device 4 is mounted on a television receiver or the like, and detects a music scene included in a broadcast program based on an acoustic signal included in the broadcast signal. In the present embodiment, an audio signal digitally encoded by PCM (Pulse Code Modulation) is input to the audio detection device 4 as in the music detection device 1.

  Below, the process of the audio | voice detection in the audio | voice detection apparatus 4 shown in FIG. 10 is demonstrated.

  The processing contents of the frame division unit 5, the windowing unit 6, and the spectrum conversion unit 7 in the voice detection device 4 are the same as those of the music detection device 1, and a description thereof is omitted.

The logarithmic spectrum calculation unit 41 converts a spectrum for each frame received from the spectrum conversion unit 7 (hereinafter referred to as an input spectrum) into a logarithm of the base 10. That is, the logarithmic spectrum calculation unit 41 calculates log ₁₀ | sp | when the input spectrum is sp. Hereinafter, log ₁₀ | sp | is referred to as a logarithmic spectrum. Then, the log spectrum calculation unit 41 outputs the log spectrum to the cepstrum calculation unit 42.

  The cepstrum calculation unit 42 performs 1024-point IFFT (Inverse Fast Fourier Transform) on the logarithmic spectrum output from the logarithmic spectrum calculation unit 41 and converts the logarithm spectrum into a cepstrum that is time-domain data. Then, the cepstrum calculation unit 42 outputs the calculated cepstrum to the fundamental frequency extraction unit 43.

  The fundamental frequency extraction unit 43 extracts the maximum cepstrum on the higher-order side (about fs / 800 or more) of the cepstrum output from the cepstrum calculation unit 42, and calculates the reciprocal of the quefrency that becomes the maximum cepstrum as the fundamental frequency (F0). . The fundamental frequency extraction unit 43 outputs the fundamental frequency (F0) to the fundamental frequency storage unit 44 and the low pass filter unit 45.

  The fundamental frequency storage unit 44 stores the fundamental frequency (F0) output from the fundamental frequency extraction unit 43. That is, the fundamental frequency storage unit 44 stores the fundamental frequency (F0) as history data for all frames.

  The low-pass filter unit 45 low-pass-filters the fundamental frequency (F0) output from the fundamental frequency extraction unit 43, that is, the fundamental frequency (F0) of the current frame, and outputs it to the phrase component analysis unit 46. The low-pass filter unit 44 extracts the fundamental frequency (F0) for the past frame from the fundamental frequency storage unit 44, performs low-pass filtering in the same way as the fundamental frequency (F0) of the current frame, and sends it to the phrase component analysis unit 46. Output. In the low-pass filter unit 44, information on the low-frequency fundamental frequency (F0), that is, the fundamental frequency (F0) that causes noise is removed without being output to the phrase component analysis unit 46 or the accent component analysis unit 47. . If the basic frequency (F0) of the current frame is not output as a result of low-pass filtering in the low-pass filter unit 44, it is not determined whether the current frame is an audio scene.

  In the present embodiment, the low-pass filter unit 45 performs a process of sequentially extracting the fundamental frequency (F0) of the past frame from the fundamental frequency storage unit 44 in order of time closest to the current frame, and performing low-pass filtering and outputting. repeat. This process is repeated until the four fundamental frequencies (F0) are output to the phrase component analysis unit 46. Finally, the low-pass filter unit 45 outputs the fundamental frequency (F0) to the phrase component analysis unit 46 for a total of five frames including the current frame and the four past frames.

  The phrase component analysis unit 46 determines whether the fundamental frequency (F0) monotonously decreases or monotonously increases (that is, changes monotonously) with respect to the fundamental frequency (F0) of the five frames output from the low-pass filter unit 45. Is analyzed). Then, the phrase component analysis unit 46 determines whether the basic frequency (F0) monotonically decreases or monotonically increases between the above five frames within a predetermined frequency range (for example, between 100 Hz and 400 Hz). Determine whether. Further, when the phrase component analysis unit 46 detects a monotonic decrease or monotonic increase (that is, monotonic change) of the fundamental frequency (F0) between the five frames, the monotonic decrease or Then, it is determined whether or not the width of the change in the fundamental frequency (F0) in the monotonic increase is within a predetermined range (for example, within 120 Hz).

  The phrase component analysis unit 46 has a monotonic decrease or monotonic increase in the fundamental frequency (F0) between the above five frames within a predetermined frequency range (for example, between 100 Hz and 400 Hz). Within a predetermined frequency range), and the width of the monotonic decrease or monotonic increase is within a predetermined range (for example, within 120 Hz, and the predetermined frequency in the claims) The monotonic decrease or monotonic increase is determined to be a phrase component representing a phrase by a human voice. Then, the phrase component analysis unit 46 outputs phrase analysis result information indicating whether or not the phrase component is included to the accent component analysis unit 47. In the present embodiment, the phrase component analysis unit 46 outputs the five frames of the fundamental frequency (F0) from the low-pass filter unit 45 to the accent analysis unit 47 together with the phrase analysis result information.

  The accent component analysis unit 47 changes the basic frequency (F0) from monotonically increasing to flat (no change) or flattening from monotonic decreasing to the basic frequency (F0) of the five frames output from the phrase component analyzing unit 46. (I.e., change from monotonic change to constant frequency). Further, the accent component analysis unit 47 analyzes whether the transition is from flat (no change) to monotonic decrease, or from flat (no change) to monotone increase (that is, change from a constant frequency to monotone change). To do. Then, the accent component analysis unit 47 shifts the fundamental frequency (F0) from the monotone increase to the flat, the transition from the monotone decrease to the flat, the transition from the flat to the monotone decrease, or from the flat between the above five frames. It is determined whether or not the transition to monotonic increase is within a predetermined frequency range (for example, between 100 Hz and 400 Hz, and within a predetermined frequency range in the claims). Further, the accent component analysis unit 47 makes a transition from the monotonic increase to the flat, the monotone decrease to the flat, the transition from the flat to the monotonous decrease, or from the flat, between the above five frames. When a transition to monotonic increase is detected, whether or not the width of the change in the fundamental frequency (F0) is within a predetermined range (for example, within 120 Hz, a predetermined frequency width in the claims). Determine whether.

  The accent component analysis unit 47 makes a transition from the monotone increase to the flat, the transition from the monotone decrease to the flat, the transition from the flat to the monotone decrease, or the monotone increase from the flat between the above five frames. If the transition to is within a predetermined frequency range (for example, between 100 Hz and 400 Hz) and the width of the change is within the predetermined range (for example, within 120 Hz), the accent by human voice It is determined that the accent component represents. Then, the accent component analysis unit 47 outputs accent analysis result information indicating whether or not an accent component is included to the voice / non-voice determination unit 48. In the present embodiment, the accent component analysis unit 47 outputs the phrase analysis result information from the phrase component analysis unit 46 to the voice / non-voice determination unit 48 together with the accent analysis result information.

  The voice / non-voice determination unit 37 determines whether or not either an accent component or a phrase component is included based on the accent analysis result information and the phrase analysis information, and the accent component or the phrase component. Is included, it is determined as an audio scene (scene in which audio is included in the acoustic signal). That is, the voice is detected. On the other hand, when neither an accent component nor a phrase component is included, it is determined that the scene is a non-audio scene.

  FIG. 11 is a diagram showing the characteristics of speech, (a) is a diagram showing a time waveform in a Japanese speech by a man, and (b) is a fundamental frequency obtained from the time waveform of (a) ( It is a figure which shows the time change of F0). FIG. 12 is a diagram showing the characteristics of speech, (a) is a diagram showing a time waveform in a Japanese speech by a woman, and (b) is a fundamental frequency obtained from the time waveform of (a) ( It is a figure which shows the time change of F0). FIG. 13 is a diagram showing the characteristics of music, (a) shows a time waveform, and (b) shows a time change of the fundamental frequency (F0) obtained from the time waveform of (a). is there.

  As shown in FIGS. 11B and 12B, in the case of human speech, a phrase component and an accent component are included, and both have a frequency in the range of 100 Hz to 400 Hz. Also, the amount of change in the phrase component and the accent component are both within about 100 Hz. On the other hand, as shown in FIG. 13B, in the case of music, the phrase component and the accent component are not included at all.

  Therefore, the monotonic decrease or monotonic increase of the fundamental frequency (F0) between the above five frames is within a predetermined frequency range (for example, between 100 Hz and 400 Hz), and the monotonic decrease or When the width of the monotonous increase is within a predetermined range (for example, within 120 Hz), it is understood that a phrase component representing a phrase in a human voice is included. Can be judged. In addition, the transition of the fundamental frequency (F0) from the monotonic increase to the flat, the transition from the monotonic decrease to the flat, the transition from the flat to the monotonic decrease, or the transition from the flat to the monotonic increase between the above five frames. If it is within a predetermined frequency range (for example, between 100 Hz and 400 Hz) and the range of change is within the predetermined range (for example, 120 Hz), an accent component representing an accent in a human voice is included. Therefore, it can be determined that human voice is included.

  In the music voice detection device provided with the music detection devices 1, 2, 3, and the voice detection device 4, music detection processing by the voice detection devices 1, 2, 3, When all the voice detection processing by the voice detection device 4 is executed, detection can be performed in a short time (minimum 0.1 seconds), and the correct answer rate is 87% for voice and 94% for music, thereby reducing false detection. .

(Sound field control device 50)
An embodiment of the sound field control device 50 according to the present invention will be described with reference to FIG. FIG. 14 is a block diagram showing a configuration of the sound field control device 50 according to the present embodiment. The sound field control device 50 corrects an acoustic signal according to a music scene, a sound scene, or the like, and controls the sound field. The sound field control device 50 includes a music determination unit 51, a memory 52, a sound field control determination unit 53, and a sound field control processing unit 54. In the present embodiment, the music determination unit 51 is included in the sound field control device 50, but may be configured independently of the sound field control device 50, and is not particularly limited. The sound field control device 50 receives the result of the music detection process performed by the music detection devices 1, 2, and 3 and the result of the sound detection process performed by the sound detection device 4.

  In the sound field control device 50, the music determination unit 51 determines whether or not music is included in the acoustic signal from the result of the input music detection process. More specifically, when at least one of the music detection processing results from the music detection devices 1, 2, or 3 indicates that music is detected (that is, at least any one device) In the case where music is detected in (2), the music determination unit 51 determines that music is included in the acoustic signal. Then, the music determination unit 51 outputs the determination result to the memory 52. The memory 52 stores a determination result from the music determination unit 51 (hereinafter referred to as music detection information). In addition, the memory 52 stores the result of the voice detection process (hereinafter referred to as voice detection information) from the voice detection device 4.

  In the present embodiment, the music detection devices 1, 2, 3, and the voice detection device 4 are provided in the music voice detection device 55, and the acoustic signal input to the voice detection device 55 is the music detection device. Music detection processing or voice detection processing is performed by each of the devices 1, 2, 3, and the voice detection device 4.

  In the sound field control device 50, the sound field control determination unit 53 determines the content of the sound field control based on the plurality of music detection information and the sound detection information stored in the memory 52. The types of sound field control include “sound field control for music scene”, “sound field control for sound scene”, and “sound field control for scene including both music and sound”. As the state of sound field control, (A) “Sound field control for music scene” is performed, (B) “Sound field control for sound scene” is performed, and (C) “Sound field control” is performed. In addition to the state in which “sound field control for a scene including both music and sound” is performed, there are four types of states: (D) a state in which sound field control is not performed (hereinafter referred to as a neutral state). .

  FIG. 15 is a diagram illustrating state transition of sound field control in the sound field control device 50. FIG. 15 shows the four states (A) to (D). Also, as shown in FIG. 15, there are 16 state transition patterns (1) to (16).

  FIG. 16 is a diagram showing conditions for each state transition. FIG. 16 shows conditions for each state transition corresponding to (1) to (16) of FIG. For example, in the state (D) (ie, the neutral state), four state transitions (1), (2), (3), and (13) can occur as shown in FIG. And in the sound field control apparatus 50, the sound field control determination part 53 performs sound field control according to the conditions shown in FIG. 16 based on the music detection information stored in the memory 52 and the sound detection information. .

  In the present embodiment, the acoustic signals input to the music detection devices 1, 2, 3, and the voice detection device 4 are digitally encoded by PCM and divided into 1024 samples per frame. When the sampling frequency of the acoustic signal is 44.1 kHz, the time per frame is 23 ms (= (1 ÷ 44100) × 1024). In the music detection devices 1 to 3 and the voice detection device 4, the music detection process or the voice detection process is performed using a plurality of consecutive frames (approximately 5 frames). Information and voice detection information are stored approximately every 0.105 seconds (= 23 ms × 5 frames). Then, the sound field control determination unit 53 analyzes the latest 10 consecutive (about 1.05 seconds) music detection information and sound detection information stored in the memory 52, and determines the contents of the sound field control. To do.

  More specifically, the sound field control determination unit 53 detects the number of times that music has been detected (hereinafter referred to as the number of times of music detection) and the sound from the 10 times of music detection information and sound detection information. The number of times (hereinafter referred to as the number of times of sound detection) is counted, and the state of the sound field control (A) to (D) is switched according to the number of times of sound detection and the number of times of music detection.

  FIG. 17 is a flowchart showing the processing contents in the sound field control determination unit 53. The processing performed by the sound field control determination unit 53 will be described with reference to FIG.

  First, in S171, the sound field control determination unit 53 determines whether or not the current sound field control state is neutral (the state (D) above). If the sound field control state is neutral, it is determined in S172 whether or not the state transition condition (1) “music detection count <2 and voice detection count> 3” shown in FIG. 16 is satisfied.

  If it is determined in S172 that the condition (1) is satisfied, sound field control of the audio scene is performed (S173). That is, the state transits to the state (B) shown in FIG. On the other hand, if it is determined in S172 that the condition (1) is not satisfied, whether or not the state transition condition (2) “music detection count> 3 and voice detection count <2” shown in FIG. 16 is satisfied in S174. Determine whether or not.

  If it is determined in S174 that the condition (2) is satisfied, the sound field control of the music scene is performed (S175). That is, the state transits to the state shown in FIG. On the other hand, if it is determined in S174 that the condition (2) is not satisfied, the condition of the state transition condition (3) “music detection count> 2 and voice detection count> 2” shown in FIG. 16 is satisfied in S176. It is determined whether or not.

  If it is determined in S176 that the condition (3) is satisfied, sound field control for a scene including music and sound is performed (S177). That is, the state transits to the state (C) shown in FIG. On the other hand, if it is determined in S176 that the condition (3) is not satisfied, the sound field control in the neutral state is continued (S178).

  In the present embodiment, the conditions shown in FIG. 16 are stored in the memory 52 in advance. However, the conditions can be changed and set again, and are not particularly limited.

  Even in the state in which the sound field control of any one of (A) to (C) is already performed, the sound field control determination unit 53 similarly performs the processes shown in S179 to S201 according to the flowchart shown in FIG. Do. Also in this case, the sound field control determination unit 53 determines the transition state based on the state transition condition shown in FIG. The state transition when the sound field control of any of (A) to (C) above is performed according to the processing flow of S179 to S201 will be described below.

  As described above, the sound field control determination unit 53 determines whether or not the current sound field control state is neutral (state (D) above) in S171, but is determined not to be neutral in S171. In S179, it is determined whether or not the sound field control state is the control state for music scene (the state (A) above). If the sound field control state is the control state for the music scene, the state transition condition (7) “music detection count <2 and voice detection count> 5 (= 3 + 2)” shown in FIG. It is determined whether or not the number of sound detection times of the determination condition (1) satisfies an offset of +2.

  If it is determined in S180 that the condition (7) is satisfied, sound field control of the audio scene is performed (S181). That is, the state transits to the state (B) shown in FIG. On the other hand, if it is determined in S180 that the condition (7) is not satisfied, in S182, the state transition condition (10) shown in FIG. 16 “music detection count> 4 (= 2 + 2) and voice detection count> 4 It is determined whether or not (= 2 + 2) ”is satisfied.

  If it is determined in S182 that the condition (10) is satisfied, the sound field control of the scene including music and sound is performed (S183). That is, the state transits to the state (C) shown in FIG. On the other hand, if it is determined in S182 that the condition (10) is not satisfied, it is determined in S184 whether or not the condition (5) “number of times of music detection <2” shown in FIG. 16 is satisfied.

  If it is determined in S184 that the condition (5) is satisfied, neutral sound field control is performed (S185). That is, the state transits to the state (D) shown in FIG. On the other hand, if it is determined in S184 that the condition (5) is not satisfied, the sound field control of the music scene is continued (S186).

  Further, as described above, the sound field control determination unit 53 determines whether or not the sound field control state is the control state for the music scene (the state (A) above) in S179, but the music scene control state is determined in S179. If it is determined that the control state is not the control state for use, it is determined in S187 whether or not the sound field control state is a control state for the sound scene (the state (B) above). If the sound field control state is a control state for an audio scene, the state transition condition (8) “music detection count> 5 (= 3 + 2) and audio detection count <2” shown in FIG. It is determined whether or not the music detection count in the determination condition (2) satisfies an offset of +2.

  If it is determined in S188 that the condition (8) is satisfied, the sound field control of the music scene is performed (S189). That is, the state transits to the state shown in FIG. On the other hand, if it is determined in S188 that the condition (8) is not satisfied, the state transition condition (12) shown in FIG. 16 “music detection count> 4 (= 2 + 2) and voice detection count> 4 in S190. It is determined whether (= 2 + 2) ”(the offset of +2 for the number of times of music detection and the number of times of sound detection of the determination condition (3)) is satisfied.

  If it is determined in S190 that the condition (12) is satisfied, sound field control of a scene including music and audio is performed (S191). That is, the state transits to the state (C) shown in FIG. On the other hand, if it is determined in S190 that the condition (12) is not satisfied, it is determined in S192 whether or not the condition (4) “Number of times of voice detection <2” shown in FIG. 16 is satisfied.

  If it is determined in S192 that the condition (4) is satisfied, neutral sound field control is performed (S193). That is, the state transits to the state (D) shown in FIG. On the other hand, if it is determined in S192 that the condition (4) is not satisfied, the sound field control of the audio scene is continued (S194).

  Further, as described above, the sound field control determination unit 53 determines whether or not the sound field control state is the control state for the music scene (the state (B) above) in S187, but the sound scene control state is determined in S187. When it is determined that the control state is not the control state, the sound field control state is a control state for the scene including both music and sound (the state (C) described above). If the sound field control state is a control state for a scene including both music and audio, the state transition condition (9) “music detection count> 5 (= 3 + 2) shown in FIG. It is determined whether or not “number of detection times 2” (an offset of +2 with respect to the number of music detection times of the determination condition (2)) is satisfied.

  If it is determined in S195 that the condition (9) is satisfied, the sound field control of the music scene is performed (S196). That is, the state transits to the state shown in FIG. On the other hand, if it is determined in S195 that the condition (9) is not satisfied, in S197, the state transition condition (11) shown in FIG. 16 “music detection count <2 and voice detection count> 5 (= 3 + 2) It is determined whether or not (the offset of +2 with respect to the number of sound detection times of the determination condition (1)) is satisfied.

  If it is determined in S197 that the condition (11) is satisfied, sound field control of the audio scene is performed (S198). That is, the state transits to the state (B) shown in FIG. On the other hand, if it is determined in S197 that the condition (11) is not satisfied, in S199, the condition of the state transition condition (6) “music detection count <2 and voice detection count <2” shown in FIG. 16 is satisfied. It is determined whether or not.

  If it is determined in S199 that the condition (6) is satisfied, neutral sound field control is performed (S200). That is, the state transits to the state (D) shown in FIG. On the other hand, if it is determined in S199 that the condition (6) is not satisfied, the sound field control of the scene including both music and sound is continued (S201).

  As described above, the sound field control determination unit 53 makes a determination based on the state transition condition shown in FIG. 16, and switches the state of the sound field control according to the determination result. That is, the state of the sound field control transitions. The sound field control processing unit 54 corrects the input acoustic signal by performing signal processing according to the determination result by the sound field control determination unit 53, and reproduces a DA converter, an amplifier, a speaker, or the like (not shown) The output PCM is played back via the.

  As a result, for example, in the neutral state, as a result of analyzing the above-described 10 times of music detection information and voice detection information, the number of times of music detection is 8, the number of times of voice detection is 1, and both voice and music are detected. When the number of times of not being performed is one, it is determined in S174 of FIG. 17 that the condition (2) “music detection count> 3 and voice detection count <2” is satisfied. In this case, the sound field control determination unit 53 determines to start the sound field control of the music scene.

  Here, the condition (2) is set in consideration that the correct answer rate of the music detection process and the voice detection process is about 90%, that is, 10% has an erroneous determination. For this reason, although the number of times of sound detection is one, it is determined that the sound field control of the music scene is performed.

  Further, in the case where the sound field control of the music scene is performed, as a result of analyzing the music detection information and the voice detection information for the next 10 times, the number of music detections is 7, the number of music detections is 3, When the number of times that none of the music is detected is 3 (in this case, the number of times that both voice and music are detected at the same time is 3), the state transition condition (5) “ It is determined that the conditions (7) and (10) are not satisfied and the number of music detection times <2 ”is not satisfied. In this case, the sound field control determination unit 53 determines to continue the sound field control of the music scene.

  From this example, in the neutral state, when the number of music detections is 7 and the number of sound detections is 3, it is determined in S176 that the state transition condition (3) is satisfied, and both music and speech are In contrast, when the sound field control of the scene including the determination is performed, in the control state of the music scene, when the number of times of music detection is seven and the number of times of sound detection is three, the sound field of the scene including both music and sound It can be seen that the control is not judged. That is, in the state where the current state is performing some kind of sound field control (that is, a state that is not neutral), a condition for state transition that gives priority to the current state of sound field control is set. The reason why such a condition is set is as follows.

  As described above, the correct answer rate of the music detection process and the voice detection process is about 90%, and about 10% is erroneously determined. Therefore, the state of sound field control may not be appropriately switched. In addition, there may be a silent part of breathing in one conversation scene, or music may be mixed for only a few seconds with sound effects. Therefore, even if it follows the scene change (that is, switching of the sound field control state) in the order of several seconds (or hundreds of ms), it cannot always be said that the comfortable switching is made for the viewer. Rather, it will make viewers tired. Therefore, when sound field control has already been performed, the current scene, that is, the state of the current sound field control is given priority over the sound field control from the neutral state. Transition conditions are set. Thereby, the switching of the state of the sound field control can be performed an appropriate number of times. That is, it is possible to realize a configuration in which the state of sound field control is switched only in a subjective time segment that the listener recognizes as one scene.

  The present invention can also be expressed as follows.

(First configuration)
Means for dividing the input audio-acoustic signal into frames divided by a predetermined time; means for converting the frequency into each frame; means for logarithmically converting the horizontal axis of the spectrum converted to the frequency; and the logarithmically converted spectrum Means for calculating the autocorrelation value, means for calculating the correlation value of the calculated autocorrelation value with a past frame, means for comparing whether the correlation value is within a predetermined value between predetermined frames, and A first configuration characterized by comprising determination means for determining a music scene when it is within a predetermined value.

(Second configuration)
Means for dividing the input audio-acoustic signal into frames divided by a predetermined time; means for converting the frequency into each frame; means for logarithmically converting the horizontal axis of the spectrum converted to the frequency; and the logarithmically converted spectrum Means for detecting the maximum value of power, means for detecting a frequency having the detected maximum value, means for comparing the detected maximum frequency with the maximum frequency of a predetermined frame in the past, and the compared frequency band A second configuration characterized by comprising means for comparing whether or not is greater than or equal to a predetermined musical scale, and determination means for determining that the music scene is greater than or equal to a predetermined musical scale.

(Third configuration)
Means for dividing the input audio-acoustic signal into frames divided by a predetermined time; means for converting each frame into a frequency; and means for calculating power equal to or lower than a predetermined frequency of the spectrum converted into the frequency and above power Means for cumulatively adding the calculated low frequency power and high frequency power to a past frame; means for calculating the ratio of the cumulatively added low frequency power and high frequency power; and the self-addition of the cumulatively added low frequency power. Means for calculating a correlation value; means for calculating a ratio of the cumulative addition value of the low frequency power and the cumulative addition value of the high frequency power; and the calculated ratio is equal to or greater than a predetermined value and the low frequency power A third configuration characterized by comprising determination means for determining a music scene when the maximum autocorrelation value is not less than a predetermined value (approximately 0.2 seconds) and not more than a predetermined value (approximately 1.5 seconds).

(Fourth configuration)
In an apparatus that divides an input audio-acoustic signal into frames divided by a predetermined time and extracts a fundamental frequency by a cepstrum method, an instantaneous frequency method, or the like for each frame, the detected fundamental frequency and a detected basic of a plurality of past frames A means for comparing each frequency with a predetermined range (approximately 100 Hz to approximately 400 Hz), a means for detecting a change amount of the fundamental frequency when the predetermined range is satisfied, and a detected change amount within a predetermined range (approximately 120 Hz) And a determination unit that includes a determination unit that determines that the scene is an audio scene when it monotonously increases or monotonously decreases.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope shown in the claims. That is, embodiments obtained by combining technical means appropriately modified within the scope of the claims are also included in the technical scope of the present invention.

  Finally, each block of the music detection devices 1, 2, 3, and the voice detection device 4 may be configured by hardware logic, or may be realized by software using a CPU as follows.

  That is, the music detection units 10, 20, 30, and the voice detection unit 40 include a CPU (central processing unit) that executes a command of a control program that realizes each function, a ROM (read only memory) that stores the program, A RAM (random access memory) for expanding the program, a storage device (recording medium) such as a memory for storing the program and various data, and the like are provided. The object of the present invention is the program code (execution format program, intermediate code program, source program) of the control programs of the music detection units 10, 20, 30 and the voice detection unit 40, which are software for realizing the functions described above. Is supplied to the music detection devices 1, 2, 3, and the voice detection device 4, and the computer (or CPU or MPU) stores the program code recorded on the recording medium. This can also be achieved by executing reading.

  Examples of the recording medium include a tape system such as a magnetic tape and a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.

  Further, the music detection devices 1, 2, 3, and the voice detection device 4 may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. For example, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication network, satellite communication. A net or the like is available. Also, the transmission medium constituting the communication network is not particularly limited. For example, even in the case of wired such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line, etc., infrared rays such as IrDA and remote control, Bluetooth ( (Registered trademark), 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like can also be used. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the program code is embodied by electronic transmission.

  Since the music detection device and the sound detection device according to the present invention can detect a music scene and a sound scene for a program being broadcast or played, a television receiver that performs optimal sound field control according to the scene. It can utilize suitably in.

It is a block diagram which shows the structure of the music detection apparatus which concerns on this invention. It is a figure which shows the relationship between 12 average temperament scales and frequency. It is a figure which shows the frequency spectrum of a trumpet, (a) is a figure which shows the frequency spectrum of a certain time, (b) is a figure which shows the frequency spectrum 23 ms after the time which shows the frequency spectrum of (a). It is a figure which shows the frequency spectrum of a koto, (a) is a figure which shows the frequency spectrum of a certain time, (b) is a figure which shows the frequency spectrum after 23 ms of the time which shows the frequency spectrum of (a). . It is a block diagram which shows the structure of the music detection apparatus which concerns on this invention. It is a figure which shows an example of the relationship between a flame | frame and a power maximum musical scale. It is a block diagram which shows the structure of the music detection apparatus which concerns on this invention. It is a figure which shows the frequency spectrum of a drum. It is a figure which shows the time transition of the spectrum power total of 100 Hz or less of a drum. It is a block diagram which shows the structure of the audio | voice detection apparatus which concerns on this invention. It is a figure which shows the characteristic of an audio | voice, (a) is a figure which shows the time waveform in the speech in Japanese by a man, (b) is the time of the fundamental frequency (F0) calculated | required from the time waveform of (a). It is a figure which shows a change. It is a figure which shows the characteristic of an audio | voice, (a) is a figure which shows the time waveform in the speech in Japanese by a woman, (b) is the time of the fundamental frequency (F0) calculated | required from the time waveform of (a). It is a figure which shows a change. It is a figure which shows the characteristic of music, (a) shows a time waveform, (b) is a figure which shows the time change of the fundamental frequency (F0) calculated | required from the time waveform of (a). It is a block diagram which shows the structure of the sound field control apparatus which concerns on this invention. It is a figure which shows the state transition of the sound field control in a sound field control apparatus. It is a figure which shows the conditions of each state transition of sound field control. It is a flowchart which shows the processing content in a sound field control determination part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Music detection apparatus 2 Music detection apparatus 3 Music detection apparatus 4 Voice detection apparatus 5 Frame division part 6 Windowing part 7 Spectrum conversion part 10 Music detection part 11 Scale spectrum calculation part (spectrum calculation means)
12 Autocorrelation coefficient calculation unit (autocorrelation value calculation means)
13 Coefficient maximum value detection unit 14 Coefficient maximum value storage unit 15 Coefficient maximum value comparison unit 16 Variance calculation unit (numericalization means)
17 Music / non-music determination unit (music determination means)
20 music detector 21 scale spectrum calculator 22 spectrum power calculator (spectrum power calculator)
23 Power maximum value detection unit (maximum scale identification number detection means)
24 power maximum value storage unit 25 power maximum value comparison unit 26 variance calculation unit (numericalization means)
27 Music / non-music determination unit (music determination means)
30 Music detection unit 31 Ultra low frequency spectrum power calculation unit (low frequency spectrum power calculation means)
32 Ultra-low frequency spectrum power storage unit 33 Ultra-low frequency spectrum power autocorrelation coefficient calculation unit 34 Coefficient maximum value determination unit (frame interval detection means)
35 High frequency spectrum power calculation unit (High frequency spectrum power calculation means)
36 Ultra low frequency / high frequency power ratio calculation unit 37 Music / non-music determination unit (music determination means)
40 voice detection unit 41 logarithm spectrum calculation unit 42 cepstrum calculation unit 43 fundamental frequency calculation unit (basic frequency extraction means)
44 Fundamental frequency storage unit 45 Low-pass filter unit 46 Phrase component analysis unit (basic frequency change detection means)
47 Accent component analysis unit (basic frequency change detection means)
48 voice / non-voice judgment unit (voice judgment means)
50 Sound Field Control Device 51 Music Judgment Unit (Music Judgment Device)
52 Memory 53 Sound Field Control Determination Unit 54 Sound Field Control Processing Unit 55 Music Audio Detection Device

Claims (12)

Spectrum calculation means for calculating a frequency spectrum for each frame representing a predetermined time of the acoustic signal from the acoustic signal;
Autocorrelation value calculating means for calculating the autocorrelation value of the frequency spectrum;
Quantification means for quantifying the magnitude of variation of the maximum value of the autocorrelation value in a plurality of consecutive frames;
A music detection apparatus comprising: music determination means for determining that the acoustic signal is music when the magnitude of the variation is smaller than a predetermined threshold value. The spectrum calculation means includes:
The music detection apparatus according to claim 1, wherein a spectrum of each frequency corresponding to a musical scale is calculated. The autocorrelation value calculating means includes:
Using sp (i) (i = 0, 1,..., N−1), which is N data representing the spectrum, the M autocorrelation values are converted into the following autocorrelation function R1 (x ) (X = 1, 2,... M).

The above numerical means is
4. The music detection apparatus according to claim 2, wherein the variance of the maximum value is calculated and the variation is digitized. Spectral power calculation means for calculating the spectral power of each frequency corresponding to the scale for each frame representing a predetermined time of the acoustic signal from the acoustic signal;
A scale identification number for identifying each frequency is assigned to each frequency of the scale, and a maximum scale identification number for detecting a maximum scale identification number that maximizes the spectrum power among the scale identification numbers for each frame. Detection means;
Quantification means for quantifying the magnitude of variation in the maximum scale identification number in a plurality of consecutive frames;
A music detection device comprising: music determination means for determining that the acoustic signal is music when the variation is larger than a predetermined threshold value. The above numerical means is
6. The music detection apparatus according to claim 5, wherein variance of the maximum scale identification number is calculated and the variation is digitized. Low frequency spectrum power calculation means for calculating a low frequency spectrum power by adding a spectrum power of a frequency equal to or lower than a predetermined first threshold value or a frequency lower than the first threshold value for each frame from an acoustic signal;
A frame interval detecting means for detecting a frame interval at which the autocorrelation value of the low frequency spectrum power in a predetermined number of consecutive plural frames is maximum;
High frequency spectrum power calculating means for calculating a high frequency spectrum power by adding spectral power of a frequency equal to or higher than a first threshold value or a frequency higher than the first threshold value for each frame from the acoustic signal;
When the ratio of the low-frequency spectrum power to the high-frequency spectrum power is equal to or greater than a second predetermined threshold and the frame interval is within a predetermined range, the acoustic signal is determined to be music. And a music determination device. The frame interval detecting means includes
Using spp (i) (i = 0, 1,..., N−1) representing N data representing the low-frequency spectrum power, M autocorrelation values are expressed by the following autocorrelation function. The music detection device according to claim 7, wherein the music detection device calculates each value of R2 (x) (x = 1, 2,... M).

A fundamental frequency extracting means for extracting a fundamental frequency for each frame from the acoustic signal;
A fundamental frequency change detecting means for detecting a change in the fundamental frequency in a predetermined number of consecutive frames;
The fundamental frequency change detecting means detects that the fundamental frequency is changing monotonously, changing from monotonic change to constant frequency, or changing from constant frequency to monotone change, and Voice determination that determines that the acoustic signal is voice when the fundamental frequency changes within a predetermined frequency range and the change width of the fundamental frequency is smaller than the predetermined frequency width. And a voice detecting device. The voice determination means is
10. The acoustic signal is determined to be speech when the change in frequency is a change within a range of approximately 100 Hz to approximately 400 Hz and the width of the change in frequency is less than approximately 120 Hz. The voice detection device according to 1.   The number of times that the acoustic signal is determined to be music within a predetermined period by the music detection device according to claim 1, and the voice detection device according to claim 9 or 10, A sound field control apparatus characterized by switching the state of sound field control according to the number of times that the acoustic signal is determined to be speech within a predetermined period.   The sound field control device according to claim 11, wherein a condition for switching the sound field control is changed according to a controlled state.

Images