JP3262204B2 - Frequency component extraction device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a sound spectrogram obtained by frequency-analyzing an acoustic signal and analyzing the acoustic signal or recognizing an acoustic event such as a sound, a musical instrument sound, a bell, and other object sounds. The present invention relates to an apparatus for extracting a frequency component that is important in reducing noise while reducing the influence of a noise component.

[0002]

2. Description of the Related Art Conventionally, regarding extraction of a frequency component from an acoustic signal, a power threshold is set on a spectrogram, a point exceeding the threshold is selected from spectral maximum points in the frequency axis direction, and this is determined by time. There is known a method of connecting in the axial direction to generate a frequency component.

A method of analyzing a sound signal or recognizing an acoustic event by assuming a model of a sound source waveform and a transfer characteristic of a target sound source and estimating parameters of the model, or assuming a spectral characteristic of the sound source. There is known a method of removing a noise component in a spectrum by using a filter designed to pass only a spectrum having the characteristic, and using the filter for analysis and recognition.

[0004]

In the former frequency component extraction method, accurate analysis of the acoustic signal is required because of reverberation and noise components inevitably included in the audio signal and arithmetic errors generated in the frequency analysis means. In addition, there is a disadvantage that it is difficult to recognize an acoustic event included therein.

[0005] The latter method is effective when the waveform and transfer characteristics of the sound source are known, or when the spectral characteristics of the sound source are known and appropriate modeling is possible. Effective processing cannot be performed on sound signals containing sound sources that do not correspond exactly, so sufficient frequency component extraction should still be performed on sound signals containing unknown sound sources or sound signals containing multiple sound sources. There was a drawback that it was difficult to do.

Therefore, in each of the above-mentioned methods, it is necessary to specify in advance what kind of sound is to be generated, such as when reverberation and noise components in a sound signal and calculation errors are large, and in actual indoor and outdoor environments. Is difficult, it is difficult to expect sufficient frequency component extraction processing in light of analysis of acoustic signals and recognition of acoustic events included in the acoustic signals or other purposes.

It is an object of the present invention to provide a frequency component extraction apparatus which introduces a rational constraint condition and extracts a frequency component contrary to the constraint, that is, a frequency component with reduced influence of noise.

[0008]

SUMMARY OF THE INVENTION A frequency component extracting apparatus according to the present invention performs frequency analysis on an acoustic signal and outputs a sound spectrogram discretized with respect to frequency and time, and frequency analyzing means. Candidate point extracting means for scanning the power value at each time point in the frequency axis direction with respect to the sound spectrogram obtained by the above to obtain a maximum point of the spectrum and extracting this as a candidate point of a point that should constitute a frequency component, One or more points located in the vicinity of the candidate point are scanned, and the state of these nearby points and the candidate points means whether these neighboring points and the candidate points are points constituting frequency components. State evaluation means for calculating an energy value to be defined; and changing the state of a candidate point or a nearby point to change the energy within a predetermined time window. State optimization means for minimizing the sum of the energy values, and whether or not the candidate point forms a frequency component is determined based on the output of the state optimization means, and it is determined that the candidate point forms a frequency component. time for the point, the frequency, and to determine the continuity of with respect to a power value, possess a time direction connecting means for connecting the points to each other with continuity, state evaluation
Means for inputting the candidate points extracted by the candidate point extracting means.
Force, set the time window, initialize the energy value,
Select an unevaluated candidate point within the time window, and select
Obtains the state of the point of, and the potential corresponding to that state
Get the value and add it to the energy value.

The present invention seeks to suppress the extraction of components that violate the constraints by introducing a loose constraint on the local structure as a potential. However, when a local constraint is introduced in a certain portion, the other portion may be wrinkled (the other portion may be in a state contrary to the constraint). That is, it is necessary to find a state that satisfies the constraint as a whole as a whole. For this purpose, the present invention uses a state evaluation unit and a state optimization unit.

[0010]

According to an embodiment of the present invention, the energy value is the sum of the energy determined by the power value and the potential determined by the local structure of the line spectrum for all candidate points in the time window.

[0012] Other frequency component extraction device of the present invention, the acoustic
Perform a frequency analysis on the signal and determine the frequency and time
Output frequency of discretized sound spectrogram
Number analysis means, and the number of samples obtained by the frequency analysis means.
Power at each time point for und spectrogram
Scan the value in the frequency axis direction to find the maximum point of the spectrum
Therefore, this is extracted as a candidate point of a point that should constitute a frequency component.
A candidate point extracting means for outputting the candidate points;
One or more points are scanned, and these neighboring points and the candidate points are
The point that constitutes a frequency component.
Energy defined by the state of neighboring points and the candidate points
State evaluation means for calculating the value, and the candidate point or the neighboring point
By changing the state, the time within the preset time window
State optimizing means for minimizing the sum of the energy values
And the candidate point is determined based on the output of the state optimization means.
Determines whether to configure the wave number component and configures the frequency component
The time, frequency, and power
Judgment of continuity with respect to values
Possess the time direction connecting means for connecting, the state optimization means, to obtain the status of candidate points from the state evaluating means sets a certain time window, select one point from the time window ,
If the state of the point is changed, check whether the energy value is smaller than before the change, and if the energy value is smaller, actually change the state.If the energy value is larger due to the state change, The probability value is calculated by dividing the energy value before the state change by the energy value after the state change.
And, the operation up to carry out the state change in the probability repeated until the entire energy value is below the set value, outputs the state at that time as the state of the best candidate point.

According to an embodiment of the present invention, the time direction connecting means inputs a set of candidate points output by the state optimizing means, arranges time windows in order of time, combines them, and selects one candidate point. Then, for the selected point, it is checked whether there are any candidate points near the time, frequency and power values, and if so, processing for connecting the candidate points is sequentially performed for all the candidate points.

[0014]

Next, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram of a frequency component extracting apparatus according to one embodiment of the present invention.

The frequency component extracting apparatus of this embodiment includes a frequency analyzing circuit 1 as frequency analyzing means, a candidate point extracting circuit 2 as candidate point extracting means, a state evaluating circuit 3 as state evaluating means, State optimization circuit 4
And a time direction connection circuit 5 as a time direction connection means, which inputs an audio signal and outputs a frequency component.

The frequency analysis circuit 1 is a circuit that performs a frequency analysis on an acoustic signal by a known means, and outputs a sound spectrogram discretized with respect to frequency and time. FIG. 2 shows an output example of the frequency analysis circuit 1.

The candidate point extracting circuit 2 scans the power value at each time point in the frequency axis direction in the sound spectrogram output from the frequency analyzing circuit 1 to detect a local maximum point, and uses this as a candidate for a point constituting a frequency component. This is a circuit that outputs points (referred to as frequency component candidate points). FIG. 3 shows an output example of the candidate point extraction circuit 2.

The state evaluation circuit 3 is a state of a frequency component candidate point obtained as an output of the candidate point extraction circuit 2 (means whether the frequency component candidate point is a point that actually constitutes a frequency component). , The energy value is calculated from the viewpoint of the local structure, that is, based on the state of the point and the state of surrounding points, and the energy value belonging to a certain time window (a certain time range set on the spectrogram) Is calculated and output. The output energy value U (ω) (where ω represents the state of whether or not it is a frequency component) is represented by a potential energy U _g (ω) based on a power value and a line spectrum as shown in Expression (1). Of the energy U _l (ω) provided by the local structure of

[0020]

(Equation 1) Of the two terms on the right side of equation (1), first, potential energy U _g
(Ω) is given by the following equation (2).

[0021]

(Equation 2) Here, i is a discrete time, j is a discrete frequency, V _g is a constant, g _{i, j} is a power value, and h _{i, j} is 1 if a frequency component exists, and if not, It is a variable that takes a value of 0.

Next, the energy U _l (ω) is obtained by adding a potential V _c (value appropriately determined by analytical learning or trial and error) determined by the local structure of the line spectrum for all points within the time window. . That is, if a point on the spectrogram is represented by s and a set of points near each other is represented by C, the energy U _l (ω) becomes

[0023]

(Equation 3) It is. FIG. 4 shows three points 4-7 and 4-4 adjacent to the candidate point 4-10 obtained by the candidate point extraction circuit 2 at time i on the frequency axis at time i-1 as neighboring points.
When -8 and 4-9 are selected, in order to obtain an energy value defined by the states of these points, six states of these four points are illustrated, and the potential V for each state is illustrated. _This shows a setting example of _c . In the example of FIG. 4, the rise of the frequency components, the value of V _c is larger than in the case of continuation of the frequency components in the fall, the
This is because it is necessary to introduce a property that the frequency component tends to last for a certain period of time (rising and falling rarely occur) into the processing. Further, the reason why the value is small without the frequency component is to prevent the noise component from being extracted by favoring a state where the frequency component is not present than a state where the frequency component is present.

In defining the energy value,
In addition, for example, a point having a frequency value of n octaves up and down or n / 2 octaves up and down (n is an integer) is regarded as a nearby point, or a point over a range wider than the range of FIG. You can also set a potential value.

The state optimization circuit 4 refers to the output of the state evaluation circuit 3 and changes the state (whether or not to constitute a frequency component) of the frequency component candidate point to minimize the total evaluation value (the evaluation function). Is inverted, and the maximum is equivalent).

The time direction connection circuit 5 includes a state optimization circuit 4
It is determined whether or not the frequency component candidate point is a point that actually constitutes a frequency component by referring to the output (the minimum evaluation total value). Are connected in the time direction among the points and output as a frequency component. In determining continuity, the time, frequency value, and power value of each point are referred to. FIG. 5 shows an output example of the time direction connection circuit 5.

Next, the flow of processing in each of the circuits 2, 3, 4, and 5 will be described with reference to flowcharts shown in FIGS.

The candidate point extraction circuit 2 inputs a spectrogram discretized with respect to time and frequency (step 101), and performs processing at each time. The number of discretized points on the frequency axis is N (the number of frequencies on the vertical axis in FIG. 2; N = 11 in FIG. 2), and at each time point n on the frequency axis is examined sequentially. First, n is set to 1 (step 10).
2) Compare the values of n and N-1 (step 103),
If n <N-1, the value of n is increased by 1 (step 10).
4), P (n-1), P (n), and P (n + 1) are obtained (step 105). Here, P (n) represents the power value at the n-th point on the frequency axis at that time. Next, “P (n−1) <P (n) and P (n)> P
(N + 1) "is true (step 106). If true, the n-th point is extracted as a candidate point (step 107), and the process returns to step 103. If false, the process returns to step 103.
If n <N−1 is not satisfied in step 103, all candidate points at that time have been extracted, so the candidate points are output (step 107), and the processing at that time is ended to proceed to the processing at the next time. move on.

The state evaluation circuit 3 inputs the candidate points extracted by the candidate point extraction circuit 2 (step 20).
1). First, a time window is set, an energy value is initialized (step 202), and an unevaluated candidate point existing in the time window is searched (step 203). If there is an unevaluated candidate point, the candidate point is selected (step 204), and the state of a point near the candidate point is obtained (step 205). The potential value corresponding to the state is retrieved from a preset table and acquired (step 206), and the acquired value is added to the energy value (step 207). The process returns to the unprocessed evaluation point search step (step 203). If the evaluation has been completed for all the candidate points existing in the time window, the processing in the time window ends, and the processing is repeated by moving the time window. When the processing has been performed for all the time windows, the processing ends.

The state optimizing circuit 4 acquires the state of the candidate point through the state evaluating circuit 3 and sets a certain time window (step 301). Next, one point is randomly selected from this time window regardless of whether or not it is a candidate point (step 302). Consider changing the state of the selected point. That is, if the point is a candidate point, it is assumed that the point is not a candidate point, and if that point is not a candidate point, it is temporarily assumed that the point is a candidate point. When such a state change is performed, it is checked whether or not the energy value is smaller than before the change (step 303). At this time, steps 205, 206 and 207 of the state evaluation circuit 3 are used. If the energy value decreases, the state change is actually performed (step 304). If the energy value increases due to the state change, q is a value obtained by dividing the energy value before the state change by the energy value after the state change, and the state is changed with a probability q (step 305). Steps 302 to 305 are repeated until the convergence is determined as the total energy change falls below a set value (step 306). If the convergence has occurred, the state at that time is regarded as the state of the optimal candidate point, and the processing is terminated.

The time direction connection circuit 5 receives a set of candidate points (referred to as frequency component points) output by the state optimization circuit 4 (step 401). First,
The time windows divided by the state optimization circuit 4 are arranged in order of time and integrated (step 402). Next, a search is made for frequency component composing points for which connection processing has not yet been performed (step 4).
03), one unprocessed frequency component composing point is selected (step 404). For the selected point, it is checked whether there is a frequency component point near the time, frequency, and power value (step 405), and if so, those frequency component points are connected (step 406), Return to step 403. If not, the process returns to step 403. In step 403, if there is no unprocessed frequency component constituent point, the process ends.

[0032]

As described above, according to the present invention, it is possible to introduce a reasonable constraint condition (frequency component continuity) for frequency component extraction. By optimizing the state of the frequency components in the above, there is an effect that it is possible to extract the frequency components contrary to the restrictions, that is, the frequency components in which the influence of noise is reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram of a frequency component extraction device according to an embodiment of the present invention.

FIG. 2 is a diagram showing an output example of a frequency analysis circuit 1.

FIG. 3 is a diagram showing an output side of a candidate point extraction circuit 2.

FIG. 4 is a diagram showing a setting example of a potential value in a state evaluation circuit 3.

FIG. 5 is a diagram showing an output side of a state determination circuit 5;

FIG. 6 is a flowchart of processing of a candidate point extraction circuit 2;

FIG. 7 is a flowchart of a process of a state evaluation circuit 3;

FIG. 8 is a flowchart of the processing of the state optimization circuit 4;

FIG. 9 is a flowchart of the processing of the time direction connection circuit 5;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 Frequency analysis circuit 2 Candidate point extraction circuit 3 State evaluation circuit 4 State optimization circuit 5 Time direction connection circuit 101-107, 201-206, 301-306, 4
01-406 steps

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G01H 3/08 G01H 3/06

Claims (4)

(57) [Claims] 1. A frequency analysis means for performing frequency analysis on an acoustic signal and outputting a sound spectrogram discretized with respect to frequency and time; and a power at each time point with respect to the sound spectrogram obtained by the frequency analysis means. Scanning a value in the frequency axis direction to obtain a local maximum point of the spectrum, extracting the local maximum point as a candidate point of a point that should constitute a frequency component, and one or more candidate points located near the candidate point. State evaluation means for scanning points, calculating whether or not these neighboring points and the candidate points are points constituting frequency components, and calculating an energy value defined by the states of the neighboring points and the candidate points; By changing the state of the candidate point or neighboring point,
State optimizing means for minimizing the total value of the energy values within a preset time window; and determining whether or not the candidate point forms a frequency component based on an output of the state optimizing means. , the time for the points that have been determined to constitute a frequency component, the frequency, and to determine the continuity of with respect to a power value, possess a time direction connecting means for connecting the points to each other with continuity, the condition evaluating unit, Extracted by the candidate point extracting means
Input candidate points, set time window, and energy value
Is initialized, and the unevaluated candidate points existing in the time window are
Select, get the state of points in the vicinity, and respond to that state
Obtain potential value and add it to energy value
That the frequency component extracting unit. Wherein said energy value, the energy determined by the power value, a potential determined by the local structure of the line spectrum, the sum but the sum of all the candidate points in the time window, the frequency of claim 1, wherein Component extraction device. 3. A frequency analysis is performed on an acoustic signal, and a frequency analysis is performed.
Sound spectrum log discretized with respect to wave number and time
Frequency analysis means for outputting a ram, and a sound spectrum obtained by the frequency analysis means
Power value at each time point on the frequency axis
Scan in the direction to find the maximum point of the spectrum,
Candidate point extraction to extract several components as candidate points
Scanning means and one or more points located near the candidate point
And the points at which these neighboring points and the candidate points constitute frequency components
State of the neighboring point and the candidate point, which means whether or not
State evaluator that calculates the energy value defined by
By changing the step and the state of the candidate point or neighboring point,
Minimize the sum of the energy values within a preset time window
State optimization means to be converted, and the candidate point is set to a frequency based on the output of the state optimization means.
Determine whether or not to configure the frequency component and configure the frequency component
, Frequency, and power values for the points determined as
Judge the continuity of
Time direction connecting means, and the state optimizing means comprises a
Get the status, set a certain time window,
If you change the state of a point after selecting one of them,
Check if the energy value is lower than before the change, and
If the energy value decreases, the state change is actually performed.
If the state change does not increase the energy value
If the energy value before the state change is
The value is divided by the energy value as the probability, and the state is changed with the probability.
Operation, the whole energy value falls below the set value
Until the optimal candidate point state is reached.
Frequency component extraction device that outputs the result . 4. The time direction connecting means inputs a set of candidate points output by the state optimizing means, arranges time windows in order of time, combines them, selects one candidate point,
For the selected point, check whether there are any candidate points near the time, frequency, and power values, and if so, perform the process of connecting those candidate points sequentially for all candidate points,
Frequency component extraction device according to claim 1 or 3 wherein.