JP5051746B2 - Feature extraction apparatus and method, and program - Google Patents

Description

  The present invention is, for example, a voice recognition device, an emotion recognition device, a voice search system, a music search system, CM broadcast confirmation, device abnormality detection by acoustic diagnosis, home crime prevention by acoustic abnormality detection, The present invention relates to a voice and acoustic signal that can be used in a support device that notifies an abnormal sound, and a general one-dimensional time-series signal feature extraction device, method, and program.

With regard to the technique using higher-order local autocorrelation features (HLAC), research on feature extraction for image data including Patent Document 1 has been promoted and applied to moving images as in Patent Document 2. In addition, it has been developed for applications such as detection of abnormal operations. Non-Patent Document 1 discloses a time-varying power spectrum as a conversion method, and discloses an example in which an existing technology is integrated with HLAC using a Fisher weight technique. Furthermore, with respect to acoustic abnormality detection using other features, for example, as in Patent Document 3, a method corresponding to an individual purpose such as detection of abnormality in a nuclear reactor has been developed.
Japanese Patent No. 2982814 JP 2006-079272 A Japanese Patent Publication No. 4-032356 Shun Kato, Tetsuya Takiguchi, Yasuo Ariki: “Phoneme Recognition Using Higher-Order Local Autocorrelation Features Using Fisher Weight Map”, Acoustical Society of Japan 2005 Autumn Meeting, 1-P-10, pp.171-172 , 2005-09

With regard to feature extraction methods using higher-order local autocorrelation features (HLAC), research has been progressed mainly for image data, and no method for acoustic signals has been established. In the case of a method using other feature amounts, it is necessary to investigate and set feature amounts and determination methods depending on the target each time.
An object of the present invention is to establish a technique for utilizing the advantages of HLAC, which is a general-purpose feature extraction method with low dependency on an object, also for an acoustic signal.

  The feature extraction method of the present invention analyzes a one-dimensional time-series signal based on non-stationary chaos analysis, calculates a higher-order local autocorrelation coefficient from a two-dimensional image generated thereby, and extracts features. This high-order local autocorrelation coefficient is calculated by converting two-dimensional image information into binary information using a threshold obtained by calculating a histogram of a two-dimensional image generated by analyzing the one-dimensional time series signal. This is performed based on the binary image information generated by doing so. The one-dimensional time series signal is an audio signal or an acoustic signal, and the unsteady chaos analysis is performed by a recurrence plot method.

  Further, the feature extraction device of the present invention includes a signal analysis unit that analyzes a one-dimensional time-series signal based on non-stationary chaos analysis, and a threshold value obtained by calculating a histogram of a two-dimensional image generated thereby, A binarization processing unit that converts two-dimensional image information into binary information, and a parameter generation unit that calculates a higher-order local autocorrelation coefficient from the binary image information generated thereby.

  In addition, the feature extraction program of the present invention is based on a procedure for analyzing a one-dimensional time-series signal based on non-stationary chaos analysis and a threshold obtained by calculating a histogram of a two-dimensional image generated thereby. A procedure for converting image information into binary information and a procedure for calculating higher-order local autocorrelation coefficients from the binary image information generated thereby are executed by a computer.

  According to the present invention, HLAC features that reflect the unsteady dynamic characteristics of the target system are extracted, and features suitable for non-linguistic information such as objects with dynamic characteristics such as consonants and abnormal sounds, and emotions. An extraction method becomes possible, and improvement in accuracy in voice recognition and abnormal sound detection is realized.

  Hereinafter, the present invention will be described based on examples. FIG. 1 is a diagram showing a schematic configuration of a feature extraction apparatus of the present invention. In the present invention, for example, feature extraction is performed by applying high-order local autocorrelation to a two-dimensional feature value obtained by nonlinearly transforming a one-dimensional signal such as an acoustic signal or an audio signal. In the present invention, the non-linear transformation is a phase space reconstruction procedure described in 1) below. The feature extraction method according to the present invention includes a voice recognition device, an emotion recognition device, a voice search system, a music search system, CM broadcast confirmation, device abnormality detection by acoustic diagnosis, home crime prevention by acoustic abnormality detection, and hearing impairment It can be used for a support device that informs a person of an abnormal sound.

One of the transient chaos analysis methods is RP (Recurrence Plot). As shown in FIG. 1, a one-dimensional time-series signal such as an audio signal is analyzed based on a non-stationary chaos analysis such as a recurrence plot in a signal analysis unit to generate a two-dimensional image. The primary reason for focusing on chaos is that a dynamic system can be constructed from observation time series data, and the primary reason for focusing on non-stationary conditions is that many biological signals are fluctuated due to time fluctuations in noise and waveforms. Therefore, it is possible to obtain broad information (features) that cannot be obtained from a classical spectrum-based analysis method.
The reason for the unsteady chaos methodology is the integration of both, and the most important reason for choosing RP is that it is effective for sound phenomena and the phase space trajectory generated from the sound is the past. RP is to be used as an effective tool for visualizing its essential information. Pattern and classify these pictures using elements that visualize chaotic mechanical properties as patterns or pictures instead of numerical values and equations, and a feature extraction technology called HLAC that has been cultivated in pattern recognition, which is a completely different field. Thus, it is useful for “sound signal processing” to derive an equivalent dynamic structure without formulating a complex high-order dynamical system. For detection of abnormalities from one-dimensional signals other than sound (such as brain waves and myoelectric signals), other non-stationary chaos tools other than RP (CML (Coupled Map Lattice) in space-time chaos theory, DVS (Deterministic Versus Stochastic) method, Coefficient family reconstruction method etc. can be used.

The binarization processing unit calculates a histogram of the two-dimensional image generated by the analysis in the signal analysis unit, and converts the two-dimensional image information into binary information using a threshold selected based on the vicinity of the mode.
The parameter generation unit calculates higher-order local autocorrelation features (HLAC features) from the binary image information generated by the binarization processing unit, and performs feature extraction.

  As described above, the present invention applies the HLAC technique to the conventional two-dimensional image data after converting the characteristic amount on the two-dimensional coordinates by signal processing on the acoustic signal. As a result, HLAC features that reflect the unsteady dynamic characteristics of the target system are extracted, and a feature extraction method suitable for non-linguistic information such as objects that have dynamic characteristics such as consonants and abnormal sounds and emotions. It becomes possible to improve the accuracy in voice recognition and abnormal sound detection.

  Hereinafter, further description will be given according to the more specific flowchart of FIG. When the multi-valued image is output in the “RP analysis” part of FIG. 2, the RP analysis ends. The steps so far correspond to the signal analysis unit. Image processing is performed on the image generated by the signal analysis unit. The binarization processing unit for mode threshold selection binarization is part of image processing.

I Signal Analysis Unit The signal analysis unit analyzes a one-dimensional time series signal such as an audio signal based on a non-stationary chaos analysis such as a recurrence plot, and generates a two-dimensional image.

1) Procedure for phase space reconstruction A dynamic system is a system in which the state of the system at a certain time is determined by a differential equation or the like. Conversely, in the phase space, even if a plurality of state variables can be observed as time-series data, it is difficult to directly associate each state variable with a variable in the dynamic system. The voice data used in this demonstration example is also a time series data of one kind of sound pressure amplitude. If this is a dynamic system, it is converted from the time series data to a multi-dimensional coordinate based on a certain rule (this Is called reconstruction), and it is possible to grasp the behavior of dynamical systems in higher dimensions. The reconstruction destination when the observation time series data is reconstructed is called a phase space or a delayed coordinate system.

When trying to reconstruct a trajectory from time series data of a certain state variable obtained from a dynamic system of m degrees of freedom into a d-dimensional phase space, a d-dimensional vector is defined by d state variables for each appropriate time delay τ. create. Here, if the time series data is x (n) and the time is n, the delay coordinates can be defined as in equation (1).

  When this dimension d and time delay τ are called embedding parameters, and the fractal dimension of the reconstructed trajectory is assumed to be m, the attractor reconstructed into a phase space larger than 2m + 1 dimension is This is guaranteed by the Embleming of Takens. When the reconstructed trajectory is an embedding of the original dynamical system trajectory, the attractor of the vector X (n) in the reconstructed phase space has the feature quantity of the state variable x (n) of the original dynamical system. Preserved topologically (this is called a diffeomorphic relationship).

2) Estimation of embedding parameters
In order to create the delay coordinates defined by equation (1), the time delay τ and the embedding dimension d must be estimated as embedding parameters. The embedding dimension estimation method used in this demonstration example is shown in a), and the time delay estimation method is shown in b).

2-a) Estimation of embedding dimension by False Nearest Neighbor (FNN) method
In the FNN method, when a point that is close in a space reconstructed in a certain dimension is separated by changing the dimension, it is set as an error near point (FNN), and when the number approaches zero This is a method in which a dimension is an embedded dimension. There are the following two scales of criteria for whether or not they are close

(First standard scale)
As shown in the previous equation (1), the data is reconstructed from the time series data x (n) into an arbitrary d-dimensional phase space. The Euclidean distance from an arbitrary point x (n) on the trajectory in the phase space to the rth closest point x (n) ⁽ r) is expressed as R _d (n, r), and the equation (2) is obtained.
At this time, if a distance obtained by extending R _d to the d + 1 dimension is R _{d + 1} , Equation (3) is obtained.

The first reference scale is to compare the distance fluctuation ratio when the embedding dimension d is expanded to d + 1 with R _tol as a threshold [Equation (4) ].

(Second standard scale)
With _RA as the attractor size,
It expresses like this. At this time, the r-th neighbor point satisfying equation (5) is defined as an error neighbor point in the second reference scale.

By increasing the embedding dimension d, the FNN ratio (False Nearest Neighbor Ratio; FNNR) determined by these two reference measures decreases when the system has determinism. Thereby, the embedding dimension when FNNR is reduced to 0 becomes an appropriate embedding dimension. Kennel has proposed by numerical experiments that if the threshold selection is determined as R _tol ≧ 10, the number of FNNs can be evaluated quantitatively and the embedding dimension can be estimated quantitatively objectively.

2-b) Estimating time delay based on mutual information When creating delayed coordinates, if the time delay is short, a highly correlated attractor is reflected in the delayed coordinate system, and if the time delay is too long, the correlation is lost. It will be reflected as a random trajectory. At such times, many non-linear statistics derived from attractors are no longer reliable, and it makes no sense to determine chaos here. Accordingly, it is very important to find an appropriate time delay, and for this reason, it is generally performed to estimate from a mutual information amount highly correlated with the time delay. The mutual information amount M of the attractor in the delayed coordinate system is defined as in Equation (6), where P (x (n)) is the probability that the time series variable x (n) will appear.

  Since the mutual information M depends on the statistic of the state variable, random signals such as white noise have M = 0 and no correlation. On the other hand, if M = ∞, the signal is completely correlated. As an appropriate time delay, it is preferable to apply the time delay when reaching the first minimum in the transition graph of the mutual information amount. When the mutual information value clearly decreased monotonously with respect to the dimension, the value when the slope was 1 / e was taken as the delay time.

2-c) Procedure for mutual estimation of embedding parameters Since the embedding dimension d and the time delay τ are mutually dependent quantities, the following calculation is repeated until both parameters converge to calculate an equilibrium solution. The procedure is shown below.

[τ → d]
Give q kinds of appropriate time delays for each vowel. Here, the upper limit of q is set to 10. For the total q types of τ in the range of 1 ≦ τ ≦ q, FNN analysis is performed to estimate q types of embedding dimension values for each τ. This set of embedded dimension values is called a dimension sequence.

[d → τ]
Next, the time delay is estimated from the mutual information amount of the time series data based on the r types of embedding dimensions. Here, r is a symbol representing the type of embedding dimension, as in q above. That is, this is equivalent to calculating the mutual information regarding the total of r types of embedding dimensions in the range of 1 ≦ d ≦ r, and obtaining r types of values by delaying the first minimum value. This set of time delay values is called a delay sequence.

[d⇔τ]
If the system is deterministic, the assumption is made that the embedding parameters are uniquely determined.
If it does so, a dimension row | line | column and a delay row | line | column will have an intersection on the two-dimensional plane of d and (tau), and this will become an optimal value.

  When there are multiple intersections, the intersection point closest to the origin of the d-τ plane is selected so that both the values of (d, τ) are small. This is because the FNN method is a method for estimating the minimum embedding dimension of the system and considers the conditions for using the first local minimum value of the mutual information as the delay time when the dimension is changed. is there. Regarding changes in the mutual information amount, there are many cases where a plurality of local minimum values appear, and determination of local minimum values may be difficult. In such a case, the value of the delay time estimated to be appropriate has a large range. In addition, the first minimum value is not always optimal in the sense that the nonlinear correlation of the phase space trajectory is maintained. For example, if the first and second minimum values have the same value and both τ are separated, there will be a difference in the delay time, but this difference may change the determination of chaos. is there. Alternatively, the system that is originally targeted has stochastic properties and may not be suitable for chaos analysis, so care must be taken when calculating the delay time. However, in general, it is common to use the first minimum value with the smallest delay time.

3) RP analysis
The RP method replaces the Euclidean norm (vector length in multiple dimensions, hereinafter referred to as the norm) information determined by an arbitrary position vector on the orbit as it is as the pixel density of the two-dimensional image, so it is independent of the higher-order trajectory. This is one of the analytical methods that can be estimated. The selection of the position vector is generally arbitrary, but in this demonstration experiment, a vector related to “time” is used. In the RP analysis method, this temporal change is visualized as a spatial arrangement on the image, so it is possible to instantly read at which point on the trajectory the change.

Let N be the number of time-series data reconstructed in the phase space. As shown in FIG. 3, the norm between the displacement vectors v_ {i} and v_ {j} that form the orbit in the phase space at the left end of the figure at the grid point (i, j) on the RP plane with N × N elements Think of coloring by (distance). Here, i and j represent element numbers on the RP plane, and are the start point index and end point index of the vector for calculating the norm. A start point index i (i = o in FIG. 3) is determined on the RP plane, and each norm is calculated from j = 1 to j = N, which is a direction of time progression as an end point index. When the starting point is replaced with i = o + 1, the starting point index is shifted in the i direction by one column on the RP plane, and the calculation in the j direction is similarly performed. In this way, the time evolution n of the orbit progresses stepwise from the front of the figure to the back (j direction) and from the left to the right (i direction) on the RP plane. The search method in this demonstration experiment was the round robin method, and the index of myself (i = j) was excluded. The pixel D (i, j) on the RP plane according to the calculated value of each norm divides the orbit from the maximum norm to the minimum norm for each fixed class value, and the norm belonging to each class is represented by the color r (k ) [Expression (7)] decided by assigning.
(However, when i ≦ j, 2 ≦ k <r, k = 0, δ_ {k} = 0)
Here, k is the number of colors, and r is a color. [delta] _k is called the resolution parameter, a class value which is divided by color index k described above, represented by the following equation (8).

3-a) Restriction of resolution parameter δ _k The minimum norm min | v _i -v _j | is shown in black and the maximum norm max | v _i -v _j | is shown in white, and the norm value between them is gradationized. It means that the norm between the displacement vectors on the trajectory is larger as the color of the corresponding point is closer to white, and the norm is smaller as it is closer to black. In addition, the range of visualization is expanded by changing the gray scale to the RGB scale according to the display application, and more detailed feature amount extraction can be performed by performing image processing on the obtained image. When the resolution parameter δ _{k is set} to a very small value, a slight change in the displacement vector on the trajectory can be captured from the obtained image. However, the resolution parameter δ _k has a constraint of δ _k <1/2 ^B. Here, B represents the number of quantization bits. Theoretically, δ _k can be set to infinitely small if it is a temporally continuous trajectory, but in the case of discrete data handled in this demonstration experiment, the phase space trajectory is also a set of discretized displacement vectors. There is also a lower limit of 1/2 ^B on the class width that divides the stretched norm. In the case of RGB resolution (3 × 8 = 24 bits), since the quantization bit number of the audio data to be used is 16 bits, the δ resolution is finer.

3-b) Multi-resolution analysis Multi-resolution analysis classifies norms from small to large scales by continuously varying the bandwidth of δ. In equation (7), the frequency distribution is calculated for k up to the upper limit r of a certain color number. For example, in the case of r = 256, one image fixed at a resolution of 256 steps is made, and one color frequency distribution (histogram) is calculated from this image. For the sake of convenience, an image having a specific resolution is expressed by the term “hierarchy”. When the upper limit value is changed from r = 256 to 128, the resolution of the generated image is fixed at r = 128, and one histogram with r = 128 is calculated. Two frequency distributions are generated from images between two layers of r = 256 and r = 128, and only w histograms are generated from a plurality of w types of layers. In this way, sequentially performing RP analysis while changing the resolution of the histogram h by w layers for each upper limit resolution is referred to as RP multi-resolution analysis. Dividing the histogram h into w levels is expressed as f ^(w) = h (k). k is a band parameter.

For example, f ^{(w = 2)} is a binary distribution up to resolution r = 2, f ^{(w = 3)} is a ternary distribution up to resolution r = 3, and f ^{(w = 256)} is a 256-value resolution distribution. think of. As a result, in the RP analysis, the pixel density distribution is calculated from RP images with various resolutions, and the change due to the hierarchy of the most frequent values (modes) of the entire distribution (hereinafter referred to as mode transition) can be noted

II Binarization processing unit (image processing)
The binarization processing unit calculates a histogram of the two-dimensional image generated by the analysis in the signal analysis unit, and converts the two-dimensional image information into binary information using a threshold selected based on the vicinity of the mode. .

Mode threshold selection type binarization method
W histograms are generated from w images generated by RP multi-resolution decomposition. When determinism is seen in the dynamic characteristics of the original time series data, the shape of this histogram is convex upward empirically. When the determinism is low and probabilistic randomness is mixed, the histogram tends to be flat. When performing deterministic chaos analysis, the mode which is the peak value of the above-mentioned histogram is searched using the property of a convex shape on the histogram. Since the histogram is the frequency of pixel density, this peak corresponds to picking up the number of colors contained most in the image. Since the mode value does not appear as a peak only with one color number r, the number of colors before and after the vertex can be easily selected within a plus or minus range with the vertex prospering. As an example, if the vertex is r and the selection range is 3, the number of colors can be narrowed down within a range of r ± 3. Here, when the colors corresponding to the negative ranges r-1, r-2, r-3 are replaced with white, and the colors corresponding to the positive ranges r + 1, r + 2, r + 3 are replaced with black, binary values are obtained. Can be converted to an image. This is to observe the behavior of multi-valued images by multi-resolution decomposition, and when the mode transition of each histogram converges, it means that it converges even if it increases further after a certain resolution. (Characteristic) is a measure of the saturation state expressed sufficiently by the number of colors at that time.

At this point, one histogram is taken out, and the colors before and after the mode correspond to the norm class with the highest frequency of the distance between two points scattered in the phase space. The distance between two points in the phase space indicated before and after the mode value is the class with the largest contribution rate that characterizes the geometric structure of the attractor. This operation takes out only the essential information from the class of norm that is most suitable for characterizing an attractor with all kinds of complex information, and constructs a binary image again. It means that it can be the most promising candidate for characterizing the dynamical system of speech.
After that, by recognizing this image as a pattern, it is possible to finally determine and recognize the audio signal.

III Parameter Generation Unit The parameter generation unit calculates a higher-order local autocorrelation coefficient from the binary image information generated by the binarization processing unit, and thereby performs feature extraction.

Higher-order Local Auto-Correlation (HLAC)
A conventional autocorrelation function with translation invariance is made higher-order and expressed by equation (9), which is called an Nth-order autocorrelation function. N is the order of the correlation coefficient.
In this paper, f (r) is considered to obtain autocorrelation features from images obtained from RP images instead of one-dimensional audio time series. At that time, the calculation is performed with high-order local autocorrelation up to N = 2 order while narrowing down to a local 3 × 3 pixel region in the image. The pixel arrangement pattern in the local area is called a mask pattern, and 25 types of mask patterns are used for a binary image, and 35 types of mask patterns are used for a 256-value grayscale image. I will take the sum of products. FIG. 4 illustrates 35 types of mask patterns of gray scale images. Correlation coefficient orders up to N = 2 are shown, and the numbers in the lattice represent the powers of the pixel values.

It is a figure which shows schematic structure of the feature extraction apparatus of this invention. It is the flowchart which actualized the characteristic extraction procedure of this invention more. It is a figure explaining RP analysis. It is a figure which illustrates 35 types of mask patterns of a gray scale image.

Claims (6)

The time series signal 1D reconstituted phase space of any dimension, the data in phase space by analyzing based on the non-stationary chaos analysis, norm between displacement vectors constituting the trajectory of the phase space Is converted into binary information by a threshold obtained by generating a two-dimensional image by calculating the pixel density of the two- dimensional image, and calculating a histogram of the two-dimensional image. the higher-order local auto-correlation coefficient calculated by applying a mask pattern on the binary image information generated, feature extraction process consisting of extracting features. The feature extraction method according to claim 1, wherein the one-dimensional time series signal is an audio signal or an acoustic signal, and the unsteady chaos analysis is performed by a recurrence plot method. By reconstructing a one-dimensional time-series signal into a phase space of an arbitrary dimension and analyzing the data in the phase space based on non-stationary chaos analysis, the norm between displacement vectors constituting the trajectory in the phase space A signal analysis unit that generates a two-dimensional image by replacing the pixel density of the two-dimensional image with
A binarization processing unit that converts two-dimensional image information into binary information according to a threshold obtained by calculating a histogram of the two-dimensional image;
A feature extraction apparatus comprising a parameter generation unit that calculates a higher-order local autocorrelation coefficient by applying a mask pattern from binary image information generated by conversion and extracts a feature. The feature extraction apparatus according to claim 3 , wherein the one-dimensional time series signal is an audio signal or an acoustic signal, and the unsteady chaos analysis is performed by a recurrence plot method. By reconstructing a one-dimensional time-series signal into a phase space of an arbitrary dimension and analyzing the data in the phase space based on non-stationary chaos analysis, the norm between displacement vectors constituting the trajectory in the phase space To generate a two-dimensional image by replacing the pixel density of the two-dimensional image with
A procedure for converting the two-dimensional image information into binary information according to a threshold obtained by calculating a histogram of the two-dimensional image generated thereby;
A feature extraction program for calculating a higher-order local autocorrelation coefficient by applying a mask pattern to binary image information generated by conversion , and executing a procedure for extracting features by a computer. The feature extraction program according to claim 5 , wherein the one-dimensional time series signal is an audio signal or an acoustic signal, and the unsteady chaos analysis is performed by a recurrence plot method.