JP2000285104A - Method and device for signal processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform signal processing that extracts a common feature which does not depend on the physical size of an object and processes it from a signal of an analysis object by performing wavelet transformation of an input signal, making its output synchronize with the input signal by a computer, performing Mellin transform and extracting a signal characteristic. SOLUTION: This method includes a wavelet transformation step in which a computer performs wavelet transformation of an input signal and a characteristic extraction step in which the characteristic of a signal is extracted by such a manner that the computer synchronizes an output of the step performing wavelet transformation with the input signal and performs Mellin transform of it. In this device, stabilizing wavelet processing part 2 performs wavelet transformation processing of an input signal 1. A Mellin transform processing part 3 performs Mellin transform of the signal 1 which is outputted from the part 2 and is subjected to stabilizing wavelet processing. A signal processing part 4 performs prescribed signal processing of an output of the part 3 and outputs the results 5.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement in analysis of time series data, which has been conventionally performed by a statistical method such as an autoregressive model or a Fourier transform. The present invention provides, for example, music recognition, voice-based personal recognition, voice recognition,
Analysis of architectural acoustics, as well as voice or music signal analysis,
It can be applied to encoding, signal separation, and signal enhancement processing.
The present invention is not limited to acoustic signals and the like. It is also widely applied to the analysis of sensor signals for collecting sequence data.

[0002]

2. Description of the Related Art Conventionally, signal processing has been performed as a basis of general information processing in order to obtain a spectrogram, that is, a "time-frequency expression". Even if you use high-speed digital conversion (for example, fast Fourier transform),
Even when using linear prediction analysis, what is sought is a vector that directly corresponds to a spectrum as a frequency expression at a certain point in time, and by having this in a time series, an expression equivalent to a spectrogram is used. Will be. These representations are derived from the spectral representation of the signal starting from the Fourier transform. For example, the most commonly used representation for features of audio signals will be a sound spectrogram. The sound spectrogram expresses a temporal change of a voice spectrum in an easy-to-read manner using a gray scale graphic expression, a contour line expression, a color display, or the like.

It is said that the spectral expression can express the characteristics of the signal better than the signal itself by the waveform itself, and that the human auditory system is not so sensitive to the relative phase relation of the signal composed of a plurality of sine waves. And the fact that a calculation method that can calculate them efficiently has been established, and it has come to be widely used because the information processing of speech and the like has been properly matched.

Heretofore, in various signal processings, performance has been improved to the utmost by viewing almost everything in the above-mentioned spectral representation. However, there is a feeling that performance improvement is already approaching its limit.
For example, a speech recognition apparatus generally needs to learn a lot of human speech in advance. However, even if a child's voice is input to a speech recognition device that has learned with a large number of adult male and female voices, it will hardly be recognized. This is basically because the physical structure of the vocal tract and vocal cords differs between adults and children, so the spectral structure and pitch period of each uttered voice are different, and as a result it is extracted from each voice This is because the feature vectors are different.

[0005] In order to solve this problem, the voice recognition device is made to learn a large number of children's voices, and a voice recognition device specially prepared only for children is used together with a device for discriminating between adults and children. There are measures to prepare. However, since a large-scale database of children's voices does not exist at present, it is not easy to prepare such a child-specific speech recognizer. Furthermore, even if such a large-scale database of children's voices is constructed with great effort, the above-mentioned solution is not very efficient.

[0006]

In order to essentially solve this problem, an expression that can automatically normalize the physical size of the vocal tract and vocal cords, which is difficult to perform with a spectrogram, is indispensable. Here, an example of only speech recognition has been given, but a problem that requires invariant acoustic feature extraction regardless of the physical size of the sound source, such as in the analysis of sounds emitted by musical instruments and the analysis of engine sounds. Comes out in various phases. For analyzing not only acoustic signals but also mechanical vibrations such as mechanical sounds and seismic waves, biological signal analysis such as brain waves, heart beat sounds, ultrasonic echoes and nerve cell signals, and general time series data Solutions to these problems are needed in a wide range of fields, such as the analysis of sensor signals.

[0007] It is therefore an object of the present invention to utilize some representation that does not depend on the physical magnitude of the source of the vibration, thereby making it essential to derive from the spectral representation as described in connection with the above example. An object of the present invention is to provide a method of performing signal processing exceeding the limit and an apparatus using the method.

[0008]

According to a first aspect of the present invention, there is provided a signal processing method, wherein a wavelet transform step of performing a wavelet transform on an input signal by a computer and an output of the wavelet transform step are synchronized with the input signal by a computer. And extracting a characteristic of the signal by performing a Mellin transform.

According to a second aspect of the present invention, in the signal processing method according to the first aspect of the present invention, in the characteristic extracting step, the characteristic extraction step converts a representation corresponding to a running spectrum obtained by the wavelet transform step into a response spectrum. Converting the time interval to logarithmic frequency expression by stabilizing in time with signal synchronization while maintaining the fine structure of the waveform, and in the time interval to logarithmic frequency expression, the value of the product or ratio of the time interval and the frequency is constant And performing a process corresponding to the Mellin transformation along the line

According to a third aspect of the present invention, in the stabilized time interval-log frequency expression in which the origin of the input signal is specified, the logarithmic time interval-logarithm of the time interval axis is logarithmically converted. Obtaining the frequency expression using a computer, and further using a computer, converting the logarithmic time interval-logarithmic frequency expression into a new expression having the product of the time interval and the frequency on the horizontal axis and the logarithmic frequency on the vertical axis;
Extracting the characteristic of the signal by performing integral conversion on the expression in the vertical axis direction or the horizontal axis direction.

According to a fourth aspect of the present invention, in the signal processing method according to the third aspect of the present invention, the expression space obtained by the integral transformation is used as a time series of expression vectors at a certain point. Further comprising the step of expressing.

A signal processing method according to a fifth aspect of the present invention provides the signal processing method according to any one of the first to fourth aspects, further comprising: The method further includes the step of providing an output subjected to frequency analysis in consideration of the characteristic to the Mellin transform step.

According to a sixth aspect of the present invention, there is provided a signal processing apparatus, comprising: a wavelet transform unit for performing a wavelet transform on an input signal converted into a predetermined format which can be processed by a computer; And characteristic extracting means for extracting the characteristic of the signal by performing the Merin conversion in synchronization with the characteristic.

According to a seventh aspect of the present invention, in addition to the configuration of the sixth aspect of the present invention, the characteristic extracting means converts the expression corresponding to the running spectrum obtained by the wavelet transform means into a response. Time interval is stabilized by signal synchronization while maintaining the fine structure of the waveform.
Means for converting to a logarithmic frequency representation, and means for performing processing equivalent to the Mellin transformation along a line in which the value of the product or ratio of the time interval and the frequency is constant in the time interval-logarithmic frequency representation And

The signal processing device according to the present invention is characterized in that the input signal converted into a format that can be processed by a computer has a stabilized time interval at which the origin is specified.
In the logarithmic frequency expression, a means for obtaining a logarithmic time interval-logarithmic frequency expression obtained by logarithmically converting the time interval axis, and a logarithmic time interval-logarithmic frequency expression are further represented by the product of the time interval and the frequency on the horizontal axis and the logarithmic frequency on the vertical axis. Means for converting into a new expression on the axis and extracting the characteristic of the signal by performing integral conversion on the expression in the vertical axis direction or the horizontal axis direction.

According to a ninth aspect of the present invention, in addition to the configuration of the eighth aspect, the signal processing apparatus further comprises:
It further includes means for expressing the expression space obtained by the integral transformation as a time series of expression vectors at a certain point.

According to a tenth aspect of the present invention, in addition to the configuration according to any one of the sixth to ninth aspects, the signal processing device according to any one of the sixth to ninth aspects further comprises: The apparatus further includes means for providing an output obtained by performing frequency analysis in consideration of the characteristic to the Mellin transform.

A signal processing apparatus according to an eleventh aspect of the present invention includes a plurality of signal processing units connected to receive an input signal, each of which performs conversion by wavelets having the same wavelet kernel function and mutually different frequencies. A wavelet bank comprising a wavelet filter, connected to receive an output of the wavelet bank, an auditory figure extracting means for extracting an auditory figure from an output of the wavelet bank, and an auditory figure extracted by the auditory figure extracting means. Size-shape image generating means for generating a size-shape image of the signal;
Feature extracting means for extracting features of the input signal from the size-shape image.

According to a twelfth aspect of the present invention, in the signal processing apparatus according to the eleventh aspect, in addition to the configuration of the eleventh aspect, the characteristic extracting means includes an impulse response line of each wavelet filter for the size-shape image. And a merin image generating means for generating a merin image by performing a Fourier transform along.

According to a thirteenth aspect of the present invention, in the signal processing device according to the twelfth aspect, in addition to the configuration of the twelfth aspect, the auditory figure extracting means detects a periodicity included in an output of the wavelet filter bank. Time strobe integration means for performing a time strobe integration on the output of each channel of the wavelet filter bank to generate a stabilized auditory image, and a time strobe integration based on the periodicity detected by the time strobe integration means. A stabilized auditory image extracting means for extracting a predetermined one cycle of the stabilized auditory image obtained by the integration as an auditory figure.

According to a fourteenth aspect of the present invention, in the signal processing device according to the thirteenth aspect of the present invention, the stabilized auditory image extracting means sets the first period of the stabilized auditory image to an auditory graphic. Means for extracting as

According to a fifteenth aspect of the present invention, in the signal processing apparatus according to the thirteenth aspect, the stabilized auditory image extracting means includes a step of setting the second period of the stabilized auditory image to an auditory graphic. Means for extracting as

According to a sixteenth aspect of the present invention, in addition to the configuration of the eleventh aspect, the output of the filter bank is different from the output of the filter bank in terms of nerve activity in the auditory nerve. And an auditory nerve firing pattern conversion means for performing conversion so as to obtain an output similar to the above and giving the conversion to the auditory figure extraction means.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION
First, in order to clarify the problems of the present invention, particularly the embodiments described below, a description will be given of the Mellin transform and acoustic physics. 1. Mellin Transform The Mellin transform is a type of integral transform similar to the Fourier transform, and is defined by the equation shown in Appendix A attached at the end of the description of the embodiment of the invention (Moriguchi, Udagawa, and Matsu, Mathematical Formula II, Iwanami Shoten, 1957, Titchm
arsh, "Introduction to the Theory of Fourier Integr
als, "Oxford UP, London, 2nd ed.) As expressed by the equation (A2) in Appendix A, even if the response of the signal to be analyzed is temporally enlarged or reduced while the response is similar, the Merin transform is performed. It is an important feature of the Mellin transform that the absolute value of the distribution obtained by the method is invariable except for a constant multiple. Appropriate signal processing is performed so that speech recognition can be performed despite the difference in spectral structure and the difference in pitch period 2. Physicality of acoustic tube Consider a lossless acoustic tube. The solution can be obtained by approximating the wave with a plane wave.The analytical solution for a sound tube of uniform caliber or horn-shaped sound tube is so well known as to be described in an elementary physics textbook. Also, the cross-sectional area of the sound tube changes. In this case, it is possible to numerically solve the wave propagating in the acoustic tube by approximating the cross-sectional area function with many small cylinders. A textbook on speech generation models teaches (for example, Nakata, Speech, Corona, Revised, 1995).

Now, consider an impulse response at the other end when one end of the acoustic tube is driven by an impulse. The important feature here is that the size of the sound tube is enlarged proportionally.
When contracted, the impulse response waveform is enlarged / reduced on the time axis. That is, the size of the physical acoustic tube is directly related to its impulse response.

A certain phoneme uttered by an adult and the same phoneme uttered by a child sound the same to a listener despite the fact that the size of each sound tube is completely different. Phonetic or English textbooks may contain vowels (vo
wel) and the corresponding articulation position (place of articulati)
on)). However, such a correspondence map does not show such a scale. Such maps can be shared by adults and children regardless of the size of their articulators. That is, regardless of the difference in the size of articulatory organs, the same phoneme can be uttered if the articulation is made similar. In other words, even if the physical size of the vocal tract is different, the same phoneme can be produced by maintaining the similarity of the vocal tract cross-sectional area functions.

If the vocal tract cross-sectional area functions are physically similar and their lengths are different, the impulse response of the vocal tract is expanded or reduced in time. Therefore, a child's voice, compared to an adult's voice, corresponds to driving a sound tube in which the impulse response of the vocal tract is reduced on the time axis with a sound pulse. Of course, because of individual differences, the above is an ideal story.However, the reduction of the impulse response on the time axis as described above is a good first-order approximation of the child's voice characteristics based on physical considerations. There should be. Not only is this analogy valid in speech, but also because violins, cellos and contrabass of different sizes produce similar sounds as instruments of the same violin family, and engines of the same shape and different sizes It can be justified by observing events other than speech, such as generating similar sounds. 3. Task assignment If we can directly create an internal representation that is invariant to the expansion and contraction of the vocal tract impulse response on the time axis as described above, we will use spectral analysis to extract higher-order formants that are difficult to extract This eliminates the need for normalization by calculating the enlargement / reduction, and the same phoneme can be processed as the same in both adults and children. The feature of having a property invariant to the expansion and contraction of the waveform on the time axis is nothing less than the feature of the melin expression that can be obtained through the melin transformation described above. In other words, it can be seen that the Mellin transform and the Mellin expression have a substantially different importance in the analysis of signals such as voices that are required now than the analysis derived from the conventional spectral expression.

However, conventionally, the Mellin transform has not been used very practically in signal processing. The reason for this is that, as described below,
(Shift varying) and its amplitude is "shift invariant"
(Sihft invariant) It was difficult to handle compared to Fourier transform. As can be seen from the equation (A1) in Appendix A, the starting point of the integration (hereinafter, this is referred to as the “origin of analysis”) needs to be determined in the Mellin transform.
If the origin of this analysis moves, the result will be different.
This is the property of "shift fluctuation". On the other hand, in the Fourier transform, it is sufficient to integrate within the range of (−∞, ∞).
There is no problem of such movement of the integration range. This is the "shift invariant" property.

For work on the melin conversion, see Um
esh et al. proposed a transformation on the frequency axis only due to the nature of the Merin transform (Umesh, Cohen, and Nelson, "Frequency-w
arping and speaker-normalization, "IEEE Int. Conf.
Acoust., Speech Signal Processing (ICASSP-97), 1
997; Umesh, Cohen, and Nelson, "Improved scale-ceps
tral analysis in speech, "IEEE Int. Conf. Acoust.,
Speech Signal Processing (ICASSP-98), 1998), and Altes proposed a combination of Fourier transform and Merin transform (Altes, "The Fourier-Mellin transform").
and mammalianhearing, "J. Acoust. Soc. Am., 63, p.
p.174-183, 1978) and application of the Mellin transform to speech recognition (Chen, Xu, and Huang, "A novel robust feature o
f speechsignal based on the Mellin transform for s
peaker-independent speech recognition, "ICASSP U9
8,1998) has also been proposed.

However, these are all Mellin transforms in the frequency axis direction using frequency amplitude information, and there is no consideration of phase information, that is, temporal information. Therefore, none of these papers mentions the specific problem of the origin of the analysis to overcome "shift variability", and does not seek a representation that preserves a stable temporal fine structure for sound. Since the information of the tone color of a sound is considered to mainly exist in this fine time structure, a method of normalizing the physical sound source dimensions while maintaining this information is desired.

In order to overcome the limitations of the current signal processing of a speech recognition device or the like, the disadvantage that the Mellin transform, which has an excellent function approaching the essence of speech and acoustic vibration, is a "shift variation" is overcome. It is necessary to perform the calculation for the signal processing accurately by utilizing it. SUMMARY OF THE INVENTION It is an object of the present invention, and in particular, the method and apparatus of the embodiments described below to derive a temporally stable representation so that the Mellin transform can be calculated to obtain a Melin representation.

[Principle of the Present Invention] The basic concept of the present invention will be described below in order to clarify the principle of the present invention, particularly the configuration and operation of the embodiment of the present invention described below. 1. SUMMARY OF THE INVENTION To overcome the "shift variation" disadvantages of the Mellin transform described above, the Mellin transform must be performed at a representation with a stable origin at any point in time.
Referring to FIG. 1, a general apparatus for realizing the solution according to the present invention includes a stabilizing wavelet processing unit 2 for performing a stabilizing wavelet transform process described later on an input signal 1, A Merin transform processor 3 for performing a Merin transform on the stabilized wavelet-processed input signal output from the stabilized wavelet processor 2,
For example, speech recognition,
A signal processing unit 4 for performing signal processing such as voice encoding and outputting a result 5; In the stabilized wavelet transform process performed by the stabilized wavelet processing unit 2, the input signal is subjected to time-frequency analysis through a wavelet filter bank, and the origin of the analysis is determined. By determining the origin of the analysis by the stabilizing wavelet processing unit 2, it becomes possible to perform the Mellin transform on the output of the stabilizing wavelet processing unit 2 by the Mellin transform processing unit 3.

In this apparatus, an input signal 1 is subjected to a stabilized wavelet transform by a stabilizing wavelet processing unit 2, and the output of the input signal 1 is defined as the starting point of the integration determined by the stabilizing wavelet processing unit 2 for the analysis. The Mellin transform 3 is performed, and a Melin expression is obtained. The obtained Mellin expression is a characteristic expression of the audio signal normalized with respect to the fluctuation of the size of the sound source and the periodicity of the waveform. This expression can also be expressed as a vector, similar to the spectrum or linear prediction coefficient mainly used in conventional speech analysis. Therefore, this Melin expression can be given as an input to any and all signal processing conventionally used, and the corresponding result 5 is obtained. For example, in a speech recognition device, it is possible to perform speech recognition by preparing a large number of feature vectors expressed in Melin in advance and performing exactly the same matching as that of the related art with the input feature vector, The hardware for that may be the same as the conventional one. 2. Wavelet Transform Referring to FIG. 2, a stabilized wavelet processing unit 2 for calculating a stabilized wavelet transform according to the present invention.
Is the input signal 6 (the same as the input signal 1 of claim 1,
Usually, it is assumed that it has periodicity. ), A wavelet transform processing section 7 comprising a filter bank for performing a wavelet transform, an amplitude compressing section 8 for compressing the amplitude of the output of the wavelet transform processing section 7 by logarithmic compression or exponential compression, and an amplitude compression section. 8, an event detection processing unit 9 for detecting an event representing periodicity and generating a detection output, and an event detection processing unit 9
, The time interval stabilization processing section 10 for stabilizing the time interval of the output waveform of the amplitude compression section 8 so as to determine the origin of the analysis as described above, and outputting it as the stabilized wavelet transform output 11 And

The equations defining the wavelet transform performed by the wavelet transform processing unit 7 are shown in equations B1 to B7 in Appendix B attached at the end of the description of the embodiment. In the wavelet transform, instead of a sine wave, which is a basis function in the Fourier transform, a function called a wavelet kernel (also called “mother wavelet”) that determines a small piece of a waveform is used. Then, a waveform obtained by expanding or reducing this wavelet kernel on the time axis (different in frequency from each other)
By examining how large the waveform to be analyzed is, the waveform to be analyzed can be analyzed in two dimensions of time and frequency.

In the Fourier transform, a sine wave is used. The sine wave is a periodic function uniformly spread in the range of (−∞, ∞) on the time axis. Therefore, in Fourier transform, it is not possible to obtain local information as to how many signals of which frequency exist in a certain part of the input signal. On the other hand, in the wavelet transform, it is possible to know local information as to which position contains a wavelet of which frequency and in what size. Therefore, the input signal can be analyzed from the two-dimensional time and frequency by the wavelet transform.

In the wavelet transform, it is known that the wavelet nucleus can be changed according to the purpose, and a wavelet nucleus having an appropriate waveform can be used for each application. For example, Daubechies wavelets, Mexican hats, French hats, Shannon wavelets, Haar wavelets, Gabor wavelets, M
Eyer wavelets are known. In the embodiment described below, a specific wavelet is used, but various wavelets described above and not described here can be used depending on the application.

In many cases, the input signal 1 having the periodicity (Equation B1) is wavelet-transformed and analyzed by the wavelet transform processing unit 7 (Combes et al. (Eds.), "Wav
elets ", Springer-Verlag, Berlin, 1989). The wavelet kernel is, for example, frequency-modulated at a predetermined frequency,
A gamma chirp function (formula B2) having a gamma distribution as an envelope can be selected. It is known that this gamma chirp function is an optimal function in the sense of minimum uncertainty in the Mellin transform (Irino and Patterso
n, "A time-domain, level-dependent auditory filter:
The gammachirp, "J. Acoust. Soc. Am., 101, pp. 412-4
19, 1997). Note that the wavelet kernel is not limited to the above-described gamma chirp function, and as described above, a waveform determined by an appropriate function depending on which feature is emphasized in the analysis can be used.

A filter bank of the wavelet transform processing unit 7 can be realized by using a set of wavelet filters (formula B3) obtained by expanding and contracting a wavelet kernel on the time axis. Here, convolution integration is performed between each signal and a filter of a filter bank arranged at equal intervals on a logarithmic frequency axis in a constant Q type in which the maximum frequency and the bandwidth are proportional (Equation B4).

Even if an external signal is temporally compressed or decompressed, the wavelet transform does not distort its output waveform. It simply moves the output of the signal to the location of the higher or lower maximum frequency filter. This is because the wavelet filter itself is obtained by enlarging or reducing the original wavelet kernel function on the time axis, and both have the same filter shape.

Logarithmic compression (Equation B5) or exponential compression (Equation B6) is performed on the obtained amplitude value of each filter output by the amplitude compression unit 8 in FIG. At this time, depending on the purpose,
There are two cases in which both the positive and negative portions of the waveform are left and the case where only the positive portion is left after half-wave rectification. In each example shown below, a case where half-wave rectification is performed is shown. When both the positive and negative parts are left, the subsequent processing is basically the same as that described below. 3. Assumptions and Stabilization Wavelet Transform of Merin Transformation As can be seen from Equation A1, the Mellin transform must always specify the origin of analysis, and if the origin deviates, the expression changes. ) "
Conversion. The point that the Mellin transform is a shift variation is disadvantageous to the shift-invariant Fourier transform,
This is why the Mellin transform has not been used so far. However, it has an attractive property for audio signal processing that it is resistant to variations in physical size as described above. Therefore, if the origin of the analysis can be determined reliably and stably, it is possible to overcome the disadvantage of the Mellin transform, which is a shift variation, and to effectively use the Mellin transform for audio signal processing. The present invention provides one solution for that.

Since the signal always flows temporally, the "wavelet spectrum" after performing the wavelet transform also corresponds to the "running spectrum" flowing temporally. Therefore, the origin of the analysis cannot be determined only from the wavelet spectrum. The origin of this analysis is determined by the event detection processing unit 9. Hereinafter, details of the processing performed by the event detection processing unit 9 will be described.

In the case of a periodic signal (Equation B2) or a pseudo-periodic signal, each wavelet filter output has one maximum value in one cycle. The present invention focuses on the point that the sound source information is expressed as a waveform when the maximum value is fixed and viewed. To this end, in the present invention, the periodicity of the filter output is detected by the event detection processing unit 9 and the time interval of the output signal of the amplitude compression unit 8 is stabilized by performing Mellin transform using the event as an origin.

A method for detecting the maximum value has already been reported (Irino and Patterson, "Temporal asymmerty").
in the auditory sytem, "J. Acoust. Soc. Am., 99, p.
p.2316-2331, 1996; Patterson and Irino, "Modeling
temporal asymmerty in theauditory sytem, "J. Acous
t. Soc. Am., 104, pp. 2967-2979, 1998). There have been many reports on pitch period detection from the past (eg, Hess, "Pitch Determination of Speech S
ignals, "Springer-Verlag, NY, 1983).

In the present invention, the time point of the maximum value in each channel is set as the start time point of the time integration performed by the time interval stabilization processing section 10 in FIG. In the time integration performed by the time interval stabilization processing unit 10, each wavelet filter output is copied with a period from a certain start time to the next start time as one cycle, and an expression for an already existing one cycle of the corresponding channel of the image buffer is provided. A new expression is generated by adding each point to. This operation is called strobe time integration (Patterson, Allerhand and Giguere, "T
ime-domain modeling of peripheral auditory proces
sing: a modular architecture and a software platfo
rm ", J. Acoust. Soc. Am., 98, 1890-1894, 1995; Patte
rson and Holdsworth, "Apparatus and methods for th
e generation of stabilised images from waveforms, "
United Kingdom Patent: 2232801 (1993), United S
tates Patent: 5,422,977 (1995), European Patent:
0473664 (1995)), and the entire operation up to this point is called a stabilized wavelet transform.

By the stabilized wavelet transform, the values of the points constituting the next cycle wavelet output, the next cycle wavelet output, and the further cycle wavelet filter output are added to the same position in the image buffer. The signal flow stops and the expression becomes stable.
Also, in this expression, the origin is always zero since the time interval from the previous peak is taken as the horizontal axis.

The stabilized wavelet transform (Equation B7) of the periodic signal (Equation B2) or the pseudo-periodic signal is a pattern in which sound source information is stored in its fine structure and is periodically repeated. Here, one cycle of the stabilized time interval pattern obtained by the stabilized wavelet transform will be referred to as a sound source information graphic (formula B8) or an auditory graphic. Since this sound source information figure is stable and the starting point is always fixed,
The problem of shift variability can be avoided and the Melin transform can be taken on this. That is, in the stabilized wavelet transform, the conditions necessary for the Mellin transform to analyze the sound source information are prepared. 4. It is known that the Mellin transform can be expressed by operators used in quantum mechanics (Cohen, "The scale transfor
m, "IEEE Trans. Acoust. Speech and Signal Processi
ng, 1993; Irino, "An optimal auditory filter," IEE
E Workshop on Applications of Signal Processing to
Audio and Acoustics, 1995; Irino, "A'gammachirp '
function as as optimal auditory filter with the Me
llin transform, "IEEE Int. Conf. Acoust., Speech S
ignal Processing (ICASSP-96), 1996). In that case,
The Merin transform is based on the time and frequency operators used by Gabor (Gabor, "Theory of communication," J. IE
E (London), 93, 42-457, 1946). That is, the product of time and frequency is an important concept for the Mellin transform. The equations defining the Mellin transformation are given in Equations B8 to B1 in Appendix B attached at the end of the embodiment.
It is shown in FIG.

In the present invention, in principle, the Mellin transform (Equation B10) is applied to the sound source information graphic (Equation B8) along an isoline (Equation B9) where the product of time and frequency is constant. Do. Here, since the parameter P of the Mellin transform is a complex number (formula B11), formula B10 can be rewritten as formula B12. As a result, as the Merin transform of the sound source information graphic, a two-dimensional expression can be obtained in which the horizontal axis is the product of the time interval and the frequency and the vertical axis is the complex variable of the Merin transform nucleus. This expression will be called a merin image.

In this expression, the sound source information is normalized and is invariant to the periodicity of the sound source and the enlargement / reduction of the physical size. Therefore, by giving the normalized sound source information to the signal processing unit 4 according to the signal processing method conventionally proposed, more excellent signal processing can be realized.

The flow of the above processing is shown in the flowchart of FIG. The calculation of the Mellin transform will be described in further detail in the first embodiment. Referring to FIG. 3, when a waveform input is received, it is subjected to a wavelet transform filter bank to perform a wavelet transform calculation.

By extracting signal period information from the output of the wavelet transform, stabilizing the output of the wavelet transform based on this information, and calculating the time interval-logarithmic frequency expression from the immediately preceding peak, Get sound source information figure.

The Mellin transform is calculated along a line on the sound source information graphic obtained in this way, where the product of the time interval and the frequency is constant. In this way, a melin image is obtained which is invariant to the periodicity of the sound source and the expansion or contraction of the physical size. 5. Time Series of Merin Image In the previous section, we showed how to calculate a Merin image from a stabilized wavelet transform at a certain point. The signal changes every moment, and the sound source information figure obtained from the stabilized wavelet transform corresponding thereto also changes. Therefore, sound source information figures are extracted at certain intervals, and a merin image is calculated based on each figure. One feature vector can be extracted from each of the Merin images. Then, as in a spectrogram, an expression in which the horizontal axis is time and the vertical axis is the Melin image vector axis, and the Melin image vectors are arranged, can be created. This is completely different from the spectrogram,
Since they are the same in form, they can be directly input to the signal processing method using the conventional spectrogram, and can be easily applied to various fields.

[Operation / Effect] Depending on the physical size of the sound source, the scale distribution of the Merin image remains unchanged even if the waveform to be analyzed is enlarged or reduced in time. This is a property not found in the Fourier spectrum. At the same time, although the expression is different from the Fourier speckle, the expression using the Melin image vector can clearly express the difference other than the enlargement / reduction of the waveform to be analyzed. In the case of speech, utterances of different vocal tract lengths can be treated in the same way by the expression using the Melin image vector. Therefore, conversely, only the difference between phonemes can be emphasized using the expression based on the Melin image vector. For example, if the expression based on the Melin image vector is used, there is a possibility that the speech recognition device learned from adult data can be used as it is for child recognition. There are many other situations where the expression using the Merin image vector can be applied, and improvement in the performance of a speech recognition device or the like can be expected. further,
By using the expression based on the Melin image vector in combination with the spectrum distribution conventionally used, audio signal processing exceeding the conventional performance can be realized. In addition, since any waveform can be used as long as it is time-series data, the present invention can be applied not only to acoustic signals such as voice and music, but also to mechanical vibrations, biological signals, and time-series measurement data. Such a technique can be applied.

In the above, the basic method of the embodiment of the present invention and its background have been described. Hereinafter, embodiments of the present invention will be described in detail. First Embodiment Referring to FIG. 4, a speech recognition apparatus according to a first embodiment of the present invention includes a stabilized wavelet processing unit 2, a Melin transform processing unit 3, , A signal processing unit 4.

The stabilizing wavelet processing unit 2 receives the audio signal 12 as an input, performs a wavelet transform on the audio signal 12, and performs a frequency analysis to perform an audio filter bank 13 and an output of the audio filter bank 13. And an auditory nerve firing pattern converter 1 for performing a conversion to obtain an output similar to the nerve activity in the auditory nerve.
4, an event detection (pitch detection) circuit 15 for detecting a maximum value in a certain neighborhood to control time integration;
The output of the event detection (pitch detection) circuit 15 is used as a signal (strobe) to extract a current fixed section output from the auditory nerve firing pattern converter 14 and perform the above-described time integration to generate and output a stabilized auditory image. And a stabilized auditory image processing unit 16. Each of these components will be described later in detail.

The Merin transform processing section 3 transforms the stabilized auditory image output from the stabilized auditory image processing section 16,
A dimension-shape image processing unit 17 for outputting a dimension-shape image, which is a new expression, and a merin image calculated from the dimension-shape image output from the dimension-shape image processing unit 17, as an expression based on the merin image vector And a merin image processing unit 18 for outputting.

The signal processing unit 4 includes the merin image processing unit 1
8, the expression based on the Merin image vector
Speech recognition circuit 19 for performing speech recognition by matching with a prepared template and outputting speech recognition result 20
including.

In the apparatus shown in FIG. 4, an input audio signal 12 is converted into a stabilized auditory image (Stabilized Auditory Image, SAI) by the Mellin conversion processing unit 3. This stabilized auditory image is an auditory version of the representation obtained with the stabilized wavelet transform 2. The stabilized auditory image is measured by the size-shape image processing unit 17 in size-
The image is converted into a shape image 17 and further converted into a melin image 18 by a merin image processing unit 18. This processing corresponds to Merin transform 3. Expressions and the like showing the stabilized wavelet-Mellin transform based on the auditory image model described below are described in Appendix C attached at the end of the description of the embodiment. 1. Configuration of Stabilized Auditory Image In this section, the operation of each component of the stabilized wavelet processing unit 2 will be described. The input audio signal 12 is
The frequency is analyzed by the auditory filter bank 13. In the apparatus of this embodiment, each auditory filter of the auditory filter bank 13 can be approximated by a gamma chirp (Equation C1) having a carrier frequency-modulated by the gamma distribution function envelope. The auditory filter bank 13 is approximately 500H
Above z, the filter is a constant Q-type filter in which the maximum frequency and the bandwidth are proportional (Equation C2). That is, the auditory filter bank is a wavelet transform (equations C3 and C4) using gamma chirp (equation C1) as a kernel function, and the parameters of this function can be set to simulate a human auditory filter (Irino and Patterson, "A time-domain,
level-dependent auditory filter: The gammachirp, "
J. Acoust. Soc. Am., 101, pp. 412-419, 1997). The auditory filter bank 13 in which the auditory filters are arranged can be constituted by an IIR filter (see, for example, JP-A-11-24696 and JP-A-11-119797).

The output of the auditory filter bank is converted by the auditory nerve firing pattern converter 14 into an auditory nerve firing pattern (Neural
Activity Pattern, NAP). In particular,
The output of the auditory filter bank 13 is subjected to half-wave rectification, and the amplitude is logarithmically compressed (Equation C5) or exponentially compressed (Equation C6). An output similar to nerve activity is obtained.

The event detection (pitch detection) circuit 15 monitors the activity of each channel, detects the maximum value in a certain vicinity, and controls time integration. Event detection (pitch detection)
The processing in the circuit 15 is performed, for example, as follows. First, the activity is smoothed to calculate an envelope. Calculate the derivative of the obtained envelope and calculate its value (envelope slope)
The peak time point of the highest activity, which is close to the point at which the value changes from positive to negative, is defined as the nearby maximum value time point (see Irino and Pa
tterson, 1996). This neighborhood maximum value occurs constantly in a signal having periodicity or pseudo-periodicity such as a voiced sound of a voice and a stationary musical instrument sound. Using this neighborhood maximum value as a signal (strobe), the current fixed section of the nerve firing pattern is taken out, and the addition of the neighborhood maximum value to the corresponding channel of the buffer of the auditory image 16 is repeated for each section. As a result, time integration is performed. Such integration is called strobe time integration (Strobe
d Temporal Integration (STI).

The processing of the STI is based on the nerve firing pattern (NA
It serves to convert the time axis of P) into a time interval axis based on the immediately preceding maximum value (Equation C7). If the strobe time integration is performed for all the channels of the auditory filter bank 13, a stabilized auditory image 16 (equation C7) can be obtained while maintaining the value of the vertical axis (logarithmic frequency axis) in the auditory filter bank 13. . This stabilized auditory image is attenuated in its entirety with a half-life of about 30 ms, and the image naturally disappears when the input signal disappears.

By integrating the stabilized auditory image in the time direction, a spectral peripheral distribution can be obtained. Since this spectral marginal distribution is similar to the spectral vector of a conventional spectrogram, an auditory spectrogram can be constructed and applied to speech recognition (see, eg, Patterson et. Al. 1995, supra). 2. Configuration of Size-Shape Image In this section, details of processing performed by the size-shape image processing unit 17 will be described. The stabilized auditory image output from the stabilized auditory image processing unit 16 has a representation having a linear time interval axis on the horizontal axis and a logarithmic frequency axis on the vertical axis. The size-shape image processing unit 17 obtains a new size-shape image by modifying this expression. This is an important step to make it easier to calculate the Melin image 18 in the next section.
FIG. 5 is a block diagram showing details of the size-shape image processing unit 17 that performs this processing. The following processing flow is shown in the flowchart of FIG. FIG.
And the description of FIG.

Referring to FIG. 5, the size-shape image processing unit 17 includes a filter delay correction unit 22 for correcting a filter delay included in the stabilized auditory image 21 and a vertical direction for the auditory image for all channels. Activity calculator 23 for calculating the total activity on the time interval axis in addition to the above, and the periodicity of the auditory image is detected based on the magnitude of the activity calculated by the activity calculator 23 Pattern detecting unit 24 for detecting a sound pattern, and an auditory figure extracting unit 25 for extracting an auditory figure to be described later from an auditory image using the periodicity detected by the periodicity detecting unit 24
A conversion unit 26 for converting the horizontal axis of the auditory graphic extracted by the auditory graphic extraction unit 25 from a linear time interval axis to a logarithmic time interval axis, to a logarithmic time interval expression; An impulse response correction unit that performs processing of moving the horizontal axis for each channel so that the straight impulse response line observed in the auditory figure whose horizontal axis is converted by the conversion unit 26 is parallel to the vertical axis. 27.

The Auditory Image Mod (Auditory Image Mod)
el, AIM) FIG. 7 shows a stabilized auditory image 21 as an example of a stabilized auditory image obtained according to (Patterson et. al. 1995, supra). FIG. 7 shows an auditory image of a click sequence sound generated at intervals of 10 ms, that is, a frequency of 100 Hz, for a little over two cycles. The vertical axis shows each channel of the filter at their maximum frequency Hz.
, And is a pseudo logarithmic frequency axis. The horizontal axis represents the time interval from the time of the vicinity maximum value at which the strobe time integration is started, and is expressed in milliseconds. Here, the time interval is a linear axis.

Referring to FIG. 7, portions having high activity along three vertical lines are arranged at the same period as the period of the original waveform. The location of 0 ms on the horizontal axis is the location where the activity of the nearby maximum value is transferred in the strobe time integration. This neighborhood maximum specifies each cycle in the case of a periodic signal, and specifies the starting point of a feature in the case of an aperiodic signal. In this way, the strobe time integration specifies the starting point or zero point of the analysis of the Mellin transform.

In the Merin transform, the wavelet filters constituting the first-stage auditory filter bank 13 are arranged on a reasonable basis, for example, at the time when the envelope of the auditory filter rises (time t = 0
Is theoretically desirable for all channels. However, in the strobe time integration, the rise of the envelope of the auditory filter itself cannot be detected, and the strobe is applied at the maximum value of the response, so that a delay time is generated with respect to the rise of the envelope. This shift can be seen by the activity on the curve that is to the left of each dense location of vertical activity in FIG. It is desirable to correct the time delay for this filter in order to make the process easy to understand.

The filter delay correction section 22 performs the correction for this. In order to perform this correction, the activity of each channel may be simply shifted to the right by the period of the reciprocal of the maximum frequency of the auditory filter (equation C).
8). FIG. 8 shows an auditory image as a result of performing the correction on FIG. This gives a good approximation of the starting point of the Mellin transform where it is placed vertically. It will be described later that it is known that the output of the Merin transform is not so affected even without performing this correction.

As described above, the stabilized auditory image processing unit 1
The strobe time integration (STI) performed at 6 stabilizes the time interval pattern that repeats into the auditory nerve firing pattern (NAP) due to the periodic sound and is shown at 0, 10, and 20 in the time interval of FIG. This causes a vertical concentration of activity in the auditory image (SAI). As can be seen with reference to FIG. 7, this vertical activity line divides the auditory image into several similar intervals at the same interval as the period of the original signal. This one section is defined as the auditory figure (AuditoryFigu) corresponding to the sound source signal.
re, AF) (Equation C9).

The activity calculator 23 adds the auditory image to all the channels in the vertical direction and calculates the total activity of the distribution on the time interval axis. The periodicity detector 24 can determine the periodicity of the pattern based on the magnitude of the activity. By using this periodicity information, the auditory figure extraction unit 25 detects the auditory image corresponding to one period of the auditory image from the auditory image corrected for the filter (FIG. 8, corresponding to the result corrected by the filter delay 22). You can extract figures.

The auditory graphic extracted by the auditory graphic extractor 25 has a linear time interval axis as a horizontal axis. When the time interval on the horizontal axis is logarithmically converted, subsequent processing can be easily performed. The conversion unit 26 for logarithmic time interval representation performs this logarithmic conversion. That is, the conversion unit 26 to logarithmic time interval representation converts the horizontal axis of the auditory figure into a logarithmic time interval axis (Equation C10). By this conversion, as shown in FIG. 9, a group of curves in the auditory diagram corresponding to the impulse response of the auditory filter can be converted into a group of straight lines that are substantially parallel and regularly arranged at 500 Hz or higher. FIG. 9 is a diagram in which the leftmost auditory graphic in FIG. 8 is converted using a logarithmic time interval axis using spline interpolation.

Referring to FIG. 9, each of the linear impulse response lines has a negative gradient and is inclined similarly to the diagonal line of the auditory figure. This expression expresses a logarithmic time interval on the horizontal axis,
The vertical axis has a logarithmic frequency, so that the Mellin transform can be easily calculated.

In order to make the calculation of the Mellin transform and the expression indicating the sound source information easy to understand, the impulse response line of the logarithmic time interval auditory diagram (Equation C10) in FIG. Therefore, this is hereinafter referred to as “perpendicular line”.), And FIG. 10 is obtained (Equation C11). This correction is performed by a conversion unit 26 for converting to a logarithmic time interval expression.
And for each channel,
This corresponds to moving the logarithmic time interval axis to the right by an amount proportional to the logarithm of the maximum frequency. The new horizontal axis in FIG. 10 is the product of the time interval and the channel maximum frequency h (equation B
It is expressed by the logarithm of 9). The vertical axis is the maximum frequency of the logarithmic axis display as in the related art.

Referring to FIG. 10, the leftmost vertical dotted line indicates the position in the auditory figure where the product h of the time interval and the maximum channel frequency is 1. In FIG. 10, h
Are drawn by broken lines corresponding to values of 1 to 5, and the activity is concentrated on any of them. That is, in the expression shown in FIG. 10, the impulse responses of all the wavelet filters are concentrated on the vertical line where the value of h is an integer, and therefore, it is understood that this expression does not depend on the enlargement / reduction of the wavelet filter. . To make this easier to understand, FIG. 11 is obtained by changing the horizontal axis to the linear axis of h.

In the example shown in FIG. 11, since the activity is directly obtained from the auditory image shown in FIG. 8 without using the logarithmic transformation, the activity on the vertical line corresponding to h = 0 is also shown. In order to perform this processing, in the auditory image shown in FIG. 8, resampling of each activity is performed at a sampling frequency proportional to the maximum frequency of each channel, and the sample points may be arranged in two dimensions as they are. Only.

As described in the previous section, in this expression, since the wavelet filter has the same expression in every channel, if the sound sources are similar and the waveform is expanded / reduced as a wavelet, the expression is always used. A representation of the same shape is obtained. The scaling of the waveform is represented in this expression as a simple translation of the activity distribution in the direction of the vertical frequency axis. Therefore,
In the sense that the information represents both the size and shape of the sound source, this expression is referred to as a size-shape image (Size
-Shape Image, SSI). As will be described later, this expression is particularly effective when expressing an auditory figure of a vowel. The above processing flow is shown in the flowchart of FIG.

The auditory figures in the size-shape images of FIGS. 10 and 11 are obtained from the leftmost auditory figure of the auditory image of FIG. 7 by the above-described series of procedures. However, it does not necessarily have to be the leftmost auditory graphic, and may be the second auditory graphic.
There is no problem in proceeding with the procedure even with an auditory figure (Equation C9) representing a period.

However, in the case of a simple click sound sequence as in this example, the selection is the same regardless of where it is selected, but when noise is added to voice or musical sound, the second auditory figure is selected. It is more advantageous to extract a signal-only component. This is because both noise and signal components are concentrated in the first auditory pattern.

The marginal distribution along the h-axis of the horizontal axis of the dimension-shape image is mainly called an impulse marginal distribution (ImpulseProfile) because the impulse response of a wavelet filter having the same shape in each channel is mainly used (formula (ImpulseProfile)). C1
2). On the other hand, what is along the vertical axis is an auditory spectrum peripheral distribution (Spectral Profile) (formula C13).
The impulse margin distribution has sound source information different from a conventional spectrum vector. Since each marginal distribution is a vector representing a dimension-shape image at a certain point in time, for example, at regular intervals (for example, 5 to 30)
If these vectors are calculated at intervals of about ms and arranged in a spectrogram format as a time series, it can be applied to speech recognition. This representation could be called a dimension-shape image spectrogram. 3. Configuration of Merin Image In this section, the merin image processing unit 1 is converted from the size-shape image output from the size-shape image processing unit 17.
8 shows the reason and the process of obtaining the melin image, and shows that this melin image corresponds to the melin image output from the melin conversion processing section 3 in FIG.

In the size-shape image output from the size-shape image processing unit 17, the response of the auditory wavelet filter occupies most of the distribution. The sound source information that would have been output to the right of these impulse response lines when a non-click sequence sound was input is expressed relatively small. Since we want to extract the sound source information itself, we want to remove the auditory filter information by some means using a deconvolution method. For this purpose, it is considered that a vertical vector is Fourier-transformed for each h of the dimension-shape image and each vector is represented by the amplitude of the spatial frequency component. Since the auditory wavelet filter information in the size-shape image does not change much in each channel, as can be seen from FIG. 10, the information will be concentrated where the spatial frequency is extremely low. On the other hand, sound information from sound sources other than click system sounds will appear at places where the spatial frequency is relatively high because the wavelet filter is forcibly excited and separate ringing occurs at various frequencies. . Thereby, the sound source information can be separated from the information of the wavelet filter itself.

This calculation is based on the expression C1 of the impulse margin distribution.
2 can be realized by replacing the weighting function W (αf _b , h) with a weighted complex sine wave defined on a logarithmic frequency represented by Expression C14. At this time, the spatial angular frequency c
/ 2π is introduced as a parameter to obtain W (αf _b , h, c),
By substituting into the expression C12, the two-dimensional expression C1
5 can be obtained. Output M obtained from equation C15
_I (h, c) will be referred to as Mellin Image 18. At this time, the horizontal axis is h, which is the same as the dimension-shape image, and the vertical axis is the spatial frequency c / 2π of the Fourier transform. The vertical translation in the size-shape image becomes a mere change in phase through the Fourier transform, and the amplitude information remains unchanged. In the size-shape image, the periodicity of the sound source has already been removed, and the size in the h-axis direction does not change. Therefore, the auditory figure represented by the Merin image represents sound source shape information that does not depend on the size of the sound source or the periodicity of the sound source excitation.

FIG. 12 shows a melin image obtained from FIG. 11 of the size-shape image of the click series sound. FIG.
As can be seen from FIG. 5, in the melin image of the click sequence sound, the activity is concentrated only at a very low spatial frequency, and there is almost no activity at a high frequency. This reflects the fact that, as described above, the click sound causes almost flat activity on the vertical line except for the low-frequency channel in the size-shape image. In the first place, the impulse response of the wavelet filter is normalized so that it has the same shape in every channel.
Since it is a shape image, theoretically, when only a single click is input, an amplitude value exists only at a place where the spatial frequency is zero. 4. Before moving on to the example of analysis of a damped oscillatory wave or a vowel, a Merin image represented by an integral in the frequency domain in this example, which is obtained as an output of the Merin image processing unit 18 (formula C15),
Consider the relationship with the melin image (formula B10) output from the melin transform processing unit 3 represented by integration in the time interval region described as a basic explanation. Taking the logarithm of the basic constraint condition (Equation B9) that the product of the time interval and the maximum frequency is constant gives Equation C16, and the derivative thereof gives Equation C17. Substituting this relationship into Equation C15, Equation C10, Equation C1
Utilizing 1 gives equation C18. This is an integral expression in a time interval region similar to Expression B10 except for the constant. This fact is based on the Merin image (Equation C15) obtained as the output of the Merin image processing unit 18 and represented by the integration in the frequency domain in this example, and the integration in the time interval domain described as a basic explanation. It is shown that the expressed merin image (formula B10) output from the merin conversion processing unit 3 is the same. 5. Hearing image / dimensions / shape image /
FIG. 13 shows an auditory image of a repetitive exponentially attenuated sine wave. This exponentially attenuated sine wave has an exponential envelope with a half-life of 2 ms, a sine wave carrier with a frequency of 2 kHz, and a repetition frequency of 100 Hz. A damped sine wave with this parameter is similar to a single formant vowel. The repeated rising portion produces a click-like response in the frequency domain at a distance from 2 kHz as activity on the vertical, with the interval between the two vertical activities indicating the periodicity of the signal. From the auditory image of FIG. 13, it can be seen that in the region of 2 kHz, the response is emphasized and extended by resonance having an attenuation envelope. This is a feature commonly found in sounds in the natural world including voice.

FIG. 14 shows a size-shape image of the auditory figure of the attenuated sine wave. The activity at a position away from 2 kHz is not much different from that of the click sequence sound in FIG.
However, in the channel around 2 kHz, the activity increases to a high value of h, and it can be seen that the slope of the column of adjacent activity gradually increases as the value of h increases. This indicates that the instantaneous frequency in a channel other than the 2 kHz channel is not the frequency of the wavelet filter, that is, the carrier frequency of the filter of each channel.

FIG. 15 shows the Mellin image of this attenuated sine wave.
Shown in Since the rising portion is like a click, the activity is concentrated at a place where the spatial frequency is very low, as in the case of the click series sound (FIG. 11). 2kH of dimension-shape image
The activity related to the resonance in the z region indicates that the vertical band-shaped active region is further increased on the Merin image, and that a portion where h is large has a response with a wide spatial frequency. The width of the band-shaped active region increases with increasing h, which corresponds to the slope between adjacent activities observed in the microstructure increasing with increasing h. This is characteristic of single resonance or single formant sound sources.

Among the band structures of the attenuated sine wave Merin image having other parameters, the band structure does not change much depending on the frequency of the carrier, the half-life of the envelope, and the repetition frequency of the signal. In other words, the information on the shape of the sound source is extracted independently of the size and the repetition frequency due to the difference in the band-like structure described above. The intensity and extent of the vertical band increases slowly with increasing half-life of the damped sinusoid. In the next section, we extend the example further and perform a similar analysis on vowels synthesized using the vocal tract cross-sectional area function. 6. The auditory image, size-shape image, and merin image of the four vowels 'a'.
a 'created. This synthetic vowel is a vocal tract cross-sectional area function of one male (Yang CS and Kasuya, H. (1995). "Dimension
differeces inthe vocal tract shapes measure from
MR images across boy, female and male subjects, "
A vowel synthesized from a vocal tract model using J. Acoust. Soc. Jpn (E), 16, pp.41-44. Consider extracting features of this vocal tract shape using a size-shape image / melin image.

One set of two voices of the four types is obtained by using the vocal tract cross-sectional area function as it is and excited by vocal cord pulses having two different frequencies of 100 Hz and 160 Hz. These auditory images are shown in FIG. 16 and FIG. Vocal tract resonance can be seen as an extension of the response in the region of resonance on the auditory image. This is the formant we call in phonetics. The second and third formants are approximately 1000Hz
And 2200 Hz. The vertical activity concentration positions in the figure are closer to each other in FIG. 17 than in FIG. 16, but it can be seen that the formant position does not change depending on the vocal cord vibration frequency.

The second set of two voices is a case in which the length of the vocal tract is reduced to 2/3 while the same vocal tract cross-sectional area function used above is kept similar and synthesized. The vocal cord vibration frequencies are 100 Hz and 160 Hz as before. The auditory images of these vowels are shown in FIGS. In these figures, the second and third formants are at the same position, but the original FIG.
17 and the frequency 1500 which is 3/2 times that of the case of FIG.
Hz and 3300 Hz, respectively. This is because the vocal tract length has become shorter. The position of the vertical activity is the same in FIGS. 16 and 18 and FIGS. 17 and 19, respectively.

The size-shape images of these four vowels are shown in the order of the auditory images in FIGS. In these auditory figures, the distinction between the response to the vocal cord pulses to the left of the auditory figure and the formants extending to the right is emphasized. The pattern of the voice information from the original long vocal tract (FIGS. 20 and 21) is basically the same. But,
Only the position of the right boundary of the auditory figure determined by the repetition frequency on the waveform differs from each other, and the higher pitch in FIG. 21 has a smaller range. Similarly, in the size-shape images of the short vocal vowels (FIGS. 22 and 23), both patterns are the same, and only the position of the right boundary is different.

Further, comparing the size-shape images of the long vocal tract and the short vocal tract, it can be seen that the response patterns of the four formants from the bottom are very similar to each other.
The difference is that the patterns of FIGS. 22 and 23 of the shorter vocal tract are translated upward in frequency compared to the patterns of FIGS. 20 and 21 of the longer vocal tract. Figure 2 of the long vocal tract
The fifth and sixth formants that can be seen in the dimension-shape image of FIG. 0 and FIG. 21 are the upper limit frequency 6000 in FIG. 22 and FIG.
Although it has moved by the same amount above Hz and cannot be seen, it can be seen if the frequency range in the figure is extended upward.

FIG. 24 shows a merin image of these four vowels.
27 to FIG. 27 show the auditory images and the size-shape images in order. The vertical axis of the melin image is the melin coefficient c / 2
At π, this corresponds to the spatial frequency in the vertical direction of the dimension-shape image, one period in the range from 100 Hz to 6000 Hz corresponds to a spatial frequency of 1. The value of the Merin image for a certain value of h is the absolute value after integration using a complex sine wave in the vertical direction of the dimension-shape image,
The one that best matches the spatial frequency and the distribution of the activity becomes larger.

Referring to FIGS. 20 to 23, the response of the vocal cord pulse has an activity at a low spatial frequency of 4 cycles / frequency or less within about 5 integers of h in the size-shape image of vowel 'a'. Can be seen. When h is 2 or greater, the formants appear as large values in separate bands in the size-shape image. When h is increased from 2 to 8, the best matching frequency is about 6 to 18, and a large value appears. When h is 8 or more, there is only one formant in the size-shape image, which indicates that a wide band-shaped active region is formed in the melin image. This is,
FIGS. 20 to 23 showing the merin image of these four vowels 'a' are the most characteristic in the common characteristics. 7. Size-shape image and merin image of five vowels 'a, i, u, e, o' in Japanese To show how different vowels are expressed in size-shape image and merin image 5
The vowel pairs were analyzed. The same vocal tract model and the same male speaker but different vocal tract cross-sectional functions (see Yang and Kas
uya, 1995). 100H using the vocal tract cross-sectional area and vocal tract length as measured
Synthesized by driving with z vocal cord pulses. In this order for the five vowels 'a, e, i, o, u', the auditory images are shown in FIGS. 28 to 32, and the size-shape images are shown in FIGS.
7. The melin image is shown in FIGS. 38 to 42, respectively.

A comparison between the auditory image and the size-shape image shows that the logarithmic transformation of the time interval axis changes the manner of emphasizing the formants. For example, the vowel '
In a '(FIG. 28), the continuation length of the resonance of the second formant is about three times longer than that of the fourth formant. However, in the dimension-shape image (FIG. 33), the continuation length of the second formant resonance with respect to the axis h of the time-frequency product is substantially the same as or slightly shorter than the fourth formant. Without such a transformation, the role of higher-order formants would be almost invisible if the Mellin transformation was taken directly on the frequency axis. Channel correction in the size-shape image effectively works to separate the wavelet impulse response from the response due to the nature of the sound source.

First, the 'a' described in the previous section (FIGS. 33 and 3)
8) Compare the size-shape image of 'e' (FIGS. 34 and 39) with the Merin image. The higher order formants in the dimension-shape image of 'e' (FIG. 34) are more concentrated than those of 'a' and extend to higher h values. This makes the 'e' merin image different from the 'a' merin image,
The value is large around 4 and 12 to 16 where the spatial frequency c / 2π is low, and the value extends to a position where h is high.

In the vowel 'i' (FIGS. 35 and 40), similar to 'e', higher formants form a group but are more concentrated. This gives rise to a point where the value of c / 2π at h 2 to 6 is large around 8. When h is 4 or more, the active region moves to c / 2π of about 15 to 20. Furthermore, as can be seen from the elongation of the resonance region in the dimension-shape image of 'i', a wide band-like region extends to a high h value of 15 or more.

In the dimension-shape image of 'o' (FIG. 36), there is a large frequency gap between the first and second formant sets and the remaining three formant sets (about 1200 Hz to 2800 Hz). As a result, in the melin image of “o” in FIG. 41, the activity at c / 2π of 4 or less is not so large. The range where the first formant is located, that is, FIG.
In the range where h is up to 5 and where c / 2π is about 5 to 8, there is an activity that reflects the interval between the first and second formants, but when the first formant disappears, c / 2π becomes 12 ~ 20
The degree of activity mainly reflects the spacing of higher formants in the order of magnitude. Continuing groups of higher formants show differences from other vowels, reflecting the diffuse activity of lower spatial frequencies where h is higher.

The vowel 'u' (FIGS. 37 and 42) is simpler than the other vowels and has a wide formant resonance bandwidth.
The activity does not extend to the place where the value of h is large in the dimension-shape image or the merin image. This may represent the characteristics of this vowel, but has lost its easily distinguishable features where h and c / 2π are large. h is 2
In the range of 5, c / 2π has a strong activity at around 7,
When h is in the range of 4 to 5, it is about 13. The band is h
Is almost nonexistent in 10 or more, and is close to 'a' in other vowels.

As described above, the melody images of the vowels are characteristically different, and the respective differences can be easily extracted from these differences. 8. Speech Recognition Device Up to the previous section, the excellent features of the Merin image have been shown, in which the sound source is almost the same for the same shape, and is different when it is different. By using such information of the merin image, an excellent voice recognition device can be realized. For example, when the activities are added in the vertical axis direction or the horizontal axis direction of the merin image, the peripheral distribution of the one-dimensional vector is obtained. By arranging both or one of these vectors in a line to form a one-dimensional vector, a feature vector representing a feature at a certain point in the auditory image is obtained.

If this feature vector is calculated, for example, at regular intervals (for example, about every 5 to 30 ms) of the auditory image, and sequentially arranged on the vertical axis to form a spectrogram, an expression which can be called a melin image spectrogram is obtained. . Even if it is combined with the above-described size-shape image spectrogram, it can be directly input to the speech recognition circuit 19 (FIG. 4) which is currently widely used. Each marginal distribution is a vector representing the sound source information at a point in time, and has a richer amount of information than the conventional amplitude spectrum. As a result, a speech recognition result 20 superior to the related art can be obtained. This is the greatest advantage of the present invention. Second Embodiment FIG. 43 shows an embodiment in which the present invention is applied to an utterance practice device for rehabilitation from practice or disability in another language, which can be applied to adults and children having different vocal tract sizes. The device is shown. This device includes a microphone 29 for converting an input voice into an electric signal, and an amplifier 30 for amplifying an electric signal output from the microphone 29.
An A / D converter 31 for performing analog / digital conversion of the electric signal amplified by the amplifier 30;
A general-purpose computer 32 for executing a program for performing audio signal processing in response to a digital signal output from the converter 31; A word character / feature amount display device 33, a DA converter 34 for converting a digital voice signal output from the general-purpose computer 32 into an analog signal, and a voice converted to an analog signal by the DA converter 34. It includes an amplifier 35 for amplifying a signal, and a speaker or headphones 36 for converting an audio signal provided from the amplifier 35 into audio.

An electric signal representing a sound output from the microphone 29 is input to a general-purpose computer 32 through an amplifier 30 and an AD converter 31. The general-purpose computer 32 performs a process to be described later on the electric signal, and supplies a signal representing the result to the phoneme / word character / feature amount display device 33 and the DA converter 34. The output of the general-purpose computer 32 is visually presented by a phoneme / word character / feature amount display device 33, and is presented audibly by a speaker or headphones 36 through a DA converter 34 and an amplifier 35.

In this general-purpose computer, processing according to the flowchart of FIG. 44 is performed. First, the already described stabilized wavelet transform is performed. Using the information, a pitch frequency, a size-shape image, and a melin image are calculated in parallel.

In the calculation of the size-shape image, information on the vocal tract length of the speaker is calculated, and in the Merin image, a normalized expression of the vocal tract length is calculated. By comparing them with the standard templates stored in advance, the phonemes and character strings spoken by the speaker can be determined and output as visual presentation information, or matched to the speaker's vocal tract length and pitch information. It is output as auditory presentation information as a synthesized sound.

In order to use the apparatus as a vocal training apparatus, visual and auditory presentation can be performed from teaching information such as generation of a practice exercise. Thus, although it is not necessary to prepare a standard template in all cases for both adults and children, accurate phonological judgment can be made, which is effective as a device for efficient practice. Third Embodiment FIG. 45 shows an embodiment in which the present invention is applied to an automatic sorter for qualities of fruits, fruits, and foods having different sizes. This automatic sorter includes a speaker 37 for irradiating a sound wave to an object to be sorted, an amplifier 38 and a DA converter 39, and a microphone 40 for receiving a sound wave returned from the item to be sorted. And an amplifier 41 for amplifying the output of the microphone 40, an A / D converter 42 for converting the output of the amplifier 41 into a digital signal, and a signal given from the A / D converter 42, which will be described later. A computer 43 for performing processing, a quality class sorting device 44 for selecting items according to control signals output from the computer 43, a display device 45 for displaying information output by the computer 43, and a computer And an alarm device 46 for issuing a warning according to the output of 43.

The processing performed by the computer 43 is shown in FIG.
6 is shown. The computer 43 generates a transmission signal for a sound emitted from the speaker 37 toward the item, and supplies the transmission signal to the DA converter 39. The computer 43 is further reflected by the product in response to the output signal generation parameter and the sound generated from the speaker 37, and is converted into an electric signal through the microphone 40, the amplifier 41, and the A / D converter 42. Based on the received signal given to 43, a stabilized wavelet transform, a size-shape image, and a calculation of a Merin image are executed,
Get an expression about the internal state of an item, independent of the size of the item. The computer 43 determines the quality grade of the item by comparing the obtained expression with a standard template stored in advance, and outputs the determination result. If the difference between the output and the standard template is larger than a predetermined value, the computer 43 determines that the item is defective and outputs a diagnosis result by the display device 45 and the alarm device 46.

According to the apparatus of this embodiment, it is possible to carry out effective sorting depending only on the internal state of the article without depending on the size of the article having the variation. This system is
The present invention can be applied not only to the above-mentioned articles but also to the diagnosis of the body and the determination of defects of products such as iron, metal products, and ceramics. Fourth Embodiment A device according to a fourth embodiment has basically the same configuration as that of the third embodiment, and has a display device 45 (monitor) for displaying an image calculated by a computer. Etc.). The display device 45 provides a means for visually presenting an expression whose size has been normalized, so that a human can directly determine the characteristics of an object. Further, if a device 46 for making a defect judgment and sounding an alarm is provided, a defect of the device can be automatically diagnosed. As a result, the present invention can be applied not only to the third embodiment but also to sonar signal processing in general.

Various other applications of the present invention are conceivable. For example, an expression independent of the size of an object can be obtained by the present invention. Therefore, in the field of architecture, if a measurement is performed using a miniature model of a concert hall, the acoustic characteristics of the concert hall after construction can be predicted. Diagnosis of aging by sound waves of the building structure itself is also included. Further, application to analysis of sonar signals in water is also possible. Fifth Embodiment FIG. 47 shows a fifth embodiment in which the present invention is applied to failure diagnosis of engines of various sizes. An output signal of a vibration sensor or a microphone 47 attached to an engine of a car, a ship, or the like is input to a computer 50 through an amplifier 41 and an AD converter 42. The computer 50 determines a defect or a failure, and the display device 51, the alarm device 52, and the engine control device 53 of the information are controlled. A device 54 for directly outputting an image is also provided.

In this computer 50, the processing shown in FIG. 48 is performed. Referring to FIG. 48, a stabilized wavelet transform is performed based on the input quasi-periodic signal, and a size-shape image and a Merin image are calculated from the result. With these images,
The state of the engine is diagnosed by comparing with a standard template stored in advance, and the result is output. At this time, if a binary signal indicating the presence or absence of a defect is obtained as a result, the defect / fault display device and the alarm device can be controlled by this signal. In contrast,
It is also possible to determine a distance measure with the standard pattern in advance, calculate the distance of how similar the distance is, and output the distance as a continuous amount. Since this information indicates the degree of abnormality such as rotation of the engine, it can be used as a signal for controlling the control device of the engine. In addition, if an image is directly output, a human can visually determine a failure.

Even if the shape of the engine is the same, the displacement varies depending on the purpose. In the same engine family, even if the dimensions are different, the state can be determined by using the same expression when using the present invention. Therefore, the engine state determination device according to the present invention can effectively determine a common cause of failure for engines of various sizes.

Furthermore, if the output from the sensor attached to the building is used, it can be applied to the defect diagnosis of the building. If the signal of the seismic wave is used, a common feature independent of the size of the epicenter can be found. it can. Further, according to the present invention, regardless of whether the signal is an artificial object, a natural object, or a signal measured by any physical system, any signal may be input as long as it is a signal from a signal source. For example, if you pick up biological signals such as heart beat sounds and brain wave signals,
Since an expression independent of the size of the body and the size of the head can be obtained, a good diagnosis result can be obtained.

As described above, according to the stabilized wavelet-Mellin transform according to the present invention, a signal expression that does not basically depend on the physical size of a sound source (for example, in the case of voice, a vocal tract that differs depending on men, women, and children) In the case of time-series data, an expression in which self-similarity (fractality) is normalized can be obtained. That is,
For an event in which a part constituting a large part has a common configuration with the original large part, the same expression can be obtained for both the large part and the small part constituting the large part. This is difficult to perform with a conventional autoregressive model or spectrum analysis, and enables signal processing that can exceed the limit of conventional time-series data processing. In addition, elements that cannot be normalized in this process can be separated in reverse, so that voice can be effectively used for personal authentication and the like.
As described above, the present invention can be widely used for signal processing that requires normalization of the physical size and self-similarity of a sound source.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

The following is an appendix cited in the description.

[0110]

(Equation 1)

[0111]

(Equation 2)

[0112]

(Equation 3)

[0113]

(Equation 4)

[Brief description of the drawings]

FIG. 1 is a schematic block diagram illustrating the principle of the present invention.

FIG. 2 is a block diagram of a stabilized wavelet processing unit 2 of FIG.

FIG. 3 is a flowchart related to FIGS. 1 and 2;

FIG. 4 is a schematic block diagram of a voice recognition device according to the first embodiment of the present invention.

5 is a block diagram of an event detection (pitch detection) circuit 15 and a stabilized auditory image processing unit 16 of FIG.

FIG. 6 is a flowchart related to FIGS. 4 and 5;

FIG. 7 is a diagram illustrating an example of a stabilized auditory image of a click sequence sound.

8 is a diagram showing a stabilized auditory image corrected from FIG. 7 by an amount corresponding to a delay of a filter.

9 is a diagram showing a stabilized auditory image in which the time interval axis of the horizontal axis in FIG. 8 is logarithmically converted and displayed.

FIG. 10 is a diagram showing a stabilized auditory image in which the impulse responses of the wavelet filters in all the channels are corrected so as to be aligned in the vertical direction.

FIG. 11 shows the stabilized auditory image shown in FIG.
FIG. 7 is a diagram in which the time interval frequency product h on the horizontal axis is converted and represented on a linear axis.

FIG. 12 is a diagram showing a melin image of a click sequence sound.

FIG. 13 is a diagram showing an auditory image of an exponentially attenuated sine wave.

FIG. 14 is a diagram showing a size-shape image of an exponentially attenuated sine wave.

FIG. 15 is a diagram showing a Merin image of an exponentially attenuated sine wave.

FIG. 16 is a diagram showing an auditory image (repetition frequency of vocal cord pulses of 100 Hz) of a Japanese vowel 'a' synthesized from a vocal tract model using the measured male speaker's vocal tract cross-sectional area function.

FIG. 17 is a diagram showing an auditory image of a Japanese vowel 'a' synthesized under the same conditions as in FIG. 16 but at a repetition frequency of vocal cord pulses of 160 Hz.

FIG. 18 shows an auditory image of Japanese vowel 'a' synthesized from a vocal tract model by reducing the vocal tract length to 2/3 of the vocal tract cross-sectional area function of FIG.
0 Hz).

FIG. 19 is a diagram showing an auditory image of a Japanese vowel 'a' synthesized under the same conditions as in FIG. 18 but at a repetition frequency of a vocal cord pulse of 160 Hz.

FIG. 20 is a diagram showing a size-shape image for FIG. 16;

FIG. 21 is a diagram showing a size-shape image for FIG. 17;

FIG. 22 is a diagram showing a size-shape image for FIG. 18;

FIG. 23 is a diagram showing a size-shape image for FIG. 19;

FIG. 24 is a view showing a merin image for FIG. 16;

FIG. 25 is a diagram showing a merin image for FIG. 17;

FIG. 26 is a diagram showing a merin image for FIG. 18;

FIG. 27 is a diagram showing a merin image for FIG. 19;

FIG. 28 is the same diagram as FIG. 16, showing an auditory image (repetition frequency of vocal cord pulse of 100 Hz) of Japanese vowel 'a' synthesized from a vocal tract model using the measured vocal tract cross-sectional area function.

29 is an auditory image of a Japanese vowel 'e' synthesized from a vocal tract model using a vocal tract cross-sectional area function of 'e' measured by the same male speaker as in FIG.
0 Hz).

FIG. 30 shows an auditory image of a Japanese vowel 'i' synthesized from a vocal tract model using a vocal tract cross-sectional function of 'i' measured by the same male speaker as in FIG.
0 Hz).

FIG. 31 shows an auditory image of a Japanese vowel 'o' synthesized from a vocal tract model using the vocal tract cross-sectional function of 'o' measured by the same male speaker as in FIG. 28 (repetition frequency of vocal cord pulse of 10)
0 Hz).

FIG. 32 shows an auditory image of a Japanese vowel 'u' synthesized from a vocal tract model using a vocal tract cross-sectional function of 'u' measured by the same male speaker as in FIG.
0 Hz).

FIG. 33 is a diagram showing a size-shape image for FIG. 28;

FIG. 34 is a diagram showing a size-shape image for FIG. 29;

FIG. 35 is a diagram showing a size-shape image for FIG. 30;

FIG. 36 is a diagram showing a dimension-shape image for FIG. 31;

FIG. 37 is a diagram showing a size-shape image for FIG. 32;

FIG. 38 is a view showing a merin image for FIG. 28;

FIG. 39 is a diagram showing a merin image for FIG. 29;

FIG. 40 is a view showing a merin image with respect to FIG. 30;

FIG. 41 is a diagram showing a merin image for FIG. 31.

FIG. 42 is a diagram showing a merin image for FIG. 32;

FIG. 43 is a block diagram of a speech training device according to a second embodiment.

FIG. 44 is a flowchart of a process performed by the general-purpose computer according to the second embodiment.

FIG. 45 is a block diagram of an article quality classifying apparatus according to a third embodiment and a sonar system according to a fourth embodiment.

FIG. 46 is a flowchart of a process performed by a computer according to the third embodiment and the fourth embodiment.

FIG. 47 is a block diagram of an engine failure diagnosis device according to a fifth embodiment.

FIG. 48 is a flowchart of a process performed by a computer according to the fifth embodiment.

[Explanation of symbols]

2 stabilizing wavelet transform processing section, 3 Merlin transform processing section, 4 signal processing section, 7 wavelet transform section,
8 amplitude compression section, 9 event detection processing section, 10-hour interval stabilization processing section, 13 auditory filter bank, 14 auditory nerve firing pattern conversion section, 15 event detection circuit, 16 stabilized auditory image processing section, 17 dimension-shape image processing , 18 Merin image processing unit, 19 speech recognition circuit, 22 filter delay correction unit, 25 auditory figure extraction unit, 26 conversion unit to logarithmic time interval expression, 27 correction unit for impulse response.

 ────────────────────────────────────────────────── ─── Continuing on the front page (72) Inventor Toshio Irino 5th Sanraya, Inaya, Seika-cho, Soraku-gun, Kyoto Pref. ATI Human Information and Communication Research Laboratories Co., Ltd. (72) Inventor Roy Di Patterson United Kingdom, W1N4A1L London, Park Crescent, within 20 Medical Research Councils

Claims (16)

[Claims] 1. A wavelet transforming step of performing a wavelet transform on an input signal by a computer, and a characteristic extracting step of extracting a characteristic of the signal by performing a Mellin transform on the output of the wavelet transforming in a computer in synchronization with the input signal. And a signal processing method. 2. The method according to claim 1, wherein the characteristic extracting step comprises: stabilizing a representation corresponding to the running spectrum obtained in the wavelet transform step with time while maintaining a fine structure of a response waveform by signal synchronization;
Converting to a logarithmic frequency expression, and performing a process corresponding to the Mellin transformation along a line where the value of the product or ratio of the time interval and the frequency is constant in the time interval-logarithmic frequency expression The signal processing method according to claim 1. Obtaining a logarithmic time-interval logarithmic frequency expression obtained by logarithmically converting the time-interval axis in a stabilized time interval-logarithmic frequency expression of the input signal whose origin is specified; Using a computer, the logarithmic time interval-logarithmic frequency expression is converted into a new expression having the product of the time interval and frequency on the horizontal axis and the logarithmic frequency on the vertical axis, and expressed in the vertical or horizontal axis direction. Extracting the characteristic of the vibration by performing an integral conversion. 4. The signal processing method according to claim 3, further comprising the step of expressing the expression space obtained by the integral transformation as a time series of expression vectors at a certain point. 5. The method according to claim 1, further comprising the step of providing an output obtained by subjecting a signal converted into a format that can be processed by a computer to a frequency analysis in consideration of auditory characteristics to the melin conversion step. The signal processing method according to any one of the above. 6. A wavelet transform means for performing a wavelet transform on an input signal converted into a predetermined format which can be processed by a computer, and a Merin transform in synchronizing an output of the wavelet transform means with the input signal to obtain a signal. A signal extraction device for extracting characteristics. 7. The characteristic extracting means, wherein the expression corresponding to the running spectrum obtained by the wavelet transform means is temporally stabilized by signal synchronization while maintaining a fine structure of a response waveform, and a time interval-logarithmic frequency expression is provided. And means for performing a process equivalent to the Mellin transform along a line where the value of the product or ratio of the time interval and the frequency is constant in the time interval-logarithmic frequency expression. The signal processing device according to claim 6, comprising: 8. In a stabilized time interval-log frequency expression in which an origin is specified of an input signal converted into a form that can be processed by a computer, a log time interval-log frequency expression in which a time interval axis is logarithmically converted is used. Means for obtaining, and further converting the logarithmic time interval-logarithmic frequency expression into a new expression having a product of the time interval and frequency on the horizontal axis and a logarithmic frequency on the vertical axis, in the vertical axis direction or the horizontal axis direction. Means for extracting the characteristic of the vibration by performing integral conversion on the expression. 9. The signal processing apparatus according to claim 8, further comprising means for expressing the expression space obtained by the integration transformation as a time series of expression vectors at a certain point. 10. The apparatus according to claim 6, further comprising means for providing an output obtained by subjecting a signal converted into a format that can be processed by a computer to a frequency analysis in consideration of auditory characteristics to the Mellin transform. The signal processing device according to any one of the above. 11. A wavelet bank comprising a plurality of wavelet filters, each connected to receive an input signal, and each of which is transformed by a wavelet having the same wavelet kernel function and having a different frequency. An auditory figure extracting means for extracting an auditory figure from an output of the wavelet bank; and generating a size-shape image of the input signal from the auditory figure extracted by the auditory figure extracting means. A signal processing apparatus, comprising: a size-shape image generating means for performing the processing; 12. The merin image generating means for generating a merin image by performing a Fourier transform on the size-shape image along an impulse response line of each of the wavelet filters. The signal processing device according to claim 11. 13. The auditory figure extracting means detects a periodicity included in an output of the wavelet filter bank, and performs a time strobe integration on an output of each channel of the wavelet filter bank, thereby stabilizing the output. A time strobe integration means for generating an auditory image, based on the periodicity detected by the time strobe integration means, a predetermined one cycle of the stabilized auditory image obtained by the time strobe integration, 13. The signal processing device according to claim 12, further comprising a stabilized auditory image extracting unit for extracting the auditory graphic. 14. The stabilized auditory image extracting means,
14. The signal processing device according to claim 13, further comprising means for extracting a first cycle of the stabilized auditory image as the auditory graphic. 15. The stabilized auditory image extracting means,
14. The signal processing device according to claim 13, further comprising: means for extracting a second period of the stabilized auditory image as the auditory graphic. 16. An auditory nerve for converting the output of the filter bank so that the output of the filter bank becomes an output similar to the nerve activity of the auditory nerve, and providing the output to the auditory figure extracting means. The signal processing device according to claim 11, further comprising a firing pattern conversion unit.