JPWO2007015489A1 - Voice search apparatus and voice search method - Google Patents

Abstract

Provided is a voice search device that does not require a standard voice pattern and has high search accuracy without being affected by individual differences in voice. Pitch equalization by equalizing pitch period of voiced sound of search target voice data Pitch equalization query voice by equalizing pitch period of voiced sound of query voice data in voice feature value space from search target voice data A partial voice search means for searching for partial voice data whose distance measure with respect to the data is equal to or less than a predetermined threshold value is provided. By equalizing the pitch period, it is possible to perform a voice search with high accuracy with little influence on gender differences and individual differences in the voice band.

Description

  The present invention relates to a voice search device for searching a portion that matches a predetermined voice from stored search target voice data.

  In recent years, there has been an increasing demand for a multimedia database that extracts only the portion of information that the viewer wants to know most from a large amount of stored video / audio data. A typical example is a News On Demand (NOD) system that extracts only the news that the viewer wants to know most from among many accumulated news programs.

  In order to construct such a multimedia database, a voice search technique for searching a portion that matches a search keyword voice (hereinafter referred to as “query voice”) from stored video / audio data such as TV news. Is needed.

  As a voice search device for searching a part that matches a query voice from search target voice data, the one described in Patent Document 1 is known.

  FIG. 12 is a diagram illustrating the configuration of the voice search device described in Patent Document 1. In this voice search device, when a voice signal is input to the voice signal input unit 102 of the search data generation unit 100, the voice signal is stored in the recording unit 201 as search target voice data. At this time, a video search index generated by the video search index generation unit 104 is added. In addition, a video signal is input to the video signal input unit 101 in synchronization with the audio signal, and stored as accumulated video data in the recording unit 201. On the other hand, the query voice is input from the keyword input unit 203 of the search processing unit 200, checked with the search target voice data in the keyword pattern matching unit 205, and the voice signal that most matches is output from the voice signal output unit 207. Hereinafter, these processes will be outlined.

  First, when an audio signal is input to the audio signal input unit 102, the audio feature pattern extraction unit 103 divides the input audio into 10 msec analysis frames. Then, fast Fourier transform is performed on each analysis frame to generate acoustic characteristic data in the generated frequency band. Further, the acoustic characteristic data is converted into N-dimensional vector data (hereinafter referred to as “feature pattern”) composed of acoustic feature amounts. Here, as the acoustic feature amount, a short-time spectrum or its logarithmic value in the generation frequency band of the input speech, logarithmic energy within a certain time of the input speech, or the like is used.

  Next, the video search index generation unit 104 extracts the first standard audio pattern from the audio feature pattern storage unit 105.

  Here, 500 standard sound patterns are stored in the sound feature pattern storage unit 105 in advance. The standard voice pattern is an analysis of pronunciations collected from multiple speakers in advance, subword units (#V, #CV, #CjV, CV, CjV, VC, QC, VQ, VV, V #: where C Is a consonant, V is a vowel, j is a stutter, Q is a prompt, and # is silence.

  The video search index generation unit 104 uses the DP matching method or HMM (Hidden Markov Model) to calculate the similarity between the first standard audio pattern and the audio feature pattern of the input audio for one audio section to be processed. It is calculated by voice recognition processing such as. Then, the section showing the highest similarity to the first standard speech pattern is detected as the “subword section”. Hereinafter, the similarity between subword sections is referred to as “score”. The video search index generation unit 104 outputs a set of a phoneme symbol of a subword section, a speech section (start time and end time), and a score as a “video search index”.

  Similarly, a subword section is detected for the second and subsequent standard audio patterns, and a video search index related to the detected subword section is output.

  If the video search index is generated for all the standard audio patterns in the audio section, the video search index generation unit 104 moves the audio section to be processed to the next adjacent audio section and executes the same processing. To do. Then, when the video search index is created over the entire section of the input audio, the process is terminated.

  The audio data of the input audio and the video search index are stored in the recording unit 201 as search target audio data. FIG. 13 is a diagram showing a part of the lattice structure of the video search index stored in the recording unit 201. In FIG. 13, the end of each audio section of the input audio divided in units of 10 msec is the end of each video search index generated for that audio section, and the video search indexes in the same audio section are arranged in the order of generation. ing. Such a lattice structure of the video search index is called a “phoneme similarity table”. “Lattice” refers to a table in which a plurality of phoneme and word candidates and their possibilities are represented in a table form for various continuous speech segments (see Non-Patent Document 1, p. 198). .

  The process of searching for a video scene using the query audio is performed as follows. First, a query voice that is a search keyword is input to the keyword input unit 203. The keyword conversion unit 204 converts the query speech into a time series of subwords. Next, the keyword pattern matching unit 205 picks up only the subwords constituting the query speech from the phoneme similarity table. Then, the sub-words on the plurality of lattices that have been picked up are connected without gaps in the order of the sub-words obtained by converting the search keyword.

  For example, when “empty” is input to the keyword input unit 203 as the query voice, the keyword conversion unit 204 generates subword sequences “SO”, “OR”, and “RA”. The keyword pattern matching unit 205 picks up subwords “SO”, “OR”, and “RA” from the phoneme similarity table and connects them without gaps. In this case, the subword “RA” is taken out from the lattice at a certain time, the subword “OR” before it is taken out from the lattice corresponding to the start time of the subword “RA”, and the subword “SO” is taken from the lattice corresponding to the start time of the subword “OR”. Take out. Then, “SO”, “OR”, and “RA” are concatenated based on the end of the last subword “RA”.

  Thus, for the keyword restored by concatenating the subwords (in the above example, “SO”, “OR”, “RA”), the sum of the scores of the restored keyword is calculated.

  In the same manner, a restoration keyword in which the end time of the subword “RA” is shifted is sequentially generated for all times, and the score is calculated for each restoration keyword (see FIG. 14).

The control unit 202 calculates the time code of the corresponding video signal from the start end time of the first subword of the restoration keyword having the highest score. Then, control is performed to reproduce the corresponding portions of the stored video data and search target audio data stored in the storage unit 201.
JP 2000-236494 A (Patent No. 3252282) JP 2005-91709 A Furui Sadaaki, “Acoustic / Voice Engineering”, Modern Science, pp. 194-210

  In the conventional speech search device, when performing speech recognition, the speech recognition is performed based on the similarity between the query speech and the standard speech pattern using the standard speech pattern stored in the speech feature pattern storage unit 105. In this case, in order to increase the recognition accuracy, it is necessary to prepare many standard voice patterns. However, as the number of standard voice patterns increases, the processing time for similarity calculation increases or the scale of the arithmetic circuit increases. In addition, when a query voice that is not registered as a standard voice pattern is input, it cannot be recognized normally, and the voice search function may not work normally.

  In general, even if the speech is for the same phoneme, the frequency band is different between men and women, and the frequency band is different between individuals even in the same gender. Therefore, since the influence of these differences appears on the similarity between the standard voice pattern and the query voice, the recognition accuracy is limited.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a voice search device that does not require a standard voice pattern and has high search accuracy without being affected by individual differences in voice.

  The first configuration of the voice search device according to the present invention is that partial voice data (partial voice-data) that matches or is similar to query voice data (query voice-data) from search target voice data (retrieval voice-data). ) For voice equalization search target voice data (pitch-equalized) that equalizes the pitch period of voiced sound of the search target voice data. The distance measure (or similar measure) for the pitch equalized query voice data obtained by equalizing the pitch period of the voiced sound of the query voice data in the voice feature value space. )) Is provided with a partial voice search means for searching for partial voice data that is equal to or lower than a predetermined threshold (or higher than a predetermined threshold).

  In this way, by equalizing the pitch periods of the search target voice data and the query voice data, gender differences and individual differences in the voice band are removed. Therefore, the distance measure and similarity measure in the feature amount space of the search target speech signal and the query speech signal in which the pitch period is equalized are hardly affected by the gender difference or individual difference in the speech band, and the phoneme string represented by the speech is represented. It depends on you. Therefore, by using this distance measure or similarity measure as a matching index, it is possible to perform a voice search with high accuracy.

  Here, as the “feature amount”, a short-time spectrum in the voice generation frequency band or its logarithmic value, logarithmic energy within a certain time, or the like can be used. When using a short-time spectrum as a feature amount, for example, a time series of feature data of each band obtained using a band filter group of about 10 to 30 channels, a spectrum calculated directly using a short-time FFT, A cepstrum obtained by cepstrum conversion, a correlation data string calculated by a correlation function, an LPC coefficient string obtained based on LPC analysis, a PARCOR coefficient, an LSP frequency, and the like are used as feature quantities.

  As the “distance scale”, various distance scales can be used according to the feature amount. For example, when a short-time spectrum is used as a feature quantity, a simple Euclidean distance, a weighted distance that takes auditory sensitivity into account, a space that is projected to a low dimension by statistical analysis such as discriminant analysis and principal component analysis Euclidean distance, Maharabinos distance, Itakura / Saito distance, COSH scale, WLR scale (weighted likelihood ratio), PWLR scale (power weighted likelihood ratio), Euclidean distance between LPC cepstrum, Euclidean distance between LPC weighted cepstrum, etc. Can be used.

  Note that the distance measure d (x, y) of the feature quantities (generally vector quantities) x and y does not necessarily satisfy the triangular inequality like the distance in the mathematical sense. However, it is desirable to have symmetry and positive value defined by the following equation, and an algorithm for efficiently calculating d (x, y) needs to exist.

  The “similarity scale” refers to a scale indicating how similar two feature quantities are. For example, a similarity that can be defined by the following equation can be used. Here, x and y represent feature amounts.

  A second configuration of the voice search device according to the present invention is the pitch configuration in which the pitch equalized query voice data is generated by equalizing the pitch period of the voiced sound of the query voice data in the first configuration. And feature data generation means for generating data obtained by converting the pitch equalization query voice data into time-series data of feature quantities (hereinafter referred to as “query feature-data”), The partial speech search means has a feature measure having a distance measure (or similarity measure) between the partial feature data included in the pitch equalization search target speech data and the query feature data equal to or less than a predetermined threshold ( (Or a predetermined threshold value or more) is searched.

  With this configuration, when query voice data is input, the pitch period equalizing means equalizes the pitch period of the voiced sound of the query voice data. Then, the feature data generation means calculates the feature amount of the query voice data with the equal pitch period, and generates query feature data. Thereby, the partial speech search means extracts a distance measure (or similarity measure) between the partial speech data of the pitch equalization search target speech data and the query feature data by threshold determination. Thereby, it is possible to search the search target voice data for voice data that matches or is similar to the query voice data.

  According to a third configuration of the voice search device of the present invention, in the first or second configuration, the partial voice search means is a search target obtained by converting the pitch equalization search target voice data into time-series data of feature quantities. Partial voice selection means for sequentially selecting partial data for the same phoneme length as the query voice data (hereinafter referred to as “selected feature data”) from the feature data while moving the selection position; and each selected feature data If the distance measure (or similarity measure) is less than or equal to a predetermined threshold (or greater than or equal to a predetermined threshold), And a matching position determining means for outputting a position in the search target voice data corresponding to the selected feature data.

  With this configuration, partial speech data having a pitch equalization search target speech data in the feature amount space (or similarity measure) equal to or smaller than a predetermined threshold (or greater than a predetermined threshold) is extracted from the search target speech data. Is possible.

  The procedure for “moving the selected position” by the partial voice selecting means is not particularly limited. For example, a method of sequentially moving the start position of partial audio data from the beginning to the end of the search target audio data, or conversely, the end position of partial audio data is sequentially moved from the end to the beginning of the search target audio data. You can take methods.

  A fourth configuration of the voice search device according to the present invention is characterized in that, in the third configuration, voice search means for storing the search target feature data is provided.

  By storing the search target voice data as search target feature data in the voice storage means in advance, it becomes possible to quickly search for partial voice data similar to the query voice data.

  According to a fifth configuration of the voice search device of the present invention, in the third or fourth configuration, the pitch equalization search target voice data is generated by equalizing the pitch period of the voiced sound of the search target voice data. Second pitch period equalizing means, and second feature data generating means for generating the search target feature data by converting the pitch equalization search target voice data into time-series data of feature quantities, It is characterized by having.

  With this configuration, the second feature data is obtained by equalizing the pitch period by the second pitch period equalizing means even when the search target voice data in the voice database is not equalized in the pitch period of the voiced sound. By calculating the feature value by the generation unit, it is possible to obtain the feature value of the search target speech data in which the pitch period is equalized.

  According to a sixth configuration of the voice search device of the present invention, in the second or fifth configuration, the pitch period equalizing means (or the second pitch period equalizing means) is configured to use the query voice data (or the search). Pitch detection means for detecting the pitch frequency of the target voice data), residual calculation means for calculating the difference between the pitch frequency and a predetermined reference frequency, and the query voice data ( Or a frequency shifter for equalizing the pitch frequency of the search target voice data).

  With this configuration, the pitch period equalizing means (or the second pitch period equalizing means) can equalize the pitch frequency of the query voice data (or the search target voice data).

  A seventh configuration of the speech search device according to the present invention is the configuration according to any one of the first to sixth configurations, wherein the search target feature data and the query feature data are the pitch equalization search target speech data and It is a time series of subband data obtained by orthogonal transformation of the pitch equalization query voice data.

  By using subbands as feature quantities in this way, it is possible to obtain the search target feature data and the query feature data at high speed using a simple filter bank, FFT, DFT, or the like.

  According to an eighth configuration of the speech search apparatus of the present invention, in the second or fifth configuration, the query feature data is averaged for each phoneme segment, and converted into time-series data of an average value. And a second section dividing means for averaging the search target feature data for each phoneme section and converting it into time-series data of average values, wherein the feature quantity scale calculating means includes the first and second feature scale calculation means. A distance scale (or similarity scale) between the time series data of the average values generated by the two section dividing means is calculated.

  In this way, by averaging the feature values in the phoneme section and performing the matching determination using the average value, the influence of noise and fluctuation is reduced, and the search accuracy is improved. Each feature is discretized in time for each phoneme section. At this time, the influence of voice expansion and contraction is removed. Therefore, the matching determination is only a simple comparison calculation, and it is not necessary to use a method with a large amount of calculation like DP matching, so that the apparatus configuration can be simplified and the calculation time can be increased.

  According to a ninth configuration of the speech search apparatus of the present invention, a query phoneme is obtained by performing phoneme labeling on the query speech data (or the search target speech data) in any one of the first to eighth configurations. Phoneme labeling processing means for generating a sequence (or search target phoneme sequence), and a phoneme sequence scale for determining a distance measure (or similarity measure) between the search target phoneme sequence corresponding to the selected feature data and the query phoneme sequence A linear sum (hereinafter referred to as a calculation means), a distance measure (or similarity measure) of the feature amount output by the feature amount scale calculation means, and a distance measure (or similarity measure) of the phoneme string scale output by the phoneme sequence scale calculation means. Total scale calculation means for calculating “total distance scale (or total similarity scale)”, and the coincidence position determination means has the total distance scale (or total similarity scale). For the following constant threshold (or above a predetermined threshold value), and outputs the position of the searched audio data corresponding to the selected feature data.

  In this way, by considering the phoneme string scale in addition to the feature quantity scale in the matching determination, the search accuracy can be improved.

  The voice search method according to the present invention is a voice search method for searching partial voice data that matches or is similar to query voice data from search target voice data, and calculates a pitch period of voiced sound of the search target voice data. A distance scale (or similarity measure) for pitch equalized query voice data obtained by equalizing the pitch period of the voiced sound of the query voice data in the voice feature value space from the equalized pitch equalization search target voice data. It has a partial voice search step of searching for partial voice data that is below a predetermined threshold (or above a predetermined threshold).

  The second configuration of the speech search method according to the present invention is the pitch configuration in which the pitch equalized query speech data is generated by equalizing the pitch cycle of the voiced sound of the query speech data in the first configuration. And a feature data generation step for generating data obtained by converting the pitch equalization query voice data into time-series data of feature quantities (hereinafter referred to as “query feature data”), and in the partial voice search step, Is a feature of the partial speech data included in the pitch equalization search target speech data, and the distance measure (or similarity measure) between the feature data and the query feature data is less than or equal to a predetermined threshold (or greater than or equal to a predetermined threshold) ) Is searched for.

  A third configuration of the speech search method according to the present invention is the search in which the pitch equalization search target speech data is converted into time-series data of feature amounts in the partial speech search step in the first or second configuration. A partial voice selection step of sequentially selecting partial data for the same phoneme length as the query voice data (hereinafter referred to as “selected feature data”) from the target feature data while moving the selection position; A feature amount scale calculating step for calculating a distance measure (or similarity measure) between data and the query feature data, and when the distance measure (or similarity measure) is a predetermined threshold value or less (or a predetermined threshold value or more), A matching position determining step of outputting a position in the search target voice data corresponding to the selected feature data.

  The fourth configuration of the speech search method according to the present invention is characterized in that, in the third configuration, a speech storage step of storing the search target feature data is provided.

  According to a fifth configuration of the voice search method of the present invention, in the third or fourth configuration, the pitch equalization search target voice data is generated by equalizing the pitch period of the voiced sound of the search target voice data. A second pitch period equalization step, and a second feature data generation step of generating the search target feature data by converting the pitch equalization search target speech data into feature amount time-series data. It is characterized by that.

  In a sixth configuration of the speech search method according to the present invention, in the second or fifth configuration, in the pitch period equalizing step (or the second pitch period equalizing step), the query voice data (or the A pitch detection step for detecting a pitch frequency of the search target voice data), a residual calculation step for calculating a difference between the pitch frequency and a predetermined reference frequency, and the query voice data so that the difference is minimized. And a frequency shift step for equalizing the pitch frequency of the search target voice data.

  According to a seventh configuration of the speech search method of the present invention, in any one of the first to sixth configurations, the search target feature data and the query feature data are the pitch equalization search target speech data and It is a time series of subband data obtained by orthogonal transformation of the pitch equalization query voice data.

  According to an eighth configuration of the speech search method of the present invention, in the second or fifth configuration, the query feature data is averaged for each phoneme segment and is converted into average time-series data. And a second section dividing step of averaging the search target feature data for each phoneme section and converting it into time-series data of average values. In the feature quantity scale calculation step, And calculating a distance measure (or similarity measure) between the time series data of the average values generated in the second interval dividing step.

  According to a ninth configuration of the speech search method of the present invention, a query phoneme is obtained by performing phoneme labeling on the query speech data (or the search target speech data) in any one of the first to eighth configurations. Phoneme labeling step of generating a sequence (or search target phoneme sequence), and phoneme sequence scale calculation step of determining a distance measure (or similarity measure) between the search target phoneme sequence and the query phoneme sequence corresponding to the selected feature data And a linear sum of the distance measure (or similarity measure) of the feature amount output in the feature amount scale operation step and the distance measure (or similarity measure) of the phoneme sequence output in the phoneme sequence scale operation step (hereinafter, A comprehensive scale calculation step of calculating “total distance scale (or total similarity scale)”, and in the matching position determination step, If the overall distance measure (or total similarity measure) is below a predetermined threshold value (or above a predetermined threshold value), and outputs the position of the searched audio data corresponding to the selected feature data.

  The program according to the present invention is read and executed by a computer, thereby causing the computer to function as the voice search device having any one of the first to eighth configurations.

  As described above, according to the present invention, by matching the pitch period of the search target voice data and the query voice data, matching the feature amount using the voice data from which the gender difference or individual difference in the voice band is removed. By performing the voice search according to the above, it is possible to improve the accuracy of the voice search with almost no influence on the gender difference or individual difference in the voice band.

  In addition, by averaging the feature quantities of search target voice data and query voice data with equal pitch period for each phoneme section, and performing a voice search by time series matching inspection of the average value of the feature quantities, noise and fluctuations are obtained. Is reduced, and the influence of the expansion and contraction of the voice is removed. As a result, the accuracy of voice search can be improved.

It is a figure showing the whole structure of the voice search device 1 which concerns on Example 1 of this invention. It is a block diagram showing the structure of the audio | voice encoder 2 of FIG. It is a block diagram showing the structure of the pitch period equalization means 10 of FIG. It is a figure explaining the outline of the signal processing in the pitch detection means 21 and the pitch average means 22. FIG. It is a figure which shows the formant characteristic of voiced sound "A". It is a figure which shows the autocorrelation and cepstrum waveform of unvoiced sound "su", and a frequency characteristic. 3 is a diagram illustrating an internal configuration of a frequency shifter 23. FIG. 6 is a diagram illustrating another example of the internal configuration of the frequency shifter 23. FIG. It is a block diagram showing the structure of the audio | voice decoder 5 of FIG. It is a block diagram showing the structure of the partial voice search means 6 of FIG. It is explanatory drawing about the number of quantization bits. 1 is a diagram illustrating a configuration of a voice search device described in Patent Document 1. FIG. 6 is a diagram showing a part of a lattice structure of a video search index stored in a recording unit 201. FIG. It is a figure showing the structure of the lattice connected in order to calculate the score about each restoration keyword.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Speech search device 2 Speech encoder 3 Speech storage means 4 Data reading means 5 Speech decoder 6 Partial speech search means 10 Pitch period equalization means 11 Feature data generation means 12a, 12b Output switching means 13 Quantizer 14 Pitch etc. Waveform encoder 15 difference bit calculator 16 pitch information encoder 17 phoneme labeling processing means 18 resampler 19 analyzer 20 resistor 21 input pitch detection means 22 pitch averaging means 23 frequency shifter 24 output pitch detection means 25 residual calculation means 26 PID controller 27 Pitch detection means 28 BPF
29 Frequency counter 31 BPF
32 Frequency counter 34 Amplifier 36 Capacitor 41 Transmitter 42 Modulator 43 BPF
44 VCO
45 Demodulator 51 Pitch equalization waveform decoder 52 Inverse quantizer 53 Synthesizer 54 Pitch information decoder 55 Pitch frequency detection means 56 Subtractor 57 Adder 58 Frequency shifter 59 Output switching means 61 Operation switching means 62 Partial voice selection means 63 64 segment division means 65 feature quantity scale calculation means 66 phoneme string scale calculation means 67 comprehensive scale calculation means 68 coincidence position determination means

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram illustrating the overall configuration of the voice search device according to the first embodiment of the present invention. The speech search apparatus 1 according to the first embodiment includes a speech encoder 2, speech storage means 3, data reading means 4, speech decoder 5, and partial speech search means 6.

  Search target speech data and query speech data are input to the speech coder 2 as input speech data. The speech encoder 2 equalizes the pitch period of voiced sound with respect to the input speech data and converts it into time-series data (feature data) of feature amounts. At this time, the pitch period information of the input voice data is separated from the feature data, encoded, and output as encoded pitch data. On the other hand, the feature data is output as a subband waveform. Furthermore, the speech encoder 2 encodes the feature data and outputs it as encoded feature data. The speech encoder 2 performs a phoneme labeling process on the input speech data, and outputs the phoneme label data including the phoneme label of each phoneme and information on the time interval.

  The voice storage means 3 stores the search target voice data that has been decomposed and encoded by the voice encoder 2 into encoded feature data, encoded pitch data, and phoneme label data. The encoded feature data and the encoded pitch data stored in the voice storage unit 3 are encoded search target feature data.

  The data reading means 4 reads partial data of the search target speech data (encoded feature data, encoded pitch data, and phoneme label data) in the speech storage means 3 in accordance with the data selection signal.

  The speech decoder 5 decodes the encoded feature data and the encoded pitch data read by the data reading unit 4 and outputs them as feature data or output speech data.

  The partial voice search means 6 searches for partial data that matches or is similar to the query voice data from the encoded search target voice data stored in the voice storage means 3.

  FIG. 2 is a block diagram showing the configuration of the speech encoder 2 of FIG. The speech encoder 2 includes a pitch period equalizing means 10, a feature data generating means 11, output switching means 12a and 12b, a quantizing means 13, a pitch equalizing waveform encoder 14, a differential bit calculator 15, a pitch information code. And a phoneme labeling means 17.

The pitch period equalizing means 10 equalizes the pitch period of the voiced sound of the input sound data x _in (t). Input audio data with equal pitch period (hereinafter referred to as “pitch equalized audio data”) x _out (t) is output from the output terminal Out_1.

The feature data generation unit 11 converts the pitch equalized audio data x _out (t) output from the output terminal Out_1 into time-series data of feature amounts. In this embodiment, a short-time frequency spectrum is used as the feature amount.

  The feature data generation means 11 includes a resampler 18 and an analyzer (Modified Discrete Cosine Transformer (MDCT)) 19.

The resampler 18 resamples each pitch section of the pitch equalized audio data x _out (t) output from the output terminal Out_1 of the pitch period equalizing means 100 so as to have the same number of samples, and completes the sampling. Output as equalized audio data x _eq (t).

The analyzer 19 performs a modified discrete cosine transform on the completely equalized speech data x _eq (t) with a fixed number of pitch sections, and generates a short-time frequency spectrum (hereinafter referred to as “feature data”) X (f). That is, in the present embodiment, the feature data is given as a time series of vector quantities (formula (4)) consisting of a short-time frequency spectrum.

Here, t is the _{time, X fi (t) (i} = 1,2, ..., n) represents the short-time spectral values of the sub-band of the frequency _{f i} at time t.

  The output switching unit 12 a switches the output destination of the feature data X (f) generated by the analyzer 19 to the partial speech search unit 6 or the speech storage unit 3 according to the switching signal input from the partial speech search unit 6. Specifically, when the search target voice data is input as the input voice data, the output destination of the feature data X (f) is switched to the voice storage unit 3. When query voice data is input as input voice data, the output destination of the feature data X (f) is switched to the partial voice search means 6.

  The quantizer 13 quantizes the feature data X (f) according to a predetermined quantization curve. The pitch equalization waveform encoder 14 encodes the feature data X (f) output from the quantizer 13 and outputs it as encoded feature data. For this encoding, an entropy encoding method such as a Huffman encoding method or an arithmetic encoding method is used.

  The difference bit calculator 15 subtracts the target bit number from the code amount of the encoded feature data output from the pitch equalization waveform encoder 14 and outputs a difference (hereinafter referred to as “difference bit number”). The quantizer 13 translates the quantization curve according to the difference bit number and adjusts the code amount of the encoded feature data to be within the range of the target bit number.

The pitch information encoder 16 encodes the residual period signal ΔV _pitch and the reference period signal AV _pitch output from the pitch period equalizing means 10 and outputs the encoded period data as encoded pitch data. For this encoding, an entropy encoding method such as a Huffman encoding method or an arithmetic encoding method is used.

  The phoneme labeling means 17 divides input speech data into phoneme sections and performs phoneme labeling on each phoneme section. And it outputs as phoneme label data which consists of information of a phoneme label and a time interval.

  The output switching unit 12 b switches the output destination of the phoneme label data generated by the phoneme labeling processing unit 17 to the partial speech search unit 6 or the speech storage unit 3. Specifically, when search target speech data is input as input speech data, the output destination of phoneme label data is switched to speech storage means 3. When query voice data is input as input voice data, the output destination of phoneme label data is switched to the partial voice search means 6.

  FIG. 3 is a block diagram showing the configuration of the pitch period equalizing means 10 of FIG. The pitch period equalizing means 10 includes an input pitch detecting means 21, a pitch averaging means 22, a frequency shifter 23, an output pitch detecting means 24, a residual calculating means 25, and a PID controller 26.

The input pitch detection means 21 detects the fundamental frequency of the pitch included in the audio signal from the input audio data x _in (t). Various methods for detecting the fundamental frequency of the pitch have been devised up to now, but representative examples are shown in this embodiment. The input pitch detection means 21 includes a pitch detection means 27, a band pass filter (hereinafter referred to as “BPF”) 28, and a frequency counter 29.

The pitch detection means 27 detects the basic pitch pitch T ₀ = 1 / f ₀ from the input voice data x _in (t). For example, assume that the input voice data x _in (t) has a waveform as shown in FIG. The pitch detection means 27 first performs a short-time Fourier transform on this waveform to derive a spectrum waveform X (f) as shown in FIG.

Usually, a speech waveform includes many frequency components in addition to the pitch, and the spectrum waveform obtained here additionally has many frequency components in addition to the fundamental frequency of the pitch and the harmonic component of the pitch. Accordingly, it is generally difficult to extract the fundamental frequency f _{0 of the} pitch from the spectrum waveform X (f). Therefore, the pitch detection means 27 performs Fourier transform again on the spectrum waveform X (f). Thereby, a spectrum waveform having a sharp peak at a point of the reciprocal number F ₀ = 1 / Δf ₀ of the harmonic interval Δf ₀ of the pitch included in the spectrum waveform X (f) is obtained (see FIG. 4C). Pitch detecting means 27, by detecting the position _{F 0} of the peak, to detect the fundamental frequency _{f 0 = Δf 0/2 =} F 0/2 pitches.

The pitch detection means 27 determines whether the input voice data x _in (t) is voiced sound or unvoiced sound from the spectrum waveform X (f). In the case of voiced _sound , 0 is output as the noise flag signal V _noise . In the case of an unvoiced sound, 1 is output as the noise flag signal V _noise . The distinction between voiced and unvoiced sounds is made by detecting the slope of the spectrum waveform X (f). FIG. 5 is a diagram showing formant characteristics of voiced sound “A”, and FIG. 6 is a diagram showing autocorrelation, cepstrum waveform, and frequency characteristics of unvoiced sound “su”. As shown in FIG. 5, the voiced sound has a formant characteristic in which the spectrum waveform X (f) is large on the low frequency side and smaller on the high frequency side as a whole. On the other hand, the unvoiced sound has a frequency characteristic that becomes larger toward the high frequency side as shown in FIG. Therefore, it is possible to determine whether the input voice data x _in (t) is voiced sound or unvoiced sound by detecting the overall inclination of the spectrum waveform X (f).

When the input voice data x _in (t) is an unvoiced sound, there is no pitch, so the pitch fundamental frequency f ₀ output by the pitch detection means 27 is a meaningless value.

The BPF 28 uses a narrow bandpass filter whose passband can be set from the outside. The BPF 28 sets the fundamental frequency f _{0 of the} pitch detected by the pitch detection means 27 as the center frequency of the pass band (see FIG. 4D). Then, the BPF 28 filters the input voice data x _in (t) and outputs a substantially sinusoidal waveform having the fundamental frequency f ₀ of the pitch (see FIG. 4E).

The frequency counter 29 outputs the basic pitch pitch T ₀ = 1 / f ₀ by counting the time interval of the zero cross point of the substantially sinusoidal waveform output from the BPF 28. The detected basic period T ₀ of the pitch is output as an output signal (hereinafter referred to as “basic frequency signal”) of the input pitch detecting means 21 (see FIG. 4F).

The pitch averaging means 22 averages the basic period signal T ₀ of the pitch output from the pitch detection means 27, and a normal low pass filter (hereinafter referred to as “LPF”) is used. The basic periodic signal V _pitch is smoothed by the pitch averaging means 22 and becomes a substantially constant signal in time within the phoneme. This smoothed fundamental period is used as the reference period T _s (reference frequency f _s = 1 / T _s ) (see FIG. 4G).

The frequency shifter 23 equalizes the pitch period of the audio signal by shifting the pitch frequency of the input audio data x _in (t) in a direction approaching the reference frequency f ₀ .

Output pitch detecting means 24 includes output audio data which is output from the frequency shifter 23 (hereinafter referred to as "pitch equalizing audio data".) From the x _out (t), to the pitch equalization audio data x _out (t) The basic period T ₀ ′ of the pitch is detected. The output pitch detection means 24 can also basically have the same configuration as the input pitch detection means 21. In the case of the present embodiment, the output pitch detection means 24 includes a BPF 31 and a frequency counter 32.

As the BPF 31, a narrow band BPF whose pass band can be set from the outside is used. The BPF 31 sets the basic frequency f _{0 of the} pitch detected by the pitch detection means 27 as the center frequency of the pass band. Then, the BPF 31 filters the pitch-equalized audio data x _out (t) and outputs a substantially sinusoidal waveform having the pitch fundamental frequency f ₀ ′. The frequency counter 32 outputs a basic pitch period T ₀ ′ = 1 / f ₀ ′ by counting the time interval between zero cross points of the substantially sinusoidal waveform output from the BPF 31. The detected basic pitch T ₀ ′ of the pitch is output as an output signal of the output pitch detecting means 24.

The residual calculation means 25 outputs a residual period ΔT _pitch obtained by subtracting the reference period T _s output from the pitch averaging means 22 from the basic period T ₀ ′ output from the output pitch detection means 24. This residual period ΔT _pitch is input to the frequency shifter 23 via the PID controller 26. The frequency shifter 23 shifts the pitch frequency of the input audio data in a direction approaching the reference frequency f ₀ in proportion to the residual frequency 1 / ΔT _pitch .

  The PID controller 26 includes an amplifier 34 and a resistor 20 connected in series, and a capacitor 36 connected in parallel to the amplifier 34. The PID controller 26 is for preventing oscillation of a feedback loop composed of the frequency shifter 23, the output pitch detection means 24, and the residual calculation means 25.

  In FIG. 3, the PID controller 26 displays an analog circuit, but may be configured by a digital circuit.

  FIG. 7 is a diagram illustrating the internal configuration of the frequency shifter 23. The frequency shifter 23 includes a transmitter 41, a modulator 42, a BPF 43, a voltage controlled oscillator (hereinafter referred to as “VCO”) 44, and a demodulator 45.

The transmitter 41 outputs a modulated carrier signal C ₁ having a constant frequency for performing frequency modulation of the input voice data x _in (t). Usually, the band of the audio signal is about 8 kHz (see FIG. 7 (i)). Therefore, the frequency of the modulated carrier signal C ₁ generated by the transmitter 41 (hereinafter referred to as “modulated carrier frequency”) is normally about 20 kHz.

The modulator 42 frequency-modulates the modulated carrier signal C ₁ output from the transmitter 41 with the input audio data x _in (t) to generate a modulated signal. This modulated signal is a signal having sidebands (upper sideband and lower sideband) having the same bandwidth as the audio signal band on both sides of the modulated carrier frequency (see FIG. 7 (ii)). ).

  The BPF 43 is a BPF having a modulation carrier frequency as a lower limit cutoff frequency and having a passband having a bandwidth larger than the bandwidth of the input voice data. As a result, the modulated signal output from the BPF 43 is a signal obtained by cutting out only the upper sideband (see FIG. 7 (iii)).

The VCO 44 outputs a signal having the same frequency as the modulation carrier signal C ₁ output from the transmitter 41 to a signal of the residual period ΔT _pitch (hereinafter referred to as “residual period signal”) input from the residual calculation means 25 via the PID controller 26. A signal obtained by modulating the frequency by ΔV _pitch (hereinafter referred to as “demodulated carrier signal”) is output.

The demodulator 45 demodulates the modulated signal of only the upper side band output from the BPF 43 with the demodulated carrier signal output from the VCO 44 to restore the audio signal (see FIG. 7 (iv)). At this time, the demodulated carrier signal is modulated with the residual periodic signal. Therefore, when demodulating the modulated signal, the deviation of the pitch frequency of the input audio data x _in (t) from the reference frequency f _s is eliminated. That is, the pitch period of the input voice data x _in (t) is equalized to the reference period T _s .

FIG. 8 is a diagram illustrating another example of the internal configuration of the frequency shifter 23. In FIG. 8, the transmitter 41 and the VCO 44 in FIG. 7 are replaced. Also with this configuration, the pitch period of the input audio data x _in (t) can be equalized to the reference period T _{s as} in the case of FIG.

  FIG. 9 is a block diagram showing the configuration of the speech decoder 5 of FIG. The audio decoder 5 is a device that decodes the audio signal encoded by the audio encoder 2. The speech decoder 5 includes a pitch equalization waveform decoder 51, an inverse quantizer 52, a synthesizer 53, a pitch information decoder 54, a pitch frequency detection means 55, a difference unit 56, an adder 57, a frequency shifter 58, and an output switch. Means 59 are provided.

  Encoded feature data and encoded pitch data are input to the audio decoder 5. The encoded feature data is encoded feature data output from the pitch equalization waveform encoder 14 of FIG. The encoded pitch data is encoded pitch data output from the pitch information encoder 16 of FIG.

The pitch equalization waveform decoder 51 decodes the encoded feature data and restores the quantized feature data of each subband (hereinafter referred to as “quantized feature data”). The inverse quantizer 52 inversely quantizes the quantized feature data, and the n subband feature data X (f) = {X (f ₁ ), X (f ₂ ),..., X (f _n ). } Is restored.

The synthesizer 53 performs inverse modified discrete cosine transform (hereinafter referred to as “IMDCT”) on the feature data X (f), and time-series data in one pitch section (hereinafter referred to as “equalized audio signal”). Generate x _eq (t). The pitch frequency detecting means 55 detects the pitch frequency of the equalized audio signal x _eq (t) and outputs it as the equalized pitch frequency signal V _eq .

On the other hand, the pitch information decoder 54 restores the reference frequency signal AV _pitch and the residual frequency signal ΔV _pitch by decoding the encoded pitch data. The difference unit 56 outputs a difference obtained by subtracting the equalized pitch frequency signal V _eq from the reference frequency signal AV _pitch as a reference frequency change signal ΔAV _pitch . The adder 57 adds the residual frequency signal ΔV _pitch and the reference frequency change signal ΔAV _pitch and outputs this as a modified residual frequency signal ΔV _pitch ″.

The frequency shifter 58 has the same configuration as the frequency shifter 23 shown in FIG. 7 or FIG. In this case, the equalized audio signal x _eq (t) is input to the input terminal In, and the modified residual frequency signal ΔV _pitch ″ is input to the VCO 44. The VCO 44 outputs the modulated carrier signal C ₁ output from the transmitter 41. A signal (hereinafter referred to as a “demodulated carrier signal”) obtained by frequency-modulating a signal having the same carrier frequency with a modified residual frequency signal ΔV _pitch ”input from the adder 57 is output. The frequency of the carrier signal is a frequency obtained by adding a residual frequency to the carrier frequency.

Thus, the fluctuation component is added to the pitch period of each pitch section of the equalized audio signal x _eq (t) in the frequency shifter 58, and the audio signal x _res (t) is restored.

  The output switching unit 59 switches the output destination of the feature data X (f) generated by the inverse quantizer 52 to the synthesizer 53 or the partial speech search unit 6 according to the switching signal input from the partial speech search unit 6. Specifically, when performing a partial speech search operation, the output destination of the feature data X (f) is switched to the partial speech search means 6. On the other hand, when the search target audio data is output to the outside, the output destination of the feature data X (f) is switched to the synthesizer 53.

  FIG. 10 is a block diagram showing the configuration of the partial voice search means 6 of FIG. The partial voice search means 6 includes an operation switching means 61, a partial voice selection means 62, section division means 63 and 64, a feature amount scale calculation means 65, a phoneme string scale calculation means 66, a total scale calculation means 67, and a matching position determination means. 68.

  The operation switching means 61 outputs a switching signal for switching the operation of the voice search device 1 to the search target voice data input / output operation for the voice storage means 3 or the partial voice search operation by the partial voice search means 6.

  The partial voice selecting unit 62 receives a data selection signal for selecting partial voice data from the search target feature data (more precisely, encoded search target feature data) stored in the voice storage unit 3. Output. This data selection signal is input to the data reading means 4. The data reading unit 4 selects and reads the search target feature data stored in the voice storage unit 3 in accordance with the data selection signal.

  The section division unit 63 uses the feature data (subband waveform) of the query speech input from the analyzer 19 of the speech coder 2 and information on the time interval of the phoneme label data of the query speech input from the phoneme labeling processing unit 17. To divide each phoneme segment. Then, the feature data is averaged for each phoneme section, and is output to the feature quantity scale calculation means 65 as time-series data of average values.

  The section dividing unit 64 uses the characteristic data (subband waveform) of the search target speech input from the inverse quantizer 52 of the speech decoder 5 as the time of the phoneme label data of the search target speech input from the data reading unit 4. Divide into phoneme sections according to the section information. Then, the feature data is averaged for each phoneme section, and is output to the feature quantity scale calculation means 65 as time-series data of average values.

The feature quantity scale calculating unit 65 calculates a distance scale D ₁ (X _q , X _o ) between the feature data input from the section dividing units 63 and 64. Here, the distance measure is expressed as a linear sum of correlation coefficients of the subband waveforms constituting the feature data.
That is, the feature data of the query speech is X _q (f), and the feature data of the search target speech is X _o (f), which are expressed by equations (5) and (6), respectively.

The correlation coefficient of each subband element of the feature data X _q (f), X _o (f) is expressed by equation (7). Here, t _j represents the j-th phoneme section. X _{q, fi} (t _j ) is the time average value of feature data X _{q, fi} (t) in the j-th phoneme section, and X _{o, fi} (t _j ) is feature data in the j-th phoneme section. X _{o, fi} (t) is a time average value.

In the first embodiment, a distance measure D ₁ (X _q , X _o ) between feature data is defined by Expression (10).

ここで、ｗ_ｉは重み係数である。重み係数ｗ_ｉは、適宜設定される。

Here, w _i is a weighting factor. The weighting factor w _i is set as appropriate.

The phoneme string scale calculation means 66 receives the phoneme label data of the query speech from the phoneme labeling processing means 17 of the speech encoder 2 and the phoneme label data of the search target speech from the data reading means 4. Phoneme sequence measure calculating unit 66 calculates using these phonemes labels predetermined distance between phonemes measure table the distance measure D ₂ data. Here, the interphoneme distance scale table represents a distance scale between two phonemes as a table for all combinations of two phonemes.

The total scale calculation means 67 is a distance scale D ₁ (X _q , X _o ) between feature data calculated by the feature quantity scale calculation means 65 and a distance scale D _{2 of} phoneme label data calculated by the phoneme string scale calculation means 66. The total distance measure D is calculated by taking the linear sum of That is, the total distance measure D is expressed by the equation (11).

ここで、Ｗ_１，Ｗ_２は重み係数であり、適宜決められる。

Here, W ₁ and W ₂ are weighting factors, which are determined as appropriate.

The coincidence position determination means 68 determines whether or not the distance measure D is equal to or smaller than a predetermined threshold value D _th , and outputs a data selection signal for selecting the partial data when D ≦ D _th .

  The operation of the voice search device 1 of the present embodiment configured as described above will be described below.

[1] Storage Operation of Search Target Voice Data First, the operation when storing the search target voice data in the voice storage means 3 will be described. In this case, the operation switching unit 61 of the partial voice search unit 6 outputs a level (for example, H level) representing the input / output operation of the search target voice data as a switching signal. Thereby, the output switching unit 12 a of the speech encoder 2 outputs the feature data X (f) generated by the analyzer 19 to the quantizer 13. The output switching unit 12 b of the speech coder 2 outputs the phoneme label data generated by the phoneme labeling processing unit 17 to the speech storage unit 3. Further, the output switching means 59 of the speech decoder 5 outputs the feature data X (f) generated by the inverse quantizer 52 to the synthesizer 53.

First, when input speech data x _in (t) is input to the speech coder 2 as search target speech data, the input pitch detection means 21 of the pitch period equalizing means 10 receives the input speech data x _in (t). It is discriminated whether it is voiced sound or unvoiced sound, and the noise flag signal V _noise is output to the output terminal OUT_4, the pitch frequency is detected from the input sound data x _in (t), and the basic frequency signal V _pitch is output to the pitch averaging means 22. To do. The pitch averaging means 22 averages the basic frequency signal V _pitch (in this case, the LPF is used so that it becomes a weighted average), and this is output as the reference frequency signal AV _pitch . The reference frequency signal AV _pitch is output from the output terminal OUT_3 and also input to the residual calculation means 25.

On the other hand, the frequency shifter 23 shifts the frequency of the input audio data x _in (t), and outputs it to the output terminal Out_1 as pitch equalized audio data x _out (t). In the initial state, the residual frequency signal ΔV _pitch is 0 (reset state), and the frequency shifter 23 uses the input audio data x _in (t) as it is as pitch equalized audio data x _out (t) to the output terminal Out_1. Is output.

Next, the output pitch detection means 24 detects the pitch frequency f ₀ ′ of the output audio data output from the frequency shifter 23. The detected pitch frequency f ₀ ′ is input to the residual calculation means 25 as the pitch frequency signal V _pitch ′.

The residual calculation means 25 generates a residual frequency signal ΔV _pitch by subtracting the reference frequency signal AV _pitch from the pitch frequency signal V _pitch ′. The residual frequency signal ΔV _pitch is output to the output terminal Out_2 and also input to the frequency shifter 23 via the PID controller 26.

The frequency shifter 23 sets a frequency shift amount in proportion to the residual frequency signal ΔV _pitch input via the PID controller 26. In this case, if the residual frequency signal ΔV _pitch is a positive value, the shift amount is set so as to decrease the frequency by an amount proportional to the residual frequency signal ΔV _pitch . If the residual frequency signal ΔV _pitch is a negative value, the shift amount is set so as to increase the frequency by an amount proportional to the residual frequency signal ΔV _pitch .

Such feedback control, the pitch period of the input speech data x _in (t) is always maintained at the reference period 1 / f _s, the pitch period of the pitch equalization speech data x _out (t) is equalized.

Thus, in the pitch period equalizing means 10, the information included _in the input voice data x _in (t) is
(A) Information indicating voiced or unvoiced sound;
(B) Information representing a speech waveform in one pitch section;
(C) Reference pitch frequency information;
(D) residual frequency information indicating the amount of deviation of the pitch frequency of each pitch section from the reference pitch frequency;
Separated. The information of (a) to (d) includes a noise flag signal V _noise , a pitch whose pitch period is equalized to a reference period 1 / f _s (a reciprocal of a weighted average of past pitch frequencies of input voice data), etc. Audio data x _out (t), reference frequency signal AV _pitch , and residual frequency signal ΔV _pitch are output. The noise flag signal V _noise is output from the output terminal Out_4, the pitch equalized audio data x _out (t) is output from the output terminal Out_1, the reference frequency signal AV _pitch is output from the output terminal Out_3, and the residual frequency signal ΔV _pitch. Is output from the output terminal Out_2.

The pitch equalized speech data x _out (t) is a speech signal from which jitter components and change components of the pitch frequency that change depending on gender differences, individual differences, phonemes, emotions, and conversation contents are removed, It is a mechanical audio signal. Therefore, since the pitch equalized voice data x _out (t) of the same voiced sound can obtain almost the same waveform irrespective of gender differences, individual differences, phonemes, emotions or conversation contents, the pitch equalized voice data x _out (t) ) Can be accurately matched for voiced sounds.

Further, since the pitch period of voiced sound equalized voice data x _out (t) is equalized to the reference period 1 / f _s , by performing subband coding in a fixed number of pitch sections, the pitch etc. Frequency spectrum X _out (f) of the digitized audio data x _out (t) is aggregated into subband components of harmonic components of the reference frequency. Since speech has a large waveform correlation between pitches, the temporal change in the spectral intensity of each subband component is gradual. Therefore, by encoding each subband component and omitting other noise components, highly efficient encoding is possible. Further, since the reference frequency signal AV _pitch and the residual frequency signal ΔV _pitch change only in a narrow range within the same phoneme due to the nature of speech, highly efficient encoding is possible. Therefore, the voiced sound component of the input voice data x _in (t) can be encoded with high efficiency as a whole.

Next, the resampler 18 calculates a resampling period by dividing the reference frequency signal AV _pitch by a constant resampling number n in each pitch section. Then, the pitch equalized voice data x _out (t) is resampled at the resampling period, and is output as equal sample number voice data x _eq (t). As a result, the number of samples in one pitch section of the pitch equalized audio data x _out (t) is set to a constant value.

Next, the analyzer 19 divides the equal sample number voice data x _eq (t) into subframes having a fixed number of pitch sections. Then, the frequency spectrum signal X (f) is generated by performing the modified discrete cosine transform for each subframe.

Here, the length of one subframe is an integral multiple of one pitch period. In this embodiment, the length of the subframe is 1 pitch period (sampling number n). Therefore, n frequency spectrum signals {X (f ₁ ), X (f ₂ ),..., X (f _n )} are output. The frequency f ₁ is the first harmonic of the reference frequency, the frequency f ₂ is the second harmonic of the reference frequency, and the frequency f _n is the nth harmonic of the reference frequency.

  Thus, by performing subband coding by dividing each subframe into subframes that are integral multiples of one pitch period and orthogonally transforming each subframe, the frequency spectrum signal of the speech waveform data is a harmonic spectrum of the reference frequency. To be aggregated. And, due to the nature of speech, the waveforms of successive pitch sections within the same phoneme are similar, so the spectrum of the harmonic component of the reference frequency is similar between adjacent subframes, and therefore the coding efficiency is increased. .

Next, the quantizer 13 quantizes the frequency spectrum signal X (f). Here, the quantizer 13 refers to the noise flag signal V _noise, switching the quantization curve in the case the noise flag signal V _noise is 0 when the 1 (unvoiced) of (voiced).

When the noise flag signal V _noise is 0 (voiced sound), the quantization curve is a quantization curve in which the number of quantization bits decreases as the frequency increases, as shown in FIG. . This corresponds to the fact that the frequency characteristic of the voiced sound has a characteristic that decreases in the low frequency region and increases in the high frequency region as shown in FIG.

On the other hand, when the noise flag signal V _noise is 1 (unvoiced sound), the quantization curve is a quantization curve in which the number of quantization bits increases as the frequency increases, as shown in FIG. The This corresponds to the fact that the frequency characteristic of the unvoiced sound has a characteristic that increases as the frequency becomes higher as shown in FIG.

  By switching the quantization curve, an optimal quantization curve is selected corresponding to voiced sound or unvoiced sound.

  As a supplement, the number of quantization bits will be described. The data format of quantization by the quantizer 13 is expressed by a real part (FL) below the decimal point and an exponent part (EXP) representing the power of 2 as shown in FIGS. However, when representing a number other than 0, the exponent (EXP) is adjusted so that the first bit of the real part (FL) is always 1.

  For example, when the real part (FL) is 4 bits and the exponent part (EXP) is 2 bits, the quantization is performed with 4 bits and the quantization is performed with 2 bits (FIG. 11 ( c) and (d)).

(1) When quantizing with 4 bits (Example 1) X (f) = 8 = [1000] ₂ (where [] ₂ represents a binary number notation)
FL = [1000] ₂ , EXP = [100] ₂
(Example 2) X (f) = 7 = [0100] ₂ is
FL = [1110] ₂ , EXP = [011] ₂
(Example 3) X (f) = 3 = [1000] ₂ is
FL = [1100] ₂ , EXP = [010] ₂

(2) When quantizing with 2 bits (Example 1) X (f) = 8 = [1000] ₂
FL = [1000] ₂ , EXP = [100] ₂
(Example 2) X (f) = 7 = [0100] ₂ is
FL = [1100] ₂ , EXP = [011] ₂
(Example 3) X (f) = 3 = [1000] ₂ is
FL = [1100] ₂ , EXP = [010] ₂

  That is, when quantizing with n bits, n bits are left from the beginning of the real part (FL), and the remaining bits are set to 0 (see FIG. 11D).

  Next, the pitch equalization waveform encoder 14 encodes the quantized frequency spectrum signal X (f) output from the quantizer 13 by an entropy encoding method, and outputs encoded feature data. Further, the pitch equalization waveform encoder 14 outputs the code amount (number of bits) of the encoded feature data to the differential bit calculator 15. The difference bit calculator 15 subtracts a predetermined number of target bits from the code amount of the encoded feature data and outputs the number of difference bits. The quantizer 13 moves the quantization curve for voiced sound up and down in parallel translation according to the number of difference bits.

For example, if the quantization curve for {f ₁ , f ₂ , f ₃ , f ₄ , f ₅ , f ₆ } is { ₆ , ₅ , ₄ , ₃ , ₂ , ₁ }, 2 is the difference bit number. If input, the quantizer 13 translates the quantization curve downward by two. As a result, the quantization curve is {4, 3, 2, 1, 0, 0}. If −2 is input as the number of difference bits, the quantizer 13 translates the quantization curve upward by two. As a result, the quantization curve becomes {8, 7, 6, 5, 4, 3}.

  In this way, by changing the quantization curve of the voiced sound up and down, the code amount of the encoded feature data of each subframe is adjusted to about the target number of bits.

On the other hand, in parallel with this, the pitch information encoder 16 encodes the reference frequency signal AV _pitch and the residual frequency signal ΔV _pitch .

On the other hand, the phoneme labeling processing means 17 divides the input speech data x _in (t) into phoneme sections and performs phoneme labeling on each phoneme section. Many techniques are known in the field of speech recognition regarding the method of dividing a phoneme section and the method of labeling a phoneme, and these known methods can be used here. The phoneme labeling processing means 17 outputs, as phoneme label data, phoneme labels obtained by phoneme labeling and information on phoneme intervals representing time intervals for each phoneme label.

  The encoded feature data, the encoded pitch data, and the phoneme label data generated as described above are output to the voice storage unit 3 and stored.

[2] Operation of Speech Decoder When the data reading means 4 reads the encoded feature data and the encoded pitch data from the speech storage means 3, these data are input to the speech decoder 5.

The pitch equalization waveform decoder 51 of the speech decoder 5 decodes the encoded feature data, and restores the frequency spectrum signal of each subband after quantization (hereinafter referred to as “quantized frequency spectrum signal”). The inverse quantizer 52 inversely quantizes the quantized frequency spectrum signal, and the frequency spectrum signals X (f) = {X (f ₁ ), X (f ₂ ),..., X (f _n )} is restored.

The synthesizer 53 performs inverse modified discrete cosine transform (hereinafter referred to as “IMDCT”) on the frequency spectrum signal X (f), and is referred to as time-series data (hereinafter referred to as “equalized audio signal”) in one pitch interval. ) X _eq (t) is generated. The pitch frequency detecting means 55 detects the pitch frequency of the equalized audio signal x _eq (t) and outputs it as the equalized pitch frequency signal V _eq .

On the other hand, the pitch information decoder 54 restores the reference frequency signal AV _pitch and the residual frequency signal ΔV _pitch by decoding the encoded pitch data. The difference unit 56 outputs a difference obtained by subtracting the equalized pitch frequency signal V _eq from the reference frequency signal AV _pitch as a reference frequency change signal ΔAV _pitch . The adder 57 adds the residual frequency signal ΔV _pitch and the reference frequency change signal ΔAV _pitch and outputs this as a modified residual frequency signal ΔV _pitch ″.

The frequency shifter 58 has the same configuration as the frequency shifter 23 shown in FIG. 7 or FIG. In this case, the equalized audio signal x _eq (t) is input to the input terminal In, and the modified residual frequency signal ΔV _pitch ″ is input to the VCO 44. The VCO 44 outputs the modulated carrier signal C ₁ output from the transmitter 41. A signal (hereinafter referred to as a “demodulated carrier signal”) obtained by frequency-modulating a signal having the same carrier frequency with a modified residual frequency signal ΔV _pitch ”input from the adder 57 is output. The frequency of the carrier signal is a frequency obtained by adding a residual frequency to the carrier frequency.

Thus, the fluctuation component is added to the pitch period of each pitch section of the equalized audio signal x _eq (t) in the frequency shifter 58, and the audio signal x _res (t) is restored.

[3] Partial Voice Data Search Operation Using Query Voice Data Next, partial voice data search operation using query voice data will be described. In this case, the operation switching unit 61 of the partial voice search unit 6 outputs a level (for example, L level) representing the partial voice search operation as a switching signal. As a result, the output switching unit 12 a of the speech encoder 2 outputs the feature data X (f) generated by the analyzer 19 to the partial speech search unit 6. The output switching unit 12 b of the speech encoder 2 outputs the phoneme label data generated by the phoneme labeling processing unit 17 to the partial speech search unit 6. The output switching means 59 of the speech decoder 5 outputs the feature data X (f) generated by the inverse quantizer 52 to the partial speech search means 6.

First, the query speech data is input to the speech coder 2 as input speech data x _in (t).

As described above, the pitch period equalizing means 1 equalizes the pitch period of the voiced sound of the input sound data x _in (t) and outputs the equalized sound data x _out (t) from the output terminal Out_1. In addition, as described above, the feature data generation unit 19 converts the pitch equalized speech data x _out (t) into feature data X (f) that includes a time series of a short-time spectrum. The feature data X (f) is output to the partial speech search means 6 via the output switching means 12a.

On the other hand, as described above, the phoneme labeling processing means 17 divides the input speech data x _in (t) into phoneme sections and performs phoneme labeling on each phoneme section. And the information of a phoneme label and a phoneme area is output as phoneme label data.

  Next, the partial voice selection means 62 of the partial voice search means 6 is data for sequentially reading the encoded feature data, the encoded pitch data, and the phoneme label data stored in the voice storage means 3 from the top of the data. Outputs a selection signal. At this time, the length of the partial data to be read out is the same phoneme length as that of the query speech data. The data reading unit 4 reads partial data from the voice storage unit 3 in accordance with the data selection signal.

  The phoneme label data read by the data reading means 4 is input to the partial voice search means 6.

  On the other hand, the encoded feature data and the partial data of the encoded pitch data read by the data reading unit 4 are input to the speech decoder 5. In the speech decoder 5, as described above, the encoded feature data is decoded by the pitch equalization waveform decoder 51, and the inverse quantizer 52 performs inverse quantization, thereby generating feature data and partial speech search Output to means 6.

  Hereinafter, the partial data of the search target feature data input from the speech decoder 5 to the partial speech search means 6 is referred to as “selected feature data”.

  In the partial speech search means 6, when query speech feature data (hereinafter referred to as “query feature data”) and phoneme label data are input from the speech encoder 2, the segment dividing means 63 converts the query feature data into phoneme. Averaging is performed for each section and converted to time-series data of average values. In this case, the query feature data may be divided into time intervals based on the information on the phoneme intervals included in the phoneme label data, and an average value may be taken for each time interval. The time-series data of the average value is input to the feature amount scale calculation means 65.

  When the selected feature data and the phoneme label data are input from the speech decoder 5 and the data reading unit 4, the section dividing unit 64 averages the selected feature data for each phoneme section and converts it into time-series data of an average value. To do. The time-series data of the average value is input to the feature amount scale calculation means 65.

The feature quantity scale calculation means 65 calculates a distance scale D ₁ (X _q , X _o ) between the time series data of the average values input from the section dividing means 63 and the section dividing means 64 according to the equation (10).

On the other hand, the phoneme string measure calculating means 66, phoneme distance measure D ₂ between the search target speech phoneme label data inputted from the phoneme label data and the data reading means of the query speech input from the speech coder 2 Calculate using the distance scale table.

The total scale calculation means 67 is a distance scale D ₁ (X _q , X _o ) between feature data calculated by the feature quantity scale calculation means 65 and a distance scale D _{2 of} phoneme label data calculated by the phoneme string scale calculation means 66. The total distance measure D is calculated by the equation (11).

The coincidence position determination means 68 determines whether or not the distance measure D is equal to or smaller than a predetermined threshold value D _th , and outputs a data selection signal for selecting the partial data when D ≦ D _th . Then, the operation switching means 61 outputs a level (for example, L level) representing the partial voice search operation as a switching signal.

  Thereby, the partial data of the searched search target data is output as output audio data.

  This embodiment can also be applied to search for information in a multimedia database in which audio information and video are recorded together.

The present invention can be used in a voice database, a multimedia database including voice information, and the like.

Claims (19)

A voice search device that searches partial voice data that matches or is similar to query voice data from search target voice data,
Pitch equalization by equalizing the pitch period of voiced sound of the search target voice data Pitch equalization by equalizing the pitch period of voiced sound of the query voice data in the voice feature amount space from the search target voice data A speech search apparatus comprising: partial speech search means for searching partial speech data whose distance measure (or similarity measure) with respect to query speech data is less than or equal to a predetermined threshold (or greater than or equal to a predetermined threshold). Pitch period equalizing means for generating the pitch equalized query voice data by equalizing the pitch period of the voiced sound of the query voice data;
Feature data generating means for generating data obtained by converting the pitch equalized query voice data into time-series data of feature quantities (hereinafter referred to as “query feature data”);
With
The partial speech search means has a feature measure having a distance measure (or similarity measure) between the partial feature data included in the pitch equalization search target speech data and the query feature data equal to or less than a predetermined threshold ( 2. The speech search apparatus according to claim 1, wherein a search is made for a search that is equal to or greater than a predetermined threshold. The partial voice search means
Of the search target feature data obtained by converting the pitch equalization search target speech data into time-series data of feature quantities, partial data for the same phoneme length as the query speech data (hereinafter referred to as “selected feature data”) is used. Partial voice selection means for sequentially selecting while moving the selection position;
Feature quantity scale calculation means for calculating a distance scale (or similarity scale) between each of the selected feature data and the query feature data;
When the distance measure (or similarity measure) is equal to or smaller than a predetermined threshold (or equal to or greater than a predetermined threshold), a matching position determination unit that outputs a position in search target audio data corresponding to the selected feature data;
The voice search device according to claim 1, further comprising: 4. A voice search apparatus according to claim 3, further comprising voice storage means for storing the search target characteristic data. Second pitch period equalizing means for generating the pitch equalization search target voice data by equalizing the pitch period of the voiced sound of the search target voice data;
Second feature data generating means for generating the search target feature data by converting the pitch equalization search target voice data into time-series data of feature quantities;
The voice search device according to claim 3 or 4, further comprising: The pitch period equalizing means (or the second pitch period equalizing means)
Pitch detection means for detecting a pitch frequency of the query voice data (or the search target voice data);
Residual calculating means for calculating a difference between the pitch frequency and a predetermined reference frequency;
6. A voice search apparatus according to claim 2, further comprising a frequency shifter for equalizing a pitch frequency of the query voice data (or the search target voice data) so that the difference is minimized. . The search target feature data and the query feature data are respectively time series of subband data obtained by orthogonal transformation of the pitch equalization search target speech data and the pitch equalization query speech data. The voice search device according to any one of claims 1 to 6. A first section dividing means for averaging the query feature data for each phoneme section and converting it into time-series data of an average value;
A second section dividing means for averaging the search target feature data for each phoneme section and converting it into time-series data of an average value;
With
6. The feature quantity scale calculating means calculates a distance scale (or similarity scale) between time series data of average values generated by the first and second section dividing means. The voice search device described. Phoneme labeling processing means for generating a query phoneme sequence (or search target phoneme sequence) by performing phoneme labeling on the query speech data (or the search target speech data);
Phoneme string scale calculation means for determining a distance scale (or similarity scale) between the search target phoneme string corresponding to the selected feature data and the query phoneme string;
A linear sum (hereinafter referred to as “total distance measure”) of the distance measure (or similarity measure) of the feature value output by the feature value measure calculating means and the distance measure (or similarity measure) of the phoneme string output means by the phoneme string measure calculating means. (Or an overall similarity scale) ”)
With
The coincidence position determination means outputs a position in search target audio data corresponding to the selected feature data when the total distance scale (or total similarity scale) is equal to or smaller than a predetermined threshold (or higher than a predetermined threshold). The voice search device according to any one of claims 1 to 8. A voice search method for searching partial voice data that matches or is similar to query voice data from search target voice data,
Pitch equalization by equalizing the pitch period of voiced sound of the search target voice data Pitch equalization by equalizing the pitch period of voiced sound of the query voice data in the voice feature amount space from the search target voice data A speech search method comprising: a partial speech search step of searching for partial speech data whose distance measure (or similarity measure) for query speech data is equal to or less than a predetermined threshold (or greater than a predetermined threshold). A pitch period equalizing step for generating the pitch equalized query voice data by equalizing the pitch period of the voiced sound of the query voice data;
A feature data generation step of generating data (hereinafter referred to as “query feature data”) obtained by converting the pitch equalization query voice data into time-series data of feature amounts;
With
In the partial speech search step, the feature amount of the partial speech data included in the pitch equalization search target speech data has a distance measure (or similarity measure) between the query feature data and a predetermined threshold value or less. The speech search method according to claim 9, wherein a search is made for a search that is (or more than a predetermined threshold). In the partial voice search step,
Of the search target feature data obtained by converting the pitch equalization search target speech data into time-series data of feature quantities, partial data for the same phoneme length as the query speech data (hereinafter referred to as “selected feature data”) is used. A partial voice selection step for sequentially selecting while moving the selection position;
A feature amount scale calculating step for calculating a distance measure (or similarity measure) between each of the selected feature data and the query feature data;
When the distance measure (or similarity measure) is equal to or less than a predetermined threshold (or greater than or equal to a predetermined threshold), a matching position determination step of outputting a position in search target audio data corresponding to the selected feature data;
The voice search method according to claim 9 or 10, characterized by comprising: The voice search method according to claim 11, further comprising a voice storage step of storing the search target characteristic data. A second pitch period equalization step of generating the pitch equalization search target voice data by equalizing the pitch period of the voiced sound of the search target voice data;
A second feature data generation step of generating the search target feature data by converting the pitch equalization search target voice data into time-series data of feature amounts;
The voice search method according to claim 11 or 12, characterized by comprising: In the pitch period equalization step (or second pitch period equalization step),
A pitch detection step of detecting a pitch frequency of the query voice data (or the search target voice data);
A residual calculation step of calculating a difference between the pitch frequency and a predetermined reference frequency;
The voice search method according to claim 11, further comprising a frequency shift step of equalizing a pitch frequency of the query voice data (or the search target voice data) so that the difference is minimized. . The search target feature data and the query feature data are respectively time series of subband data obtained by orthogonal transformation of the pitch equalization search target speech data and the pitch equalization query speech data. The voice search method according to any one of claims 10 to 15. A first section division step of averaging the query feature data for each phoneme section and converting the average to time-series data of an average value;
A second section dividing step of averaging the search target feature data for each phoneme section and converting the averaged time-series data into average values;
Have
The distance scale (or similarity scale) between the time series data of the average values generated in the first and second section division steps is calculated in the feature quantity scale calculation step. Or the voice search method of 14. A phoneme labeling step of generating a query phoneme sequence (or search target phoneme sequence) by performing phoneme labeling on the query speech data (or the search target speech data);
A phoneme sequence scale calculation step for determining a distance measure (or similarity measure) between the search target phoneme sequence corresponding to the selected feature data and the query phoneme sequence;
A linear sum (hereinafter, “total”) of the distance measure (or similarity measure) of the feature amount output in the feature amount scale operation step and the distance measure (or similarity measure) of the phoneme sequence output in the phoneme sequence scale operation step. A total scale calculation step for calculating a distance scale (or a total similarity scale).
With
In the matching position determination step, when the total distance measure (or total similarity measure) is equal to or smaller than a predetermined threshold (or equal to or larger than a predetermined threshold), a position in the search target audio data corresponding to the selected feature data is output. The voice search method according to claim 10, wherein the voice search method is a voice search method. A program that causes a computer to function as the voice search device according to any one of claims 1 to 8 by being read and executed by the computer.

Images