JP2709385B2 - Speech database construction method with multi-layer labels - Google Patents

Description

Description: BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for constructing a speech database having a multi-layer label, and in particular, digitizes a speech signal waveform and classifies the speech waveform for each phoneme based on the characteristics of the signal. Do,
The present invention relates to a method for constructing a speech database having a multi-layer label in which a label is assigned to each phoneme. [Problems to be Solved by Conventional Techniques and Inventions] In order to improve various techniques for performing speech processing such as a speech recognition algorithm, a speech synthesis algorithm, and a speaker recognition / adaptive algorithm, it is necessary to use various environments. It is necessary to collect and organize changes in phonetic features. To this end, it is essential to maintain a phonetic-labeled speech database. Conventional speech databases are mainly used for evaluating the performance of speech processing devices such as speech recognition devices and for speech research and development.
Broadly classified into types. The former is, for example, a voice database consisting of words of city names and others, but it is merely an analog voice and is not labeled. On the other hand, the latter research speech database has a drawback that even if it is labeled, since it is a label of only a phoneme or a symbol of a unit equivalent thereto, a speech event cannot be efficiently selected. Therefore, a main object of the present invention is to provide various audio feature labels in a hierarchical manner so that the selection and extraction of audio data can be efficiently performed based on the label information and can be used for various research purposes for audio. Such a voice database configuration method is provided. [Means for Solving the Problems] The present invention relates to a speech database having a configuration in which a speech signal waveform is digitized, the speech waveform is divided for each phoneme based on the characteristics of the signal, and each phoneme label is assigned. An audio data file for storing digitized audio signal waveforms, and a plurality of layers of label files provided corresponding to each audio storage address of the audio data file, and a portion to which a phoneme label is assigned is regarded as one layer. The description of various voice features reflecting actual voice phenomena is described in a label file over a plurality of types, and a label describing each layer is provided within each phoneme or between phonemes, and a digitized voice waveform and The correspondence with the multilayer label describing the physical characteristics is performed. [Operation] In the method for constructing a speech database having a multi-layer label according to the present invention, various portions that reflect actual speech phenomena are formed by using a portion provided with a phoneme label as one layer in a digitized speech signal waveform. Describes speech features in a label file for multiple types and assigns descriptive labels within or between phonemes, and associates digitized speech waveforms with multi-layer labels that describe their physical features. Thus, the present invention can be used for various research purposes with respect to the efficiency of selection and extraction of voice data and voice based on label information. [Embodiment of the Invention] FIG. 1 is a diagram showing an example of a label in each layer provided for each phoneme, within a phoneme and between phonemes of an audio signal according to the present invention, and FIG. 2 shows a label display method in each layer. FIG. 3 is a diagram showing the notation method and its symbol in the event layer, FIG. 4 is a diagram showing the notation method and its symbol in the allophone layer, and FIG. 5 is a diagram showing the notation inside the computer. FIG. 4 is a diagram illustrating a data format of a label file. Hereinafter, the present invention will be described with reference to FIG. 1 to FIG. FIG. 1 shows the labeling performed in accordance with the waveform of the audio signal, the rate of change of the spectrum, and the power. As shown in FIG. It is composed of an event layer as a layer, an allophone layer as a third layer, a fusion layer as a fourth layer, a vowel central layer as a fifth layer, and a comment layer as a sixth layer. The phonetic symbol layer performs segmentation for each phoneme based on the spectrum change of the uttered voice as a clue, and divides the Hepburn-type Roman alphabet notation into vowel parts and consonant parts and describes them in the corresponding voice sections. For example, when the voice of “atoshimatsu” is uttered, the vowel part and the consonant part of each phoneme are divided, and “a”, “t”, “o”, “sh”, “i”,
Describe “m”, “a”, “tsu”. In this way, by associating the vowel part and the consonant part of the uttered voice with the voice section, the expression of the language environment can be easily realized. However, if a phoneme boundary cannot be determined due to allophone or fusion, a symbol corresponding to the second or lower layer is assigned. The event layer is obtained by dividing each phoneme section divided by the phonetic symbol layer into a plurality of sections in accordance with a change in spectrum, and adding a label to reflect the actual utterance well. It is represented by the notation as shown below. In other words, "<" indicates a transition to a vowel, and indicates a transition section (a section in which the formant structure has not yet been arranged even though energy exists in the low frequency range) accompanied by a vowel at the beginning (including a semi-vowel). It represents. “>” Indicates a vowel, which is a transitional section accompanied by a ending vowel and a relatively long vowel before silence (including a sound repellent). Section). “ ^* >” Indicates a transition from a vowel to a voiced consonant, and is a transitional section in which a transition from a vowel part (including a vowel sound) to a voiced consonant part is made. “Tr” is a section in which the spectral pattern is disturbed for some reason. “Cl, ^* cl” is a closing (closure) section associated with a bursting (fricative) sound and a pause section accompanying a prompting sound, and * indicates a voiced case. “P,
“t, k, b, d, g” indicates a section other than the closure in the plosive, “mm” is a nasal consonant section, and “pau” is a pause section at a word boundary. , sh, z, dj, f "is a fricative interval," w, y "is a semi-vowel interval," r "is a flowing sound interval, and" a, i, u, e, o "is a vowel interval. And “j” is a relentless sound section, “N” is a sound-repelling section, and “ts, ch” indicates a section other than the closure in the affricate. More specifically, with reference to the above, the section until the first vowel "a" is generated is a transition section indicating a rise to a vowel,
“<” Is assigned. The section following the vowel "a" is a transitional section accompanied by the ending of the vowel "a", so ">" is added. In the next section, "cl" is given as a closure accompanying the plosive, and in the next section, "t" is given as a section other than the closed section in the plosive. The next section is a vowel “o”, and then “sh” is given as a fricative section. Further, the next section is a vowel “i”, followed by “>” as a transition section following the nasal consonant section “mm”, the vowel “a”, and the vowel “a”. "Cl" is given as a closed section, and "ts" is given as a section other than the closed section in the affricate. Next to the event layer is a tonalization layer, and a segment is provided and a symbol is provided when so-called tonicization different from the Roman alphabet notation occurs. As the abnormal sound, two types of sections, ie, unvoiced sound and fricative sound, are described. If abnormal noise has occurred, the segment is performed from the time of occurrence of abnormal noise to the end time, regardless of the boundary of the phonetic symbol layer. As shown in FIG. 4, the notation symbol is “d
“v” and “fr” as a section in which the vowel is fricated by the influence of the subsequent fricative. In the example shown in FIG. 1, “fr” is interposed between the phonemes “o” and “sh”. Is given and “dv” is given because the last phoneme “u” is unvoiced. The nasalization of vowels or voiced plosives, which is one of the tonalization phenomena, is The fourth layer is a fusion layer, in which continuous phonemes are fused and describe a non-separable continuous part on a spectrum. In the example shown in Fig. 1, the symbol "tsu" is added because the last two phonemes "ts" and "u" cannot be distinguished consecutively. The fifth layer is the vowel central layer, and the phonemes segmented by the phonetic symbol layer retain distinct vowel characteristics. In the example shown in Fig. 1, a pointer indicating the center of each of the vowels "a", "o", "i", and "a" is described. This is a comment layer, in which a comment about a phenomenon that cannot be described in the above-described first to fifth layers is described, and the correspondence between each label layer and the audio waveform is associated as shown in FIG. That is, the label signal is
It is written between the start value indicating the start time and the end value indicating the end time. Then, the link with the actual audio waveform is obtained by extracting the audio data file by designating symbols indicating the generator, the type of word, etc., and by using the start time and the end time, the correspondence between the label and the audio waveform is obtained. Make a connection. FIG. 6 is a flowchart from A / D conversion to input of label data for constructing a voice database having a multi-layer label according to the present invention. Next, a method of constructing a speech database having a multi-layer label according to the present invention will be described. The recording conditions are as follows: In a quiet environment such as a recording studio, utterances are clearly separated by words and once recorded on a magnetic tape by PCM. Then, via a computer workstation, the audio signal PCM-recorded on the magnetic tape is converted to 16-bit A / D by sampling at 20 kHz and stored in a magnetic disk via a computer workstation. Then, the stored voice data is cut out for each word, and a 512-point FFT (Fast Fourier Transform) is performed, and the frame period is 2.5 msec.
, And the result is displayed in a gray scale by a laser printer. The result is represented as a sonagraph as shown in FIG. Looking at the shaded display, the phonemes are classified and labeled, and label data is input from the keyboard. That is, according to the spectrum of the speech waveform shown in FIG. 1, the speech symbol layer, event layer, allophone layer, fusion layer, vowel central layer, and comment layer are respectively labeled according to FIGS. . Then, input the labeled data from the keyboard of the computer terminal,
For each label, as shown in FIG. 5, the section start time and the end time represented by each label are recorded simultaneously, thereby associating with the waveform data. [Effects of the Invention] As described above, according to the present invention, the speech database has not only a mere surface label in Roman alphabet notation of phonemes but also a multi-layer label structure in which the actual phonation phenomenon is described in detail. Can be used for various research purposes of speech. For example, speech recognition can be applied to the development and evaluation of recognition algorithms and error analysis, speech synthesis can be applied to the construction and evaluation of synthesis rules, and perception can be applied to
It can be applied to, for example, associating the sound of a voice with a physical quantity.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a diagram showing an example of a label assigned to each layer of an audio signal according to the present invention. FIG. 2 is a diagram showing a label display method in each layer. FIG. 3 is a diagram showing a notation method and symbols in the event layer. FIG. 4 is a diagram showing a notation method and symbols in the abnormal sound layer. FIG. 5 is a diagram showing a data format of a label file in the computer. FIG. 6 is a flowchart from A / D conversion to input of label data for constructing a voice database having a multi-layer label according to the present invention.

   ────────────────────────────────────────────────── ─── Continuation of front page    (72) Inventor Kazuya Takeda               Sanraya, Sayaka-gun, Kyoto               No. 5 ATR Co., Ltd.               Within the Translation and Telephone Laboratories (72) Inventor Nao Kuwahara               Sanraya, Sayaka-gun, Kyoto               No. 5 ATR Co., Ltd.               Within the Translation and Telephone Laboratories (72) Inventor Shigeru Katagiri               Sanraya, Sayaka-gun, Kyoto               No.5 Viewed by AT R Co., Ltd.               Auditory Institute                (56) References JP-A-61-296396 (JP, A)

Claims (1)

(57) [Claims] An audio database having a configuration in which an audio signal waveform is digitized, the audio waveform is divided for each phoneme based on the characteristics of the signal, and each phoneme label is added, wherein the audio data for storing the digitized audio signal waveform And a plurality of layers of label files provided corresponding to the respective voice storage addresses of the voice data file, and a portion to which a phoneme label is assigned is regarded as one layer, and various voices reflecting actual voice phenomena are included. A plurality of types described in the label file and a label describing each layer within each phoneme or between phonemes is given, a multi-layered label describing the digitized voice waveform and its physical characteristics. A voice database having a multi-layer label. 2. 2. The method according to claim 1, wherein one of the multilayer labels includes a phonetic symbol layer displayed in Roman characters for each phoneme unit. 3. One of the multilayer labels is an event layer in which each section divided by the phonetic symbol layer is divided into a plurality of sections in accordance with changes in speech characteristics and labeled so as to reflect the actual pronunciation well. 3. The method for constructing a speech database having a multi-layer label according to claim 2, including: 4. 3. The method according to claim 2, wherein one of the multi-layer labels includes an allophone layer describing sections of devoice and fricative sound. 5. One of the multilayer labels is a combination of successive phonemes,
3. The method according to claim 2, further comprising a fusion layer describing a non-separable continuous part. 6. 3. The method according to claim 2, wherein one of the multilayer labels includes a vowel central layer describing a pointer indicating the center of a vowel. 7. 3. The multi-layer label according to claim 2, wherein one of the multi-layer labels includes a comment layer that describes a comment that cannot be described in the phonetic symbol layer, the event layer, the allophone layer, the fusion layer, and the vowel central layer. A method for constructing a speech database having a multilayer label according to any one of claims 6 to 13.