JPS63237098A - Voice data base configuration system having multi-layer label - Google Patents

Abstract

(57) [Summary] This bulletin contains application data before electronic filing, so abstract data is not recorded.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a speech database configuration method with multilayer labels, and in particular, to digitizing speech signal waveforms and classifying the speech waveforms into phonemes based on signal characteristics. do the
Speech database construction method with multi-layered labels such as labels given to each phoneme [Problems to be solved by conventional technology and invention] Speech recognition algorithm, speech synthesis algorithm.

In order to improve speech processing technologies such as speaker recognition and adaptive algorithms, it is necessary to collect and organize changes in phonological features under various environments. To this end, it is essential to develop a speech database with phonological labels.

Conventional speech databases are mainly divided into two types: those for performance evaluation of speech processing devices such as speech recognition devices, and those for speech research and development. For the former, for example, a city name
There is an audio database consisting of i and others, but it is just a collection of analog audio and is not labeled. On the other hand, the latter research speech database had the disadvantage that even if it was labeled, it was only labeled with symbols for phonemes or similar units, making it impossible to efficiently select speech events.

Therefore, the main purpose of the present invention is to hierarchically assign various audio feature labels so that the label information can be used to improve the efficiency of selecting and extracting audio data and for various research purposes related to audio. The purpose of the present invention is to provide a method for configuring such a speech database.

[Means for Solving the Problems] This invention provides a method for digitizing a speech signal waveform, dividing the speech waveform into phonemes based on the characteristics of the signal, and constructing a speech f-database in which each phoneme is given a label. The part to which a phoneme label is attached is taken as one layer, and various phonetic features that reflect actual phonetic phenomena are described over multiple types, and a layer corresponding to each feature is provided to create a layer within each phoneme. Alternatively, a label is assigned to each layer between phonemes, and a digitized speech waveform is associated with a multilayer label describing its physical characteristics.

"Operation" In the speech database configuration method with multilayer labels according to the present invention, by assigning various hierarchical speech feature labels to digitized speech waveforms, speech data can be selected based on label information. It can be used for various research purposes such as improving the efficiency of speech extraction and speech.

[J? Example of I-A
FIG. 2 is a diagram showing the label display method in each layer, FIG. 3 is a diagram showing the notation method and its symbols in the event layer, and FIG. 4 is a diagram showing the label display method in the event layer. FIG. 5 is a diagram showing the notation method and its symbols, and FIG. 5 is a diagram showing the data format of the label file inside the computer.

The present invention will be explained below with reference to FIGS. 1 to 5. Figure 1 shows the waveform, spectral change rate, and power of the vocal Ot arch with labels corresponding to the waveform, spectrum change rate, and power.As shown in Figure 2, the labels are for the phonetic symbol layer as the first layer. , the event layer as the second layer, the allophonic layer as the third layer, the fusion layer as the fourth layer, the vowel center layer as the fifth layer, and the comment layer as the sixth layer. Consisting of The phonetic symbol layer performs segmentation for each phoneme using the spectral changes of the uttered sounds as clues, and divides the Hepburn Roman alphabet into vowels and consonants, and describes them in the corresponding speech sections. For example, when you pronounce the sound "atoshimatsu", the vowel part and consonant part of each phoneme are divided and 1laI1. “[
)l, 110+1, 1131111,
II i II.

11. Describe lZ&a'' and ``tsu''. In this way, by associating the vowel and consonant parts of the uttered voice with respective speech intervals, the expression of the linguistic environment can be easily realized. However, If the 8-element boundary cannot be determined due to allophoneization or fusion, the corresponding symbol is given in the second layer and below.

In the event layer, each phoneme interval divided by the phonetic symbol layer is divided into multiple parts according to changes in the spectrum, and labels are given to better reflect the actual utterances, as shown in Figure 3. It is represented by the notation symbol as shown. That is, r* < If indicates transition to a vowel, and i? i
It represents the 'A crossing section (a section in which the formant structure is not yet complete despite the presence of energy in the low range) that accompanies the initial vowel (including semi-vowels). ti〉pr indicates the transition from a vowel, and the transition period accompanying a vowel at the end of a word and a relatively long pre-silent vowel (including a plosive) (despite the presence of energy in the low range, the formant structure collapses). section).

“ゞ〉” indicates the transition from a vowel to a voiced consonant, and is a transitional section in which the vowel part (including the plethora) transitions to the voiced consonant part. 1llrlt is a section where the spectrum pattern is disturbed for some reason.

"c l, "c l" is a closure section J5 accompanying a plosive (affricate) and a pause section accompanying a consonant, and * indicates a voiced case. "p, t ,
k, b.

d, g" indicate sections other than closures within plosives. "nv" is a nasal consonant section, and "pau" is a pause section at a word boundary. "s, h, sh.

Z, dj, f'' are fricative intervals, 'w, y' are semi-vowel intervals, ', II are warming intervals, 'a'
,i.

``u, e, o'' is the basic a interval, tlJ)l is the persistent tone interval, ``N11 is the plucked tone interval,'' ts,
ch” indicates a section other than the closure within the affricate.

To explain the above event layer in more detail with reference to FIG. 1, the interval until the first vowel "a N" is pronounced is a transitional interval indicating the rise to the vowel.
゛〈'' is added.The section following vowel 1ias is vowel 1
">" is added because it is a transitional section accompanying the ending of 1a11. In the next section, 'at' is added as a closure accompanying the plosive, and in the next section, '℃' is added as a section other than the closed section within the plosive.The next section is the vowel ll011 , and then IJhll is given as a fricative interval.Furthermore, the next interval ]n is a vowel 11i11, followed by a nasal consonant interval “”Ilm”,
As a transition interval following the vowel IfaN and the vowel “aII゛〉
” is added, and then 1 is added as a closed section accompanying the affricate.
1 CI 11 is given, and °'ts'' is given as a section other than the open section within the affricate.

Next to the event layer is an allophone layer, in which segments are provided and symbols are provided when so-called allophones, which differ from the Roman alphabet, occur. As for allophones, we describe the two categories of devoicing and fricatives. If allophones have occurred, the segment is defined as the period from the time the allophones occur to the end, regardless of the boundaries of the phonetic symbol layer. As shown in Figure 4, the notation symbols are 11 for devoiced sections.
d■II and 'fr' are given as the interval where the vowel becomes a fricative due to the influence of the following fricative.In the example shown in Figure 1, 'r' is added between the phoneme 110IT and 'sh'.
"r" is added, and "d v " is added because the last phoneme "u" is devoiced.In addition, the LH conversion of a vowel or voiced plosive, which is one of the allophonic phenomena, is Since it is difficult to judge on the spectrum, it is not included in the allophone layer.

41st? il is a fusion layer in which consecutive phonemes are fused to describe a continuous part that cannot be separated on the spectrogram. The segment boundaries are those of the audio signal layer. In the example shown in Figure 1, the last two phonemes "ts",
The symbol "tsu" is given because the svectrodalums of "u" are consecutive and cannot be distinguished.

The fifth layer is the vowel center layer, which describes pointers indicating the centers where phonemes segmented in the phonetic symbol layer retain distinct vowel characteristics. In the example shown in Figure 1, the vowel “t
Pointers indicating the centers of aPl, "10II, Ill Il, and "'a" are written. Sixth
The layer is a comment layer, and the above-mentioned 1st to 5111th layers
Comments are written about phenomena that cannot be described.

Note that the correspondence between each label layer and the audio waveform is as shown in FIG. That is, a label symbol is written with a start value indicating its start time and an end value indicating its end time.

And the link with the actual speech waveform is the speaker.

By specifying a symbol indicating the type of word, etc., an audio data file is retrieved, and based on the start time and end time, GJ is performed based on the correspondence between the label and the audio waveform.

FIG. 6 is a flowchart from A/D conversion to input of label data for constructing a speech database with multilayer labels according to the present invention.

Next, a method of configuring a speech database with multilayer labels according to the present invention will be explained. The recording conditions are as follows: in as quiet an environment as possible, such as in a recording studio, the words are uttered clearly, separated into words, and then recorded as PCM onto magnetic tape. Then, off-line, the audio signal recorded in PCM on the magnetic tape is subjected to 16-bit Δ/D conversion by sampling at 20 kHz via the computer work stage 1, and is stored on the magnetic disk. Then, the stored audio data is cut out every 5n, subjected to 512-point FFT (RS-speed Fourie L transform), and subjected to spectrum analysis with a frame period of 2.5 II sec, and the results are displayed in gradation on a radar printer. Do the following.

The results are expressed as a tuna graph as shown in FIG.

Using this gradation display, phonemes are classified and labeled, and label data is input from the keyboard. That is,
According to the spectrum of the speech waveform shown in Figure 1, the phonetic symbol layer, event layer, allophonic layer, fusion layer, vowel center layer,
Each comment layer is labeled according to FIGS. 1 to 4 described above. Then, by inputting the labeled data from the keyboard of the computer terminal and simultaneously recording the section start time and end time represented by each label, as shown in Figure 5, the waveform Make a correspondence with the data.

[Effects of the Invention] As described above, according to the present invention, the speech database has a multi-layered label structure that describes the actual vocalization phenomenon in detail, rather than just superficial labels based on the Roman alphabet notation of phonemes. , can be used for various research purposes of speech. For example, in voice recognition! Il! It can be applied to the development and evaluation of recognition algorithms and error analysis; in speech synthesis, it can be applied to the construction of synthesis rules; and in perception, it can be applied to the correspondence between the sound of speech and physical quantities.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing an example of labels given to each layer of an audio signal in the present invention. FIG. 2 is a diagram showing a label display method in each layer. FIG. 3 is a diagram showing the notation method and its symbols in the event layer. FIG. 4 is a diagram without showing the notation method and its symbols in the allophone layer. FIG. 5 is a diagram showing the data format of the label file inside the t3 machines. FIG. 6 is a flow diagram from Δ/D conversion to input of label data for constructing a speech database with multilayer labels according to the present invention. Patent Applicant: A.T.R. Automatic Translation Telephone Research Institute A.T.R. Co., Ltd. Figure 3 Note 1: All long vowels are written with one vowel letter. Figure 4 Figure 6

Claims (8)

[Claims] (1) In a speech database that has a structure in which the speech signal waveform is digitized, the speech waveform is divided into phonemes based on the characteristics of the signal, and each phoneme is labeled, the parts with phoneme labels are combined into one We describe the characteristics of various sounds that reflect actual speech phenomena in multiple types, and create layers corresponding to each feature. An audio database configuration method having multilayer labels, which is characterized in that a descriptive label is assigned and a digitized audio waveform is associated with a multilayer label that describes its physical characteristics. (2) The digitized audio signal waveform is stored in an audio data file, and each label is stored in a label file, and each label corresponds to a storage address of each phoneme in the audio data file. The multilayer label according to claim 1, wherein a value is assigned and the audio data file and the label file are linked by making a correspondence between the value and the audio waveform. Speech database configuration method. (3) A speech database configuration system having multilayer labels according to claim 1, wherein one of the multilayer labels includes a phonetic symbol layer in which each phoneme unit is displayed in Roman letters. (4) One of the multi-layer labels is that each section divided by the phonetic symbol layer is divided into a plurality of parts according to changes in phonetic characteristics, and labels are given so as to better reflect the actual pronunciation. A speech database configuration method with multi-layer labels as claimed in claim 3, including an event layer. (5) One of the multilayer labels includes an allophonetic layer that describes a section of devoicing and fricativeization. . (6) A speech database configuration method having a multilayer label according to claim 3, wherein one of the multilayer labels includes a fusion layer in which consecutive phonemes are fused and describe an inseparable continuous part. . (7) One of the multilayer labels includes a vowel center layer that describes a pointer indicating the center of the vowel.
Speech database construction method with multi-layered labels as described in Section 1. (8) One of the multilayer labels includes a comment layer in which comments are written about phenomena that cannot be described in the phonetic symbol layer, the event layer, the allophonic layer, the fusion layer, and the vowel center layer. A speech database configuration method having multilayer labels according to any one of Items 7 to 7.