JPH0372994B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention will be explained in the following order. A. Industrial application field B Summary of disclosure C. Conventional technology D. Problem that the invention aims to solve E. Means to solve the problem F Example F1 speech recognition system environment F1a General explanation (Figures 2 to 7) F1b auditory model and speech recognition system sounds
Its realization in the Hibiki processor (Fig. 8)
~Figure 14) F1c precision matching (Fig. 4, Fig. 15) F1d Basic high-speed matching (Figs. 16 to 18)
figure) F1e Alternative high-speed matching (Figures 19 to 20)
figure) F1f Matching based on first J level
(Figure 20) F1g phoneme tree structure and high-speed matching example
(Figure 21) F1h language model (Figure 2, Figure 22) F2 word phonetic symbol and feeneem mark
Construction of basic form of Coff model F2a Construction of basic phonetic forms (Figure 4, Table 2) F2b Construction of the basic form of Finim (23rd
(Fig. 24) F3 Alphabet of synthesized pheneem phoneme
Generation (Figure 4) F4 Short phonetic Markov model word
Automatic format of basic form (Fig. 1A, 1B
Figures, Figures 21, 23, 25 to 3
0 figure) F5 comparison result F6 Alternative Example F7 table G Effect of invention A. Industrial application field The present invention generates acoustic models of words in a vocabulary.
related to the field of Acoustic models are mainly used for speech recognition.
It is used to understand the sounds of words, but it is also used to explore the phonetic types of words.
can be incorporated into any phonetic application. B Summary of disclosure The present invention is based on the fine-me of a given word.
Phoneme: Because it is often obtained from the front end
The basic form (named after its initials)
Automatic creation of basic forms of phonetic forms with reduced length compared to
Disclose construction. Specifically, the present invention
is () Finim basic of Finim phoneme
Define each word in the vocabulary by its morphology, and
phonemes (each of which has at least one feeney)
Define the alphabet of (corresponding to the m phoneme),
() Finim String according to voice input
In a system that generates a
Each fineme phoneme in the word denoted by phoneme
, and (b) combination.
If the adverse effects of
Combine at least one pair of adjacent synthesized phonemes into one synthesized phoneme
By merging with
The basic form of the word is synthesized into a shortened word of the phoneme.
Allows conversion to basic form. C. Conventional technology Regarding the invention that forms the background of the present invention,
U.S. Patent Application No. 06/665401 (dated October 26, 1984)
Application), No. 06/672974 (filed on November 19, 1984),
and No. 06/697174 (filed on February 1, 1985)
There is. In probabilistic speech recognition methods, the acoustic waveform is initially
Label ie feeney by sound processor
(minimal sound such as that coming from the front end)
is converted to a string of Label (that
each identifying a monophonic symbol) are generally around 200
Selected from different label alphabets.
Ru. Generation of such labels is described in various papers.
and as described in the aforementioned U.S. patent application Ser. No. 06/665,401.
ing. When performing speech recognition using labels, Markov
model machine (i.e. stochastic finite state machine)
) is being discussed. The Markov model is
Usually includes multiple states and transitions between states. Change
In other words, the Markov model (a) confirms the certainty of each transition that occurs.
rate, (b) respectively to generate each label at various transitions.
has a probability assigned to it relative to the probability of
It is normal to do so. Markov model, or
Markov sources can be found in various papers, e.g.
IEEE Bulletin: Pattern analysis and computer information
(PAMI) Vol. 5 No. 2 (March 1983 issue)
Le R. Part et al.'s paper “Recognizing Continuous Speech”
L.R. Bahl et al, “A Maximum Likelihood Method”
Likelihood Approach to Continuous Speecb
“Rocognition”, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol.
PAMI-5, No. 2, March 1983)
There is. Performs a matching process when recognizing speech.
to find out which word (in case of multiple words) in the vocabulary.
(in some cases) is generated by a sound processor.
The highest likelihood of generating a string of labels
Decide what will happen. one such matuchin
The programming procedure is described in the aforementioned U.S. Patent Application No. 06/672,974.
has been done. According to this, acoustic matching consists of (a) word
Each word in the vocabulary is processed by a Markov model phoneme machine.
characterized by the sequence, and (b) the acoustic processor.
generate a label string generated by
that of the sequence of phoneme machines representing each word
This is done by determining the likelihood of each. each
The sequence of phoneme machines that a word represents is a word.
It is called the basic form. When deciding on the basic form of a word, first
Determine the nature of the phoneme machine used to construct morphology.
It is necessary to Said US Patent No. 06/697174
The basic form of work based on Finim is
is shown. In other words, Finim's a
200 finims forming a Rufa bet (collection)
A Markov model is prepared for each of
When spoken (as produced by a sound processor)
) Generate 0, 1 or 2 or more finemes.
Display the probability of a particular finem. given
Finim's Markov model phoneme machine is
The change in pronunciation of each utterance results in a given fee.
Probability of not producing pheneem other than neem, or
or some fees other than the given fee
Generates a probability of producing neem. Finim's
The basic form determines the phoneme map in the basic form of each word.
The number of syns is approximately the number of finems per word.
Almost equal. In general, sound at a rate of 1 per centisecond
Generate finemes well with processor
In this case, there are 60 to 100 finemes per word. As an alternative, the said U.S. Pat.
Phonetic phonemes, as described in issue 672974
A word basic form of the machine may be constructed.
In this case, each phoneme machine corresponds to a single note in the phonetic alphabet,
Contains 7 states and 13 transitions. D. Problem that the invention aims to solve The Finim phoneme machine is structured by two states.
easily and automatically determined. deer
However, Finim's basic form is quite long
Therefore, the matching process takes a long time.
Requires calculation. On the other hand, the basic form of phonetic symbols
is short and requires less calculation, but the precision
is not easily and automatically determined due to the low
In this case, the phonetic symbols must be entered manually. What is the basic form of finem and the basic form of phonetic symbols?
Although both are effective and helpful, they vary depending on usage.
optimization may not be possible. Therefore, it is an object of the present invention to
largely avoids the disadvantages of basic forms of phonetics and phonetics.
At the same time, the basics encompassing the main advantages of each
It is to provide form. Furthermore, it is an object of the present invention to create a short and simple phonetic map.
Automatically form word basic form of Ruyuf model
It is to be. Furthermore, it is an object of the present invention to
It is shorter than the basic form of Neem (accurate
, the target word (which has the characteristics of the basic form of the phonetic type)
The objective is to provide the basic form of the code. E. Means to solve the problem According to the invention, each word in the vocabulary is
It is determined by the basic form of Finim.
Assume. In addition, the alpha alphabet of synthesized phonemes is also included.
Assume that it has been issued. phooneme
(Phoneme)
It is desirable to select synthesized phonemes that are this
If so, each phoneme machine will produce a phonie
It corresponds to the element of the alphabet and its font.
The beam element in the feeneme string
Transitions and transitions used when matching to neem
Store label probabilities. The elements in the alpha alphabet of synthesized phonemes are
element in the alphabet of the phoneme
can do. Simple basic form of synthesized phonemes obtained by the present invention
The state is constructed as follows. Say the target word once, or more than once if possible.
A string of finemes with each utterance
is generated. Generated Fini of target word
string (may be multiple strings)
) that generates a string based on
Finim word basics with (simultaneous) probability
A form is selected. Then each string of target words
Align the ring to the Finem basic form.
Ru. In this way, each frame in the basic form of Finim
the enim phoneme for each string (of the target word).
the corresponding fineme in the
Ru. Each fee in the basic word form of feeneem
The neem phoneme is replaced by its corresponding synthetic phoneme.
and thereby form the basic form of synthesized phonemes.
Ru. Next, the following steps are executed. (a) Select pairs of adjacent synthesized phonemes. (b) For each fineme string, the selected
The fee aligned for pairs of adjacent synthesized phonemes
Determine the substring of neem. (c) For all fineme strings.
Generate each determined substring
Determine the single synthesized phoneme that yields the highest joint probability
do. (d) Step for all pairs of adjacent synthesized phonemes.
Repeat steps (a) to (c) and create one synthesized phoneme for each pair.
decide. (e) Each pair of synthesized phonemes in the basic form is replaced by
to replace it with one synthesized phoneme determined by
measure the negative effects of (f) One synthesized phoneme with minimal negative impact and its corresponding response.
Replace the phoneme pairs. (g) One synthesized phoneme and its corresponding synthesized phoneme pair
After replacing the
supply. (h) Steps (a) to (g) for this new basic form.
Repeat. Step (h) converts the basic form of the synthesized phoneme to the desired length.
shorten to, i.e. replace common pairs with sets
until the negative effects of
Repeat as many times as necessary. Furthermore, for a given phoneme alpha
When a matching synthesized phoneme alphabet is selected,
The shortened basic form generated by the present invention is
For use in applications related to phonetic systems
It is equipped with a phonetic display of words that can be used. F Example F1 speech recognition system environment F1a General explanation (Figures 2 to 7) Figure 2 is an overview diagram of the speech recognition system 1000.
Show the diagram. This system is a stack deco
1002 and the audio processor connected to it.
Setusa (AP) 1004, high-speed approximate acoustic matuchin
Array processor 1006 performs precision
Array processor that performs acoustic matching
1008, language model 1010, and work
Station 1012 is included. The audio processor 1004 processes audio waveform input.
A string of bells, each with a corresponding
It is changed to a fe-neem, which roughly identifies the single note symbol.
It is designed to be replaced. With the system of the present invention
The sound processor 1004 is unique to human hearing.
Based on a special model, U.S. Patent Application No. 06/
No. 65401 (filed on October 26, 1984)
Ru. Label from sound processor 1004, e.g.
In other words, Finim is a stack decoder 1002.
sent to. FIG. 3 shows the stack decoder 10
02 logic element is shown. That is, the stack de-
The coder 1002 connects the search device 1020 and
Workstation 1012 connected to
Tough Ace 1022, 1024, 1026 and
contains 1028. of these interfaces
Each includes a sound processor 1004, an array processor
Setsa 1006, 1008 and language model 1
010 respectively. During operation, the feedback from the sound processor 1004 is
Neem is processed by the array process by the search device 1020.
The data is sent to Tsusa 1006 (high-speed matching).
The high-speed matching procedure described below is based on the
Patent Application No. 06/672974 (filed on November 19, 1984)
is also listed. The purpose of matching is to simply
In other words, for a given label string, at least
By determining the single most likely word
be. Fast matching searches for words in a word vocabulary.
the string of the given incoming label.
is designed to reduce the number of candidate words in
There is. Fast matching is a stochastic finite state machine
(also referred to as Markov model in this specification)
It is something that can be developed. After fast matching reduces the number of candidate words,
Stack decoder 1002 uses language model 10
10 and, if possible, the existing Mieji.
Based on each candidate word in the high-speed matching candidate list,
Determine the contextual likelihood of the code. Precision matching allows these words to be
Fast matching with moderate likelihood as words
From candidate list, inspect based on language model calculations
It is desirable to do so. Precision matching is also done in the US.
Patent Application No. 06/672974 (filed on November 19, 1984)
It is described in. Precision matching is shown in Figure 4.
It is implemented by a Markov model phoneme machine as shown.
go After precision matching, call the language model again
It is desirable to determine the word likelihood. Book
The stack decoder 1002 of the invention is a high speed decoder.
matching, precision matching, and language models.
Using the information obtained from the use of the generated label
most likely path for the word in the string
i.e. designed to determine the sequence
There is. Find the most likely sequence of words
The two traditional methods are Viterbi recovery.
encoding and single stack decoding. these
Each of the techniques provides pattern analysis and machine information.
IEEE Bulletin on PAMI Volume 5 No. 2, 1983
Articles published outside of L.R.B. in the March issue,
“Maximum likelihood approach to continuous speech recognition” (L.R. Bahl et al.
al, “A Maximum Likelihood Approach to
“Continuous Speech Recognition”, IEEE
Transactions on Pattern Analysis and
Machine Intelligence, Vol. PAMI-5, No. 2,
March 1983)
It is listed. The single stack decoding method uses
are listed in a single stack according to their likelihood.
and decrypted based on this single stack.
Ru. Single stack decoding is
Since the degrees differ somewhat, normalization is commonly used.
This is based on the fact that The Viterbi method does not require normalization and is generally small.
Suitable for small tasks. Another alternative is to use a small vocabulary system.
Decryption may be performed by the system. this place
possible word sequence if
examines the word combinations and determines which combinations are generated.
highest probability of producing a label string with
Determine whether you have The computational demands of this method are large.
This is not practical for large vocabulary systems. Stack decoder 1002 actually
acts to control the elements of, but is executed
There are not many calculations. Therefore, the stack decoder
1002 is a VM (virtual machine)/system program.
Product Introduction Release 3
(1983), as described in publications such as
IBM VM/370 Operating System Controls
Desired to include a 4341 processor running under control.
Delicious. Array processors that perform a significant amount of calculations
Setsa is a floating point system
This is achieved using a commercially available 190L manufactured by (FPS).
Ru. New features including multiple stacks and unique decision methods
This method is L.R.Bahl
Invented by outsiders. This method is shown in Figure 5.
6 and 7. Figures 5 and 6 show that between consecutive labels
Multiple consecutive labels Y generated at “interval”₁,Y₂……
It is shown. FIG. 6 also shows multiple word paths, i.e.
Path A, path B, and path C are shown.
In the context of Figure 5, path A is the entry “to be
path B to entry “two b”, path B to entry “two b”, path B to entry “two b”
C would correspond to the entry "too". subject
For word passes, the highest probability of termination is
The label that the target word path has (i.e. the equivalent
label spacing). Label like this
It is called a “boundary label.” of a word path W representing a sequence of words.
, the most likely end time (between two words)
Displayed in label string as “border label”
) is published in IBM Technology Disclosure Bulletin, Volume 23, No.
No. 4, September 1980, outside El R Bar
Paper “Fast Acoustic Matching Calculation” (L.R.Bahl et al.
al, “Faster Acoustic Match Computation”,
IBM Technical Disclosure Bulletin, Vol.23,
No. 4, September 1980)
can be discovered by any known method. easy
In other words, this paper focuses on two important points:
Section: (a) How many label strings Y are in the word
(or word sequence)
Is there? (b) At what label spacing (of the label string)
whether the partial sentence (corresponding to the part) ends It explains how to tackle this issue. For any given word pass, the label strip
each from the first label of the string to the border label
, or the “likelihood” associated with the label interval
has the value "all of the likelihood values for a given word path"
collectively calculates the “likelihood vector” for a given word path.
Therefore, for each word pass,
There is a corresponding likelihood vector. Likelihood value L_tis shown in Figure 6.
It is shown. Word Pass W¹,W²,...,W^sof a gathering of
“Likelihood envelope” Λ at label interval t_tis mathematically
It is defined as follows. Λ_t=max(L_t(W¹),...,L_t(W^s)) That is, for each label interval, the likelihood envelope is
the latest associated with any word pass in said collection.
Contains a high likelihood value. Figure 6 shows the likelihood envelope 1040
It is shown. If the word path corresponds to a complete sentence,
considered “complete”. For the complete path, please enter
When a speaker reaches the end of a sentence, e.g.
It is desirable to be identified by pressing the button.
The input entered is between the labels marking the end of the sentence.
synchronized with the interval. The complete word password is
It cannot be extended by adding words. part
A typical word path accommodates incomplete sentences and
can be done. Partial paths are “alive” or “dead”
Word Pass is classified as a “pass”.
is already extended, the “dead”
It is “alive” when it has not yet been extended. child
already extended by at least one
by forming a longer extended word path of
The path that is in the
There is no such thing. Each word path is
Characterizing it as “good” or “bad”
is possible. The word pass is marked with its boundary label.
The label corresponding to the word path whose word path is the
Good if it has a likelihood value that is within the large likelihood envelope
It is a password. Otherwise, word
Pass is a bad word pass. maximum likelihood envelope
Decrease each value by a certain value to find the good (bad) limit
It is desirable to act as a level.
Yes, but not necessarily necessary. There is a stack element for each label interval.
Ru. Each living word pass looks like this
label corresponding to the boundary label of the living path
Assigned to the stack element corresponding to the interval. vinegar
The task elements are listed in order of likelihood value (
) 0.1 or more
May have birds. Next, the stack decoder 1002 in FIG.
The steps to be executed will be explained below. Form a likelihood envelope and determine which word path is better
Determining the stack decoding method shown in Figure 7
There is a mutual relationship as shown in the flowchart. In the flowchart of FIG. 7, block 1050
So, first, the null path is the first stack (0)
to go into. In block 1052, the previously confirmed
The (complete) stack element containing the complete path
If available, it will be supplied. (complete) stack required
Each complete path in the element has a likelihood associated with it
has a vector. highest likelihood for that boundary label
The likelihood vector of the complete path with
Determine the likelihood envelope. If (complete) stack element
If there is no complete path to
The interval is initialized to −∞. Additionally, the complete path
Even if is not specified, the maximum likelihood envelope is −∞
may be initialized to . Initial envelope settings
is performed in blocks 1054 and 1056. After the maximum likelihood envelope is initialized, a predetermined amount △
△ provision that exceeds the reduced likelihood by
form a good region of and below the reduced likelihood
△ Forms poorly defined areas. If △ is large, it is large.
The higher the password, the more likely it is that the password can be extended.
The number becomes larger. L_tlog to determine_Tenusing
If the value of △ is 2, a satisfactory result can be obtained.
It will be done. The value of △ is uniform along the length of the label interval
Although it is desirable, it is not necessary to
isn't it. The word path has a border label of △−good region.
If the word path has a likelihood within the range
is marked as “good”. In other cases, the
The code path is marked as “bad”. As shown in Figure 7, the likelihood envelope is updated and the word
A “good” (extendable) path, or
or as a “bad” path.
Blog about finding the longest unmarked word pass
It starts with 1058. Not marked with 2 or more
short word path corresponds to the longest word path length
If the border label is on the stack
A word path with a likelihood of is selected. War
block 1060 if a path is found.
, the likelihood at that boundary label is △− well-specified region.
Find out if it is within the area. If within a good area
If not, in block 1062, △
Mark the path within the area, and in block 1058,
Find unmarked living paths. too
If it is within the acceptable range, in block 1064, △
Mark the path within the defined good area and block 1
In step 066, update the likelihood envelope and mark it as “good”.
Contains the likelihood value of the tracked path. That is,
For each label interval, the updated likelihood value is (a) the current likelihood value within its likelihood envelope; (b) Associated with word passes marked “good”
likelihood value is determined as the larger likelihood value between. this
The operations take place in blocks 1064 and 1066.
Ru. After the envelope has been updated, block 1058
Return, longest unmarked, best alive
Find the password again. This loop uses unmarked word
Iterates until there are no more spaces left. Not marked
If you run out of passwords, block 1070
The shortest word path marked as “good” in
is selected. If 2 or more have the shortest length
If you have a “good” word pass, block 10
72, find the word with the highest likelihood for that boundary label.
The selected shortest path is
It will be extended. That is, at least one possibility
If the subsequent word
language models, precision matching, and language models.
determined by successful implementation of Dell procedures.
Ru. For each likely subsequent word, extended
A word pass is formed. In detail,
The lengthened word path is the shortest word path selected.
Append a promising trailing word to the end of the decoded path.
Formed by adding The selected shortest word path is
After forming the code path, the selected word
The path is removed from the stack in which it was an entry.
instead, each extended word
The path is inserted into the appropriate stack. In particular, extension
The word path that has been created will be
entry into the stack (block 107).
2). An extended path is formed and its stack is re-formed.
Once formed, return to block 1052 and continue the process.
The process is repeated. Therefore, at each iteration, the shortest, best “good”
selected and extended. in some iteration
Word paths marked as “bad” paths are
Repetition can result in a “good” pass. Then,
Is the live word pass a “good” pass or a “bad” pass?
The characteristics of a “good pass” are uniquely attached to each iteration.
given. In fact, the likelihood envelope is one iteration and
Since it does not change significantly from next iteration to
The calculations that determine whether a path is good or bad are done efficiently.
be exposed. Furthermore, normalization becomes unnecessary. If you want to identify a complete sentence, block 1074
It is desirable to include. i.e. alive
What remains unmarked in Word Pass
If there is no “good” word pass that should be extended
If so, decryption ends. each of its boundary labels
The complete word path with the highest likelihood
the most likely word for the power label string.
identified as a code sequence. For continuous speech where sentence ends are not identified, path extension
The length is continuous, i.e. the system
line for a given number of words desired by the user of the system.
be exposed. F1b auditory model and speech recognition system acoustics
Its realization in the processor (Figures 8 to 1)
Figure 4) FIG. 8 shows an audio processor 110 as described above.
A specific example of 0 is shown. Acoustic wave input (e.g.
natural sounds) are sampled at a predetermined rate.
Enters A/D converter 1102. representative sample
The processing speed is 1 sample every 50 microseconds.
Ru. time to shape the edges of the digital signal.
A window generator 1104 is provided. Time window occurs
The output of the
FFT (Fast Fourier Transform) device that provides torque output
Enter 1106. Then, the output of the FFT device 1106 is labeled
L₁L₂...L_fis processed to generate . Features
Selection device 1108, cluster device 1110, prototype
The device 1112 and the encoder 1114 cooperate.
generate a label. When generating labels, the original
places points (or vectors) in space based on selected features.
formed as a vector). The audio input is selected
A counterpart that can be compared to the original due to the same characteristics
special to supply points (or vectors) into space.
It is marked. In detail, when defining the prototype, the cluster
The set of points is assigned to each class by the setting 1110.
group as data. The method of forming clusters is
probability component (such as a Gaussian distribution) applied to speech
Based on cloth. The prototype of each class is (class
(with respect to the center locus or other features of the data)
is generated by the location 1112. The generated prototype
and acoustic input (both with the same features selected).
) enters the encoder 1114. Symbolization device 1
114 performs a comparison procedure so that a particular sound
Assign a label to the acoustic input. The selection of appropriate features represents the acoustic (speech) wave input.
This is an important element when removing the wasu label. here
The acoustic processor described has improved feature selection.
Includes device 1108. Follow this sound processor
Then, an auditory model is taken and the speech recognition system
used in audio processors. auditory model,
This will be explained with reference to FIG. FIG. 9 shows a portion of the human inner ear. describe in detail
If so, the inner hair cells 1200 and the fluid-containing groove 1
Distal end 1202 extending to 204 is shown in detail.
There is. In addition, upstream from the inner hair cells 1200, outer hair cells 1200
hair cell 1206 and distal end 1 extending into groove 1204
208 is shown. 1200 inner hair cells and outer hair
Cell 1206 has nerves that transmit information to the brain.
It matches. In particular, Nieuron undergoes electrochemical changes.
The electrical pulses are carried along the nerves to the brain,
It will be processed. Electrochemical changes are fundamental
Stimulated by mechanical movement of membrane 1210. The basement membrane 1210 serves as a frequency analyzer for acoustic wave input.
The area along the basement membrane 1210
Is it conventional to respond to each critical frequency band?
is known. responds to the corresponding frequency band.
Each portion of the basement membrane 1210 has an acoustic waveform
Affects the perceived volume of the input. That is,
The volume of the tone can be determined by combining two tones of similar power intensity.
than if the bands occupy the same frequency band.
If the two tones are in separate critical frequency bands
It is perceived as larger when the By the basement membrane 1210
There are 22 defined critical frequency bands.
I know that. Match the frequency response of the basement membrane 1210
Therefore, the present invention is advantageous in that the critical frequency band is
Physically convert the input acoustic waveform into some or all parts.
and then for each defined critical frequency band.
The signal components are examined separately. This feature is
The signal from the FFT device 1106 (Fig. 8) is
for each critical frequency band tested.
Providing separate signals to feature selection device 1108
This is done by A separate input is also provided by the time window generator 1104.
Blot on a time frame (preferably 25.6 ms)
is blocked. Therefore, the feature selector 1108 has 22
It is desirable to include the following signals. of these signals
each for a given frequency band per time frame
represents the strength of the sound. The signal is a conventional critical band filter in Figure 10.
Preferably, the filter is filtered by filter 1300. Next
, the signal is individually expressed as a function of frequency and the change in volume as a function of frequency.
Processed by an acoustic equalization converter 1302 that perceives
do. By the way, given dB level at one frequency
The perceived loudness of the first tone of the
the volume of the second tone of the same dB level at the frequency of
It may be different. Volume equalization converter 1302
are based on empirical data and each frequency
Convert the band signals so that each is measured on the same loudness scale.
be determined. For example, volume equalization converter 1
302 is a 1933 Fletscher and Munsson
(Fletcher and Munson) with some changes.
By
can be imaged. Figure 11 has been changed to the above study.
Shows the result of adding . According to Figure 11, at 40dB
A 1KHz tone has a volume level of 60dB compared to a 100Hz tone.
It turns out that it corresponds to Bell. The volume equalization converter 1302 converts the music shown in FIG.
Adjust the volume according to the line, equal regardless of frequency
produces a volume of In addition to frequency dependence, Figure 11 shows
As is clear from examining the wave number, the change in power is
Does not respond to changes in volume. That is, the intensity of the sound,
That is, amplitude fluctuations are perceived at all points.
is not reflected in similar changes in volume. for example,
At a frequency of 100Hz, 10dB around 110dB
Perceived volume change is 10dB around 20dB
is much larger than the perceived volume change. this
The difference is a volume compression device that compresses the volume in a predetermined way
1304. Volume compression device 1304
places the loudness amplitude measurement in phons into units of son.
By changing the power P to its cube root P^1/3to
Can be compressed. Figure 12 shows known phone pairs determined empirically.
Shows the relationship between Thorns. With the use of sone units,
The model of the present invention is almost correct even with large audio signal amplitude.
Maintain a solid state. 1 sone is a 1KHz tone with 40dB volume and regulation.
has been established. FIG. 10 shows a new time-varying response device 13.
06 is shown. This device has each critical frequency
Band-related volume equalization and volume compression signals
Works better. In detail, the tested frequency
For every few bands, the neural firing rate f changes each time frame.
It can be determined by The firing rate f is the acoustic process of the present invention.
It is defined as follows according to the f=(So+DL)n (1) However, n is the amount of neurotransmitter; So is the acoustic wave
Spontaneous firing related to neural firing independent of shape input
fire constant; L is the volume measurement; D is the displacement constant.
So.n is a spontaneous phenomenon that occurs regardless of the presence or absence of sound wave input.
DLn corresponds to the neural firing rate due to acoustic wave input.
Corresponds to firing rate. The important point is that in the present invention, the value of n is determined by the following equation:
It has the characteristic of changing over time.
Ru. dn/dt=Ao−(So+Sh+DL)n (2) where Ao is the recruitment constant; Sh is the spontaneous neurotransmission
is the material attenuation constant. The new relationship shown in equation (2)
is that neurotransmitters are not produced at a certain rate at Ao.
(a) Decay (Sh.n), (b) Spontaneous firing (So・
n), and (c) neural firing due to acoustic wave input (DL・
n) is taken into consideration. these
The modeled phenomenon is at the location shown in Figure 9.
Assume that it happens. As is clear from equation (2), the following amounts of neurotransmitters and
and the next firing rate is at least as high as the current amount of neurotransmitters.
It is proportional to the square of the sound processor of the present invention.
This shows the fact that it is nonlinear. Sunawa
The amount of neurotransmitter in state (t+△t) is
of neurotransmitters in state (t+dn/dt・△t)
equals quantity. Then, n(t+△t)=n(t)+(dn/dt)・△t (3) holds true. Equations (1), (2) and (3) describe the operation of the time-varying signal analyzer.
represent. A time-varying signal analyzer shows how the auditory system changes over time.
It is adaptive and the auditory nerve signal is non-directive with the acoustic wave input.
It shows the fact that they are linearly related.
By the way, the acoustic processor of the present invention is based on the nervous system.
to better follow the apparent temporal changes in
The best way to perform nonlinear signal processing in speech recognition systems
This is the first model offered. Reduce the number of unknown terms in equations (1) and (2)
Therefore, in the present invention, the following
Use the formula. So+Sh+DL=1/T (4) However, T is the audio wave input generated
After that, the auditory response drops to 37% of its maximum value.
This is the measured value of the time it takes to complete the process. T is a function of volume
With the sound processor of the present invention, various
Known display of volume level response attenuation
Take it out from the graph. That is, a tone of constant volume
When a response is generated, initially a high level response is generated.
After that, the response is determined by the time constant T.
and decays toward steady-state levels. sound
If there is no acoustic wave input, T=T₀It is. This is 50
It is about milliseconds. Volume is L_naxIf T=
T_naxIt is. This is about 30 milliseconds. Ao=
By setting it to 1, 1/(So+Sh) becomes
If L=0, it is determined to be 5 centiseconds. L is
L_naxSo, L_naxIn the case of = 20 sones, the following formula holds true. So+Sh+D(20)=1/30 (5) According to the above data and formula, So and Sh are
It is determined by equations (6) and (7) shown below. So＝DL_nax/ [R+(DL_naxT₀R)-1〕 (6) Sh=1/T₀−So (7) however, R = f stable state | L_nax/f stable state | L=0 (8) f steady state is when dn/dt is 0 for a given sound
Expresses the firing rate in terms of volume. R is the only variable left in the sound processor
It is. Therefore, the performance of this processor is R
Just change it and it will change. That is, R is the performance
One parameter that can be adjusted to change the
meter, typically stable against transient effects
It means minimizing the effects of the condition. similar
Output pattern is inconsistent when using audio input
This is generally due to differences in frequency response,
difference, background noise and (sound signal stability)
(affects the dynamic part but not the transient part)
Minimize steady-state effects since they are caused by distortion.
It is desirable to The value of R is
configuration to optimize the error rate of the system.
It is desirable that The most
A suitable value is R=1.5. In that case, So and Sh
The values are 0.0888 and 0.11111, respectively, and D
The value is 0.00666. Figure 13 shows the operation of the audio processor according to the present invention.
This is a flowchart. Preferably sample at 20kHz
daisi during a 25.6 ms time frame.
The talized audio passes through a Hanning window 1320 and its
The output of the DFT1322 is at 10 ms intervals.
It is desirable to perform a double Fourier transform. conversion
The output is filtered in block 1324 to provide at least
One frequency band (preferably all critical frequencies)
several bands or at least 20 bands)
Each provides a power density output. Next, power
- Density is block 1326, recorded size
is converted to volume level. This operation is the first
This can be easily carried out by changing the graph in Figure 1. So
Summary of the process after (limitations of block 1330)
(including value updates) is shown in FIG. In Figure 14, first, the filtered frequency
Sensory limit Tf and audible limit T for each band m_h
are set to be 120dB and 0dB, respectively.
(block 1340). Then the voice cowl
total frame register and histogram
Reset system registers (block 134)
2). Each histogram contains bins,
(in a given frequency band)
or similar measurements within their respective ranges.
represents the number of samples, or the number of counts. Main departure
In light, the histogram (for a given frequency band
) within each of multiple volume ranges.
It is preferable to express the number of centiseconds of the period. example
For example, in the third frequency band, 10dB and 20dB
There may be 20 centiseconds between powers. Similarly,
In the 20th frequency band, between 50dB and 60dB,
If there are 150 centiseconds out of a total of 1000 centiseconds,
There is a case. Total number of samples (i.e. centiseconds)
and the percentiles are taken from the counts contained in the bins.
Served. Block 1344 identifies each frequency band.
The frames at the output of the filter are examined and blocked.
1346, the appropriate histogram (per filter)
1) The bins inside are incremented. block 1348
, the total number of bins whose amplitude exceeds 55 dB is the filter
data (i.e., frequency band), and
Determine the number of filters that indicate the presence of voices. block
1350, with a minimum number (e.g. 20
If there is no filter in 6), block 1
At 344, the next frame is inspected. existence of voice
If there are enough filters to show, block 135
2, increment the voice counter. voice counter
The audio appears in block 1354 for 10 seconds and the block
New Tf and T in Lock 1356_hThe value of
is incremented until determined for each router. New Tf and T for given filter_hThe value of is
It is determined as follows. For Tf, the maximum of 1000 bins
dB of the bin holding the 35th sample from the top
The value (i.e. the 96.5th percentile of volume) is
BIN_Hand Tf is defined as Tf=BIN_HSet to +40dB
be done. T_hIf , from the lowest bin (0.01)
Hold the (total number of bins - audio count)th value
The dB value of the bin is BIN_Lis defined as That is,
BIN_Lis classified as audio in the histogram.
bins of 1% of the number of samples excluding those
Ru. T_his T_h=BIN_LDefined as −30dB. Blocks 1330 and 1332 in FIG.
, the amplitude of the sound updates the limit value as mentioned above
and changes per son based on the updated limit value.
converted and compressed. Introducing the son unit and compressing
An alternative method is to use the filter (after the bin has been incremented).
Take the amplitude "a" and convert it to dB using the following formula. a^dB=20log_Ten(a)−10 (9) Next, each of the filter amplitudes is equivalent to
Range between 0dB and 120dB to give a volume of
compressed into a^egl=120(a^dB−T_h)/(Tf−T_h) (Ten) Then a^eglis the volume level (phone unit) using the following formula.
) to an approximate value of the volume in son units.
(Mapping a 1kHz signal to 1 at 40dB) is desirable.
Yes. L^dB=(a^egl-30)/4 (11) Then, an approximation of the volume in sones L_sis given by the following equation
available. L_s=10(L^dB)/20 (12) At step 1334, L_sis the input of equations (1) and (2)
output firing rate per frequency band
Determine f. For 22 frequency bands, the 22-dimensional
The vector is acoustic over successive time frames.
Characterize the wave input. However, in general, 20
Frequency bands are usually scaled in mel
The filter bank is used for testing. Block 1 before processing the next time frame.
337, determine the “next state” of n according to equation (3).
Ru. The acoustic processor described above has a firing rate f and a neural
When having a DC pedestal with a large transmission mass n
Needs improvement in usage. That is,
The dynamic cleanliness of the f and n equation terms is important.
If so, use the following formula to lower the pedestal height.
Geru. Steady state and no acoustic input signal present
(L=0), Equation (2) is expressed as follows in the stable state.
can be solved for the partial state n′. n′=A/(So+Sh) (13) The internal state of the amount of neurotransmitter n(t) is as follows
are shown as steady state part and fluctuating part as
Ru. n(t)=n′+n″(t) (14) Combining equations (1) and (14) yields
Fire rate is obtained. f(t)=(So+D・L)(n′+n″(t))(15) The term So・n′ is a constant, but all other terms
is the variable part of n or expressed by (D・L)
contains the passed input signal. Subsequent processing is done using the output vector.
Since it is only related to the squared difference between torques, the constant term is
It will be ignored. From equations (15) and (13), we get the following equation:
It will be done. f″(t)=(So+D・L)・[{n″(t)+D・L
・
A}/(So+Sh)〕 (16) Considering equation (3), the “next state” is as follows.
Ru. n(t+△t)=n′(t+△t)+n″(t+△t) (17) n(t+△t)=n″(t)+A−(So+Sh+D・L) ・(n′+n″(t) (18) n(t+△t)＝n″(t)−(Sh・n″(t) −(So＋Ao・L^A)・n″(t) −(Ao・L^A・D)/(So+Sh)+Ao −(So・Ao)＋(Sh・Ao)) /(So+Sh) (19) Equation (19) can be written as follows if all constant terms are ignored.
I'm going to growl. n″(t+△t)＝n″(t)(1-So・△t)−f″(t) (20) Equations (15) and (20) are calculated for each 10 ms
Output expression applied to each filter during the time frame
and configure the state update expression. Using these expressions
The result is a vector of 20 elements every 10 ms,
Each element of this vector is scaled by mel
each frequency in the filter bank
Corresponds to the firing rate of the band. Regarding the embodiment described above, the flowchart in FIG.
Special calculation of fire rate f and “next state” n(t+△t)
Equation (11) and
(16), f, dn/dt and n(t+△t)
This applies except for replacing the expression. A unique value for each equation term (i.e., t₀=
5csec, t_L=3csec, Ao=1, R=1.5 and
Lmax=20) can be set to other values,
The So, Sh, and D terms are set to different values for the other terms.
When set, each desired value 0.0888,
The value will be different from 0.11111 and 0.00666. The present invention can be applied to various software or hardware.
It can be implemented by a. F1c Precision matching (Fig. 4, Fig. 15, Fig. 1)
Figure 6) Figure 4 shows an example of a phonetic phoneme machine, 2000.
shows. Each phonetic matching machine uses a probabilistic
is a state machine limited to (a) Multiple states Si; (b) Multiple transitions tr (Sj | Si): A certain transition is different
Between states, some transitions transition between the same states, and each
Transitions have corresponding probabilities; (c) Corresponding labels for each label that can be generated by a specific transition.
actual label probability It is characterized by having the following. In Figure 4, seven states S₁~S₇and 13 history
Transfer tr1 to tr13 are precision matching phoneme machine 20
00, and the three transitions tr11, tr
The paths of tr12 and tr13 are shown with dashed lines.
At each of these three transitions, the phoneme produces a label.
can change from one state to another without achieving
Ru. Therefore, such a transition is called a null transition.
Ru. Generate labels along transitions tr1 to tr10.
can be done. In detail, transition tr1 to tr
At least one label along each of the 10
There may be unique probabilities that are generated.
Ru. For each transition, the system can generate
There is a probability associated with each label. In other words, too
can be selectively generated by an acoustic channel.
If there are 200 possible labels, each transition (not null)
The associated “actual label probability” is 200
each of which has a corresponding label for a particular transition
corresponds to the probability generated by a phoneme. transition tr
The actual label probability of 1 is given by the symbol P, as shown.
and 1 to 200 surrounded by the following brackets.
Represented in columns. Each of these numbers is
Represents a bell. For label 1, precision matchin
The phoneme machine 2000 generates label 1 at transition tr1.
There is a probability P[1] of various actual labels
Probabilities are written in relation to labels and corresponding transitions.
It is remembered. The string with label y1y2y3... is given
Precision matching phoneme machine 20 that corresponds to the phonemes of
00, the matching procedure is performed.
Ru. For procedures related to precision matching phoneme machine
This will be explained with reference to FIG. Figure 15 is a trellis diagram of the phoneme machine in Figure 4.
be. As in the case of the phoneme machine, this training
Squirrel diagram is also in state S₁from state S₇null transition to state S₁
from state S₂transition to, and state S₁from state S_Fourfart
shows the transition of Transitions between other states are also shown.
Ru. In addition, the trellis diagram is horizontally
Indicates the time. Starting probability q₀, and q₁is the phoneme
Time t=t of that phoneme₀or t=t₁to each of
represents the probability of having a start time at . At the start of each
The respective transitions in time are also shown. China
For example, the interval between consecutive start (and end) times
must have a length equal to the label time interval.
desirable. Using precision matching phoneme machine 2000
how much a given phoneme is in the label of the incoming string
When determining a tight match,
Find the end time distribution of the phoneme and map the phoneme.
Used to determine the cutting value. End time distribution
The way to perform precision matching depends on
All described in this invention with respect to the tucking procedure
This is common to all phoneme machine embodiments. precision pine
When generating the finishing time distribution to perform the
Precision matching phoneme machine 2000 is accurate and complex.
Requires complicated calculations. First, according to the trellis diagram of FIG. 15, time t
=t₀required to get the start and end times in
Find out about calculations. The phoneme map shown in Figure 4
In the case of the example of the
Ru. Pr(S₇,t=t₀)=q₀T(1→7)+Pr(S₂,t=
t₀)・T(2→7)+Pr(S₃,t=t₀)・T(3
→7) (21) However, Pr represents the probability, and T is the 2 in parentheses.
represents the transition probability between two states. This formula is t
=t₀There are three states in which the end time can be
The probability of each is shown. Furthermore, t=t₀end time of
is state S₇limited to current occurrences. Next, end time t=t₁When we examine the state S₁other than
calculations must be made for every state of
do not have. state S₁starts at the end time of the previous phoneme.
For convenience of explanation, state S_FourOnly calculations related to are shown. S_FourIn this case, the calculation becomes: Pr(S_Four,t=t₁)Pr(S₁,t=t₀)・T(1→
4)・Pr(y₁,1→4)+Pr(S_Four,t=t₀)・
T(4→4)・Pr(y₁,4→4) (22) Equation (22) is given at time t=t₁The phoneme machine is in state S_Four
The probability that is is the following two terms: (a) Time t=t₀In state S₁With probability that state S₁mosquito
state S_FourMultiplying the transition probability to
given label y in the string₁is in state S₁from state S_Four
The value obtained by multiplying the probability of transition to (b) time t = t₀
In state S_FourWith probability that state S_Fourfrom to itself
Multiply by the transition probability, and further state S_Fourto itself
given label y as the one to be transferred₁probability of generating
This shows that it is determined by the sum of the value obtained by multiplying
vinegar. Similarly, (state S₁(excluding) other conditions
calculation is also performed, and the phoneme is determined at time t=t₁in certain conditions
generate the corresponding probability that is the state. In general, given
When determining the probability of being in the target state at the time of
The matching is (a) Each state and state before the transition leading to the target state
and the respective probabilities of each of the previous states; (b) for each of said previous states, its label string;
Each of the previous states and the current
Labels that must be generated on transitions between states
recognize the value representing the probability of (c) that representing the probability and label probability of each previous state;
Combine each value and target by corresponding transition
gives the probability of the state. The overall probability of being in the target state is
is determined from the target state probabilities for all transitions.
state S₇The calculations for the three null transitions
term, whose phoneme is in state S₇For phonemes ending in
time t=t₁Allows you to start and end with
Ru. Time t=t₀and t=t₁determine the probability for
Determining the probabilities of other pairs of end times, such as when
It is desirable to do this so as to form an end time distribution.
Delicious. The end time distribution value of a given phoneme is
How well do the phonemes of match to the incoming labels?
Displays whether the How well words match incoming labels
the phoneme that represents the word.
are processed sequentially. Each phoneme is the end time of the probability value
Generate a distribution. The phoneme matching value is
By summing the time probabilities and taking the logarithm of that sum,
can be obtained. The start time distribution of the next phoneme is the end time
It is derived by normalizing the distribution. This positive
In normalization, for example, we define each of those values as
scale by dividing by the sum of
Make sure that the sum of the ringed values is 1. Examining a given word or word string
There are at least two ways to determine the number of phonemes h that should be
be. In the depth-first method, the computation follows the basic form.
(continuously subtotal by each successive phoneme)
). the given sound that this subtotal follows
If it is found that the elementary position is below a predetermined limit value,
The calculation ends. Another method, breadth-first method
calculates similar phoneme positions in each word.
cormorant. The calculation consists of calculating the first phoneme of each word, followed by
calculate the second phoneme of each word, and so on.
Do it next. In the breadth-first method, each word's
Calculated values along the same number of phonemes are relative to the same phoneme.
Compare by position. In either method, matching
The word with the largest sum of values is the goal we were looking for.
Word. Precise matching is done using APAL (Array Processor).
It is realized in the assembly language). this
is Floating Point Systems, Inc.
(Floating Point Systems, Inc.) assembly
It is 190L. Precise matching is based on the actual label probability (i.e.
That is, a given phoneme has a given label y at a given transition.
generation probability), transition probability for each phoneme machine, and
and the given time after the given start time of the given phoneme.
to remember each of the probabilities of being in a given state in
Requires considerable memory. The aforementioned 190L is
End time, preferably based on the log sum of end time probabilities.
matching value, previously generated end time probability
start time based on and sequential phonemes in the word
Word matching gain based on the matching value of
Setup to calculate each of the points
It will be done. Furthermore, precise matching is achieved by matching hands.
It is desirable to calculate the tail probabilities of the order. Ending sure
rate is the likelihood of consecutive labels independent of words.
Measure. In a simple example, a given tail probability is
corresponds to the likelihood of a label following another label.
Ru. This likelihood can be determined, for example, by using a certain sample voice.
easily determined from a string of labels generated by
be done. Therefore, precise matching is the basic form, Marco
including the statistics of the model, and the tail probabilities.
Provide sufficient storage. Each word has about 10 phonemes
For a vocabulary of 5000 words containing , the basic format is 5000
Requires ×10 memory capacity. (Marco for each phoneme)
70 distinct phonemes, 200 distinct
labels, and the certainty that any label will be generated.
If there are 10 transitions with rate, the statistic is 70×10
×200 storage locations are required.
Ru. However, the phoneme machine has three parts (open
beginning part, middle part and end part),
It is desirable that statistical tables correspond to this. (Three
It is desired that each part contains one self-loop of
Delicious. ) Therefore, the memory requirement is reduced to 70 x 3 x 200
do. For the tail probability, 200 x 200 memory locations
– sion is required. This array has 50K integer
numbers and 82K floating point storage.
Works on the legs. F1d Basic high-speed matching (Figs. 16 to 18)
figure) Is precision matching calculations expensive?
reduces the required computations without sacrificing too much accuracy.
Basic high-speed matching and alternative high-speed matching
Ching. High-speed matching is precision matching
It is desirable to use it in conjunction with. high speed matsuchi
ng extracts promising candidate words from the vocabulary.
Precision matching is often
If so, it is executed on the candidate words from this list. The fast approximate acoustic matching method is based on the above-mentioned U.S. patent.
Described in Application No. 06/672974 (filed on November 19, 1984)
has been done. In fast approximate acoustic matching, each sound
A phoneme machine is a phoneme machine with all phoneme machines in a given phoneme machine.
Transition sets the actual label probability for each label at a particular position.
It is desirable to simplify it by replacing it with a replacement value.
Delicious. A specific replacement value uses that replacement value.
If the matching value of a given phoneme is
Precision when the replacement value does not replace the actual label probability
Excessive matching value obtained by dense matching
It is desirable to choose to evaluate. This article
One way to guarantee that in a given phoneme machine
If any probability corresponding to a given label in its permutation
Choose each replacement value so that it is no larger than the set value.
This is a method of choosing. Actual labels in the phoneme machine
By replacing the probabilities with the corresponding replacement values,
requirements for determining word matching scores.
The amount of calculation can be significantly reduced. More replacement
Since it is desirable to overestimate the value of
Matching scores achieved without replacement before
Less than what is to be determined. Acoustic with language decoder with Markov model
In a particular embodiment of performing matching, each
Through shaping, phonemes are (a) multiple states and transition paths between states; (b) Probabilities T(i→j) - each of which is equal to the current state Si
represents the probability of transitioning to state Sj given
(However, Si and Sj may be in the same state or different.
The transition tr(Sj
｜Si), (c) Actual label probabilities (each actual label probability
P(y_k|i→j) is given by the phoneme machine,
In a given transition from one state to the next
label y_k(k is a symbol that identifies the label)
(representing the probability that
Ru. Each phoneme machine is (a) Each y in each of the above phoneme machines_kone particular
The value p′(y_k), (b) At each transition in a given phoneme machine, each actual
The output probability p(y_k｜i→j), the corresponding y_k
One particular value p′(y_k) divided by
means to apply, including. The replacement value must be at least a specific phoneme map.
y corresponding to any transition in thin_klabel actual
It is desirable that the magnitude is the maximum label probability.
Fast matching implementation corresponds to incoming labels
10 words chosen as most likely to occur in the vocabulary
Form a list of candidate words on the order of 100 to 100.
It is used as follows. Candidate words are determined by the language model and
It is desirable to be subject to precise matching.
stomach. The number of words considered for precise matching is
By truncating to the order of 1% of the words in the vocabulary.
Computation costs are significantly reduced while maintaining accuracy.
be done. Basic high-speed matching is used for all transitions.
Treat the actual label probability for a given label as one value.
Simplify by replacing the given label with the given
It can be generated by the phoneme machine. Sunawa
That is, for a given phoneme machine with a probability that the label occurs
Regardless of the transition in the
Replace with a specific value. This value is overestimated and
At least for any transition in a given phoneme machine
The maximum probability size of the label that occurs is
desirable. Let the label probability replacement value be
as the maximum value of the actual label probability for a given label
By setting, basic high-speed matching
The generated matching values are at least accurate
Matching values such as those resulting from the use of matching
guaranteed to be the same size. like this
Basically, high-speed matching generally involves matching each phoneme.
overestimates the value of the
Generally selected as a candidate word. precision ma
The words considered as candidates by tuching are basically
Pass according to fast matching. Figure 16 shows the basic high-speed matching phoneme machine 30.
Indicates 00. Labels (symbols and
) is the basic fast map along with the start time distribution.
Enter Tsuching Phoneme Machine 3000. Start time minutes
Entering the cloth and label strings can be done using the details above.
Similar to the input of a dense matching phoneme machine. start
Time is sometimes not distributed over multiple times.
Instead, e.g.
It may also represent the exact (phoneme start) time of the following
Ru. However, if the audio is continuous,
The completion time distribution (as explained in more detail later)
Used to form the start time distribution. Basic high speed
The matching phoneme machine 3000 has an end time distribution
and the generated end time distribution
A matching value for a specific phoneme is generated. War
The matching score for C is based on the constituent phonemes (at least
is the matching value of the first h phoneme of the word).
Defined as the sum. FIG. 17 shows the basic high-speed matching calculation. base
This high-speed matching calculation uses start time distribution, phoneme
the number or length of labels generated by
each label y_kThe replacement value p′(y_k)only
is connected with. The actual label of a given label in a given phoneme machine
Replace all Bell probabilities with their corresponding replacement values
By doing this, basic fast matching can reduce the transition probability to
(given phoneme machining)
actual label (which may be different for each transition)
Probability and probability of being in a given state at a given time
It becomes unnecessary to include the rate. By the way, the length distribution is a precise matching model.
Determined from To explain in detail, the length distribution
For each length, this procedure examines each condition individually.
and determine each transition path for each state.
This is desirable. As a result, the actual state of the inspection
The situation is (a) Given a particular label length, (b) Regardless of the output along the transition, This may occur. specific to each target state
The probabilities of all transition paths of length are summed and then
, the sums of all objective states are added and the distribution
represents the probability for a given length of . Each of the above steps
is executed iteratively over the length of . good matuchin
Following the form of the algorithmic procedure, these calculations are
Training is known as the technique of f-modeling.
It is done on a squirrel diagram. along the trellis structure
For transition paths that share branches, for each common branch
The calculation only needs to be done once, and the result is a common branch.
is added to each path containing. In Figure 17, two restrictions are included as an example.
It is. First, the label generated by the phoneme
The length of each has a probability of 1₀,1₁,1₂and 1₃have
may be 0, 1, 2 or 3. start
Time is also limited, each with probability q₀,q₁,q₂Oyo
biq₃Only four start times with . vinegar
That is, L(1₀,1₁,1₂,1₃) and Q(q₀,q₁,
q₂,q₃) is assumed. These limitations make it difficult for the eye to
The ending distribution of target phonemes is defined as the following formula:
Ru. Φ₀=q₀1₁ Φ₁=q₁1₀+q₀1₁p₁ Φ₂=q₂1₀+q₁1₁p₂+q₀1₂p₁p₂ Φ₃=q₃1₀+q₂1₁p₃+q₁1₂p₂p₃ +q₀1₃p₁p₂p₃ Φ_Four=q₃1₁p_Four+q₂1₂p₃p_Four +q₁1₃p₂p₃p_Four Φ_Five=q₃1₂p_Fourp_Five+q₂1₃p₃p_Fourp_Five Φ₆=q₃1₃p_Fourp_Fivep₆ Examining these equations, we find that Φ₃are four start times
It can be seen that it contains terms corresponding to each of the following. So
The first term of is the phoneme at time t=t₃start with and length
0 label (phoneme ends at the same time as it begins)
represents the probability of generating . The second term is that the phoneme is at time t
=t₂, and the label length is 1, and
represents the probability that label 3 is generated by that phoneme.
Was. The third term is that the phoneme is at time t=t₁Start with and
Label length is 2 (i.e. labels 2 and 3)
, and labels 2 and 3 are
Represents the probability of generation. Similarly, the fourth term is the phoneme
is time t=t₀and the label length is 3.
Yes, and the three labels 1, 2 and 3 represent that sound.
represents the probability generated by the element. Calculations and precision matching required for basic high-speed matching
Comparing the calculations required for
It turns out that it is also relatively easy. By the way,
The value of p′(y) is
remain the same value as in the case of label length probability
It is. Additionally, length and start time limitations
This makes later end time calculations easier. example
If, Φ₆So, the phoneme is at time t=t₃Start with 3 levels.
Bells 4, 5 and 6 all sound at their end time.
It must be generated and used by the elements. When generating matching values for target phonemes,
The end time probabilities along the given end time distribution are summed.
It will be done. If you wish, take the logarithm and obtain the following equation.
Ru. Matching value = log_Ten(Φ₀＋……＋Φ₆) As mentioned above, word matching scores are
The matching values of consecutive phonemes in a given word are combined.
easily determined by measuring Next, we will explain how to generate the start time distribution using Figure 18.
I will explain. In Figure 18a, the word
THE₁is broken down into its constituent phonemes and repeated.
In Figure 18b, the string of labels is on the time axis.
shown along. Figure 18c is the first start
Shows time distribution. The initial start time distribution is (silence
(in preceding words, which may contain words)
drawn from the end time distribution of the new preceding phoneme.
Ru. Label input and start time distribution in Figure 18c
Based on the end time distribution of phoneme DH Φ_DHis generated
(Fig. 18d). At the start of the next phoneme UH1
For the duration distribution, the previous phoneme ending distribution is at the limit of Figure 18d.
Determined by recognizing the time when the value A is exceeded.
It will be done. A is determined individually for each end time distribution.
Ru. A is the function of the sum of the end time distribution values of the target phoneme.
It is a number. Therefore, the interval between time a and time b is
represents the time at which the start time distribution of element UH1 is set.
vinegar. In Figure 18e, the interval between time c and time d
is, the end time distribution of the phoneme DH exceeds the limit value A,
and at the time when the start time distribution of the next phoneme is set.
Equivalent to. The value of the start time distribution is, for example, the critical value
Divide each end time value by the sum of end times that exceed A.
Obtained by normalizing the end time distribution. The basic high-speed matching phoneme machine 3000 is
Written by Floating Point Systems, Inc.
Executed using assembler 190L using APAL program.
It is being expressed. Also, according to the description herein,
books using other hardware and software.
It is also possible to develop specific forms of the invention. F1e Alternative high-speed matching (Fig. 19, 20
figure) alone or preferably with precision matching and
(or) base fast used with language models
Matching significantly reduces computational requirements.
To further reduce the computational requirements, the present invention further
, two lengths (minimum length L_nioand maximum length L_nax)of
precision by forming a uniform label length distribution between
Simplify dense matching. Basic high speed Matsuchin
In the case of labels of a given length (i.e., 1₀,
1₁,1₂etc.) generally obtain different values.
Ru. Alternative high-speed matching allows each label
Replace the length probability with one uniform value. The minimum length is the probability of non-zero in the initial length distribution.
preferably equal to the minimum length of the
Other lengths can be selected if desired. Most
The choice of major length is more arbitrary than the choice of minimum length, but
The probability that the length is smaller than the minimum and larger than the maximum is
Set to 0. The probability of length is the minimum length and maximum length
By setting it so that it exists only between
One pseudo-distribution can be shown. one way and
The uniform probability is the average probability due to the pseudo distribution.
can be set. As an alternative, uniform
The probability is set as the maximum value of the length probability, and the uniform value
It can be replaced with. By making the probabilities of all label lengths equal
The effect of
This can be easily recognized from the equation for the end time distribution. In detail
In other words, the probability of length can be taken as a constant.
can. L_nioset to 0 and the probabilities of all lengths
at the end by replacing the value of one constant with the value of one constant
The time distribution is displayed as follows. Θm=Φm/1=qm+Θm−1Pm (23) However, "1" is one uniform replacement value,
The value of Pm is the location generated at time m for a given phoneme.
It is desirable to correspond to a replacement value for a given label.
stomach. For the above equation for Θm, the matching value is:
defined as Matching value = log_Ten(Θ₀+Θ₁+… +Θm＋log_Ten10 (24) Comparison of basic high-speed matching and alternative high-speed matching
Compared to the alternative, the number of additions and multiplications required is
By using the matching phoneme machine,
Width decreases. L_nioIf = 0, basic high speed pine
Ching must consider the probability of length
So, it required 40 multiplications and 20 additions, but
For alternative fast matching, Θm is determined iteratively.
Therefore, one time for each successive Θm
It turns out that we only need one multiplication and one addition. Figures 19 and 20 show alternative high-speed matuchin
The simplification of the calculation by the algorithm is shown in detail. Figure 19a
is the minimum length L_nioPhoneme machine 310 corresponding to =0
An example of 0 is shown below. Maximum length is uniform length distribution
Assume that it is infinite so that Figure 19b shows the sound
A trellis diagram resulting from an elementary machine 3100 is shown.
Assume that the start time after qn is outside the start time distribution
Then, if m<n, each decision of consecutive Θm
All the equations require one addition and one multiplication.
If you want to determine a later end time, use one multiplication.
All you need to do is arithmetic, no addition is necessary. Figure 20a shows the minimum length L_nioThe specific case when = 4
An embodiment of the phoneme machine 3200 is shown in FIG. 20b.
shows the corresponding trellis diagram. L_nio=4
Therefore, the trellis diagram in Fig. 20b has the symbol
yields a probability of 0 along the paths U, V, W and Z.
Ru. Θ_Fourand Θ_oIf the end time is between, multiply by 4 times
One addition is required. greater than n+4
For the end time, only one multiplication is required;
No calculation required. This example is based on the above-mentioned FPS company.
It is realized with APAL code on 190L. The desired additional state is shown in Figure 19 or 20.
It can be added to the example. F1f Matching based on first J level (first
Figure 20) Basic high-speed matching and alternative high-speed matching
In order to further improve the
Consider only the matching of the first J label of the
Do it like this. Sound Pro whose label is Sound Channel
Rate of 1 label every centisecond by Setsa
A reasonable value for J is
It is 100. In other words, the audio of the order in 1 second.
The corresponding labels are supplied to the phonemes and phoneme machines.
Confirm the matching with the incoming label. inspect
By limiting the number of labels, there are two advantages.
It will be done. The first is a reduction in decoding delay, and the second is a reduction in decoding delay.
compares the scores for short and long words.
The problem of comparison can be fully avoided. rice cake
Of course, the length of J can be changed as desired.
Ru. Effects of limiting the number of labels inspected
is observed using the trellis diagram in Figure 20b.
I can do it. Without the improvement according to the present invention,
The fast matching score is the bottom row of this drawing.
is the sum of the probabilities of Θm along (row). Sunawa
T, t=t₀(L_nio= 0) or t=t_Four(L_nio=
state S at each time starting at (case 4)_Fourprobability that
is determined as Θm, then all Θm
are summed. L_nioIf = 4, t_Fourany previous
state S at time_FourThe probability that it is is 0. to said improvement
Therefore, taking the sum of Θm ends at time J.
Ru. In Figure 20b, time J is time t_o+2corresponds to
Ru. The inspection of J labels beyond the interval up to time J is completed.
When determining the matching score by
, the sum of the following two probabilities occurs. First, before
As mentioned, along the bottom row of this trellis diagram.
There is a tsuta rho calculation. However, this calculation is performed at time J
-1. status at each time up to time J-1
S_FourThe probabilities that are are summed to give a raw score. Second
, the phoneme becomes S at time J.₀~S_Foureach state of
There is a column score corresponding to the sum of probabilities. child
The column score for is calculated as follows. Column score =_Four 〓^f=0 Pr(Sf, J) (25) The phoneme matching score is the raw score and the column score.
Obtained by summing the points and taking the logarithm of the sum.
It will be done. To continue high-speed matching of the next phoneme
is the lowest row (preferably including time J).
), the start time of the next phoneme is calculated.
Take out the cloth. Determine the matching score for each of the J consecutive phonemes.
Then, as mentioned above, the sum of all phonemes is that phoneme
is the sum of all matching scores. Basic high-speed matching and alternative high-speed matching described above
How to generate end time probabilities in the Ching example
When examining the column score, it is found that the column score is determined quickly.
It turns out that it does not easily fit the calculations. inspect
The above-mentioned high-speed match improves to limit the number of labels.
better accommodate matching and alternative matching.
Therefore, the present invention replaces column scores with additional row scores.
make it possible to change. That is, (Fig. 20
b) state S between times J and J+K_Fourthe sound that is
The raw additional low score is determined. However, K is left
is the maximum number of states in the phoneme machine of meaning. that
Therefore, if any phoneme machine has 10 states,
If the bottom of the trellis diagram is
10 end times are added along the row of the section, and the
A probability is determined for each. Until time J+K
All probabilities along the lowest row of (time J+
K) are added, and the map of a given phoneme is
Generate a tucking score. As mentioned above, consecutive
Add up the matching values of the phonemes and match the words.
get points. This example is an APAL on the FPS 190L mentioned above.
This is realized in code, but other
other components on other hardware, such as when
It can also be realized by a code. F1g phoneme tree structure and high-speed matching example
(Figure 21) Basic fast matching or alternative fast matching
(with or without a max label limit
) to determine the phoneme matching value.
The calculation time required to determine the results is significantly reduced.
Furthermore, use the words in the list obtained through high-speed matching to
Even when performing dense matching, the computational complexity is
Significant savings. Once the phoneme matching value is determined,
As shown in FIG. 21, the branches of the tree structure 4100
A comparison is made along the path to determine which path of phonemes occurs most.
Determine if it is possible. In Figure 21, (point 4
102 to branch 4104) spoken word
Phoneme matching of “the” phoneme DH and UH1
The sum of the values is for each phoneme branching from the phoneme MX.
must be significantly higher than for the sequence of
Must be. By the way, the phoneme map of the first phoneme MX
Tutting values are calculated only once and then spread out
Used for each basic format. (branch 4104 and
See 4106. ) Furthermore, the first stem of the branch
The total score calculated along the
any other sequence of lower or branching than
If you find out that the total score is lower than the total score of
All basic forms extending from the initial sequence are the same.
Sometimes they are removed from the list of candidate words. example
For example, the basic form related to branches 4108 to 4118
is determined that MX is not a likely path.
Then, they are thrown away at the same time. High-speed matching example and tree structure make it possible to
A list of candidate words in fixed order is generated, and
The computation involved is significantly saved. Regarding memory requirements, phoneme tree structure, phoneme organization,
The measured value and the tail probability are to be memorized.
There is. For tree structure, 25000 arcs and each arc
There are four data words that characterize the first
The data word represents the index of the subsequent arc or phoneme.
Was. The second data word is the subsequent data word along the branch.
Represents the number of phonemes. The third data word is a tree structure
Indicates which node the arc is placed on. The fourth data word represents the current phoneme. subordinate
So, in the case of this tree structure, the storage space is 25000 × 4
is necessary. Fast matching allows 100 different
There are ivy phonemes and 200 different ivy finemes.
A fineem is one that is produced somewhere in the phoneme.
Since we have a probability of 100×200 statistical probabilities
Storage space is required. 200 for trailing structures
×200 storage space is required. Therefore, the high speed
In the case of tsuching, the space to store 100K integers and
60K floating point storage space is sufficient.
be. F1h language model (Figure 2, Figure 22) As mentioned above, regarding the word in context (triple
Contains a language model that stores information (such as letters)
This increases the probability of correctly selecting the word.
You can The language model is described in the above paper.
has been done. Language model 1010 (Figure 2) is a unique character
It is desirable to have i.e. modified triple
law is used. According to the invention, the sample sample
Examine the text and identify triple words in a fixed order in the vocabulary
and the likelihood of each word pair and single word
Confirm. And the most promising Mie work
A list of codes and word pairs is formed. Furthermore,
Triple words not in the list of triple words, and
It is the likelihood of a word pair that is not in the list of word pairs.
Each will be determined. According to the language model, the target word is reduced to 2 words.
If it continues, this target word and the previous two words are
It is determined whether the word is in the list of triple words.
Ru. If it is in the list of triple words, the triple word
Specifies the remembered probability assigned to the
be done. The target word and the two preceding words are triple words.
If it's not in the list, add the target word and
Are adjacent preceding words in the list of word pairs?
judge whether in the list of word pairs.
, the probability of the word pair and the triple word
Multiply the probability that there is no triple word in the list of words, and
Assign the product to the target word. Before including target word
Each triple word and word pair are triple words.
not in the list of words and the list of word pairs.
is the probability of only the target word, and the triple word
The probability that is not in the list of triple words, as well as the word
Multiply the probability that the do pair is not in the list of word pairs and
Assign the product to the target word. Figure 22 shows high-speed matching and precision matching.
Select words from the vocabulary using language modeling and language models.
5000 shows a flowchart 5000 for selecting. block 5002
, define the words in the vocabulary and block 500.
4, each word is the basic phonetic symbol of the Markov model.
Display by form. Block 5006, basic
The phonemes of the morphology are arranged in a tree structure (as shown in Figure 21).
Arranged. Markov models for various words
is block 5008, and the sample text
Format by reading (described later). each marco
The probabilities associated with the basic word forms of the word model (e.g.
For example, transition probabilities and labels at various transitions
probability) is used for precision matching (to be performed later).
It will be held for a long time. Given a Markov model phoneme
The probability of a given label in block 5 is
010, and the replacement value is overestimated as much as possible.
Replacement, block 5012, basic high speed mating
to be able to execute the log. sound professional
The label string from Setusa (described above) is
Approximate matching with lock 5014 (not shown)
provides input to block 5012. block
5016 (not shown) performs basic high-speed matching.
If you are trying to improve the distribution of uniform lengths,
Similar values are used in block 5018. Block 5
020, if you want to further improve it, block 50
22 to limit matching to the first J label.
Ru. After the desired approximate fast matching, the probabilistic order of candidates is
The list of complementary words is stored in block 5024 (not shown).
) to feed the language model. Next, Blotsu
5026 (not shown) performs approximate matching and
and the language model as determined by the language model.
Perform precision matching of promising words
Ru. Precision match of block 5026 (not shown)
The words from the block 5028 (not shown)
) to the language model and block 5030
(not shown) to select the most likely word.
can be done. Phonetics of the F2 word and Marko of Finim
Construction of the basic form of the f-model F2a Construction of basic phonetic forms (Figure 4, Table 1,
2 table) Matrix that can be used to form the basic form
One type of Lukov model phoneme machine is based on phonetic symbols.
It is something that can be developed. That is, each of the phoneme machines is
corresponds to the unit of a given phonetic symbol. For a given word, each corresponds to
A single phone sheet of phonemes with each phoneme machine
There is a kensu. Each phoneme machine has several states and
and transitions between them, among them:
Something that can also generate finem output
Yes, there are some that cannot (called null transitions).
As mentioned above, the statistics for each phoneme machine are (a) the probability of a given transition occurring, and (b) A given transition produces a particular finem.
Includes the probability that At each non-null transition, each fee
It is desirable to have a probability associated with neem.
For the Finim Alphabet shown in Table 1.
There are about 200 finim. Figure 4 shows the sound
A phoneme machine used to form the basic form of a mark
shows. The sequence of such a phoneme machine is
given word by word. generate known word
During the shaping phase, the statistical values, i.e. the probabilities
enters the phoneme machine. Phoneme machine for various phonetic symbols
The transition probability and Finim probability in are
uttered a single sound of a known phonetic symbol at least once
Write the fineme string generated when
including, well-known forward/backward al.
determined during shaping by applying the
It will be done. Statistics for one phoneme identified as phoneme DH
A sample of is shown in Table 2. As a rough estimate
Therefore, the transitions tr1, tr2 and
Label output probability distribution of tr8, transitions tr3, tr4, tr
5 and tr9 label output probability distributions and transitions
The label output probability distribution of tr6, tr7 and tr10 is
Each is represented by a single distribution. child
This is determined by the arc (i.e. transition) of each column in Table 2.
indicated by the assignment of labels 4, 5 or 6 to
has been done. Table 2 shows the beginning, center, or middle of the phoneme DH.
or the probability of each transition generated respectively at the end.
and the probability of the label (i.e., finem).
vinegar. For DH phonemes, for example, the state S₁from state S₂
The probability of transitioning to state S is calculated as 0.07243, and the probability of transitioning to state S₁mosquito
state S_FourThe probability of transitioning to is 0.92757. (what
Then, there are two possible transitions from the initial state.
Therefore, the sum of both probabilities is equal to 1. )
Regarding the label output probability, the DH phoneme is
The last part of the element, i.e. the column labeled 6 in Table 2
The probability of generating Fenenu AE13 (see Table 1) is
It has a rate of 0.091. Table 2 also includes each node.
The counts associated with the mode (i.e. state) are shown.
ing. The node count is determined by the number of its phonemes during shaping.
represents the number of times that the corresponding state has occurred. Table 2
Such statistics exist for each phoneme machine.
Ru. The phoneme machine of the phonetic alphabet is a sequence of word basic forms.
Sequencing is generally performed by phoneticians.
and usually not automatically. The basic form of phonetic symbols is precision mating and high-speed roughening.
It has been used with some success in computational acoustic matching.
There is. However, the basic form of phonetic symbols is
Depends on the decision and is not automatic, so it may not be accurate
Sometimes there is. F2b Construction of basic form of Finim (Fig. 23,
Figure 24) An alternative to the basic form of phonetic symbols is the basic form of finem.
It is. Each phoneme machine corresponds to a single note in the phonetic symbol.
Instead of each phoneme machine in the basic form,
Each fee in Neem Alphabet
Corresponds to neem. Easy feeneem phoneme improvement
has two states S₁and S₂as well as multiple transitions
(each with its own probability) containing (the second
(See Figure 3). A transition is a null between two states.
Unable to generate finem in transition.
stomach. The first non-null transition between two states is then
The probability of each generated finem is
have The second non-null transition is state S₁Self-rule in
each fineme is generated there.
Has probability. Therefore, the simple feeneem phoneme
The machine will play 0, 1, or 2 or more finems.
can be generated. Figure 24 is Finim
Figure 3 shows a trellis diagram based on phonemes. Markov model phoneme machining in finem units
When constructing a phoneme, a phoneme machine based on multiple utterances is used.
It is preferable to use the The phoneme machine is
given by forming it on the basis of number utterances.
It is possible to capture changes in the pronunciation of words. child
One way to capture changes in the pronunciation of
Disclosed in Application No. 738933 (filed May 29, 1985)
It is. Build the basic form of the word by said application
When selecting word segments in a vocabulary, follow the method below.
(e.g. each word or given syllable)
or part of it)
To construct. This method is explained in detail below.
Contains steps. (a) Each of the multiple utterances of a word segment
Convert to each fineme string.
Ru. (b) Finim's Markov model phoneme machine
form a set of (c) Generate multiple fineme strings
Best single phoneme machine P₁Determine. (d) Generate multiple fineme strings
Format P₁P₂or P₂P₁The best two-phoneme basic form of
Determine. (e) The best two for each finem string.
Align basic phoneme forms. (f) Each finem string has a left part and a right part.
The left part is the first part of the basic form of the two phonemes.
Corresponding to the phoneme machine, the right part is the two-phoneme basic form.
correspond to the second phoneme machine in the state. (g) identify each left part as a left substring;
and identify each right part as a right substring.
Separate. (h) Set the left substring to multiple utterances.
Corresponding set of finem strings and
Process in the same way, and also
If the basic form of a single phoneme is divided into more than
higher subs than in the case of the good two-phoneme basic form.
Prohibited if there is a probability of generating a tring
do. (i) Set the right substring to multiple utterances.
Corresponding set of finem strings and
Process in the same way, and also
If the basic form of a single phoneme is divided into more than
higher subs than in the case of the good two-phoneme basic form.
Prohibited if there is a probability of generating a tring
do. (j) A single phoneme that is not segmented is divided into the corresponding fini
in an order that corresponds to the order of the system substrings.
Link. In certain embodiments, the method further includes the steps below.
Including steps. (k) The connected basic forms are
aligned and concatenated bases for each of the
For each phoneme in the morphology, each corresponding fee
Common substrings in neem strings
The substring corresponding to a given phoneme that is a set of
Identify the ring. (l) For each set of common substrings,
has the highest joint probability of producing a bust string
Determine the phoneme machine to use. (m) For each common substring, the concatenated bases
The phonemes in the morphology are divided into the highest joint probabilities determined.
Replace with phoneme. The basic form obtained by replacing phonemes is improved.
This is the basic form. Steps (k) to (m) above
If it iterates until no elements are replaced, further modifications can be made.
A refined basic form is obtained. a given word segment (e.g. word)
There are other ways to construct the Finim basic form of
Ru. For example, the basic form is
output of the sound processor in response to a single utterance of the
It can be taken out. How to build the Finim basic form
Generally, the phonetic base of the same word, regardless of
It has the advantage of being more precise and accurate than form.
It is characterized by Furthermore, the basic form of phonetic symbols is
Usually constructed manually by phoneticians,
Finim basic forms are generally constructed automatically.
Ru. However, the basic form of Finim is generally
It is 10 times the length of the basic phonetic form, which is quite large in terms of calculations.
This is a disadvantage. In its most basic form, the invention
Work from the Finim basic form built in
basic form of composition of segment (e.g. word)
related to the formation of Regarding the basic form of Finim, accuracy and basic
While maintaining the ability to automatically construct this form,
The problem of shortening the length of the basic forms of phonetic symbols is an important one.
It's getting old. The present invention addresses this problem and
Simple phonetic Markov symbols representing words in the vocabulary
Enables automatic generation of basic form of model phoneme machine
Ru. The basic form of phonetic type is useful in speech recognition, e.g.
For example, the fast approximate matching procedure described above as well as the fine
Can be easily used in dense matching procedures
Ru. F3 Alphabet of composite pheneem phoneme
Generation (Figure 4) When extracting the alpha alphabet of a synthesized phoneme,
Sorted using word backward algorithm
There is a predetermined set of shaped N-fee-neem phonemes.
Assume that N-Fineam phoneme set
is given, the alpha alphabet of the synthesized phoneme is
It is retrieved by executing the steps below. (a) Select a phoneme pair from the current set of phonemes.
Ru. (b) Determine the entropy for each phoneme in the selected pair.
Set. The entropy for each phoneme is defined as
be justified. Entropy = −〓pilog₂pi Take the sum of all finim (i=1200).
Each pi produces a particular phoneme, the fineem i.
There are 200 different finems with a probability of
be. (c) Merging two phonemes of the selected pair into a single phoneme.
The synthesized phoneme (or the corresponding
Determine the entropy of the cluster. synthesis
Each output probability associated with a phoneme has two paired
is the average output probability of the phoneme. (d) The synthesized phoneme and each phoneme merged by the following formula
Phonemes and phis based on entropy measurements of
Determine the loss of common information in neem. Loss = E synthesis - (E phoneme 1 + E phoneme 2)/2 As the loss of common information increases,
The influence of a phoneme on the two phonemes of the pair becomes smaller. (e) Steps (a) to (d) for each pair of finem phonemes.
Repeat. (f) Minimum of a given selected pair of merged phonemes.
Select synthesized phonemes representing the loss of the merged
Replace two phonemes with their composite phoneme and
The number of phonemes being reset is decreased by 1. (g) Step (a) until the number of phonemes in the set is n.
-Repeat (f). Each remaining phoneme is a composite phoneme
It is. The typical 200 phonemes in the Fhineem phoneme are:
Approximately 50 unique compositions in the synthesized phoneme alphabet
It is desirable to reduce it to phonemes. Each synthesized phoneme has seven shapes as shown in Figure 4.
state, expressed by a Markov model of 13 transitions.
This is desirable. Each transition is
It has a transition probability corresponding to the likelihood. each non-null
A transition (tr1 to tr10) has multiple outputs related to it.
each output probability is a specific non-null transition.
represents the likelihood of a given finem generated by
It is desirable to The set 50 element synthesized phonemes are shown in section F4 below.
used to construct a simple basic form, as shown.
It is desirable that Synthetic phonemes in the alphabet
The number and composition of synthesized phonemes are based on known choices in phonetic technology.
Similar to selected phonemes and alphabets
You can choose as follows. F4 Word base of short phonetic Markov model
Automatic formation of this form (Fig. 1A, Fig. 1B, Fig. 2)
(Fig. 1, Fig. 23, Fig. 25 to Fig. 30) According to the invention, a simple Markov model basic
The form is formed as follows. First, the word
There is a basic form of Finim built for each,
Each basic form is formed according to one of the methods described above.
Assume that The basic form of Finim is
It is desirable to construct a number from vocalizations and store it in memory.
Delicious. In general, the finem basic form of the word
is on the order of 60 to 100 finemes in length. In Figure 25, the target word is f-fe-neem.
displayed by strings, each string having its
Corresponds to each utterance of the word. Block 1
Each of ~9 (hereinafter omitted) is processed by a sound processor.
Display the generated finem (i.e. label)
Was. In this example, the label is
One file is generated every centisecond. each string
The finem in the game is probably the pronunciation of the target word.
It will be different for each. In accordance with the present invention, the finem base of the target word is
It is assumed that this form is known. Figure 26 is
Shows the basic form of Finim. Compatible with target word
A sequence of feeneem phonemes (feenie
(Basic form) is preformed. Hui
The basic form of the neem, as mentioned above, has multiple vocalizations.
By the method of constructing the basic form of Finim,
It is desirable to take it out. f utterance of the target word
For each, the finem term
Eneem String FS₁~FS_fThere is. Hui
Neem is an alpha abet of 200 fine neem,
That is, it is selected from the set. Fhineem sound
When using the simple phoneme model shown in Figure 23,
Generate 0, 1, or 2 or more finemes in
can be played, but generally one feeneem sound
Each element generates one finem on average. Attachment
Each unique file indicated by a letter P
The basic form of fe-neem based on the neem phoneme is the second
It is shown in Figure 6. According to the present invention, each synthesized phoneme is
At least one of the N phonemes in the alphabet
, each finem phoneme is one composite phoneme.
corresponds only to Synthesis in a set of synthesized phonemes
The number of phonemes n is the number of phonemes in the set of fineme phonemes.
The number of phonemes is less than N. new al
Alphabet is considerably more than the original Alphabet.
Fewer elements, e.g.
40 or 50 elements compared to the 200 element order of
It is desirable to have the element. Alphabet's sound
One way to determine the prime is briefly shown in the previous section.
There is. Alphabets of n-synthetic phonemes, words in vocabulary
Basic form of finem and target word for each
Assuming multiple utterances of the target word, a simple
The basic form is formed by the following method. first
, each utterance is expressed as a feeney as shown in Figure 25.
Mu String FS₁~FS_fDisplayed by each of
be done. Then each finem string is
The basic form of the target word in Figure 26
aligned against. The alignment results are shown in Figure 27.
vinegar. In Figure 27, each string shows
The basic form of Finim is shown aligned to
It is. This alignment is IEEE Bulletin Volume 64, 1976
F. Zienerik, April issue, pp. 532-556.
Paper “Continuous speech recognition using statistical methods” (F.
Jelinek, “Continuous Speech Recognition by
Statistical Methods, “Proceedings of the
IEEE, Vol. 64, April 1976, pp 532-556).
Viterbi was discussed in various papers.
It is desirable to obtain this by performing an alignment method. The subscript of each phoneme in Figure 27 is
Identify the phonemes among the 200 phonemes of Tsuto. each
A phoneme string is a sequence of the same phonemes, i.e.
WachiP₁P₁P₃P_FourP₂...P_FourIt is displayed by.
However, the f-th string
The alignment to the finemes in the
There is. For a typical word lasting about 1 second, about 100
There are about 100 fineem phonemes.
Ru. Figure 28 shows how to place the feeneem phoneme into a synthesized phoneme.
The results are shown below. Each phoneme in Figure 27 corresponds to
Synthetic phonemes (each of which has the symbol CP_i) with
is being given. Is subscript i in the set of synthesized phonemes?
Identify synthesized phonemes. Phoneme P₁and P₂is CP_Ten
, the phoneme P₃is CP₁₁, the phoneme P_Fouris CP₁₃replaced with
It is. Number of synthesized phonemes for each string in Figure 28
is the string FS in Figure 27₁~FS_feach sound of
Equal to a prime number. Each of the synthesized phonemes is structurally a sound of the aforementioned phonetic type.
7 states with 13 transitions, similar to
This is desirable. Next, in Figure 28, a simple 2
The finem phoneme of the state is placed in the synthesized phoneme of the 7 states.
Can be replaced. Next, in Figure 29, each fineme strike
The same two synthesized sounds aligned to the ring
One element (i.e., the leftmost synthesized phoneme and the synthesized phoneme to its right)
is replaced by a synthesized phoneme. In detail, (part
In figure 28) CP_TenTwo adjacent neighbors identified as
The synthesized phoneme is one synthesized phoneme CP_Tenreplaced by
Ru. Similarly, in Figure 30, two adjacent tone components
The phoneme is again replaced by one synthesized phoneme. sand
That is, in Figure 30, the adjacent composition (of Figure 29)
Phoneme CP₁₁and C.P.₁₃is one synthesized phoneme CP₈placed in
It has been replaced. From Figures 29 and 30, each series
The number of synthesized phonemes increases by one by combining the synthesized phonemes.
It can be seen that it decreases. As explained later, two
By continuing the process of merging the synthesized phonemes of
It is possible to obtain a basic form of a specific length. The process of iteratively merging two synthesized phonemes is
It is based on proximity. synthetic phoneme
For a given sequence of synthesized phoneme pairs,
examine each resulting from the combination of its two phonemes.
Determine adverse effects. Adverse effects of merger
The phoneme pair with the smallest result is the closest one.
and will be considered as a candidate for the next merger. Measuring the adverse effects of mergers
Although there are various methods for determining the
It is preferable to define it as MAX (a/A, b/B).
Yes. A and B are the calculated lengths of the basic form
is a selectable limit value that limits . a and b
is defined as below. Display pairs of adjacent synthesized phonemes in their sequence
Two synthesized phonemes CP_nand C.P._oIf that
The phis aligned for the two synthesized phonemes
Continuous F of neem_nand F_oThere is. synthetic phoneme
C.P._nois, Finim F_noThe joint probability of generating
Defined as the maximizing synthesized phoneme. F_noteeth,
Continuation F of two adjacent finems_nand F_o
It is a concatenation of Synthetic phoneme CP_noare two adjacent
Synthetic phoneme CP of_nand C.P._ois the closest,
If scheduled to be merged, CP_nand
C.P._oIt is a phoneme that replaces . L_n,L_oand L_no
is a synthesized phoneme CP_n, C.P._oand C.P._nogives F_n,
F_oand F_norepresents the log probability of each. Person in charge
The number a is obtained by the following formula. a=L_n+L_o−L_no Therefore, a is a synthesized phoneme CP_nand C.P._omerger of
This is a measure of the decrease in probability due to composition of two
As the adverse effect of phoneme merging increases, the
The value also increases. Next, b is a “paired t-test”.
It is determined by a method known as vs. T-inspection
A common method of research is to examine various texts, e.g.
Published by Duxbari, Massachusetts, W.
“Introduction to Probability and Statistics” by Mendenhall, Vol.
5th edition, 1979, pp. 294-297 (W. Mendenhall,
“Lntroduction to Probability and Statistics”.
5th Edition, Duxbury Press, Massachusetts,
1979, pp. 294-297). vs. T-inspection
According to the survey, T.₁is a continuous F of finem_nin
The finem () is a synthesized phoneme CP_nand () combination
Phoneme CP_noIf it is produced by a comparison of
A pair-t test is performed on the difference in the log probability of the finem between
is the t-statistic obtained by performing the
Ru. Similarly, T₂is a continuous F of finem_oin
The finem () is a synthesized phoneme CP_oand () combination
Phoneme CP_noIf it is produced by a comparison of
These are the Finim's t-statistics. For example,
Enim continuous F_oFor each finem in the
Synthetic phoneme CP that generates that finem_ocertainty
rate, and the synthesized phoneme that generates its finem
C.P._noThere is a probability of between these two probabilities
Comparisons can be made pairwise. This process
Iterated for each series of enims, that
There is a separate sequence associated with each word utterance.
I understand. Z₁and Z₂is T₁and T₂The t-distribution of N(0,
1) After converting to distribution, T₁and T₂taken by
represents the value. Next, b is obtained by the following formula
Ru. b=MAX(Z₁,Z₂) b is a synthesized phoneme CP_nand C.P._oCP by merging_no
, the difference in the finem probability is
Indicates how significant (statistically) it is. Maximum of a/A and b/B for each pair of synthesized phonemes
By taking the value, the adverse effects of the merger are minimized.
The pair can be determined. Then this closest
It is possible to merge pairs and replace them with one synthesized phoneme.
Wear. Examination of all synthesized phoneme pairs as well as the two most
The merger of adjacent synthesized phonemes that are close together is the basic form of
the number of synthesized phonemes reaches a specified limit, or
The distance between the closest phoneme pair is greater than 1
(In either case, a or b or both are each
continues until the limit value is exceeded). Synthetic phonemes reflect the alphabet of the phonetic alphabet.
is selected, the resulting synthesized phoneme
The sequence is a simple sequence that is acceptable for speech recognition.
Pronunciation of words, not just the basic form
It also provides automatically generated descriptions. The latter result is phonetic
It can also be used in the field of Figures 1A and 1B are the basics of the method of the invention.
8000 shows a typical flowchart 8000. Follow this flowchart
repeat the target word once or, if possible, multiple times.
utter over and over. Block 8002, respectively.
This vocalization produces a fineme string.
to be accomplished. In block 8004, the generated target word
based on the string(s) of
Best to generate string(s)
Finim word basic with (simultaneous) probability
Select form. Basic form of each word in the vocabulary
is preformed in block 8006.
Ru. Next, in block 8008, each of the target words is
Strings are aligned to the selected base form
be done. In this way, each fin in the basic form
The phoneme pairs in each string (of the target word)
aligned to the corresponding finem. then
In block 8009, each fineme phoneme corresponds to
is replaced with a synthesized phoneme. Next, follow the steps below.
Execute. (a) Selecting pairs of adjacent synthesized phonemes (block
8010). (b) For each fineme string, the selected
Feeney to align against adjacent phoneme pairs
Determine the system substring (block 80)
12). (c) less than the alphabet of the finem phoneme.
of a synthesized phoneme formed to have no elements
From Alphabet (block 8013),
each of the fineme strings
The best match that produces the determined substring
Determine synthesized phonemes with time probabilities (block
8014). (d) By block 8016, adjacent synthesized phonemes are
Reverse steps (a) to (c) above for all pairs of
Revenge. Single synthesis for each pair of adjacent synthesized phonemes
Determine the phoneme. (e) Each phoneme pair in the basic form is determined instead.
The negative effects of replacing it with a single synthesized phoneme
Measure (block 8020). (f) the minimum adverse effect does not exceed the prescribed limit;
(block 8024), resulting in minimal adverse effects.
Replace a synthesized phoneme and its corresponding phoneme pair
(Block 8022). (g) Replaced synthesized phonemes and their corresponding phoneme pairs.
After that, a new phoneme basic form is provided (block
8026). (h) If the number of phonemes is larger than a predetermined desired number (block
8028) and repeat steps (a) to (g).
Search for a new basic form. Step (h) shortens the basic form to the desired length.
(block 8030).
or by replacing phoneme pairs with synthetic phonemes.
The negative impact exceeds the limit value (block 8024)
Repeat until. This step continues until all words are processed.
and iterate through each word in the vocabulary (block 8).
040). Therefore, for each word in the vocabulary, the desired
There is a basic form of a synthesized phoneme that is shortened to the length of
or shorten without exceeding adverse impact thresholds.
There is a basic form that cannot be done. The basic form shortened by this procedure is a phonetic type.
In basic form, precise acoustic matching and (also
) Perform fast approximate acoustic matching as described above
It can be used when That is, the second
The tree structure 4100 in Figure 1 (various phoneme sequences
represents a phoneme stored in memory) is
The present invention may include shortened basic forms.
Wear. In this case, each phoneme includes a synthesized phoneme. Therefore, the basic form of each word in FIG.
A sequence of synthesized phonemes in a shortened basic form.
each synthesized phoneme in the sequence is represented by the arrival
Matching against labels or finems
Displayed by the phoneme machine used when executing
That is planned. F5 comparison result Speech recognition systems using various types of basic forms
The results of the performance tests are shown below. First, a vocabulary of 62 keyboard characters and their
For systems with arbitrary 600 utterances, manual
The number of errors in the basic form of the phonetic alphabet determined by is 28.
It was hot. Finim base constructed after a single utterance
The number of errors in this embodiment was 14. And several times
Based on the basic form of Finim constructed from the vocalizations of
The number of errors encountered was 5. Second, 2000 common words for business correspondence.
Vocabulary and has a manuscript of 2070 arbitrary utterances, usually
A system using the basic form of the phonetic symbol of
Using the basic form of Finim constructed from voices.
108 errors compared to 42 resulting errors.
Ra occurred. Finally, the “shortened” basics explained in Section F6
By using the morphology, the following results were obtained.
Vocabulary of 2000 typical office words and 2070
If you have a manuscript of any word, use the quick overview mentioned above.
Through arithmetic acoustic matching, the basic form of the phonetic symbol is
Errors related to 8 were shortened.
The basic form has 2 errors associated with it.
Ivy. From these results, we can develop a filter based on multiple vocalizations.
The basic form of neem was generated from a single utterance.
Basic forms of standard phonetic symbols and basic finemes
It turns out to be more accurate than the form. Also, shortened
The basic form of
It can be seen that it has been significantly improved. Furthermore, these
From the results, a good practice is based on multiple utterances.
This is a shortened basic form. F6 Alternative Example For example, the set of synthesized phonemes is different from the one mentioned above.
It can be determined by different methods. Also,
Approach measurements like MAX(a/A, b/B) are
a/A or b/B or produce comparable results.
can be reshaped as other scales. Furthermore, the present invention constructs a shortened basic form.
The method is based on multiple utterances of words,
Uses a fineme string that corresponds to each utterance.
It is preferable to treat it as soon as possible. However, the book
The invention provides shortened bases from a single utterance of the target word.
It can also be used to construct this form. single
Based on one utterance, one substring
The composition that yields the highest probability of producing a finem
Phonemes are considered with replacement of adjacent phoneme pairs. 1st
Since there is only a ring, there are no joint probabilities to consider.
stomach. From changes in pronunciation, shortening based on multiple utterances
It is accurate and good due to its basic form.

【table】

【table】

[Table] G Effects of the Invention According to the present invention, a given word can be converted from a word basic form consisting of finemone phonemes to a shortened word basic form consisting of synthesized phonemes.

[Brief explanation of drawings]

1A and 1B are flowcharts showing the method of the present invention, FIG. 1C is a diagram showing the arrangement relationship between FIGS. 1A and 1B, and FIG. 2 is a schematic block diagram of a system environment in which the present invention can be implemented. , FIG. 3 is a detailed block diagram of the stack decoder in the system environment of FIG. 2, and FIG. 4 is a detailed block diagram of the stack decoder in the system environment of FIG. Figure 5 is a diagram showing the phoneme machine, Figure 5 is a diagram showing successive stack decoding steps, Figure 6 is a diagram showing the likelihood vector and likelihood envelope of each word path, and Figure 7 is a diagram showing the stack decoding procedure. Flow chart, Figure 8 shows the elements of the acoustic processor, Figure 9 shows the parts of a typical human ear representing the locations forming the components of the acoustic model, and Figure 10 shows the parts of the acoustic processor. The block diagram shown in Fig. 11 is used for designing an audio processor.
Figure 12 is a diagram showing the relationship between sound intensity and frequency; Figure 12 is a diagram showing the relationship between horns and horns; Figure 13 is a diagram showing how the acoustic characteristics of Figure 8 are expressed by the acoustic processor; Figure 13 shows how the limits are updated; Figure 15 shows the trellis or lattice of the precision matching procedure; and Figure 16 shows the phoneme machine used to perform the matching. , Figure 17 is a time distribution diagram used in a matching procedure with specific conditions, Figure 18 is a time distribution diagram used in a matching procedure with specific conditions.
Figures a to e illustrate the interrelationships between phonemes, label strings, and start and end times determined by the matching procedure; Figures a and b show a specific phoneme machine with a minimum length of 0; Figures 20a and 20b show a specific phoneme machine with a minimum length of 4 and the corresponding trellis, Figure 21 allows processing of multiple words at the same time. Figure 22 is a diagram showing the tree structure of phonemes;
Figure 3 is a diagram showing the Finim phoneme machine, No. 24.
is a trellis diagram of a plurality of sequential feeneem phoneme machines, FIG. 25 is a diagram showing a plurality of feeneem strings generated in response to corresponding utterances of a target word, and FIG.
FIG. 27 shows the finemme base form (containing a sequence of finem phonemes) with the highest joint probability of producing a string; FIG. The figure shows the replacement of each fineme phoneme and its corresponding synthesized phoneme, and Figure 29 is a new base in which the common pairs of adjacent synthesized phonemes associated with each string shown in Figure 28 are replaced with the same single synthesized phoneme. A diagram showing the format, FIG. 30, is a diagram showing a new basic format in which the common pairs of adjacent synthesized phonemes associated with each string shown in FIG. 29 are replaced with the same single synthesized phoneme. 1000...Voice recognition system, 1002...
Stack decoder, 1004...Acoustic processor, 1006, 1008...Array processor, 1010...Language model, 1012...Workstation, 1020...Search device, 102
2,1024,1026,1028...interface.

Claims (1)

[Claims] 1. A first word Markov model formed by concatenating first word partial Markov models each corresponding to a set of first phonemes each representing an acoustic type that can be assigned to a minute time interval. , respectively for each word in the vocabulary, each of which corresponds to one or more of the first phonemes; A set of second word partial Markov models with complex transitions are prepared corresponding to each of the set of second phonemes, and the second word partial Markov model is prepared based on the first word Markov model.
The second model formed by concatenating word partial Markov models
A method for constructing a word Markov model, the method comprising: (a) generating a string of first phonemes in response to the utterance of a target word; and (b) constructing a first word Markov model of the target word according to the generation. (c) replacing each of the first word partial Markov models of the first word Markov model with a corresponding second word partial Markov model; forming a string of word partial Markov models; (d) forming a string of adjacent second word partial Markov models in the string of second word partial Markov models;
selecting a pair of models; (e) determining a substring of the first phoneme that is aligned to the selected pair of adjacent second word partial Markov models; and (f) second word partial Markov models. (g) select the second word partial Markov model that produces the determined substring with the highest probability; (h ) Repeat steps (d) to (g) for each pair of adjacent second word partial Markov models in the string of second word partial Markov models, and (i) align the second word partial Markov models to the determined substring; (j) measuring the adverse effect of replacing each pair of adjacent second word partial Markov models with a correspondingly selected second word partial Markov model according to a predetermined criterion; synthesizing a second word Markov model by replacing at least one adjacent pair of second word partial Markov models with a correspondingly selected second word partial Markov model based on the adverse effect; Characteristic Ward Markov model construction method.