JPH0372996B2 - - Google Patents

Description

[Detailed description of the invention]

The present invention will be explained in the following order. A. Industrial application field B. Conventional technology C. Problem that the invention aims to solve D. Means to solve the problem E Examples E-1 Voice recognition system E-1A Configuration overview (Figures 1 and 2) E-1B Auditory model and its implementation (Figure 4) E-1C Detailed verification (Figure 3, Figure 11) E-1D Basic high-speed verification (Figure 12) E-1E Another high-speed verification (Figures 15 and 16) E-1F Matching based on first J label (first
Figure 6) E-1G Tree structure and high-speed matching of single notes (No. 17)
figure) E-1H language model E-1J stack decoder (Figure 21, 2
Figure 2) E-1K Construction of basic phonetic format (Figure 3) E-1L Construction of basic phonemic forms (Figure 23) E-2 Determination from vocabulary by polling
Selecting new words (Figures 25-29) F Effect of invention G table A. Industrial application field This invention relates broadly to speech recognition technology, and in particular to simple speech recognition technology.
A short list of likely words selected from the vocabulary of
It concerns techniques for forming strikes. B. Conventional technology In probabilistic processing methods for speech recognition
The audio waveform is first labeled by the audio processing device.
or converted into a string of phonemes. each type of sound
Those labels that represent
Alphabets consists of about 200 different labels.
selected from the list. Regarding the generation of such labels
has been described in various papers such as those listed below.
It's being ignored. Proceedings of the IEEE, 64
vol., pp. 532-556 (1976).
Continuous Speech Recognition
The paper entitled “by Statistical Methods”. When adopting labels for speech recognition
Markov model sound machine (probabilistic finite state machine)
There has been some discussion about
Ru. Markov models typically have multiple states and
have transitions between these states. In addition,
An ordinary Markov model has (a) the probability of each transition occurring;
and form each label at various transitions.
Probability values for individual probabilities are not assigned.
Ru. Note that the Markov model can be used for pattern analysis and
Proceedings of the IEEE on Machine Intelligence (IEEE
Transaction on Pattern Analysis and
Machine Intelligence), Volume PAMI-5, No. 2,
L.R. Bahl, F. Zierline, March 1983
Jelinek, and R.L. Mercer
“Maximum Likelihood Method for Continuous Speech Recognition (A
Maximum Likelihood Approach to
Continuous Speech Recognition)”
It is stated in the text etc. In speech recognition, the speech processing device
Which word in the vocabulary given the bell sequence
to determine which is the most likely
processing) is executed. Regarding the applicant's application filed on November 19, 1984
As shown in U.S. Patent Application No. 672,974,
Phonetic matching is performed using (a) a sequence of Markov model sound machines;
(b) Characterize each word in the vocabulary by
A sound machine that produces a sequence of labels generated by
The individual probability of each column representing a word in
Furthermore, the sound machine word is made by determining
Each column representing the word basic form (word
baseform). As described in the above-mentioned U.S. patent application,
The basic word form is made up of several phonetic sound machines.
can do. In this column, each sound machine is
Preferably corresponds to phonetic sounds, with 7 states and 13 states.
It has a transition. Alternatively, each word base form is a phonemic sound machine.
It may also be formed as a row of rows. phonemic sound machine
is a Markov model that is simpler than a tonal sound machine.
preferably composed of two states.
It will be done. And between the first state and the second state
is empty such that no label can be generated.
There is a transition (null transition). Also, the first condition
Between the state and the second state, there is an alphanumeric
A non-empty transition such that one label can be generated from
There is also a shift. In the first state, the label forms
becomes a self-loop that can be And at the training stage
Meanwhile, statistics for each phonetic sound machine are determined.
Ru. That is, for each phoneme, the probability of each transition is
and the probability that each label is generated at each non-empty transition.
The rate is calculated from the known pronunciations. vocabulary basics
The form is created by connecting phonemic sound machines.
It is formed. A system that adopts phonemic basic forms and phonetic basic forms.
In the system, a sound is generated in response to an unknown audio input.
Corresponds to the sequence of labels generated by the voice processor
Find the most likely word (or string of words) to
The final goal is to produce. achieve this goal
One way to do this is described in the above-mentioned US patent application.
It is being Especially in that way, the number of words
However, from the total number of words in the vocabulary, we select the most likely candidates.
is first reduced to a list of words, and then this list
The searched words can then be processed using a more detailed matching procedure or language model.
be considered in the Dell procedure, where preferably the most
Most likely words are selected. number of candidate words
In reducing the
According to the method, an approximation is applied to the sound machine,
This allows high-speed processing without requiring excessive calculations.
Reason is brought to you. When achieving word reduction,
Approximate phonetic matching using calculations based on matching grids
is executed. This approximate phonetic matching
Words are processed based on more detailed matching processing and language models.
Useful for deciding whether to apply further processing.
is found to be sufficient. C. Problem that the invention aims to solve The purpose of this invention is to
Which part of the vocabulary corresponds to the sequence of phonetic labels
To determine whether a word has a relatively high likelihood,
A fast and easy-to-calculate method and how to perform it.
The objective is to provide equipment for D. Means to solve the problem The invention should be examined in detail
A non-traditional way to reduce word count
Teach. That is, the present invention
Each label in the list “votes” for each word in the vocabulary.
Polling such that a table is set up
(polling) method. this vote
is the fact that a given word produced a given label.
Reflects likelihood (certainty). The value of that vote is
The probability of the label output and the probability obtained during the training session
is calculated from the calculated transition probability statistics. According to one embodiment of the invention, the label string is
The target word is
selected. From the voting table, each row in that column
A bell is recognized and each label corresponds to the desired word
votes will be determined. and for the target word
All votes for a label are accumulated and combined, making sure
A score of uniqueness is given. for each word in the vocabulary
By repeating this process, each word is
You will get a score for the certainty that you will. It seems certain
The list of complementary words can be obtained from the likelihood score.
can. In the second example, each label corresponds to each word in the vocabulary.
There is also a second table containing penalties for
It is formed. Penas assigned to a given label
lutei for words that do not generate that given label.
Represents certainty. In the second embodiment, the label
calculates the likelihood score for a given word based on the
Both label candidates and penalties are considered when determining
considered. To account for length, the likelihood score is preferably
In calculating the certainty score of a word,
The conversion is based on the number of labels considered. Furthermore, along the generated label of a certain word,
If the end point of the process is not determined, the present invention:
The target word has multiple successive likelihood scores
Likelihood scores can be calculated over time
Specifies what should be calculated in the interval. The present invention further
for the target word, preferably in the vocabulary.
compared to the certainty scores of all other words.
Specifies that the highest likelihood score is given. According to the invention, each word has at least one
represented by a string of modal finite-state sound machines,
and the audio processing device generates an audio label in response to the audio input.
Confirmation from the vocabulary of words, such as generating bells
Teach students how to choose the correct word. That person
(a) Each label in the alphabet is
Vote on words and label each label for a given word
where the votes of generate the label that gives that vote.
The first table represents the likelihood of a given word.
It has a step of forming. Moreover, the method is
(b) put a pen on each label for each word in the vocabulary;
given label of a given word
The penalty assigned to that given word is
Likelihood of a given label not generated by the model
Forming a second table representing
and (c) for a given label sequence,
votes for all labels in the column for that particular unit.
Penalty for all labels not in column for word
to determine the likelihood of a specific word.
It has a stage of Furthermore, the above method preferably provides for each word
to give a score of certainty.
Repeating steps (a), (b) and (c) above for each word
It also has If desired, the method described above can be
Can be used in combination with Application No. 672974.
Ru. E Example E-1 Voice recognition system E-1A Configuration overview FIG. 1 shows an overview of the speech recognition system 1000.
A block diagram is shown. This system 100
0 is the stack decoder 1002 and its stack
Audio processing device 100 connected to the Tsuku decoder
4 and used when performing fast approximate speech matching.
array processor 1006 and detailed audio
array processor used when performing
1008, language model 1010, and workspace
It is equipped with a station 1012. The audio processing device 1004 widens the audio waveform input.
A label that identifies the type of sound it corresponds to, in a specific sense.
or is designed to convert to a string of phonemes.
Ru. In this system, the audio processing device 10
04 is based on the unique model of the human ear.
Regarding this, the applicant's patent application filed in 1980-
It is described in No. 211229. Labels or phonemes from speech processing unit 1004
enters stack decoder 1002. Logical
Stack decoder 1002 is shown in FIG.
It can be expressed as a block element. vinegar
That is, the stack decoder 1002
-Face 1022, 1024, 1026 and 1
028, the audio processing device 1004, high-speed illumination
Synthesis processor 1006, detailed verification processor 10
08, and the language model 1010, and further
Search to contact work station 1012
It has block 1020. In operation, from the audio processing device 1004,
Phonemes are searched by search block 1020 in a fast matching process.
You will be led to Rosetsusa 1006. This high-speed matching process
As will be explained later, the
Also described in U.S. Patent Application No. 672,974 mentioned above.
Ru. Simply stated, the purpose of this matching is to
Determine the most likely word for a column of labels.
Is Rukoto. Fast matching examines words in the vocabulary and
to reduce the number of candidate words for the force label sequence.
It is intended to be a sea urchin. This fast matching is done by Marco
A stochastic finite state machine, also called a f-model,
Based on. Fast matching reduces the number of candidate words
, the stack decoder 1002 uses the language model 1
010, the language model 1010
Preferably, it is based on existing trigrams.
Then, the sentence of each candidate word in the fast matching candidate list.
Determine contextual certainty. Preferably, based on the calculation results of the language model,
Through detailed verification, the plausibility of the word being spoken is confirmed.
Words with high probability are candidates for high-speed matching.
It will be considered later. About this detailed matching procedure
is described in the above-mentioned U.S. Patent Application No. 672,974.
Ru. The detailed matching procedure is shown in Figure 3.
implemented by a Markov model sound machine such as
will be carried out. After that detailed matching, preferably from the exact word
The language model is called again to determine the
It will be done. High-speed matching, detailed matching, and language model
The present invention utilizes information obtained through application.
The tag decoder 1002 processes the generated label
Most likely path or column of words corresponding to column
is designed to determine. Two methods for finding the most probable word sequence
As a conventional technology, Viterbi decoding
decoding and single stack decoding.
Ru. These techniques were developed by L.R. Barr and F. Jie mentioned above.
Described in the paper by Linetsk and R.L. Mercer.
Ru. In particular, Viterbi decoding is the fifth
In chapter 6, single stack decoding
described in the chapter. In single stack decoding technology
varies during a single stack depending on the certainty.
Routes of various lengths are listed. Single stack
Decoding is more or less accurate due to the length of the path.
depending on the
We must take into account the fact that
No. On the other hand, Vyterbi's technology does not require such standardization.
generally practical for small tasks
It is. Another technique is to use each possible combination of words
as possible word sequences and which combinations
has the highest probability of producing the generated label sequence
By determining which has
Decoding can also be done per stem.
Ru. However, the amount of calculation required for this technique is large.
This makes it impractical for lexical systems. Stack decoder 1002 includes substantially other
It plays the role of controlling the block circuit, but many
No calculations are performed. Therefore, the stack decoder
1002 is preferably a Virtual Machine/
System Product Introduction Release 3
(1983) and other publications.
Under VM/370 operating system
Contains a running 4341 processor. Equivalent calculation
This array processor is not commercially available.
Implemented by Floating Point System (FPS) 190L
ing. Multiple stacking and the best word sequence or path
A new system with a unique decision strategy for determining
The technology was developed by L.R. Barr, F. Zielinetsk, and R.L. Ma.
About this later as it was invented by Sir
state E-1B auditory model and its realization FIG. 4 shows an audio processing device 1100 (in FIG.
1004) is illustrated.
Ru. In this figure, an audio wave input (e.g.
conversation) is an analog-digital (A/D)
input to the converter 1102, and the A/D converter
1102 samples the input at a given rate.
pull. Typical sampling rate is 50 ma
One sample every microsecond. A/D converter
1102.
A time window generator 1104 is provided to
Ru. The output of the time window generator 1104 is a fast Fourier
It is input to the conversion circuit (FFT) 1106 and FFT1
106 gives a frequency spectrum for each time window.
Ru. The output of FFT1106 is then labeled y₁y₂‥y_femits
Processed to voice. to generate a label
In addition, a feature selection block 1108, a cluster block
Tsuku 1110, prototype block 111
2. Four times called label creation block 1114
Road blocks work together. In the generation of labels,
The protocol is configured based on selected features or audio input.
Based on a point (or vector) in space
space that can be determined and then compared to the protocol.
to give the corresponding points (or vectors) in
characterized by the same selected features as
I get kicked. In particular, when deciding on a protocol, the cluster block
The lock 1110 allows the collection of points to be separated into individual clusters.
Classified as a star. to determine the cluster
The method is applied to speech, such as a Gaussian distribution.
Based on probability distribution. cluster centroid or
Prototypes for each cluster relative to other features
is generated by prototype block 1112.
be born. and by the same selected features.
The generated protocol and voice are characterized by
The input is entered into label creation block 1114.
Ru. Label creation block 1114 performs a comparison procedure.
This will cause one label to be assigned to a specific audio input.
assigned to power. Selection of appropriate features depends on the audio (conversation) wave input.
This is an important factor in obtaining a label that allows here
The audio processing device described in
It has a feature selection block 1108. this audio
According to the processing unit, an auditory model is obtained, which
It is applied to the speech processing device of the speech recognition system.
To explain the auditory model, please refer to Figure 5.
Ru. Figure 5 is a diagram showing a part of the inside of the human ear.
It is. In particular, internal hair cells 1200 absorb fluid.
End 1202 protruding into retaining passageway 1204
It is shown as having internal hair cells
Upstream are external hair cells 1206, which also
with an end 1208 projecting into the passageway 1204
It is shown as. and internal hair cells 12
00 and external hair cells 1206 send information to the brain.
It has nerves connected to it. In particular, the new euro
electricity sent through nerves to the brain for processing
undergoes electrochemical changes that result in physical stimulation. This electric
Gas chemical changes are caused by mechanical movement of the basement membrane 1210.
be stimulated by Conventionally, the basilar membrane 1210 has a frequency response to audio wave input.
It acts as a wave number analyzer and
It is possible for each part to respond to individual critical frequency bands.
Are known. And the different basement membrane 1210
When a part responds to its corresponding frequency band
This means that the perceived sound for a sound wave input is
Affects size. That is, two similar
than when sounds of intensity occupy the same frequency band.
Also, the two sounds are in different critical frequency bands.
In some cases, the sound is perceived to be louder.
Ru. Approximately 22 determined by basement membrane 1210
It is known that a critical frequency band exists.
Ru. to match the frequency response of the basement membrane 1210.
In this embodiment, the audio processing device 1100 preferably performs the following steps.
converts the audio wave input into a part of the above critical frequency band or
is separated into all its separated critical frequency bands
The signal components are examined individually for each region. This feature is
Filter the signal from FFT1106 (Figure 4) as appropriate.
Then, during the feature selection block 1108, the
It is possible to provide separate signals for each critical frequency band.
This is accomplished by Its separate input is also a time window generator 1104
within a time frame (preferably of 25.6 ms) by
is blocked. Therefore, the feature selection block
The check element 1108 preferably includes 22 signals.
for a given period, each of which corresponds to one time frame.
Represents the strength of sound in the wave number band. The filter action is preferably of the conventional type shown in FIG.
This is done by a critical band filter 1300. Individual
Another signal is then equal to the loudness of the sound.
(loudness) converter 1302.
Ru. This converter 1302 converts the perceived sound into
Consider the change in magnitude as a function of frequency.
In this respect, a given dB level of a certain frequency
The first sound in is loud enough to be heard.
and at the same dB level at a different frequency.
Note that the second sound may be different from the second sound.
sea bream. Converter 1302 converts various frequencies
so that the bands are measured on the same size scale
In order to convert the signals of each frequency band into
Based on data. For example, converter 1302
is preferably a 1933 Fletcher
(Fletcher) and Munson (Munson) research
Based on this, and by making some changes to it.
Correlation of sound intensity to sound volume isosensory curve
I do. The modified results of these studies are
This is shown in Figure 7. In other words, (1000
As can be seen from the isosensitive curve passing through the point (Hz, 40dB)
, 1000Hz, 40dB sound becomes 100Hz, 60dB sound,
They are comparable in size. The converter 1302 has equal sensitivity regardless of the frequency.
Preferably follow the curve of Figure 7 to give the degree of
to adjust the volume. As can be further seen from Figure 7, to the frequency
There is a dependence not only on sound intensity and sound volume.
It does not correspond to Sato. i.e. the intensity or vibration of the sound.
A change in width is a similar change in audible loudness.
Not necessarily reflected by change. for example,
At 100Hz, the sound intensity changes by 10dB at 110dB.
Since the sound intensity changes by 10dB with 20dB,
The change in the loudness of the audible sound is due to the former
is much larger. This difference is a predetermined
A size scaling block that compresses the size in style.
Processed by lock element 1304. suitably
is the magnitude scaling block element 1304
is the loudness of the sound expressed in phons
Replacing the amplitude measurement with sone
, the intensity P is its cube root P^1/3Compress it into Figure 8 shows the experimentally obtained Thorn vs. Huon
It is a figure showing data. Adopting Thorne
Due to the above model, the large amplitude of conversational speech
is almost accurate. Furthermore, what is 1 sone?
It is defined as a sound level of 1KHz, 40dB.
Ru. Referring again to Figure 6, each critical frequency
Equisensitivity corresponding to band, scaled by size
A time-varying response block element that processes the programmed signal.
A child 1306 is illustrated. In particular, for each time frame
the neural network for each frequency band being tested.
Neural firing rate f is determined
be done. The neural firing rate is the result of this speech processing.
According to the device, it is expressed as follows. f=(So+DL)n (1) In this formula, n is the nerve transmission signal
(neurotransmitter) amount, So is the audio wave input
Spontaneous fatigue associated with unrelated neural firing
Eyring constant, L is the magnitude measurement, D is the variation
It is a positional constant. In other words, (So)n is the sound wave input
Spontaneous nerve damage that occurs regardless of whether there is force or not.
corresponding to earring rate, DLn is based on audio wave input
corresponds to the firing rate. What is important is that the value of n is determined by this audio processing device.
Assuming that it changes over time based on the following relational expression,
It is characterized by: dn/dt=Ao−(So+Sh+DL)n (2) In this equation, Ao is the supply constant and Sh is the spontaneous
is the decay constant of the neutral transmission signal. As shown in equation (2)
The novel relationship that has been shown is that neural transmission signals are
It is being formed at a rate of (a) collapse (Sh×n),
(b) spontaneous firing; and (c) acoustic wave input.
It is important to consider that the nerve damage caused by
This was taken into consideration. These modeled phenomena
The estimated position of is shown in FIG. Equation (2) also calculates the following quantities and facades of the neural transmission signal:
earing rate is at least an expression of the amount of neural transmission signals.
This sound means that it depends multiplicatively on the existing conditions.
It also reflects the fact that voice processing devices are nonlinear.
ing. That is, God at time (t+△t)
The amount of nerve transmission signal is the nerve transmission signal at time t
is the amount of (dn/dt)・△t, and this
Expressing this in the formula, n(t+△t)=n(t)+(dn/dt)・△t (3) Equations (1), (2) and (3) indicate that the above listening system
It is adaptive over time, and auditory nerve signals are
It is said that the signal is nonlinear with respect to the audio wave input.
Write a time-varying signal analyzer that takes advantage of the fact that
It is something. In this respect, this audio processing device is divine.
better match the apparent time changes in the physiological system
As described above, nonlinear signals in speech recognition systems are
The first model that embodies the signal processing is given below. In order to reduce the number of unknowns in equations (1) and (2),
This audio processing device is applicable to a constant sound volume L.
The following equation (4) is used. So+Sh+DL=1/T (4) In this equation, T is the time when the audio wave input is generated.
However, the maximum value decreases to 37%, but the auditory response is
It is a measure of the time it takes for an answer to be answered. T is the loudness of the sound
According to this audio processing device, it is a function of
Declining response to different loudness levels
This is the value obtained from the graph representing . Sunawa
In other words, when a sound of a certain loudness is generated, it
1, then the response is
It falls to the steady state level with a time constant T.
If there is no audio wave input, T = To (about 50 milliseconds)
It is. L_naxIf T=T_nax(about 30ms)
It is. By setting Ao=1, L=
When 0, I/(So+Sh) is 5 centiseconds.
Also, L is L_naxand L_nax=20 sones
Then, equation (5) becomes as follows. So+Sn+D(20)=1/30 (5) Based on the above data and formula, So and Sh are calculated by formula (6) and
(7), it is determined as follows. So＝DL_nax/(R+(DL_naxToR) −1 (6) Sh=1/To−So (7) Here R=fs | L=L_nax/fs｜L=0 (8) Here, fs | is the loudness of a given sound when dn/dt=0.
Represents the firing rate at high temperature. R is the only variable left in the audio processing device.
Ru. Therefore, in order to change the performance of the audio processing device,
For example, only R can be changed. That is, R is
Typically, minimizing steady-state effects versus transition effects
Adjust to change performance meaning
is a single parameter that can be steady state
Minimizing the effect is due to differences in frequency response,
due to differences in speakers, background noise, and
Affects the steady state part of the conversation that is not the transition part of the conversation.
Generally speaking, the distortion exerted on the same audio input is
Because it produces a non-constant output pattern,
Delicious. The value of R is preferably
by optimizing the overall error rate of the system.
will be played. The appropriate value found in this way is
R=1.5. Then, So=0.0888, Sh=
It becomes 0.1111, and D becomes 0.00666. Referring to FIG. 9, the flowchart of this audio processing device
A chart is shown. In Figure 9, favorable
25.6ms, preferably sampled at 20KHz
The digitized audio within the time frame of Hanning
(Hanning) window 1320, Hanning window 1
The output from 320 is preferably flashed at 10 millisecond intervals.
1322. child
The output thus converted is filtered by element 1324.
at least one frequency band (preferably
All critical frequency bands or at least 20
intensity density corresponding to each of the critical frequency bands)
gives the output of The density of this intensity is then a logarithmic variable.
The loudness level is determined by the conversion step 1326.
is converted to This is based on the graph in Figure 7.
It is easily executed. The processing below is
10th step 1330;
As shown in the figure. In Figure 10, sensory threshold Tf and auditory sound
threshold T_his each filtered frequency
For band m, in step 1340 T_f=
120dB, T_h=0dB is the initial setting. after that,
audio counter, all frame registers,
The histogram register is reset in step 1342.
is set. Each histogram is the number of samples or counts
contains a bin representing the value, and between the counted values
The intensity within a given frequency band or its equivalent.
Different measurements are in individual ranges. current hiss
The totogram is preferably for each given frequency band.
, the loudness of the sound falls within several ranges of loudness.
represents the number of centiseconds in a period. for example,
In the third frequency band, the intensity is 10dB and 20dB
There may be an interval of 20 centiseconds in between. Similarly,
In the 20th frequency band, 50dB and 60dB
During that time, 150 centiseconds out of the total 1000 centiseconds
It can exist. Total number of samples (or centiseconds)
Then, the percentile is obtained from the count value contained in the bin.
It will be done. In step 1344, each frequency band filter is
The frames from the router output are checked and appropriate
One bin in the histogram for each filter.
It is incremented at step 1346. Amplitude is 55dB
The total number of bins exceeding
audio data (i.e., frequency band).
The number of filters representing the presence is determined. stop
the minimum number of sounds (e.g. 20).
If none of the filters (6 of them) exist, the next filter is
The frame is checked at step 1344. too
In step 1350, a sufficient number to represent the voice is selected.
If the filter exists, in step 1352
A voice counter is incremented. The audio counter
Step 1354 until you hear 10 seconds of audio.
is incremented at step 1352, then step 13
T for each filter in 56_fand T_hnew value of
is determined. T_fand T_hThe new value of for a given filter is
It is determined as follows. T_ffor 1000 bins
35th from the top (i.e. 96.5th percentile of audio)
BIN is the dB value of the bin that holds samples of_H
is defined as Then T_fis T_f＝BIN_H+
It is set to 40dB. T_h, the lowest bin
A bit that holds the 0.01th sample in percentile from
dB value of BIN_Lis defined as Sunawa
T-BIN_Lis a sample classified as audio.
of the number of samples in the histogram excluding the number of pulls.
It is a bottle that is 1%. Then, T_his T_h=
BIN_LDefined as −30dB. Returning to FIG. 9, the audio amplitude is determined at step 133.
Based on the thresholds updated at 0 and 1332,
Converted to thorn in step 1332 and scaled
be logged. As mentioned above, how to do this
It is. To get Thorn and scale
Alternatively, the file (after the bin has been incremented)
Using the filter amplitude “a”, based on the following formula:
May be converted to dB. a^aB=20log_Ten(a)-10 (9) Then each filter amplitude is equal based on the following formula:
range between 0 and 120 to give a larger size.
scaled. a^eql=120(a^dB−T_h)/(T_f−T_h) (Ten) a^eqlis preferably determined by the following formula:
(1 KHz signal at 40 dB)
(by matching) the approximate size in the Thorn
is converted to L^dB=(a^eql-30)/4 (11) And the size in Thorn is approximated as follows
be done. L_s(approximate) = 10 (L^dB)/20 (12) The size represented by Thorn is then input as
In step 1334, equations (1) and (2) are given, which
output fire array corresponding to each frequency band.
The switching rate f is determined (step 1335). twenty two
If there are several frequency bands, a continuous time frame (frame)
A 22-dimensional vector specifies the audio wave input
signal. However, in general,
(mel: unit of pitch)
Twenty frequency bands were tested using router banks.
It will be done. Before processing the next time frame (step 1336)
Then, in step 1337, the next state of n is calculated based on equation (3).
The status is determined. The above-mentioned audio processing device has a firing rate f and a medium
It has a DC pedestal with a large amount of vertical transmission signal n.
In case of application, it is improved. That is,
The terms in the equations for f and n also have a dynamic range.
To reduce pedestal height, if important.
The following formula is obtained. First, it is in a steady state and there is no audio wave input (L
= 0), equation (2) is the internal state in the steady state
For n′, it is solved as follows. n′=A/(So+Sh) (13) The internal state of the amount of nerve transmission signal n(t) is steady
By the part n′ and the time-varying part n″, we have the following
be harassed. n(t)=n′+n″(t) (14) Combining equation (1) and equation (14), the fire
The following equation for the ring rate is obtained: f(t)=(So+DL)(n′+n(t)) (15) In this equation, the term So×n′ is a constant.
and all other terms are the time-varying part of n, and
and DL.
The rest of the process is to calculate the squared difference between the output vectors.
The constant term can be ignored since it is involved in the There
Then, using equation (13), remove the constant term from equation (15).
Letting the equation be expressed as f″(t), f″(t)=(So+DL)n″(t) +DLA/(So+Sh) (16) Considering equation (3), the next state is n(t+△t)=n′(t+△t)+n″(t+△t) (17) =n′(t+△t)+n″(t)+(Ao −(So+Sh+DL)n(t))△t (18) =n″(t+△t)+n″(t)−(So+Sh) n′(t)−DLn′(t)△t+Ao△t −(So＋Sh＋DL)n″(t)△t (19) In equation (19), we decide to ignore all constant terms.
From this, the following formula is obtained. n″(t+△t)=n″(t)(1-Sh△t)−f″(t)△t
(20) Equations (15) and (20) are now
Output method applied to each filter for each time frame of seconds
Construct an equation and a state update equation. these
The result of applying the equation is that every 10 milliseconds
is a 20-element vector, and each component of the vector is
individual scaled filter banks.
Corresponds to the firing rate of the frequency band. Regarding the embodiment shown earlier, f, dn/
The expressions of dt and n(t+1) are respectively
A spatial representation of the ring rate f and the next state n(t+
Δt). Note that the values contributing to various equation terms (e.g. to
= 5 centiseconds, t_Lnax= 3 centiseconds, Ao = 1, R
= 1.5, Lmax = 20) may be set to another value,
Then, So, Sh and D are each preferred
different from the new derived values 0.0888, 0.11111 and 0.00666.
Please note that the value is This audio model was developed by the inventor as a floating point
PL/I program on system FPS190L hardware
It was implemented using the ramming language, but other
using different software or hardware.
It can also be used. E-1C Detailed verification FIG. 3 shows a sample detailed verification sound machine 200.
0 is shown. Each detailed matching sound machine is
(a) Multiple states Si, (b) Multiple transitions tr(Sj | Si)
(The transitions include those between different states and those between different states.
There are things that return to themselves from a state of
Each transition is associated with a probability), (c)
correspond to each label that can occur in a particular transition.
is characterized by the actual label probability
It is a stochastic finite state machine. In FIG. 3, detailed verification sound machine 2000
7 states S₁~S₇And the 13 states tr1 to tr13 are
It is given. As can be seen from Figure 3,
The sound machine 2000 has three transitions indicated by dashed lines.
It has tr11, tr12 and tr13. these three
At each transition, the sound can generate a label.
change and follow from one state to another
Therefore, such a transition is called a zero transition. one
On the other hand, labels are generated along the transition from tr1 to tr10.
It can be done. Especially for each transition from tr1 to tr10, one
or more labels, depending on the number of times they are generated.
It can have different probabilities. Preferably, individual
can occur in this system at the transition of
There are probabilities associated with labels. Sunawa
If it occurs selectively depending on the audio channel,
If there are 200 labels that can be
Each transition (that is not a zero transition) has a
has 200 “actual label probabilities” and
Each probability corresponds to a sound at a particular transition
Corresponds to the probability that a label will be generated. In Figure 3
then the actual label probability for transition tr1 is P
[i] (i is an integer from 1 to 200, and is the label number
). for example,
In the case of label 1, the detailed verification sound machine 2000
There is a probability P[1] that label 1 occurs at transition tr1
do. The actual various label probabilities are
and their corresponding transitions. Now, corresponding to a given sound, the column of labels y₁y₂y₃
... was given to the detailed verification sound machine 2000.
The matching procedure is executed. Detail verification sound machine
The relevant procedure is explained with reference to Figure 11.
Ru. Figure 11 is a grid diagram of the sound machine in Figure 3.
Ru. Similar to the sound machine in Figure 3, this grid diagram is
state S₁from state S₇and a zero transition to state S₁empty
Status S₂to & condition S₁from state S_Fourshows the transition to
Ru. Also shown are transitions between different states. child
The grid diagram also has a horizontal time scale.
Ru. In Figure 11, the probability q at the start point is₀and q₁
are, respectively, for a sound, when the sound is at time t=t₀
and t=t₁represents the probability of having a start time. each
start time t₀and t₁The various transitions are illustrated in
It is shown. In addition, the time between sequential start times
The interval is preferably equal to the length of the label time interval.
Please note that how much a given sound is labeled in the input string
Detailed matching sound machine 200 to determine the close
If 0 is adopted, the minute at the end of that note.
The cloth is found and it determines the match value of that sound
used for. Based on the distribution of end times
The concept of
common to all embodiments of the sound machine disclosed.
Ru. end point distribution to perform detailed matching.
When forming, detailed verification sound machine 2000
performs rigorous and complex calculations. Referring to FIG. 11, time t=t₀start time and
Regarding the calculations required to have both end times.
Let's think about it first. This is the example shown in Figure 3.
Since the field follows the sound machine structure, the following probability
applies. Pr(S₇, t=t₀)=q₀T(1→7)+ Pr(S₂, t=t₀)T(2→7)+ Pr(S₃, t=t₀)T(3→7) (21) In this equation, Pr is the certainty in the state shown in parentheses.
The rate, T, is in the direction of the arrow of the state number shown in parentheses.
is the transition probability. Equation (21) shows that the end time is t=t₀
The individual probabilities corresponding to the three conditions such that
Display. Furthermore, t=t₀The end time is
For example, state S₇Occurrence is limited to
You can see that. Then end time t=t₁If we pay attention to the state S₁other than
calculations must be made for all states of
You can see that there isn't. state S₁is the end of the previous note.
starts at the end of the period. For convenience of explanation, state S_Four
Only calculations for are shown. state S_FourIn this case, the calculation formula is as follows. Pr(S_Four, t=t₁)=Pr(S₁, t=t₀) T(1→4)Pr(y₁｜1→4)+Pr (S_Four, t=t₀)T(4→4)Pr (y₁｜4→4) (22) In other words, equation (22) means that time t=t₁The sound machine
state S_FourThe probability that is depends on the sum of the following two terms:
-ing That is, one of the terms
is (a)t=t₀In state S₁For the probability that the state S₁from
state S_FourThe transition probability to and the state S₁from state S_Fourto
given label y when there is a transition₁is generated
The other term is (b) time.
t=t₀In state S_FourWith probability that state S_Fourfrom itself
Probability of transition to body and state S_Fourfrom to itself
given label y when there is a transition₁Probability that is generated
It is multiplied by Similarly, another state (but state S₁(excluding)
Even if time t=t₁The probability that the sound is in a certain state is
Calculations are performed to find . Generally, places
When determining the probability of being in a particular state at a given time
(a) the detailed verification procedure that led to that particular state;
each previous state with a transition, and each state before that.
(b) for each previous state.
and each previous state to match the label column.
is not raised on the transition between state and current state.
Recognize the value representing the probability of a label that should not be
(c) Redefine the probability of each previous state and the label probability.
into the corresponding transition.
The purpose is to find the probability of the above-mentioned specific state.
The overall probability of being in that particular state is that
that state for all transitions that lead to it
is determined from the probability of In addition, state S₇calculation for
The sound is in state S₇When the sound ends at time t
=t₁Make it possible to start and end with
This includes terms related to three zero transitions.
Please be careful. time t=t₀and t=t₁As well as the probability calculation for
Then, the probability calculation for another end time column is
preferably done to form the ending distribution
Ru. The value of the distribution at the end corresponding to a given note is
How well does a given phone match the input label?
It shows how it is. How many words are in the input label column?
If you want to determine how well the word matches,
A number of single notes are processed. And each note is
Generate the ending distribution of probability values. for that single note
The matching value is obtained by adding the ending probabilities and then
It is obtained by taking the logarithm of the sum. Also, the following
The onset time distribution of a single note is scaled, e.g.
Divide each value by the total value so that the sum of the values is 1.
Standardize the ending point distribution by scaling the value of
It can be obtained by Note that a given word or word string is tested.
The method for determining the number h of power tones is at least as follows:
There are also two. First, the depth first method
method), each individual
It is done to calculate sequentially moving small forms for each single note.
It will be done. And the subtotal follows that basic format.
Below the expected threshold corresponding to the position of a given note
When it is found that , the calculation ends. Alright
In the breadth first method,
is calculated for the positions of similar phonemes in each word.
It will be done. i.e. following the first phonetic sound of each word
As a calculation, calculate the second phonetic sound of each word.
etc. will be carried out. In the breadth-first method,
The calculated value along the same number of phonemes in various words is that
Compare at the same relative pitch position along these notes.
be done. In both of these methods, the matching value
The word(s) with the largest sum is the one you are looking for.
It is a target for Detailed verification can be done using APAL (Array Processor
assembly language). In addition, APAL is
Floating Point System Co., Ltd.
(Floating Point System, Inc) 190L specific access
It is Sembla. For detailed matching, the actual probability of each label (i.e.
That is, at a given transition, a given note has a given label.
probability of generating y) and the transition for each sound machine.
transition probability and a given time after a specified start time
memorize the probability that a given note is in a given state in
It is recognized that considerable memory is required for
sea bream. The FPS (floating point system) 190L mentioned above is
the end time, e.g. the sum (preferably the end point probability)
(logarithm of the sum of ) or previously generated
Start time based on end point probability and word
Word matching based on match values corresponding to consecutive phonemes
Set to perform various calculations such as total score.
will be uploaded. Additionally, detailed matching is preferred.
In the matching procedure, the tail probability (tail
probability). Terminal probability is the word
We measured ordinal certainty of labels without involving
It is something. In a simpler embodiment, given
The terminal probability of is the probability of a label following another label.
correspond to the nature. For example, how much is this certainty?
Is it a label sequence generated from a few sample sounds?
It is easily determined. Therefore, the detailed matching is based on the basic form and Mark
statistics for the f-model and to contain the terminal probabilities.
requires sufficient storage capacity. For example, each
Vocabulary of 5000 words, each word consisting of about 10 sounds
, the basic format requires 5000x10 memory.
be. Also, (each note is accompanied by a Markov model)
) 70 different notes and 200 different labels
and 10 labels with a probability that each label is generated.
If a transition exists, its statistics include 70×10×
200 positions are required. However, the sound machine
Three parts: a beginning part, a middle part,
Split into ending parts and accompanied by corresponding statistics
(preferably 3 in consecutive parts)
(contains two self-loops). Therefore, the necessary notes
The storage capacity is reduced to 70x3x200. Regarding the terminal probability
then 200x200 storage locations are required. child
array fills up 50K integer and 82K floating point storage.
Gives proper movement. In addition, detailed matching is based on phonemic rather than phonetic sounds.
Can be performed by using sound
Please note that. E-1D Basic high-speed verification As mentioned above, detailed matching is computationally expensive.
Therefore, the amount of calculation required can be reduced even if some accuracy is sacrificed.
Basic fast matching and other fast matching to reduce
will be carried out. This fast matching is preferably
used in combination with matching, i.e. fast matching
lists likely candidate words from the vocabulary,
and detailed information for each candidate word listed at most.
A match is made. A high-speed approximate voice matching technique is described in the above-mentioned present application.
No. 672,974.
In high-speed approximate speech matching technology, it is preferable to
is each transition in all transitions in a given sound machine.
Place the actual probability corresponding to the label with a specific replacement value.
By changing each sound machine, each sound machine is simplified.
Ru. A particular replacement value is preferably
The match value corresponding to a given sound when used
If the replacement value of does not replace the actual label probability, then
Overestimation of matching values achieved by detailed matching
selected to be of value. guarantee this condition
One method is to
Any probability corresponding to Bell is greater than the replacement value
By choosing each replacement value as not. Onma
The actual label probability in the syn is the corresponding permutation
for a certain word by replacing it with the value
Significantly reduces the amount of calculation required to determine matching scores.
Do a little. Furthermore, the replacement value is preferably overestimated.
Therefore, the obtained matching score is not replaced.
not less than the score that would have been determined. In a linguistic decoder with a Markov model
In certain embodiments, where phonetic matching is performed using
is such that each sound machine is equipped with the following (a) to (c).
characterized by training. (a) Multiple states and transition paths between those states. (b) Transition tr(Sj|Si) with probability T(i→j). So
Each of is the state given the current state Si
Represents the probability of transition to Sj. Furthermore, this Sj and Si
can be in the same state or in different states.
good. (c) Actual label probability P(Yk|i→j). Each label probability P(Yk|i→j) is one state
For a given transition from to its next state,
Let the probability that label y.k is generated by the sound machine be
express. Here k is for identifying the label.
It is a subscript. Each sound machine: (d) in each sound machine;
To assign a single specific value P′(Yk) to each Yk of
and (e) at each transition in a given sound machine.
Then, each actual output probability P(Yk|i→j) is expressed as
For a single specific value P′(Yk) assigned to Yk
It has means for reversing and replacing it. suitably
whose replacement value is any transition of a particular sound machine
The actual maximum label of the corresponding Yk label in
At least equal in magnitude to probability. Fast verification procedure
is the most probable from the vocabulary corresponding to the input label
Approximately 10 to 100 units selected as the best words.
It is employed to determine a list of word candidates. child
A language model is preferably applied to these word candidates.
A detailed match is performed. Do it like this
The number of words considered by detailed matching is
By reducing it to approximately 1%, the calculation cost becomes significant.
accuracy is reduced without any loss in accuracy. The basic fast matching is given in a given sound machine.
for all transitions such that a label of
A single value that represents the actual label probability for a given label
to simplify detailed matching by replacing
Ru. That is, for a given label whose probability of occurrence is
Regardless of the transitions in the sound machine, the probability is a single specific value
replaced by This value is
Maximum probability and minimum probability of labels occurring in thin transitions
The overestimated values are at least the same size.
Ru. Label for a given label in a given sound machine
Set the replacement value of the probability to the maximum value of the actual label probability
generated using basic fast matching by
The matching values obtained are at least as detailed as possible.
It is as large as the matching value obtained. In this way, the base
Book-like fast matching typically uses more words.
Matching value of each phone so that it is widely selected as a candidate
Or overestimate. That is, based on detailed verification
Words that are considered candidates are also based on this basic
Pass criteria based on fast matching. Referring to Figure 12, for basic high-speed matching,
A meno sound machine is shown. (Symbols and phonemes
Also called. ) label along with the starting point distribution
Input into a basic high speed verification tone machine. this start
The time distribution and label column inputs are the detailed matching described above.
Similar to what is input into a sound machine. Furthermore, open
The starting point distribution sometimes spans multiple times.
Rather than the distribution, for example, the point at the beginning of the sound.
Denotes the exact time that follows a period of silence.
I would like you to recognize that there are also However, the audio is
If it is continuous, the starting point distribution (for which
(described in more detail later)
used. Sound machine 3000 ends point distribution
and corresponding to a specific sound from its ending time distribution.
Generates a match value. The match value for a word is the phoneme
(at least the first h sound of the word) and
It is defined as Referring to Figure 13, the basic high-speed matching operation
is shown diagrammatically. This basic fast match
The operation is generated by the starting point distribution and the single note.
associated with the number or length of labels Yk and each label Yk.
replacement value P′_Ykbe involved only in And given
All actual for a given label in a sound machine
To replace the label probability with the corresponding replacement value
, basic fast matching converts transition probabilities into length distributions
Replaced by the probability (for each transition in a given sound machine
the actual label probabilities (which may be different) and
The necessity of having a probability of being in a given state at a given time
remove the need. In this regard, the length distribution is
Determined from the file. In particular, for each length in the length distribution
Accordingly, the detailed matching procedure involves checking each state individually.
For each state, select (a) a specific label length.
is given, and (b) regarding the output along the transition.
The currently inspected condition may not be present.
Determine the various possible transition paths. specific
All transition paths of a certain length leading to each transition path
The probabilities corresponding to are summed and then in that distribution
To display the probability of a given length of
The sums corresponding to all states of are added. Verification
The procedure is repeated for each length. Verification procedure
Following the preferred form of
The trellis diagram is known in the field of economic models.
This is done with reference to the diagram). That is, the lattice
For transition paths that share branches along the structure, the common
Calculations need to be done only once for each branch passed through
The calculated value is added to each path that includes a common branch.
available. In FIG. 13, two limitations are illustratively included.
It is rare. First, the latitude produced by a single note is
The length of each bell is 1₀,1₁,1₂, and 1₃certainty
Assume that it is one of 0, 1, 2, or 3 with a rate of
Ru. In addition, the start time is also limited, which allows each
is the probability p₀,q₁,q₂and q₃of four starting times with
It becomes possible to With these limitations, the following equation
determines the end point distribution of the target note. φ₀=q₀1₀ φ₁=q₁1₀+q₀1₁p₁ φ₂=q₂1₀+q₁1₁p₂+q₀1₂p₁p₂ φ₃=q₃1₀+q₂1₁p₃+q₁1₂p₂p₃+q₀1₃p₁p₂p₃ φ_Four=q₃1₁p_Four+q₂1₂p₃p_Four+q₁1₃p₂p₃p_Four φ_Five=q₃1₂p_Fourp_Five+q₂1₃p₃p_Fourp_Five φ₆=q₃1₃p_Fourp_Fivep₆ Looking at these equations, φ₃However, there are four openings.
It can be seen that terms corresponding to each start time are included.
The first term is that the single note is at time t=t₃starts with
generate the label length (i.e. at the same time
It must be a single note that begins and ends.) Represents the probability.
vinegar. The second term is the time t=t₂started with
The length of the label is 1, and the label 3 is the single note.
represents the probability of being generated. The third term is simply
sound is time t=t₁and the label length is 2.
Yes (i.e. labels 2 and 3), labels 2 and 3
and 3 are generated by a single note.
Similarly, the fourth term indicates that the single note is at time t=t₀started with
, the label length is 3, and the three labels 1,
Expresses the probability that 2 and 3 are generated by a single note.
vinegar. The amount of calculation required for basic high-speed matching and detailed matching
When comparing the amount of calculation required for
It turns out that it is also relatively easy. Regarding this point
and P′_YkThe value is the same as the label length probability for all
remains the same for each representation of the expression. In addition, long
There is a limit to the start time, so please wait until the end of the later
Calculations in between become easier. For example, φ₆smell
So, for a single note, time t=t₃I have to leave at
, and the three labels 4, 5 and 6 should be,
generated by a single note whose end time should be applied
Must-have. Emit the matching value of the target single note.
The end time that follows the predetermined end time distribution when
Point probabilities are summed. If desired, give the following formula
Therefore, the logarithm of the sum is taken. Match value = log_Ten(φ₀＋‥‥＋φ₆) As mentioned before, matching on a word
The score value is based on consecutive single sounds in a particular word.
easily by summing the matching values for
Desired. Now, to explain the generation of the starting point distribution.
Please refer to FIG. 14 for details. Figure 14a Smell
The word THE₁is repeated and it becomes an elemental
divided into single notes. Figure 14b shows that over time
The label column is shown. In Figure 14c,
1 starting point distribution is shown. This starting point
The distribution (includes silence, i.e. “words” without sound)
(in the previous word) previous recent single note
This is obtained from the end point distribution. Figure 14
Based on the label input and starting time distribution of c,
Start time distribution φ of sound DH_DHis generated. next unit
The start point distribution of the note UH is the end point of the previous single note.
Period when the fabric exceeds the threshold (A) in Figure 14 d
Determined by recognizing the interval. (A) is
It is determined separately for each end point distribution. suitably
(A) is the end time distribution of the target single note.
It is a function of the total value. In this way, at times a and b
The period between
(See Figure 14e)
(see). Period between times c and d in Figure 14e
During this period, the end point distribution of the single note DH exceeds the threshold.
In addition, the starting point distribution of the next note is set.
Corresponds to the period. The value of the starting point distribution is, for example,
At each end, the sum of end times exceeding the threshold (A)
Normalize the end point distribution by dividing the time
It can be obtained by This basic high-speed matching sound machine 3000 is
APAL printer in Floating Point Systems' 190L.
It was executed using a program. Follow the above teachings
to develop a specific form of matching procedure.
You can also use hardware and software.
Wear. E-1E Another high-speed match Basic fast matching alone or detailed matching
or combined with a language model to perform the necessary calculations.
Significantly reduce the amount. Further reduces the amount of calculation required
In order, the minimum length L_nioand maximum length L_nax
We define a uniform label length distribution between the two lengths.
The method described here allows detailed matching by
further simplify. In basic fast matching
is the probability of a single note producing a label of a given length,
i.e. 1₀,1₁,1₂etc. typically have different values.
It's on. However, this other high-speed matching
For example, the probability for each length of the label is a single uniform value.
Can be replaced. Preferably, the minimum length is a fraction of the original length.
equal to the smallest length in the cloth that has non-zero probability
However, you may choose other lengths if desired.
stomach. Choosing the maximum length is more arbitrary than choosing the minimum length
but less than the minimum length and greater than the maximum length
in that the probability of a long length is set to zero.
is important. Let the length probability be the maximum length and minimum length
By setting it so that it exists only between
A uniform pseudodistribution is given. in a certain way
is the uniform probability is the mean probability on that pseudodistribution and
It can be set by Or a uniform
The probability is the length replaced by the above uniform value.
It may be set as the maximum value of the probability of Characterize all label length probabilities as equal
The effect of this is at the end of basic high-speed matching.
It is easy to refer to the formula shown above corresponding to the cloth.
recognized. In particular, the length probability is treated as a constant term.
It can be washed out. L_niois set to zero and all length probabilities are simply
By replacing it with a constant value of 1,
The cloth is characterized as follows. θ_n=φ_n/1=q_n+θ_n-1Pn Here “1” is a single uniform replacement value,
The value of Pm is preferably determined during a given note at time m.
corresponds to the generated replacement value for the given label. For the above formula for θm, the matching value is as follows
It is defined as follows. Match value = log_Ten(θ₀+θ₁+… +θ_n) + log_Ten (1) Compare basic fast match and this alternative fast match
Then the number of additions and multiplications required is
Significantly reduced by adopting another fast matching method.
I know that it will happen. L_nio= 0, the basic height
In fast matching, length probabilities must be taken into account.
Therefore, 40 multiplications and 20 additions are required.
That's what I found out. Using this alternative fast match
Then, θ_nis found recursively, and for each continuous θ_nNitsu
Only one multiplication and one addition are required. How this alternative fast match simplifies calculations
Figures 15 and 16 are given to illustrate the
is being given. In Figure 15a, the sound machine
Example 3100 has a minimum length L_nio=0 corresponds to
Ru. The maximum length at this time is determined by the length distribution being uniform.
defined as infinite so that it can be characterized as
has been done. In Figure 15b, the sound machine 3
A grid diagram generated from 100 is shown. child
Here, q_oThe start time after is outside the start time distribution.
For each successive θ for m<n_n
To determine all of , one addition and one multiplication are required.
is necessary. Also, to find out the end time
requires only one multiplication and no addition. Figure 16 shows L_nioThe case where =4 is shown. Figure 16
a has a special example of a sound machine 3200 for that purpose.
Therefore, Figure 16b shows the corresponding grid diagram.
Rawasu. L_nio= 4, so the grid diagram in Figure 16b is
Zero along the path marked by u, v, w and z
Has probability. θ_Fourand θ_othose ends extending between
For the end time, four multiplications and one addition are required.
Please note one thing. However, greater than n+4
In the case of high end time, only one multiplication and addition is
unnecessary. This example uses APAL code on FPS190L.
was executed using the code. In addition, the first
Even if a state is added to the embodiment of Fig. 5 or Fig. 16,
Please note a good thing. E-1F Matching based on first J label Basic high-speed matching and further high-speed matching above
The strings input into the sound machine to be refined into
Only the first J labels of the
It is intended that this will be taken into consideration. every 1/100th of a second
The audio processing equipment of the audio channel is
Assuming that the label is generated by the position, J
The corresponding value of is 100. In other words, about 1 second
The labels corresponding to the sounds of the degree are single sounds and sound machines.
to check the match between the incoming labels and
Given. Limiting the number of labels inspected
As a result, two advantages are realized. First, de
Coding delay is reduced. Second, short
Problems when comparing word scores to longer word scores
is substantially avoided. Of course, the length of J is
It may be changed as desired. Effect of limiting the number of labels inspected
can be seen by referring to the grid diagram in Figure 16b.
I can take it. If the above improvements are not made, the high speed
The matching score is θ along the bottom line of this figure._nof the probability of
It is Japanese. That is, (L_nio= 0) t=t₀Ma
Taha (L_nio= 4) t=t_Foureach time starting at
Between state S_FourThe probability of being in is θ_nTomo Tomera
, all θ_nare then summed. L_nio=4
If t_Fourstate S at any time before_FourIt is certain that
There is no rate. However, using the above improvement, θ_n
The sum ends at time J. In Figure 16b, time
interval J is time t_o+2corresponds to End inspection of J label after J time interval or more
This means that the following two certainties are required when calculating matching scores:
yields a total rate. First, as mentioned above, the lattice
There is a row calculation along the bottom row of the diagram, but it is
up to interval J-1. At each hour up to time J-1
state S_FourThe probabilities in are combined to get the score value for the row.
It is measured. The second is state S at time J₀~S_Foureach state of
There is a score for the row that corresponds to the sum of the probabilities that there is a single note in
do. That is, the score for the row is Row score =_Four 〓^f=0 Pr(S_f, J) This matching value for a single note is the row score and column score.
by summing and then taking the logarithm of that sum.
can get. Continue fast matching for next note
The time J is preferably included in order to along the bottom line
The value is used to obtain the starting point distribution of the next note.
be done. Calculate the match value for each successive note.
After calculating, as mentioned before, all the single notes
is the sum of the matching scores of all the phones.
becomes. The basic high-speed matching described above and another high-speed matching
Examining how the ending point distribution is generated, we find that
Column score counts easily match fast matching calculated values
Notice what I don't do. Limit the number of labels examined
The improved method of determining
In order to
Request to replace points with scores from another line
do. That is, time J and J+K (K is the sound machine
state S between (maximum number of states)_Four(Figure 16
The scores of the separate rows are calculated for the notes in b).
Ru. Therefore, if the sound machine has 10 states
If the lattice on which the probabilities are computed is
Adding 10 end times along the bottom row of the diagram
It becomes. Then, in order to obtain the matching value of a given single note,
All confirmations along the bottom row up to time J+K
The rate and probability at time J+K are added.
Also, as before, to get the word matching score
Successive phone matching values are summed. This example uses APAL code on FPS190L.
It was carried out using However, other codes and other
You can also do it using hardware.
Ru. E-1G Tree structure of single notes and high-speed matching Use basic fast match or another fast match
Therefore, even if you limit the maximum label, it will not be done.
Even if the
Computation time is significantly reduced. Furthermore, fast matching
A detailed match is performed on the words obtained by
This computational time savings is large enough even when
It's a huge amount. Once the matching value of a single note is calculated, the 17th
As shown, along the branches of the tree structure 4100
This determines which path of the note is most likely.
Mustard squid is determined. In Figure 17,
(Connects from 4102 to branch 4104) DH and
The sum of matching values of the single notes corresponding to DH1 is the single note MX
Rather than the various tones that branch out from the
corresponds to the word “the”.
Should. In this regard, the single note of the first MX note
The match value is calculated only once, then that MX note
(branch 4) is used for each basic form extending from
104 and 4106). Furthermore, the first branch
All score values calculated along the column are calculated for the columns of other branches.
much lower than the total score value or close to the threshold
If you find that the
All basic forms extending from the columns of are candidates at the same time.
removed from complementary words. For example, MX is sure
When it turns out that it is not the correct route, branch 4108
The basic formats associated from to branch 4118 are the same.
sometimes abandoned. By using this high-speed matching and tree structure,
candidate words with extremely low computational complexity.
You will get an ordered list of . Regarding memory requirements, the sound tree structure and the sound
statistics and terminal probabilities are what should be remembered.
Please note that. Regarding the tree structure,
25,000 arcs and 4 data that characterize each arc
ta word exists. That first data word
The key changes the index to the next arc or note.
Was. The second data word is the next one along that branch.
represents the number of single notes. third data word
is the node in the dendrogram at which the arc is located.
Shows what is happening. In addition, a fourth data
C represents the current single note. Therefore, the dendritic structure
In this case, 25000×4 storage locations are required.
In fast matching, 100 different single notes and 200
There are black phonemes. Then, a single sound with a phoneme
If we have a single probability generated somewhere in
Requires memory locations for 100 x 200 statistical probabilities
It is. Finally, for the terminal probability, 200 × 200
A memory location is required. This fast matching requires
100K integer and 60K floating point storage is sufficient.
Ru. E-1H language model As mentioned before, words in context
A language module that stores information such as related triplet letters.
Increase your chances of making accurate word selections by using
You can also do this. The language model 1010 (Figure 1) has unique properties.
have. In particular, the converted trigraph system is used.
ing. According to this method, a single
words, an ordered pair of words, and an ordered pair of words.
In order to determine the certainty of the triple words that were eclipsed.
Sample text is checked. And the most
List of probable triplets and most likely pairs
A list of words is formed. Additionally, triple words
The probability that is not in the triple word list above, and the paired word is
The probabilities of the above pairs of words not being in the list form each
be done. According to this language model, a given word has two
When two words follow, this word and its successor
are present in the triple word list above.
A judgment is made as to whether the And if
If so, the memory assigned to that triple word.
The probability that the Also, if the word at hand
and the following two words are in the triple word list
If not, the current word and the next word are
Determine whether or not the word exists in the paired word list.
A decision is made. And if so, triplicate
The probability that a word is not in the triple word list is
The pairwise probabilities are multiplied and the product is assigned to the word at hand.
It is applied. Or, if the current word and the next
(and the next) words are paired with the triple word list
If not on the list, only the current word
The probability that a triple word is not in the triple word list and
is multiplied by the probability that the word is not in the paired word list.
Ru. This product is then assigned to the word at hand.
Ru. Referring to Figure 18, the
A flowchart showing the details of the sound machine is shown.
It is. In step 5002, words in the vocabulary are
(typically about 5000) are defined. Then each
Words are represented by rows of sound machines (sound machines).
step 5004). This sound machine can e.g.
It is represented as a target sound machine, but instead
It can also contain phonemic sounds. words phonetically
Depending on the row of sound machines or the row of phonemic sound machines.
The representation will be explained below. In step 5006, the basic form of the word is
are arranged in the tree structure. Basic form of each word
The statistics of each sound machine in F. Zielinecz
(Jelinek) “Continuous speech using statistical methods”
Continuous Speech Recognition by
Published in a paper titled “Statistical Methods”
The well-known forward and backward
backward) by training based on the algorithm
It is determined. In step 5009, the
should be substituted with the actual parameter value or statistic
The value is determined. For example, the actual label output probability
The value to be assigned to is determined. Step 5010
Then, the sounds in each word basic form are approximate substitutions.
The determined value is stored as a stored actual value.
Replaced by the actual probability. Related to basic fast matching
All approximations to
be done. Next, we will discuss whether or not to improve the ability of phonetic matching.
A determination is made (step 5011). and,
If not, for basic approximate matching
The determined value is set so that it can be used.
and other evaluation values related to other approximations are set.
No (step 5012). It has also been improved
If an approximate match is desired, then step 5
018 is executed. Then a uniform string
The length is determined (step 5018), and
Improvements are requested (step 5020).
A judgment is made as to whether or not. And if near
If further improvement of similar matching is desired,
Phonetic matching is performed on the first J in the generated string.
label (step 5022).
Next, whether or not improved approximate matching was selected
Regardless, the calculated parameter value is
12, and at this point each word is in its basic form.
Each sound machine in the
has been trained with the desired approximation value. E-1J stack decoder A preferred method used in the speech recognition system shown in Figure 1.
The stack decoder is based on the applicant's speech recognition group.
L. Bahl and F. Zielinetsk from the Group
(Jelinek), and by R.L. Mercer.
Invented. Therefore, this suitable stack deco
The reader will be explained below. Figures 19 and 20 show the sequential label spacing.
or multiple labels occurring sequentially at a label location.
Bell Y₁Y₂‥‥It is shown. Figure 20 also shows some generated words.
The paths, i.e. path A, path B and path are illustrated.
ing. In the context of Figure 19, route A is the entry
Corresponding to “to be or”, route B is the entry “two
b” and route C corresponds to the entry “too”.
do. Corresponding to the current word path, the word path
the label with the highest probability that it has finished
(or equivalently, label spacing) and such
Such labels are called "boundary labels." For a word string W that represents a word string, there are two
label strings as “boundary labels” between words.
Most likely finish time revealed during the ring
However, IBM Technical Disclosure Bull
Tein (Technical Disclosure Bulletin)
volume 23, number 4, September 1980, L.R. Barr
“Fast
Faster Acoustic Match
Computation)”
can be found by such methods. briefly describe
and this document addresses two similar matters, namely
(a) How many of the label string Y are words?
(or word string) or
(b) At what label spacing is one label string
whether the partial sentence corresponding to the part ends or not.
Discuss ways to get involved. For any word path, the label string
Each label from the first label to the border label
“Likelihood value” associated with a label or label interval
exists. Putting these together, a given word history
The probability values of all the tracts for that given word
Represents the “likelihood vector” of the route. Therefore, each
For each word path, there is a corresponding likelihood vector.
Ru. Likelihood value L_tis shown in FIG. Word route W¹,W²，‥‥W³corresponds to a gathering of
The “likelihood envelope” Λ at the label interval t_tis a number
Scientifically defined as: Λ_t=max(L_t(W¹),...L_t(W^S)) That is, for each label interval, the likelihood envelope
A line is associated with any word path within that collection.
Contains the highest likelihood value. Figure 20 shows the likelihood envelope
Line 1040 is illustrated. If a word path corresponds to a complete sentence,
It is considered “complete”. A complete route is preferred.
Suitably, the speaker's input allows the speaker to e.g.
by pressing a button when the end is reached.
It is identified by This input is synchronized with the label spacing.
The end of the sentence is marked. complete word path
cannot be extended by adding words.
“Partial” word paths correspond to incomplete sentences,
Can be extended. Partial pathways are classified as “living” or “dead”
be done. That is, if a word path is already extended
If so, the word path is “dead” and
If it has not been extended yet, it is “alive”. this
Using classification, you can
already extended to form a lengthened word path
The route that has been extended will be extended at a later time.
It is never considered again. Each word path is also related to the likelihood envelope
Can also be characterized as “good” or “bad”
can. That is, if the label corresponding to the boundary label
If the word path is within Δ of the maximum likelihood envelope,
The word path is good if it has a likelihood value of
be. Otherwise, the word path is “bad”
That's why. Note that preferably, but not necessarily, Δ is
is a constant value, and this value determines the maximum likelihood envelope
Each value of will act as a threshold level for good or bad.
It is greatly reduced. There is one stack element for each label interval.
Ru. Each living word path is
The step corresponding to the label interval corresponding to the threshold level
Assigned to the tact element. The stack element is
Having 0, 1 or more route entries
and those entries follow the order of their likelihood values.
is listed. Next, in the stack decoder 1002 of FIG.
The steps to be executed will now be explained. Form a likelihood envelope to determine which word path is better.
The sample flowchart in Figure 22
Interrelate with the steps shown in the chart.
It will be done. In the flowchart of Figure 22, zero
(null) The path is first entered in step 5050.
Input to Tack 0. And decided in advance
If there are complete paths with
A stack (complete) element containing is given (stack
5052). Each complete element in a stack (complete) element
A path has an associated likelihood vector.
Ru. of the complete path with the highest likelihood for the boundary label.
The likelihood vector initially determines the maximum likelihood envelope.
Set. If a complete path in a stacked (complete) element
does not exist, the maximum likelihood envelope is
Initialized as −∞ in the interval. Furthermore, if completed
If all paths are not identified, the maximum likelihood envelope is still
The contact line can be initialized as −∞. envelope
Steps 5054 and 50 for line initialization.
56. After the maximum likelihood envelope is initialized, its envelope
The line is lowered by a predetermined value Δ, which lowers the
A Δ-good region is formed above the lowered likelihood value.
A Δ-defective region is formed below the likelihood value.
Ru. The larger the value of Δ, the more word paths
is considered to be extendable. L_tTo calculate
Menilog_Tenis used, set Δ to 2.0.
values give satisfactory results. The value of this Δ is
Preferably, but not required, the length of the label spacing
uniform along. If the word path is a boundary label in the Δ− good region
If the word path has a likelihood in
Marked as “good”. Otherwise,
The word path is marked as "bad". As shown in Figure 22, the likelihood envelope is updated,
Mark word paths as good (extensible) or bad
The longest unmarked loop for
It starts by finding a new word path (starting from
Step 5058). And if marked
If there are two or more word paths that do not exist, then the longest word path
If it exists in the stack corresponding to the length of
the word path with the greatest likelihood for that boundary label
is selected. If the word path is found,
The likelihood at that boundary label is within the Δ-good region
marked as “good” when, otherwise
Marked as “bad” (step 506)
0). If a word route is marked as “bad”
, another unmarked living path is found.
is marked (step 5062). if word
If a route is marked as “good”, the “good”
Likelihood envelope to include the likelihood value of the marked path
The line is updated. That is, for each label interval,
(a) Current likelihood value in the likelihood envelope and (b) “Good” and mark
between the likelihood values associated with the checked word paths.
, the likelihood value updated as a larger likelihood value is
It is determined. This means that steps 5064 and
This is indicated by step 5066. envelope
is marked as the longest and best after updated
The missing living word path is found again (Stetstu
5058). The loop then checks that the unmarked word path is
The remaining ones are returned until there are none left. So
Then all unmarked words will be found.
When the shortest word path marked as “good” is
selected. If the shortest length marked as good is also
If one or more paths exist, the boundary label
The path with the highest likelihood in the
(Step 5070). The shortest path selected is
will be extended. i.e. preferably fast
Matching, Language Models, Detailed Matching, and Language Models
As described above, by executing the procedure
At least one likely following word is determined.
It will be done. With each word that follows this certainty,
A lengthened word path is formed. In particular, extended
The selected word path is the terminal of the selected shortest word path.
formed by adding words that most likely follow
be done. a single word extended to that selected shortest word path.
After the word path is formed, the selected word path is
The path is the star that contained the path as an entry.
removed from Tsuku, each extended word path is
It is placed in the appropriate stack for that purpose. Special
, the extended word path is the extended word path
The elements placed on the stack corresponding to the boundary labels of
entry (step 5072). Regarding step 5072, the selected route is
The extending action is explained with reference to FIG.
Ru. After the route is found in step 5070,
The following procedures are carried out, which results in:
Single or multiple word definitions based on suitable approximate matching
The road will be extended. That is, in step 6000 (FIG. 21)
In this case, the voice processing device 1002 (FIG. 1) performs the above-mentioned
Generate a label string as shown below.
The column of labels indicates the possible execution of step 6002.
It is given as input to make the calculation. step
At 6002, the candidate word is determined according to the teachings shown previously.
To obtain an ordered list of
Approximate matching or improved approximate matching procedures
One is executed. Then the language model (as mentioned above)
The rule is applied in step 6004. language model
The remaining words after being applied are processed in step 600.
generated in a detailed matching processor that runs
will be entered along with the specified label. Detailed matching remains
yields a list of word suggestions, and this list is preferred.
Preferably, in step 6008, the language model is
It will be done. In this way, approximate matching, detailed matching, and
The likely words determined by the word model are
The route found in step 5070 of FIG.
used for extension. Step 6008 (22nd
Each of the probable words determined in
headings so that an extended word path can be formed.
found so that a word path can be formed.
added to each word path. Referring again to Figure 22, the extended path is
After the stack is formed and the stack is re-formed, the steps
The process is repeated by returning to 5052.
Ru. In this way, each iteration returns the shortest best word history.
It consists of selecting a path and extending it. once
Word paths marked as “bad” by eye processing are
It may become "good" in subsequent iterative processing. Living
Characterize the word path that came as “good” or “bad”
In this way, each iteration is independent
done to. In practice, the likelihood envelope is
It does not change significantly from the first iteration to the next iteration.
and whether the word path is “good” or “bad”.
Calculations to determine whether there is
Ru. Furthermore, no standardization is required. When a complete sentence is identified, it is preferable to
Step 5074 is executed. i.e. marked
There are no unused word paths left and there are no good word paths left that should be extended.
When there is no suitable word path, decoding
ends. Even the highest likelihood for each boundary label
A complete word path is added to the input label string.
is considered to be the most probable word sequence for
Ru. Furthermore, continuous speech in which the end of a sentence is not identified
If the route extension is continuous or system
The number of words to be repeated according to the user's preference is
Continue. E-1K Construction of basic phonetic formats can be used in forming the basic form
Also based on speech as a Markov model sound machine
There is. That is, each sound machine uses the International Phonetic Alphabet.
(International Phonetic Alphabet)
corresponds to a given phonetic sound such as For a given word, each to an individual sound machine.
There is a sequence of corresponding phonetic sounds. each sound
Machines can exist in multiple states and in between those states.
, and some of those transitions are
Something that can and is also capable of producing phonemic output
(called zero transition) cannot do that. aforementioned
As before, the statistics associated with each sound machine are (a)
(b) the probability that a given transition occurs and (b) the probability that a given transition occurs.
and the likelihood that the phoneme is generated. Preferably, each non-
In the zero transition, each phoneme has some probability associated with it.
It is being responded to. At the end of the detailed description of the invention.
The phoneme alphabets shown in Table I above have a preferred
There are actually 200 phonemes. phonetic sound
The sound machine used to form the machine is the first
This is shown in Figure 3. A row of such sound machines is
It is provided for each word. Statistics or probability is
Sound reinforcement during the training period when known words are pronounced.
input into the and various phonetic sounds.
The transition probabilities and phoneme probabilities in syn are known
Occurs when a phonetic sound is pronounced at least once.
Focusing on the phoneme strings that appear, the well-known forward-backward
(forward-backward) algorithm applied
This may be determined during the training period. Example of statistics for one single note identified as a single note DH
is shown in the table listed at the end of the detailed description of the invention.
ing. As an approximation, the sound machine in Figure 3
The label output probability distribution of transitions tr1, tr2 and tr8 is single
is expressed by the distribution of Also, transition tr3,
tr4, tr5 and tr9 are represented by a single distribution,
Transitions tr6, tr7 and tr10 are also expressed by a single distribution
be done. This means that in Table 2, the individual columns
Assign arcs (i.e. transitions) to 4, 5 or 6
This can be seen by Table 2 shows the accuracy of each transition.
rates and labels (i.e. phonemes) beyond the single DH
Indicates the probability of occurrence at the end, middle, and rear end.
are doing. For this DH sound, for example, state S₁mosquito
state S₂The probability of transitioning to is calculated to be 0.07243.
It is calculated. state S₁from state S_FourThe transition probability to is
0.92757 (in this case, it is possible from the initial state
Since there are only these two transitions, the sum of their probabilities is
is 1). Regarding the label output probability, DH
The note is at the end of the single note, i.e. in the 6th line of Table 2.
The probability of generating phoneme AE13 (see Table 1) is 0.091.
be. Table 2 also shows the information related to each node (state).
There is a count value. This node count value is the training
Displays the number of times the sound is in the corresponding state during the period.
vinegar. Statistics like Table 2 are found for each sound machine.
Ru. Arranging phonetic sound machines into word basic forms
is typically performed by phoneticians and is commonly
is not executed automatically. E-1L Construction of basic phonemic forms Figure 23 shows specific examples of phonemic sounds.
Ru. This phonemic sound has two states and three transitions.
have. In this figure, the zero transition is the dashed line
, which is a state in which no label is formed.
It represents the route from state 1 to state 2. state 1
A self-loop transition with any number of labels there
make it possible to be generated from. state 1 and
Non-zero transitions between state 2 cause the label to form
It is permissible to each transition and each
During the training stage, the probabilities associated with the labels are
Similar to that described in relation to the basic forms of voice types.
determined by the method. Phonemic word basic forms combine phonemic sounds.
It is constructed by letting one for this
This method was filed on February 1, 1985 by the applicant.
No. 697,174. good
Preferably, the basic phonemic form of a word is
produced by pronouncing a word multiple times. this thing
is a patent application filed on May 29, 1985 by the applicant.
No. 738,933.
Briefly, the basic form is extracted from a large number of pronunciations.
One way to generate it is as follows:
Ru. (a) The multiple pronunciations of a word fragment can be expressed as individual phoneme strings.
Convert to ring. (b) Define a set of phonemic Markov model sound machines.
to justify (c) To generate the above-mentioned large number of phoneme strings.
The best single sound machine P₁Determine. (d) To generate the above-mentioned large number of phoneme strings.
, format P₁P₂or P₂P₁The best two sounds consisting of
Decide on the basic format. (e) Corresponding to each phoneme string, select the best two
Arrange basic sound forms. (f) Split each phoneme string into left and right parts.
Separate. At this time, the left part is in two-note basic form.
corresponds to the first sound machine, and the right part corresponds to the second sound machine.
Corresponds to the second basic type sound machine. (g) Each left part as a left substring and each right part as a left substring.
and the right substring, respectively.
Ru. (h) Supports multiple pronunciations for left substring sets
are processed in the same way as the set of phoneme strings
Ru. This process is based on the idea that a single basic form of sound is the best
When it has a higher probability than the two-tone basic form,
Prevent substrings from separating further.
Contains steps. (j) Supports multiple pronunciations for the right substring set
are processed in the same way as the set of phoneme strings
Ru. This process is based on the idea that a single basic form of sound is the best
When it has a higher probability than the two-tone basic form,
Prevent substrings from separating further.
Contains steps. (k) unseparated single tones, their
Sounds correspond to the order of their corresponding phoneme substrings
join in the order you want. The number of these model elements is typically the word
approximately equal to the number of phonemes obtained corresponding to the pronunciation of
stomach. The basic formal model is then
, and in response to that utterance, a string of labels is generated.
training by inputting the sound into the audio processing device that generates the sound.
be refined (or incorporate statistics). and known pronunciations and uttered labels.
Based on the well-known forward-backward
−backward) word model according to the algorithm
statistics are obtained. Figure 24 shows the grids corresponding to phonemic sounds.
It is shown. This grid provides detailed phonetic matching.
It is considerably simpler than the lattice in Figure 11 associated with
Ru. E-2 Confirmation from vocabulary by polling
choice of words Referring to FIG. 25, a diagram of one embodiment of the present invention is shown.
A roach is illustrated. This flowchart
In step 8000, the
First, the vocabulary of words is determined. These words are
Standard business communication vocabulary or techniques, depending on the user.
corresponds to technical vocabulary. At this time, 5000 pieces or
There are more words in the vocabulary, but
The number of words can be changed. Each word is from the teaching in Chapter E-1K or E-1L above.
As shown, the sequence of Markov model sound machines
will be displayed. That is, each word consists of successive sounds
Basic form of vocal sound machine construction, or sequential
As the basic form of the phonemic sound machine constructed
can be expressed. Next, in step 8006, each label of each word is
A “vote” is calculated each time. The step 8006 of calculating votes is the 25th, 2nd
6, 27, 28 and 29.
Ru. Figure 26 shows a given sound machine P_paudio label of
FIG. The totals shown in this figure
The numbers are taken from the statistics generated during the training period.
It is something. That is, during the training period, the known
The known pronunciation corresponding to the tone sequence is uttered,
A label string is generated in response.
I want you to remember that. In this way, it is known
Each label is generated when a single note is pronounced.
The number of times is obtained during the training period. Furthermore, as shown in Fig. 26,
Such a distribution map is generated for each note. From the training data, the information contained in Figure 26
Not only can information be obtained, but also information about a given note can be obtained.
You can also get the expected value of the label. That is, for a given unit
When a known pronunciation corresponding to a word is generated,
The number of labels generated is recorded. given single note
The number of labels corresponding to the known pronunciations
It is recorded every time it is born. And this information?
, the most probable corresponding to a given note, and
The expected number is determined. Figure 27 shows each
A diagram showing the expected number of labels for each note.
be. If these sounds correspond phonemically
For example, the expected number of labels for each note is
The average value should typically be about 1. audio
For typical sounds, the number of labels can vary widely.
Can be extended to. Extracting information from training data is described above.
“Continuous speech recognition using statistical methods”
(Continuous Speech Recognition by
It is detailed in a paper entitled “Statistical Methods”.
forward-backward
achieved by using algorithms
Ru. Simply stated, the forward-backward algorithm
teeth, (a) From the initial state of the Markov model to state i
Look ahead and check the state on the “forward path”.
determine the statistics leading to i, (b) From the final state of the Markov model to the state (i+
1) Look backwards and follow the “rear route”.
The statistics from state (i+1) to the final state are
By deciding, Between state i and state (i+1) in a single note
consists of determining the probability of each transition in . situation
The transition probability from i to state (i+1) and its
The label output for a given label string
the probability of a particular transition occurring given the
combined with another statistic in determining the Shown in Figure 28 in relation to word 1 and word 2.
As shown, each word is a sequence of predetermined single sounds
It is known. The sound sequence for each word,
Provides the information mentioned in connection with Figures 25 and 26
and how many times a given label is issued for a particular word W.
determining whether it is most likely that
I can do it. For words like word 1, the label
The number of times 1 is expected is the sound P₁about label
1 count value and sound P₃Counting of label 1 for
value and sound P₆Add the count value etc. of label 1 for
It is something that has been learned. Similarly, for word 1, the label
The number of times Le2 is expected is the sound P₁label about
The count value of Le 2 and the sound P₃Label 2 total for
This is the addition of numerical values, etc. for each label of word 1
The expected counts are for each of the 200 labels.
calculated by performing the above steps.
Ru. The count value for a particular label is
Let the raw number be the total number of labels generated during the training period.
Show what you cut or show the number during the training period.
Indicates the number of bell occurrences itself. In Figure 29, specific words (e.g.
The expected count value for each label in 1) is shown.
has been done. For a given word, the period as shown in Figure 29
From the counted value of the expected label, each label of the word is
Bell's "votes" are calculated. For a given word W′
The votes for label L′ at
represents the likelihood that the This vote indicates that the word W′
It corresponds to the logarithm of the probability of producing L′. preferred
In other words, the vote value is expressed by the following formula. votes = log_Ten{Pr(L′|W′)} The values of these votes are as shown in Figure 30.
stored in the table. For each of words 1 to W,
Each label is a vote number represented by a double subscript V.
Has value. This first subscript corresponds to the label.
Accordingly, the second subscript corresponds to a word. Therefore,
For example, V₁₂is the vote value of label 1 for word 2
be. Referring again to Figure 25, when an unknown voice input
including step 8008 of generating a label in response.
and polling to find likely candidates from the vocabulary.
The process of selecting words is illustrated. This place
The process is performed by the audio processing device 1004 (FIG. 1).
executed. For the word in question, utter it in the table in Figure 30.
The generated label is look-up.
be done. And for that word, each generated
The vote value of the given label is searched. The value of this vote is
Then save up to give the total vote value for that word.
are accumulated (step 8010). For example, the label
If 1, 3 and 5 are generated, the vote value V₁₁,
V₃₁and V₅₁will be calculated and combined.
If the value of the vote is the logarithm of the probability, then they
to give the overall voting value for word 1
are totaled. A similar procedure is performed for each word in the vocabulary.
This will cause labels 1, 3 and 5 to be attached to each word.
You will have to “vote” for that. According to one embodiment of the invention, for each word the
The obtained vote value serves as the likelihood score value for that word.
Fulfill. and also the value of the highest accumulated vote.
and n words (n is a planned integer) are candidate words.
These will later be determined as detailed above.
It will be processed in the matching and language model. In another embodiment, a “penalty” is added along with the vote value.
i” is calculated. That is, for each word,
Penalty is calculated and allocated (step
8012). This penalty applies if the current label
represents the likelihood that it is not caused by a given word.
vinegar. There are various ways to calculate the penalty.
There is a law. expressed by the basic phonetic form
One method for calculating word penalties
, each phonetic sound generates only one label
involved in assuming that. given label and identification
For the phonemic sound of , the pena of its given label
Lutei is labeled differently from that particular sound.
corresponds to the logarithm of the probability generated by an elementary sound.
Ru. Therefore, the sound P₂The penalty for label 1 is
Any label from label 2 to label 200 is
Corresponds to the logarithm of the probability, which is one label generated
do. Note that one label is output for each phonetic sound.
Although it is not accurate to assume that
To calculate the Lutei, it must be satisfactory.
I understand. In this way, each note has a label.
Once a penalty is determined, a word is known and
The penalty for composing a sequence of notes is easy to
It is determined. Figure 31 shows the penalty for a single label for each word.
has been done. Each penalty has two subscripts.
PEN and the first subscript refers to the label.
The second subscript represents the word. Returning to FIG. 25 again, at step 8008 the
The generated label is labeled Alphabetical.
To find out which labels are not occurring at the same time
will be inspected. In this way, each label
The penalty for not incurring a penalty is calculated. given
To get the total penalty for the word of the given
Penalty for each label not raised for each word
Tey will be searched and all such penalty
is accumulated (step 8014). If each pena
If Lutei corresponds to the logarithm of the “zero” probability, then
The penalty for a given word is
Similarly, it is summed for all labels. child
This procedure is repeated for each word in the vocabulary.
Each word is then a string of generated labels.
given the overall vote value and the overall
There will be penalties. For each word in the vocabulary, the overall vote value and overall
Once Narutei is obtained, combine the two values.
The likelihood score value is determined by
Tup 8016). Furthermore, if you wish, you can also
The value is weighted more heavily than the overall penalty.
or vice versa. Furthermore, the likelihood score value of each word is preferably
scaled based on the number of labels voting.
(Step 8018). In particular, the overall vote value and
The overall penalty (both of which are the sum of the logarithms of the probabilities)
) are added together, and then the final
The sum value is used to calculate the vote value and penalty.
Divide by the number of phonetic labels generated and considered.
As a result, the likelihood score value is scaled. Yet another aspect of the invention provides voting and penalty
(i.e., polling) operation,
to decide which labels to consider for
Related. Word endings are identified and their corresponding
If the label is known, preferably
Everything that occurs between the start time and the known end time
all labels are considered. But the end time
When something is known that is not known (Step 8)
020), the present invention provides the following method. vinegar
That is, a standard end time is defined and
repeats at successive time intervals after the specified end time.
A likelihood score value is calculated (step 802).
2). For example, after 500 ms, each
The (scaled) likelihood score value of the word is calculated.
and this behavior is then applied to the start time of the word's pronunciation.
The process continues until 1000 milliseconds have passed. In this example
, each word has 10 (scaled) likelihoods
It will have a degree score value. Next, which of the 10 likelihood score values for a given word?
There is a method for making choices about whether to allocate
Applicable. In particular, the likelihood obtained for a given word is
obtained at the same time interval for a column of degree score values.
The highest likelihood score value is compared to the likelihood score value of other words.
A degree score value is selected (step 8024). child
The highest likelihood score value for that same time interval is
is subtracted from all other likelihood score values in .
Then, the highest likelihood score value for a given time interval is
words with a lower likelihood are set to zero, and other less likely words are set to zero.
The word's likelihood score value will have a negative value. stop
Therefore, the base of a given word also has small negative values (near zero).
b) The relative likelihood score value with the highest likelihood score value
is assigned to that word. When a likelihood score value is assigned to each word, the most
n words that are assigned high likelihood scores
is the candidate word obtained by polling.
selected (step 8026). In one embodiment of the invention, polling
The n words obtained by
The words in this list are given as
detailed matching and language model processing
Ru. This reduced value obtained by polling
The list of numbers used in this example is as shown above.
It functions as an alternative to the high-speed phonetic verification that was performed. This point
, phonetic fast matching gives a tree-like lattice structure.
Eh, in this tree structure, the basic form of the word as a sequential sound.
An expression is entered that also contains the same initial note.
It is observed that words follow a common branch along a tree structure.
be noticed. However, for a vocabulary of 2000 words,
Akira's polling method is a high-speed method with a tree-like lattice structure.
The processing speed can be two to three times faster than matching.
I understand. However, unlike that, phonetic fast matching and point
It can also be used in combination with
Ru. That is, the trained Markov model and
From the generated string of labels, step
Approximate high-speed matching in parallel with polling in 8028
is executed. And one list is a phonetic reference.
and the other list is given by Pauli.
given by ng. In the conventional method, 1
Entries on one list increase the other list
used when making However, the best word suggestions
The method that seeks to further reduce the number of
, only words that appear in both lists are
Retained for further processing. step 80
The interaction of these two technologies in 30
depends on system accuracy and computational goals. yet another
As an example of
It may then be applied to polling lists. The device for performing polling is shown in Figure 32.
It is shown. In this figure, element 8102
memorizes the word model trained as described above
do. And from the statistics applied to the word model
The vote generator 8104 generates a label for each word.
Calculate the value of the vote in the table and record the value of the vote in the vote table.
It is stored in the storage device 8106. Similarly, the penalty generator 8108
Calculate the penalty for each label of each word in the vocabulary.
The value is stored in the penalty table storage device 8.
110. The word likelihood score calculation device 8112 calculates unknown sounds.
Emitted by the voice processing device 8114 in response to voice input.
Receive the generated label. And the word selection element
For a given word selected by child 8116
The word likelihood score calculation device 8112 calculates the selected word likelihood score value.
The votes for each label generated for the selected word and each label
Combined with penalty that no bell occurs
Ru. The device 8112 performs a likelihood score, as described above.
Also includes a means to scale the values.
Ru. Although a likelihood score calculation device is not required,
also calculates the likelihood score value at successive time intervals after the reference time.
may include means for repeating the calculation of
stomach. The likelihood score calculation device 8112 is a word list device.
The word score value is given to the device 8120, and the word list device
The setting 8120 is based on the assigned likelihood score value.
Arrange words. The word list obtained through polling is
Combining the list obtained by similar high-speed matching
In some embodiments, list comparison device 8122 is provided.
It is being This device 8122 has as input:
Polling list and phonetic from word list device
Fast matching (as mentioned in some of the examples above)
Receives a polling list from. To reduce the amount of memory and computation required, we
Some features are included. First, the value of votes
and penalty are expressed as integers ranging from 0 to 255.
It can be automated. Second, the actual page
The value of penalty is penalty = a x (value of votes) + b
Approximately calculated corresponding to the vote value from the formula
can be replaced by a penalty.
In this formula, a and b are constants, and their values are
is determined by least squares regression. Thirdly, label
file so that each class contains at least one label.
can be grouped into phonetic classes.
Wear. And the assignment of labels to classes is
information between the resulting phonetic classes and words.
Collect labels hierarchically to maximize information
Determined by Furthermore, according to the present invention, the period of silence is
be detected and ignored (by law)
stomach. The invention also works on IBM MVS systems.
Executed using PL/I, but with other systems
It can also be executed using other programming languages. Furthermore, within the scope of its technical idea, the present invention
Various modifications are possible. For example, the end time of a word in its basic form
summing the number of expected labels for each sound in
It may be determined by In addition, the vote value and
The label string generated
For example, the odd numbered label or the first m
only for selected labels like the labels in
It may also be calculated. However, the tip of the word
Let the vote value and penalty for each label between and the trailing edge be
It is preferable to consider. Furthermore, the present invention adds the logarithm of the probability.
It is also intended to use a variety of voting formulas other than
ing. The present invention allows each label to vote for each word in the vocabulary.
and this vote typically differs for each word.
to get a short list of candidate words when
Widely applied to polling equipment and polling methods
It will be done. F Effect of invention As described above, according to this invention, voice recognition
Create a list of words for detailed matching
Try using the polling method to
Therefore, such a word list can be processed in a short processing time.
can get.

【table】

【table】

【table】

【table】 [Brief explanation of drawings]

Fig. 1 is a schematic block diagram of a speech recognition system, Fig. 2 is a block diagram showing the block diagram of Fig. 1 in more detail, Fig. 3 is a diagram of an acoustic sound machine, and Fig. 4 is a block diagram showing the block diagram of Fig. 1 in more detail. Block diagram of audio processing device, No. 5
The figure is a cross-sectional view showing the inside of the human ear.
A block diagram of a part of the audio processing device, FIG. 7 is a diagram of the iso-sensory curve, FIG. 8 is a diagram showing the correspondence between son and phon, and FIG. 9 is a processing flowchart of the audio processing device. Fig. 10 is a detailed floor cheat for updating the threshold value in Fig. 9, Fig. 11 is a diagram showing a detailed matching grid, Fig. 12 is a block diagram of a high-speed matching sound machine, and Fig. 13 is a high-speed FIG. 14 is a diagram showing the matching operation; FIG. 14 is a diagram showing the correlation between a single note, a label string, and start and end times;
FIG. 15 is a diagram showing a minimum length 0 sound machine and its starting point distribution, FIG. 16 is a diagram showing a minimum length 4 sound machine and its time chart, and FIG. 17 is a diagram showing a minimum length 4 sound machine and its time chart.
Figure 18 shows a sound tree structure; Figure 18 is a flowchart representing the steps carried out in the training sound machine for performing phonetic matching;
Figure 20 shows the sequential steps of stack decoding; Figure 20 shows the likelihood vectors for word paths and likelihood envelopes;
FIG. 21 is a flowchart of the procedure for extending the found path, FIG. 22 is a flowchart of the operation of the stack decoder, FIG. 23 is a diagram of the phonemic sound machine, and FIG. 24 is a flowchart of the procedure for extending the found path. , Figures 25.1 and 25.2 are diagrams representing the phonemic sound grid, Figures 25.1 and 25.2 are diagrams representing the polling method of the present invention,
Figure 5 shows the combination of Figures 25.1 and 25.2, Figure 26 shows the label count distribution, and Figure 27 shows how each single note generates each label during the training period. Figure 28 is a diagram showing the number of times the word is generated, Figure 29 is a diagram showing the sound sequence that generates the word, and Figure 29 is
Figure 30 is a diagram showing the expected count value for a certain word for each label. Figure 30 is a diagram showing the vote value for each label and word. Figure 31 is a diagram showing the penalty value for each label and word. , FIG. 32 is a block diagram of the polling device of the present invention.

Claims (1)

[Claims] 1. Quantizing audio at predetermined small time intervals,
In addition to generating labels according to the quantized audio data and performing preprocessing for speech recognition, at least one
A Markov model having two state transitions and a label output probability in which each of the above labels is output in each of these state transitions is set for each note,
The input speech to be recognized is converted into the string with the label, and the string with the label is matched with the single note or the string of the single note with reference to the probability data of the Markov model, and based on this matching, the input speech is converted into the string with the label. In a speech recognition method that recognizes speech, before performing the above recognition, (a) for each of the above labels and each word in the vocabulary, the label is generated at each of the above minute intervals when the word is uttered; (b) accumulating the label generation likelihoods of a plurality of labels generated according to the input speech to be recognized for a predetermined word, with reference to a table displaying each label generation likelihood; and a step of determining whether the predetermined word is a candidate word of the input speech according to the accumulated value, and performing the recognition on the word determined to be the candidate word. Method. 2. Quantize the audio at predetermined small time intervals,
In addition to generating labels according to the quantized audio data and performing preprocessing for speech recognition, at least one
A Markov model having two state transitions and a label output probability in which each of the above labels is output in each of these state transitions is set for each note,
The input speech to be recognized is converted into the string with the label, and the string with the label is matched with the single note or the string of the single note with reference to the probability data of the Markov model, and based on this matching, the input speech is converted into the string with the label. In a speech recognition device that recognizes speech, (a) for each label in the label set and each word in the vocabulary, a first label generating likelihood that the word generates the label at each minute interval is determined. (b) for each of said labels in said label set and for each of said words in said vocabulary, means for forming a table of label failures in which said word does not generate said label for each of said microintervals; (c) combining the label generation likelihood and the label non-generation likelihood associated with a predetermined word for a label generated in response to input speech; , means for determining the likelihood that the word corresponds to the input speech, and determining whether the predetermined word is a candidate for the input speech to be recognized based on the likelihood of the means (c). A speech recognition device characterized in that the above-mentioned recognition is performed on candidate words.