CN110364142A - Phoneme of speech sound recognition methods and device, storage medium and electronic device - Google Patents
Phoneme of speech sound recognition methods and device, storage medium and electronic device Download PDFInfo
- Publication number
- CN110364142A CN110364142A CN201910578724.7A CN201910578724A CN110364142A CN 110364142 A CN110364142 A CN 110364142A CN 201910578724 A CN201910578724 A CN 201910578724A CN 110364142 A CN110364142 A CN 110364142A
- Authority
- CN
- China
- Prior art keywords
- phonetic feature
- feature
- phonetic
- phoneme
- speech
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 230000015654 memory Effects 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 11
- 210000005036 nerve Anatomy 0.000 claims description 11
- 238000010586 diagram Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 9
- 238000012549 training Methods 0.000 description 9
- 238000013528 artificial neural network Methods 0.000 description 8
- 230000007246 mechanism Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000004891 communication Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000003062 neural network model Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 235000013399 edible fruits Nutrition 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000000306 recurrent effect Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 230000008859 change Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 230000007787 long-term memory Effects 0.000 description 1
- 230000005389 magnetism Effects 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007306 turnover Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
- G10L15/16—Speech classification or search using artificial neural networks
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
- G10L2015/025—Phonemes, fenemes or fenones being the recognition units
Landscapes
- Engineering & Computer Science (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Machine Translation (AREA)
Abstract
The invention discloses a kind of phoneme of speech sound recognition methods and devices, storage medium and electronic device.Wherein, this method comprises: being extracted from the multiple speech frames being ranked up sequentially in time and one-to-one multiple first phonetic features of multiple speech frames；Determine that multiple crucial phonetic features, the probability for the phoneme that each key phonetic feature corresponds in set of phonemes are more than or equal to destination probability threshold value from multiple first phonetic features；Determine that phonetic feature set corresponding with each key phonetic feature, each phonetic feature set include one or more phonetic features adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features；Fusion Features are carried out to the phonetic feature in each phonetic feature set respectively, obtain multiple convergence voice features, the corresponding convergence voice feature of each phonetic feature set；Identify phoneme corresponding with each convergence voice feature respectively in set of phonemes.
Description
Technical field
The present invention relates to computer fields, in particular to a kind of phoneme of speech sound recognition methods and device, storage medium
And electronic device.
Background technique
Currently, main end-to-end modeling method to be applied is the other feature coding mode of frame level, example in acoustic model modeling
Such as, CTC (Connectionist Temporal Classification, the classification of connection timing) model.Which has one
Condition is unrelated to assume (i.e. current output is only related to input feature vector, unrelated with history output).
However, due to having relevance between voice data, although the unrelated hypothesis of above-mentioned condition obtains modeling
Acoustic model although there is succinctly stable advantage, recognition result accuracy rate is low, and recognition effect is poor.That is, phase
Phoneme of speech sound recognition methods in the technology of pass has that recognition result accuracy rate is low.
Summary of the invention
The embodiment of the present invention provides a kind of phoneme of speech sound recognition methods and device, storage medium and electronic device, at least
The phoneme of speech sound recognition methods of the solution in the related technology technical problem low there are recognition result accuracy rate.
According to an aspect of an embodiment of the present invention, a kind of phoneme of speech sound recognition methods is provided, comprising: from according to the time
It is extracted in multiple speech frames that sequence is ranked up and one-to-one multiple first phonetic features of multiple speech frames；From multiple
Multiple crucial phonetic features are determined in first phonetic feature, wherein each key phonetic feature corresponds in set of phonemes
The probability of one phoneme is more than or equal to destination probability threshold value；Determine phonetic feature corresponding with each key phonetic feature
Set, wherein each phonetic feature set include in corresponding crucial phonetic feature and multiple first phonetic features with it is corresponding
The adjacent one or more phonetic features of crucial phonetic feature；The phonetic feature in each phonetic feature set is carried out respectively
Fusion Features obtain multiple convergence voice features, wherein the corresponding convergence voice feature of each phonetic feature set；In sound
Phoneme corresponding with each convergence voice feature is identified respectively in element set.
According to another aspect of an embodiment of the present invention, a kind of phoneme of speech sound identification device is additionally provided, comprising: extract single
Member, for being extracted from the multiple speech frames being ranked up sequentially in time and multiple speech frames one-to-one multiple the
One phonetic feature；First determination unit, for determining multiple crucial phonetic features from multiple first phonetic features, wherein
The probability for the phoneme that each key phonetic feature corresponds in set of phonemes is more than or equal to destination probability threshold value；Second
Determination unit, for determining phonetic feature set corresponding with each key phonetic feature, wherein each phonetic feature set
Including one adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features or
Multiple phonetic features；Integrated unit is obtained for carrying out Fusion Features to the phonetic feature in each phonetic feature set respectively
Multiple convergence voice features, wherein the corresponding convergence voice feature of each phonetic feature set；Recognition unit, in sound
Phoneme corresponding with each convergence voice feature is identified respectively in element set.
Another aspect according to an embodiment of the present invention, additionally provides a kind of storage medium, and meter is stored in the storage medium
Calculation machine program, wherein the computer program is arranged to execute the above method when operation.
Another aspect according to an embodiment of the present invention, additionally provides a kind of electronic device, including memory, processor and deposits
Store up the computer program that can be run on a memory and on a processor, wherein above-mentioned processor passes through computer program and executes
Above-mentioned method.
In embodiments of the present invention, crucial phonetic feature is determined by the phonetic feature according to speech frame；It is determining and every
The corresponding phonetic feature set of a key phonetic feature, wherein each phonetic feature set includes corresponding crucial phonetic feature
And one or more phonetic features adjacent with corresponding crucial phonetic feature in multiple first phonetic features；To each voice
Phonetic feature in characteristic set carries out Fusion Features, obtains multiple convergence voice features；And it is identified respectively in set of phonemes
Phoneme corresponding with each convergence voice feature out, on the basis for determining crucial phonetic feature based on the other feature coding of frame level
On, phonetic feature section (phonetic feature set) is determined using crucial phonetic feature, it is (single so as to extract more accurately section
Member) grade another characteristic, and then the accuracy rate of recognition result can be improved, the phoneme of speech sound recognition methods solved in the related technology is deposited
In the low technical problem of recognition result accuracy rate.
Detailed description of the invention
The drawings described herein are used to provide a further understanding of the present invention, constitutes part of this application, this hair
Bright illustrative embodiments and their description are used to explain the present invention, and are not constituted improper limitations of the present invention.In the accompanying drawings:
Fig. 1 is a kind of schematic diagram of the application environment of phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 2 is a kind of flow diagram of optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 3 is a kind of schematic diagram of optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 4 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 5 is a kind of schematic diagram of optional CTC model according to an embodiment of the present invention；
Fig. 6 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 7 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Fig. 8 is a kind of schematic diagram of optional attention model according to an embodiment of the present invention；
Fig. 9 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Figure 10 is a kind of schematic diagram of voice data according to an embodiment of the present invention；
Figure 11 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Figure 12 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Figure 13 is the schematic diagram of another optional phoneme of speech sound recognition methods according to an embodiment of the present invention；
Figure 14 is a kind of structural schematic diagram of optional phoneme of speech sound identification device according to an embodiment of the present invention；
Figure 15 is a kind of structural schematic diagram of optional electronic device according to an embodiment of the present invention.
Specific embodiment
In order to enable those skilled in the art to better understand the solution of the present invention, below in conjunction in the embodiment of the present invention
Attached drawing, technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
The embodiment of a part of the invention, instead of all the embodiments.Based on the embodiments of the present invention, ordinary skill people
The model that the present invention protects all should belong in member's every other embodiment obtained without making creative work
It encloses.
It should be noted that description and claims of this specification and term " first " in above-mentioned attached drawing, "
Two " etc. be to be used to distinguish similar objects, without being used to describe a particular order or precedence order.It should be understood that using in this way
Data be interchangeable under appropriate circumstances, so as to the embodiment of the present invention described herein can in addition to illustrating herein or
Sequence other than those of description is implemented.In addition, term " includes " and " having " and their any deformation, it is intended that cover
Cover it is non-exclusive include, for example, the process, method, system, product or equipment for containing a series of steps or units are not necessarily limited to
Step or unit those of is clearly listed, but may include be not clearly listed or for these process, methods, product
Or other step or units that equipment is intrinsic.
According to an aspect of an embodiment of the present invention, a kind of phoneme of speech sound recognition methods is provided.Optionally, above-mentioned voice
Phoneme recognition method can be, but not limited to be applied in application environment as shown in Figure 1.As shown in Figure 1, above-mentioned phoneme of speech sound is known
Other method is related to terminal device 102 and passes through the interaction between network 104 and server 106.
Terminal device 102 can acquire or obtain from other equipment the multiple voices being ranked up sequentially in time
Frame, and multiple speech frames are sent to server 106 by network 104.Terminal device 102 can also acquire or from other equipment
Target speech data is obtained, and target speech data is sent to server 106 by network 104, by server 106 from target
Multiple speech frames are obtained in voice data.
Server 106 can extract and multiple speech frames after getting multiple speech frames from multiple speech frames
One-to-one multiple first phonetic features；Multiple crucial phonetic features are determined from multiple first phonetic features, wherein every
The probability for the phoneme that a key phonetic feature corresponds in set of phonemes is more than or equal to destination probability threshold value；It determines
Phonetic feature set corresponding with each key phonetic feature, wherein each phonetic feature set includes corresponding crucial voice
The one or more phonetic features adjacent with corresponding crucial phonetic feature in feature and multiple first phonetic features；It is right respectively
Phonetic feature in each phonetic feature set carries out Fusion Features, obtains multiple convergence voice features, wherein each voice is special
A corresponding convergence voice feature is closed in collection；Identify sound corresponding with each convergence voice feature respectively in set of phonemes
Element.
Optionally, in the present embodiment, above-mentioned terminal device can include but is not limited at least one of: mobile phone is put down
Plate computer, desktop computer etc..Above-mentioned network can include but is not limited at least one of: wireless network, cable network, wherein
The wireless network includes: the network of bluetooth, WIFI and other realization wireless communications, which may include: local area network, city
Domain net, wide area network etc..Above-mentioned server can include but is not limited at least one of: for using target nerve network model
The equipment that target sequence model is handled.Above-mentioned only a kind of example, the present embodiment do not do any restriction to this.
Optionally, in the present embodiment, as an alternative embodiment, as shown in Fig. 2, above-mentioned phoneme of speech sound identifies
The process of method may comprise steps of:
S202 is extracted one-to-one with multiple speech frames from the multiple speech frames being ranked up sequentially in time
Multiple first phonetic features；
S204 determines multiple crucial phonetic features from multiple first phonetic features, wherein each key phonetic feature
Probability corresponding to a phoneme in set of phonemes is more than or equal to destination probability threshold value；
S206 determines phonetic feature set corresponding with each key phonetic feature, wherein each phonetic feature set
Including one adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features or
Multiple phonetic features；
S208 carries out Fusion Features to the phonetic feature in each phonetic feature set respectively, obtains multiple convergence voices
Feature, wherein the corresponding convergence voice feature of each phonetic feature set；
S210 identifies phoneme corresponding with each convergence voice feature respectively in set of phonemes.
Optionally, above-mentioned phoneme of speech sound recognition methods can be by destination server execution, can be, but not limited to apply
In the tasks such as speech recognition, language translation.
For example, multiple speech frames are from the language to be translated for corresponding to first language (for example, Chinese) by taking language translation as an example
The speech frame got in sound data.As shown in figure 3, multiple first voices can be extracted from multiple speech frames by module one
Feature is determined multiple crucial phonetic features from multiple first phonetic features by module two, and exports Key Words to module three
Sound signature identification determines phonetic feature set corresponding with each key phonetic feature by module three, and respectively to each language
Phonetic feature in sound characteristic set carries out Fusion Features, is identified respectively in set of phonemes and each fusion language by module four
The corresponding phoneme of sound feature.After identifying each phoneme, is determined in voice data to be translated by the phoneme identified and include
Word (or sentence), by comprising word (or sentence) be translated as the word (or sentence) of second language.
Through this embodiment, on the basis of determining crucial phonetic feature based on the other feature coding of frame level, key is utilized
Phonetic feature determines phonetic feature section (phonetic feature set), to extract more accurately section (unit) grade another characteristic, determines
Phoneme corresponding with each phonetic feature section, there are recognition result accuracys rate for the phoneme of speech sound recognition methods for solving in the related technology
Low technical problem improves the accuracy rate of recognition result.
Above-mentioned voice recognition mode is explained below with reference to Fig. 2.
In step S202, extracted from the multiple speech frames being ranked up sequentially in time and multiple speech frames one
One corresponding multiple first phonetic features.
Multiple speech frames can be the speech frame got from target speech data.Above-mentioned target speech data can be
One Duan Yuyin of target duration, for example, the voice of one section of 2s.
Before the multiple speech frames obtained in target speech data, the available target voice number of destination server
According to.Above-mentioned target speech data, which can be, is sent to destination server by network by terminal, is also possible to by preserving target
The server of voice data is sent to destination server.Wherein, above-mentioned terminal can be the terminal for recording target speech data,
It can be the terminal for preserving target speech data, can also be other terminals that request handles target speech data.
Optionally, in the present embodiment, extracted from multiple speech frames it is multiple correspondingly with multiple speech frames
Before first phonetic feature, target speech data can be divided according to scheduled duration, obtain multiple unit frames；According to mesh
The period is marked, multiple speech frames are determined from multiple unit frames, wherein each speech frame includes one or more unit frames.
After getting target speech data, it can be obtained from target speech data using various ways above-mentioned multiple
Speech frame: target speech data is divided into multiple unit frames；Multiple speech frames are sampled out from multiple unit frames, alternatively, will
Multiple unit frames are combined, and obtain multiple speech frames.
The mode that target speech data is divided into multiple unit frames may is that according to scheduled duration to target speech data
It is divided, obtains multiple unit frames.Above-mentioned scheduled duration can satisfy following division condition: can recognize that specific voice
Feature.Above-mentioned scheduled duration can also meet following division condition: the quantity for the phonetic feature for including is less than or equal to 1.On
It states scheduled duration and can according to need and set, for example, it may be 10ms.By setting scheduled duration, it is ensured that can
It identifies phonetic feature, and leakage identification or wrong identification will not be caused since duration is too long.
For example, the voice data for being 2s for length, can be that scheduled duration draws the voice data according to 10ms
Point, obtain 200 unit frames.
Multiple unit frames are being obtained, can determine multiple speech frames from multiple unit frames according to target period,
In, each speech frame includes one or more unit frames.
In order to reduce the complexity of calculating, the efficiency of raising phoneme of speech sound identification can be sampled multiple unit frames,
Alternatively, being combined to multiple unit frames.For example, one can be extracted according to every N number of unit frame (target period is N number of unit frame)
The mode of a or multiple unit frames is sampled, and multiple speech frames are obtained.In another example can be one according to every M unit frame
The mode of group carries out unit frame combination, obtains multiple speech frames.
For example, for 200 unit frames for being divided the voice data of 2s for scheduled duration according to 10ms, it can
In a manner of extracting a unit frame according to every 2 unit frames, 100 speech frames are obtained, can also be taken out according to every 4 unit frames
The mode for taking a unit frame obtains 50 speech frames, can also carry out unit frame in such a way that every 4 unit frames are one group
Combined mode obtains 50 speech frames.
Through this embodiment, by being divided to obtain unit frame to voice data, and to the side that unit frame is sampled
Formula obtains speech frame, it is possible to reduce the computation complexity of phoneme of speech sound identification improves the efficiency of phoneme of speech sound identification.
After obtaining multiple speech frames, it is special that destination server can extract multiple first voices from multiple speech frames
Sign, wherein corresponded between multiple speech frames and multiple first phonetic features.
Identified from speech frame phonetic feature mode can there are many, for existing speech feature extraction mode, only
The phonetic feature to be extracted can be used for carrying out phoneme of speech sound identification, be used equally for the phoneme of speech sound identification side in the present embodiment
Method.
In order to improve the validity of the phonetic feature extracted, phonetic feature can be carried out using target nerve network model
Extraction.
Optionally, in the present embodiment, extracted from the multiple speech frames being ranked up sequentially in time with it is multiple
One-to-one multiple first phonetic features of speech frame may include: successively to be input to each speech frame in multiple speech frames
Target nerve network model, wherein target nerve network model is for extracting the first phonetic feature corresponding with each speech frame；
Obtain multiple first phonetic features of target nerve network model output.
Above-mentioned target nerve network model can be frame grade encoding device model (that is, the part Encoder), can be all kinds of
The model of deep-neural-network, can include but is not limited at least one of: multilayer LSTM (Long Short-Term
Memory, shot and long term memory network), for example, BiLSTM (two-way LSTM), UniLSTM (derivative LSTM)；Multilayer convolutional network；
FSMN (Feedforward Sequential Memory Networks, feed-forward type serial memorization network), TDNN (Time
Delay Neural Network, time-delay neural network).
For example, as shown in figure 4, each speech frame in multiple speech frames successively can be input to CNN
In (Convolution Neural Networks, convolutional neural networks), is extracted by CNN and exported corresponding with each speech frame
The first phonetic feature.
Through this embodiment, speech feature extraction is carried out by using neural network model, can according to need and carries out net
Network model training improves the accuracy and validity of speech feature extraction.
In step S204, multiple crucial phonetic features are determined from multiple first phonetic features, wherein each key
The probability that phonetic feature corresponds to a phoneme in set of phonemes is more than or equal to destination probability threshold value.
For each of extracting the first phonetic feature, which can be determined according to the first phonetic feature extracted
The corresponding probability with each phoneme in set of phonemes of sound feature.
Above-mentioned phoneme (phone) can be the element for forming each voice, be marked off according to the natural quality of language come
Minimum linguistic unit.It can be analyzed according to the articulation of syllable, a movement constitutes a phoneme.For Chinese, sound
Element can be divided into vowel and consonant, e.g., Chinese syllableThere is a phoneme,There are two phoneme,There are three phonemes.Into
When row phoneme recognition, the tone (for example, high and level tone, rising tone, upper sound, falling tone) in syllable can be identified, it can also be with nonrecognition sound
Tone in section.
For each first phonetic feature, the probability of each phoneme corresponded in set of phonemes and can be 1 (normalizing
Change processing).In whole the first phonetic features, the first phonetic feature of part due to comprising Limited information, may can not be true
Its fixed probability for corresponding to each phoneme in set of phonemes, these first phonetic features can be ignored；The first phonetic feature of part
Represented information is indefinite, correspond to set of phonemes in each phoneme probability be less than destination probability threshold value (for example,
80%), these first phonetic features are not belonging to crucial phonetic feature；Information represented by the first phonetic feature of part is clear,
Probability corresponding to a certain phoneme in set of phonemes is more than that (probability for being determined as a certain phoneme is greater than destination probability threshold value
80%) these first phonetic features, are determined as crucial phonetic feature.
The determination of crucial phonetic feature can be carried out using various ways.As long as the voice can be determined according to phonetic feature
Feature corresponds to the mode of each phoneme probability in set of phonemes, is used equally for the determination of crucial phonetic feature.
Optionally, in the present embodiment, determine that multiple crucial phonetic features can wrap from multiple first phonetic features
It includes: determining multiple peak locations from multiple first phonetic features using CTC model, wherein each peak location corresponding one
A key phonetic feature.
CTC model can be as shown in figure 5, CTC model includes an encoder (encoder), by x1…xTIt is sequentially inputted to
In encoder, and using Softmax function (normalization exponential function) to the output (h of encoderenc) handled, it obtains
Each input x (x1…xT) it is each y (y1…yT) probability (P (y1|x)…P(yT|x))。
CTC mainly solves traditional RNN, and (Recurrent Neural Network, Recognition with Recurrent Neural Network are that one kind is used for
The neural network of processing sequence data) in model, the correspondence problem of the sequence of annotated sequence and input.By in label symbol collection
In plus a blank character blank, be then labeled using RNN, can not determine some effectively export when then export blank
Symbol；A significant character is then exported when determining some effective unit enough, therefore, in CTC can obtain label (mark
Label) in significant character peak location.
For example, as shown in fig. 6, CNN after identifying multiple first phonetic features, can use CTC criterion, export more
A peak location, each peak location correspond to a crucial phonetic feature, and peak location is the mark of crucial phonetic feature.
Through this embodiment, the positioning that crucial phonetic feature is carried out using CTC model, does not need to mark in training pattern
The convenience that model training and model use can be improved in the boundary of each phoneme.
In step S206, phonetic feature set corresponding with each key phonetic feature is determined, wherein each voice
Characteristic set includes adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features
One or more phonetic features.
For each of determining crucial phonetic feature, phonetic feature corresponding with each crucial phonetic feature can be determined
Set.For current key phonetic feature, corresponding phonetic feature set includes: current key phonetic feature and multiple
The one or more phonetic features adjacent with current key phonetic feature in first phonetic feature.
It can be using the determining phonetic feature set corresponding with each key phonetic feature of various ways.For example, can incite somebody to action
One or more phonetic features in current key phonetic feature, multiple first phonetic features before and after current speech feature
It is determined as phonetic feature set corresponding with current key phonetic feature.In another example can be by current key phonetic feature, multiple
One or more phonetic features in first phonetic feature before current speech feature are determined as and current key phonetic feature pair
The phonetic feature set answered.For another example can be by current speech feature in current key phonetic feature, multiple first phonetic features
One or more phonetic features later are determined as phonetic feature set corresponding with current key phonetic feature.
Optionally, in the present embodiment, determine that phonetic feature set corresponding with each key phonetic feature can wrap
It includes: determining that the second phonetic feature corresponding with the current key phonetic feature in multiple crucial phonetic features and third voice are special
Sign, wherein the second phonetic feature be multiple first phonetic features in front of current key phonetic feature and with current key language
Sound feature non-conterminous first crucial phonetic feature, third phonetic feature is in multiple first phonetic features in current key language
After sound feature and with non-conterminous first crucial phonetic feature of current key phonetic feature；It determines and current key language
The corresponding current speech characteristic set of sound feature, wherein current speech characteristic set is the subclass of target voice characteristic set,
Target voice characteristic set includes the second phonetic feature, third phonetic feature and the second phonetic feature and third phonetic feature
Between the first phonetic feature.
For the current speech feature in multiple crucial phonetic features, can determine first before current speech feature and
With current key phonetic feature non-conterminous first crucial phonetic feature (the second phonetic feature) and current speech feature it
Afterwards and with current key phonetic feature non-conterminous first crucial phonetic feature (third phonetic feature), then by the second language
The first phonetic feature between sound feature, third phonetic feature and the second phonetic feature and third phonetic feature is determined as target
Phonetic feature set, then select from target voice characteristic set one or more phonetic features as with current key voice
The corresponding phonetic feature set of feature.
It should be noted that for first crucial phonetic feature, corresponding second phonetic feature is first first
Phonetic feature, for the last one crucial phonetic feature, corresponding third phonetic feature is the last one first phonetic feature.
For example, for 12 the first phonetic features corresponding with 12 speech frames, wherein crucial phonetic feature are as follows: the 3rd,
6,7 and 10 the first phonetic features.For the 1st crucial phonetic feature, corresponding target voice characteristic set are as follows: 1-6
A first phonetic feature.For the 2nd crucial phonetic feature, corresponding target voice characteristic set are as follows: 3-10 first
Phonetic feature.For the 3rd crucial phonetic feature, corresponding target voice characteristic set are as follows: 3-10 the first voices are special
Sign.For the 4th crucial phonetic feature, corresponding target voice characteristic set are as follows: 7-12 the first phonetic features.
Through this embodiment, by determining that the second phonetic feature corresponding with current key phonetic feature and third voice are special
Sign, determines target voice characteristic set corresponding with current key phonetic feature by the second phonetic feature and third phonetic feature,
It, can be to avoid not so as to determine phonetic feature set corresponding with current key phonetic feature by target voice characteristic set
With the influence between key phonetic feature, guarantee the accuracy of phoneme recognition.
In step S208, Fusion Features are carried out to the phonetic feature in each phonetic feature set respectively, are obtained multiple
Convergence voice feature, wherein the corresponding convergence voice feature of each phonetic feature set.
It, can be by the language in current speech characteristic set for the current speech characteristic set in multiple phonetic feature set
Sound feature carries out Fusion Features, obtains convergence voice feature corresponding with current speech characteristic set.
Fusion Features can be carried out using various ways, for example, can be using each voice to current speech characteristic set
Feature is weighted summation.The weight of each phonetic feature may be the same or different.For example, can be according to current speech
The distance between each phonetic feature and current key phonetic feature of characteristic set assign different power for different phonetic features
Weight, closer at a distance from current key phonetic feature, weight is bigger.
It should be noted that the distance between two phonetic features can according to speech frame corresponding with each phonetic feature it
Between distance be indicated, the distance between two speech frames can for two voice frame start positions, final position or appoint
Time difference between same position of anticipating.
Optionally, in the present embodiment, Fusion Features are carried out to the phonetic feature in each phonetic feature set respectively, obtained
It may include: that the phonetic feature in each phonetic feature set is input to target from attention respectively to multiple convergence voice features
In power layer, multiple convergence voice features are obtained, wherein target is used for from attention layer to the voice in each phonetic feature set
Feature is weighted summation, obtains convergence voice feature corresponding with each phonetic feature set.
It can be used from attention (Self-Attention) layer and the phonetic feature in each phonetic feature set carried out
Fusion Features, the feature of extraction unit length scale obtain convergence voice feature.
It is a kind of model using from attention mechanism from attention model.It is different from the attention mechanism of standard
Be: in the attention of standard, Query vector is related to output label, is returned in RNN by the label of output
It obtains；In self-attention, Query vector passes through transformation by encoder itself and generates.
For example, as shown in fig. 7, being exported according to multiple peak locations of CTC output and CNN more for from attention layer
A first phonetic feature determines phonetic feature section corresponding with each peak location, and exports the corresponding fusion of each phonetic feature section
Phonetic feature.For example, set of voice features corresponding with the 1st key phonetic feature is combined into: 1-6 the first phonetic features.The
1-6 the first phonetic features are input to from attention layer, are corresponding with the 1st crucial phonetic feature from the output of attention layer
Convergence voice feature.
Through this embodiment, section level characteristics are extracted using from attention layer, it is ensured that phonetic feature merges accurate
Property, and then improve the accuracy rate of phoneme of speech sound identification.
In step S210, phoneme corresponding with each convergence voice feature is identified respectively in set of phonemes.
After obtaining multiple convergence voice features, can according to obtained multiple convergence voice features, obtain with each
The corresponding phoneme of convergence voice feature.
For the present fusion phonetic feature in multiple convergence voice features, can be obtained according to present fusion phonetic feature
The current convergence voice feature corresponds to the probability of each phoneme in set of phonemes, and is corresponded to according to present fusion phonetic feature
The probability of each phoneme in set of phonemes determines phoneme corresponding with each convergence voice feature.
Optionally, in the present embodiment, sound corresponding with each convergence voice feature is identified respectively in set of phonemes
Element may include: the decoder that each convergence voice feature is sequentially inputted to target attention model, obtain merging with each
The corresponding phoneme of phonetic feature, wherein above-mentioned decoder is used for according at least to present fusion phonetic feature currently entered and makes
The previous phoneme handled with previous phonetic feature of the decoder to present fusion phonetic feature, obtain with currently
The corresponding current phoneme of convergence voice feature.
Attention is a kind of for promoting the mechanism of the effect of the Encoder+Decoder model based on RNN
(Mechanism), commonly referred to as Attention Mechanism.Attention Mechanism can be applied to machine and turn over
It translates, speech recognition, many fields such as image labeling (Image Caption).Attention is imparted to model and is distinguished discrimination
Ability, for example, assigning different weights in machine translation, speech recognition application for each word in sentence, making neural network
The study of model becomes more flexible (soft), while Attention itself can be used as a kind of alignment relation, and interpretation is defeated
Enter/export the alignment relation between sentence, what knowledge interpretation model has acquired on earth.
The structure of attention model can be as shown in Figure 8.Wherein, x1…xTFor the input of encoder, hencFor encoder
Output；For attention layers of a upper outputs, (a upper input for attention model is xu-1), cuFor attention layer
(this input of attention model is x to this state outputu), yu-1It is exported for one on attention model,For decoding
This output of device, P (yu|yu-1,…,y0, x) and it is this output of attention model.
Decoder (decoder) network in target attention (Attention) model can be used to determine and each melt
Close the corresponding phoneme of phonetic feature.Above-mentioned target attention model can be standard Attention model, be also possible to improved
Attention model, as long as sound corresponding with each convergence voice feature can be obtained according to multiple convergence voice features of input
The network model of element, is used equally for phoneme corresponding with each convergence voice feature constant current journey really.
For example, as shown in figure 9, the multiple convergence voice features exported from attention layer can be input to attention model
Decoder in, by the decoder according to the present fusion phonetic feature of input and sound corresponding with convergence voice feature before
Element determines phoneme corresponding with present fusion phonetic feature.
Through this embodiment, the knowledge of phoneme corresponding with each convergence voice feature is carried out using the decoder of attention model
Not, the accuracy rate of phoneme of speech sound identification can be improved.
It, can be according to identifying after identifying phoneme corresponding with each convergence voice feature respectively in set of phonemes
Multiple phonemes, obtain phonotactics corresponding with multiple speech frames.
Since the same phoneme likely corresponds to multiple speech frames, it can in the multiple crucial phonetic features identified
The case where same phoneme capable of being corresponded to there are at least two crucial phonetic features.
For example, as shown in Figure 10, for " hello ", containing 5 phonemes " n ", " i ", " h ", " a ", " o ", correspond to 12
A speech frame, wherein " n " corresponds to the 1-4 speech frame, and " i " corresponds to the 5-7 speech frame, and " h " corresponds to 8-9
Speech frame, " a " correspond to the 10-11 speech frame, and " h " corresponds to the 12nd speech frame.For " n ", the Key Words that identify
Sound feature is the first phonetic feature corresponding with the 3rd, 4 speech frame, for other phonemes, the crucial phonetic feature that identifies
Only one, then, the group of the phoneme corresponding with each convergence voice feature of final output is combined into " nnihao ".
Optionally, in the present embodiment, sound corresponding with each convergence voice feature is identified respectively in set of phonemes
After element, phoneme corresponding with each convergence voice feature can be combined according to the language form belonging to set of phonemes,
It obtaining target and shows information, wherein target shows that information is one or more syllables corresponding with multiple speech frames, alternatively, with
The corresponding one or more words of multiple speech frames；Target is shown that information is output to display equipment and shows.
While the multiple phonemes identified, each syllable can be determined.It, can according to the rule of different language type
To merge the phoneme recognition result for corresponding to identical phoneme, one or more syllables are obtained, and according to different language class
The rule of type determines one or more words corresponding with obtained one or more syllables.
One or more syllables corresponding with multiple speech frames are being obtained, alternatively, after one or more words, it can be with mesh
The mode of mark display information is exported to be shown to display equipment (for example, terminal device).
Through this embodiment, multiple phonemes that the language form according to belonging to set of phonemes will identify that be determined as one or
Multiple syllables, alternatively, one or more words, and by showing that equipment is shown, it can clearly show that out phoneme recognition knot
Fruit improves user experience.
Above-mentioned phoneme of speech sound recognition methods is illustrated below with reference to optional example.In this example, the first language is extracted
Sound feature uses deep layer convolutional neural networks model, carries out Fusion Features and uses from attention layer, identifies and merge
The corresponding phoneme of phonetic feature uses the decoder of standard attention model.
Two kinds of end-to-end modeling methods can be applied in acoustic model modeling: one is CTC；Another kind is
Attention.Main in CTC model only includes an encoder (encoder), the i.e. other feature coding module of frame level, has letter
Clean stable advantage, shortcoming is unrelated there are a condition it is assumed that i.e. current output is only related to input feature vector, with history
It exports unrelated.Attention model have two main modulars of encoder and decoder (decoder), output not only with it is defeated
Enter feature in relation to also related to history output, it is more perfect than CTC on probabilistic model.Meanwhile Attention can capture it is longer
The feature of range, is not limited by before and after frames.
The combination of two kinds of modeling patterns can be combined two methods by multitask training frame, as shown in figure 11,
Encoder module is shared, optimizes the loss function of an interpolation in training, shown in loss function such as formula (1):
LMTL=λ LCTC+(1-λ)LAttention (1)
Wherein, LMTLFor the loss function after combination, LCTCFor the loss function of CTC, LAttentionFor Attention model
Loss function.
However, in such a way that multitask training frame combines two methods, CTC and Attention output unit collection
Closing must be identical, the unit range information that Attention can not be provided using CTC, and CTC and Attention due to one
Frame level not Shu Chu one unit rank export, need specially treated convergence strategy.
Phoneme of speech sound recognition methods in this example is a kind of Acoustic Modeling method, in conjunction with existing CTC, ATTENTION,
The end-to-end modeling technique of Self-Attention, on the basis of the peak location that CTC model provides, several lists in effective use front and back
The bounds of member first extract more accurate unit rank length characteristic using Self-attention layers, and then use standard
Attention Decoder layer, can further repair mistake on the basis of CTC, reach more preferably recognition accuracy.
As shown in figure 12, modeling corresponding to the phoneme of speech sound recognition methods in this example can be divided into following four
Module: module one, frame grade encoding device model；Module two, pronunciation unit boundary and position discrimination module；Module three, section are (single
Member) level characteristics coder module；Module four, decoder (output unit differentiation) module.
For frame grade encoding device model, all kinds of deep-neural-network models can be used, for example, multilayer LSTM, multilayer
Convolutional network, FSMN or TDNN network.For pronunciation unit boundary and position discrimination module, CTC criterion can be used, it can be defeated
Pronunciation unit peak location out.For section (unit) level characteristics coder module, Self-attention layers can be used,
The feature of self-attention network extraction unit length scale is used in the range of the covering each N number of unit in left and right.For hair
Sound unit differentiates output module, can be using the Decoder network in standard Attention model.
The pronunciation unit set of module two and the output unit set of module four can be different, if pronunciation unit set is using upper
Hereafter relevant phoneme (context-dependent phoneme), output unit set use syllable (syllable).
As shown in figure 13, it is exported for frame grade encoding device model for encoder output layers, wherein dark circle represents CTC
The spike of effective label under criterion；Self-attention layers are extracted the certain unit in left and right from attention mechanism by unsupervised
More advanced feature (be in such as figure around each unit range) in bounds；In the section that self-attention layers are extracted
On the basis of (unit) level characteristics, sentencing for final output unit is further carried out using the decoder of standard attention
Not.
By this example, by self-attention layers, section (unit) is extracted using the unit range information that CTC is provided
Level characteristics, by introducing self-attention layers among CTC and attention, so that the output of attention is disobeyed
Rely the output with original CTC, makes model have the ability to repair the mistake that inserts and delete introduced in CTC model, and final unified
It is exported by the Decoder layer of Attention, without the concern for the convergence strategy with CTC, improves the convenience of processing.
It should be noted that for the various method embodiments described above, for simple description, therefore, it is stated as a series of
Combination of actions, but those skilled in the art should understand that, the present invention is not limited by the sequence of acts described because
According to the present invention, some steps may be performed in other sequences or simultaneously.Secondly, those skilled in the art should also know
It knows, the embodiments described in the specification are all preferred embodiments, and related actions and modules is not necessarily of the invention
It is necessary.
Other side according to an embodiment of the present invention additionally provides a kind of for implementing above-mentioned phoneme of speech sound recognition methods
Phoneme of speech sound identification device, as shown in figure 14, which includes:
(1) extraction unit 1402, for extracted from the multiple speech frames being ranked up sequentially in time with it is multiple
One-to-one multiple first phonetic features of speech frame；
(2) first determination units 1404, for determining multiple crucial phonetic features from multiple first phonetic features,
In, the probability for the phoneme that each key phonetic feature corresponds in set of phonemes is more than or equal to destination probability threshold value；
(3) second determination units 1406, for determining phonetic feature set corresponding with each key phonetic feature,
In, each phonetic feature set include in corresponding crucial phonetic feature and multiple first phonetic features with corresponding Key Words
The adjacent one or more phonetic features of sound feature；
(4) integrated unit 1408 are obtained for carrying out Fusion Features to the phonetic feature in each phonetic feature set respectively
To multiple convergence voice features, wherein the corresponding convergence voice feature of each phonetic feature set；
(5) recognition unit 1410, for identifying sound corresponding with each convergence voice feature respectively in set of phonemes
Element.
Optionally, above-mentioned phoneme of speech sound identification device can be by destination server execution, can be, but not limited to apply
In the tasks such as speech recognition, language translation.
Optionally, extraction unit 1402 can be used for executing above-mentioned steps S202, and the first determination unit 1404 can be used for
Above-mentioned steps S204 is executed, the second determination unit 1406 can be used for executing above-mentioned steps S206, and integrated unit 1408 can be used
In executing above-mentioned steps S208, recognition unit 1410 can be used for executing above-mentioned steps S210.
Through this embodiment, on the basis of determining crucial phonetic feature based on the other feature coding of frame level, key is utilized
Phonetic feature determines phonetic feature section (phonetic feature set), to extract more accurately section (unit) grade another characteristic, determines
Phoneme corresponding with each phonetic feature section, there are recognition result accuracys rate for the phoneme of speech sound recognition methods for solving in the related technology
Low technical problem improves the accuracy rate of recognition result.
As a kind of optional embodiment, above-mentioned apparatus further include:
(1) division unit, for being extracted and multiple languages from the multiple speech frames being ranked up sequentially in time
Before one-to-one multiple first phonetic features of sound frame, target speech data is divided according to scheduled duration, is obtained more
A unit frame；
(2) determination unit, for determining multiple speech frames from multiple unit frames according to target period, wherein each
Speech frame includes one or more unit frames.
Through this embodiment, by being divided to obtain unit frame to voice data, and to the side that unit frame is sampled
Formula obtains speech frame, it is possible to reduce the computation complexity of phoneme of speech sound identification improves the efficiency of phoneme of speech sound identification.
As a kind of optional embodiment, extraction unit 1402 includes:
(1) first input module, for each speech frame in multiple speech frames to be successively input to target nerve network
Model, wherein target nerve network model is for extracting the first phonetic feature corresponding with each speech frame；
(2) module is obtained, for obtaining multiple first phonetic features of target nerve network model output.
Through this embodiment, speech feature extraction is carried out by using neural network model, can according to need and carries out net
Network model training improves the accuracy and validity of speech feature extraction.
As a kind of optional embodiment, the first determination unit 1404 includes:
First determining module is more for being determined from multiple first phonetic features using connection timing classification CTC model
A peak location, wherein corresponding one crucial phonetic feature of each peak location.
Through this embodiment, the positioning that crucial phonetic feature is carried out using CTC model, does not need to mark in training pattern
The convenience that model training and model use can be improved in the boundary of each phoneme.
As a kind of optional embodiment, the second determination unit 1406 includes:
(1) second determining module, it is corresponding with the current key phonetic feature in multiple crucial phonetic features for determining
The second phonetic feature and third phonetic feature, wherein the second phonetic feature be multiple first phonetic features in current key
Before phonetic feature and with non-conterminous first crucial phonetic feature of current key phonetic feature, third phonetic feature is more
In a first phonetic feature after current key phonetic feature and with non-conterminous first key of current key phonetic feature
Phonetic feature；
(2) third determining module, for determining current speech characteristic set corresponding with current key phonetic feature,
In, current speech characteristic set is the subclass of target voice characteristic set, and target voice characteristic set includes that the second voice is special
The first phonetic feature between sign, third phonetic feature and the second phonetic feature and third phonetic feature.
Through this embodiment, by determining that the second phonetic feature corresponding with current key phonetic feature and third voice are special
Sign, determines target voice characteristic set corresponding with current key phonetic feature by the second phonetic feature and third phonetic feature,
It, can be to avoid not so as to determine phonetic feature set corresponding with current key phonetic feature by target voice characteristic set
With the influence between key phonetic feature, guarantee the accuracy of phoneme recognition.
As a kind of optional embodiment, integrated unit 1408 includes:
(1) input module, for the phonetic feature in each phonetic feature set to be input to target from attention respectively
In layer, multiple convergence voice features are obtained, wherein target is used for special to the voice in each phonetic feature set from attention layer
Sign is weighted summation, obtains convergence voice feature corresponding with each phonetic feature set.
Through this embodiment, section level characteristics are extracted using from attention layer, it is ensured that phonetic feature merges accurate
Property, and then improve the accuracy rate of phoneme of speech sound identification.
As a kind of optional embodiment, recognition unit 1410 includes:
(1) second input module, for each convergence voice feature to be sequentially inputted to the decoding of target attention model
Device obtains phoneme corresponding with each convergence voice feature, wherein decoder is used for according at least to present fusion currently entered
Phonetic feature and the previous sound handled using previous phonetic feature of the decoder to present fusion phonetic feature
Element obtains current phoneme corresponding with present fusion phonetic feature.
Through this embodiment, the knowledge of phoneme corresponding with each convergence voice feature is carried out using the decoder of attention model
Not, the accuracy rate of phoneme of speech sound identification can be improved.
As a kind of optional embodiment, above-mentioned apparatus further include:
(1) assembled unit, for identified respectively in set of phonemes phoneme corresponding with each convergence voice feature it
Afterwards, the language form according to belonging to set of phonemes is combined phoneme corresponding with each convergence voice feature, obtains target
Show information, wherein target shows that information is one or more syllables corresponding with multiple speech frames, alternatively, with multiple voices
The corresponding one or more words of frame；
(2) output unit, for target to be shown that information is output to display equipment and shows.
Through this embodiment, multiple phonemes that the language form according to belonging to set of phonemes will identify that be determined as one or
Multiple syllables, alternatively, one or more words, and by showing that equipment is shown, it can clearly show that out phoneme recognition knot
Fruit improves user experience.
The another aspect of embodiment according to the present invention, additionally provides a kind of storage medium, is stored in the storage medium
Computer program, wherein the computer program is arranged to execute the step in any of the above-described embodiment of the method when operation.
Optionally, in the present embodiment, above-mentioned storage medium can be set to store by executing based on following steps
Calculation machine program:
S1 is extracted more correspondingly with multiple speech frames from the multiple speech frames being ranked up sequentially in time
A first phonetic feature；
S2 determines multiple crucial phonetic features from multiple first phonetic features, wherein each key phonetic feature pair
Should the probability of a phoneme in set of phonemes be more than or equal to destination probability threshold value；
S3 determines phonetic feature set corresponding with each key phonetic feature, wherein each phonetic feature set packet
Include adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features one or more
A phonetic feature；
S4 carries out Fusion Features to the phonetic feature in each phonetic feature set respectively, and it is special to obtain multiple convergence voices
Sign, wherein the corresponding convergence voice feature of each phonetic feature set；
S5 identifies phoneme corresponding with each convergence voice feature respectively in set of phonemes.
Optionally, in the present embodiment, those of ordinary skill in the art will appreciate that in the various methods of above-described embodiment
All or part of the steps be that the relevant hardware of terminal device can be instructed to complete by program, the program can store in
In one computer readable storage medium, storage medium may include: flash disk, read-only memory (Read-Only Memory,
ROM), random access device (Random Access Memory, RAM), disk or CD etc..
Another aspect according to an embodiment of the present invention additionally provides a kind of for implementing above-mentioned phoneme of speech sound recognition methods
Electronic device, as shown in figure 15, which includes: processor 1502, memory 1504, transmitting device 1506 etc..It should
Computer program is stored in memory, which is arranged to execute the implementation of any of the above-described method by computer program
Step in example.
Optionally, in the present embodiment, above-mentioned electronic device can be located in multiple network equipments of computer network
At least one network equipment.
Optionally, in the present embodiment, above-mentioned processor can be set to execute following steps by computer program:
S1 is extracted more correspondingly with multiple speech frames from the multiple speech frames being ranked up sequentially in time
A first phonetic feature；
S2 determines multiple crucial phonetic features from multiple first phonetic features, wherein each key phonetic feature pair
Should the probability of a phoneme in set of phonemes be more than or equal to destination probability threshold value；
S3 determines phonetic feature set corresponding with each key phonetic feature, wherein each phonetic feature set packet
Include adjacent with corresponding crucial phonetic feature in corresponding crucial phonetic feature and multiple first phonetic features one or more
A phonetic feature；
S4 carries out Fusion Features to the phonetic feature in each phonetic feature set respectively, and it is special to obtain multiple convergence voices
Sign, wherein the corresponding convergence voice feature of each phonetic feature set；
S5 identifies phoneme corresponding with each convergence voice feature respectively in set of phonemes.
Optionally, it will appreciated by the skilled person that structure shown in figure 15 is only to illustrate, electronic device can also
To be smart phone (such as Android phone, iOS mobile phone), tablet computer, palm PC and mobile internet device
The terminal devices such as (Mobile Internet Devices, MID), PAD.Figure 15 it does not make to the structure of above-mentioned electronic device
At restriction.For example, electronic device may also include than shown in Figure 15 more perhaps less component (such as network interface) or
With the configuration different from shown in Figure 15.
Wherein, memory 1504 can be used for storing software program and module, such as the phoneme of speech sound in the embodiment of the present invention
Recognition methods and the corresponding program instruction/module of device, the software that processor 1502 is stored in memory 1504 by operation
Program and module identify thereby executing various function application and phoneme of speech sound, that is, realize above-mentioned phoneme of speech sound recognition methods.
Memory 1504 may include high speed random access memory, can also include nonvolatile memory, as one or more magnetism is deposited
Storage device, flash memory or other non-volatile solid state memories.In some instances, memory 1504 can further comprise phase
The memory remotely located for processor 1502, these remote memories can pass through network connection to terminal.Above-mentioned network
Example include but is not limited to internet, intranet, local area network, mobile radio communication and combinations thereof.
Above-mentioned transmitting device 1506 is used to that data to be received or sent via a network.Above-mentioned network specific example
It may include cable network and wireless network.In an example, transmitting device 1506 includes a network adapter (Network
Interface Controller, NIC), can be connected by cable with other network equipments with router so as to interconnection
Net or local area network are communicated.In an example, transmitting device 1506 is radio frequency (Radio Frequency, RF) module,
For wirelessly being communicated with internet.
The serial number of the above embodiments of the invention is only for description, does not represent the advantages or disadvantages of the embodiments.
If the integrated unit in above-described embodiment is realized in the form of SFU software functional unit and as independent product
When selling or using, it can store in above-mentioned computer-readable storage medium.Based on this understanding, skill of the invention
Substantially all or part of the part that contributes to existing technology or the technical solution can be with soft in other words for art scheme
The form of part product embodies, which is stored in a storage medium, including some instructions are used so that one
Platform or multiple stage computers equipment (can be personal computer, server or network equipment etc.) execute each embodiment side of the present invention
The all or part of the steps of method.
In the above embodiment of the invention, it all emphasizes particularly on different fields to the description of each embodiment, does not have in some embodiment
The part of detailed description, reference can be made to the related descriptions of other embodiments.
In several embodiments provided herein, it should be understood that disclosed client, it can be by others side
Formula is realized.Wherein, the apparatus embodiments described above are merely exemplary, such as the division of unit, and only one kind is patrolled
Volume function division, there may be another division manner in actual implementation, such as multiple units or components can combine or can be with
It is integrated into another system, or some features can be ignored or not executed.Another point, it is shown or discussed mutual
Coupling, direct-coupling or communication connection can be through some interfaces, the indirect coupling or communication connection of unit or module, can
To be electrically or in the form of others.
Unit may or may not be physically separated as illustrated by the separation member, shown as a unit
Component may or may not be physical unit, it can and it is in one place, or may be distributed over multiple networks
On unit.It can some or all of the units may be selected to achieve the purpose of the solution of this embodiment according to the actual needs.
It, can also be in addition, the functional units in various embodiments of the present invention may be integrated into one processing unit
It is that each unit physically exists alone, can also be integrated in one unit with two or more units.Above-mentioned integrated list
Member both can take the form of hardware realization, can also realize in the form of software functional units.
The above is only the preferred embodiment of the present invention, it is noted that those skilled in the art are come
It says, various improvements and modifications may be made without departing from the principle of the present invention, these improvements and modifications also should be regarded as
Protection scope of the present invention.
Claims (15)
1. a kind of phoneme of speech sound recognition methods characterized by comprising
It is extracted from the multiple speech frames being ranked up sequentially in time multiple correspondingly with the multiple speech frame
First phonetic feature；
Multiple crucial phonetic features are determined from the multiple first phonetic feature, wherein each crucial phonetic feature
Probability corresponding to a phoneme in set of phonemes is more than or equal to destination probability threshold value；
Determine phonetic feature set corresponding with each crucial phonetic feature, wherein each phonetic feature set
Including in the corresponding crucial phonetic feature and the multiple first phonetic feature with the corresponding crucial phonetic feature
Adjacent one or more phonetic features；
Fusion Features are carried out to the phonetic feature in each phonetic feature set respectively, obtain multiple convergence voice features,
Wherein, the corresponding convergence voice feature of each phonetic feature set；
Identify phoneme corresponding with each convergence voice feature respectively in the set of phonemes.
2. the method according to claim 1, wherein from the multiple language being ranked up sequentially in time
It is extracted in sound frame with before one-to-one the multiple first phonetic feature of the multiple speech frame, the method is also wrapped
It includes:
Target speech data is divided according to scheduled duration, obtains multiple unit frames；
According to target period, the multiple speech frame is determined from the multiple unit frame, wherein each speech frame packet
Include one or the multiple unit frame.
3. the method according to claim 1, wherein from the multiple voice being ranked up sequentially in time
It is extracted in frame and includes: with one-to-one the multiple first phonetic feature of the multiple speech frame
Each speech frame in the multiple speech frame is successively input to target nerve network model, wherein the target mind
Through network model for extracting first phonetic feature corresponding with each speech frame；
Obtain the multiple first phonetic feature of the target nerve network model output.
4. the method according to claim 1, wherein being determined from the multiple first phonetic feature described more
It is a key phonetic feature include:
Multiple peak locations are determined from the multiple first phonetic feature using connection timing classification CTC model, wherein every
Corresponding one of a peak location crucial phonetic feature.
5. the method according to claim 1, wherein determining institute corresponding with each crucial phonetic feature
Predicate sound characteristic set includes:
Determine the second phonetic feature corresponding with the current key phonetic feature in the multiple crucial phonetic feature and third
Phonetic feature, wherein second phonetic feature is in the multiple first phonetic feature in the current key phonetic feature
Before and with non-conterminous first crucial phonetic feature of the current key phonetic feature, the third phonetic feature is institute
It states after the current key phonetic feature and non-conterminous with the current key phonetic feature in multiple first phonetic features
First crucial phonetic feature；
Determine current speech characteristic set corresponding with the current key phonetic feature, wherein the current speech feature
Collection is combined into the subclass of target voice characteristic set, and the target voice characteristic set includes second phonetic feature, described
First phonetic feature between third phonetic feature and second phonetic feature and the third phonetic feature.
6. the method according to claim 1, wherein special to the voice in each phonetic feature set respectively
Sign carries out Fusion Features, and obtaining the multiple convergence voice feature includes:
The phonetic feature in each phonetic feature set is input to target from attention layer respectively, is obtained the multiple
Convergence voice feature, wherein the target from attention layer be used for the phonetic feature in each phonetic feature set into
Row weighted sum obtains convergence voice feature corresponding with each phonetic feature set.
7. the method according to claim 1, wherein identified respectively in the set of phonemes with it is each described
The corresponding phoneme of convergence voice feature includes:
Each convergence voice feature is sequentially inputted in the decoder of target attention model, is obtained and each institute
State the corresponding phoneme of convergence voice feature, wherein the decoder is used for special according at least to present fusion voice currently entered
Seeking peace, it is previous to be handled using previous phonetic feature of the decoder to the present fusion phonetic feature
Phoneme obtains current phoneme corresponding with present fusion phonetic feature.
8. method according to any one of claim 1 to 7, which is characterized in that identified respectively in the set of phonemes
Out after phoneme corresponding with each convergence voice feature, the method also includes:
According to language form belonging to the set of phonemes, group is carried out to phoneme corresponding with each convergence voice feature
It closes, obtains target and show information, wherein the target shows that information is one or more sounds corresponding with the multiple speech frame
Section, alternatively, one or more word corresponding with the multiple speech frame；
The target is shown that information is output to display equipment and shows.
9. a kind of phoneme of speech sound identification device characterized by comprising
Extraction unit, for being extracted from the multiple speech frames being ranked up sequentially in time and the multiple speech frame one
One corresponding multiple first phonetic features；
First determination unit, for determining multiple crucial phonetic features from the multiple first phonetic feature, wherein each
The probability for the phoneme that the key phonetic feature corresponds in set of phonemes is more than or equal to destination probability threshold value；
Second determination unit, for determining phonetic feature set corresponding with each crucial phonetic feature, wherein each
The phonetic feature set include in the corresponding crucial phonetic feature and the multiple first phonetic feature with it is corresponding
The adjacent one or more phonetic features of the key phonetic feature；
Integrated unit obtains multiple for carrying out Fusion Features to the phonetic feature in each phonetic feature set respectively
Convergence voice feature, wherein each corresponding convergence voice feature of the phonetic feature set；
Recognition unit, for identifying phoneme corresponding with each convergence voice feature respectively in the set of phonemes.
10. device according to claim 9, which is characterized in that first determination unit includes:
First determining module is more for being determined from the multiple first phonetic feature using connection timing classification CTC model
A peak location, wherein corresponding one of each peak location crucial phonetic feature.
11. device according to claim 9, which is characterized in that second determination unit includes:
Second determining module, for determining corresponding with the current key phonetic feature in the multiple crucial phonetic feature the
Two phonetic features and third phonetic feature, wherein second phonetic feature is in the multiple first phonetic feature described
Before current key phonetic feature and with non-conterminous first crucial phonetic feature of the current key phonetic feature, it is described
Third phonetic feature be the multiple first phonetic feature in after the current key phonetic feature and with the current pass
Key phonetic feature non-conterminous first crucial phonetic feature；
Third determining module, for determining current speech characteristic set corresponding with the current key phonetic feature, wherein
The current speech characteristic set is the subclass of target voice characteristic set, and the target voice characteristic set includes described the
It is described between two phonetic features, the third phonetic feature and second phonetic feature and the third phonetic feature
First phonetic feature.
12. device according to claim 9, which is characterized in that the integrated unit includes:
Input module, for the phonetic feature in each phonetic feature set to be input to target from attention layer respectively
In, obtain the multiple convergence voice feature, wherein the target is used for from attention layer to each phonetic feature set
In phonetic feature be weighted summation, obtain convergence voice feature corresponding with each phonetic feature set.
13. the device according to any one of claim 9 to 12, which is characterized in that described device further include:
Assembled unit, for identified respectively in the set of phonemes phoneme corresponding with each convergence voice feature it
Afterwards, the language form according to belonging to the set of phonemes is combined phoneme corresponding with each convergence voice feature,
It obtaining target and shows information, wherein the target shows that information is one or more syllables corresponding with the multiple speech frame,
Alternatively, one or more word corresponding with the multiple speech frame；
Output unit, for the target to be shown that information is output to display equipment and shows.
14. a kind of storage medium, which is characterized in that be stored with computer program in the storage medium, wherein the computer
Program is arranged to execute method described in any one of claim 1 to 8 when operation.
15. a kind of electronic device, including memory and processor, which is characterized in that be stored with computer journey in the memory
Sequence, the processor are arranged to execute side described in any one of claim 1 to 8 by the computer program
Method.
Priority Applications (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910775838.0A CN110534092B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775376.2A CN110428809B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910578724.7A CN110364142B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775364.XA CN110473518B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910741010.3A CN110335592B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910578724.7A CN110364142B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Related Child Applications (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775364.XA Division CN110473518B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910741010.3A Division CN110335592B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775376.2A Division CN110428809B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775838.0A Division CN110534092B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110364142A true CN110364142A (en) | 2019-10-22 |
| CN110364142B CN110364142B (en) | 2022-03-25 |
Family
ID=68216113
Family Applications (4)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775364.XA Active CN110473518B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775838.0A Active CN110534092B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910578724.7A Active CN110364142B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775376.2A Active CN110428809B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Family Applications Before (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775364.XA Active CN110473518B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
| CN201910775838.0A Active CN110534092B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910775376.2A Active CN110428809B (en) | 2019-06-28 | 2019-06-28 | Speech phoneme recognition method and device, storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (4) | CN110473518B (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853669A (en) * | 2019-11-08 | 2020-02-28 | 腾讯科技（深圳）有限公司 | Audio identification method, device and equipment |
| CN110992929A (en) * | 2019-11-26 | 2020-04-10 | 苏宁云计算有限公司 | Voice keyword detection method, device and system based on neural network |
| CN111489737A (en) * | 2020-04-13 | 2020-08-04 | 深圳市友杰智新科技有限公司 | Voice command recognition method and device, storage medium and computer equipment |
| CN111627457A (en) * | 2020-05-13 | 2020-09-04 | 广州国音智能科技有限公司 | Voice separation method, system and computer readable storage medium |
| CN111639157A (en) * | 2020-05-13 | 2020-09-08 | 广州国音智能科技有限公司 | Audio marking method, device, equipment and readable storage medium |
| CN114724544A (en) * | 2022-04-13 | 2022-07-08 | 北京百度网讯科技有限公司 | Voice chip, voice recognition method, device and equipment and intelligent automobile |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111276131B (en) * | 2020-01-22 | 2021-01-12 | 厦门大学 | Multi-class acoustic feature integration method and system based on deep neural network |
| CN112750425B (en) * | 2020-01-22 | 2023-11-03 | 腾讯科技（深圳）有限公司 | Speech recognition method, device, computer equipment and computer readable storage medium |
| CN111489738B (en) * | 2020-04-13 | 2021-02-23 | 深圳市友杰智新科技有限公司 | Feature extraction method and voice command identification method based on multi-head attention mechanism |
| CN112037798B (en) * | 2020-09-18 | 2022-03-01 | 中科极限元(杭州)智能科技股份有限公司 | Voice recognition method and system based on trigger type non-autoregressive model |
| CN112382278B (en) * | 2020-11-18 | 2021-08-17 | 北京百度网讯科技有限公司 | Streaming voice recognition result display method and device, electronic equipment and storage medium |
| CN112397093B (en) * | 2020-12-04 | 2024-02-27 | 中国联合网络通信集团有限公司 | Voice detection method and device |
| CN113449590B (en) * | 2021-05-14 | 2022-10-28 | 网易（杭州）网络有限公司 | Speaking video generation method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2389219A (en) * | 2002-05-29 | 2003-12-03 | L Labs Inc | User interface, system and method for automatically labelling phonic symbols to speech signals for correcting pronunciation |
| CN103544140A (en) * | 2012-07-12 | 2014-01-29 | 国际商业机器公司 | Data processing method, display method and corresponding devices |
| CN105374352A (en) * | 2014-08-22 | 2016-03-02 | 中国科学院声学研究所 | Voice activation method and system |
| CN105895087A (en) * | 2016-03-24 | 2016-08-24 | 海信集团有限公司 | Voice recognition method and apparatus |
| CN108615526A (en) * | 2018-05-08 | 2018-10-02 | 腾讯科技（深圳）有限公司 | The detection method of keyword, device, terminal and storage medium in voice signal |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5588091A (en) * | 1989-05-17 | 1996-12-24 | Environmental Research Institute Of Michigan | Dynamically stable associative learning neural network system |
| CN109256135B (en) * | 2018-08-28 | 2021-05-18 | 桂林电子科技大学 | End-to-end speaker confirmation method, device and storage medium |
| CN109215662B (en) * | 2018-09-18 | 2023-06-20 | 平安科技（深圳）有限公司 | End-to-end speech recognition method, electronic device, and computer-readable storage medium |
| CN109472024B (en) * | 2018-10-25 | 2022-10-11 | 安徽工业大学 | Text classification method based on bidirectional circulation attention neural network |
| CN109461438B (en) * | 2018-12-19 | 2022-06-14 | 合肥讯飞数码科技有限公司 | Voice recognition method, device, equipment and storage medium |
| CN109902622B (en) * | 2019-02-26 | 2020-06-09 | 中国科学院重庆绿色智能技术研究院 | Character detection and identification method for boarding check information verification |
-
2019
- 2019-06-28 CN CN201910775364.XA patent/CN110473518B/en active Active
- 2019-06-28 CN CN201910775838.0A patent/CN110534092B/en active Active
- 2019-06-28 CN CN201910578724.7A patent/CN110364142B/en active Active
- 2019-06-28 CN CN201910775376.2A patent/CN110428809B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| GB2389219A (en) * | 2002-05-29 | 2003-12-03 | L Labs Inc | User interface, system and method for automatically labelling phonic symbols to speech signals for correcting pronunciation |
| CN103544140A (en) * | 2012-07-12 | 2014-01-29 | 国际商业机器公司 | Data processing method, display method and corresponding devices |
| CN105374352A (en) * | 2014-08-22 | 2016-03-02 | 中国科学院声学研究所 | Voice activation method and system |
| CN105895087A (en) * | 2016-03-24 | 2016-08-24 | 海信集团有限公司 | Voice recognition method and apparatus |
| CN108615526A (en) * | 2018-05-08 | 2018-10-02 | 腾讯科技（深圳）有限公司 | The detection method of keyword, device, terminal and storage medium in voice signal |
Non-Patent Citations (2)
| Title |
|---|
| JULIAN SALAZAR: "SELF-ATTENTION NETWORKS FOR CONNECTIONIST TEMPORAL CLASSIFICATION IN SPEECH RECOGNITION", 《INTERNATIONAL CONFERENCE ON ACOUSTICS SPEECH AND SIGNAL PROCESSING ICASSP》 * |
| NIKO MORITZ ET AL.: "Triggered Attention for End-to-End Speech Recognition", 《ICASSP 2019 - 2019 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP)》 * |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110853669A (en) * | 2019-11-08 | 2020-02-28 | 腾讯科技（深圳）有限公司 | Audio identification method, device and equipment |
| CN110853669B (en) * | 2019-11-08 | 2023-05-16 | 腾讯科技（深圳）有限公司 | Audio identification method, device and equipment |
| CN110992929A (en) * | 2019-11-26 | 2020-04-10 | 苏宁云计算有限公司 | Voice keyword detection method, device and system based on neural network |
| WO2021103712A1 (en) * | 2019-11-26 | 2021-06-03 | 苏宁云计算有限公司 | Neural network-based voice keyword detection method and device, and system |
| CN111489737A (en) * | 2020-04-13 | 2020-08-04 | 深圳市友杰智新科技有限公司 | Voice command recognition method and device, storage medium and computer equipment |
| CN111489737B (en) * | 2020-04-13 | 2020-11-10 | 深圳市友杰智新科技有限公司 | Voice command recognition method and device, storage medium and computer equipment |
| CN111627457A (en) * | 2020-05-13 | 2020-09-04 | 广州国音智能科技有限公司 | Voice separation method, system and computer readable storage medium |
| CN111639157A (en) * | 2020-05-13 | 2020-09-08 | 广州国音智能科技有限公司 | Audio marking method, device, equipment and readable storage medium |
| CN111639157B (en) * | 2020-05-13 | 2023-10-20 | 广州国音智能科技有限公司 | Audio marking method, device, equipment and readable storage medium |
| CN114724544A (en) * | 2022-04-13 | 2022-07-08 | 北京百度网讯科技有限公司 | Voice chip, voice recognition method, device and equipment and intelligent automobile |
| CN114724544B (en) * | 2022-04-13 | 2022-12-06 | 北京百度网讯科技有限公司 | Voice chip, voice recognition method, device and equipment and intelligent automobile |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110428809B (en) | 2022-04-26 |
| CN110534092A (en) | 2019-12-03 |
| CN110473518B (en) | 2022-04-26 |
| CN110534092B (en) | 2022-04-26 |
| CN110428809A (en) | 2019-11-08 |
| CN110473518A (en) | 2019-11-19 |
| CN110364142B (en) | 2022-03-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110364142A (en) | Phoneme of speech sound recognition methods and device, storage medium and electronic device | |
| CN110335592A (en) | Phoneme of speech sound recognition methods and device, storage medium and electronic device | |
| CN108536679B (en) | Named entity recognition method, device, equipment and computer readable storage medium | |
| JP7490804B2 (en) | System and method for streaming end-to-end speech recognition with asynchronous decoders - Patents.com | |
| EP3371807B1 (en) | Generating target phoneme sequences from input speech sequences using partial conditioning | |
| CN108711420A (en) | Multilingual hybrid model foundation, data capture method and device, electronic equipment | |
| CN110534087A (en) | A kind of text prosody hierarchy Structure Prediction Methods, device, equipment and storage medium | |
| CN110032630B (en) | Dialectical recommendation device and method and model training device | |
| CN112528637B (en) | Text processing model training method, device, computer equipment and storage medium | |
| CN108711421A (en) | A kind of voice recognition acoustic model method for building up and device and electronic equipment | |
| EP3979098A1 (en) | Data processing method and apparatus, storage medium, and electronic apparatus | |
| CN107112005A (en) | Depth nerve SVMs | |
| CN110444203A (en) | Audio recognition method, device and electronic equipment | |
| CN112233664A (en) | Network training method, device, equipment and storage medium | |
| CN107871496A (en) | Audio recognition method and device | |
| CN110457714B (en) | Natural language generation method based on time sequence topic model | |
| CN114596844A (en) | Acoustic model training method, voice recognition method and related equipment | |
| CN110010136A (en) | The training and text analyzing method, apparatus, medium and equipment of prosody prediction model | |
| CN113590078A (en) | Virtual image synthesis method and device, computing equipment and storage medium | |
| CN111597341A (en) | Document level relation extraction method, device, equipment and storage medium | |
| CN107122378A (en) | Object processing method and device | |
| CN117581233A (en) | Artificial intelligence system for sequence-to-sequence processing with dual causal and non-causal limited self-attention for streaming applications | |
| CN112669824B (en) | Method for constructing acoustic model, voice recognition system and voice recognition method | |
| CN108694939A (en) | Phonetic search optimization method, device and system | |
| Singh et al. | Overview of neural machine translation for English-Hindi |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |