TW201830377A - Speech point detection method and speech recognition method - Google Patents

Abstract

The invention discloses a speech point detection method and a speech recognition method, and relates to the technical field of the speech recognition; the method comprises: extracting speech features from a speech data and inputting the speech features into a silence model; outputting a label according to the speech features to indicate whether the speech data is a silence frame; confirming a speech endpoint of a segment of speech according to the labels of the speech data of consecutive frames: during the non-activated state, if the length of the speech data of the non-silence frames is greater than a preset first threshold, it is determined that the first non-silence frame of the speech data is the starting point of a segment of speech; during the activated state, if the length of the speech data of the silence frame is greater than a preset second threshold, it is determined that the first silence frame of the speech data is the ending point of a speech data. The benefits of the above mentioned solution are: the problem of inaccurate detection of speech point in the prior art and excessive requirements for the detection environment can be solved.

Description

Speech endpoint detection method and speech recognition method

The present invention relates to the field of speech recognition technologies, and in particular, to a speech endpoint detection method and a speech recognition method.

With the development of speech recognition technology, speech recognition has become more and more widely used in people's lives. When the user uses the voice recognition technology in the handheld device, the voice recognition button is usually used to control the start and end time of the voice segment to be recognized, but when the user uses the voice recognition technology in the smart home environment, The distance between the start end and the end point of the voice paragraph can be manually determined by the remote selection of the sound pickup device. In this case, another method is needed to automatically judge the start and end time of the voice, that is, the voice endpoint detection. Voice Active Detection (VAD).

The traditional endpoint detection method is mainly based on the sub-band energy, that is, calculating the energy of each frame of speech data in a certain frequency band, and comparing with a preset energy threshold to determine the start end point and the end end point of the speech. This endpoint detection method has a high requirement for the detection environment, and it must perform speech recognition in a quiet environment to ensure the accuracy of the detected speech endpoint. In a relatively noisy environment, different kinds of noise will affect the energy of different sub-bands, which will cause greater interference to the above-mentioned endpoint detection methods, especially low signal-to-noise ratio and non-stationary In the information environment, the calculation of the energy of the sub-band will cause a lot of interference, which will make the final test result inaccurate. Only the accuracy of the voice endpoint detection can ensure that the voice is correctly collected and correctly recognized. The inaccurate result of the endpoint detection may cause the speech to be truncated or more noise to be recorded, which may result in the speech recognition not being able to decode the entire sentence, thus causing problems such as underreporting or false positives, and even causing the whole sentence. Decode all errors and reduce the accuracy of the speech recognition results.

According to the above problems in the prior art, a technical solution for a voice endpoint detection method and a voice recognition method is provided, which aims to solve the problem that the voice endpoint detection is inaccurate and the detection environment is too high in the prior art. The above technical solutions specifically include:

A voice endpoint detection method, wherein a pre-training forms a muting model for determining whether the voice data is a silence frame, and then acquiring a segment of voice of the externally input voice data including consecutive frames, and performing the following steps: Step S1, Extracting a voice feature of each frame of voice data, and inputting the voice feature into the mute model; Step S2, the mute model outputs a tag associated with each frame of voice data according to the voice feature, and the tag is used to indicate whether the voice data is a silence frame; Step S3, confirming the voice endpoint of a voice according to the label of the voice data of the continuous frame: When the sound pickup device that collects the voice is in a non-activated state, if the length of the voice data of the non-silent frame continuously appears is greater than a preset first The threshold value is used to determine that the voice data of the first frame is a non-silent frame is the starting end point of a voice; when the voice pickup device that collects the voice is in the startup state, if the voice data of the silent frame continuously appears, the length of the voice data is greater than a preset number. The second threshold determines that the voice data of the first frame is a silence frame is the end point of a voice.

Preferably, the voice endpoint detection method is configured to form a silent model by using the following method: Step A1, inputting a preset plurality of training voice data, and extracting voice features of each training voice data; Step A2 According to the corresponding voice feature, the automatic labeling operation is performed for each frame of the training voice data to obtain a label corresponding to each frame of voice data; the label is used to indicate that the corresponding one frame of voice data is a silent frame or a non-silent frame; Step A3, The silent model is obtained according to the training voice data and the corresponding label training; the first node and the second node are disposed on the output layer of the silent model; the first node is used to indicate the label of the corresponding silent frame; the second node is used to indicate the corresponding non- The label of the mute frame.

Preferably, the voice endpoint detection method, wherein each training voice data corresponding to the external input is preset with an annotated text to mark the text content corresponding to the training voice data; then step A2 specifically includes: Step A21, Obtaining the voice feature and the corresponding annotated text; Step A22, using the acoustic model formed by the pre-training to forcibly align the voice feature and the corresponding annotated text to obtain an output label corresponding to the phoneme of each frame of the voice feature; Step A23, The forced alignment training is post-processed with speech data to map the output label of the mute phoneme to the tag representing the mute frame and the output tag of the non-silent phoneme to the tag representing the non-silent frame.

Preferably, the speech endpoint detection method, wherein, in step A22, the acoustic model formed by the pre-training is a Gaussian mixture model-hidden Markov model, or a deep neural network-hidden Markov model.

Preferably, the speech endpoint detection method, wherein the muting model is a deep neural network model including a multi-layer neural network.

Preferably, the speech endpoint detection method includes at least one non-linear transformation between every two layers of neural networks of the muting model.

Preferably, the voice endpoint detection method, wherein each layer of the neural network of the silent model is a fully connected neural network, or a convolutional neural network, or a recurrent neural network.

Preferably, the voice endpoint detection method, wherein the silence model is a deep neural network model including a multi-layer neural network; the first node and the second node are disposed on an output layer of the silent model; The second node is used to indicate the label of the corresponding non-silent frame. Step S2 specifically includes: Step S21: After the voice feature is input into the mute model, the forward calculation by the multi-layer neural network is respectively associated with the output layer. a first value of the first node and a second value associated with the second node; Step S22, comparing the first value with the second value: if the first value is greater than the second value, A node is used as a label of the voice data and is output; if the first value is smaller than the second value, the second node is used as a label of the voice data and output. A speech recognition method, wherein the speech endpoint detection method is used to detect a start endpoint and an end endpoint of a segment of speech that need to be identified.

The foregoing technical solution has the beneficial effects of providing a voice endpoint detection method, which can solve the problem that the voice endpoint detection is inaccurate and the detection environment is too high in the prior art, thereby improving the accuracy of the voice endpoint detection and extending the endpoint. The ubiquity of the detection method improves the overall speech recognition process.

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

It should be noted that the embodiments in the present invention and the features in the embodiments may be combined with each other without conflict.

The invention is further illustrated by the following figures and specific examples, but is not to be construed as limiting.

According to the above problems existing in the prior art, there is provided a speech endpoint detection method, in which a pre-training forms a muting model for judging whether speech data is a silence frame, and then acquiring externally input speech data including consecutive frames. a piece of speech, and performing the following steps as shown in FIG. 1: Step S1, extracting the speech features of each frame of speech data, and inputting the speech features into the mute model; Step S2, the mute model outputs the association according to the speech features For each frame of voice data, the label is used to indicate whether the voice data is a silent frame; Step S3, the voice endpoint of a voice is confirmed according to the label of the voice data of the continuous frame: When the voice pickup device that collects the voice is in a non-activated state If the length of the voice data of the non-silent frame is greater than a preset first threshold, determining that the voice data of the first frame is a non-silent frame is a starting end of a voice; when the voice collecting device is in the voice collection In the startup state, if the length of the voice data of the silent frame continuously exceeds a preset second threshold, it is determined A mute voice frame data of a voice terminal end.

Specifically, in this embodiment, a mute model is first formed, and the mute model can be used to determine whether each frame of speech data in a segment of speech is a mute frame. The so-called silent frame refers to voice data that does not contain valid voices that require voice recognition; the so-called silent frame refers to voice data that contains valid voices that require voice recognition.

Then, in the embodiment, after the training forms the silent model, the voice features of each frame of the voice data input from the external input are extracted, and the extracted voice features are input into the silent model to output the voice data associated with the frame. s Mark. In this embodiment, there are two tags in total, which are respectively used to indicate that the frame voice data is a silent frame/non-silent frame.

In this embodiment, after the mute and non-mute classification of each frame of voice data is obtained, the speech endpoint is determined. However, instead of a frame of non-silent frame, a speech start can be considered, and a silence frame cannot be regarded as a speech end. Instead, the start end of a speech is determined according to the number of consecutive silent/non-silent frames. And end endpoints. Specifically:

When the sound collecting device that collects the voice is in a non-activated state, if the length of the voice data of the non-silent frame is greater than a preset first threshold, determining that the voice data of the first frame is a non-silent frame is a voice. Starting point

When the sound collecting device that collects the voice is in the activated state, if the length of the voice data of the silent frame is greater than a preset second threshold, the voice data of the first frame being the silence frame is determined to be the end point of the voice.

In a preferred embodiment of the present invention, the first threshold may take a value of 30, and the second threshold may take a value of 50. That is, when the sound collection device that collects the voice is in the non-activated state, if the length of the non-silent frame continuously appears to be greater than 30 (the consecutive 30 frames of non-silent frames appear), it is determined that the first frame of the non-silent frame is the beginning of the segment of the voice. point.

When the sound collecting device that collects the voice is in the startup state, if the length of the silent frame continuously appears to be greater than 50 (the consecutive 50 frames of the silence frame appear), it is determined that the first frame silence frame is the end point of the segment of the voice.

In another preferred embodiment of the present invention, the first threshold may also take a value of 70, and the second threshold may take a value of 50.

In other embodiments of the present invention, the values of the first threshold and the second threshold may be freely set according to actual conditions to meet the requirements of voice endpoint detection in different environments.

In a preferred embodiment of the present invention, the silent model can be pre-trained by the following method as shown in FIG. 2: Step A1, inputting a preset plurality of training voice data, and extracting each training voice data The voice feature; Step A2, according to the corresponding voice feature, the automatic labeling operation is performed for each frame of the training voice data, and a label corresponding to each frame of voice data is obtained; the label is used to indicate that the corresponding one frame of voice data is a silent frame or a non- Silent frame; Step A3, according to the corresponding voice feature, perform automatic labeling operation for each frame of training voice data, and obtain a label corresponding to each frame of voice data; the label is used to indicate that the corresponding one frame of voice data is a silent frame or non-mute a first node and a second node are disposed on an output layer of the mute model; a first node is used to indicate a tag of the corresponding mute frame; and a second node is used to represent a tag corresponding to the non-silent frame.

Specifically, in this embodiment, a preset plurality of training voice data is first input. The so-called training voice data refers to voice data in which the text content is known in advance. The training voice data can be extracted according to the Chinese voice data set of the voice recognition system that has been trained in Chinese in advance, and has the annotation text corresponding to the training voice data. That is, the training voice data input in the above step A1 is the same as the voice data applied when training the acoustic model of the subsequent voice recognition.

In this embodiment, after the training voice data is input, the voice features are extracted for each training voice data. The extraction of speech features can also be performed using speech features extracted while training the acoustic model of speech recognition. Common speech features may include Mel Frequency Cepstrum Coefficient (MFCC), Perceptual Linear Predictive (PLP) or Filter-Bank (FBANK) features. As such, in other embodiments of the invention, other similar speech features may be employed to accomplish the training of the silent model.

In this embodiment, in the above step A2, before the training input parameter as the silent model, it is first necessary to perform an automatic labeling operation on the training voice data to align each frame of the voice data frame. In the above automatic labeling operation, each frame of voice data will obtain a label, and the above automatic labeling processing method will be described in detail below. After the automatic labeling operation, the silent model can be trained.

In the preferred embodiment of the present invention, each of the training voice data corresponding to the external input is preset with an annotated text to mark the text content corresponding to the training voice data; then the step A2 is specifically shown in FIG. The method may include: Step A21: acquiring a voice feature and a corresponding annotated text; Step A22: forcibly aligning the voice feature and the corresponding annotated text by using an acoustic model formed by the pre-training to obtain an output corresponding to the phoneme of each frame of the voice feature. Step A23, post-processing the forced-aligned training voice data to map the output label of the silent phoneme to the label representing the silence frame, and map the output label of the non-silent phoneme to the label indicating the non-silent frame on.

Specifically, in the embodiment, if the automatic labeling operation is performed on the training voice data manually, a large amount of labor costs are required, and the labeling of the noise may be inconsistent in the labeling results of different labeling personnel. , thereby affecting the process of the subsequent training model. Therefore, an efficient and feasible automatic labeling method is provided in the technical solution of the present invention.

In the above method, the speech features of the speech data for each frame and the corresponding annotated text are first acquired, and then the speech features and the annotated text are forcibly aligned.

In this embodiment, the acoustic features of the subsequent speech recognition (ie, the acoustic model formed by the pre-training) may be used to forcibly align the speech features and the annotated text. The acoustic model of speech recognition in the present invention may be a Gaussian Mixture Model-Hidden Markov Model (GMM-HMM) or a deep neural network--Hidden Markov Model (Deep Neural Network) - Hidden Markov Model, DNN-HMM), or other suitable model. The modeling unit in the above acoustic model is a phone level, such as a Context Independent Phone (ci-phone) or a Context Dependent Phone (cd-phone). The forced alignment operation using the acoustic model described above can align the training speech data frame to the phoneme level.

In this embodiment, in the above step A23, after the forced-aligned training voice data is post-processed, the voice data corresponding to the mute label can be obtained. In the above post-processing operation, part of the phoneme is generally regarded as a silent phoneme, and other phonemes are regarded as non-silent phonemes. After the above mapping, each frame of voice data can be associated with a mute/unmute tag.

In a preferred embodiment of the invention, the mute model can then be trained using the speech features and frame aligned tags obtained above. The above silent model may be a deep neural network model including a multilayer neural network. Each layer of the above silent model may be a fully connected neural network, a convolutional neural network, a recurrent neural network, etc., and each two layers of neural networks may contain one or more nonlinear transformations, such as signoid nonlinearity. Transform, tanh nonlinear transform, maxpool nonlinear transform, RELU nonlinear transform or softmax nonlinear transform.

In a preferred embodiment of the invention, as shown in FIG. 4, the silent model includes a multilayer neural network 41 and includes an output layer 42. A first node 421 and a second node 422 are disposed in the output layer 42 of the mute model. The first node 421 is used to indicate a label of a corresponding silent frame, and the second node 422 is used to indicate a label corresponding to a non-silent frame. A softmax nonlinear transform or other non-linear transform operation may be performed on the first node 421 and the second node 422 of the output layer 42, or may not use a non-linear transform operation.

In the preferred embodiment of the present invention, the step S2 is specifically as shown in FIG. 5, and includes: Step S21: After the voice feature is input into the mute model, the forward calculation by the multi-layer neural network respectively obtains the correlation in the output layer. a first value of a node and a second value associated with the second node; Step S22, comparing the first value with the second value: if the first value is greater than the second value, the first value is The node is used as a label of the voice data and is output; if the first value is smaller than the second value, the second node is used as a label of the voice data and output.

Specifically, in this embodiment, the voice feature is input into the trained silent model, and the multi-layer neural network performs forward calculation, and finally obtains two output nodes (first node and second node) in the output layer. The values are the first value and the second value. Then comparing the size of the first value and the second value: if the first value is larger, the first node is selected as the label of the voice data and output, that is, the voice data is a silence frame; accordingly, if the second If the value is large, the second node is selected as the label of the voice data and output, that is, the voice data is a non-silent frame at this time.

In a preferred embodiment of the present invention, a complete process of the voice endpoint detection method is as follows: First, a pre-trained Chinese speech recognition system is prepared, wherein the selected speech recognition system has a Chinese speech data set and has Annotated text for voice data.

The acoustic characteristics of the acoustic model of the above speech recognition system are FBANK features, so the FBANK feature is still used when training the silent model.

The speech data is extracted from the training speech data, and the corresponding annotation text is input into the speech recognition system for forced alignment, each frame speech feature is corresponding to the phoneme level label, and then the non-silent phoneme in the alignment result is mapped to the non-mute On the label, the mute phoneme is mapped to the mute tag to complete the training profile tag preparation of the mute model.

Then, using the above-mentioned training voice data and corresponding label training to form a silent model, when the voice endpoint is detected by using the silence model formed by the above training, the voice features of each frame of the voice are extracted and sent to the training. In the silent model, after the forward calculation of the multi-layer neural network, the first value of the first node and the second value of the second node are output, and then the magnitudes of the two values are compared, and the corresponding value of the output value is larger. The label of the node is used as a label of the frame voice data to indicate that the frame voice data is a silent frame/non-silent frame.

Finally, it is determined whether there is a silent frame/non-silent frame of consecutive frames: when the sound collecting device that collects the voice is in a non-activated state, if there are consecutive 30 frames of non-silent frames, the first of the consecutive 30 frames of non-silent frames The frame voice data is used as the starting end point of the entire speech to be recognized; when the sound collecting device that collects the voice is in the activated state, if there are consecutive 50 frames of the silence frame, the first frame of the continuous 50 frames of the silenced frame is the voice data. As the end point of the entire speech to be recognized.

In a preferred embodiment of the present invention, a voice recognition method is further provided, wherein the voice endpoint detection method is used to detect a start endpoint and an end endpoint of a voice to be identified to determine a range of voices to be recognized. The speech recognition technology is then used to identify the speech.

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the embodiments and the scope of the present invention, and those skilled in the art should be able to The resulting solutions to the obvious variations are intended to be included within the scope of the invention.

S1‧‧‧Step S1

S2‧‧‧Step S2

S3‧‧‧Step S3

A1‧‧‧Step A1

A2‧‧‧Step A2

A3‧‧‧Step A3

A21‧‧‧Step A21

A22‧‧‧Step A22

A23‧‧‧Step A23

41‧‧‧Multilayer neural network

42‧‧‧Output layer

421‧‧‧ first node

422‧‧‧second node

S21‧‧‧Step S21

S22‧‧‧Step S22

1 is a schematic overall flow chart of a voice endpoint detection method in a preferred embodiment of the present invention; FIG. 2 is a schematic flowchart of training to form a silent model in a preferred embodiment of the present invention; In a preferred embodiment, a schematic diagram of automatic labeling of training speech data on the basis of FIG. 2; FIG. 4 is a diagram showing the structure of a muting model including a multi-layer neural network in a preferred embodiment of the present invention; BRIEF DESCRIPTION OF THE DRAWINGS Figure 5 is a flow diagram of processing and outputting tags associated with voice data on the basis of Figure 1 in a preferred embodiment of the present invention.

Claims (9)

A speech endpoint detection method, wherein a pre-training forms a muting model for judging whether the speech data is a silence frame, and then acquiring a segment of the voice of the externally input speech data including consecutive frames, and performing the following steps: Step S1 Extracting a speech feature of the speech data of each frame, and inputting the speech feature into the mute model; Step S2, the mute model outputs a tag associated with the speech data of each frame according to the speech feature, the tag is used for Indicates whether the voice data is a silent frame; Step S3, confirming a voice endpoint of the voice according to the label of the voice data of the continuous frame: when the sound collecting device that collects the voice is in a non-activated state, if the voice is continuously unmuted If the length of the voice data of the frame is greater than a preset first threshold, determining that the voice data of the first frame is the unvoiced frame is a starting end of the voice; when the sound collecting device that collects the voice is in an activated state If the length of the voice data of the silent frame continuously exceeds a preset second threshold, determining that the first frame is the static The frame of speech data for end points of the speech period ends. The voice endpoint detection method of claim 1, wherein the silent model is pre-trained by the following method: Step A1, inputting a preset plurality of training voice data, and extracting each of the training voices The voice feature of the data; Step A2, according to the corresponding voice feature, performing automatic labeling operation on the training voice data for each frame, obtaining a label corresponding to the voice data of each frame; the label is used to indicate the corresponding frame of the voice The data is a silent frame or a non-silent frame; Step A3, the mute model is obtained according to the training voice data and the corresponding tag; the output node of the mute model is provided with a first node and a second node; the first node The label used to indicate the corresponding silence frame; the second node is used to indicate the label corresponding to the non-silent frame. The voice endpoint detection method of claim 2, wherein each of the training voice data corresponding to the external input is preset with an annotation text to mark the text content corresponding to the training voice data; Step A2 specifically includes: Step A21: acquiring the voice feature and the corresponding tagged text; Step A22, forcibly aligning the phonetic feature and the corresponding tagged text by using an acoustic model formed by pre-training to obtain each frame The voice feature corresponds to an output tag of the phoneme; Step A23, post-processing the training voice data subjected to the forced alignment to map the output tag of the mute phoneme to the tag representing the mute frame, and to be unmuted The output label of the phoneme is mapped to the label representing the non-silent frame. The speech endpoint detection method according to claim 3, wherein in the step A22, the acoustic model formed by the pre-training is a Gaussian mixture model-hidden Markov model, or a deep neural network-hidden Markov model. . The speech endpoint detection method of claim 1, wherein the muting model is a deep neural network model including a multi-layer neural network. The speech endpoint detection method of claim 5, wherein each of the two layers of the muting model includes at least one non-linear transformation between the neural networks. The speech endpoint detection method of claim 5, wherein the neural network of each layer of the muting model is a fully connected neural network, or a convolutional neural network, or a recurrent neural network. The voice endpoint detection method of claim 2, wherein the muting model is a deep neural network model including a multi-layer neural network; the first node and the second node are disposed on an output layer of the muting model; The first node is used to indicate the label corresponding to the silence frame; the second node is used to indicate the label corresponding to the non-silent frame; then the step S2 specifically includes: Step S21, after the voice feature is input into the silent model, The forward calculation of the multi-layer neural network respectively obtains a first value associated with the first node in the output layer and a second value associated with the second node; Step S22, the first value is The second value is compared: if the first value is greater than the second value, the first node is used as the label of the voice data and is output; if the first value is less than the second value, The second node is used as the tag of the voice data and output. A speech recognition method, wherein the speech endpoint detection method as described in claims 1-8 detects the start endpoint and the end endpoint of a speech that needs to be recognized.