WO2025256262A1 - 语音识别的方法、服务器、存储介质和程序产品 - Google Patents

语音识别的方法、服务器、存储介质和程序产品

Info

Publication number
WO2025256262A1
WO2025256262A1 PCT/CN2025/089471 CN2025089471W WO2025256262A1 WO 2025256262 A1 WO2025256262 A1 WO 2025256262A1 CN 2025089471 W CN2025089471 W CN 2025089471W WO 2025256262 A1 WO2025256262 A1 WO 2025256262A1
Authority
WO
WIPO (PCT)
Prior art keywords
speech
prediction
features
speech recognition
text
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2025/089471
Other languages
English (en)
French (fr)
Inventor
安柯宇
张仕良
李泽瑞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Publication of WO2025256262A1 publication Critical patent/WO2025256262A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/16—Speech classification or search using artificial neural networks

Definitions

  • This disclosure relates to computer technology, and more particularly to a method, server, storage medium, and program product for speech recognition.
  • This disclosure provides a speech recognition method, server, storage medium, and program product to address the problems of slow speed and low efficiency in autoregressive speech recognition models.
  • this disclosure provides a speech recognition method, including:
  • the speech data is input into a speech recognition model, the speech features of the speech data are extracted by the speech recognition model, the speech features are processed by connection time sequence classification to obtain a preliminary text sequence prediction result, and the speech recognition result of the speech data is generated by decoding based on the preliminary text sequence prediction result and the speech features.
  • this disclosure provides a speech recognition method, including:
  • the speech data to be recognized is obtained;
  • the speech data is input into a speech recognition model, the speech features of the speech data are extracted by the speech recognition model, the speech features are processed by connection time sequence classification to obtain a preliminary text sequence prediction result, and the speech recognition result of the speech data is generated based on the preliminary text sequence prediction result and the speech features.
  • the speech recognition result of the speech data is returned to the terminal device.
  • this disclosure provides a server, including:
  • At least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to cause the server to perform the methods provided by any of the foregoing aspects.
  • this disclosure provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the method provided in any of the foregoing aspects.
  • this disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the methods provided in any of the foregoing aspects.
  • the speech recognition method, server, storage medium, and program products disclosed herein extract speech features from speech data through a speech recognition model, perform connection-time classification processing on the speech features to align speech with text, and generate a preliminary text sequence prediction result.
  • the length of this preliminary text sequence prediction result is also the length of the final speech recognition result.
  • the speech recognition result of the speech data is decoded to achieve non-autoregressive speech recognition.
  • the final speech recognition result can be generated in one decoding step, which greatly saves the time overhead of decoding and outputting the speech recognition result, improves the speed and efficiency of speech recognition, and can achieve a high recognition accuracy.
  • Figure 1 is a schematic diagram of an example system architecture to which this disclosure applies;
  • Figure 2 is a flowchart of a speech recognition method provided in an exemplary embodiment of the present disclosure
  • Figure 3 is a basic framework diagram of a speech recognition model provided in an exemplary embodiment of this disclosure.
  • Figure 4 is a framework diagram of a speech recognition model provided in an exemplary embodiment of this disclosure.
  • FIG. 5 is a detailed flowchart of speech recognition provided by an exemplary embodiment of this disclosure.
  • Figure 6 is a flowchart of a speech recognition model training method provided in an exemplary embodiment of this disclosure.
  • Figure 7 is a framework diagram of speech recognition model training provided in an exemplary embodiment of this disclosure.
  • Figure 8 is a flowchart of a speech recognition method provided in an exemplary embodiment of the present disclosure.
  • Figure 9 is a schematic diagram of the structure of a server provided in an embodiment of this disclosure.
  • the user information including but not limited to user device information, user attribute information, etc.
  • data including but not limited to data used for analysis, data stored, data displayed, etc.
  • the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.
  • a neural network architecture that performs sequence-to-sequence mapping.
  • speech recognition the input sequence is acoustic features, and the output sequence is the recognized text.
  • CTC Connectionist Temporal Classification
  • a token also known as a discrete token or discrete representation, is a discrete representation of speech or text.
  • the discrete representation of speech is called a speech token
  • the discrete representation of text is called a text token.
  • the vocabulary used by the model contains both speech and text tokens that may appear in the prediction results.
  • Each speech token in the vocabulary corresponds to a speech unit, such as a phoneme, syllable, or speech segment.
  • Each text token in the vocabulary corresponds to a text unit, such as a character or word.
  • this disclosure provides a speech recognition method, specifically including: acquiring speech data to be recognized; inputting the speech data into a speech recognition model; extracting speech features from the speech data through the speech recognition model; performing Connectivity-Time Classification (CTC) processing on the speech features to obtain a preliminary text sequence prediction result; aligning speech and text through CTC processing to generate a preliminary text sequence prediction result, the length of which is also the length of the final speech recognition result; furthermore, based on the preliminary text sequence prediction result and speech features, decoding to generate the speech recognition result of the speech data; employing a non-autoregressive approach, the final speech recognition result can be generated in a single decoding step, significantly saving the time overhead of decoding and outputting the speech recognition result, improving the speed and efficiency of speech recognition, and achieving recognition accuracy comparable to that of autoregressive speech recognition models.
  • CTC Connectivity-Time Classification
  • Figure 1 is a schematic diagram of an example system architecture to which this disclosure applies.
  • the system architecture includes a server and end-side devices.
  • the server and end-side devices have a communication link that enables communication between them.
  • a server is a computing device deployed in the cloud or locally, such as a cloud cluster.
  • the server stores a non-autoregressive speech recognition model.
  • the server is responsible for performing speech recognition on given speech data based on the non-autoregressive speech recognition model, obtaining the speech recognition result, i.e., the text corresponding to the speech data.
  • the server inputs the speech data into the speech recognition model, extracts speech features from the speech data through the speech recognition model, performs connection-time classification processing on the speech features to obtain a preliminary text sequence prediction result, and decodes and generates the speech recognition result of the speech data based on the preliminary text sequence prediction result and speech features.
  • Edge devices can be electronic devices that run downstream applications/systems/services. Specifically, they can be hardware devices with network communication, computing, and information display capabilities, including but not limited to smartphones, tablets, desktop computers, local servers, and cloud servers. Edge devices need to perform speech recognition tasks when running downstream applications/systems/services. These downstream applications/systems/services can provide speech-to-text services, multilingual speech recognition, etc. To implement the functions of downstream applications/systems/services, speech recognition needs to be performed based on given speech data, and the subsequent processing logic of the downstream application/system/service needs to be implemented based on the speech recognition results. For example, returning the text content of the speech data to the user; analyzing user needs based on the speech recognition results of user input to achieve human-computer interaction; and implementing multilingual translation functions based on the speech recognition results of user input.
  • an edge device when it needs to perform a speech recognition task, it sends a speech recognition request to the server.
  • This speech recognition request contains the speech data to be recognized.
  • the server responds to the speech recognition request, obtains the speech data to be recognized, inputs the speech data into the speech recognition model, extracts the speech features from the speech data through the speech recognition model, performs connection-time classification processing on the speech features, and obtains a preliminary text sequence prediction result. Based on the preliminary text sequence prediction result and the speech features, the server decodes and generates the speech recognition result of the speech data.
  • the server returns the speech recognition result of the voice data to the end device.
  • the end device receives the speech recognition result of the voice data returned by the server and continues to execute subsequent processing logic based on the speech recognition result.
  • the server can provide the speech recognition model's Application Program Interface (API) to the edge device.
  • the edge device can send a request to the server to invoke the speech recognition model based on the API, and this request contains the speech data to be recognized.
  • the server extracts the speech data to be recognized from the request, inputs the speech data into the speech recognition model for speech recognition, obtains the speech recognition result, and returns the speech recognition result to the edge device.
  • the edge device receives the speech recognition result returned by the server and continues to execute subsequent processing logic based on the speech recognition result.
  • a cloud server's speech-to-text service users can submit voice data to be processed to the cloud server via their edge devices.
  • This data can include audio data generated from various scenarios such as meeting recordings, phone recordings, and voice input.
  • the cloud server inputs the voice data into a speech recognition model, extracts the voice features, performs connection-time classification on these features, and obtains a preliminary text sequence prediction result. Based on the preliminary text sequence prediction result and the voice features, the system decodes and generates the speech recognition result of the voice data, thus achieving speech-to-text conversion and obtaining the corresponding text.
  • the cloud server provides the user with the text corresponding to the voice data.
  • the cloud server sends the text corresponding to the voice data to the end device.
  • the cloud server stores the text corresponding to the voice data in a designated location and provides the user with download information for the text corresponding to the voice data. The user then downloads the text corresponding to the voice data to their local device according to the download information.
  • speech recognition method provided in this embodiment can be applied to speech recognition in any language and is suitable for multilingual speech recognition scenarios.
  • FIG. 2 is a flowchart of a speech recognition method provided in an exemplary embodiment of this disclosure.
  • the execution entity in this embodiment is the server in the aforementioned system architecture. As shown in Figure 2, the specific steps of the method are as follows:
  • Step S201 Obtain the speech data to be recognized.
  • a device when it needs to perform speech recognition, it can send a speech recognition request to the server.
  • This request contains the speech data to be recognized.
  • the server receives the speech recognition request from the device and extracts the speech data to be recognized from it.
  • users can also upload voice data to the server for processing via their end-user devices.
  • users can upload recorded meeting recordings, call recordings, and other voice files to the server.
  • the server receives the voice data uploaded via the end-user device.
  • the server provides the speech recognition model's API to the edge device.
  • the edge device can send a request to the server to invoke the speech recognition model based on the API, and this request contains the speech data to be recognized.
  • the server extracts the speech data to be recognized from the request.
  • the voice data to be identified can be voice data of various formats.
  • the server can receive the voice data to be identified sent by other devices, or read the pre-stored voice data from the storage device, or obtain the voice data to be identified from a third-party platform.
  • Step S202 Input the speech data into the speech recognition model, extract the speech features of the speech data through the speech recognition model, perform connection time sequence classification processing on the speech features to obtain the preliminary text sequence prediction result, and decode and generate the speech recognition result of the speech data based on the preliminary text sequence prediction result and speech features.
  • the server After acquiring the speech data to be recognized, the server inputs the speech data into the speech recognition model, which then performs speech recognition processing on the input speech data to obtain the speech recognition result.
  • a speech recognition model is used to extract speech features from the input speech data.
  • Connectivity-Time Classification (CTC) is then applied to these features, and the speech is aligned with the text using CTC to obtain a preliminary text sequence prediction result. Further, based on the preliminary text sequence prediction result and the speech features, the speech recognition model decodes and generates the speech recognition result of the speech data.
  • the preliminary text sequence prediction result includes multiple text tag prediction probability distributions, each of which can determine one text tag.
  • the preliminary text sequence prediction result determines a text tag sequence, and its length refers to the length of the text tag sequence determined by the preliminary text sequence prediction result.
  • the length of the preliminary text sequence prediction result determines the length of the final speech recognition result; therefore, the length of the final speech recognition result can be obtained by using the length of the text tag sequence corresponding to the preliminary text sequence prediction result.
  • the method in this embodiment extracts speech features from the speech data using a speech recognition model during speech recognition.
  • speech and text are aligned, generating a preliminary text sequence prediction result.
  • the length of this preliminary text sequence prediction result is the same as the length of the final speech recognition result.
  • the speech recognition result of the speech data is decoded, achieving non-autoregressive speech recognition.
  • the final speech recognition result can be generated in a single decoding step, significantly reducing the time overhead of decoding and outputting the speech recognition result, improving the speed and efficiency of speech recognition, and achieving recognition accuracy comparable to autoregressive speech recognition models.
  • Figure 3 is a basic framework diagram of the speech recognition model provided in this embodiment.
  • the speech recognition model used in this embodiment includes: a feature extraction unit, a connection-time classification unit, and a decoding unit.
  • the system includes a feature extraction unit for extracting speech features from the speech data, a connection-temporal classification unit for performing connection-temporal classification on the speech features to obtain preliminary text sequence prediction results, and a decoding unit for decoding and generating speech recognition results from the speech data based on the preliminary text sequence prediction results and speech features.
  • the feature extraction unit can be implemented using any existing speech feature extraction model, such as convolutional neural networks (CNNs) used to extract speech features from speech data, or speech feature extraction models based on Transformers, etc. This embodiment does not make any specific limitations.
  • CNNs convolutional neural networks
  • the feature extraction unit may include an acoustic feature extraction module and an encoding module.
  • the acoustic feature extraction module is used to extract acoustic features from the speech data.
  • the acoustic features of the speech data may be Mel frequency cepstrum coefficients (MFCC), linear predictive coding (LPC), etc.
  • MFCC Mel frequency cepstrum coefficients
  • LPC linear predictive coding
  • the acoustic feature extraction module may be implemented using a neural network model that extracts the corresponding type of acoustic features from the speech data; no specific limitation is made here.
  • the encoding module is used to encode the acoustic features of the speech data to extract high-level representations of the acoustic features of the speech data and obtain the speech features of the speech data.
  • the encoding module can be implemented using any speech coding model used to encode and generate high-level representations of acoustic features, such as the encoder in a speech recognition model based on CNN or Transformer, etc. This embodiment does not make any specific limitation.
  • connection timing classification unit includes a connection timing classification module and a post-processing module.
  • connection-time classification module is used to perform connection-time classification prediction on speech features, and obtain the connection-time classification prediction results.
  • the connection-time classification prediction results include: prediction frames corresponding to each frame in the speech data, and the prediction frames include the text tag prediction probability distribution.
  • the post-processing module is used to post-process the connection time-series classification prediction results to obtain preliminary text sequence prediction results.
  • the temporal classification module can be a deep learning network implementing the CTC algorithm, consisting of multiple layers, including linear layers and softmax layers.
  • the linear layers are used to learn the linear relationship between the input speech features and the output text tags, while the softmax layers are used to convert the network output into predicted probabilities, representing the probability of the output text tags.
  • connection time-series classification module can use extended bidirectional long short-term memory (BLSTM) network, hidden Markov model (HMM) or other models to implement the CTC algorithm.
  • BLSTM long short-term memory
  • HMM hidden Markov model
  • step S202 the speech features are processed by connection temporal classification to obtain preliminary text sequence prediction results, which can be implemented in the following way:
  • Connectivity temporal classification prediction is performed on speech features to obtain the connectivity temporal classification prediction result (also known as the CTC prediction result).
  • the connectivity temporal classification prediction result includes: the prediction frame corresponding to each frame in the speech data, and the prediction frame includes the text tag prediction probability distribution.
  • T represent the number of frames in the speech data.
  • the connection-time classification prediction result of the speech data includes T text tag prediction probability distributions, each corresponding to a frame in the speech data, also known as the T-frame text tag prediction probability distribution.
  • vocab_size represent the size of the vocabulary used by the speech recognition model, which contains text tags that may appear in the speech recognition result.
  • a special text tag representing the null character referred to as the specific tag in this embodiment, denoted as ⁇
  • ⁇ a special text tag representing the null character
  • the text tag prediction probability distribution of each frame in the connection-time classification prediction result contains the probability distribution of that frame corresponding to these vocab_size+1 text tags (including text tags in the vocabulary and the specific tag ⁇ ).
  • the text tag with the highest probability in the text tag prediction probability distribution of each frame can be used as the text tag determined by that text tag prediction probability distribution.
  • connection temporal classification prediction result is obtained, resulting in a frame-level prediction result aligned with the length of the speech data. Specifically, this can be achieved using existing CTC algorithms/models, which will not be elaborated here.
  • connection-time classification prediction results are post-processed to obtain preliminary text sequence prediction results, which can be achieved in the following way:
  • Consecutive repeating frames in the connection-time classification prediction results are merged to obtain a merged prediction result.
  • Repeating frames include multiple prediction frames with the same text label corresponding to the highest probability.
  • Specific frames are then removed from the merged prediction result to obtain a preliminary text sequence prediction result.
  • Specific frames refer to prediction frames whose text label corresponds to a specific label with the highest probability.
  • the predicted probability distribution of each text tag included in the connection-time classification prediction result is referred to as a prediction frame, and the connection-time classification prediction result contains multiple prediction frames.
  • the text tag corresponding to the highest probability in each prediction frame is taken as the text tag determined by the prediction frame.
  • multiple prediction frames with the same determined text tag are called repeating frames. Consecutive repeating frames refer to those that appear consecutively in the connection-time classification prediction result.
  • a specific frame refers to a prediction frame where the determined text tag is the specific tag " ⁇ " representing the empty character.
  • connection-time classification prediction results consecutive repeating frames in the prediction results are first merged into a new prediction frame, and then specific frames are removed to obtain a preliminary text sequence prediction result.
  • the length of the text sequence contained in this preliminary text sequence prediction result i.e., the number of text tokens contained
  • the average of the consecutive repeating frames in the connection-time classification prediction results can be used to merge the consecutive repeating frames and obtain a new prediction frame.
  • the text tag prediction probability distribution of the new prediction frame is the average of the text tag prediction probability distributions of the repeating frames.
  • the merged prediction result is obtained after merging all consecutive repeating frames in the connection-time classification prediction results.
  • the corresponding 5-frame text tag prediction probability distribution (i.e., 5 prediction frames) can be obtained, denoted as p1, p2, p3, p4, and p5.
  • the text tag sequence determined by these 5 prediction frames is [a, ⁇ , b, b, ⁇ ], where the 3rd and 4th prediction frames are two consecutive prediction frames, and the text tag determined by these two prediction frames is the same, both being "b", therefore the 3rd and 4th prediction frames are consecutive repeating frames. Merging p3 and p4 yields a new prediction frame.
  • the merged new prediction frame is the average of p3 and p4.
  • the text tag sequence determined by the merged prediction result is [a, ⁇ , b, ⁇ ]. Further, specific frames in the merged prediction result that have the specific text tag " ⁇ " are removed, resulting in a preliminary text sequence prediction result.
  • the preliminary text sequence prediction result determines the text tag sequence [a, b].
  • the preliminary text sequence prediction results include two text tag prediction probability distributions: the first is p1, and the second is the average of p3 and p4, forming a posterior probability matrix with dimensions [2, vocab_size+1].
  • the prediction frame with the highest probability of corresponding consecutive repeating frames to the determined text tags can also be used as the merged prediction frame.
  • a frame-level connection-time classification prediction result is obtained by performing connection-time classification prediction on the speech features. Further, by merging consecutive repeating frames in the connection-time classification prediction result and deleting specific frames, a token-level preliminary text sequence prediction result is obtained. The length of this preliminary text sequence prediction result is also the length of the final speech recognition result. Predicting a preliminary text sequence prediction result of the same length as the speech recognition result using CTC has good noise resistance and vocabulary modeling capabilities, which can improve the accuracy of speech recognition.
  • the decoding unit includes a mapping module and a decoding module.
  • the mapping module is used to map the preliminary text sequence prediction results to hidden layer embeddings.
  • the decoding module is used to decode the speech features and hidden layer embeddings to obtain the speech recognition results of the speech data.
  • the mapping module can be implemented using an embedding layer (also known as an embedding layer), which maps the initial text sequence prediction results to the feature space to obtain a hidden layer embedding consistent with the dimension of the speech features.
  • an embedding layer also known as an embedding layer
  • the decoding module can be implemented using a transformer decoder, taking the speech features and hidden layer embeddings of the speech data as input to decode and generate the final speech recognition result.
  • the decoding module can be implemented using a bidirectional transformer decoder.
  • the decoding module can be implemented using a unidirectional transformer decoder. The solution in this embodiment can be applied to both offline speech recognition and popular speech recognition scenarios.
  • Figure 4 is a framework diagram of a speech recognition model provided in an exemplary embodiment of this disclosure.
  • the speech recognition model specifically includes: an acoustic feature extraction module, an encoding module, a connection time-series classification module, a post-processing module, a mapping module, and a decoding module.
  • the acoustic feature extraction module is used to extract the acoustic features of speech data.
  • the encoding module is used to encode the acoustic features of speech data to obtain the speech features of the speech data.
  • connection temporal classification module is used to perform connection temporal classification prediction on speech features and obtain the connection temporal classification prediction results.
  • the post-processing module is used to post-process the connection time-series classification prediction results to obtain preliminary text sequence prediction results.
  • the decoding module is used to decode speech features and hidden layer embeddings to obtain speech recognition results from the speech data.
  • FIG. 5 is a detailed flowchart of the speech recognition provided in this embodiment of the present disclosure.
  • the detailed process of speech recognition based on the speech recognition model shown in Figure 4 is as follows:
  • Step S501 Obtain the speech data to be recognized.
  • Step S502 Extract the acoustic features of the speech data.
  • Step S503 Encode the acoustic features of the speech data to obtain the speech features of the speech data.
  • Step S504 Perform connection temporal classification prediction on the speech features to obtain the connection temporal classification prediction result.
  • Step S505 Merge consecutive repeating frames in the connection time-series classification prediction results and delete specific frames to obtain preliminary text sequence prediction results.
  • Step S506 Map the preliminary text sequence prediction results to hidden layer embeddings.
  • Step S507 Decode the speech features and hidden layer embeddings to obtain the speech recognition results of the speech data.
  • the method in this embodiment extracts speech features from the speech data using a speech recognition model during speech recognition.
  • speech and text are aligned, generating a preliminary text sequence prediction result.
  • the length of this preliminary text sequence prediction result is the same as the length of the final speech recognition result.
  • the speech recognition result of the speech data is decoded, achieving non-autoregressive speech recognition.
  • the final speech recognition result can be generated in a single decoding step, significantly reducing the time overhead of decoding and outputting the speech recognition result, improving the speed and efficiency of speech recognition, and achieving recognition accuracy comparable to autoregressive speech recognition models.
  • FIG. 6 is a flowchart of a speech recognition model training method provided in an exemplary embodiment of this disclosure. As shown in Figure 6, the training steps of the speech recognition model used in the aforementioned embodiment are as follows:
  • Step S601 Obtain the training set, which includes speech samples and reference text for the speech samples.
  • the training set used to train the speech recognition model includes speech samples and reference text for the speech samples.
  • the reference text for the speech samples is also the predicted recognition result of the speech samples.
  • the training set used in this embodiment can come from a public dataset used for training speech recognition tasks, or it can be constructed by obtaining a large number of speech samples and annotating the reference text of the speech samples. No specific limitation is made here.
  • Step S602 Input the speech sample into the speech recognition model, extract the speech features of the speech sample through the speech recognition model, perform connection time-series classification processing on the speech features of the speech sample to obtain the preliminary text sequence prediction result of the speech sample, and generate the prediction recognition result of the speech sample based on the preliminary text sequence prediction result and speech features of the speech sample.
  • the speech samples in the training set are input into the speech recognition model, and the speech recognition model performs speech recognition processing on the input speech samples to obtain the predicted recognition results of the speech samples.
  • a speech recognition model extracts speech features from the input speech sample. Connectivity-Time Classification (CTC) is then applied to these features.
  • CTC Connectivity-Time Classification
  • the CTC method aligns the speech with the text, yielding a preliminary text sequence prediction result for the speech sample.
  • the speech recognition model decodes and generates a predicted recognition result for the speech sample based on the preliminary text sequence prediction result and the speech features.
  • the preliminary text sequence prediction result includes multiple text tag prediction probability distributions, each of which can determine one text tag.
  • the preliminary text sequence prediction result determines a text tag sequence, and its length refers to the length of the text tag sequence determined by the preliminary text sequence prediction result.
  • the length of the preliminary text sequence prediction result determines the length of the final speech recognition result; in other words, the length of the preliminary text sequence prediction result is the length of the text tag sequence corresponding to the final speech recognition result.
  • the speech features of the speech samples are subjected to connection temporal classification processing to obtain preliminary text sequence prediction results for the speech samples. Specifically, this can be achieved in the following manner:
  • connection temporal classification prediction to obtain the connection temporal classification prediction results of the speech samples.
  • the connection temporal classification prediction results of the speech samples include: prediction frames corresponding to each frame in the speech samples, and the prediction frames include the text tag prediction probability distribution.
  • connection temporal classification prediction results are post-processed to obtain preliminary text sequence prediction results. Specifically, consecutive repeating frames in the connection temporal classification prediction results of the speech samples are merged to obtain a merged prediction result for the speech samples, where repeating frames include multiple prediction frames with the same text label corresponding to the highest probability; specific frames are deleted from the merged prediction results of the speech samples to obtain preliminary text sequence prediction results for the speech samples, where specific frames refer to prediction frames with the highest probability corresponding to a specific label.
  • the speech features of the speech samples are processed by connection temporal classification to obtain preliminary text sequence prediction results of the speech samples.
  • connection temporal classification For the specific implementation principle and technical effect, please refer to the relevant content of the foregoing embodiment, which will not be repeated here.
  • a sampler can be added to mix the hidden embedding of the speech sample with the embedding of the reference text's tag sequence to obtain a mixed embedding of the speech sample.
  • This mixed embedding is then used to replace the unmixed hidden embedding and input into the decoding module.
  • the decoding module then performs decoding processing based on the speech features of the speech sample and the mixed embedding to obtain the predicted recognition result of the speech sample. This helps the decoding module better learn the relationships between different text tags, enhances the decoding module's ability to model the vocabulary, and thus improves the accuracy of speech recognition.
  • Figure 7 is a framework diagram of the speech recognition model training provided in this embodiment.
  • a sampler is added as shown in Figure 7.
  • the input of the sampler includes the hidden layer embedding of the speech sample and the labeled sequence of the reference text for the speech sample, and the output is the hybrid embedding of the speech sample.
  • the output hybrid embedding serves as the input to the decoding module.
  • the predicted recognition results of the speech samples are generated. This can be achieved in the following way:
  • the initial text sequence prediction results of the speech sample are mapped to the hidden layer embedding of the speech sample; the hidden layer embedding of the speech sample is mixed with the embedding of the labeled sequence of the reference text of the speech sample to obtain the hybrid embedding of the speech sample; the speech features of the speech sample and the hybrid embedding of the speech sample are decoded to obtain the prediction and recognition results of the speech sample.
  • the mapping module maps the preliminary text sequence prediction result of the speech sample to the hidden layer embedding of the speech sample, and inputs the hidden layer embedding of the speech sample into the sampler.
  • the embedding of the labeled sequence of the reference text of the speech sample is also input into the sampler.
  • the sampler mixes the hidden layer embedding of the speech sample with the embedding of the labeled sequence of the reference text of the speech sample to obtain the mixed embedding of the speech sample, and then inputs the mixed embedding of the speech sample into the decoding module.
  • the latent embeddings of the generated speech data are directly input into the decoding module (as shown by the dashed arrow in Figure 7; " ⁇ " indicates that the latent embeddings are no longer directly input into the decoding module).
  • the decoding module decodes and generates the predicted recognition result of the speech data based on the speech features and latent embeddings of the speech data. As shown in Figure 7, during the training process, the latent embeddings of the speech samples are input into the sampler.
  • the sampler mixes the latent embeddings of the speech samples with the embeddings of the reference text's tag sequence, and the resulting mixed embedding replaces the latent embeddings and is input into the decoding module.
  • the decoding module decodes and generates the predicted recognition result of the speech samples based on the speech features and mixed embeddings of the speech samples. This helps the decoding module better learn the relationships between different text tags, enhances the decoding module's ability to model the vocabulary, and thus improves the accuracy of speech recognition.
  • a portion of frames are sampled from the hidden layer embedding of the speech sample, and another portion of frames are sampled from the embedding of the reference text of the speech sample.
  • the sampled frames are then combined in sequence to obtain the mixed embedding of the speech sample.
  • the sampling rules specify the rules or implementation logic for which frames to sample from the hidden layer embedding of the speech sample and the embedded token sequence of the reference text. Based on the sampling rules, the number of frames to be sampled from the hidden layer embedding and the embedded token sequence of the reference text can be determined.
  • the sampling rule can be configured to set the sampling ratio of the latent embedding and the reference text's labeled sequence, or to set two different sampling probabilities for the latent embedding and the reference text's labeled sequence.
  • the sampling rule can be configured and adjusted according to actual application needs and experience, and this embodiment does not impose specific limitations here.
  • the sampling rule can be configured to set the sampling ratio of the hidden layer embedding and the embedded token sequence of the reference text. Based on the length (i.e. the number of frames) of the hidden layer embedding of the speech sample, to obtain a hybrid embedding of the same length as the hidden layer embedding, a portion of the frames can be randomly sampled from the hidden layer embedding of the speech sample as the corresponding frames in the hybrid embedding, and the remaining portion can use the corresponding frames in the embedded token sequence of the reference text, so that the number of frames sampled from the hidden layer embedding of the speech sample and the number of frames using the embedded token sequence of the reference text conform to the sampling ratio configured by the sampling rule.
  • the sampling rule can be configured with two different sampling probabilities for the hidden layer embedding and the embedded sequence of the reference text.
  • a hybrid embedding of the same length as the hidden layer embedding for any frame in the hybrid embedding, based on the sampling probability, one frame is sampled from the corresponding frame of the hidden layer embedding of the speech sample and the corresponding frame of the embedded sequence of the reference text. After sampling each frame based on the sampling probability, the hybrid embedding is obtained.
  • connection-time classification module continuously improves, and the accuracy of connection-time classification prediction continuously improves.
  • the sampling rules can be adjusted to increase the sampling of the hidden layer embedding determined based on the connection-time classification prediction results.
  • the accuracy of connection temporal classification prediction can be calculated based on the preliminary text sequence prediction results of the speech samples and the labeled sequence of the reference text of the speech samples. Based on the accuracy of connection temporal classification prediction, the sampling rules of the sampler are adjusted so that the number of frames sampled by the sampler from the hidden layer embedding of the speech samples increases as the accuracy of connection temporal classification prediction improves.
  • connection-time classification prediction is the proportion of correctly predicted labels in the preliminary text sequence prediction results of the speech samples.
  • a correctly predicted label refers to a predicted label in the preliminary text sequence prediction results that matches the corresponding label in the label sequence of the reference text.
  • the mapping relationship between the segmented intervals of the accuracy of connection-time classification prediction and the sampling ratio or sampling probability in the sampling rule can be configured. As the number of training iterations increases, when the segmented interval of the currently calculated accuracy of connection-time classification prediction changes, the sampling ratio or sampling probability in the sampling rule is adjusted.
  • the adjustment rule of the sampler's sampling rule based on the accuracy of connection-time classification prediction can be configured and adjusted according to the actual application scenario and experience, and is not specifically limited here.
  • the reference text's tag sequence can be input into a mapping module, which maps the reference text's tag sequence to a feature space, thus obtaining the embedding of the reference text's tag sequence.
  • the embedding of the reference text's tag sequence includes the embedding of each tag in the reference text's tag sequence, forming a tag embedding sequence.
  • the hidden layer embedding of the speech sample contains the hidden layer embeddings of multiple tags.
  • the speech recognition model training process may not require adding a sampler, and the same model framework as in the aforementioned embodiments may be used.
  • the hidden layer embeddings of the generated speech samples are directly input into the decoding module.
  • the decoding module generates the predicted recognition result of the speech data based on the speech features of the speech samples and the hidden layer embeddings.
  • Step S603 Calculate the connection temporal classification loss based on the preliminary text sequence prediction results of the speech samples and the labeled sequence of the reference text of the speech samples.
  • the initial text sequence prediction result includes multiple text tag prediction probability distributions, forming a sequence of text tag prediction probability distributions. Each text tag prediction probability distribution can determine one text tag.
  • the initial text sequence prediction result can determine a text tag sequence.
  • the probability that the text-labeled sequence determined by the preliminary text sequence prediction of the speech sample matches the labeled sequence of the reference text of the speech sample can be calculated based on the preliminary text sequence prediction result of the speech sample. In other words, the probability of obtaining the labeled sequence of the reference text of the speech sample through CTC prediction is calculated. Furthermore, the connection temporal classification loss is calculated based on the probability of obtaining the labeled sequence of the reference text of the speech sample through CTC prediction.
  • L ⁇ sub>CTC ⁇ /sub> represents the connection-time classification loss.
  • x represents the speech features of the speech sample.
  • z represents the labeled sequence of the reference text, and
  • x) represents the probability of predicting the labeled sequence of the reference text for the speech sample using CTC based on the speech features of the speech sample.
  • (x,z) ⁇ S represents the speech features of any speech sample in the training set and the labeled sequence of the reference text.
  • ln( ⁇ ) represents the natural logarithm operation.
  • connection time classification loss can also be modified in other ways.
  • the natural logarithm operation ln( ⁇ ) in formula (1) can be replaced by the logarithm operation log( ⁇ ) with base 10. This embodiment does not make specific limitations here.
  • Step S604 Calculate the cross-entropy loss based on the predicted recognition results of the speech samples and the reference text of the speech samples.
  • the cross-entropy loss function is used to calculate the cross-entropy loss value based on the predicted recognition result of the speech sample and the reference text of the speech sample.
  • any method for calculating cross-entropy loss used in speech recognition model training can be used, which will not be elaborated here.
  • Step S605 Adjust the parameters of the speech recognition model based on the connection time classification loss and cross-entropy loss.
  • the combined loss is calculated based on the connection-time classification loss and the cross-entropy loss. Based on the combined loss, the parameters of the speech recognition model are adjusted through backpropagation.
  • connection time-series classification loss and cross-entropy loss can be summed as the comprehensive loss.
  • connection-time classification loss and cross-entropy loss are weighted and summed according to the configured weighting coefficients to obtain the comprehensive loss.
  • the weighting coefficients of the connection-time classification loss and cross-entropy loss can be configured and adjusted according to actual application needs and experience, and are not specifically limited here.
  • the parameters of the speech recognition model are adjusted through backpropagation based on the calculated comprehensive loss.
  • the training strategy used to train the speech recognition model using the training set including but not limited to the selected optimization algorithm, learning rate, batch size, and training stopping conditions, can be configured and adjusted according to the actual application scenario, and is not specifically limited here.
  • the speech recognition model trained by the method in this embodiment extracts speech features from speech samples, performs CTC processing on these features to align speech with text, and generates a preliminary text sequence prediction result for the speech sample.
  • the length of this preliminary text sequence prediction result is also the length of the final speech recognition result.
  • the predicted recognition result of the speech sample is decoded, achieving non-autoregressive speech recognition.
  • the final speech recognition result can be generated in a single decoding step, significantly saving the time overhead of decoding and outputting the speech recognition result, improving the speed and efficiency of speech recognition, and achieving recognition accuracy comparable to autoregressive speech recognition models.
  • FIG 8 is a flowchart of a speech recognition method provided in an exemplary embodiment of this disclosure.
  • the execution entity in this embodiment is a server deployed with a speech recognition model. As shown in Figure 8, the specific steps of the method are as follows:
  • Step S801 In response to the terminal device's request to call the speech recognition model, obtain the speech data to be recognized.
  • the server can provide the speech recognition model's API to the edge device.
  • the edge device can send a request to the server to invoke the speech recognition model based on the speech recognition model's API, and this request contains the speech data to be recognized.
  • the server responds to the request sent by the end device to invoke the speech recognition model and extracts the speech data to be recognized from the request.
  • Step S802 Input the speech data into the speech recognition model, extract the speech features of the speech data through the speech recognition model, perform connection time-series classification processing on the speech features to obtain preliminary text sequence prediction results, and generate speech recognition results of the speech data based on the preliminary text sequence prediction results and speech features.
  • the server After acquiring the speech data to be recognized, the server inputs the speech data into the speech recognition model to perform speech recognition and obtain the speech recognition result.
  • speech data is input into a speech recognition model, which extracts speech features from the speech data. These features are then subjected to connection-time classification processing to obtain preliminary text sequence prediction results. Based on these preliminary text sequence prediction results and the speech features, a speech recognition result for the speech data is generated.
  • a speech recognition model which extracts speech features from the speech data. These features are then subjected to connection-time classification processing to obtain preliminary text sequence prediction results. Based on these preliminary text sequence prediction results and the speech features, a speech recognition result for the speech data is generated.
  • Step S803 Return the speech recognition result of the speech data to the end device.
  • the server After obtaining the speech recognition results from the speech data, the server returns the speech recognition results to the end device.
  • the edge device receives the speech recognition results returned by the server and continues to execute subsequent processing logic based on the speech recognition results.
  • the method in this embodiment extracts speech features from the speech data using a speech recognition model during speech recognition.
  • speech and text are aligned, generating a preliminary text sequence prediction result.
  • the length of this preliminary text sequence prediction result is the same as the length of the final speech recognition result.
  • the speech recognition result of the speech data is decoded, achieving non-autoregressive speech recognition.
  • the final speech recognition result can be generated in a single decoding step, significantly reducing the time overhead of decoding and outputting the speech recognition result, improving the speed and efficiency of speech recognition, and achieving recognition accuracy comparable to autoregressive speech recognition models.
  • Figure 9 is a schematic diagram of a server structure provided in an embodiment of this disclosure.
  • the server includes a memory 901 and a processor 902.
  • the memory 901 is used to store computer execution instructions and can be configured to store various other data to support operations on the server.
  • the processor 902 is communicatively connected to the memory 901 and is used to execute the computer execution instructions stored in the memory 901 to implement the technical solutions provided in any of the above method embodiments. Their specific functions and the technical effects they can achieve are similar and will not be repeated here.
  • the server may also include other components such as a firewall 903, a load balancer 904, a communication component 905, and a power supply component 906.
  • Figure 9 only schematically illustrates some components and does not imply that the server only includes the components shown in Figure 9.
  • Figure 9 uses a cloud server deployed in the cloud as an example for illustrative purposes; the server can also be deployed locally, and this embodiment does not impose specific limitations.
  • This disclosure also provides a computer-readable storage medium storing computer-executable instructions.
  • a processor executes the computer-executable instructions, it implements the method of any of the foregoing embodiments.
  • the specific functions and technical effects to be achieved are not described here.
  • This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the methods of any of the foregoing embodiments.
  • the computer program is stored in a readable storage medium, and at least one processor of the server can read the computer program from the readable storage medium.
  • the execution of the computer program by the at least one processor causes the server to perform the technical solutions provided in any of the above method embodiments; the specific functions and achievable technical effects are not elaborated here.
  • This disclosure provides a chip, including a processing module and a communication interface.
  • the processing module is capable of executing the technical solutions of the server in the aforementioned method embodiments.
  • the chip further includes a storage module (e.g., a memory), which stores instructions.
  • the processing module executes the instructions stored in the storage module, and the execution of the instructions stored in the storage module causes the processing module to execute the technical solutions provided in any of the aforementioned method embodiments.
  • the integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium.
  • These software functional modules, stored in a storage medium include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this disclosure.
  • processor can be a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), etc.
  • a general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in this disclosure can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules in at least one processor.
  • the memory may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.
  • RAM random access memory
  • non-volatile memory such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.
  • the aforementioned storage can be object storage (OSS).
  • OSS object storage
  • the aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
  • SRAM Static Random Access Memory
  • EEPROM Electrically Erasable Programmable Read-Only Memory
  • EPROM Erasable Programmable Read-Only Memory
  • PROM Programmable Read-Only Memory
  • ROM Read-Only Memory
  • magnetic storage flash memory
  • flash memory magnetic disk, or optical disk.
  • the aforementioned communication components are configured to facilitate wired or wireless communication between the device containing the communication components and other devices.
  • the device containing the communication components can access wireless networks based on communication standards, such as mobile hotspots (WiFi), second-generation (2G), third-generation (3G), fourth-generation (4G)/Long Term Evolution (LTE), fifth-generation (5G), or combinations thereof.
  • the communication components receive broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.
  • the communication components also include a Near Field Communication (NFC) module to facilitate short-range communication.
  • the NFC module may be implemented based on Radio Frequency Identification (RFID), infrared, Ultra Wide Band (UWB), Bluetooth, and other technologies.
  • RFID Radio Frequency Identification
  • UWB Ultra Wide Band
  • Bluetooth and other technologies.
  • the aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.
  • the aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
  • SRAM static random access memory
  • EEPROM electrically erasable programmable read-only memory
  • EPROM erasable programmable read-only memory
  • PROM programmable read-only memory
  • ROM read-only memory
  • magnetic storage flash memory
  • flash memory magnetic disk
  • optical disk optical disk
  • An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium.
  • the storage medium can be an integral part of the processor.
  • the processor and storage medium can reside within an application-specific integrated circuit (ASIC).
  • ASIC application-specific integrated circuit
  • the processor and storage medium can exist as discrete components within an electronic device or host device.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Machine Translation (AREA)
  • Telephonic Communication Services (AREA)

Abstract

本公开提供一种语音识别的方法、服务器、存储介质和程序产品。本公开的方法，在对语音数据进行语音识别时，通过语音识别模型提取语音数据的语音特征，对语音特征进行CTC处理，实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度；进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得较高的识别准确率。

Description

语音识别的方法、服务器、存储介质和程序产品
本公开要求申请号为202410753634.8的中国专利申请的优先权，该中国专利申请于2024年06月12日提交中国专利局，申请名称为“语音识别的方法、服务器、存储介质和程序产品”，其全部内容通过引用结合在本公开中。
技术领域
本公开涉及计算机技术，尤其涉及一种语音识别的方法、服务器、存储介质和程序产品。
背景技术
在语音识别系统中，除了识别准确率外，识别速度也是一项关键的指标。目前流行的语音识别模型多是自回归transformer模型，以自回归的方式产生输出，每一时间步的输出需依赖之前时间步的输出，这使得解码输出识别结果的时间与输出识别结果的长度成正比，这导致解码输出的速度慢，语音识别的效率低。尤其在输出序列较长的语音识别场景中，语音识别的速度很慢、效率很低。
发明内容
本公开提供一种语音识别的方法、服务器、存储介质和程序产品，用以解决自回归的语音识别模型语音识别速度慢、效率低的问题。
第一方面，本公开提供一种语音识别方法，包括：
获取待识别的语音数据；
将所述语音数据输入语音识别模型，通过所述语音识别模型提取所述语音数据的语音特征，对所述语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据所述初步的文本序列预测结果和所述语音特征，解码生成所述语音数据的语音识别结果。
第二方面，本公开提供一种语音识别方法，包括：
响应于端侧设备对语言识别模型的调用请求，获取待识别的语音数据；
将所述语音数据输入语音识别模型，通过所述语音识别模型提取所述语音数据的语音特征，对所述语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据所述初步的文本序列预测结果和所述语音特征，生成所述语音数据的语音识别结果；
向所述端侧设备返回所述语音数据的语音识别结果。
第三方面，本公开提供一种服务器，包括：
至少一个处理器；以及与所述至少一个处理器通信连接的存储器；其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述服务器执行如前述任一方面所提供的方法。
第四方面，本公开提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如前述任一方面所提供的方法。
第五方面，本公开提供一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如前述任一方面所提供的方法。
本公开提供的语音识别的方法、服务器、存储介质和程序产品，在对语音数据进行语音识别时，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度；进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得较高的识别准确率。
附图说明
此处的附图被并入说明书中并构成本公开的一部分，示出了符合本公开的实施例，并与说明书一起用于解释本公开的原理。
图1为本公开所适用的一示例系统架构的示意图；
图2为本公开一示例性实施例提供的语音识别方法的流程图；
图3为本公开一示例性实施例提供的语音识别模型的基础框架图；
图4为本公开一示例性实施例提供的语音识别模型的框架图；
图5为本公开一示例性实施例提供的语音识别的详细流程图；
图6为本公开一示例性实施例提供的语音识别模型训练方法流程图；
图7为本公开一示例性实施例提供的语音识别模型训练的框架图；
图8为本公开一示例性实施例提供的语音识别方法流程图；
图9为本公开实施例提供的一种服务器的结构示意图。
通过上述附图，已示出本公开明确的实施例，后文中将有更详细的描述。这些附图和文字描述并不是为了通过任何方式限制本公开构思的范围，而是通过参考特定实施例为本领域技术人员说明本公开的概念。
具体实施方式
这里将详细地对示例性实施例进行说明，其示例表示在附图中。下面的描述涉及附图时，除非另有表示，不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反，它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。
需要说明的是，本公开所涉及的用户信息(包括但不限于用户设备信息、用户属性信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
首先对本公开所涉及的名词进行解释：
Transformer：一种完成序列到序列映射的神经网络结构。在语音识别这一任务中，输入序列是声学特征，输出序列是识别文本。
连接时序分类(Connectionist Temporal Classification，简称CTC)：通过引入特定标签(表示空白符的标签)和自弧实现声学特征和文本的对齐。
标记(token)：也称为离散标记、或离散token，是指语音或文本的离散表征。为了区分文本和语音的离散表征，将语音的离散表征称为语音标记，将文本的离散表征称为文本标记。模型使用的词表中包含预测结果中可能出现的语音标记和文本标记，词表中每一语音标记对应一个语音单元，如音素、音节、语音片段等。词表中每一文本标记对应一个文本单元，如字、词等。
针对目前的自回归语音识别模型语音识别速度慢、效率低的问题，本公开提供一种语音识别方法，具体包括：获取待识别的语音数据，将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类CTC处理，得到初步的文本序列预测结果，通过CTC处理实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度；进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，采用非自回归的方式，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得与自回归语音识别模型相当的识别准确率。
图1为本公开所适用的一示例系统架构的示意图。如图1所示，该系统架构包括服务器和端侧设备。其中，服务器与端侧设备之间具有可通信的通信链路，能够实现服务器与端侧设备间的通信连接。
服务器是部署在云端或本地的具有计算能力的设备，例如云集群等。服务器存储有非自回归的语音识别模型。服务器负责基于非自回归的语音识别模型，对给定的语音数据进行语音识别，得到语音数据的语音识别结果，即语音数据对应的文本。具体地，服务器将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果。
端侧设备可以是运行下游应用/系统/服务的电子设备，具体可以为具有网络通信功能、运算功能以及信息显示功能的硬件设备，其包括但不限于智能手机、平板电脑、台式电脑、本地服务器、云端服务器等。端侧设备运行下游应用/系统/服务时需要执行语音识别任务。端侧设备运行的下游应用/系统/服务可以实现语音转写服务、多语种语音识别等。在实现下游应用/系统/服务的功能时，需要基于给定语音数据进行语音识别，并基于语音识别结果实现下游应用/系统/服务后续的处理逻辑。例如，向用户返回语音数据的内容文本；基于用户输入语音的语音识别结果分析用户需求，实现人机交互；基于用户输入语音的语音识别结果实现多语种翻译功能等。
基于图1所示的系统架构，端侧设备在需要执行语音识别任务时，向服务器发送语音识别请求，该语音识别请求包含待识别的语音数据。服务器响应于语音识别请求，获取待识别的语音数据，将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果；根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果。
进一步地，服务器向端侧设备返回语音数据的语音识别结果。端侧设备接收服务器返回的语音数据的语音识别结果，基于语音识别结果继续执行后续的处理逻辑。
示例性地，服务器可以将语音识别模型的(Application Program Interface，简称API)提供给端侧设备。端侧设备可以基于语音识别模型的API向服务器发送对语音识别模型的调用请求，该调用请求包含待识别的语音数据。服务器响应于接收到端侧设备发送的对语音识别模型的调用请求，从调用请求中提取待识别的语音数据，将语音数据输入语音识别模型进行语音识别，得到语音识别结果，并向端侧设备返回语音识别结果。端侧设备接收服务器返回的语音识别结果，基于语音识别结果继续执行后续的处理逻辑。
在一示例场景中，以云服务器的语音转写服务为例，用户可以通过端侧设备向云服务器提交待处理的语音数据，例如会议录音、电话录音、语音输入数据等各类场景产生的音频数据等。云服务器将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现语音转文本，得到语音数据对应的文本。
进一步地，云服务器向用户提供语音数据对应的文本。例如，云服务器向端侧设备发送语音数据对应的文本。或者，云服务器将语音数据对应的文本存储到指定位置，并向用户提供语音数据对应的文本的下载信息。用户通过端侧设备根据语音数据对应的文本的下载信息，将语音数据对应的文本下载到本地。
需要说明的是，本实施例提供的语音识别方法，可以应用于任一语种的语音识别，并且适用于多语种语音识别的场景。
下面以具体地实施例对本公开的技术方案以及本公开的技术方案如何解决上述技术问题进行详细说明。下面这几个具体的实施例可以相互结合，对于相同或相似的概念或过程可能在某些实施例中不再赘述。下面将结合附图，对本公开的实施例进行描述。
图2为本公开一示例性实施例提供的语音识别方法的流程图。本实施例的执行主体为前述系统架构中的服务器。如图2所示，该方法具体步骤如下：
步骤S201、获取待识别的语音数据。
在实际应用中，端侧设备在需要进行语音识别时，可以向服务器发送语音识别请求，该语音识别请求包含待识别的语音数据。服务器接收端侧设备发送的语音识别请求，并从语音识别请求中提取待识别的语音数据。
在一些应用场景中，用户还可以通过所使用的端侧设备向服务器上传待处理的语音数据。例如，用户可以将录制的会议录音、通话录音等语音文件上传到服务器。服务器接收通过端侧设备上传的语音数据。
在一些应用场景中，服务器将语音识别模型的API提供给端侧设备。端侧设备可以基于语音识别模型的API向服务器发送语音识别模型的调用请求，该调用请求包含待识别的语音数据。服务器响应于接收到端侧设备发送的对语音识别模型的调用请求，从调用请求中提取待识别的语音数据。
本实施例中，待识别的语音数据可以是各类格式的语音数据，服务器可以接收其他设备发送的待识别的语音数据，或者从存储设备中读取预先存储的语音数据，或者从第三方平台获取待识别的语音数据，此处对于服务器获取待识别的语音数据的方式和来源不做具体限定。
步骤S202、将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果。
在获取到待识别的语音数据之后，服务器将语音数据输入语音识别模型，通过语音识别模型对输入的语音数据进行语音识别处理，获得语音识别结果。
具体地，通过语音识别模型，提取输入的语音数据的语音特征，对语音特征进行连接时序分类CTC处理，通过CTC的方法将语音与文本对齐，得到一个初步的文本序列预测结果。进一步地，通过语音识别模型，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果。
其中，初步的文本序列预测结果包括多个文本标记预测概率分布，每一文本标记预测概率分布能够确定一个文本标记。初步的文本序列预测结果能够确定一个文本标记序列，初步的文本序列预测结果的长度是指初步的文本序列预测结果确定的文本标记序列的长度。初步的文本序列预测结果的长度决定了最终的语音识别结果的长度，初步的文本序列预测结果的长度作为最终的语音识别结果对应的文本标记序列的长度，由此可以得到最终语音识别结果的长度。
本实施例的方法，在实现语音识别时，通过语音识别模型提取语音数据的语音特征，通过对语音特征进行CTC处理，实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度。进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得与自回归语音识别模型相当的识别准确率。
图3为本实施例提供的语音识别模型的基础框架图。如图3所示，本实施例使用的语音识别模型包括：特征提取单元、连接时序分类单元和解码单元。
其中，特征提取单元，用于提取语音数据的语音特征。连接时序分类单元，用于对语音特征进行连接时序分类处理，得到初步的文本序列预测结果。解码单元，用于根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果。
特征提取单元可以采用现有任意一种语音特征提取模型实现，例如用于提取语音数据的语音特征的卷积神经网络(Convolutional Neural Networks，简称CNN)、或基于Transformer的语音特征提取模型等，本实施例此处不做具体限定。
在一可选实施例中，特征提取单元可以包括：声学特征提取模块和编码模块。其中，声学特征提取模块用于提取语音数据的声学特征。语音数据的声学特征可以是语音数据的梅尔频率倒谱系数(Mel Frequency Cepstrum Coefficient，简称MFCC)、线性预测编码(Linear Predictive Coding，简称LPC)等，声学特征提取模块可以采用提取语音数据的对应类型的声学特征的神经网络模型实现，此处不做具体限定。
编码模块用于对语音数据的声学特征进行编码，以提取语音数据的声学特征的高层表征，获得语音数据的语音特征。编码模块可以采用任意一种用于编码生成声学特征的高层表征的语音编码模型实现，例如，基于CNN或Transformer的语音识别模型中的编码器等，本实施例此处不做具体限定。
在一可选实施例中，连接时序分类单元包括：连接时序分类模块和后处理模块。
连接时序分类模块用于：对语音特征进行连接时序分类预测，得到连接时序分类预测结果。连接时序分类预测结果包括：与语音数据中各帧对应的预测帧，预测帧包括文本标记预测概率分布。
后处理模块用于：对连接时序分类预测结果进行后处理，得到初步的文本序列预测结果。
其中，连接时序分类模块可以是一个实现CTC算法的深度学习网络，由多个层组成，包括线性(linear)层和归一化(softmax)层。其中线性层用于学习输入的语音特征到输出的文本标记之间的线性关系，而softmax层则用于将网络的输出转换为预测概率，表示输出可能的文本标记的概率。
示例性地，连接时序分类模块可以采用扩展的双向长短期记忆网络(Bi-directional Long Short Term Memory，简称BLSTM)、隐马尔科夫模型(Hidden Markov Model，简称HMM)等用于实现CTC算法的模型实现，本实施例此处不做具体限定。
前述步骤S202中，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，具体可以采用如下方式实现：
对语音特征进行连接时序分类预测，得到连接时序分类预测结果(也称为CTC预测结果)。其中，连接时序分类预测结果包括：与语音数据中各帧对应的预测帧，预测帧包括文本标记预测概率分布。
示例性地，用T表示语音数据的帧数，语音数据的连接时序分类预测结果包括T个文本标记预测概率分布，T个文本标记预测概率分布分别与语音数据中的各帧对应，也称为T帧文本标记预测概率分布。用vocab_size表示语音识别模型所使用的词表的大小，该词表包含语音识别结果中可能出现的文本标记。在CTC处理中，增加了一个表示空字符的特殊文本标记(本实施例中称为特定标记，记为∈)，共有vocab_size+1种文本标记。连接时序分类预测结果中每一帧的文本标记预测概率分布包含该帧对应于这vocab_size+1种文本标记(包括词表中的文本标记和特定标记∈)概率分布。每一帧的文本标记预测概率分布中最大概率对应的文本标记，可以作为该文本标记预测概率分布确定的文本标记。
例如，假设语音数据共有5帧，提取语音数据中每一帧的语音特征，得到5帧语音特征。通过对5帧语音特征进行CTC处理，可以得到对应的5帧文本标记预测概率分布，分别表示为p1、p2、p3、p4、p5，构成一个维度是[5,vocab_size+1]的后验概率矩阵。
本实施例中，通过对语音特征进行连接时序分类预测，得到连接时序分类预测结果，得到一个与语音数据长度对齐的帧级别的预测结果，具体可以采用已有的CTC算法/模型实现，此处不再赘述。
进一步地，对连接时序分类预测结果进行后处理，得到初步的文本序列预测结果，具体可以采用如下方式实现：
将连接时序分类预测结果中连续的重复帧合并，得到合并预测结果，其中重复帧包括最大概率对应的文本标记相同的多个预测帧。将合并预测结果中的特定帧删除，得到初步的文本序列预测结果，特定帧是指最大概率对应的文本标记为特定标记的预测帧。
本实施例中，为了描述方便，将连接时序分类预测结果包括的各文本标记预测概率分布称为预测帧，连接时序分类预测结果包含多个预测帧。每一预测帧中最大概率对应的文本标记作为预测帧确定的文本标记。这里将确定的文本标记相同的多个预测帧称为重复帧。连续的重复帧是指：在连接时序分类预测结果中连续出现的重复。特定帧是指确定的文本标记是表示空字符的特定标记“∈”的预测帧。
在对连接时序分类预测结果进行后处理时，先将连接时序分类预测结果中出现的连续的重复帧合并成一个新的预测帧，再将其中的特定帧去除，可以得到初步的文本序列预测结果。该初步的文本序列预测结果包含的文本序列的长度(即包含的文本token的数量)将作为最终语音识别结果对应的文本序列的长度，是一个token级的预测结果。
可选地，在实现将连接时序分类预测结果中连续的重复帧合并，得到合并预测结果时，可以将连接时序分类预测结果中连续的重复帧求平均，来实现连续重复帧的合并，得到一个新的预测帧。新的预测帧的文本标记预测概率分布为重复帧的文本标记预测概率分布的平均值。连接时序分类预测结果中所有连续的重复帧合并完成后得到合并预测结果。
示例性地，假设语音数据共有5帧，提取语音数据中每一帧的语音特征，得到5帧语音特征。通过对5帧语音特征进行CTC处理，可以得到对应的5帧文本标记预测概率分布(即5个预测帧)，分别表示为p1、p2、p3、p4、p5。假设这5个预测帧确定的文本标记序列为[a，∈，b，b，∈]，其中第3个和第4个预测帧是连续的两个预测帧，并且这两个预测帧确定的文本标记相同，都是“b”，因此第3个和第4个预测帧是连续的重复帧，合并p3和p4后得到一个新的预测帧。合并后的新的预测帧为p3和p4的平均值。合并预测结果确定的文本标记序列为[a，∈，b，∈]。进一步地，将合并预测结果中确定的文本标记是特定标记“∈”的特定帧去除，得到初步的文本序列预测结果。初步的文本序列预测结果确定的文本标记序列为[a，b]。初步的文本序列预测结果包括两个文本标记预测概率分布，第一个是p1，第二个是p3和p4的平均值，形成一个维度是[2,vocab_size+1]的后验概率矩阵。
另外，在实现将连接时序分类预测结果中连续的重复帧合并，得到合并预测结果时，也可以根据连续的重复帧确定的文本标记，将连续的重复帧对应于所确定的文本标记的概率最大的预测帧，作为合并后的预测帧。
本实施例中，通过对语音特征进行连接时序分类预测，得到一个帧级别的连接时序分类预测结果；进一步地，通过将连接时序分类预测结果中连续的重复帧合并，并删除特定帧，得到一个token级别的初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度。通CTC的方式来预测一个与语音识别结果等长的初步的文本序列预测结果，具有很好的抗噪能力和对词表的建模能力，可以提升语音识别的准确度。
在一可选实施例中，解码单元包括：映射模块和解码模块。映射模块用于将初步的文本序列预测结果映射为隐层嵌入。解码模块用于对语音特征和隐层嵌入进行解码处理，得到语音数据的语音识别结果。
其中，映射模块可以采用嵌入层(也称为Embedding层)实现，用于将初步的文本序列预测结果映射到特征空间，得到与语音特征维度一致的隐层嵌入。
解码模块可以采用transformer解码器实现，以语音数据的语音特征和隐层嵌入作为输入，解码生成最终的语音识别结果。在实际应用中，用于线下语音识别时，解码模块可以采用一个双向的transformer解码器(即双向的transformer decoder)实现。用于流式语音识别时，解码模块可以采用一个单向的transformer解码器(即双向的transformer decoder)实现。本实施例的方案可以应用于线下语音识别和流行语音识别的场景。
图4为本公开一示例性实施例提供的语音识别模型的框架图。在一可选实施例中，如图4所示，语音识别模型具体包括：声学特征提取模块、编码模块、连接时序分类模块、后处理模块、映射模块和解码模块。
其中，声学特征提取模块，用于提取语音数据的声学特征。
编码模块，用于对语音数据的声学特征进行编码，获得语音数据的语音特征。
连接时序分类模块，用于对语音特征进行连接时序分类预测，得到连接时序分类预测结果。
后处理模块，用于对连接时序分类预测结果进行后处理，得到初步的文本序列预测结果。
映射模块，用于将初步的文本序列预测结果映射为隐层嵌入。
解码模块，用于对语音特征和隐层嵌入进行解码处理，得到语音数据的语音识别结果。
本实施例中，语音识别模型所包含的各个模块的实现原理和技术效果参见前述实施例，此处不再赘述。
图5为本公开实施例提供的语音识别的详细流程图。基于图4所示的语音识别模型进行语音识别的详细流程如下：
步骤S501、获取待识别的语音数据。
步骤S502、提取语音数据的声学特征。
步骤S503、对语音数据的声学特征进行编码，获得语音数据的语音特征。
步骤S504、对语音特征进行连接时序分类预测，得到连接时序分类预测结果。
步骤S505、将连接时序分类预测结果中连续的重复帧合并，并删除特定帧，得到初步的文本序列预测结果。
步骤S506、将初步的文本序列预测结果映射为隐层嵌入。
步骤S507、对语音特征和隐层嵌入进行解码处理，得到语音数据的语音识别结果。
本实施例中各步骤的具体实现原理和技术效果参见前述实施例，此处不再赘述。
本实施例的方法，在实现语音识别时，通过语音识别模型提取语音数据的语音特征，通过对语音特征进行CTC处理，实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度。进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得与自回归语音识别模型相当的识别准确率。
图6为本公开一示例性实施例提供的语音识别模型训练方法流程图。如图6所示，前述实施例使用的语音识别模型的训练步骤如下：
步骤S601、获取训练集，训练集包括语音样本和语音样本的参考文本。
其中，训练语音识别模型所使用的训练集包括语音样本和语音样本的参考文本。语音样本的参考文本也即是语音样本的预测识别结果。
本实施例中使用的训练集可以来自于用于语音识别任务训练的公开数据集，也可以通过获取大量语音样本并标注语音样本的参考文本自行构建，此处不做具体限定。
步骤S602、将语音样本输入语音识别模型，通过语音识别模型提取语音样本的语音特征，对语音样本的语音特征进行连接时序分类处理，得到语音样本的初步的文本序列预测结果，根据语音样本的初步的文本序列预测结果和语音特征，生成语音样本的预测识别结果。
本实施例中，基于训练集，将训练集中的语音样本输入语音识别模型，通过语音识别模型对输入的语音样本进行语音识别处理，获得语音样本的预测识别结果。
具体地，通过语音识别模型，提取输入的语音样本的语音特征，对语音样本的语音特征进行连接时序分类CTC处理，通过CTC的方法将语音与文本对齐，得到语音样本的一个初步的文本序列预测结果。进一步地，通过语音识别模型，根据语音样本的初步的文本序列预测结果和语音特征，解码生成语音样本的预测识别结果。具体实现原理和技术效果参见前述实施例的相关内容，此处不再赘述。
其中，初步的文本序列预测结果包括多个文本标记预测概率分布，每一文本标记预测概率分布能够确定一个文本标记。初步的文本序列预测结果能够确定一个文本标记序列，初步的文本序列预测结果的长度是指初步的文本序列预测结果确定的文本标记序列的长度。初步的文本序列预测结果的长度决定了最终的语音识别结果的长度，初步的文本序列预测结果的长度也即是最终的语音识别结果对应的文本标记序列的长度。
在一可选实施例中，对语音样本的语音特征进行连接时序分类处理，得到语音样本的初步的文本序列预测结果，具体可以采用如下方式实现：
对语音样本的语音特征进行连接时序分类预测，得到语音样本的连接时序分类预测结果，述语音样本的连接时序分类预测结果包括：与语音样本中各帧对应的预测帧，预测帧包括文本标记预测概率分布。
进一步地，对连接时序分类预测结果进行后处理，得到初步的文本序列预测结果。具体地，将语音样本的连接时序分类预测结果中连续的重复帧合并，得到语音样本的合并预测结果，其中重复帧包括最大概率对应的文本标记相同的多个预测帧；将语音样本的合并预测结果中的特定帧删除，得到语音样本的初步的文本序列预测结果，特定帧是指最大概率对应的文本标记为特定标记的预测帧。
本实施例中对语音样本的语音特征进行连接时序分类处理，得到语音样本的初步的文本序列预测结果，具体实现原理和技术效果参见前述实施例的相关内容，此处不再赘述。
在一可选实施例中，在模型训练过程中，可以增加一个采样器，通过采样器将语音样本的隐层嵌入与语音样本的参考文本的标记序列的嵌入混合，获得语音样本的混合嵌入，使用语音样本的混合嵌入来代替混合前的隐层嵌入输入解码模块，通过解码模块根据语音样本的语音特征和语音样本的混合嵌入进行解码处理，得到语音样本的预测识别结果，可以帮助解码模块更好地学习不同文本标记间的关系，增强解码模块对词表的建模能力，从而提升语音识别的准确度。
图7为本实施例提供的语音识别模型训练的框架图。示例性地，在图4所示语音识别模型的基础上，增加采样器的位置如图7所示，采样器的输入包括语音样本的隐层嵌入和语音样本的参考文本的标记序列，输出是语音样本的混合嵌入。输出的混合嵌入作为解码模块的输入。
在模型训练过程中，根据语音样本的初步的文本序列预测结果和语音特征，生成语音样本的预测识别结果，具体可以采用如下方式实现：
将语音样本的初步的文本序列预测结果映射为语音样本的隐层嵌入；根据语音样本的隐层嵌入与语音样本的参考文本的标记序列的嵌入混合，获得语音样本的混合嵌入；根据语音样本的语音特征和语音样本的混合嵌入进行解码处理，得到语音样本的预测识别结果。
具体地，如图7所示，映射模块将语音样本的初步的文本序列预测结果映射为语音样本的隐层嵌入，并将语音样本的隐层嵌入输入采样器。将语音样本的参考文本的标记序列的嵌入输入采样器。通过采样器根据语音样本的隐层嵌入与语音样本的参考文本的标记序列的嵌入混合，获得语音样本的混合嵌入，并将语音样本的混合嵌入输入到解码模块。
在前述实施例中，执行语音识别任务的过程中，产生的语音数据的隐层嵌入会直接输入解码模块(如图7中虚线箭头所示，用“×”表示隐层嵌入不再直接输入解码模块)，解码模块根据语音数据的语音特征和隐层嵌入解码生成语音数据的预测识别结果。如图7所示，在训练过程中，语音样本的隐层嵌入输入采样器，通过采样器将语音样本的隐层嵌入与语音样本的参考文本的标记序列的嵌入混合后，将得到的混合嵌入代替隐层嵌入输入解码模块，解码模块根据语音样本的语音特征和混合嵌入解码生成语音样本的预测识别结果，可以帮助解码模块更好地学习不同文本标记间的关系，增强解码模块对词表的建模能力，从而提升语音识别的准确度。
具体地，在实现将语音样本的隐层嵌入与语音样本的参考文本的标记序列的嵌入混合，获得语音样本的混合嵌入时，根据配置的采样规则，从语音样本的隐层嵌入中采样一部分帧，从语音样本的参考文本的标记序列的嵌入中采样另一部分帧，将采样得到的帧按顺序组合，得到语音样本的混合嵌入。
其中，采样规则规定了从语音样本的隐层嵌入和参考文本的标记序列的嵌入中采样哪些帧的规则或实现逻辑，基于采样规则可以确定限制从隐层嵌入和参考文本的标记序列的嵌入中采样帧的数量。
示例性地，采样规则可以配置隐层嵌入和参考文本的标记序列的嵌入的采样比例，或者为隐层嵌入和参考文本的标记序列的嵌入配置两个不同的采样概率等，采样规则可以根据实际应用需求和经验进行配置和调整，本实施例此处不做具体限定。
可选地，采样规则可以配置隐层嵌入和参考文本的标记序列的嵌入的采样比例，根据语音样本的隐层嵌入的长度(即帧数)，要采样获得一个与隐层嵌入等长的混合嵌入，可以从语音样本的隐层嵌入随机采样部分帧，作为混合嵌入中的对应帧，剩余部分使用参考文本的标记序列的嵌入中的对应帧，使得从语音样本的隐层嵌入采样的帧数与使用参考文本的标记序列的嵌入的帧数符合采样规则配置的采样比例。
可选地，采样规则可以配置隐层嵌入和参考文本的标记序列的嵌入配置两个不同的采样概率。要采样获得一个与隐层嵌入等长的混合嵌入，对于混合嵌入中的任一帧，基于采样概率，在语音样本的隐层嵌入的对应帧和参考文本的标记序列的嵌入的对应帧中采样一个帧。基于采样概率完成各个帧的采样后，即可获得混合嵌入。
在一可选实施例中，随着训练迭代次数的增加，连接时序分类模块的能力不断地提升，连接时序分类预测的准确性不断提升，可以调整采样规则，已增加对基于连接时序分类预测结果确定的隐层嵌入的采样。
具体地，可以根据语音样本的初步的文本序列预测结果和语音样本的参考文本的标记序列，计算连接时序分类预测的准确率；根据连接时序分类预测的准确率，调整采样器的采样规则，使得采样器从语音样本的隐层嵌入中采样的帧数，随着连接时序分类预测的准确率的提高而增多。
其中，连接时序分类预测的准确率为语音样本的初步的文本序列预测结果中预测正确的标记在初步的文本序列预测结果中所占的比例。预测正确的标记是指初步的文本序列预测结果中与参考文本的标记序列中对应标记一致预测标记。
示例性地，可以配置连接时序分类预测的准确率的分段区间与采样规则中采样比例或采样概率的映射关系，随着训练迭代次数的增加，在当前计算得到的连接时序分类预测的准确率所在的分段区间发生变换时，调整采样规则中采样比例或采样概率。另外，根据连接时序分类预测的准确率对采样器的采样规则的调整规则，可以根据实际应用场景和经验配置和调整，此处不做具体限定。
本实施例中，可以通过将参考文本的标记序列输入映射模块，通过映射模块将参考文本的标记序列映射到特征空间，得到参考文本的标记序列的嵌入。参考文本的标记序列的嵌入包括参考文本的标记序列中每个标记的嵌入，是一个标记嵌入序列。语音样本的隐层嵌入包含多个标记的隐层嵌入。
在另一可选实施例中，语音识别模型训练过程中也可以不增加采样器，使用与前述实施例中相同模型框架。在语音识别模型训练过程中，产生的语音样本的隐层嵌入直接输入解码模块，解码模块根据语音样本的语音特征和隐层嵌入解码生成语音数据的预测识别结果。
步骤S603、根据语音样本的初步的文本序列预测结果和语音样本的参考文本的标记序列，计算连接时序分类损失。
其中，初步的文本序列预测结果包括多个文本标记预测概率分布，是一个文本标记预测概率分布的序列。每一文本标记预测概率分布能够确定一个文本标记。初步的文本序列预测结果能够确定一个文本标记序列。
基于CTC算法，根据语音样本的初步的文本序列预测结果和语音样本的参考文本的标记序列，可以计算出语音样本的初步的文本序列预测结果确定的文本标记序列与语音样本的参考文本的标记序列一致的概率，也即通过CTC预测获得语音样本的参考文本的标记序列的概率。进一步地，根据通过CTC预测获得语音样本的参考文本的标记序列的概率，计算连接时序分类损失。
示例性地，可以采用如下公式(1)计算连接时序分类损失：
LCTC＝-∑(x,z)∈Sln(p(z|x))   (1)
其中，LCTC表示连接时序分类损失。x表示语音样本的语音特征。z表示参考文本的标记序列，p(z|x)表示根据语音样本的语音特征，通过CTC预测获得语音样本的参考文本的标记序列的概率。(x,z)∈S表示训练集中任一语音样本的语音特征和参考文本的标记序列。ln(·)表示自然对数运算。
另外，所使用的连接时序分类损失的计算方式还可以有其他变形，如公式(1)中的自然对数运算ln(·)可以替换成以10为底的对数运算log(·)等，本实施例此处不做具体限定。
步骤S604、根据语音样本的预测识别结果和语音样本的参考文本，计算交叉熵损失。
该步骤中，使用交叉熵损失函数(Cross-Entropy Loss)，根据语音样本的预测识别结果和语音样本的参考文本，计算交叉熵损失函数值，得到交叉熵损失。具体可以使用语音识别模型训练中任意一种交叉熵损失的计算方式实现，此处不再赘述。
步骤S605、根据连接时序分类损失和交叉熵损失，调整语音识别模型的参数。
该步骤中，根据连接时序分类损失和交叉熵损失计算综合损失，根据综合损失，通过反向传播调整语音识别模型的参数。
可选地，将连接时序分类损失和交叉熵损失求和，作为综合损失。
可选地，根据配置的加权系数，对连接时序分类损失和交叉熵损失加权求和，得到综合损失。其中，连接时序分类损失和交叉熵损失的加权系数可以根据实际应用需求和经验进行配置和调整，此处不做具体限定。
示例性地，根据计算得到的综合损失，通过反向传播调整语音识别模型的参数。另外，本实施例中，使用训练集训练语音识别模型所使用的训练策略，包括但不限于所选择的优化算法、学习率、批量大小(batch)、训练停止条件等，可以根据实际应用场景进行配置和调整，此处不做具体限定。
本实施例的方法训练获得的语音识别模型，通过提取语音样本的语音特征，通过对语音样本的语音特征进行CTC处理，实现语音与文本的对齐，生成语音样本的一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度。进一步地，根据语音样本的初步的文本序列预测结果和语音特征，解码生成语音样本的预测识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得与自回归语音识别模型相当的识别准确率。
图8为本公开一示例性实施例提供的语音识别方法流程图。本实施例的执行主体为部署有语音识别模型的服务器。如图8所示，该方法具体步骤如下：
步骤S801、响应于端侧设备对语言识别模型的调用请求，获取待识别的语音数据。
本实施例中，服务器可以将语音识别模型的API提供给端侧设备。端侧设备可以基于语音识别模型的API向服务器发送对语音识别模型的调用请求，该调用请求包含待识别的语音数据。
服务器响应于接收到端侧设备发送的对语音识别模型的调用请求，从调用请求中提取待识别的语音数据。
步骤S802、将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据初步的文本序列预测结果和语音特征，生成语音数据的语音识别结果。
在获取到待识别的语音数据之后，服务器将语音数据输入语音识别模型进行语音识别，得到语音识别结果。
具体地，将语音数据输入语音识别模型，通过语音识别模型提取语音数据的语音特征，对语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据初步的文本序列预测结果和语音特征，生成语音数据的语音识别结果。具体实现原理和技术效果参见前述实施例的相关内容，本实施例此处不再赘述。
步骤S803、向端侧设备返回语音数据的语音识别结果。
在获得语音数据的语音识别结果之后，服务器向端侧设备返回语音识别结果。
端侧设备接收服务器返回的语音识别结果，基于语音识别结果继续执行后续的处理逻辑。
本实施例的方法，在实现语音识别时，通过语音识别模型提取语音数据的语音特征，通过对语音特征进行CTC处理，实现语音与文本的对齐，生成一个初步的文本序列预测结果，该初步的文本序列预测结果的长度也即是最终语音识别结果的长度。进一步地，根据初步的文本序列预测结果和语音特征，解码生成语音数据的语音识别结果，实现非自回归的语音识别，通过一步解码即可生成最终的语音识别结果，大大节省了解码输出语音识别结果的时间开销，提升了语音识别的速度和效率，并且可以取得与自回归语音识别模型相当的识别准确率。
图9为本公开实施例提供的一种服务器的结构示意图。如图9所示，该服务器包括：存储器901和处理器902。存储器901，用于存储计算机执行指令，并可被配置为存储其它各种数据以支持在服务器上的操作。处理器902，与存储器901通信连接，用于执行存储器901存储的计算机执行指令，以实现上述任一方法实施例所提供的技术方案，其具体功能和所能实现的技术效果类似，此处不再赘述。
可选的，如图9所示，该服务器还包括：防火墙903、负载均衡器904、通信组件905、电源组件906等其它组件。图9中仅示意性给出部分组件，并不意味着服务器只包括图9所示组件。图9中仅以服务器为部署在云端的云服务器为例进行示例性地说明，服务器还可以部署在本地，本实施例此处不做具体限定。
本公开实施例还提供一种计算机可读存储介质，计算机可读存储介质中存储有计算机执行指令，当处理器执行计算机执行指令时，实现前述任一实施例的方法，具体功能和所能实现的技术效果此处不再赘述。
本公开实施例还提供一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现前述任一实施例的方法。计算机程序存储在可读存储介质中，服务器的至少一个处理器可以从可读存储介质读取计算机程序，至少一个处理器执行计算机程序使得服务器执行上述任一方法实施例所提供的技术方案，具体功能和所能实现的技术效果此处不再赘述。
本公开实施例提供一种芯片，包括：处理模块与通信接口，该处理模块能执行前述方法实施例中服务器的技术方案。可选的，该芯片还包括存储模块(如，存储器)，存储模块用于存储指令，处理模块用于执行存储模块存储的指令，并且对存储模块中存储的指令的执行使得处理模块执行前述任一方法实施例所提供的技术方案。
上述以软件功能模块的形式实现的集成的模块，可以存储在一个计算机可读取存储介质中。上述软件功能模块存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)或处理器执行本公开各个实施例方法的部分步骤。
应理解，上述处理器可以是中央处理单元(Central Processing Unit，简称CPU)、图形处理器(graphics processing unit，简称GPU)，还可以是其它通用处理器、数字信号处理器(Digital Signal Processor，简称DSP)、专用集成电路(Application Specific Integrated Circuit，简称ASIC)等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本公开所公开的方法的步骤可以直接体现为硬件处理器执行完成，或者用至少一个处理器中的硬件及软件模块组合执行完成。
存储器可能包含高速随机存取存储器(Random Access Memory，简称RAM)，也可能还包括非易失性存储，例如至少一个磁盘存储器，还可以为U盘、移动硬盘、只读存储器、磁盘或光盘等。
上述存储器可以是对象存储(Object Storage Service，简称OSS)。
上述存储器可以由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(Static Random Access Memory，简称SRAM)，电可擦除可编程只读存储器(Electrically Erasable Programmable Read Only Memory，简称EEPROM)，可擦除可编程只读存储器(Erasable Programmable Read Only Memory，简称EPROM)，可编程只读存储器(Programmable Read Only Memory，简称PROM)，只读存储器(Read Only Memory，简称ROM)，磁存储器，快闪存储器，磁盘或光盘。
上述通信组件被配置为便于通信组件所在设备和其他设备之间有线或无线方式的通信。通信组件所在设备可以接入基于通信标准的无线网络，如移动热点(WiFi)，第二代移动通信系统(2G)、第三代移动通信系统(3G)、第四代移动通信系统(4G)/长期演进(Long Term Evolution，简称LTE)、第五代移动通信系统(5G)等移动通信网络，或它们的组合。在一个示例性实施例中，通信组件经由广播信道接收来自外部广播管理系统的广播信号或广播相关信息。在一个示例性实施例中，通信组件还包括近场通信(Near Field Communication，简称NFC)模块，以促进短程通信。例如，在NFC模块可基于射频识别(Radio Frequency Identification，简称RFID)技术，红外技术，超宽带(Ultra Wide Band，简称UWB)技术，蓝牙技术和其他技术来实现。
上述电源组件，为电源组件所在设备的各种组件提供电力。电源组件可以包括电源管理系统，一个或多个电源，及其他与为电源组件所在设备生成、管理和分配电力相关联的组件。
上述存储介质可以是由任何类型的易失性或非易失性存储设备或者它们的组合实现，如静态随机存取存储器(SRAM)，电可擦除可编程只读存储器(EEPROM)，可擦除可编程只读存储器(EPROM)，可编程只读存储器(PROM)，只读存储器(ROM)，磁存储器，快闪存储器，磁盘或光盘。存储介质可以是通用或专用计算机能够存取的任何可用介质。
一种示例性的存储介质耦合至处理器，从而使处理器能够从该存储介质读取信息，且可向该存储介质写入信息。当然，存储介质也可以是处理器的组成部分。处理器和存储介质可以位于专用集成电路中。当然，处理器和存储介质也可以作为分立组件存在于电子设备或主控设备中。
需要说明的是，在本文中，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者装置不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者装置所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括该要素的过程、方法、物品或者装置中还存在另外的相同要素。
上述本公开实施例的顺序仅仅为了描述，不代表实施例的优劣。另外，在上述实施例及附图中的描述的一些流程中，包含了按照特定顺序出现的多个操作，但是应该清楚了解，这些操作可以不按照其在本文中出现的顺序来执行或并行执行，仅仅是用于区分开各个不同的操作，序号本身不代表任何的执行顺序。另外，这些流程可以包括更多或更少的操作，并且这些操作可以按顺序执行或并行执行。需要说明的是，本文中的“第一”、“第二”等描述，是用于区分不同的消息、设备、模块等，不代表先后顺序，也不限定“第一”和“第二”是不同的类型。“多个”的含义是两个以上，除非另有明确具体的限定。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到上述实施例方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件，但很多情况下前者是更佳的实施方式。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，空调器，或者网络设备等)执行本公开各个实施例的方法。
本领域技术人员在考虑说明书及实践这里公开的发明后，将容易想到本公开的其它实施方案。本公开旨在涵盖本公开的任何变型、用途或者适应性变化，这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。
以上仅为本公开的优选实施例，并非因此限制本公开的专利范围，凡是利用本公开说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本公开的专利保护范围内。

Claims (17)

  1. 一种语音识别方法，其中，包括：
    获取待识别的语音数据；
    将所述语音数据输入语音识别模型，通过所述语音识别模型提取所述语音数据的语音特征，对所述语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据所述初步的文本序列预测结果和所述语音特征，解码生成所述语音数据的语音识别结果。
  2. 根据权利要求1所述的方法，其中，所述对所述语音特征进行连接时序分类处理，得到初步的文本序列预测结果，包括：
    对所述语音特征进行连接时序分类预测，得到连接时序分类预测结果，所述连接时序分类预测结果包括：与所述语音数据中各帧对应的预测帧，所述预测帧包括文本标记预测概率分布；
    对所述连接时序分类预测结果进行后处理，得到初步的文本序列预测结果。
  3. 根据权利要求2所述的方法，其中，所述对所述连接时序分类预测结果进行后处理，得到初步的文本序列预测结果，包括：
    将所述连接时序分类预测结果中连续的重复帧合并，得到合并预测结果，其中所述重复帧包括多个预测帧，所述多个预测帧中最大概率对应的文本标记相同；
    将所述合并预测结果中的特定帧删除，得到初步的文本序列预测结果，所述特定帧是指最大概率对应的文本标记为特定标记的预测帧。
  4. 根据权利要求3所述的方法，其中，将所述连接时序分类预测结果中连续的重复帧合并，得到合并预测结果，包括：
    将所述连接时序分类预测结果中连续的重复帧合并成一个新的预测帧，得到合并预测结果，所述新的预测帧的文本标记预测概率分布为所述重复帧的文本标记预测概率分布的平均值。
  5. 根据权利要求1-4中任一项所述的方法，其中，所述根据所述初步的文本序列预测结果和所述语音特征，解码生成所述语音数据的语音识别结果，包括：
    将所述初步的文本序列预测结果映射为隐层嵌入；
    对所述语音特征和所述隐层嵌入进行解码处理，得到所述语音数据的语音识别结果。
  6. 根据权利要求1-5中任一项所述的方法，其中，所述提取所述语音数据的语音特征，包括：
    提取所述语音数据的声学特征；
    对所述语音数据的声学特征进行编码，获得所述语音数据的语音特征。
  7. 根据权利要求1-6中任一项所述的方法，其中，所述语音识别模型包括：声学特征提取模块、编码模块、连接时序分类模块、后处理模块、映射模块和解码模块，
    所述声学特征提取模块，用于提取所述语音数据的声学特征；
    所述编码模块，用于对所述语音数据的声学特征进行编码，获得所述语音数据的语音特征；
    所述连接时序分类模块，用于对所述语音特征进行连接时序分类预测，得到连接时序分类预测结果；
    所述后处理模块，用于对所述连接时序分类预测结果进行后处理，得到初步的文本序列预测结果；
    所述映射模块，用于将所述初步的文本序列预测结果映射为隐层嵌入；
    所述解码模块，用于对所述语音特征和所述隐层嵌入进行解码处理，得到所述语音数据的语音识别结果。
  8. 根据权利要求1-7中任一项所述的方法，其中，还包括：
    获取训练集，所述训练集包括语音样本和所述语音样本的参考文本；
    将所述语音样本输入所述语音识别模型，通过所述语音识别模型提取所述语音样本的语音特征，对所述语音样本的语音特征进行连接时序分类处理，得到所述语音样本的初步的文本序列预测结果，根据所述语音样本的初步的文本序列预测结果和语音特征，生成所述语音样本的预测识别结果；
    根据所述语音样本的初步的文本序列预测结果和所述语音样本的参考文本的标记序列，计算连接时序分类损失；
    根据所述语音样本的预测识别结果和所述语音样本的参考文本，计算交叉熵损失；
    根据所述连接时序分类损失和所述交叉熵损失，调整所述语音识别模型的参数。
  9. 根据权利要求8所述的方法，其中，所述对所述语音样本的语音特征进行连接时序分类处理，得到所述语音样本的初步的文本序列预测结果，包括：
    对所述语音样本的语音特征进行连接时序分类预测，得到所述语音样本的连接时序分类预测结果，所述语音样本的连接时序分类预测结果包括：与所述语音样本中各帧对应的预测帧，所述预测帧包括文本标记预测概率分布；
    将所述语音样本的连接时序分类预测结果中连续的重复帧合并，得到所述语音样本的合并预测结果，其中所述重复帧包括最大概率对应的文本标记相同的多个预测帧；
    将所述语音样本的合并预测结果中的特定帧删除，得到所述语音样本的初步的文本序列预测结果，所述特定帧是指最大概率对应的文本标记为特定标记的预测帧。
  10. 根据权利要求8或9所述的方法，其中，根据所述语音样本的初步的文本序列预测结果和语音特征，生成所述语音样本的预测识别结果，包括：
    将所述语音样本的初步的文本序列预测结果映射为所述语音样本的隐层嵌入；
    将所述语音样本的隐层嵌入与所述语音样本的参考文本的标记序列的嵌入混合，获得所述语音样本的混合嵌入；
    对所述语音样本的语音特征和所述语音样本的混合嵌入进行解码处理，得到所述语音样本的预测识别结果。
  11. 根据权利要求10所述的方法，其中，所述将所述语音样本的隐层嵌入与所述语音样本的参考文本的标记序列的嵌入混合，获得所述语音样本的混合嵌入，包括：
    根据配置的采样规则，从所述语音样本的隐层嵌入中采样至少一帧，从所述语音样本的参考文本的标记序列的嵌入中采样缺失的帧，将采样得到的帧按顺序组合，得到所述语音样本的混合嵌入。
  12. 根据权利要求11所述的方法，其中，还包括：
    根据所述语音样本的初步的文本序列预测结果和所述语音样本的参考文本的标记序列，计算连接时序分类预测的准确率；
    根据所述连接时序分类预测的准确率，调整所述采样规则，使得从所述语音样本的隐层嵌入中采样的帧数，随着所述连接时序分类预测的准确率的提高而增多。
  13. 根据权利要求1-12中任一项所述的方法，其中，所述获取待识别的语音数据，包括：
    接收端侧设备发送的语音识别请求，从所述语音识别请求中提取待识别的语音数据；
    或者，
    接收通过端侧设备上传的语音数据。
  14. 一种语音识别方法，其中，包括：
    响应于端侧设备对语言识别模型的调用请求，获取待识别的语音数据；
    将所述语音数据输入语音识别模型，通过所述语音识别模型提取所述语音数据的语音特征，对所述语音特征进行连接时序分类处理，得到初步的文本序列预测结果，根据所述初步的文本序列预测结果和所述语音特征，生成所述语音数据的语音识别结果；
    向所述端侧设备返回所述语音数据的语音识别结果。
  15. 一种服务器，其中，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；
    其中，所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述服务器执行权利要求1-14任一项所述的方法。
  16. 一种计算机可读存储介质，其中，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1-14任一项所述的方法。
  17. 一种计算机程序产品，包括计算机程序，其中，该计算机程序被处理器执行时实现如权利要求1-14任一项所述的方法。
PCT/CN2025/089471 2024-06-12 2025-04-17 语音识别的方法、服务器、存储介质和程序产品 Pending WO2025256262A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202410753634.8A CN118645105A (zh) 2024-06-12 2024-06-12 语音识别的方法、服务器、存储介质和程序产品
CN202410753634.8 2024-06-12

Publications (1)

Publication Number Publication Date
WO2025256262A1 true WO2025256262A1 (zh) 2025-12-18

Family

ID=92658807

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2025/089471 Pending WO2025256262A1 (zh) 2024-06-12 2025-04-17 语音识别的方法、服务器、存储介质和程序产品

Country Status (2)

Country Link
CN (1) CN118645105A (zh)
WO (1) WO2025256262A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN118645105A (zh) * 2024-06-12 2024-09-13 阿里巴巴（中国）有限公司 语音识别的方法、服务器、存储介质和程序产品
CN120164486A (zh) * 2025-03-20 2025-06-17 北京百度网讯科技有限公司 目标音频生成方法及装置、电子设备和介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112466288A (zh) * 2020-12-18 2021-03-09 北京百度网讯科技有限公司 语音识别方法、装置、电子设备及存储介质
CN113362813A (zh) * 2021-06-30 2021-09-07 北京搜狗科技发展有限公司 一种语音识别方法、装置和电子设备
CN113436620A (zh) * 2021-06-30 2021-09-24 北京有竹居网络技术有限公司 模型的训练方法、语音识别方法、装置、介质及设备
CN116682432A (zh) * 2022-09-23 2023-09-01 荣耀终端有限公司 语音识别方法、电子设备及可读介质
CN117953896A (zh) * 2023-08-10 2024-04-30 马上消费金融股份有限公司 语音识别方法、装置、电子设备及存储介质
US20240144917A1 (en) * 2022-10-26 2024-05-02 Google Llc Exporting modular encoder features for streaming and deliberation asr
CN118645105A (zh) * 2024-06-12 2024-09-13 阿里巴巴（中国）有限公司 语音识别的方法、服务器、存储介质和程序产品

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112466288A (zh) * 2020-12-18 2021-03-09 北京百度网讯科技有限公司 语音识别方法、装置、电子设备及存储介质
CN113362813A (zh) * 2021-06-30 2021-09-07 北京搜狗科技发展有限公司 一种语音识别方法、装置和电子设备
CN113436620A (zh) * 2021-06-30 2021-09-24 北京有竹居网络技术有限公司 模型的训练方法、语音识别方法、装置、介质及设备
CN116682432A (zh) * 2022-09-23 2023-09-01 荣耀终端有限公司 语音识别方法、电子设备及可读介质
US20240144917A1 (en) * 2022-10-26 2024-05-02 Google Llc Exporting modular encoder features for streaming and deliberation asr
CN117953896A (zh) * 2023-08-10 2024-04-30 马上消费金融股份有限公司 语音识别方法、装置、电子设备及存储介质
CN118645105A (zh) * 2024-06-12 2024-09-13 阿里巴巴（中国）有限公司 语音识别的方法、服务器、存储介质和程序产品

Also Published As

Publication number Publication date
CN118645105A (zh) 2024-09-13

Similar Documents

Publication Publication Date Title
KR102401942B1 (ko) 번역품질 평가 방법 및 장치
US20230127787A1 (en) Method and apparatus for converting voice timbre, method and apparatus for training model, device and medium
CN110516253B (zh) 中文口语语义理解方法及系统
JP5901001B1 (ja) 音響言語モデルトレーニングのための方法およびデバイス
WO2022178969A1 (zh) 语音对话数据处理方法、装置、计算机设备及存储介质
WO2025256262A1 (zh) 语音识别的方法、服务器、存储介质和程序产品
CN114171016B (zh) 语音交互的方法、装置、电子设备及存储介质
CN108419094A (zh) 视频处理方法、视频检索方法、装置、介质及服务器
US20260031083A1 (en) Data processing system and method for speech recognition model, and speech recognition method
CN110310619A (zh) 多音字预测方法、装置、设备及计算机可读存储介质
US20240078385A1 (en) Method and apparatus for generating text
CN113362804A (zh) 一种合成语音的方法、装置、终端及存储介质
CN115081459B (zh) 口语文本生成方法、装置、设备及存储介质
CN112669842A (zh) 人机对话控制方法、装置、计算机设备及存储介质
CN119360818A (zh) 基于人工智能的语音生成方法、装置、计算机设备及介质
CN115168544B (zh) 信息提取方法及电子设备和存储介质
CN118070072A (zh) 基于人工智能的问题处理方法、装置、设备及存储介质
CN112036186A (zh) 语料标注方法、装置、计算机存储介质及电子设备
CN115620726A (zh) 语音文本生成方法、语音文本生成模型的训练方法、装置
CN113793598A (zh) 语音处理模型的训练方法和数据增强方法、装置及设备
CN117556057A (zh) 知识问答方法、向量数据库构建方法及装置
CN114882880B (zh) 基于解码器的语音唤醒方法及其相关设备
CN114118068B (zh) 训练文本数据的扩增方法、装置及电子设备
CN110717316A (zh) 字幕对话流的主题分割方法及装置
CN115359786A (zh) 多意图语义理解模型训练和使用方法以及装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 25820942

Country of ref document: EP

Kind code of ref document: A1