WO2025016325A1 - 语音处理方法、存储介质及系统 - Google Patents

语音处理方法、存储介质及系统 Download PDF

Info

Publication number
WO2025016325A1
WO2025016325A1 PCT/CN2024/105264 CN2024105264W WO2025016325A1 WO 2025016325 A1 WO2025016325 A1 WO 2025016325A1 CN 2024105264 W CN2024105264 W CN 2024105264W WO 2025016325 A1 WO2025016325 A1 WO 2025016325A1
Authority
WO
WIPO (PCT)
Prior art keywords
speech
segments
decoding
feature
conversion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/105264
Other languages
English (en)
French (fr)
Inventor
陈谦
郑斯奇
张仕良
张庆林
王雯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Alibaba China Co Ltd
Original Assignee
Alibaba China Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Alibaba China Co Ltd filed Critical Alibaba China Co Ltd
Publication of WO2025016325A1 publication Critical patent/WO2025016325A1/zh
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/16—Speech classification or search using artificial neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/28—Constructional details of speech recognition systems
    • G10L15/30—Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
    • G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique
    • G10L25/30—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique using neural networks

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a speech processing method, storage medium and system.
  • Speech processing is a technology that uses computers to analyze, recognize, synthesize and understand human language. With the advent of the intelligent era, the application of speech processing is becoming more and more extensive, and it is of great significance to provide a more intelligent human-computer interaction experience. In related technologies, different speech processing models are used to process speech data streams. However, due to different construction principles, various speech processing models have different degrees of defects when performing speech processing, resulting in low processing efficiency of speech data streams and poor recognition accuracy.
  • the embodiments of the present disclosure provide a voice processing method, a storage medium, and a system to at least solve the technical problems of low processing efficiency and poor recognition accuracy when processing voice data streams in related technologies.
  • a speech processing method comprising: obtaining speech information to be processed, wherein the speech information to be processed comprises: a plurality of speech segments; performing feature conversion on the plurality of speech segments to obtain conversion results; performing feature decoding on the conversion results to obtain decoding results; and determining a target speech processing result based on the decoding results.
  • a speech processing method including: obtaining speech information to be recognized, wherein the speech information to be recognized includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain multiple speech feature vectors; performing feature decoding on the multiple speech feature vectors to obtain multiple text segments; and determining a target speech recognition result based on the multiple text segments.
  • a speech processing method including: obtaining speech information to be processed, wherein the speech information to be processed includes: multiple speech segments; using a speech processing large model to perform feature conversion on the multiple speech segments to obtain conversion results, and performing feature decoding on the conversion results to obtain decoding results; determining a target speech processing result based on the decoding result.
  • a speech processing system including: a processor; and a memory, connected to the processor and configured to provide the processor with instructions for processing the following processing steps: obtaining speech information to be processed, wherein the speech information to be processed includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain conversion results; performing feature decoding on the conversion results to obtain decoding results; and determining a target speech processing result based on the decoding result.
  • multiple voice segments in the voice information to be processed are obtained, and then feature conversion is performed on the multiple voice segments to obtain conversion results, and then feature decoding is performed on the conversion results to obtain decoding results, and finally the target voice processing results are determined based on the decoding results, thereby achieving the purpose of efficiently processing the voice information to obtain voice processing results corresponding to the multiple voice segments, thereby achieving the technical effect of improving the processing efficiency and recognition accuracy when processing the voice data stream, and thus solving the technical problems of low processing efficiency and poor recognition accuracy when processing the voice data stream in the related art.
  • FIG1 shows a hardware structure block diagram of a computer terminal (or mobile device) for implementing a speech processing method
  • FIG2 is a flow chart of a speech processing method according to Embodiment 1 of the present disclosure.
  • FIG3 is a schematic diagram of a speech processing method according to Embodiment 1 of the present disclosure.
  • FIG4 is a schematic diagram of another speech processing method according to Embodiment 1 of the present disclosure.
  • FIG5 is a flow chart of a speech processing method according to Embodiment 2 of the present disclosure.
  • FIG6 is a flow chart of a speech processing method according to Embodiment 3 of the present disclosure.
  • FIG7 is a flow chart of a speech processing method according to Embodiment 4 of the present disclosure.
  • FIG8 is a structural block diagram of a speech processing device according to Embodiment 1 of the present disclosure.
  • FIG9 is a structural block diagram of a speech processing device according to Embodiment 2 of the present disclosure.
  • FIG10 is a structural block diagram of a speech processing device according to Embodiment 3 of the present disclosure.
  • FIG11 is a structural block diagram of a speech processing device according to Embodiment 4 of the present disclosure.
  • FIG. 12 is a structural block diagram of a computer terminal according to Embodiment 6 of the present disclosure.
  • the technical solution provided by the present disclosure is mainly implemented by large-scale model technology.
  • the large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than ten trillion model parameters.
  • the large model can also be called the foundation model/foundation model (Foundation Model).
  • the large model is pre-trained by large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters.
  • This model can adapt to a wide range of downstream tasks and has good generalization ability, such as large-scale language model (Large Language Model, LLM), multi-modal pre-training model (multi-modal pre-training model), etc.
  • LLM Large Language Model
  • multi-modal pre-training model multi-modal pre-training model
  • the pre-trained model can be fine-tuned through a small number of samples so that the large model can be applied to different tasks.
  • the large model can be widely used in natural language processing (NLP), computer vision and other fields, and can be specifically applied to computer vision tasks such as visual question answering (VQA), image description (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present disclosure, data processing by a large model of speech processing in a speech processing scenario is used as an example for explanation.
  • Connectionist Temporal Classification (CTC) model is a deep learning model for sequence classification. It uses an end-to-end training method and can process variable-length input and output sequences. The CTC model performs classification tasks by mapping the input sequence to the output sequence. In the inference phase, the CTC model uses a forward algorithm to calculate all possible tag combinations and their probabilities at each time step, and selects the output result from them.
  • Attention-based Encoder-Decoder It is a sequence-to-sequence model used to map an input sequence of variable length to another output sequence of variable length.
  • the model mainly consists of an encoder and a decoder.
  • the encoder converts the input sequence into a vector representation of fixed length and can pass information to the decoder through this vector.
  • the decoder uses the vector as the initial state to generate the output sequence.
  • RNN-T Recurrent Neural Network-Transducer
  • Different speech processing models are used in related technologies to process speech data streams.
  • various speech processing models have defects to varying degrees when performing speech processing, resulting in low processing efficiency and poor recognition accuracy of speech data streams.
  • the CTC streaming model framework has an independent frame assumption, so its modeling context capability is relatively weak;
  • the AED model requires the use of an additionally designed block attention mechanism to achieve streaming recognition, which is not suitable for processing speech data stream tasks;
  • the RNN-T model has poor long-term dependence, and its recognition accuracy may decrease as the length of the input audio increases.
  • a method embodiment of speech processing is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.
  • the above-mentioned speech processing method provided by the embodiment of the present disclosure can be applied to the application scenario shown in Figure 1, but is not limited thereto.
  • the speech processing large model is deployed in the server 10, and the server 10 can
  • One or more client devices 20 may be connected via a local area network, a wide area network, an Internet connection, or other types of data networks.
  • the client devices 20 herein may include but are not limited to: smart phones, tablet computers, laptop computers, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc.
  • the client device 20 may interact with the user through a graphical user interface to implement the call to the large model, thereby implementing the method provided by the embodiment of the present disclosure.
  • a system composed of a client device and a server can perform the following steps: the server obtains voice information to be processed, wherein the voice information to be processed includes: multiple voice segments; feature conversion is performed on the multiple voice segments to obtain conversion results; feature decoding is performed on the conversion results to obtain decoding results; and the target voice processing result is determined based on the decoding results.
  • a target voice processing type is selected from a plurality of candidate voice processing types, wherein the plurality of candidate voice processing types include: voice recognition type, voice translation type, and voice interaction type; in response to a second touch operation on the graphical user interface, a plurality of voice segments corresponding to the target voice processing type are determined; in response to a third touch operation on the graphical user interface, the plurality of voice segments are converted into a plurality of voice feature vectors, feature decoding is performed on the plurality of voice feature vectors to obtain a plurality of decoded segments, and the plurality of decoded segments are spliced to obtain the target voice processing result; and the target voice processing result is displayed in the graphical user interface.
  • the embodiment of the present disclosure can be performed in the client device.
  • FIG2 is a flow chart of a speech processing method according to Embodiment 1 of the present disclosure. As shown in FIG2, the method includes the following steps:
  • Step S21 obtaining voice information to be processed, wherein the voice information to be processed includes: a plurality of voice segments;
  • Step S22 performing feature conversion on the multiple speech segments to obtain conversion results
  • Step S23 feature decoding is performed on the conversion result to obtain a decoding result
  • Step S24 determining the target speech processing result based on the decoding result.
  • the above-mentioned voice information to be processed can be obtained by collecting the user's voice through the audio component on the terminal device, such as using the mobile phone microphone component to obtain multiple voice clips of the user; the voice information to be processed can also be extracted from the recorded audio files, such as the voice messages in the application software; the voice information to be processed can also be obtained from audio resources on the Internet, such as live broadcasts, online music radio stations, etc.; the voice information to be processed can also be obtained from smart devices. Voice information to be processed obtained through the above channels can be used for voice recognition, voice synthesis, sentiment analysis, speech recognition, etc. Applications such as person recognition.
  • the multiple voice segments in the above-mentioned voice information to be processed may include continuous multi-person conversation content or discrete personal voice content, which is not limited in the embodiment of the present disclosure.
  • Feature conversion refers to the process of converting the speech data to be processed into a new feature representation in a certain way. Feature conversion can reduce the dimension of the data, remove redundant information, extract useful features, and improve the robustness and generalization ability of the algorithm.
  • Feature decoding is performed on multiple speech feature vectors to obtain decoding results, and the target speech processing result can be determined based on the decoding results.
  • autoregressive decoding classification decoding
  • generative decoding sequence-to-sequence decoding
  • path decoding path decoding
  • attention decoding can be used. These decoding methods can be selected according to specific tasks and application scenarios.
  • autoregressive decoding is a common decoding method, which is usually used in sequence generation tasks such as natural language processing and speech recognition.
  • sequence generation tasks such as natural language processing and speech recognition.
  • the basic idea of autoregressive decoding is that in a generation process, each step generates each element in the sequence in turn, and the result of each generation will affect the next generation.
  • This can be represented by a conditional probability model. For example, when using RNN to generate a text sequence, one character is generated at a time, and the probability of each character's occurrence depends on the previously generated characters and the generation conditions.
  • Categorical decoding maps the speech feature vector to a fixed set of categories, each category represents a phoneme, syllable or word. This decoding method is often used in speech recognition tasks.
  • Generative decoding uses a generative model, such as a generative adversarial network and a variational autoencoder, to map the speech feature vector to the sound waveform space, thereby generating a speech signal corresponding to the input feature.
  • Sequence-to-sequence decoding uses a recurrent neural network, such as a long short-term memory network and a gated recurrent unit, to map a speech feature vector sequence to another speech feature vector sequence to achieve tasks such as speech conversion and speech synthesis; path decoding maps the speech feature vector to a specific path, which can represent the duration and boundary of a phoneme, syllable, word or speech.
  • Attention decoding uses the attention mechanism to align the input speech feature vector sequence with another speech feature vector sequence to generate the corresponding output sequence.
  • step S22 feature conversion is performed on multiple speech segments, and the conversion results include: at multiple consecutive moments, a large speech processing model is used to perform speech encoding or speech segmentation on the multiple speech segments in turn to obtain multiple speech feature vectors, wherein the multiple moments correspond one-to-one to the multiple speech segments.
  • the speech processing model includes: a feature conversion model and a feature decoding model.
  • the main function of the feature conversion model is to convert the speech signal from the time domain or frequency domain to the feature domain.
  • Common feature conversion models include Fourier transform, discrete cosine transform, wavelet transform, etc., which can extract the spectral features, energy features, etc. of the speech signal.
  • the feature decoding model (is another important model in speech processing, and its main function is to decode the speech signal in the feature domain into the original time domain or frequency domain signal.
  • the feature decoding model can restore the feature signal to the original speech signal by inverse transformation, thereby realizing the restoration of the speech signal.
  • the feature conversion model and feature decoding model are usually used together.
  • the former is used to convert speech signals into feature representations, and the latter is used to decode feature representations into original signals.
  • Such a processing process can extract useful information from speech signals and reduce signal redundancy, thereby facilitating subsequent tasks such as speech recognition, speech synthesis, and speech enhancement.
  • a large speech processing model is used to sequentially perform speech encoding on multiple speech segments using a speech encoder or perform speech segmentation using a speech segmentor to obtain multiple speech feature vectors.
  • the multiple speech segments can be preprocessed first, such as cutting speech segments of fixed length, performing speech signal enhancement, removing noise, etc., and then extracting speech features, that is, converting the speech segments into feature representations that can be understood by the speech encoder, wherein the above-mentioned speech features may include Mel-Frequency Cepstral Coefficients (MFCC), spectrogram, Linear Predictive Cepstral Coding (LPCC), etc.
  • MFCC Mel-Frequency Cepstral Coefficients
  • LPCC Linear Predictive Cepstral Coding
  • a speech encoder model is constructed by establishing a neural network model that accepts speech features as input and generates encoded speech feature vector representations as output, wherein the above-mentioned speech encoder models include autoencoders, variational autoencoders, etc.
  • Each The speech features of the speech segment corresponding to each moment are input into the speech encoder model to obtain the corresponding encoded speech feature vectors, which can be used to perform tasks such as recognition, classification, and clustering of the speech segments.
  • the multiple speech segments can be preprocessed first, such as intercepting a speech segment of a fixed length, performing speech signal enhancement, removing noise, etc., and then extracting speech features, that is, converting the speech segments into feature representations that can be understood by the speech encoder, wherein the above-mentioned speech features may include MFCC, spectrogram, LPCC, etc. Then, a speech segmentation model is constructed, and a neural network segmentation model is established to accept speech features as input and generate output speech feature vectors after segmentation.
  • Commonly used speech segmentation models include convolutional neural networks, recurrent neural networks, convolution-recurrent neural networks, etc.
  • the speech features corresponding to the speech segment at each moment are input into the speech segmentation model to obtain the corresponding segmentation results.
  • the output format of the speech segmentation results may include annotated text and timestamp sequence.
  • the segmentation results may also be post-processed and optimized, such as removing repeated words, merging adjacent words, performing phoneme-level segmentation, etc.
  • the segmentation results and related information such as original speech data, segmentation text, timestamp sequence, etc., are stored as speech feature vectors corresponding to the speech segments.
  • multiple speech feature vectors are obtained, which can be used for tasks such as streaming speech recognition and streaming speech translation, thereby further improving the speech processing efficiency.
  • step S23 feature decoding is performed on the conversion result to obtain the decoding result, which includes: at multiple consecutive moments, a large speech processing model is used to perform feature decoding on multiple speech feature vectors in turn to obtain multiple decoding segments.
  • the feature decoding model is a multimodal language model.
  • a multimodal language model is a language model that can process inputs of multiple different modalities (such as text, images, audio, etc.). It can understand and generate multiple different types of data at the same time, and can effectively integrate and interact with these different modal data.
  • the goal of a multimodal language model is to improve the performance and performance of a language model in understanding and generation tasks by combining information from different modalities.
  • a multi-modal language model based on a decoder structure is used to sequentially decode multiple speech feature vectors to obtain multiple decoded segments.
  • each decoding output will be used as part of the next decoding input. Therefore, the decoding output result can be used and the decoding can be repeated to obtain decoded segments at multiple consecutive moments until a specific termination condition is reached or the maximum number of rounds is reached.
  • they can also be post-processed, such as deduplication, sorting, filtering invalid information, etc.
  • a large speech processing model is used to sequentially perform feature decoding on multiple speech feature vectors to obtain multiple decoded segments, including: at multiple consecutive moments, a large speech processing model is used to sequentially perform feature decoding on multiple speech feature vectors and context information to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segment and the historical moment decoding segment associated with the current moment speech feature vector in the multiple speech feature vectors.
  • the above-mentioned context information refers to the information associated with the speech segment at the historical moment and the decoded segment at the historical moment.
  • the context information includes the speech segment at the historical moment, the decoded segment, and its related feature vectors.
  • the context information can provide more context, help accurately understand the speech information at the current moment, and generate more fluent and natural speech segments.
  • a large speech processing model is used to perform feature decoding on multiple speech feature vectors and context information in turn to obtain multiple decoded segments, including: at multiple consecutive moments, a large speech processing model is used to perform feature decoding on the current moment speech feature vector, the speech segment at the previous moment adjacent to the current moment speech feature vector, and the decoded segment at the previous moment adjacent to the current moment speech feature vector in turn to obtain the decoded segment at the current moment, until all multiple speech feature vectors are decoded to obtain multiple decoded segments.
  • the speech feature vector at the current moment depends on the speech segments at the previous historical moments, which are also the context information.
  • the decoded segments at the current moment are also affected by the decoded segments at the previous moments, so the decoded segments at the previous moments also constitute part of the context information.
  • the previous moment adjacent to the second moment is the first moment
  • the speech feature vector of the second moment the speech segment of the first moment and the decoded segment of the first moment are feature decoded in turn to obtain the decoded segment of the second moment, until all the speech feature vectors are decoded to obtain multiple decoded segments.
  • feature decoding is performed on the conversion result to obtain a decoding result, including: using a multimodal language model to feature decode the conversion result to obtain a decoding result; or, using a multimodal language model to non-feature decode the conversion result to obtain a decoding result.
  • feature decoding refers to the use of speech feature vectors to assist in the decoding process when decoding the conversion results using a multimodal language model. Feature decoding can more accurately predict the probability of sentence generation and improve the accuracy and effect of decoding.
  • Non-feature decoding refers to not using speech feature vectors during the decoding process, but only relying on a multimodal language model to decode the conversion results.
  • a multimodal language model can be used to flexibly decode the conversion results to further improve speech recognition efficiency and recognition accuracy.
  • a graphical user interface is provided by a terminal device, and the content displayed by the graphical user interface at least partially includes a speech processing scenario, and the speech processing method further includes:
  • Step S31 in response to a first touch operation on the graphical user interface, selecting a target speech processing type from a plurality of candidate speech processing types, wherein the plurality of candidate speech processing types include: a speech recognition type, a speech translation type, and a speech interaction type;
  • Step S32 in response to a second touch operation on the graphical user interface, determining a plurality of voice segments corresponding to the target voice processing type
  • Step S33 in response to a third touch operation on the graphical user interface, converting the plurality of speech segments into a plurality of speech feature vectors, performing feature decoding on the plurality of speech feature vectors to obtain a plurality of decoded segments, and splicing the plurality of decoded segments to obtain a target speech processing result;
  • Step S34 displaying the target speech processing result in the graphical user interface.
  • the above graphical user interface is a voice processing interface.
  • the speech processing scenario shown in the figure can be a speech recognition scenario, a speech translation scenario or a speech interaction scenario.
  • the above-mentioned graphical user interface also includes a first control (or a first touch area).
  • a target voice processing type is selected from a plurality of candidate voice processing types, wherein the plurality of candidate voice processing types include: voice recognition type, voice translation type, and voice interaction type.
  • the above-mentioned graphical user interface also includes a second control (or a second touch area).
  • a second touch operation acting on the second control or the second touch area
  • multiple voice segments corresponding to the target voice processing type are determined.
  • the above-mentioned graphical user interface also includes a third control (or a third touch area).
  • a third touch operation acting on the third control or the third touch area
  • multiple voice segments are converted into multiple voice feature vectors
  • the multiple voice feature vectors are feature decoded to obtain multiple decoded segments
  • the multiple decoded segments are spliced to obtain the target voice processing result.
  • the target speech processing results are displayed in the graphical user interface so that users can intuitively view the speech processing results and speech processing progress, further enhancing the user's intelligent interactive experience.
  • first touch operation, the second touch operation and the third touch operation can all be operations in which a user touches the display screen of the terminal device with a finger and touches the terminal device.
  • the touch operation can include single-point touch and multi-point touch, wherein the touch operation of each touch point can include click, long press, heavy press, swipe, etc.
  • the first touch operation, the second touch operation and the third touch operation can also be touch operations implemented by input devices such as a mouse and a keyboard.
  • FIG3 is a schematic diagram of a speech processing method according to Embodiment 1 of the present disclosure. As shown in FIG3 , the method includes the following steps:
  • Step S301 obtaining voice information to be processed, wherein the voice information to be processed includes: a plurality of voice segments;
  • Step S302 at a plurality of consecutive moments, using a large speech processing model to sequentially perform speech encoding or speech segmentation on a plurality of speech segments to obtain a plurality of speech feature vectors, wherein the plurality of moments correspond one-to-one to the plurality of speech segments;
  • Step S303 at a plurality of consecutive moments, using the speech processing large model to sequentially perform feature decoding on the speech feature vector at the current moment, the speech segment at the previous moment adjacent to the speech feature vector at the current moment, and the decoded segment at the previous moment adjacent to the speech feature vector at the current moment to obtain the decoded segment at the current moment, until all the speech feature vectors are decoded to obtain a plurality of decoded segments;
  • Step S304 determining the target speech processing result based on the decoding result.
  • FIG4 is a schematic diagram of another speech processing method according to Embodiment 1 of the present disclosure.
  • speech input segment 1 (Speech chunk1) is converted into a plurality of speech feature vectors after passing through a speech encoder or a speech segmenter (also called speech discretization). These vectors are fed into a multimodal language model for feature decoding, and a text segment 1 (Text chunk1) is generated based on the decoding result.
  • speech input segment 2 (Speech chunk2) is also converted into a speech feature vector through a speech encoder or a speech segmenter, and is fed into a multimodal language model for feature decoding to generate a text segment 2 (Text chunk2).
  • streaming decoding is achieved, and thus the speech recognition method of the embodiment of the present disclosure can be used for tasks such as streaming speech recognition, streaming speech translation, and streaming speech interaction.
  • the speech recognition method of the embodiment of the present disclosure can realize the function of converting speech into text more accurately and efficiently in scenarios such as speech recognition and speech translation, thereby improving the user experience and providing better services to users.
  • the speech recognition method of the embodiment of the present disclosure can also be applied to the field of voice interaction, such as smart home, smart customer service and other fields, to provide users with a more intelligent interactive experience.
  • the speech recognition method of the embodiment of the present disclosure solves the contradiction between the newly input speech features and the generated text in the traditional streaming model framework, and improves the accuracy.
  • the streaming model framework based on the decoder is more unified in model structure, which is convenient for optimization and expansion.
  • the speech recognition method of the embodiment of the present disclosure proposes a new decoder-based streaming model framework, which is different from traditional streaming model frameworks such as CTC, AED, RNN-T, and has better applicability and accuracy.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • user information including but not limited to user device information, user personal information, etc.
  • data including but not limited to data used for analysis, stored data, displayed data, etc.
  • the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware.
  • the technical solution of the present disclosure, or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, a disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present disclosure.
  • a storage medium such as ROM/RAM, a disk, or an optical disk
  • FIG5 is a flow chart of a speech processing method according to Embodiment 2 of the present disclosure. As shown in FIG5, the method comprises the following steps:
  • Step S51 obtaining voice information to be recognized, wherein the voice information to be recognized includes: a plurality of voice segments;
  • Step S52 performing feature conversion on the multiple speech segments to obtain multiple speech feature vectors
  • Step S53 performing feature decoding on the multiple speech feature vectors to obtain multiple text segments
  • Step S54 determining a target speech recognition result based on the multiple text segments.
  • the embodiments of the present disclosure can be applied, but not limited to, to any scenario involving voice interaction in the fields of health, science, society, language, and art, especially in application scenarios such as voice recognition and voice translation in the fields of smart homes, smart customer service, and smart conferences.
  • the voice processing method in the embodiments of the present disclosure can more accurately and efficiently realize the function of converting voice into text, and further provide users with a more intelligent interactive experience.
  • FIG6 is a flow chart of a speech processing method according to Embodiment 3 of the present disclosure. As shown in FIG6 , the method includes the following steps:
  • Step S61 obtaining voice information to be translated, wherein the voice information to be translated includes: a plurality of first voice segments;
  • Step S62 performing feature conversion on the plurality of first speech segments to obtain a plurality of speech feature vectors
  • Step S63 performing feature decoding on the plurality of speech feature vectors to obtain a plurality of second speech segments
  • Step S64 determining a target speech translation result based on the multiple second speech segments.
  • the embodiments of the present disclosure can be applied, but not limited to, to any scenario involving voice interaction in the fields of health, science, society, language, and art, especially in application scenarios such as voice recognition and voice translation in the fields of smart homes, smart customer service, and smart conferences.
  • the voice processing method in the embodiments of the present disclosure can more accurately and efficiently realize the function of converting voice into text, and further provide users with a more intelligent interactive experience.
  • FIG. 7 is a flow chart of a speech processing method according to Embodiment 4 of the present disclosure. As shown in FIG. 7 , the method includes the following steps:
  • Step S71 obtaining voice information to be processed, wherein the voice information to be processed includes: a plurality of voice segments;
  • Step S72 using a large speech processing model to perform feature conversion on the plurality of speech segments to obtain conversion results, and feature decoding on the conversion results to obtain decoding results;
  • Step S73 determining the target speech processing result based on the decoding result.
  • the embodiments of the present disclosure can be applied, but not limited to, to any scenario involving voice interaction in the fields of health, science, society, language, and art, especially in application scenarios such as voice recognition and voice translation in the fields of smart homes, smart customer service, and smart conferences.
  • the voice processing method in the embodiments of the present disclosure can more accurately and efficiently realize the function of converting voice into text, and further provide users with a more intelligent interactive experience.
  • using a large speech processing model to perform feature conversion on a plurality of speech segments to obtain conversion results includes:
  • a large speech processing model is used to sequentially perform speech encoding or speech segmentation on multiple speech segments to obtain multiple speech feature vectors, wherein the multiple moments correspond to the multiple speech segments one by one;
  • a large speech processing model is used to perform feature decoding on multiple speech feature vectors and context information in turn to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segment and the historical moment decoding segment associated with the current moment speech feature vector in the multiple speech feature vectors.
  • FIG8 is a structural block diagram of a speech processing device according to Embodiment 1 of the present disclosure. As shown in FIG8 , the speech processing device includes:
  • the acquisition module 801 is configured to acquire voice information to be processed, wherein the voice information to be processed includes: a plurality of voice segments;
  • the conversion module 802 is configured to perform feature conversion on multiple speech segments to obtain conversion results
  • the decoding module 803 is configured to perform feature decoding on the conversion result to obtain a decoding result
  • the determination module 804 is configured to determine the target speech processing result based on the decoding result.
  • the conversion module 802 is also configured to use a large speech processing model to perform speech encoding or speech segmentation on multiple speech segments in sequence at multiple consecutive moments to obtain multiple speech feature vectors, wherein the multiple moments correspond one-to-one to the multiple speech segments.
  • the decoding module 803 is further configured to use a large speech processing model to perform feature decoding on multiple speech feature vectors in sequence at multiple consecutive moments to obtain multiple decoding segments.
  • the decoding module 803 is also configured to use a large speech processing model to perform feature decoding on multiple speech feature vectors and context information in sequence at multiple consecutive moments to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segments and historical moment decoding segments associated with the current moment speech feature vector among the multiple speech feature vectors.
  • the decoding module 803 is also configured to use a large speech processing model to perform feature decoding on the speech feature vector at the current moment, the speech segment at the previous moment adjacent to the speech feature vector at the current moment, and the decoded segment at the previous moment adjacent to the speech feature vector at the current moment in sequence at multiple consecutive moments to obtain a decoded segment at the current moment, until all multiple speech feature vectors are decoded and multiple decoded segments are obtained.
  • the speech processing large model includes: a feature conversion model and a feature decoding model.
  • the feature decoding model is a multimodal language model.
  • the decoding module 803 is further configured to: use a multimodal language model to perform feature decoding on the conversion result to obtain a decoding result; or use a multimodal language model to perform non-feature decoding on the conversion result to obtain a decoding result.
  • the speech processing device also includes: a response module 805, which is configured to respond to a first touch operation applied to the graphical user interface and select a target speech processing type from multiple candidate speech processing types, wherein the multiple candidate speech processing types include: speech recognition type, speech translation type, and speech interaction type; the response module 805 is also configured to respond to a second touch operation applied to the graphical user interface and determine multiple speech segments corresponding to the target speech processing type; the response module 805 is also configured to respond to a third touch operation applied to the graphical user interface and convert multiple speech segments into multiple speech feature vectors, perform feature decoding on the multiple speech feature vectors to obtain multiple decoded segments, and splice the multiple decoded segments to obtain a target speech processing result; and a display module 806 is configured to display the target speech processing result in the graphical user interface.
  • a response module 805 which is configured to respond to a first touch operation applied to the graphical user interface and select a target speech processing type from multiple candidate speech processing types, wherein the multiple candidate speech processing types include:
  • multiple speech segments in the speech information to be processed are obtained, and then the multiple speech segments are feature converted to obtain conversion results, and then the conversion results are feature decoded to obtain decoding results, and finally the target speech processing result is determined based on the decoding results, so as to achieve the goal of The purpose of efficiently processing voice information to obtain voice processing results corresponding to multiple voice segments is achieved, thereby achieving the technical effect of improving the processing efficiency and recognition accuracy when processing voice data streams, and further solving the technical problems of low processing efficiency and poor recognition accuracy when processing voice data streams in related technologies.
  • the acquisition module 801, conversion module 802, decoding module 803 and determination module 804 correspond to steps S21 to S24 in Embodiment 1, and the four modules and corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 1.
  • the modules or units may be hardware components or software components stored in a memory and processed by one or more processors.
  • FIG. 9 is a structural block diagram of a speech processing device according to Embodiment 2 of the present disclosure. As shown in FIG. 9 , the speech processing device includes:
  • the acquisition module 901 is configured to acquire voice information to be recognized, wherein the voice information to be recognized includes: a plurality of voice segments;
  • the conversion module 902 is configured to perform feature conversion on the multiple speech segments to obtain multiple speech feature vectors
  • a decoding module 903 is configured to perform feature decoding on a plurality of speech feature vectors to obtain a plurality of text segments;
  • the determination module 904 is configured to determine a target speech recognition result based on the multiple text segments.
  • multiple voice segments in the voice information to be recognized are obtained, and then the multiple voice segments are feature converted to obtain conversion results, and then the conversion results are feature decoded to obtain decoding results, and finally the target voice recognition results are determined based on the decoding results, thereby achieving the purpose of efficiently processing the voice information to obtain voice recognition results corresponding to the multiple voice segments, thereby achieving the technical effect of improving the recognition efficiency and recognition accuracy when processing the voice data stream, and thus solving the technical problems of low recognition efficiency and poor recognition accuracy when processing the voice data stream in the related art.
  • the acquisition module 901, conversion module 902, decoding module 903 and determination module 904 correspond to steps S51 to S54 in Embodiment 2, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 2. It should be noted that the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors.
  • FIG. 10 is a structural block diagram of a speech processing device according to Embodiment 3 of the present disclosure. As shown in FIG. 10 , The speech processing device comprises:
  • the acquisition module 1001 is configured to acquire voice information to be translated, wherein the voice information to be translated includes: a plurality of first voice segments;
  • the conversion module 1002 is configured to perform feature conversion on the plurality of first speech segments to obtain a plurality of speech feature vectors
  • a decoding module 1003 is configured to perform feature decoding on a plurality of speech feature vectors to obtain a plurality of second speech segments;
  • the determination module 1004 is configured to determine a target speech translation result based on the multiple second speech segments.
  • multiple voice segments in the voice information to be translated are obtained, and then feature conversion is performed on the multiple voice segments to obtain conversion results, and then feature decoding is performed on the conversion results to obtain decoding results, and finally a target voice translation result is determined based on the decoding results, thereby achieving the purpose of efficiently processing the voice information to obtain voice translation results corresponding to the multiple voice segments, thereby achieving the technical effect of improving the translation efficiency and translation accuracy when processing the voice data stream, and thus solving the technical problems of low translation efficiency and poor translation accuracy when processing the voice data stream in the related art.
  • the acquisition module 1001, conversion module 1002, decoding module 1003 and determination module 1004 correspond to steps S61 to S64 in Embodiment 3, and the four modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 3.
  • the modules or units may be hardware components or software components stored in a memory and processed by one or more processors.
  • FIG. 11 is a structural block diagram of a speech processing device according to Embodiment 4 of the present disclosure. As shown in FIG. 11 , the speech processing device includes:
  • the acquisition module 1101 acquires the voice information to be processed, wherein the voice information to be processed includes: a plurality of voice segments;
  • the processing module 1102 uses a speech processing large model to perform feature conversion on multiple speech segments to obtain conversion results, and performs feature decoding on the conversion results to obtain decoding results;
  • the determination module 1103 determines the target speech processing result based on the decoding result.
  • the processing module 1102 is further configured to: at multiple consecutive moments, use the speech processing large model to perform speech encoding or speech segmentation on multiple speech segments in sequence to obtain multiple speech feature vectors, wherein the multiple moments correspond to the multiple speech segments one by one; at multiple consecutive moments, use the speech processing large model to perform speech encoding or speech segmentation on multiple speech segments in sequence to obtain multiple speech feature vectors.
  • Feature decoding is performed on multiple speech feature vectors and context information to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segment and the historical moment decoded segment associated with the current moment speech feature vector in the multiple speech feature vectors.
  • the purpose of efficiently processing the speech information to obtain speech recognition results corresponding to multiple speech segments is achieved, thereby achieving the technical effect of improving the recognition efficiency and recognition accuracy when processing speech data streams, and thus solving the technical problems of low recognition efficiency and poor recognition accuracy when processing speech data streams in related technologies.
  • the acquisition module 1101, the processing module 1102 and the determination module 1103 correspond to steps S71 to S73 in Embodiment 4, and the three modules and the corresponding steps implement the same examples and application scenarios, but are not limited to the contents disclosed in Embodiment 4.
  • the modules or units may be hardware components or software components stored in a memory and processed by one or more processors.
  • the embodiment of the present disclosure may provide a computer terminal, which may be any computer terminal device in a computer terminal group.
  • the computer terminal may also be replaced by a terminal device such as a mobile terminal.
  • the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.
  • the above-mentioned computer terminal can execute the program code of the following steps in the speech processing method: obtaining speech information to be processed, wherein the speech information to be processed includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain conversion results; performing feature decoding on the conversion results to obtain decoding results; and determining the target speech processing result based on the decoding result.
  • the above-mentioned computer terminal can execute the program code of the following steps in the speech processing method: obtaining speech information to be recognized, wherein the speech information to be recognized includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain multiple speech feature vectors; performing feature decoding on the multiple speech feature vectors to obtain multiple text segments; and determining the target speech recognition result based on the multiple text segments.
  • the computer terminal can execute the program code of the following steps in the speech processing method: obtaining speech information to be translated, wherein the speech information to be translated includes: a plurality of first speech segments; performing feature conversion on the plurality of first speech segments to obtain a plurality of speech feature vectors; performing feature conversion on the plurality of speech feature vectors
  • the target speech translation result is determined based on the plurality of second speech segments.
  • Figure 12 is a structural block diagram of a computer terminal according to Embodiment 6 of the present disclosure.
  • the computer terminal may include: one or more (only one is shown in the figure) processors 122, a memory 124, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.
  • the memory can be configured to store software programs and modules, such as the program instructions/modules corresponding to the speech processing method and device in the embodiment of the present disclosure, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned speech processing method.
  • the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.
  • the memory may further include a memory remotely arranged relative to the processor, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
  • the processor can call the information and application stored in the memory through the transmission device to execute the following steps: obtain the voice information to be processed, wherein the voice information to be processed includes: multiple voice segments; perform feature conversion on the multiple voice segments to obtain conversion results; perform feature decoding on the conversion results to obtain decoding results; determine the target voice processing result based on the decoding result.
  • the processor may also execute the following program code: at multiple consecutive moments, use a large speech processing model to perform speech encoding or speech segmentation on multiple speech segments in sequence to obtain multiple speech feature vectors, wherein multiple moments correspond one-to-one to multiple speech segments.
  • the processor may also execute the program code of the following steps: at a plurality of consecutive moments, a large speech processing model is used to sequentially perform feature decoding on a plurality of speech feature vectors to obtain a plurality of decoded segments.
  • the processor may also execute the program code of the following steps: at multiple consecutive moments, a large speech processing model is used to sequentially perform feature decoding on multiple speech feature vectors and context information to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segments and historical moment decoded segments associated with the current moment speech feature vector among the multiple speech feature vectors.
  • the processor may further execute the following program code: at a plurality of consecutive moments, the speech feature vector at the current moment, the speech segment at the previous moment adjacent to the speech feature vector at the current moment, and the decoded segment at the previous moment adjacent to the speech feature vector at the current moment are sequentially decoded using the speech processing large model to obtain the decoded segment at the current moment, until all the speech feature vectors are decoded to obtain a plurality of speech feature vectors. Decode the fragment.
  • the speech processing large model includes: a feature conversion model and a feature decoding model.
  • the feature decoding model is a multimodal language model.
  • the processor may also execute program code of the following steps: performing feature decoding on the conversion result using a multimodal language model to obtain a decoding result; or performing non-feature decoding on the conversion result using a multimodal language model to obtain a decoding result.
  • the processor may also execute program codes of the following steps: in response to a first touch operation on a graphical user interface, selecting a target speech processing type from a plurality of candidate speech processing types, wherein the plurality of candidate speech processing types include speech recognition type, speech translation type, and speech interaction type; in response to a second touch operation on the graphical user interface, determining a plurality of speech segments corresponding to the target speech processing type; in response to a third touch operation on the graphical user interface, converting the plurality of speech segments into a plurality of speech feature vectors, performing feature decoding on the plurality of speech feature vectors to obtain a plurality of decoded segments, and splicing the plurality of decoded segments to obtain a target speech processing result; and displaying the target speech processing result in the graphical user interface.
  • the processor may also execute the program code of the following steps: obtaining speech information to be recognized, wherein the speech information to be recognized includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain multiple speech feature vectors; performing feature decoding on the multiple speech feature vectors to obtain multiple text segments; and determining the target speech recognition result based on the multiple text segments.
  • the processor may also execute the program code of the following steps: obtaining speech information to be translated, wherein the speech information to be translated includes: a plurality of first speech segments; performing feature conversion on the plurality of first speech segments to obtain a plurality of speech feature vectors; performing feature decoding on the plurality of speech feature vectors to obtain a plurality of second speech segments; and determining a target speech translation result based on the plurality of second speech segments.
  • the processor may also execute the program code of the following steps: obtaining speech information to be processed, wherein the speech information to be processed includes: multiple speech segments; performing feature conversion on the multiple speech segments using a speech processing large model to obtain conversion results, and performing feature decoding on the conversion results to obtain decoding results; and determining the target speech processing result based on the decoding result.
  • the processor may also execute the following steps of program code: at multiple consecutive moments, use a large speech processing model to perform speech encoding or speech segmentation on multiple speech segments in turn to obtain multiple speech feature vectors, wherein multiple moments correspond to multiple speech segments one by one; at multiple consecutive moments, use a large speech processing model to perform feature decoding on multiple speech feature vectors and context information in turn to obtain multiple decoded segments, wherein the context information is related to the speech feature vector at the current moment in the multiple speech feature vectors.
  • the associated historical moment voice fragment and the historical moment decoding fragment are determined.
  • multiple voice segments in the voice information to be processed are obtained, and then the multiple voice segments are feature converted to obtain conversion results, and then the conversion results are feature decoded to obtain decoding results, and finally the target voice processing results are determined based on the decoding results, thereby achieving the purpose of efficiently processing the voice information to obtain the voice processing results corresponding to the multiple voice segments, thereby achieving the technical effect of improving the processing efficiency and recognition accuracy when processing the voice data stream, and thus solving the technical problems of low processing efficiency and poor recognition accuracy when processing the voice data stream in the related art.
  • the structure shown in FIG. 12 is for illustration only, and the computer terminal may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, and a mobile Internet device (MID).
  • FIG. 12 does not limit the structure of the above-mentioned electronic device.
  • the computer terminal may also include more or fewer components (such as a network interface, a display device, etc.) than those shown in FIG. 12, or have a configuration different from that shown in FIG. 12.
  • a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
  • the embodiment of the present disclosure also provides a computer-readable storage medium.
  • the computer-readable storage medium can store the program code executed by the speech processing method provided in the above embodiment.
  • the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.
  • the storage medium is configured to store program codes for executing the following steps: obtaining voice information to be processed, wherein the voice information to be processed includes: multiple voice segments; performing feature conversion on the multiple voice segments to obtain conversion results; performing feature decoding on the conversion results to obtain decoding results; and determining a target voice processing result based on the decoding result.
  • the storage medium is configured to store program code for executing the following steps: at multiple consecutive moments, a large speech processing model is used to perform speech encoding or speech segmentation on multiple speech segments in turn to obtain multiple speech feature vectors, wherein the multiple moments correspond one-to-one to the multiple speech segments.
  • the storage medium is configured to store program codes for executing the following steps: Code: At multiple consecutive moments, a large speech processing model is used to sequentially decode multiple speech feature vectors to obtain multiple decoded segments.
  • the storage medium is configured to store program code for executing the following steps: at multiple consecutive moments, a large speech processing model is used to sequentially perform feature decoding on multiple speech feature vectors and context information to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segment and the historical moment decoded segment associated with the current moment speech feature vector in the multiple speech feature vectors.
  • the storage medium is configured to store program code for executing the following steps: at multiple consecutive moments, a large speech processing model is used to sequentially perform feature decoding on the speech feature vector at the current moment, the speech segment at the previous moment adjacent to the speech feature vector at the current moment, and the decoded segment at the previous moment adjacent to the speech feature vector at the current moment to obtain a decoded segment at the current moment, until all multiple speech feature vectors are decoded to obtain multiple decoded segments.
  • the speech processing large model includes: a feature conversion model and a feature decoding model.
  • the feature decoding model is a multimodal language model.
  • the storage medium is configured to store program code for executing the following steps: using a multimodal language model to perform feature decoding on the conversion result to obtain a decoding result; or, using a multimodal language model to perform non-feature decoding on the conversion result to obtain a decoding result.
  • the storage medium is configured to store program codes for executing the following steps: in response to a first touch operation applied to a graphical user interface, selecting a target speech processing type from a plurality of candidate speech processing types, wherein the plurality of candidate speech processing types include: speech recognition type, speech translation type, and speech interaction type; in response to a second touch operation applied to the graphical user interface, determining a plurality of speech segments corresponding to the target speech processing type; in response to a third touch operation applied to the graphical user interface, converting the plurality of speech segments into a plurality of speech feature vectors, performing feature decoding on the plurality of speech feature vectors to obtain a plurality of decoded segments, and splicing the plurality of decoded segments to obtain a target speech processing result; and displaying the target speech processing result in the graphical user interface.
  • the storage medium is configured to store program code for executing the following steps: obtaining speech information to be recognized, wherein the speech information to be recognized includes: multiple speech segments; performing feature conversion on the multiple speech segments to obtain multiple speech feature vectors; performing feature decoding on the multiple speech feature vectors to obtain multiple text segments; and determining a target speech recognition result based on the multiple text segments.
  • the storage medium is configured to store program codes for executing the following steps: obtaining speech information to be translated, wherein the speech information to be translated includes: a plurality of first speech segments; performing feature conversion on the plurality of first speech segments to obtain a plurality of speech feature vectors; performing feature conversion on the plurality of speech feature vectors; The target speech translation result is determined based on the plurality of second speech segments.
  • the storage medium is configured to store program codes for executing the following steps: obtaining speech information to be processed, wherein the speech information to be processed includes: multiple speech segments; performing feature conversion on the multiple speech segments using a speech processing large model to obtain conversion results, and performing feature decoding on the conversion results to obtain decoding results; and determining a target speech processing result based on the decoding result.
  • the storage medium is configured to store program codes for executing the following steps: at multiple consecutive moments, using a large speech processing model to perform speech encoding or speech segmentation on multiple speech segments in turn to obtain multiple speech feature vectors, wherein the multiple moments correspond one-to-one to the multiple speech segments; at multiple consecutive moments, using a large speech processing model to perform feature decoding on multiple speech feature vectors and context information in turn to obtain multiple decoded segments, wherein the context information is determined by the historical moment speech segment and the historical moment decoded segment associated with the current moment speech feature vector in the multiple speech feature vectors.
  • multiple voice segments in the voice information to be processed are obtained, and then the multiple voice segments are feature converted to obtain conversion results, and then the conversion results are feature decoded to obtain decoding results, and finally the target voice processing results are determined based on the decoding results, thereby achieving the purpose of efficiently processing the voice information to obtain the voice processing results corresponding to the multiple voice segments, thereby achieving the technical effect of improving the processing efficiency and recognition accuracy when processing the voice data stream, and thus solving the technical problems of low processing efficiency and poor recognition accuracy when processing the voice data stream in the related art.
  • the embodiment of the present disclosure also provides a speech processing system, including:
  • a processor and a memory, connected to the processor, configured to provide the processor with instructions for performing the following processing steps:
  • Acquire speech information to be processed wherein the speech information to be processed includes: a plurality of speech segments; perform feature conversion on the plurality of speech segments to obtain conversion results; perform feature decoding on the conversion results to obtain decoding results; and determine a target speech processing result based on the decoding results.
  • multiple voice segments in the voice information to be processed are obtained, and then the multiple voice segments are feature converted to obtain conversion results, and then the conversion results are feature decoded to obtain decoding results, and finally the target voice processing results are determined based on the decoding results, so that the purpose of efficiently processing the voice information to obtain the voice processing results corresponding to the multiple voice segments is achieved, thereby achieving the technical effect of improving the processing efficiency and recognition accuracy when processing the voice data stream, and then
  • the invention solves the technical problems of low processing efficiency and poor recognition accuracy when processing voice data stream in the related technology.
  • the disclosed technical content can be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the units is only a logical function division.
  • multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
  • the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium.
  • the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure.
  • the aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Machine Translation (AREA)

Abstract

一种语音处理方法、存储介质及系统，该方法包括：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段（S21）；对多个语音片段进行特征转换，得到转换结果（S22）；对转换结果进行特征解码，得到解码结果（S23）；基于解码结果确定目标语音处理结果（S24）。该方法解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。

Description

语音处理方法、存储介质及系统 技术领域
本公开涉及计算机技术领域，具体而言，涉及一种语音处理方法、存储介质及系统。
背景技术
语音处理是使用计算机对人类语言进行分析、识别、合成和理解的技术，随着智能化时代的到来，语音处理的应用领域越来越广泛，其对于提供更加智能化的人机交互体验具有重要意义。相关技术中采用不同的语音处理模型对语音数据流进行处理，而各种不同的语音处理模型由于构造原理不同，在进行语音处理时，均存在不同程度的缺陷，从而导致语音数据流的处理效率低下、识别准确性差。
针对上述的问题，目前尚未提出有效的解决方案。
发明内容
本公开实施例提供了一种语音处理方法、存储介质及系统，以至少解决相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
根据本公开实施例的一个方面，提供了一种语音处理方法，包括：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。
根据本公开实施例的又一方面，还提供了一种语音处理方法，包括：获取待识别的语音信息，其中，待识别的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向量进行特征解码，得到多个文本片段；基于多个文本片段确定目标语音识别结果。
根据本公开实施例的又一方面，还提供了一种语音处理方法，包括：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果；基于解码结果确定目标语音处理结果。
根据本公开实施例的又一方面，还提供了一种计算机可读存储介质，计算机可读存储介质包括存储的可执行程序，其中，在可执行程序运行时控制计算机可读存储介质所在设备执行本公开实施例中任意一项所述的语音处理方法。
根据本公开实施例的又一方面，还提供了一种语音处理系统，包括：处理器；以及存储器，与处理器连接，被设置为处理器提供处理以下处理步骤的指令：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。
在本公开实施例中，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
容易注意到的是，上面的通用描述和后面的详细描述仅仅是为了对本公开进行举例和解释，并不构成对本公开的限定。
附图说明
此处所说明的附图用来提供对本公开的进一步理解，构成本公开的一部分，本公开的示意性实施例及其说明用于解释本公开，并不构成对本公开的不当限定。在附图中：
图1示出了一种用于实现语音处理方法的计算机终端(或移动设备)的硬件结构框图；
图2是根据本公开实施例1的一种语音处理方法的流程图；
图3是根据本公开实施例1的一种语音处理方法的示意图；
图4是根据本公开实施例1的又一种语音处理方法的示意图；
图5是根据本公开实施例2的一种语音处理方法的流程图；
图6是根据本公开实施例3的一种语音处理方法的流程图；
图7是根据本公开实施例4的一种语音处理方法的流程图；
图8是根据本公开实施例1的一种语音处理装置的结构框图；
图9是根据本公开实施例2的一种语音处理装置的结构框图；
图10是根据本公开实施例3的一种语音处理装置的结构框图；
图11是根据本公开实施例4的一种语音处理装置的结构框图；
图12是根据本公开实施例6的一种计算机终端的结构框图。
具体实施方式
为了使本技术领域的人员更好地理解本公开方案，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本公开一部分的实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都应当属于本公开保护的范围。
需要说明的是，本公开的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的本公开的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
本公开提供的技术方案主要采用大模型技术实现，此处的大模型是指具有大规模模型参数的深度学习模型，通常可以包含上亿、上百亿、上千亿、上万亿甚至十万亿以上的模型参数。大模型又可以称为基石模型/基础模型(Foundation Model)，通过大规模无标注的语料进行大模型的预训练，产出亿级以上参数的预训练模型，这种模型能适应广泛的下游任务，模型具有较好的泛化能力，例如大规模语言模型(Large Language Model,LLM)、多模态预训练模型(multi-modal pre-training model)等。
需要说明的是，大模型在实际应用时，可以通过少量样本对预训练模型进行微调，使得大模型可以应用于不同的任务中。例如，大模型可以广泛应用于自然语言处理(Natural Language Processing，NLP)、计算机视觉等领域，具体可以应用于如视觉问答(Visual Question Answering，VQA)、图像描述(Image Caption，IC)、图像生成等计算机视觉领域任务，也可以广泛应用于基于文本的情感分类、文本摘要生成、机器翻译等自然语言处理领域任务。因此，大模型主要的应用场景包括但不限于数字助理、智能机器人、搜索、在线教育、办公软件、电子商务、智能设计等。在本公开实施例中，以语音处理场景下通过语音处理大模型进行数据处理为例进行解释说明。
首先，在对本公开实施例进行描述的过程中出现的部分名词或术语适用于如下解释：
连续时序分类(Connectionist Temporal Classification，CTC)模型：是一种用于序列分类的深度学习模型，采用端到端的训练方式，可以处理变长输入序列和输出序列。CTC模型通过将输入序列映射到输出序列上来进行分类任务，在推断阶段，CTC模型使用前向算法计算每个时间步上可能出现的所有标记组合及其概率，并从中选择输出结果。
基于注意力的编码器-解码器(Attention-based Encoder-Decoder，AED)模型：是一种序列到序列模型，用于将一个不定长的输入序列映射到另一个不定长的输出序列，该模型主要由编码器和解码器组成，编码器将输入序列转换为固定长度的向量表示，并能够通过这个向量向解码器传递信息，解码器使用该向量作为初始状态生成输出序列。
循环神经网络-变换器(Recurrent Neural Network-Transducer，RNN-T)模型：是一种端到端的语音识别模型，其采用序列建模和转录技术来将语音信号转化为文本。
相关技术中采用不同的语音处理模型对语音数据流进行处理，而各种不同的语音处理模型由于构造原理不同，在进行语音处理时，均存在不同程度的缺陷，从而导致语音数据流的处理效率低下、识别准确性差。具体的，CTC流式模型框架存在独立帧假设，因此其建模上下文能力相对较弱；AED模型需要使用额外设计的分块注意力机制来实现流式识别，而这并适合处理语音数据流任务；RNN-T模型的长时间依赖性较差，随着输入音频长度的增加，其识别精度可能会下降。
由此可知，相关技术中存在对语音数据流进行处理时的处理效率低、识别准确性差的技术问题，针对上述的问题，目前尚未提出有效的解决方案。
实施例1
根据本公开实施例，还提供了一种语音处理的方法实施例，需要说明的是，在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行，并且，虽然在流程图中示出了逻辑顺序，但是在某些情况下，可以以不同于此处的顺序执行所示出或描述的步骤。
考虑到大模型的模型参数量庞大，且移动终端的运算资源有限，本公开实施例提供的上述语音处理方法可以应用于如图1所示的应用场景，但不仅限于此。在如图1所示的应用场景中，语音处理大模型部署在服务器10中，服务器10可 以通过局域网连接、广域网连接、因特网连接，或者其他类型的数据网络，连接一个或多个客户端设备20，此处的客户端设备20可以包括但不限于：智能手机、平板电脑、笔记本电脑、掌上电脑、个人计算机、智能家居设备、车载设备等。客户端设备20可以通过图形用户界面与用户进行交互，实现对大模型的调用，进而实现本公开实施例所提供的方法。
在本公开实施例中，客户端设备和服务器构成的系统可以执行如下步骤：服务器获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。在客户端设备中，响应作用于图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；响应作用于图形用户界面的第二触控操作，确定目标语音处理类型对应的多个语音片段；响应作用于图形用户界面的第三触控操作，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果；在图形用户界面内显示目标语音处理结果。需要说明的是，在客户端设备的运行资源能够满足语音处理大模型的部署和运行条件的情况下，本公开实施例可以在客户端设备中进行。
在上述运行环境下，本公开提供了如图2所示的语音处理方法。图2是根据本公开实施例1的一种语音处理方法的流程图，如图2所示，该方法包括以下步骤：
步骤S21，获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；
步骤S22，对多个语音片段进行特征转换，得到转换结果；
步骤S23，对转换结果进行特征解码，得到解码结果；
步骤S24，基于解码结果确定目标语音处理结果。
上述待处理的语音信息可以通过终端设备上的音频组件对用户语音进行采集而获得，如采用手机麦克风组件获得用户的多个语音片段；待处理的语音信息还可以从已经录制好的音频文件中进行提取而得到，如应用软件中的语音消息；待处理的语音信息还可以从互联网上的音频资源获取，如网络直播、网络音乐电台等；待处理的语音信息还可以为从智能设备中获取语音指令或会话信息。以上渠道所获取的待处理的语音信息都可以进行语音识别、语音合成、情感分析、说话 人识别等应用。
需要说明的是，上述待处理语音信息中的多个语音片段中可以包括连续的多人对话内容或者离散的个人语音内容，本公开实施例不予限制。
进一步的，对待处理的语音信息中的多个语音片段进行特征转换，得到多个语音特征向量。其中，特征转换指的是将待处理的语音数据通过一定方式转换成新的特征表示的过程，通过特征转换可以减少数据的维度、去除冗余信息、提取有用的特征，并提高算法的鲁棒性和泛化能力。
对多个语音特征向量进行特征解码，得到解码结果，基于解码结果能够确定目标语音处理结果。在对多个语音特征向量进行特征解码的过程中，可以采用自回归解码、分类解码、生成解码、序列到序列解码、路径解码和注意力解码等方式，这些解码方式可以根据具体的任务和应用场景选择使用。
其中，自回归解码是一种常见的解码方式，通常应用于自然语言处理、语音识别等序列生成任务中。自回归解码的基本思想是在一次生成的过程中，每一步依次生成序列中的每一个元素，每次生成的结果会影响下一次的生成。这可以用一个条件概率模型来表示，如在使用RNN生成文本序列时，一次生成一个字符，每个字符的出现概率取决于前面已经生成的字符以及生成的条件。分类解码则是将语音特征向量映射到一个固定的类别集合中，每个类别代表一个音素、音节或单词，这种解码方式常用于语音识别任务。生成解码则是使用生成模型，例如生成对抗网络、变分自编码器，将语音特征向量映射到声音波形空间中，从而生成与输入特征相对应的语音信号。序列到序列解码则是使用循环神经网络，例如长短时记忆网络、门控循环单元，将语音特征向量序列映射到另一个语音特征向量序列，实现语音转换、语音合成等任务；路径解码则是将语音特征向量映射到一条特定的路径上，该路径可以表示音素、音节、单词或语音的时长和边界。注意力解码则是使用注意力机制，将输入的语音特征向量序列与另一个语音特征向量序列进行对齐，从而生成对应的输出序列。
基于上述步骤S21至步骤S24，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
需要说明的是，本公开实施例可以但不限于应用于健康、科学、社会、语言和艺术等领域中任何涉及语音交互的场景中，尤其涉及智能家居、智能客服、智能会议等领域内的语音识别、语音翻译等应用场景下，本公开实施例中的语音处理方法可以更加准确、高效地实现语音转换成文本的功能，进一步为用户提供更加智能化的交互体验。
下面对本公开实施例中的语音处理方法进行进一步介绍。
在一种可选的实施例中，在步骤S22，对多个语音片段进行特征转换，得到转换结果包括：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应。
在一种可选的实施例中，语音处理大模型包括：特征转换模型与特征解码模型。特征转换模型的主要功能是将语音信号从时域或频域转换到特征域。常见的特征转换模型包括傅里叶变换、离散余弦变换、小波变换等，它们可以提取出语音信号的频谱特征、能量特征等。特征解码模型(是语音处理中的另一个重要模型，其主要功能是将特征域的语音信号解码为原始的时域或频域信号。特征解码模型可以通过逆变换的方式将特征信号恢复为原始的语音信号，从而实现对语音信号的还原。
特征转换模型和特征解码模型通常配合使用，前者用于将语音信号转换为特征表示，后者用于将特征表示解码为原始信号。这样的处理过程可以提取语音信号中的有用信息，并减少信号的冗余，从而方便后续的语音识别、语音合成、语音增强等任务的进行。
具体的，在连续多个时刻，采用语音处理大模型依次对多个语音片段采用语音编码器进行语音编码或者采用语音分词器进行语音分词，得到多个语音特征向量。
对多个语音片段采用语音编码器进行语音编码时，可以对多个语音片段先进行预处理，如截取固定长度的语音段、进行语音信号增强、去除噪声等，进而提取语音特征，即将语音片段转换为可以被语音编码器理解的特征表示，其中，上述语音特征可以包括梅尔频率倒谱系数(Mel-Frequency Cepstral Coefficients，MFCC)、语谱图、线性预测倒谱相关系数(Linear Predictive Cepstral Coding，LPCC)等。随后构建语音编码器模型，通过建立一个神经网络模型，接受语音特征作为输入并生成编码后的语音特征向量表示作为输出，其中，上述语音编码器模型包括自编码器(Autoencoder)、变分自编码器(Variational Auto-Encoder)等。将每 个时刻对应语音片段的语音特征输入到语音编码器模型中，能够得到对应的编码后的语音特征向量，这些语音特征向量可以用于对语音片段进行识别、分类、聚类等任务。
需要注意的是，不同的语音编码器模型在输入输出的参数设置和网络结构设计上可能存在差异，具体的实现方式需要根据具体的任务和数据进行调整和优化，本公开不予限制。
对多个语音片段采用语音分词器进行语音分词时，可以对多个语音片段先进行预处理，如截取固定长度的语音段、进行语音信号增强、去除噪声等，进而提取语音特征，即将语音片段转换为可以被语音编码器理解的特征表示，其中，上述语音特征可以包括MFCC、语谱图、LPCC等。随后构建语音分词模型，通过建立一个神经网络分词模型，接受语音特征作为输入并生成分词后的输出语音特征向量，常用的语音分词模型包括卷积神经网络、循环神经网络、卷积-循环神经网络等。将每个时刻对应语音片段的语音特征输入到语音分词模型中，得到对应的分词结果，语音分词结果的输出格式可以包括标注文本和时间戳序列，还可以对分词结果进行后处理和优化，如去除重复词汇、合并相邻词汇、进行音素级别的分词等，最后存储分词结果及相关信息，如原始语音数据、分词文本、时间戳序列等，将其作为语音片段对应的语音特征向量。
需要注意的是，对于不同的语音分词任务，要根据数据的特征和需求进行有针对性的处理和模型设计。同时，语音分词模型的建立、分词结果的评价和后处理等环节也需要进行精细化调整，才能得到更准确、更符合要求的语音分词结果，本公开不予限制。
基于上述可选实施例，通过在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，从而用于流式语音识别和流式语音翻译等任务，进一步提升语音处理效率。
在一种可选的实施例中，在步骤S23，对转换结果进行特征解码，得到解码结果包括：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段。
在一种可选的实施例中，特征解码模型为多模态语言模型。多模态语言模型是一种能够处理多种不同模态(如文本、图像、音频等)输入的语言模型。它能够同时理解和生成多种不同类型的数据，并能够将这些不同模态的数据进行有效的集成和交互。多模态语言模型的目标是通过结合不同模态的信息，提高语言模型在理解和生成任务中的性能和表现。
具体的，在多个连续时刻，采用基于解码器结构(Decoder)的多模态语言模型(Multi-modal Language Model)依次对多个语音特征向量进行特征解码，得到多个解码片段。在特征解码过程中，每一次解码输出会被作为下一次解码输入的一部分，因此可以利用解码输出结果，并重复进行解码来获得连续多个时刻的解码片段，直到达到特定终止条件或达到最大轮数。在得到多个解码片段后，还可以对它们进行后处理，如去重、排序、过滤无效信息等。
基于上述可选实施例，通过在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段，能够实现语音特征向量的快速解码，进而提升语音处理效率。
在一种可选的实施例中，在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段包括：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
上述上下文信息是指与历史时刻语音片段与历史时刻解码片段相关联的信息。在语音识别或语音合成任务中，上下文信息包括历史时刻的语音片段、解码片段以及其相关特征向量等信息。上下文信息可以提供更多的上下文语境，帮助准确理解当前时刻的语音信息以及生成更加流畅自然的语音片段。
基于上述可选实施例，通过在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，进一步提升语音识别的准确性。
在一种可选的实施例中，在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段包括：在连续多个时刻，采用语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个解码片段。
具体来说，在每个时刻，当前时刻的语音特征向量是依赖于前面历史时刻的语音片段的，这些历史时刻的语音片段也就是上下文信息。同样地，在语音识别或语音合成的解码过程中，当前时刻的解码片段也会受到历史时刻解码片段的影响，因此历史时刻的解码片段也构成了上下文信息的一部分。
以连续多个时刻中的第一时刻和第二时刻为例，与第二时刻相邻的前一时刻为第一时刻，依次对第二时刻语音特征向量、第一时刻语音片段以及第一时刻解码片段进行特征解码，以得到第二时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个解码片段。
基于上述可选实施例，通过在连续多个时刻，采用语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，快速得到多个解码片段，能够提高解码效率和解码准确性，进一步提升语音识别的准确性。
在一种可选的实施例中，对转换结果进行特征解码，得到解码结果包括：采用多模态语言模型对转换结果进行特征解码，得到解码结果；或者，采用多模态语言模型对转换结果进行非特征解码，得到解码结果。
具体的，特征解码是指利用多模态语言模型对转换结果进行解码时，使用了语音特征向量，用于辅助解码过程，通过特征解码，可以更准确地预测句子的生成概率，提高解码的准确性和效果。非特征解码则是指在解码过程中不使用语音特征向量，只依靠多模态语言模型对转换结果进行解码。
基于上述可选实施例，可以利用多模态语言模型对转换结果灵活解码，进一步提高语音识别效率和识别准确率。
在一种可选的实施例中，通过终端设备提供一图形用户界面，图形用户界面所显示的内容至少部分地包含一语音处理场景，语音处理方法还包括：
步骤S31，响应作用于图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；
步骤S32，响应作用于图形用户界面的第二触控操作，确定目标语音处理类型对应的多个语音片段；
步骤S33，响应作用于图形用户界面的第三触控操作，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果；
步骤S34，在图形用户界面内显示目标语音处理结果。
在上述可选的实施例中，上述图形用户界面为语音处理界面，在语音处理界 面中所显示的语音处理场景可以是语音识别场景、语音翻译场景或者语音交互场景。
上述图形用户界面还包括第一控件(或第一触控区域)，当检测到作用于第一控件(或第一触控区域)的第一触控操作时，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型。
上述图形用户界面还包括第二控件(或第二触控区域)，当检测到作用于第二控件(或第二触控区域)的第二触控操作时，确定目标语音处理类型对应的多个语音片段。
上述图形用户界面还包括第三控件(或第三触控区域)，当检测到作用于第三控件(或第三触控区域)的第三触控操作时，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果。
在图形用户界面内显示目标语音处理结果，以便用户直观查看语音处理结果和语音处理进度，进一步提升用户的智能化交互体验。
需要说明的是，上述第一触控操作、上述第二触控操作和上述第三触控操作均可以是用户用手指接触上述终端设备的显示屏并触控该终端设备的操作。该触控操作可以包括单点触控、多点触控，其中，每个触控点的触控操作可以包括点击、长按、重按、划动等。上述第一触控操作、上述第二触控操作和上述第三触控操作还可以是通过鼠标、键盘等输入设备实现的触控操作。
图3是根据本公开实施例1的一种语音处理方法的示意图，如图3所示，该方法包括以下步骤：
步骤S301，获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；
步骤S302，在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应；
步骤S303，在连续多个时刻，采用所述语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个解码片段；
步骤S304，基于解码结果确定目标语音处理结果。
基于上述步骤S301至步骤S304，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
图4是根据本公开实施例1的又一种语音处理方法的示意图，如图4所示，在第一时刻，语音输入片段1(Speech chunk1)通过语音编码器或语音分词器(也称为语音离散化)后，转换成多个语音特征向量，这些向量被送入多模态语言模型以进行特征解码，基于解码结果生成文本片段1(Text chunk1)。接下来，在第二时刻，语音输入片段2(Speech chunk2)也通过语音编码器或语音分词器转换成语音特征向量，并输入到多模态语言模型中进行特征解码，生成文本片段2(Text chunk2)，以此类推，实现了流式解码，由此本公开实施例的语音识别方法可用于流式语音识别、流式语音翻译以及流式语音交互等任务。
从实际功能价值的角度来看，本公开实施例的语音识别方法在语音识别、语音翻译等场景下可以实现更加准确和高效的语音转换成文本的功能，从而提高使用体验，为用户提供更好的服务。同时，本公开实施例的语音识别方法还可以应用于语音交互领域，例如智能家居、智能客服等领域，为用户提供更加智能化的交互体验。从解决问题的角度来看，本公开实施例的语音识别方法解决了传统流式模型框架中新输入的语音特征和已生成文本之间的矛盾，提高了准确率。同时，基于解码器的流式模型框架在模型结构上更加统一，便于优化和扩展。此外，本公开实施例的语音识别方法提出了一种新型的基于解码器的流式模型框架，与传统的CTC、AED、RNN-T等流式模型框架有所不同，具有更好的适用性和准确性。
需要说明的是，本公开所涉及的用户信息(包括但不限于用户设备信息、用户个人信息等)和数据(包括但不限于用于分析的数据、存储的数据、展示的数据等)，均为经用户授权或者经过各方充分授权的信息和数据，并且相关数据的收集、使用和处理需要遵守相关国家和地区的相关法律法规和标准，并提供有相应的操作入口，供用户选择授权或者拒绝。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本公开并不受所描述的动作 顺序的限制，因为依据本公开，某些步骤可以采用其他顺序或者同时进行。其次，本领域技术人员也应该知悉，说明书中所描述的实施例均属于优选实施例，所涉及的动作和模块并不一定是本公开所必须的。
通过以上的实施方式的描述，本领域的技术人员可以清楚地了解到根据上述实施例的方法可借助软件加必需的通用硬件平台的方式来实现，当然也可以通过硬件。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质(如ROM/RAM、磁碟、光盘)中，包括若干指令用以使得一台终端设备(可以是手机，计算机，服务器，或者网络设备等)执行本公开各个实施例所述的方法。
实施例2
根据本公开实施例，还提供了一种语音处理方法，图5是根据本公开实施例2的一种语音处理方法的流程图，如图5所示，该方法包括以下步骤：
步骤S51，获取待识别的语音信息，其中，所述待识别的语音信息包括：多个语音片段；
步骤S52，对所述多个语音片段进行特征转换，得到多个语音特征向量；
步骤S53，对所述多个语音特征向量进行特征解码，得到多个文本片段；
步骤S54，基于所述多个文本片段确定目标语音识别结果。
基于上述步骤S51至步骤S54，通过获取待识别的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音识别结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音识别结果的目的，从而实现了提升对于语音数据流进行处理时的识别效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的识别效率低、识别准确性差的技术问题。
需要说明的是，本公开实施例可以但不限于应用于健康、科学、社会、语言和艺术等领域中任何涉及语音交互的场景中，尤其涉及智能家居、智能客服、智能会议等领域内的语音识别、语音翻译等应用场景下，本公开实施例中的语音处理方法可以更加准确、高效地实现语音转换成文本的功能，进一步为用户提供更加智能化的交互体验。
在本公开的上述实施例中没有详述的部分，可以参见实施例1的相关描述， 不予赘述。
实施例3
根据本公开实施例，还提供了一种语音处理方法，图6是根据本公开实施例3的一种语音处理方法的流程图，如图6所示，该方法包括以下步骤：
步骤S61，获取待翻译的语音信息，其中，待翻译的语音信息包括：多个第一语音片段；
步骤S62，对多个第一语音片段进行特征转换，得到多个语音特征向量；
步骤S63，对多个语音特征向量进行特征解码，得到多个第二语音片段；
步骤S64，基于多个第二语音片段确定目标语音翻译结果。
基于上述步骤S61至步骤S64，通过获取待翻译的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音翻译结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音翻译结果的目的，从而实现了提升对于语音数据流进行处理时的翻译效率和翻译准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的翻译效率低、翻译准确性差的技术问题。
需要说明的是，本公开实施例可以但不限于应用于健康、科学、社会、语言和艺术等领域中任何涉及语音交互的场景中，尤其涉及智能家居、智能客服、智能会议等领域内的语音识别、语音翻译等应用场景下，本公开实施例中的语音处理方法可以更加准确、高效地实现语音转换成文本的功能，进一步为用户提供更加智能化的交互体验。
在本公开的上述实施例中没有详述的部分，可以参见实施例1的相关描述，不予赘述。
实施例4
根据本公开实施例，还提供了一种语音处理方法，图7是根据本公开实施例4的一种语音处理方法的流程图，如图7所示，该方法包括以下步骤：
步骤S71，获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；
步骤S72，采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果；
步骤S73，基于解码结果确定目标语音处理结果。
基于上述步骤S71至步骤S73，通过获取待处理的语音信息，进而采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果，最后基于解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音识别结果的目的，从而实现了提升对于语音数据流进行处理时的识别效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的识别效率低、识别准确性差的技术问题。
需要说明的是，本公开实施例可以但不限于应用于健康、科学、社会、语言和艺术等领域中任何涉及语音交互的场景中，尤其涉及智能家居、智能客服、智能会议等领域内的语音识别、语音翻译等应用场景下，本公开实施例中的语音处理方法可以更加准确、高效地实现语音转换成文本的功能，进一步为用户提供更加智能化的交互体验。
在一种可选的实施例中，采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果包括：
在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应；
在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
在本公开的上述实施例中没有详述的部分，可以参见实施例1的相关描述，不予赘述。
实施例5
根据本公开实施例，还提供了一种实施上述语音处理方法的语音处理装置，图8是根据本公开实施例1的一种语音处理装置的结构框图，如图8所示，该语音处理装置包括：
获取模块801，被设置为获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；
转换模块802，被设置为对多个语音片段进行特征转换，得到转换结果；
解码模块803，被设置为对转换结果进行特征解码，得到解码结果；
确定模块804，被设置为基于解码结果确定目标语音处理结果。
可选地，转换模块802还被设置为在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应。
可选地，解码模块803还被设置为在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段。
可选地，解码模块803还被设置为在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
可选地，解码模块803还被设置为在连续多个时刻，采用语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个解码片段。
可选地，语音处理大模型包括：特征转换模型与特征解码模型。
可选地，特征解码模型为多模态语言模型。
可选地，解码模块803还被设置为：采用多模态语言模型对转换结果进行特征解码，得到解码结果；或者，采用多模态语言模型对转换结果进行非特征解码，得到解码结果。
可选地，语音处理装置还包括：响应模块805，被设置为响应作用于图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；响应模块805还被设置为响应作用于图形用户界面的第二触控操作，确定目标语音处理类型对应的多个语音片段；响应模块805还被设置为响应作用于图形用户界面的第三触控操作，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果；显示模块806，被设置为在图形用户界面内显示目标语音处理结果。
在本公开实施例中，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对 语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
此处需要说明的是，上述获取模块801、转换模块802、解码模块803和确定模块804对应于实施例1中的步骤S21至步骤S24，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例1所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件。
图9是根据本公开实施例2的一种语音处理装置的结构框图，如图9所示，该语音处理装置包括：
获取模块901，被设置为获取待识别的语音信息，其中，待识别的语音信息包括：多个语音片段；
转换模块902，被设置为对多个语音片段进行特征转换，得到多个语音特征向量；
解码模块903，被设置为对多个语音特征向量进行特征解码，得到多个文本片段；
确定模块904，被设置为基于多个文本片段确定目标语音识别结果。
在本公开实施例中，通过获取待识别的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音识别结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音识别结果的目的，从而实现了提升对于语音数据流进行处理时的识别效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的识别效率低、识别准确性差的技术问题。
此处需要说明的是，上述获取模块901、转换模块902、解码模块903和确定模块904对应于实施例2中的步骤S51至步骤S54，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例2所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件。
图10是根据本公开实施例3的一种语音处理装置的结构框图，如图10所示， 该语音处理装置包括：
获取模块1001，被设置为获取待翻译的语音信息，其中，待翻译的语音信息包括：多个第一语音片段；
转换模块1002，被设置为对多个第一语音片段进行特征转换，得到多个语音特征向量；
解码模块1003，被设置为对多个语音特征向量进行特征解码，得到多个第二语音片段；
确定模块1004，被设置为基于多个第二语音片段确定目标语音翻译结果。
在本公开实施例中，通过获取待翻译的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音翻译结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音翻译结果的目的，从而实现了提升对于语音数据流进行处理时的翻译效率和翻译准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的翻译效率低、翻译准确性差的技术问题。
此处需要说明的是，上述获取模块1001、转换模块1002、解码模块1003和确定模块1004对应于实施例3中的步骤S61至步骤S64，四个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例3所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件。
图11是根据本公开实施例4的一种语音处理装置的结构框图，如图11所示，该语音处理装置包括：
获取模块1101，获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；
处理模块1102，采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果；
确定模块1103，基于解码结果确定目标语音处理结果。
可选地，处理模块1102还被设置为：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应；在连续多个时刻，采用语音处理大模型依次 对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
在本公开实施例中，通过获取待处理的语音信息，进而采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果，最后基于解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音识别结果的目的，从而实现了提升对于语音数据流进行处理时的识别效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的识别效率低、识别准确性差的技术问题。
此处需要说明的是，上述获取模块1101、处理模块1102和确定模块1103对应于实施例4中的步骤S71至步骤S73，三个模块与对应的步骤所实现的实例和应用场景相同，但不限于上述实施例4所公开的内容。需要说明的是，上述模块或单元可以是存储在存储器中并由一个或多个处理器处理的硬件组件或软件组件。
实施例6
本公开的实施例可以提供一种计算机终端，该计算机终端可以是计算机终端群中的任意一个计算机终端设备。可选地，在本实施例中，上述计算机终端也可以替换为移动终端等终端设备。
可选地，在本实施例中，上述计算机终端可以位于计算机网络的多个网络设备中的至少一个网络设备。
在本实施例中，上述计算机终端可以执行语音处理方法中以下步骤的程序代码：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。
在本实施例中，上述计算机终端可以执行语音处理方法中以下步骤的程序代码：获取待识别的语音信息，其中，待识别的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向量进行特征解码，得到多个文本片段；基于多个文本片段确定目标语音识别结果。
在本实施例中，上述计算机终端可以执行语音处理方法中以下步骤的程序代码：获取待翻译的语音信息，其中，待翻译的语音信息包括：多个第一语音片段；对多个第一语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向 量进行特征解码，得到多个第二语音片段；基于多个第二语音片段确定目标语音翻译结果。
可选地，图12是根据本公开实施例6的一种计算机终端的结构框图。如图12所示，该计算机终端可以包括：一个或多个(图中仅示出一个)处理器122、存储器124、存储控制器、以及外设接口，其中，外设接口与射频模块、音频模块和显示器连接。
其中，存储器可被设置为存储软件程序以及模块，如本公开实施例中的语音处理方法和装置对应的程序指令/模块，处理器通过运行存储在存储器内的软件程序以及模块，从而执行各种功能应用以及数据处理，即实现上述的语音处理方法。存储器可包括高速随机存储器，还可以包括非易失性存储器，如一个或者多个磁性存储装置、闪存、或者其他非易失性固态存储器。在一些实例中，存储器可进一步包括相对于处理器远程设置的存储器，这些远程存储器可以通过网络连接至计算机终端。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
处理器可以通过传输装置调用存储器存储的信息及应用程序，以执行下述步骤：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。
可选地，上述处理器还可以执行如下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应。
可选地，上述处理器还可以执行如下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段。
可选地，上述处理器还可以执行如下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
可选地，上述处理器还可以执行如下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个 解码片段。
可选地，语音处理大模型包括：特征转换模型与特征解码模型。
可选地，特征解码模型为多模态语言模型。
可选地，上述处理器还可以执行如下步骤的程序代码：采用多模态语言模型对转换结果进行特征解码，得到解码结果；或者，采用多模态语言模型对转换结果进行非特征解码，得到解码结果。
可选地，上述处理器还可以执行如下步骤的程序代码：响应作用于图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；响应作用于图形用户界面的第二触控操作，确定目标语音处理类型对应的多个语音片段；响应作用于图形用户界面的第三触控操作，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果；在图形用户界面内显示目标语音处理结果。
可选地，上述处理器还可以执行如下步骤的程序代码：获取待识别的语音信息，其中，待识别的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向量进行特征解码，得到多个文本片段；基于多个文本片段确定目标语音识别结果。
可选地，上述处理器还可以执行如下步骤的程序代码：获取待翻译的语音信息，其中，待翻译的语音信息包括：多个第一语音片段；对多个第一语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向量进行特征解码，得到多个第二语音片段；基于多个第二语音片段确定目标语音翻译结果。
可选地，上述处理器还可以执行如下步骤的程序代码：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果；基于解码结果确定目标语音处理结果。
可选地，上述处理器还可以执行如下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应；在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关 联的历史时刻语音片段与历史时刻解码片段确定。
采用本公开实施例，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
本领域普通技术人员可以理解，图12所示的结构仅为示意，计算机终端也可以是智能手机(如Android手机、iOS手机等)、平板电脑、掌上电脑以及移动互联网设备(Mobile Internet Devices，MID)等终端设备。图12其并不对上述电子装置的结构造成限定。例如，计算机终端还可包括比图12中所示更多或者更少的组件(如网络接口、显示装置等)，或者具有与图12所示不同的配置。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令终端设备相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：闪存盘、只读存储器(Read-Only Memory，ROM)、随机存取器(Random Access Memory，RAM)、磁盘或光盘等。
实施例7
本公开的实施例还提供了一种计算机可读存储介质。可选地，在本实施例中，上述计算机可读存储介质可以保存上述实施例所提供的语音处理方法所执行的程序代码。
可选地，在本实施例中，上述存储介质可以位于计算机网络中计算机终端群中的任意一个计算机终端中，或者位于移动终端群中的任意一个移动终端中。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到转换结果；对转换结果进行特征解码，得到解码结果；基于解码结果确定目标语音处理结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代 码：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量进行特征解码，得到多个解码片段。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对当前时刻语音特征向量、当前时刻语音特征向量相邻的前一时刻语音片段以及当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至多个语音特征向量全部解码完毕，得到多个解码片段。
可选地，语音处理大模型包括：特征转换模型与特征解码模型。
可选地，特征解码模型为多模态语言模型。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：采用多模态语言模型对转换结果进行特征解码，得到解码结果；或者，采用多模态语言模型对转换结果进行非特征解码，得到解码结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：响应作用于图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；响应作用于图形用户界面的第二触控操作，确定目标语音处理类型对应的多个语音片段；响应作用于图形用户界面的第三触控操作，将多个语音片段转换为多个语音特征向量，对多个语音特征向量进行特征解码以得到多个解码片段，以及对多个解码片段进行拼接以得到目标语音处理结果；在图形用户界面内显示目标语音处理结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取待识别的语音信息，其中，待识别的语音信息包括：多个语音片段；对多个语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向量进行特征解码，得到多个文本片段；基于多个文本片段确定目标语音识别结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取待翻译的语音信息，其中，待翻译的语音信息包括：多个第一语音片段；对多个第一语音片段进行特征转换，得到多个语音特征向量；对多个语音特征向 量进行特征解码，得到多个第二语音片段；基于多个第二语音片段确定目标语音翻译结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：获取待处理的语音信息，其中，待处理的语音信息包括：多个语音片段；采用语音处理大模型对多个语音片段进行特征转换以得到转换结果，以及对转换结果进行特征解码以得到解码结果；基于解码结果确定目标语音处理结果。
可选地，在本实施例中，存储介质被设置为存储用于执行以下步骤的程序代码：在连续多个时刻，采用语音处理大模型依次对多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，多个时刻与多个语音片段一一对应；在连续多个时刻，采用语音处理大模型依次对多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，上下文信息由多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
采用本公开实施例，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
实施例8
本公开的实施例还提供了一种语音处理系统，包括：
处理器；以及存储器，与所述处理器连接，被设置为所述处理器提供处理以下处理步骤的指令：
获取待处理的语音信息，其中，所述待处理的语音信息包括：多个语音片段；对所述多个语音片段进行特征转换，得到转换结果；对所述转换结果进行特征解码，得到解码结果；基于所述解码结果确定目标语音处理结果。
采用本公开实施例，通过获取待处理的语音信息中的多个语音片段，进而对所述多个语音片段进行特征转换，得到转换结果，随后对所述转换结果进行特征解码，得到解码结果，最后基于所述解码结果确定目标语音处理结果，达到了对语音信息进行高效处理以获得多个语音片段对应的语音处理结果的目的，从而实现了提升对于语音数据流进行处理时的处理效率和识别准确性的技术效果，进而 解决了相关技术中对语音数据流进行处理时的处理效率低、识别准确性差的技术问题。
上述本公开实施例序号仅仅为了描述，不代表实施例的优劣。
在本公开的上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见其他实施例的相关描述。
在本公开所提供的几个实施例中，应该理解到，所揭露的技术内容，可通过其它的方式实现。其中，以上所描述的装置实施例仅仅是示意性的，例如所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，单元或模块的间接耦合或通信连接，可以是电性或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本公开各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本公开的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本公开各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述仅是本公开的优选实施方式，应当指出，对于本技术领域的普通技术人员来说，在不脱离本公开原理的前提下，还可以做出若干改进和润饰，这些改进和润饰也应视为本公开的保护范围。

Claims (14)

  1. 一种语音处理方法，包括：
    获取待处理的语音信息，其中，所述待处理的语音信息包括：多个语音片段；
    对所述多个语音片段进行特征转换，得到转换结果；
    对所述转换结果进行特征解码，得到解码结果；
    基于所述解码结果确定目标语音处理结果。
  2. 根据权利要求1所述的语音处理方法，其中，对所述多个语音片段进行特征转换，得到所述转换结果包括：
    在连续多个时刻，采用语音处理大模型依次对所述多个语音片段进行语音编码或者语音分词，得到多个语音特征向量，其中，所述多个时刻与所述多个语音片段一一对应。
  3. 根据权利要求2所述的语音处理方法，其中，对所述转换结果进行特征解码，得到所述解码结果包括：
    在所述连续多个时刻，采用所述语音处理大模型依次对所述多个语音特征向量进行特征解码，得到多个解码片段。
  4. 根据权利要求3所述的语音处理方法，其中，在所述连续多个时刻，采用所述语音处理大模型依次对所述多个语音特征向量进行特征解码，得到所述多个解码片段包括：
    在所述连续多个时刻，采用所述语音处理大模型依次对所述多个语音特征向量和上下文信息进行特征解码，得到所述多个解码片段，其中，所述上下文信息由所述多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
  5. 根据权利要求4所述的语音处理方法，其中，在所述连续多个时刻，采用所述语音处理大模型依次对所述多个语音特征向量和所述上下文信息进行特征解码，得到所述多个解码片段包括：
    在所述连续多个时刻，采用所述语音处理大模型依次对所述当前时刻语音特征向量、所述当前时刻语音特征向量相邻的前一时刻语音片段以及所述当前时刻语音特征向量相邻的前一时刻解码片段进行特征解码以得到当前时刻解码片段，直至所述多个语音特征向量全部解码完毕，得到所述多个解码片段。
  6. 根据权利要求2所述的语音处理方法，其中，所述语音处理大模型包括：特征转换模型与特征解码模型。
  7. 根据权利要求6所述的语音处理方法，其中，所述特征解码模型为多模态语言模型。
  8. 根据权利要求7所述的语音处理方法，其中，对所述转换结果进行特征解码，得到所述解码结果包括：
    采用所述多模态语言模型对所述转换结果进行特征解码，得到所述解码结果；或者，
    采用所述多模态语言模型对所述转换结果进行非特征解码，得到所述解码结果。
  9. 根据权利要求1所述的语音处理方法，其中，通过终端设备提供一图形用户界面，所述图形用户界面所显示的内容至少部分地包含一语音处理场景，所述语音处理方法还包括：
    响应作用于所述图形用户界面的第一触控操作，从多种候选语音处理类型中选取目标语音处理类型，其中，所述多种候选语音处理类型包括：语音识别类型、语音翻译类型、语音交互类型；
    响应作用于所述图形用户界面的第二触控操作，确定所述目标语音处理类型对应的所述多个语音片段；
    响应作用于所述图形用户界面的第三触控操作，将所述多个语音片段转换为所述多个语音特征向量，对所述多个语音特征向量进行特征解码以得到多个解码片段，以及对所述多个解码片段进行拼接以得到所述目标语音处理结果；
    在所述图形用户界面内显示所述目标语音处理结果。
  10. 一种语音处理方法，包括：
    获取待识别的语音信息，其中，所述待识别的语音信息包括：多个语音片段；
    对所述多个语音片段进行特征转换，得到多个语音特征向量；
    对所述多个语音特征向量进行特征解码，得到多个文本片段；
    基于所述多个文本片段确定目标语音识别结果。
  11. 一种语音处理方法，包括：
    获取待处理的语音信息，其中，所述待处理的语音信息包括：多个语音片段；
    采用语音处理大模型对所述多个语音片段进行特征转换以得到转换结果，以及对所述转换结果进行特征解码以得到解码结果；
    基于所述解码结果确定目标语音处理结果。
  12. 根据权利要求11所述的语音处理方法，其中，采用所述语音处理大模型对所述多个语音片段进行特征转换以得到所述转换结果，以及对所述转换结果进行特征解码以得到所述解码结果包括：
    在连续多个时刻，采用语音处理大模型依次对所述多个语音片段进行语音编 码或者语音分词，得到多个语音特征向量，其中，所述多个时刻与所述多个语音片段一一对应；
    在所述连续多个时刻，采用所述语音处理大模型依次对所述多个语音特征向量和上下文信息进行特征解码，得到多个解码片段，其中，所述上下文信息由所述多个语音特征向量中当前时刻语音特征向量相关联的历史时刻语音片段与历史时刻解码片段确定。
  13. 一种计算机可读存储介质，所述计算机可读存储介质包括存储的可执行程序，其中，在所述可执行程序运行时控制所述计算机可读存储介质所在设备执行权利要求1至12中任意一项所述的语音处理方法。
  14. 一种语音处理系统，包括：
    处理器；以及
    存储器，与所述处理器连接，被设置为所述处理器提供处理以下处理步骤的指令：
    获取待处理的语音信息，其中，所述待处理的语音信息包括：多个语音片段；
    对所述多个语音片段进行特征转换，得到转换结果；
    对所述转换结果进行特征解码，得到解码结果；
    基于所述解码结果确定目标语音处理结果。
PCT/CN2024/105264 2023-07-14 2024-07-12 语音处理方法、存储介质及系统 Pending WO2025016325A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310868220.5A CN116798409A (zh) 2023-07-14 2023-07-14 语音处理方法、存储介质及系统
CN202310868220.5 2023-07-14

Publications (1)

Publication Number Publication Date
WO2025016325A1 true WO2025016325A1 (zh) 2025-01-23

Family

ID=88038534

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/105264 Pending WO2025016325A1 (zh) 2023-07-14 2024-07-12 语音处理方法、存储介质及系统

Country Status (2)

Country Link
CN (1) CN116798409A (zh)
WO (1) WO2025016325A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116798409A (zh) * 2023-07-14 2023-09-22 杭州阿里巴巴飞天信息技术有限公司 语音处理方法、存储介质及系统

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021015189A (ja) * 2019-07-11 2021-02-12 中部電力株式会社 マルチモーダル音声認識装置およびマルチモーダル音声認識方法
CN112382278A (zh) * 2020-11-18 2021-02-19 北京百度网讯科技有限公司 流式语音识别结果显示方法、装置、电子设备和存储介质
CN113270086A (zh) * 2021-07-19 2021-08-17 中国科学院自动化研究所 一种融合多模态语义不变性的语音识别文本增强系统
CN113470620A (zh) * 2021-07-06 2021-10-01 青岛洞听智能科技有限公司 一种语音识别方法
CN114974237A (zh) * 2022-05-13 2022-08-30 北京百度网讯科技有限公司 流式语音的处理和模型训练方法、装置、设备和介质
CN116798409A (zh) * 2023-07-14 2023-09-22 杭州阿里巴巴飞天信息技术有限公司 语音处理方法、存储介质及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112489620B (zh) * 2020-11-20 2022-09-09 北京有竹居网络技术有限公司 语音合成方法、装置、可读介质及电子设备
CN112466288B (zh) * 2020-12-18 2022-05-31 北京百度网讯科技有限公司 语音识别方法、装置、电子设备及存储介质
CN113838456B (zh) * 2021-09-28 2024-05-31 中国科学技术大学 音素提取方法、语音识别方法、装置、设备及存储介质
CN115565533A (zh) * 2022-09-21 2023-01-03 科大讯飞股份有限公司 语音识别方法、装置、设备及存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2021015189A (ja) * 2019-07-11 2021-02-12 中部電力株式会社 マルチモーダル音声認識装置およびマルチモーダル音声認識方法
CN112382278A (zh) * 2020-11-18 2021-02-19 北京百度网讯科技有限公司 流式语音识别结果显示方法、装置、电子设备和存储介质
CN113470620A (zh) * 2021-07-06 2021-10-01 青岛洞听智能科技有限公司 一种语音识别方法
CN113270086A (zh) * 2021-07-19 2021-08-17 中国科学院自动化研究所 一种融合多模态语义不变性的语音识别文本增强系统
CN114974237A (zh) * 2022-05-13 2022-08-30 北京百度网讯科技有限公司 流式语音的处理和模型训练方法、装置、设备和介质
CN116798409A (zh) * 2023-07-14 2023-09-22 杭州阿里巴巴飞天信息技术有限公司 语音处理方法、存储介质及系统

Also Published As

Publication number Publication date
CN116798409A (zh) 2023-09-22

Similar Documents

Publication Publication Date Title
US12462115B2 (en) System and method for temporal attention behavioral analysis of multi-modal conversations in a question and answer system
CN109859736B (zh) 语音合成方法及系统
CN110909613B (zh) 视频人物识别方法、装置、存储介质与电子设备
CN113936637B (zh) 基于多模态知识图谱的语音自适应补全系统
WO2024140434A1 (zh) 基于多模态知识图谱的文本分类方法、设备及存储介质
CN112083806A (zh) 一种基于多模态识别的自学习情感交互方法
CN107885756A (zh) 基于深度学习的对话方法、装置及设备
WO2024140430A9 (zh) 基于多模态深度学习的文本分类方法、设备及存储介质
CN113392265A (zh) 多媒体处理方法、装置及设备
CN114373443B (zh) 语音合成方法和装置、计算设备、存储介质及程序产品
US11984113B2 (en) Method and server for training a neural network to generate a textual output sequence
CN113314119A (zh) 语音识别智能家居控制方法及装置
WO2024114303A1 (zh) 音素识别方法、装置、电子设备及存储介质
WO2025092406A1 (zh) 语音处理方法、会议语音增强方法和语音模型训练方法
US20250329334A1 (en) Speech processing method and apparatus, device, and medium
CN117077787A (zh) 文本生成方法、装置、冰箱及存储介质
WO2025016325A1 (zh) 语音处理方法、存储介质及系统
Gupta et al. A multimodal fusion model for real-time environment emotion recognition using audio-visual-textual features
CN113223513A (zh) 语音转换方法、装置、设备和存储介质
CN117591636A (zh) 基于生活支援的多模态人机交互系统及方法
CN117290524A (zh) 用于信息推荐的方法、装置和家电设备
CN108989551A (zh) 位置提示方法、装置、存储介质及电子设备
Sompalyam et al. LipCraft: Building a Visual Speech Recognition System with Machine Learning and Streamlit
CN115695943A (zh) 数字人视频生成方法、装置、设备及存储介质
CN120015036B (zh) 一种语音意图识别方法及相关装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24842318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE