WO2021012734A1

WO2021012734A1 - Audio separation method and apparatus, electronic device and computer-readable storage medium

Info

Publication number: WO2021012734A1
Application number: PCT/CN2020/086757
Authority: WO
Inventors: 高立志
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-07-25
Filing date: 2020-04-24
Publication date: 2021-01-28
Also published as: CN110473566A

Abstract

An audio separation method and apparatus, an electronic device and a computer-readable storage medium. The method comprises: acquiring speech (S11); performing noise filtering on the speech (S12); extracting voiceprint feature data from the filtered speech, inputting the voiceprint feature data into a pre-set speech classification model for classification to obtain a classification result, encoding, according to the classification result, the speech corresponding to the same voiceprint feature data and storing same as a separate speech file, and performing separation processing on the speech (S13); and recognizing the speech after separation processing to acquire recognized text of the speech (S14). A pre-set speech classification model is used to perform separation processing on filtered speech according to the voiceprint features of the speech, and the speech after separation processing is recognized to acquire recognized text of the speech, so that the speech text of words spoken by different people in the speech can be recognized, thereby improving the accuracy of speech recognition.

Description

Audio separation method, device, electronic equipment and computer readable storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 25, 2019, the application number is 201910678465.5, and the invention title is "audio separation method, device, electronic equipment, and computer-readable storage medium", and its entire content Incorporated in this application by reference.

Technical field

This application relates to the field of speech processing, and in particular to an audio separation method, device, electronic equipment, and computer-readable storage medium.

Background technique

At present, in general speech recognition, if there are multiple people speaking, the recognized text contains the content of multiple people's speech. The inventor realizes that it is impossible to distinguish who said the text, which affects the recognition effect and accuracy.

Summary of the invention

In view of the above, it is necessary to propose an audio separation method, device, electronic equipment, and computer-readable storage medium to improve the accuracy of speech recognition.

The first aspect of the present application provides an audio separation method, the method including:

Get voice

Noise filtering the voice;

Extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and encode the voice corresponding to the same voiceprint feature data according to the classification result. Store the voice as a separate voice file and separate the voice; and

Recognizing the speech after the separation process to obtain the recognized text of the speech.

The second aspect of the application provides an audio separation device, which includes:

Acquisition module, used to acquire voice;

A noise filtering module for performing noise filtering on the voice;

The voice separation module is used to extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and classify the same voiceprint feature data according to the classification result The corresponding speech is encoded and stored as a separate speech file to separate the speech; and

The text recognition module is used to recognize the speech after the separation process to obtain the recognized text of the speech.

A third aspect of the present application provides an electronic device including a processor configured to implement the audio separation method when executing a computer program stored in a memory, and the audio separation method includes:

Get voice

Noise filtering the voice;

A fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the audio separation method is implemented, and the audio separation method includes:

Get voice

Noise filtering the voice;

This application uses a preset voice classification model to separate the filtered voice according to the voiceprint characteristics of the voice, and to recognize the voice after the separation process to obtain the recognized text of the voice, which can identify different people in the voice The speech text of spoken words improves the accuracy of speech recognition.

Description of the drawings

Fig. 1 is a flowchart of an audio separation method in an embodiment of the present application.

Fig. 2 is a schematic diagram of an application environment of an audio separation method in an embodiment of the present application.

Fig. 3 is a schematic diagram of a page audio separation device in an embodiment of the present application.

Fig. 4 is a schematic diagram of an electronic device in an embodiment of the present application.

Detailed ways

In order to be able to understand the above objectives, features and advantages of the application more clearly, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the application and the features in the embodiments can be combined with each other if there is no conflict.

In the following description, many specific details are set forth in order to fully understand the present application. The described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field of this application. The terms used in the description of the application herein are only for the purpose of describing specific embodiments, and are not intended to limit the application.

Preferably, the audio separation method of the present application is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculation and/or information processing in accordance with pre-set or stored instructions. Its hardware includes, but is not limited to, a microprocessor and an application specific integrated circuit (ASIC) , Field-Programmable Gate Array (FPGA), Digital Processor (Digital Signal Processor, DSP), embedded equipment, etc.

The electronic device may be a computing device such as a desktop computer, a notebook computer, a tablet computer, and a cloud server. The device can interact with the user through a keyboard, a mouse, a remote control, a touch panel, or a voice control device.

Example 1

Fig. 1 is a flowchart of an audio separation method in an embodiment of the present application. According to different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

Referring to FIG. 1, the audio separation method specifically includes the following steps:

Step S11: Acquire voice.

Please refer to FIG. 2, which shows an application environment diagram of an audio separation method in an embodiment of this application. In this embodiment, the method is applied in a terminal device 1. The terminal device 1 includes a voice acquisition unit 11. The terminal device 1 acquires voice through the voice acquiring unit 11. In this embodiment, the voice acquisition unit 11 may include, but is not limited to, an electric, capacitive, piezoelectric, electromagnetic, and semiconductor microphone. In another embodiment, the terminal device 1 can receive a voice sent by an external device 2 communicatively connected with the terminal device 1. In other embodiments, the terminal device 1 obtains voice from the storage device of the terminal device 1.

Step S12: Perform noise filtering on the voice.

In one embodiment, the terminal device 1 filters the environmental noise in the voice. For example, when the terminal device 1 acquires voice through the voice acquiring unit 11 from a noisy environment, the voice includes environmental noise of the environment where the terminal device 1 is located. In a specific embodiment, the terminal device 1 detects whether the voice decibel of the acquired voice is within a preset decibel threshold range, and when the voice decibel of the voice is not within the preset decibel threshold range, the terminal device 1 The speech is noise filtered. The preset decibel threshold can be set as required, and in this embodiment, the preset decibel threshold range can be set to 70-80db. The terminal device 1 selects the voice information whose voice decibel exceeds the first decibel threshold as the environmental noise from the voice, and deletes the environmental noise whose voice decibel exceeds the first decibel threshold, so as to realize the Voice noise filtering. In this embodiment, the first decibel threshold can be set as required, for example, the first decibel threshold can be set to 80db.

In one embodiment, the terminal device 1 filters the environmental noise in the speech through a deep learning voiceprint noise reduction method. In a specific implementation method, the voiceprint noise reduction method through deep learning to filter the environmental noise in the voice includes: establishing a machine learning and deep learning model; establishing a voiceprint recognition model; and passing the acquired voice through Machine learning and deep learning models are used for learning, and the environmental noise in the speech is recognized and distinguished; the speech after recognition by the machine learning and deep learning models is filtered to remove the speech that does not belong to human speech audio Ambient noise, get the voice that has undergone preliminary screening; determine whether the voice that has undergone the preliminary screening reaches the preset threshold; when it is determined that the voice that has undergone the preliminary screening reaches the preset threshold, the voice that has reached the preset threshold and the voiceprint The recognition model is compared and extracted, the voice frequency and the spectral image that are consistent with the voiceprint recognition model are retained, the voices that are not consistent with the voiceprint recognition model are eliminated, and the voice processed by voiceprint noise reduction is obtained.

In this embodiment, the terminal device 1 uses a large amount of obtained speech environment audio and a large amount of specific person's speech audio to build a machine learning and deep learning model; all the speech environment audio and specific person's speech audio are converted into the form of pop chart It is imported into the terminal device 1, and through a large amount of repeated training, the environmental noise (environmental sound) is distinguished from the voice pop chart of a specific person's speech through machine learning and deep learning.

In this embodiment, each individual’s unique voiceprint can be observed using the grammar. Acquire the voiceprint of a specific speaker, perform feature extraction operation on the voiceprint of the person, use the existing voiceprint of the specific speaker to build a voiceprint spectrogram, and perform the features on the voiceprint spectrogram After extraction, a voiceprint recognition model that only belongs to the person can be established. The modeling methods of voiceprint recognition models are divided into three types: text-related, text-independent and text prompt. Since the input voice content cannot be determined, the text-independent type is selected for voiceprint modeling, thereby obtaining the voiceprint recognition model. Among them, the text irrelevant includes: GMM-UBM, GMM-SVM, GMM-UBM-LF, i-vector/PLDA). In this embodiment, GMM-UBM is selected to build the voiceprint modeling of the speaker confirmation system. When the voices and test voices of multiple speakers are input, the MFCC feature vector is extracted, and after a large amount of human voiceprint data is repeatedly trained and MAP adaptive Process and confirm the decision to obtain a human voiceprint recognition model with a high voiceprint recognition rate. In this embodiment, the MFCC feature vector extraction process includes input sample speech, the sample speech pre-emphasis, framing, and windowing, the processed sample speech is subjected to Fourier transform, Mel frequency filtering, and Log logarithmic energy , Calculate the cepstrum of the sample and output the MFCC image.

In another embodiment, the terminal device 1 filters out white noise in the speech. Among them, white noise refers to noise with equal noise energy contained in frequency bands of equal bandwidth within a wider frequency range. In this implementation manner, the white noise in the speech can be removed by the wavelet transform algorithm or the Kalman filter algorithm.

Step S13: Extract voiceprint feature data from the filtered voice, input the voiceprint feature data into the preset voice classification model for classification to obtain a classification result, and assign the same voiceprint feature data to the corresponding The speech is encoded and stored as a separate speech file, and the speech is processed separately.

Since everyone in the real world has specific voiceprint characteristics, which are gradually formed by our vocal organs during the growth process, no matter how similar others imitate our speech, the voiceprint characteristics are actually significantly different of. Therefore, in this embodiment, the voiceprint feature can be used to verify the speaker's identity and distinguish the speaker's voice. In specific practical applications, the voiceprint feature data includes, but is not limited to, Mel cepstrum coefficient MFCC, perceptual linear prediction coefficient PLP, depth feature Deep Feature, and energy regularization spectrum coefficient PNCC. After the voice is filtered by noise, the terminal device 1 uses wavelet transform technology to extract the Mel cepstrum coefficient MFCC, the perceptual linear prediction coefficient PLP, the depth feature Deep Feature, or the energy normalized spectral coefficient PNCC, etc. According to the Mel Cepstral Coefficient MFCC, Perceptual Linear Prediction Coefficient PLP, Deep Feature or Energy Normalized Spectral Coefficient PNCC, the voiceprint feature data is input into the preset voice classification model to obtain the classification result. As a result, the voice corresponding to the same voiceprint feature data is encoded and stored as a separate voice file.

In this embodiment, the preset speech classification model includes at least one of the following: a vector machine model, a random model, and a neural network model. Specifically, the terminal device uses a pre-trained preset voice classification model to determine the category of the voiceprint feature data according to the extracted voiceprint feature data. In this application, the categories of the voiceprint feature data include: a first voiceprint feature category, a second voiceprint feature category, and a third voiceprint feature category. In this embodiment, the training process of the preset voice classification model includes:

1) Acquire the voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample, and label the voiceprint feature category of the voiceprint feature data of the positive sample, so that the voiceprint feature data of the positive sample carry the voiceprint feature category label.

For example, to select 500 voiceprint feature data corresponding to the first voiceprint feature category, the second voiceprint feature category, and the third voiceprint feature category respectively, and to label each voiceprint feature data category, you can use "1" as For the voiceprint feature label of the first voiceprint feature category, "2" is used as the voiceprint feature label of the second voiceprint feature category, and "3" is used as the voiceprint feature label of the third voiceprint feature category.

2) The voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample are randomly divided into a training set with a first preset ratio and a verification set with a second preset ratio, and the training set is used to train the Preset a speech classification model, and use the verification set to verify the accuracy of the preset speech classification model after training.

First distribute the training samples in the training set with different voiceprint characteristics to different folders. For example, the training samples of the first voiceprint feature category are distributed to the first folder, the training samples of the second voiceprint feature category are distributed to the second folder, and the training samples of the third voiceprint feature category are distributed to the third folder. Folder. Then extract the training samples of the first preset ratio (for example, 70%) from different folders as the total training samples to train the preset voice classification model, and take the remaining second samples from different folders. A preset proportion (for example, 30%) of training samples is used as a total test sample to verify the accuracy of the preset voice classification model after training.

3) If the accuracy rate is greater than or equal to the preset accuracy rate, the training ends, and the trained preset voice classification model is used as a classifier to identify the category of the voiceprint feature data; if the accuracy rate is less than When the accuracy rate is preset, the number of positive samples and the number of negative samples are increased to retrain the preset voice classification model until the accuracy rate is greater than or equal to the preset accuracy rate.

In this embodiment, the terminal device 1 is also used to perform enhanced amplification processing on the voice corresponding to the same voiceprint feature data; and encode the voice after the enhanced amplification processing. That is, the terminal device 1 separates the voices with different voiceprint features from the voice according to the voiceprint features, respectively strengthens the amplification process of the separated voices, and encodes the voices corresponding to the same voiceprint features. Stored as a separate voice file, and stored separately as a voice file.

Step S14: Recognizing the speech after the separation process to obtain the recognized text of the speech.

In this embodiment, the terminal device 1 converts the separated speech into text through speech recognition, and uses it as the initial speech recognition text; and matches the initial speech recognition text with a preset text database to obtain the matched Speech recognition text.

In this embodiment, the specific process of the terminal device 1 converting the separated speech into text through speech recognition includes:

1) Extract audio features of the voice and convert them into acoustic feature vectors of preset length;

2) Decode the feature vector into word order according to the decoding algorithm;

3) Obtain the subwords corresponding to the word order through the HMM phoneme model, and the subwords are initials and vowels;

4) Combine multiple subwords into text according to the preset pronunciation dictionary;

5) Use the language model grammar rules to decode to get the optimal sequence and get the text.

In this embodiment, the grammar rule is the Viterbi algorithm. For example, the voice to be recognized is "Hello", which is transformed into a 39-dimensional acoustic feature vector after feature extraction, and multiple corresponding sub-words /n//i//h//ao are obtained through multiple HMM phoneme models /, Splice multiple sub-words into characters according to the preset pronunciation dictionary, such as you, Nepal; good, number. Decode by Viterbi algorithm to obtain the optimal sequence "Hello" and output the text.

In this embodiment, at least two text databases may be preset, for example, a first text database and a second text database. The first text database can be dedicated to storing multiple modal particles, such as "um", "ah", "yeah", etc. The modal particles have nothing to do with the content of the meeting and easily affect the readability of the speech converted into text. The second text database can be dedicated to storing multiple professional words and their corresponding pinyin, such as "feature vector", "feature matrix", "tensor analysis", etc. The professional words are more complex, so they tend to appear in batches during the process of speech recognition error. In this application, a third text database can also be preset according to the actual situation, specifically for storing sentences such as names or place names. This article does not make specific restrictions on the number of pre-set databases and corresponding contents of this article.

In this embodiment, the terminal device 1 matching the initial voice recognition text with a preset text database specifically includes:

1) Match the initial speech recognition text with a preset first text database to obtain a first matching result; and

2) Match the first matching result with a preset second text database to obtain a second matching result;

Specifically, the matching the initial speech recognition text with a preset first text database includes: determining whether there is a first word in the initial speech recognition text that matches a word in the preset first text database; When it is determined that there is a first word that matches a word in the preset first text database in the initial voice recognition text, the first word that matches in the initial voice recognition text is processed.

Preferably, the processing of the matching first word in the initial speech recognition text may further include: judging whether the matching first word is based on the pre-trained modal particle model based on the deep learning network Is the modal particle to be deleted; when it is determined that the first matching word is the modal particle to be deleted, the first matching word in the initial speech recognition text is eliminated; when the matching first word is determined When a word is not a modal particle to be deleted, the first matching word in the initial speech recognition text is retained.

For example, suppose the initial speech recognition text is "this is very easy to use", and the modal word "this" is stored in the preset first text database, then the initial speech recognition text is matched with the preset first text database to determine The matched word is "this", and then judge whether the matched first word "this" is the modal particle to be deleted according to the pre-trained modal particle model based on the deep learning network. The network's modal particle model determines that the matched first word "this" does not belong to the modal particle to be deleted in "this is very useful", then the first matching word in the initial speech recognition text is retained, The first matching result obtained is "This is pretty easy to use".

For another example, suppose the initial speech recognition text is "this, we are going to have a meeting", and the first text database is preset to store the modal word "this", then the initial speech recognition text is matched with the preset first text database to determine The matched word is "this", and then judge whether the matched first word "this" is the modal particle to be deleted according to the pre-trained modal particle model based on the deep learning network. The network's modal particle model determines that the first matching word "this" belongs to the modal particle to be deleted in "this, we are going to have a meeting", and then the first matching word in the initial speech recognition text is eliminated. The first matching result obtained was "We are going to have a meeting."

Specifically, the matching the first matching result with a preset second text database includes:

1) Convert the words in the first matching result to the first pinyin;

2) Determine whether there is a second pinyin that is the same as the first pinyin in the preset second text database;

3) When it is determined that there is a second pinyin that is the same as the first pinyin in the preset second text database, the words corresponding to the second pinyin are extracted as the words corresponding to the first pinyin.

For example, suppose that the first matching result is "this is an original giant earthquake", and the words in the first matching result are converted to the first pinyin as "zhe shi yige yuanshi Juzhen"; the second text database is preset to store professional words "Matrix" and the corresponding second pinyin "juzheng", when it is determined that there is a second pinyin identical to the first pinyin in the preset second text database, the word "juzheng" corresponding to the second pinyin " "Matrix" is extracted as the word corresponding to the first pinyin "juzheng", and the second matching result obtained is "This is an original matrix".

This application converts the separated speech into text through speech recognition technology, as the initial speech recognition text; and matches the initial speech recognition text with a preset text database to obtain the matched speech recognition text, which can be recognized The voice text of the words spoken by different people in the voice is convenient for the recorder to gather information.

Example 2

FIG. 3 is a schematic diagram of an audio separation device 40 in an embodiment of the application.

In some embodiments, the audio separation device 40 runs in an electronic device. The audio separation device 40 may include multiple functional modules composed of program code segments. The program code of each program segment in the audio separation device 40 can be stored in a memory and executed by at least one processor to perform the audio separation function.

In this embodiment, the audio separation device 40 can be divided into multiple functional modules according to the functions it performs. Referring to FIG. 3, the audio separation device 40 may include an acquisition module 401, a noise filtering module 402, a speech separation module 403, and a text recognition module 404. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, and are stored in a memory. In some embodiments, the functions of each module will be detailed in subsequent embodiments.

The acquiring module 401 is used for acquiring voice.

The acquisition module 401 acquires voice through the voice acquisition unit 11. In this embodiment, the voice acquisition unit 11 may include, but is not limited to, an electric, capacitive, piezoelectric, electromagnetic, and semiconductor microphone. In another embodiment, the acquisition module 401 can receive the voice sent by the external device 2 communicatively connected with the terminal device 1. In other embodiments, the acquiring module 401 acquires the voice from the storage device of the terminal device 1.

The noise filtering module 402 is configured to perform noise filtering on the speech.

In one embodiment, the noise filtering module 402 filters the environmental noise in the speech. In a specific embodiment, the noise filtering module 402 detects whether the voice decibel of the acquired voice is within a preset decibel threshold range, and when the voice decibel of the voice is not within the preset decibel threshold range, the noise filtering module 402 Perform noise filtering on the voice. The preset decibel threshold can be set as required. In this embodiment, the preset decibel threshold range can be set to 70-80db. The noise filtering module 402 selects the voice information whose voice decibel exceeds the first decibel threshold as the environmental noise from the voice, and deletes the environmental noise whose voice decibel exceeds the first decibel threshold, so as to realize the The noise filtering of the speech. In this embodiment, the first decibel threshold can be set as required, for example, the first decibel threshold can be set to 80db.

In one embodiment, the noise filtering module 402 filters the environmental noise in the speech by using a deep learning voiceprint noise reduction method. In a specific implementation method, the voiceprint noise reduction method through deep learning to filter the environmental noise in the voice includes: establishing a machine learning and deep learning model; establishing a voiceprint recognition model; and passing the acquired voice through Machine learning and deep learning models are used for learning, and the environmental noise in the speech is recognized and distinguished; the speech after recognition by the machine learning and deep learning models is filtered to remove the speech that does not belong to human speech audio Ambient noise, get the voice that has undergone preliminary screening; determine whether the voice that has undergone the preliminary screening reaches the preset threshold; when it is determined that the voice that has undergone the preliminary screening reaches the preset threshold, the voice that has reached the preset threshold and the voiceprint The recognition model is compared and extracted, the voice frequency and the spectral image that are consistent with the voiceprint recognition model are retained, the voices that are not consistent with the voiceprint recognition model are eliminated, and the voice processed by voiceprint noise reduction is obtained.

In this embodiment, the noise filtering module 402 uses a large amount of obtained speech environment audio and a large amount of specific person's speech audio to build a machine learning and deep learning model; converts all the speech environment audio and specific person's speech audio into a pop chart. The format is imported into the terminal device 1, and through a lot of repeated training, the environmental noise (environmental sound) and the voice pop chart of a specific person's speech are distinguished through machine learning and deep learning.

In another embodiment, the noise filtering module 402 filters white noise in the speech. Among them, white noise refers to noise with equal noise energy contained in frequency bands of equal bandwidth within a wider frequency range. In this implementation manner, the white noise in the speech can be removed by the wavelet transform algorithm or the Kalman filter algorithm.

The voice separation module 403 is configured to use a preset voice classification model to perform separation processing on the filtered voice according to the voiceprint features of the voice.

In this embodiment, the voice separation module 403 uses a preset voice classification model to separate the filtered voice according to the voiceprint features of the voice, including: extracting voiceprint feature data from the filtered voice, and dividing the voiceprint The feature data is input into the preset voice classification model for classification to obtain the classification result. According to the classification result, the voice corresponding to the same voiceprint feature data is coded and stored as a separate voice file, thus realizing the separation processing of the voice .

Since everyone in the real world has specific voiceprint characteristics, which are gradually formed by our vocal organs during the growth process, no matter how similar others imitate our speech, the voiceprint characteristics are actually significantly different of. Therefore, in this embodiment, the voiceprint feature can be used to verify the speaker's identity and distinguish the speaker's voice. In specific practical applications, the voiceprint feature data includes, but is not limited to, Mel cepstrum coefficient MFCC, perceptual linear prediction coefficient PLP, depth feature Deep Feature, and energy regularization spectrum coefficient PNCC. After the voice is filtered by noise, the voice separation module 403 uses wavelet transform technology to extract the Mel cepstrum coefficient MFCC, the perceptual linear prediction coefficient PLP, the depth feature Deep Feature or the energy normalized spectral coefficient PNCC, etc. Voiceprint feature data, and input the voiceprint feature data into the preset voice classification model to obtain the classification result according to the Mel cepstrum coefficient MFCC, the perceptual linear prediction coefficient PLP, the depth feature Deep Feature or the energy normalized spectrum coefficient PNCC. As a result of the classification, the voice corresponding to the same voiceprint feature data is encoded and stored as a separate voice file.

In this embodiment, the preset speech classification model includes at least one of the following: a vector machine model, a random model, and a neural network model. Specifically, the terminal device uses a pre-trained preset voice classification model to determine the category of the voiceprint feature data according to the extracted voiceprint feature data. In this application, the categories of the voiceprint feature data include: a first voiceprint feature category, a second voiceprint feature category, and a third voiceprint feature category. In this embodiment, the training process of inputting the voiceprint feature data into the preset voice classification model for classification to obtain the classification result includes:

In this embodiment, the voice separation module 403 is also used to perform enhanced amplification processing on the voice corresponding to the same voiceprint feature data; and encode the voice after the enhanced amplification processing. That is, the terminal device 1 separates the voices with different voiceprint features from the voice according to the voiceprint features, respectively strengthens the amplification process of the separated voices, and encodes the voices corresponding to the same voiceprint features. Stored as a separate voice file, and stored separately as a voice file.

The text recognition module 404 is configured to recognize the speech after the separation process to obtain the recognized text of the speech.

In this embodiment, the text recognition module 404 converts the separated speech into text through speech recognition, as the initial speech recognition text; and matches the initial speech recognition text with a preset text database to obtain the matching Speech recognition text.

In this embodiment, the specific process for the text recognition module 404 to convert the separated speech into text through speech recognition includes:

In this embodiment, at least two text databases may be preset, for example, a first text database and a second text database. The first text database can be dedicated to storing multiple modal particles, such as "um", "ah", "yeah", etc. The modal particles have nothing to do with the content of the meeting and easily affect the readability of the speech converted into text. The second text database can be dedicated to storing multiple professional words and their corresponding pinyin, such as "feature vector", "feature matrix", "tensor analysis", etc. The professional words are more complex, so they tend to appear in batches during the process of speech recognition error. In this application, a third text database can also be preset according to the actual situation, specifically for storing sentences such as names or place names. This article does not make specific restrictions on the number of pre-set text databases and corresponding content.

In this embodiment, the text recognition module 404 matching the initial speech recognition text with a preset text database specifically includes:

1) Convert the words in the first matching result to the first pinyin;

Example 3

FIG. 4 is a schematic diagram of a preferred embodiment of the electronic device 7 of this application.

The electronic device 7 includes a memory 71, a processor 72, and a computer program 73 that is stored in the memory 71 and can run on the processor 72. When the processor 72 executes the computer program 73, the steps in the above audio separation method embodiment are implemented, such as steps S11 to S14 shown in FIG. 1. That is, the audio separation method includes: acquiring speech; performing noise filtering on the speech; extracting voiceprint feature data from the filtered voice, and inputting the voiceprint feature data into a preset voice classification model for classification to obtain a classification result According to the classification result, the voice corresponding to the same voiceprint feature data is encoded and stored as a separate voice file to separate the voice; and the voice after the separation is recognized to obtain the information of the voice Recognize the text. Alternatively, when the processor 72 executes the computer program 73, the functions of the modules/units in the foregoing audio separation device embodiment are implemented, for example, the modules 401 to 404 in FIG. 3.

Exemplarily, the computer program 73 may be divided into one or more modules/units, and the one or more modules/units are stored in the memory 71 and executed by the processor 72 to complete This application. The one or more modules/units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 73 in the electronic device 7. For example, the computer program 73 can be divided into an acquisition module 401, a noise filtering module 402, a speech separation module 403, and a text recognition module 404 in FIG. 3. For specific functions of each module, refer to the second embodiment.

In this embodiment, the electronic device 7 and the terminal device 1 are the same device. In other embodiments, the electronic device 7 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. Those skilled in the art can understand that the schematic diagram is only an example of the electronic device 7 and does not constitute a limitation on the electronic device 7. It may include more or less components than those shown in the figure, or a combination of certain components, or different components. Components, for example, the electronic device 7 may also include input and output devices, network access devices, buses, and the like.

The so-called processor 72 may be a central processing module (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor 72 can also be any conventional processor, etc. The processor 72 is the control center of the electronic device 7 and connects the entire electronic device 7 with various interfaces and lines. Parts.

The memory 71 may be used to store the computer program 73 and/or modules/units. The processor 72 runs or executes the computer programs and/or modules/units stored in the memory 71 and calls the computer programs and/or modules/units stored in the memory 71. The data in 71 realizes various functions of the electronic device 7 described above. The memory 71 may mainly include a storage program area and a storage data area. The storage program area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may The data (such as audio data, phone book, etc.) created according to the use of the electronic device 7 is stored. In addition, the memory 71 may include a high-speed random access memory, and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), and a Secure Digital (SD) Card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

If the integrated module/unit of the electronic device 7 is implemented in the form of a software function module and sold or used as an independent product, it may be stored in a computer-readable storage medium, which may be non-easy. Loss of sex can also be volatile. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the above audio separation method can be realized, which includes: acquiring speech; filtering the speech noise; extracting voiceprint feature data from the filtered voice, and inputting the voiceprint feature data Perform classification to a preset voice classification model to obtain a classification result, encode and store the voice corresponding to the same voiceprint feature data as a separate voice file according to the classification result, and perform separation processing on the voice; and The subsequent voice is recognized to obtain the recognized text of the voice. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory) , Random Access Memory (RAM, Random Access Memory), electrical carrier signal, telecommunications signal, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or deleted in accordance with the requirements of the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to the legislation and patent practice, the computer-readable medium Does not include electrical carrier signals and telecommunication signals.

In the several embodiments provided in this application, it should be understood that the disclosed electronic device and method may be implemented in other ways. For example, the electronic device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

In addition, the functional modules in the various embodiments of the present application may be integrated in the same processing module, or each module may exist alone physically, or two or more modules may be integrated in the same module. The above-mentioned integrated modules can be implemented in the form of hardware, or in the form of hardware plus software functional modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the application and not to limit them. Although the application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the application can be Modifications or equivalent replacements are made without departing from the spirit and scope of the technical solution of this application.

Claims

An audio separation method, wherein the method includes:

Get voice

Noise filtering the voice;

Extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and encode the voice corresponding to the same voiceprint feature data according to the classification result. Store the voice as a separate voice file and separate the voice; and

Recognizing the speech after the separation process to obtain the recognized text of the speech.
The audio separation method according to claim 1, wherein the training process of the preset voice classification model comprises:

Acquire the voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample, and label the voiceprint feature category of the voiceprint feature data of the positive sample, so that the voiceprint feature data of the positive sample carry the voiceprint feature category label;

The voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample are randomly divided into a training set of a first preset ratio and a verification set of a second preset ratio, and the training set is used to train the preset A voice classification model, and use the verification set to verify the accuracy of the preset voice classification model after training;

If the accuracy rate is greater than or equal to the preset accuracy rate, the training is ended, and the preset voice classification model after training is used as a classifier to identify the category of the voiceprint feature data; and

If the accuracy rate is less than the preset accuracy rate, the number of positive samples and the number of negative samples are increased to retrain the preset voice classification model until the accuracy rate is greater than or equal to the preset accuracy rate.
The audio separation method according to claim 1, wherein said performing noise filtering on said voice comprises:

The voice information whose voice decibel exceeds the first decibel threshold is selected from the voice as environmental noise, and the environmental noise whose voice decibel exceeds the first decibel threshold is deleted.
The audio separation method according to claim 1, wherein said performing noise filtering on said voice comprises:

Establish a machine learning and deep learning model; establish a voiceprint recognition model; learn the acquired voice through the machine learning and deep learning model, and identify and distinguish the environmental noise in the voice; pass the machine learning And the speech after recognition by the deep learning model is filtered, and the environmental noise in the speech that is not human speech audio is removed, and the speech that has undergone preliminary screening is obtained; it is determined whether the speech after the preliminary screening reaches the preset threshold; When the preliminarily screened voice reaches the preset threshold, the voice that reaches the preset threshold is compared and extracted with the voiceprint recognition model, and the voice frequency and spectral image that are consistent with the voiceprint recognition model are retained, and those with The voiceprint recognition model does not conform to the voice, and the voiceprint noise reduction processed voice is obtained.
The audio separation method according to claim 1, wherein said recognizing the speech after the separation process to obtain the recognized text of the speech comprises:

Convert the separated speech into text through speech recognition as the initial speech recognition text; and

The initial speech recognition text is matched with the preset text database to obtain the matched speech recognition text.
5. The audio separation method according to claim 5, wherein said converting the separated speech into text through speech recognition comprises:

Extracting audio features of the voice and converting them into acoustic feature vectors of preset length;

Decoding the feature vector into word order according to a decoding algorithm;

Obtain the subwords corresponding to the word order through the HMM phoneme model, where the subwords are initials and vowels;

Combine multiple sub-words into text according to the preset pronunciation dictionary; and

Use the Viterbi algorithm to decode the optimal sequence and get the text.
The audio separation method according to claim 1, wherein:

The voiceprint features include Mel cepstrum coefficient MFCC, perceptual linear prediction coefficient PLP, depth feature Deep Feature, and energy regularization spectrum coefficient PNCC.
An audio separation device, wherein the device includes:

Acquisition module, used to acquire voice;

A noise filtering module for performing noise filtering on the voice;

The voice separation module is used to extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and classify the same voiceprint feature data according to the classification result The corresponding speech is encoded and stored as a separate speech file to separate the speech; and

The text recognition module is used to recognize the speech after the separation process to obtain the recognized text of the speech.
An electronic device, wherein the electronic device includes a processor configured to implement an audio separation method when executing a computer program stored in a memory, and the audio separation method includes:

Get voice

Noise filtering the voice;

Extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and encode the voice corresponding to the same voiceprint feature data according to the classification result. The voice is stored as a separate voice file to perform separation processing; and the voice after the separation processing is recognized to obtain the recognized text of the voice.
The electronic device according to claim 9, wherein the training process of the preset voice classification model comprises:

Acquire the voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample, and label the voiceprint feature category of the voiceprint feature data of the positive sample, so that the voiceprint feature data of the positive sample carry the voiceprint feature category label;

The voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample are randomly divided into a training set of a first preset ratio and a verification set of a second preset ratio, and the training set is used to train the preset A voice classification model, and use the verification set to verify the accuracy of the preset voice classification model after training;

If the accuracy rate is greater than or equal to the preset accuracy rate, the training is ended, and the preset voice classification model after training is used as a classifier to identify the category of the voiceprint feature data; and

If the accuracy rate is less than the preset accuracy rate, the number of positive samples and the number of negative samples are increased to retrain the preset voice classification model until the accuracy rate is greater than or equal to the preset accuracy rate.
9. The electronic device of claim 9, wherein said performing noise filtering on said voice comprises:

The voice information whose voice decibel exceeds the first decibel threshold is selected from the voice as environmental noise, and the environmental noise whose voice decibel exceeds the first decibel threshold is deleted.
9. The electronic device of claim 9, wherein said performing noise filtering on said voice comprises:

Establish a machine learning and deep learning model; establish a voiceprint recognition model; learn the acquired voice through the machine learning and deep learning model, and identify and distinguish the environmental noise in the voice; pass the machine learning And the speech after recognition by the deep learning model is filtered, and the environmental noise in the speech that is not human speech audio is removed, and the speech that has undergone preliminary screening is obtained; it is determined whether the speech after the preliminary screening reaches the preset threshold; When the preliminarily screened voice reaches the preset threshold, the voice that reaches the preset threshold is compared and extracted with the voiceprint recognition model, and the voice frequency and spectral image that are consistent with the voiceprint recognition model are retained, and those with The voiceprint recognition model does not conform to the voice, and the voiceprint noise reduction processed voice is obtained.
9. The electronic device of claim 9, wherein the recognizing the speech after the separation process to obtain the recognized text of the speech comprises:

Convert the separated speech into text through speech recognition as the initial speech recognition text; and

The initial speech recognition text is matched with the preset text database to obtain the matched speech recognition text.
The electronic device according to claim 13, wherein said converting the speech after the separation process into text through speech recognition comprises:

Extracting audio features of the voice and converting them into acoustic feature vectors of preset length;

Decoding the feature vector into word order according to a decoding algorithm;

Obtain the subwords corresponding to the word order through the HMM phoneme model, where the subwords are initials and vowels;

Combine multiple sub-words into text according to the preset pronunciation dictionary; and

Use the Viterbi algorithm to decode the optimal sequence and get the text.
A computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the audio separation method, and the audio separation method includes:

Get voice

Noise filtering the voice;

Extract voiceprint feature data from the filtered voice, input the voiceprint feature data into a preset voice classification model for classification to obtain a classification result, and encode the voice corresponding to the same voiceprint feature data according to the classification result. The voice is stored as a separate voice file to perform separation processing; and the voice after the separation processing is recognized to obtain the recognized text of the voice.
15. The computer-readable storage medium of claim 15, wherein the training process of the preset speech classification model comprises:

Acquire the voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample, and label the voiceprint feature category of the voiceprint feature data of the positive sample, so that the voiceprint feature data of the positive sample carry the voiceprint feature category label;

The voiceprint feature data of the positive sample and the voiceprint feature data of the negative sample are randomly divided into a training set of a first preset ratio and a verification set of a second preset ratio, and the training set is used to train the preset A voice classification model, and use the verification set to verify the accuracy of the preset voice classification model after training;

If the accuracy rate is greater than or equal to the preset accuracy rate, the training is ended, and the preset voice classification model after training is used as a classifier to identify the category of the voiceprint feature data; and

If the accuracy rate is less than the preset accuracy rate, the number of positive samples and the number of negative samples are increased to retrain the preset voice classification model until the accuracy rate is greater than or equal to the preset accuracy rate.
15. The computer-readable storage medium of claim 15, wherein said performing noise filtering on said speech comprises:

The voice information whose voice decibel exceeds the first decibel threshold is selected from the voice as environmental noise, and the environmental noise whose voice decibel exceeds the first decibel threshold is deleted.
15. The computer-readable storage medium of claim 15, wherein said performing noise filtering on said speech comprises:

Establish a machine learning and deep learning model; establish a voiceprint recognition model; learn the acquired voice through the machine learning and deep learning model, and identify and distinguish the environmental noise in the voice; pass the machine learning And the speech after recognition by the deep learning model is filtered, and the environmental noise in the speech that is not human speech audio is removed, and the speech that has undergone preliminary screening is obtained; it is determined whether the speech after the preliminary screening reaches the preset threshold; When the preliminarily screened voice reaches the preset threshold, the voice that reaches the preset threshold is compared and extracted with the voiceprint recognition model, and the voice frequency and spectral image that are consistent with the voiceprint recognition model are retained, and those with The voiceprint recognition model does not conform to the voice, and the voiceprint noise reduction processed voice is obtained.
15. The computer-readable storage medium according to claim 15, wherein the recognizing the speech after the separation process to obtain the recognized text of the speech comprises:

Convert the separated speech into text through speech recognition as the initial speech recognition text; and

The initial speech recognition text is matched with the preset text database to obtain the matched speech recognition text.
19. The computer-readable storage medium of claim 19, wherein said converting the separated speech into text through speech recognition comprises:

Extracting audio features of the voice and converting them into acoustic feature vectors of preset length;

Decoding the feature vector into word order according to a decoding algorithm;

Obtain the subwords corresponding to the word order through the HMM phoneme model, where the subwords are initials and vowels;

Combine multiple sub-words into text according to the preset pronunciation dictionary; and

Use the Viterbi algorithm to decode the optimal sequence and get the text.