CN115662457A - Voice noise reduction method, electronic device and storage medium - Google Patents

Abstract

The invention discloses a voice noise reduction method, electronic equipment and a storage medium. In the method, original voice data is obtained; predicting pure voice data corresponding to the original voice data; comparing the raw speech data and the clean speech data to determine a noise characteristic parameter; determining a noise reduction strategy matched with the noise characteristic parameters; and based on the noise reduction strategy, carrying out noise reduction processing on the original voice data. Therefore, noise reduction processing can be carried out in a targeted manner according to the noise characteristics of the original voice data, and a better noise reduction effect is achieved.

Description

Voice noise reduction method, electronic device and storage medium

Technical Field

The invention belongs to the technical field of voice processing, and particularly relates to a voice noise reduction method, electronic equipment and a storage medium.

Background

With the continuous development of voice technology, various voice interaction devices have been integrated into various aspects of people's lives. The wireless wallet wheat is favored by a plurality of users with the characteristic of convenient voice interaction, and particularly has wide application in outdoor program recording scenes and outdoor live broadcast scenes.

At present, the near-field sound pickup and noise reduction of the wireless neckband microphones on the market are basically realized through a traditional noise reduction algorithm in a chip, and the volume cannot be dynamically adjusted. Through traditional filter function, carry out rough machining to the original audio frequency that the tie wheat was picked up and was got, give the recording terminal, the expression of making an uproar falls all the same under each scene, can't satisfy the complicated service environment of tie wheat, leads to the performance of making an uproar is relatively poor under outdoor scene.

In view of the above problems, the industry has not provided a better solution for the moment.

Disclosure of Invention

An embodiment of the present invention provides a voice noise reduction method, an electronic device, and a storage medium, which are used to solve at least one of the above technical problems.

In a first aspect, an embodiment of the present invention provides a speech noise reduction method, including: acquiring original voice data; predicting pure voice data corresponding to the original voice data; comparing the raw speech data and the clean speech data to determine a noise characteristic parameter; determining a noise reduction strategy matched with the noise characteristic parameters; and based on the noise reduction strategy, carrying out noise reduction processing on the original voice data.

In a second aspect, an embodiment of the present invention provides an electronic device, which includes: the computer-readable medium includes at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the above-described method.

In a third aspect, the present invention provides a storage medium, in which one or more programs including execution instructions are stored, where the execution instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to perform the steps of the above-mentioned method of the present invention.

In a fourth aspect, the present invention also provides a computer program product comprising a computer program stored on a storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the steps of the above method.

The embodiment of the invention has the beneficial effects that:

and outputting corresponding pure voice data by a prediction algorithm aiming at the original voice data, and determining a noise characteristic parameter based on voice comparison so as to better analyze the noise characteristics of the original voice data. And then, a matched noise reduction strategy is determined according to the noise characteristic parameters, so that noise reduction processing can be carried out in a targeted manner according to the noise characteristics of the original voice data, and a better noise reduction effect is realized.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.

FIG. 1 shows a flow diagram of an example of a method of speech noise reduction according to an embodiment of the invention;

FIG. 2 shows a flowchart of an example of training a VDCNN according to an embodiment of the invention;

fig. 3 shows a schematic diagram of a training principle of the VDCNN according to an embodiment of the present invention;

FIG. 4 shows a flow diagram of an example of a method of speech noise reduction according to an embodiment of the invention;

FIG. 5A is a simulation diagram showing the noise reduction performance effect of a wireless neckline microphone in the prior art;

FIG. 5B is a simulation diagram showing the noise reduction performance effect of the wireless neckline microphone improved by the voice noise reduction method according to the embodiment of the invention;

fig. 6A is a simulation diagram illustrating a sound volume effect of a wireless neckline microphone in the prior art;

fig. 6B is a simulation diagram illustrating a sound pickup volume adjustment effect of a wireless neckline microphone improved by a voice noise reduction method according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an embodiment of an electronic device according to the invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.

The invention may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The invention may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.

As used herein, a "module," "system," and the like are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, or software in execution. In particular, for example, an element may be, but is not limited to being, a process running on a processor, an object, an executable, a thread of execution, a program, and/or a computer. Also, an application or script running on a server, or a server, may be an element. One or more elements may be in a process and/or thread of execution and an element may be localized on one computer and/or distributed between two or more computers and may be operated by various computer-readable media. The elements may also communicate by way of local and/or remote processes in accordance with a signal having one or more data packets, e.g., signals from data interacting with another element in a local system, distributed system, and/or across a network of the internet with other systems by way of the signal.

Finally, it should be further noted that the terms "comprises" and "comprising," when used herein, include not only those elements but also other elements not expressly listed or inherent to such processes, methods, articles, or devices. Without further limitation, an element defined by the phrases "comprising 8230; \8230;" 8230; "does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

Fig. 1 shows a flowchart of an example of a speech noise reduction method according to an embodiment of the present invention. The implementation subject of the method according to the embodiment of the present invention may be a terminal (e.g., a mobile phone, a smart speaker, etc.) having a voice capture function and a processing capability, and shall not be limited herein. In some service scenes, the method provided by the embodiment of the invention can be applied to wireless leading wire microphones to realize real-time noise reduction processing on the acquired voice data. In the form of application products, the method can be integrated on a chip of a hardware product; in addition, the function of noise reduction of the audio frequency of the wireless microphone can be realized through a mobile phone APP or computer software, the power consumption and hardware resource requirements are lower because a voice algorithm does not run on a product chip, but the noise reduction is realized by combining software, so that the learning and use cost of a user is increased.

As shown in fig. 1, in step 110, raw speech data is acquired. Wherein, the surrounding audio is collected based on the audio collector to determine the original voice data.

In step 120, pure speech data corresponding to the original speech data is predicted. Here, the pure voice data corresponding to the original voice data may be determined using a preset prediction algorithm.

In step 130, the raw speech data and the clean speech data are compared to determine a noise characteristic parameter. Specifically, the original speech data is compared with the pure speech data, and the noise characteristic parameter is obtained based on the difference information, for example, the intensity and distribution of noise can be determined according to the difference information, steady-state noise and transient noise can be identified, and the like.

In step 140, a noise reduction strategy matching the noise characteristic parameters is determined. Illustratively, a targeted noise reduction strategy can be adopted according to the characteristics of the noise type, the noise intensity, the noise distribution and the like.

In step 150, the original speech data is denoised based on the denoising strategy.

According to the embodiment of the invention, corresponding pure voice data is predicted aiming at the original voice data, and the noise characteristics are obtained through audio data comparison, so that a targeted noise reduction strategy is realized, and the noise reduction effect is improved.

It should be noted that, in the related art, some noise reduction strategies are implemented by recognizing the environment (e.g., an office or a bus, etc.) where the voice is located, and applying corresponding noise reduction strategies based on the environment scene. However, such noise reduction strategies have the obvious disadvantage that the noise characteristics in certain environmental scenarios are not constant, for example, the voice noise in the "driving situation" of the public transportation environment is of a light noise type, while the voice noise in the "getting on/off situation" of the public transportation environment is of a heavy noise type, and the noise reduction strategies adapted to the environment result in the disadvantage that the noise reduction effect cannot be stabilized.

In contrast, in the embodiment of the invention, the original voice data corresponding to the preset acquisition time is analyzed, and the corresponding noise reduction strategy is determined aiming at the original voice data, so that the aim of matching the noise reduction strategy in real time can be achieved, and the constant voice noise reduction effect is ensured.

In some examples of the embodiments of the present invention, the noise reduction operation as in steps 110-150 in fig. 1 is performed for each piece of original speech data input in real time, so that the noise reduction strategy is adjusted in real time to achieve the real-time noise reduction effect, but at the same time, a large resource consumption is also caused. In another example of the embodiment of the present invention, based on the noise reduction policy, noise reduction processing is performed on the original voice data corresponding to the subsequent preset time period, so as to reduce system resource consumption of the voice device during voice noise reduction. In addition, the noise reduction strategy is also configured to be updated according to a preset period, so that the requirement of system resource configuration can be reduced while the requirement of voice noise reduction is met, and the method is applicable to wider hardware and scenes.

As to the implementation details of step 140, on one hand, the noise level corresponding to the noise characteristic parameter may be determined, and then the noise reduction strategy having the noise reduction depth corresponding to the noise level may be determined. Specifically, comprehensive evaluation can be performed according to dimensions such as the category and the intensity of the noise, so as to obtain a corresponding level of the noise, so as to implement a noise reduction strategy of a corresponding noise reduction depth. Illustratively, the system is configured with a plurality of noise reduction algorithms corresponding to different depths, e.g., high fidelity, light noise reduction, medium noise reduction, and deep noise reduction, the light noise reduction algorithm being implemented when fewer noise components are detected in the raw speech data, the medium noise reduction algorithm being implemented when the noise in the raw speech data is detected to be primarily stationary noise, and the deep noise reduction algorithm being implemented when more types of noise (e.g., including stationary and transient noise) and greater intensity are detected in the raw speech data. Thus, the depth of the noise is judged, and customized depth noise reduction is realized.

On the other hand, the voice characteristic parameters corresponding to the pure voice data can be determined, and then the noise reduction strategy is determined based on the voice characteristic parameters and the noise characteristic parameters. In the example of this embodiment, when performing a voice noise reduction operation, the voice characteristic parameter and the noise characteristic parameter in the original voice data are considered comprehensively, so that voice information is not weakened while voice noise reduction is performed, and a better noise reduction effect is ensured. In some embodiments, the human voice characteristic parameter includes human voice spectral characteristic information, and the noise characteristic parameter includes noise spectral characteristic information. For example, when there is an overlapping spectrum interval between the human voice spectrum feature information and the noise spectrum feature information, the noise reduction depth of the noise for the overlapping spectrum interval should be reduced as much as possible to achieve the preservation of the human voice information.

With respect to the implementation details of step 120, in some examples of the embodiment of the present invention, pure speech data corresponding to the original speech data may be predicted based on a Very Deep Convolutional Neural Network (VDCNN). Here, the neural network model has noisy speech data as input data and clean speech data as output data.

Fig. 2 shows a flowchart of an example of training a VDCNN according to an embodiment of the present invention.

As shown in fig. 2, in step 210, the pure human voice and the pure noise audio are overlapped and mixed to obtain a noisy audio.

In step 220, frequency domain feature extraction is performed on the obtained noisy audio and the pure voice, respectively, to obtain corresponding feature vectors.

In step 230, the ultra-deep convolutional neural network is trained based on the obtained training set, and the difference mapping between the noisy audio and the clean voice is learned to determine the corresponding ultra-deep convolutional neural network.

Fig. 3 shows a schematic diagram of a training principle of the VDCNN according to an embodiment of the present invention. In the embodiment of the invention, the training of difference judgment and accurate identification of the noisy audio and the pure voice is integrated into a network model, so that dynamic judgment and effective noise reduction of a noise reduction strategy are realized. During training, the corresponding pure voice is subjected to feature extraction to obtain a group of corresponding vectors which are used as labels of clean voice. A Multi-Target Loss is used to learn the detailed information between pure human voice and noisy frequencies and zero-filling before each convolutional layer to keep the size of all eigenvectors the same. Therefore, the ultra-deep convolutional neural network (VDCNN) is trained on the basis of frequency domain information of the audio set, and a detection/noise reduction model with strong generalization and excellent performance is obtained.

Fig. 4 shows a flowchart of an example of a speech noise reduction method according to an embodiment of the present invention.

As shown in fig. 4, in step 410, the decision link of the noise reduction strategy is performed.

Specifically, the noise of the product at present is detected, namely, the input audio is pre-judged through a Very Deep Convolutional Neural Network (VDCNN) to obtain an expected clean voice, and a noise reduction strategy is judged by comparing the difference between the expected clean voice and the input audio, so that dynamic decision is realized, high universality is ensured, and the product is adaptive to various scenes of life;

in step 420, a smart noise reduction procedure is performed.

Specifically, the human voice and the noise are distinguished through a noise reduction model taking a Very Deep Convolutional Neural Network (VDCNN) as a root, steady-state and transient noises are effectively suppressed, the human voice is reserved and appropriately enhanced, and the expected clean human voice of the noise reduction model is obtained.

In step 430, the gain element is adapted.

Specifically, the sound intensity level corresponding to the voice data subjected to noise reduction is determined, and then audio gain adjustment processing is performed on the voice data subjected to noise reduction according to the sound intensity level and a preset level threshold. For example, when the sound intensity of the voice data is too low, the audio gain amplification operation is performed, and when the sound intensity of the voice data is too high, the audio gain reduction operation is performed, thereby ensuring that the output audio is in a reasonable sound intensity interval.

It should be noted that, due to the repetition of epidemic situations in recent years, the self-media is quickly burned out, and the self-media is used as a collar-clip wheat for the basic people, but the practical experience is that the use feeling is not good especially outdoors, and if the noise is too large, the later period is needed basically. If worn far from the person's mouth, the pick-up volume may be very small.

In some service application scenarios, the voice noise reduction method of the embodiment of the present invention may be applied to a wireless lapel microphone, and while the voice noise reduction effect is ensured, the dynamic adjustment of the voice volume is realized through an Adaptive Gain Control (AGC) technology, and the voice volume is not broken when the voice volume is too close to the mouth of a person, and the voice volume is not too small when the voice volume is too far away from the mouth of a person.

For the wireless leading wire wheat, considering the multiple outdoor use scenes and different wearing habits of users, the problems are solved by a deep learning noise reduction model and an adaptive gain algorithm. In practical application, the noise of the product at present is detected, namely, the input audio is pre-judged through a Very Deep Convolutional Neural Network (VDCNN) to obtain an expected clean voice, and a noise reduction strategy is judged by comparing the difference between the expected clean voice and the input audio, so that dynamic decision is realized, high universality is ensured, and the product is adaptive to various scenes of life. And distinguishing human voice and noise by using a noise reduction model taking the VDCNN as a root, effectively inhibiting steady-state and transient noises, reserving and properly enhancing the human voice, and obtaining the expected clean human voice of the noise reduction model. The dynamic adjustment of the human voice volume is realized through the self-adaptive AGC technology, and the human voice volume cannot be broken when being too close to the human mouth and cannot be too small when being too far away.

Further, the noise reduction performance of the wireless collar clamp microphone is subjected to simulation test. Fig. 5A is a simulation diagram illustrating the noise reduction performance effect of the wireless neckline microphone in the prior art. Fig. 5B is a simulation diagram illustrating the noise reduction performance effect of the wireless neckband microphone improved by the voice noise reduction method according to the embodiment of the present invention. The comparison of intuitive simulation data shows that the noise reduction performance of the voice data is obviously improved.

Fig. 6A is a simulation diagram illustrating the sound volume effect of a wireless neckline microphone in the prior art. Fig. 6B is a simulation diagram illustrating the sound volume adjustment effect of the wireless neckband microphone improved by the voice noise reduction method according to the embodiment of the present invention. As can be seen from comparison of visual simulation data, stable sound volume output can be realized.

By the embodiment of the invention, an intelligent noise reduction (including adaptive gain) scheme is implemented on the lapel microphone, so that the noise processing capability and the sound pickup stability of the product are enhanced. Furthermore, as the collarband microphone has more excellent processing on the noise in life and more stable pickup, the use scene of the collarband microphone is expanded, the volume is not required to be adjusted manually, and the use of a user is more convenient.

It should be noted that for simplicity of explanation, the foregoing method embodiments are described as a series of acts or combination of acts, but those skilled in the art will appreciate that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention. In the foregoing embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to the related descriptions of other embodiments.

In some embodiments, the present invention provides a non-transitory computer readable storage medium, in which one or more programs including executable instructions are stored, and the executable instructions can be read and executed by an electronic device (including but not limited to a computer, a server, or a network device, etc.) to perform any of the above voice noise reduction methods of the present invention.

In some embodiments, the present invention further provides a computer program product comprising a computer program stored on a non-volatile computer-readable storage medium, the computer program comprising program instructions that, when executed by a computer, cause the computer to perform any of the above-described speech noise reduction methods.

In some embodiments, an embodiment of the present invention further provides an electronic device, which includes: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a voice noise reduction method.

Fig. 7 is a schematic diagram of a hardware structure of an electronic device for executing a speech noise reduction method according to another embodiment of the present invention, and as shown in fig. 7, the electronic device includes:

one or more processors 710 and a memory 720, one processor 710 being illustrated in fig. 7.

The apparatus for performing the voice noise reduction method may further include: an input device 730 and an output device 740.

The processor 710, the memory 720, the input device 730, and the output device 740 may be connected by a bus or other means, as exemplified by the bus connection in fig. 7.

The memory 720, which is a non-volatile computer-readable storage medium, may be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the voice noise reduction method in embodiments of the present invention. The processor 710 executes various functional applications of the server and data processing by executing nonvolatile software programs, instructions and modules stored in the memory 720, namely, implements the voice noise reduction method of the above-mentioned method embodiment.

The memory 720 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to use of the voice interactive apparatus, and the like. Further, the memory 720 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 720 optionally includes memory located remotely from processor 710, which may be connected to the voice interaction device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input device 730 may receive input numeric or character information and generate signals related to user settings and function control of the voice interactive apparatus. The output device 740 may include a display device such as a display screen.

The one or more modules are stored in the memory 720 and, when executed by the one or more processors 710, perform the speech noise reduction method of any of the method embodiments described above.

The product can execute the method provided by the embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method. For technical details that are not described in detail in this embodiment, reference may be made to the method provided in the embodiment of the present invention.

Electronic devices of embodiments of the present invention exist in a variety of forms, including but not limited to:

(1) Mobile communication devices, which are characterized by mobile communication functions and are primarily targeted at providing voice and data communications. Such terminals include smart phones, multimedia phones, functional phones, and low-end phones, among others.

(2) The ultra-mobile personal computer equipment belongs to the category of personal computers, has the functions of calculation and processing, and generally has the mobile internet access characteristic. Such terminals include PDA, MID, and UMPC devices, among others.

(3) Portable entertainment devices such devices may display and play multimedia content. The devices comprise audio and video players, handheld game consoles, electronic books, intelligent toys and portable vehicle-mounted navigation devices.

(4) Other onboard electronic devices with data interaction functions, such as a vehicle-mounted device mounted on a vehicle.

The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.

Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a general hardware platform, and certainly can also be implemented by hardware. Based on such understanding, the above technical solutions substantially or contributing to the related art may be embodied in the form of a software product, which may be stored in a computer-readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments.

Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, and not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of speech noise reduction, comprising:

acquiring original voice data;

predicting pure voice data corresponding to the original voice data;

comparing the raw speech data and the clean speech data to determine a noise characteristic parameter;

determining a noise reduction strategy matched with the noise characteristic parameters;

and performing noise reduction processing on the original voice data based on the noise reduction strategy.

2. The method of claim 1, wherein said predicting clean speech data corresponding to said original speech data comprises:

and predicting pure voice data corresponding to the original voice data based on the ultra-deep convolutional neural network.

3. The method of claim 1, wherein the determining a noise reduction strategy that matches the noise characteristic parameter comprises:

determining voice characteristic parameters corresponding to the pure voice data;

and determining a noise reduction strategy based on the human voice characteristic parameters and the noise characteristic parameters.

4. The method of claim 1, wherein the determining a noise reduction strategy that matches the noise characteristic parameter comprises:

determining a noise level corresponding to the noise characteristic parameter;

determining a noise reduction strategy having a noise reduction depth corresponding to the noise level.

5. The method of claim 3, wherein the vocal feature parameters comprise vocal spectral feature information and the noise feature parameters comprise noise spectral feature information.

6. The method of claim 1, wherein after denoising the raw speech data based on the denoising strategy, the method further comprises:

determining the sound intensity grade corresponding to the voice data after noise reduction;

and performing audio gain adjustment processing on the voice data subjected to noise reduction according to the sound intensity level and a preset level threshold.

7. The method of claim 1, wherein after determining a noise reduction strategy that matches the noise characteristic parameter, the method further comprises:

and based on the noise reduction strategy, carrying out noise reduction processing on the subsequent original voice data corresponding to the preset time period.

8. The method of claim 7, wherein the noise reduction strategy is further configured to be updated according to a preset period.

9. An electronic device, comprising: at least one processor, and a memory communicatively coupled to the at least one processor, wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the method of any of claims 1-8.

10. A storage medium on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.

Images