KR102503895B1 - Audio signal processing method and appratus - Google Patents

Abstract

A method and apparatus for processing a sound signal including a voice signal and noise. A sound signal processing method according to an embodiment includes obtaining an input sound signal including a voice signal and noise, generating a first signal by removing noise from the input sound signal, and inputting the noise through a user interface related to noise control. and obtaining a parameter for controlling the strength of the noise, and generating an output acoustic signal by performing a weighted sum of the input acoustic signal and the first signal based on the parameter.

Description

Sound signal processing method and apparatus {AUDIO SIGNAL PROCESSING METHOD AND APPRATUS}

It relates to a method and apparatus for processing an acoustic signal. More specifically, it relates to a method and apparatus for processing a voice signal and an acoustic signal including noise.

Noise cancellation technology is a technology that removes various unintended distortions of an input signal, including interference of other signals, from an acoustic signal. For example, when a voice signal input through a microphone includes noise due to noise generated in the surrounding environment, noise other than the voice may be removed through a background noise removal technique. That is, the noise cancellation technology is a kind of voice preprocessing technology that excludes or mitigates the influence of unwanted signals input to the microphone. Noise cancellation technology can be used to improve call quality by removing background noise that interferes with communication during a call, and can also be used to increase the efficiency of voice recognition in a voice recognition system.

A lot of research has been conducted for noise removal, and traditional noise removal algorithms have been developed using statistical techniques, but recently many algorithms using deep learning have been developed. In the case of using a deep learning model trained to output a denoised signal, it is necessary to have all the weights of several models in order to adjust the strength of denoising, which requires excessive software capacity for storing the weights of the various models. , there is a problem of long latency as memory is allocated to many weight values.

It is possible to provide a voice signal processing technology that controls the degree of noise cancellation according to the strength of the set background noise, and to provide a technology that allows the user to adjust the degree of noise cancellation through a user interface in an environment where a voice signal is received. there is.

By adjusting the synthesis ratio of the noise-removed voice signal and the input voice signal in the section with and without audio, the output signal is synthesized with a natural connection between the section with and without audio. technology can be provided.

It is possible to provide a technique for determining a situation requiring noise removal in a call environment and inducing a user to appropriately control background noise removal through an interface in a situation requiring noise removal.

However, the technical challenges are not limited to the above-described technical challenges, and other technical challenges may exist.

A sound signal processing method performed by a processor according to one aspect includes obtaining an input sound signal including a voice signal and noise; generating a first signal by removing the noise from the input sound signal; obtaining a parameter input through a user interface related to noise control and controlling the intensity of the noise; and generating an output acoustic signal by performing a weighted sum of the input acoustic signal and the first signal based on the parameter.

The sound signal processing method may include obtaining a signal-to-noise ratio (SNR) value of the input sound signal; and determining whether to activate the user interface based on the SNR value.

The generating of the output acoustic signal may include obtaining a speech presence probability corresponding to each section of the first signal; and performing a weighted sum of the input acoustic signal and the first signal for each section based on the voice presence probability and the parameter.

The weighted summing may include performing a weighted sum by assigning a higher weight to the input sound signal than the first signal, based on the parameter, in a section in which the voice presence probability is high; and performing a weighted sum by assigning a higher weight to the first signal than to the input sound signal, based on the parameter, in a section in which the voice presence probability is low.

The weighted summing step generates a second signal having a high proportion of the input acoustic signal and a third signal having a high proportion of the first signal by weighting the input acoustic signal and the first signal based on the parameter. step; and performing a weighted sum of the second signal and the third signal for each section based on the voice presence probability.

In the obtaining of the voice presence probability, the voice signal in the corresponding frame corresponds to each of the frames generated by dividing the first signal into specific time units based on a voice activity detection algorithm. It may include obtaining a probability of being included.

The generating of the first signal may include generating the first signal by performing a noise removal algorithm in the time domain of the input sound signal.

The generating of the first signal may include generating the first signal based on a deep learning model trained to output a noise-removed acoustic signal from an acoustic signal including noise.

An acoustic signal processing method according to one aspect includes generating a first signal by removing noise from an input acoustic signal including a voice signal and noise; obtaining a speech presence probability in response to the first signal; obtaining a parameter related to the degree of noise removal; and controlling a degree of removal of the noise by combining the input sound signal and the first signal based on at least one of a probability that the voice exists and the parameter.

The controlling may include: controlling a degree of removal of the noise by weighting and summing the input sound signal with a higher weight than the first signal based on the parameter in a section in which the sound is likely to exist; and controlling a degree of removal of the noise by weighting the first signal with a higher weight than the input sound signal based on the parameter in a section in which the probability of the presence of the voice is low.

The controlling may include dividing the first signal into a voice section and a non-voice section by applying a voice activity detection algorithm to the first signal; weighting the first signal corresponding to the speech section and the input sound signal corresponding to the speech section by assigning a higher weight to the input sound signal than the first signal based on the parameter; and weighting the first signal corresponding to the non-voice section and the input audio signal corresponding to the non-voice section by placing a higher weight on the first signal than the input audio signal based on the parameter. can

An acoustic signal processing apparatus according to one side obtains an input acoustic signal including a voice signal and noise, removes the noise from the input acoustic signal, generates a first signal, and inputs the first signal through a user interface related to noise control. obtaining a parameter for controlling the intensity of the noise, and generating an output acoustic signal by weighting the input acoustic signal and the first signal based on the parameter;

It includes at least one processor.

The processor may obtain a signal-to-noise ratio (SNR) value of the input sound signal, and determine whether to activate the user interface based on the SNR value.

In generating the output acoustic signal, the processor obtains a speech presence probability corresponding to each section of the first signal, and based on the speech presence probability and the parameters, the input acoustic signal and a weighted sum of the first signal for each section.

In the weighted sum, the processor performs a weighted sum by assigning a higher weight to the input sound signal than the first signal, based on the parameter, in a section in which the voice presence probability is high, and in a section in which the voice presence probability is low, Based on the parameter, a weighted sum may be performed by placing a higher weight on the first signal than on the input sound signal.

In the weighted sum, the processor weights the input sound signal and the first signal based on the parameter, so that a second signal having a high weight ratio of the input sound signal and a third signal having a high weight ratio of the first signal A signal may be generated, and based on the voice presence probability, the second signal and the third signal may be weighted summed for each section.

In generating the first signal, the processor may generate the first signal by performing a noise removal algorithm in the time domain of the input sound signal.

When generating the first signal, the processor may generate the first signal based on a deep learning model learned to output an acoustic signal from which noise is removed from an acoustic signal including noise.

The sound signal processing apparatus according to one side generates a first signal by removing noise from an input sound signal including a speech signal and noise, and in response to the first signal, a speech presence probability Obtaining a parameter related to the degree of noise removal, and synthesizing the input sound signal and the first signal based on at least one of the probability that the voice exists and the parameter, thereby determining the degree of noise removal. It includes at least one processor that controls.

By providing a user interface capable of controlling the intensity of background noise in an environment where a voice signal is received, the user can control the degree of noise removal of the voice signal in real time while receiving the voice signal, and in a situation where noise removal is required, the user can Intuitively, an effect that can appropriately control background noise removal can be derived.

A noise canceling signal with improved quality may be output by synthesizing an output signal in which a connection between a section including audio and a section not including audio is natural.

1 is an exemplary view of a voice signal processing system according to an embodiment;
2a and 2b are exemplary diagrams of a noise control interface configuration provided to a user's terminal.
3 is an exemplary view of a noise control interface screen provided during a group video call;
4 is an exemplary view of a voice signal processing system according to an embodiment;
5 and 6 are flowcharts illustrating a method for processing a sound signal according to an embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for illustrative purposes only, and may be changed and implemented in various forms. Therefore, the form actually implemented is not limited only to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical idea described in the embodiments.

Although terms such as first or second may be used to describe various components, such terms should only be construed for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element.

It should be understood that when an element is referred to as being “connected” to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but one or more other features or numbers, It should be understood that the presence or addition of steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in this specification, it should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same reference numerals are given to the same components regardless of reference numerals, and overlapping descriptions thereof will be omitted.

1 is an exemplary diagram of a voice signal processing system according to an embodiment.

Referring to FIG. 1 , a voice signal processing system 100 according to an embodiment may correspond to a system for controlling a degree of noise removal of a sound signal.

The system 100 according to an embodiment may include an apparatus 110 that receives an acoustic signal containing noise and generates an output acoustic signal having a controlled degree of noise removal according to a set noise intensity. In other words, when the noise intensity is set to be high, the device 110 generates an output acoustic signal with a low degree of noise removal from the input sound signal, and when the noise intensity is set to be low, the output sound signal with a high degree of noise cancellation is generated from the input sound signal. By generating, it is possible to control the degree of noise removal. The device 110 is a device that performs a sound signal processing method according to an embodiment, and may correspond to a device that generates an output sound signal by processing an input sound signal according to the sound signal processing method according to an embodiment. . The device 110 may include at least one processor, memory, and input/output device, and may correspond to, for example, a server or a user's device (eg, a mobile phone, a computer, etc.).

The memory constituting the device 110 according to an embodiment may be a volatile memory or a non-volatile memory, and may store information related to a sound signal processing method or a program in which the sound signal processing method is implemented.

At least one processor constituting the device 110 according to an embodiment may perform a sound signal processing method. A processor constituting the device 110 may execute a program and control the device 110 . Program codes executed by the processor may be stored in a memory constituting the device 110 .

The device 110 according to an embodiment is connected to an external device (eg, a personal computer or network) through an input/output device, can exchange data, receive input data from a user, or output data to a user. can provide

Referring to FIG. 1 , an apparatus 110 may include a noise removal unit 111 and a noise control unit 112 . Each component of the audio signal processing apparatus 110 may correspond to a module, and each component may perform a process for processing a sound signal. The configuration of the device 110 shown in FIG. 1 is divided to describe the acoustic signal processing method according to an embodiment in functional units, and the configuration of the device is not limited. As described above, the acoustic signal processing method according to an embodiment may be performed by at least one processor constituting the device 110 .

The noise removal unit 111 according to an embodiment may correspond to a module that receives a voice signal and an acoustic signal 101 including noise and outputs a noise-removed signal. Noise included in the input sound signal 101 may include background noise or background noise generated when noise generated in the surrounding environment is received through the input device, in addition to the voice signal. The noise removal unit 111 may perform a process of removing noise from the input sound signal 101 using a noise removal algorithm. The noise removal algorithm may include various algorithms for removing noise from sound signals, such as an algorithm for removing noise using a statistical technique and an algorithm for removing noise using a deep learning model.

According to one embodiment, the noise removal algorithm may include an algorithm performed in the time domain of the acoustic signal. In other words, an algorithm for removing noise from an acoustic signal expressed in the time domain may be included. A noise removal algorithm performed in the time domain of the acoustic signal may be distinguished from a noise removal algorithm performed in the frequency domain of the audio signal.

For example, the noise removal algorithm is a one-dimensional convolution ( convolution) to estimate the denoised signal. Here, the deep learning model learning method may include a process of mapping a result of applying a one-dimensional convolution to an acoustic signal including noise expressed in the time domain into an acoustic signal from which noise is removed. Hereinafter, an acoustic signal from which noise is removed by a noise removal algorithm from the acoustic signal 101 including noise and voice signals input to the system may be referred to as a first signal.

The noise controller 112 according to an embodiment synthesizes the first signal from which noise is removed from the input sound signal 101 and the input sound signal 101 to generate an output sound signal 102 having a controlled degree of noise removal. may correspond to a module. The noise control unit 112 may perform a process of generating the output acoustic signal 102 having a controlled degree of noise removal using a noise control algorithm. The degree of noise removal may be controlled based on input through the noise control interface 120 .

According to one embodiment, the noise control algorithm for generating the output acoustic signal 102 may include a weighted sum of the first signal and the input acoustic signal 101 . A weight corresponding to the first signal for the weighted sum and a weight corresponding to the input acoustic signal 101 may be determined based on a parameter for controlling the intensity of noise acquired through the noise control interface 120. According to an embodiment, a parameter for controlling the intensity of noise may be input from a user terminal through the noise control interface 120 .

For example, the output acoustic signal 102 may be expressed as Equation 1 below.

Here, x (t), y (t) and z (t) are acoustic signals expressed in the time domain, and represent the acoustic signals as a function of time (t), respectively the input acoustic signal, the first signal and the output corresponds to the signal. α is a parameter related to the intensity of noise input from the user terminal through the noise control interface 120, and may have a real value of 0 or more and 1 or less.

According to an embodiment, the parameter α related to the intensity of noise is determined as a weight of the first signal, and (1-α) is determined as a weight of the input sound signal 101, and the first signal and the input sound are determined according to the determined weight. An output acoustic signal 102 may be generated by a weighted sum of the signals 101 .

According to an embodiment, the weighted sum of the input sound signal and the first signal may be performed for each section or frame. A frame means a section in which a signal expressed in the time domain is divided into predetermined time units. The weighted sum of the input sound signal and the first signal for each frame is a weighted sum of the first frame of the input sound signal and the first frame of the first signal according to weights, wherein the first frame of the sound signal and the first frame of the first signal A frame may correspond to the same time interval. For example, when the size of the frame is 1s, the input sound signal of the 0-1s frame and the first signal of the 0-1s frame are weighted and the input sound signal of the 1-2s frame and the first signal of the 1-2s frame It is possible to weight-sum all of the input sound signals and all of the first signals in a weighted-sum manner. The weight of the input sound signal and the weight of the first signal for the weighted sum may be different for each frame. For example, when the size of the frame is 1s, the input sound signal x(0) of the 0~1s frame and the first signal y(0) of the 0~1s frame are α ₀ x(0)+(1-α ₀ ) y (0), and the input sound signal of the 1 to 2 s frame and the first signal of the 1 to 2 s frame are α ₁ x (1) + (1-α ₁ ) y (1) It can be a weighted sum, and α ₀ and α ₁ can be determined independently of each other.

According to an embodiment, the noise controller 112 may generate the output sound signal 102 in consideration of the speech presence probability. This will be described in detail in FIG. 4 below.

System 100 according to one embodiment may include a noise control interface 120 . The noise control interface 120 may include a user interface (UI) that supports interaction between a user and the system 100 for noise control. The noise control interface 120 may be mounted on a device different from the device 110 performing the acoustic signal processing method and connected to the device 110 through a network to transmit/receive data. For example, the device 110 corresponds to a server related to sound signal processing, and the noise control interface 120 is mounted on a user's terminal and transmits data received from the user's terminal to the device 110 through a network. , Data may be received from the device 110 through the network and displayed on the user terminal.

Unlike what is shown in FIG. 1 , the noise control interface 120 according to an embodiment may be mounted on the device 110 that performs a sound signal processing method. For example, the device 110 may correspond to a user terminal and the noise control interface 120 may correspond to a user interface provided to the user terminal.

The acoustic signal processing apparatus 110 according to an embodiment may obtain an input related to noise control input through the noise control interface 120 . The input related to noise control may include an input for controlling the intensity of noise. For example, a user may input a parameter for controlling the intensity of noise through the interface 120, and the acoustic signal processing apparatus 110 may obtain the input parameter through a network.

The intensity of noise means the size of noise included in the output sound signal 102, and the degree to which noise is removed from the input sound signal 101 may be determined according to the setting of the noise intensity. For example, when the noise intensity of the noise is set to be small, it means that the output sound signal contains less noise, so the degree of noise removal can be set to be larger. On the other hand, when the noise intensity is set high, the degree of noise removal may be set small.

For example, a configuration of a noise control interface provided to a user's terminal may refer to FIGS. 2A and 2B. Referring to FIG. 2A , the user may set the intensity of noise included in the output sound signal 102 by changing the position of the displayed bar 201 within the reference line 202 through a provided noise control interface. As the bar 201 is positioned to the left of the reference line 202, it indicates a small noise intensity, and as it is positioned to the right, it may indicate a large noise intensity. Referring to FIGS. 2A and 2B, since the bar 211 shown in FIG. 2B is located to the right of the reference line than the bar 201 shown in FIG. 2A, the intensity of noise is set higher in FIG. 2B than in FIG. 2A. The noise control interface according to an embodiment may receive an input for changing the position of the bar 201 from a user, parameterize a point in the reference line 202 at which the bar 201 is located, and obtain a parameter related to noise intensity. there is.

The configuration of the interface shown in FIGS. 2A and 2B is an example of the configuration of the noise control interface 120 according to an embodiment. In addition, the noise control interface 120 may include an interface for inputting the strength of noise as a numerical value. and may include an interface supporting various inputs such as touch input through a touch screen, key input through a keyboard, and click input through a mouse.

The user can transmit an input related to noise control to the system by adjusting the intensity of noise included in the output acoustic signal 102 through the noise control interface 120, and the system can send an input related to the noise control received through the interface. The output acoustic signal 102 may be generated by adjusting the degree of noise removal based on the input.

According to one embodiment, the noise control interface 120 may be provided in an environment that receives a voice signal. For example, the noise control interface 120 may be provided in an environment where a voice signal of the other party is received during a voice call or a video call.

For example, FIG. 3 is a diagram illustrating a noise control interface provided during a group video call. Referring to FIG. 3 , when a group video call is conducted through a user terminal, a noise control interface may be provided in an environment for receiving video and audio transmitted from a terminal of a call counterpart. In the case of a group video call, a noise control interface may be provided in response to voice signals of each of a plurality of call parties, and the user may use the noise control interface displayed at the bottom of the interface where the video of the call party is displayed to correspond to the video of the call party. It is possible to control the noise intensity of the voice signal received from The user may control the noise intensity of the voice signal received from the call counterpart through an interface corresponding to each of the plurality of call counterparts, thereby receiving the counterpart's voice signal with the adjusted noise removal degree. In addition, since noise intensities corresponding to a plurality of counterparts may be set differently, the degree of noise removal may be differently adjusted according to the set noise intensities, and the sound signal may be output.

Referring back to FIG. 1 , activation of the noise control interface 120 according to an embodiment may be determined based on a signal-to-noise ratio (SNR) of the input acoustic signal 101.

SNR (signal-to-noise ratio) represents the ratio of the received signal to the noise signal. When the strength of the received signal is V _s and the strength of noise is V _n , alog ₍ V _s /V _n ) ( dB), where a is a positive constant. The higher the SNR, the greater the signal is greater than the noise. For example, when the SNR of the voice signal is 20 dB rather than 7 dB, the voice is heard louder and the noise is small. If the SNR is negative, such as 10 dB, it means that noise is greater than speech.

The device 110 according to an embodiment may obtain an SNR value of the input sound signal 101 and determine whether to activate the user interface 120 based on the SNR value. For example, when the SNR value is less than a predetermined reference value, it may be determined that the input acoustic signal 101 contains a lot of noise, and it may be determined that the noise control interface 120 is activated.

According to an embodiment, when it is determined to activate the interface 120, the interface 120 is activated, and the device 110 may perform a noise removal algorithm and a noise control algorithm.

Referring back to FIG. 3 , in the case of a group video call, a noise control interface may be provided corresponding to each voice signal of a plurality of call parties, and the SNR value of each input sound signal received from the plurality of call parties Based on this, it may be determined whether to activate the noise control interface. Whether or not the noise control interface is activated may be displayed on the interface in various ways. For example, whether or not the noise control interface is activated may be indicated by a difference in color between the figures 301 and 302 and a color difference between the figures 303 and 304 displayed on the interface. When the noise control interface is activated, the user can move the position of the bar 305 representing the noise intensity to input a parameter for controlling the noise intensity through the input device, whereas when the noise control interface is not activated, The user cannot move the position of the bar 306 representing the noise intensity through the input device.

4 is an exemplary diagram of a voice signal processing system according to an embodiment.

Referring to FIG. 4 , the apparatus for performing the acoustic signal processing method according to an exemplary embodiment may further include a voice activity detection unit configured to obtain a voice presence probability. The configuration of the device shown in FIG. 4 is divided to describe the acoustic signal processing method according to an embodiment in functional units, like in FIG. 1 , and the configuration of the device is not limited.

The voice activity detection unit 410 according to an embodiment may correspond to a module that obtains a voice presence probability of a first signal from which noise is removed from an input acoustic signal. The voice activity detection unit 410 may perform a process of obtaining a voice presence probability of the first signal by using a voice activity detection (VAD) algorithm. The voice activity detection algorithm is an algorithm for determining a voice section and a silent section (or non-voice section) in an acoustic signal, and may include various algorithms for detecting voice activity. As a result of the voice activity detection algorithm, a voice presence probability, which is a probability that an acoustic signal used as an input of the algorithm is a voice signal, may be output.

The voice activity detection unit 410 according to an embodiment may obtain a voice presence probability for each frame of the first signal according to a voice activity detection algorithm.

The noise controller 420 according to an exemplary embodiment may generate an output audio signal by performing a weighted sum of the input audio signal and the first signal based on a parameter for controlling a voice presence probability and noise intensity.

The noise control unit 420 according to an embodiment weights the input sound signal and the first signal based on a parameter for controlling the noise intensity, thereby obtaining a second signal having a high ratio of the input sound signal and a high ratio of the first signal. An output acoustic signal may be generated by generating a third signal and performing a weighted sum of the second signal and the third signal based on a voice presence probability of the first signal.

For example, when the parameter controlling the intensity of noise input through the noise control interface is set to α ₁ (where α ₁ is an arbitrary real number greater than or equal to 0 and less than 0.5), the input sound signal x (t ) and the first signal y(t), the second signal z ₀ (t) having a high proportion of the input acoustic signal x(t) may be synthesized.

Here, since α ₁ is an arbitrary real number greater than or equal to 0 and less than 0.5, the weight (1−α ₁ ) of the input acoustic signal x(t) is greater than the weight α ₁ of the first signal y(t). Accordingly, a second signal z ₀ (t) having a higher proportion of the input acoustic signal x(t) than the first signal y(t) may be generated.

In addition, as shown in Equation 3 below, a third signal z ₁ (t) having a high proportion of the first signal y (t) is synthesized by weighted summing the input sound signal x (t) and the first signal y (t) can

Here, α ₂ is a parameter determined based on α ₁ input through the noise control interface, and may correspond to a parameter determined smaller as α ₁ increases. For example, α ₂ can be determined as (1−α ₁ ).

When α ₂ is determined as (1-α ₁ ), in response to α ₁ , which is an arbitrary real number greater than 0 and less than 0.5, since α ₂ is determined as a real number greater than 0.5 and less than 1, the input sound signal x(t) The weight (1-α ₂ ) of is smaller than the weight α ₂ of the first signal y(t). Accordingly, an output third signal z ₁ (t) having a higher proportion of the first signal y(t) than the input acoustic signal x(t) may be generated.

The output acoustic signal z(t) is generated by a weighted sum of the second signal z ₀ (t) and the third signal z ₁ (t) based on the speech presence probability p of the first signal as shown in Equation 4 below: It can be.

According to an embodiment, when the voice presence probability is high, the second signal having a high proportion of the input acoustic signal is more included in the output acoustic signal, and when the voice presence probability is small, the third signal having a high proportion of the first signal By synthesizing the first signal and the input acoustic signal so that more signals are included in the output acoustic signal, an output acoustic signal having a noise removal degree controlled for each section may be generated. A weight α ₁ for synthesizing the second signal and the third signal and By adjusting α ₂ , an output audio signal in which a part including the audio signal and a part not including the audio signal are naturally connected can be generated.

According to an embodiment, when the voice presence probability p of the first signal is obtained for each frame, the noise controller 420 determines the input sound signal and the first signal based on the voice presence probability and a parameter for controlling the noise intensity. An output acoustic signal may be generated by performing a weighted sum for each frame. For example, when the audio presence probability in frame t ₁ of the first signal is p ₁ , the output acoustic signal z(t ₁ ) corresponding to frame t ₁ may be expressed as in Equation 5 below.

According to an embodiment, when the voice presence probability p of the first signal is obtained for each frame, the noise controller 420 determines whether each frame of the first signal is a frame with a high probability of voice presence or a frame with a low voice presence probability. By determining, it is possible to generate an output acoustic signal.

Whether the specific frame of the first signal is a frame with a high probability of voice presence or a frame with a low probability of voice presence may be determined by comparing the voice presence probability obtained in the specific frame of the first signal with a predetermined threshold value. For example, based on 0.5, if the audio presence probability of a specific frame of the first signal is greater than or equal to 0.5, a frame having a high audio presence probability may be determined, and if it is less than 0.5, a frame having a low audio presence probability may be determined. According to an embodiment, a frame with a high probability of voice existence may be divided into a voice section, and a frame with a low voice presence probability may be divided into a non-voice section. In other words, the frames of the first signal may be divided into a speech section and a non-voice section based on the speech existence probability for each frame of the first signal output by applying the speech utterance detection algorithm to the first signal.

Among the frames of the first signal, the noise control unit 420 performs a weighted sum by placing a higher weight on the input sound signal than on the first signal, based on a parameter for controlling the noise intensity, in a frame having a high probability of voice presence among the frames of the first signal. can Among the frames of the first signal, the noise control unit 420 may perform a weighted sum by assigning a higher weight to the first signal than to the input sound signal, based on a parameter for controlling the noise intensity, in a frame having a low voice presence probability.

For example, when the parameter controlling the intensity of noise input through the noise control interface is set to α ₁ (where α ₁ is an arbitrary real number between 0 and 0.5), frames having a high probability of voice presence among frames of the first signal In (t ₁ ), the first signal y(t ₁ ) and the input sound signal x(t ₁ ) may be synthesized as shown in Equation 6 below.

Here, since α ₁ is an arbitrary real number between 0 and 0.5, the weight (1−α ₁ ) of the input acoustic signal x(t ₁ ) is greater than or equal to the weight α ₁ of the first signal y(t ₁ ). Accordingly, in a frame having a high voice presence probability, an output sound signal z(t ₁ ) having a higher proportion of the input sound signal x(t ₁ ) than the first signal y(t ₁ ) may be generated.

On the other hand, in a frame having a low voice presence probability among frames of the first signal, the first signal y(t ₂ ) and the input sound signal x(t ₂ ) may be synthesized as shown in Equation 7 below.

Here, α ₂ is a parameter determined based on α ₁ input through the noise control interface, and may correspond to a parameter determined smaller as α ₁ increases. For example, α ₂ can be determined as (1−α ₁ ).

When α ₂ is determined as (1-α ₁ ), in response to α ₁ , which is an arbitrary real number between 0 and 0.5, since α ₂ is determined as a real number between 0.5 and 1, the input sound signal x(t ₂ ) has a weight (1-α ₂ ) that is less than or equal to the weight α ₂ of the first signal y(t ₂ ). Accordingly, in a frame having a low voice presence probability, an output acoustic signal z(t ₂ ) having a higher proportion of the first signal than the input acoustic signal may be generated.

According to an embodiment, by determining whether each frame of the first signal is a frame with a high probability of speech existence or a frame with a low probability of speech existence (or by determining whether it corresponds to a speech section or a non-voice section), output In the process of generating the sound signal, in the above-described Equation 4, p = 1 in the case of a frame with a high probability of audio presence and p = 0 in the case of a frame with a low probability of audio presence, the second signal and the third signal are weighted and output. It can be understood as generating an acoustic signal.

5 and 6 are flowcharts illustrating a method for processing a sound signal according to an exemplary embodiment.

More specifically, FIGS. 5 and 6 illustrate an operation process of an acoustic signal system in an embodiment in which the acoustic signal processing device is a server and a noise control interface is mounted in a user terminal.

Referring to FIG. 5 , the server may receive ( 510 ) an input sound signal input through an input device such as a microphone of a user terminal through a network. The server may acquire the first signal by performing the above-described noise removal algorithm on the received input sound signal (530). The user terminal may receive a noise control input from a user through a noise control interface and transmit the received noise control input to a server, and the server may acquire the noise control input received through the noise control interface (520). According to one embodiment, the operation of obtaining a noise control input and the operation of performing a noise removal algorithm may be performed in parallel.

Based on the obtained noise control input, the server may generate an output sound signal by performing a noise control algorithm of weighted summing the first signal and the input sound signal (540). The server may transmit the output audio signal to the user terminal through the network (550), and the user terminal receiving the output audio signal may output the output audio signal through an output device such as a speaker.

According to one embodiment, the input sound signal received from the server may correspond to a sound signal on which a noise removal algorithm is performed on the user terminal. In other words, the user terminal may perform a process of removing noise from the sound signal received through the input device.

Referring to FIG. 6 , the server receiving the input sound signal from the user terminal obtains an SNR value from the received input sound signal (610) prior to performing subsequent operations shown in FIG. You can decide whether to activate it or not. The SNR value may be calculated by a processor of the server or may be calculated from an external device and obtained by the server.

The server may activate the noise control interface of the user terminal (620) when the SNR value meets a predetermined criterion based on the obtained SNR value. The predetermined criterion may correspond to a criterion related to an SNR value for determining whether to perform a noise removal and control algorithm on an input sound signal. For example, by comparing the SNR value with a predetermined threshold value or threshold range, it may be determined to activate the noise control interface if the SNR value is greater than or less than the threshold value or if the SNR value is within the threshold range.

When the noise control interface is activated in the server, the noise control interface may be activated such that the noise control interface is displayed on the user terminal, and the user may transmit and receive data to and from the server through the noise control interface.

If the server determines to activate the noise control interface based on the SNR value, a noise cancellation algorithm may be performed. Also, since a noise control input can be obtained from a server through a noise control interface, a noise control algorithm can be performed.

On the other hand, if the SNR value does not correspond to the predetermined criterion, the server may not activate the noise control interface and the noise removal algorithm may not be performed. When the noise control interface is not activated in the user terminal, the user cannot input parameters related to noise control through the interface. The noise control algorithm is not performed in the server, and if parameters related to noise control input through the noise control interface are not received, the noise control algorithm is not performed. In this case, the sound signal input to the user terminal may be output through the output device without undergoing a separate process for noise removal and control in the server.

5 and 6 show a case in which a user terminal that transmits an input sound signal to a server and a user terminal that receives an output sound signal from a server are the same terminal, but according to an exemplary embodiment, a user terminal that transmits an input sound signal to a server A user terminal sending a message and a user terminal receiving an output sound signal from the server may correspond to different terminals. For example, in a video call or voice call environment, since an audio signal input through a specific user terminal is output through the terminal of the other party, the user terminal transmitting the input audio signal to the server through the network and the server through the network. User terminals receiving the output sound signal may correspond to different terminals.

The embodiments described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, the devices, methods and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA). array), programmable logic units (PLUs), microprocessors, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. The device can be commanded. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. , or may be permanently or temporarily embodied in a transmitted signal wave. Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in the art of computer software. may be Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

The hardware device described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on this. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

Claims (20)

In the acoustic signal processing method performed in the processor,
obtaining an input sound signal including a voice signal and noise;
generating a first signal by removing the noise from the input sound signal;
obtaining a speech presence probability in response to the first signal;
obtaining a parameter input through a user interface related to noise control and controlling the strength of the noise;
generating a second signal and a third signal having different synthesis ratios of the input acoustic signal and the first signal, based on the parameter; and
generating an output acoustic signal by weighted summing the second signal and the third signal based on the voice presence probability;
including,
Acoustic signal processing method.
According to claim 1,
obtaining a signal-to-noise ratio (SNR) value of the input sound signal; and
Based on the SNR value, determining whether to activate the user interface.
Including more,
Acoustic signal processing method.
According to claim 1,
The voice presence probability is obtained corresponding to each section of the first signal,
The output acoustic signal is generated by weighting the second signal and the third signal for each section based on the voice presence probability and the parameter obtained for each section of the first signal.
Acoustic signal processing method.
delete According to claim 1,
Generating the output acoustic signal
weighting the second signal and the third signal for each section by assigning a higher weight to the second signal than the third signal based on the voice presence probability when the voice presence probability is greater than or equal to a predetermined threshold; and
When the voice presence probability is less than a predetermined threshold, weighted summing the second signal and the third signal for each interval by assigning a higher weight to the third signal than the second signal based on the voice presence probability
including,
Acoustic signal processing method.
According to claim 1,
The step of obtaining the voice presence probability is
Corresponding to each of the frames generated by dividing the first signal into specific time units,
Obtaining a probability that the voice signal is included in a corresponding frame based on a voice activity detection algorithm
including,
Acoustic signal processing method.
According to claim 1,
Generating the first signal
generating the first signal by performing a noise removal algorithm in the time domain of the input sound signal;
including,
Acoustic signal processing method.
According to claim 1,
Generating the first signal
Generating the first signal based on a deep learning model learned to output an acoustic signal from which noise is removed from an acoustic signal including noise
including,
Acoustic signal processing method.
delete delete delete A computer program stored in a medium to be combined with hardware to execute the method of any one of claims 1 to 3 and 5 to 8.
Acquiring an input acoustic signal including a voice signal and noise;
removing the noise from the input sound signal to generate a first signal;
Corresponding to the first signal, obtaining a speech presence probability;
Obtaining a parameter input through a user interface related to noise control and controlling the intensity of the noise;
By weighting the input acoustic signal and the first signal based on the parameter, a second signal having a higher proportion of the input acoustic signal than the first signal and a first signal having a higher proportion of the first signal than the input acoustic signal 3 generate signals,
generating an output acoustic signal by weighted summing the second signal and the third signal based on the speech presence probability;
including at least one processor,
Acoustic signal processing device.
According to claim 13,
The processor
Obtaining a signal-to-noise ratio (SNR) value of the input sound signal;
Based on the SNR value, determining whether to activate the user interface,
Acoustic signal processing device.
According to claim 13,
The voice presence probability is obtained corresponding to each section of the first signal,
The second signal and the third signal are generated by weighting the input acoustic signal and the first signal for each section based on the voice presence probability and the parameter obtained for each section of the first signal.
Acoustic signal processing device.
delete According to claim 13,
The processor
In generating the output acoustic signal,
When the voice presence probability is greater than or equal to a predetermined threshold, a weighted sum of the second signal and the third signal is given for each section by assigning a higher weight to the second signal than the third signal based on the voice presence probability;
When the voice presence probability is less than a predetermined threshold, weighted summing the second signal and the third signal for each interval by placing a higher weight on the third signal than on the second signal based on the voice presence probability.
Acoustic signal processing device.
According to claim 13,
The processor
In generating the first signal,
Generating the first signal by performing a noise removal algorithm in the time domain of the input sound signal,
Acoustic signal processing device.
According to claim 13,
The processor
In generating the first signal,
Generating the first signal based on a deep learning model learned to output an acoustic signal from which noise is removed from an acoustic signal containing noise,
Acoustic signal processing device.




delete

Images