KR20220081129A - Audio signal processing method and appratus - Google Patents

Abstract

It relates to a method and apparatus for processing an acoustic signal including a voice signal and noise. An acoustic signal processing method according to an embodiment includes: obtaining an input acoustic signal including a voice signal and noise; generating a first signal by removing noise from the input acoustic signal; and obtaining a parameter controlling the intensity of the noise and generating an output acoustic signal by weighted summing the input acoustic signal and the first signal based on the parameter.

Description

Acoustic signal processing method and apparatus {AUDIO SIGNAL PROCESSING METHOD AND APPRATUS}

It relates to a method and apparatus for processing an acoustic signal. More particularly, it relates to a method and apparatus for processing a sound signal including a voice signal and noise.

Noise cancellation technology is a technology that removes various unintentional distortion of an input signal, including interference of other signals, from an acoustic signal. For example, when a voice signal input through a microphone includes noise due to noise generated in the surrounding environment, noise other than the voice may be removed through background noise removal technology. That is, the noise removal technology is a kind of voice pre-processing technology that excludes or mitigates the effects of unwanted signals input to the microphone. The noise removal technology may be used to increase call quality by removing background noise that interferes with communication during a call, and may be used to increase the efficiency of voice recognition in a voice recognition system.

Many studies have been conducted to remove noise, and traditional noise removal algorithms have been developed using statistical techniques, but recently, many algorithms using deep learning have been developed. When using a deep learning model trained to output a denoised signal, it is necessary to have all the weights of several models in order to control the strength of denoising, which requires excessive software capacity to store the weights of several models. , there is a problem in that latency increases as memory is allocated to many weight values.

It is possible to provide a voice signal processing technology that controls the degree of noise removal according to the set background noise intensity, and it is possible to provide a technology for controlling the degree of noise removal through the user interface in an environment in which the user receives a voice signal. have.

By adjusting the synthesis ratio of the noise-removed voice signal and the input voice signal in the section including the voice and the section without the voice, a natural output signal is synthesized by connecting the section with the voice and the section without the voice technology can be provided.

It is possible to provide a technology that determines a situation in which noise removal is required in a call environment, and induces a user to appropriately control background noise removal through an interface in a situation in which noise removal is required.

However, the technical tasks are not limited to the above-described technical tasks, and other technical tasks may exist.

An acoustic signal processing method performed by a processor according to one aspect includes: acquiring an input acoustic signal including a voice signal and noise; generating a first signal by removing the noise from the input sound signal; obtaining a parameter input through a user interface related to noise control and controlling the intensity of the noise; and generating an output acoustic signal by weighted summing the input acoustic signal and the first signal based on the parameter.

The sound signal processing method may include: obtaining a signal-to-noise ratio (SNR) value of the input sound signal; and determining whether to activate the user interface based on the SNR value.

The generating of the output sound signal may include: acquiring a speech presence probability corresponding to each section of the first signal; and weighted summing the input sound signal and the first signal for each section based on the voice presence probability and the parameter.

The weighted summing may include: weighted summing the input sound signal with a higher weight than the first signal based on the parameter in the section in which the voice presence probability is high; and weighted summing the first signal with a higher weight than the input sound signal, based on the parameter, in a section in which the voice presence probability is low.

In the weighted summing step, the input sound signal and the first signal are weighted and summed based on the parameter, thereby generating a second signal having a high proportion of the input sound signal and a third signal having a high proportion of the first signal. step; and weighted summing the second signal and the third signal for each section based on the voice presence probability.

The obtaining of the voice presence probability may include, in response to each of the frames generated by dividing the first signal into specific time units, based on a voice activity detection algorithm, that the voice signal is included in the corresponding frame. It may include obtaining a probability of being included.

The generating of the first signal may include generating the first signal by performing a noise removal algorithm in a time domain of the input sound signal.

The generating of the first signal may include generating the first signal based on a deep learning model trained to output an acoustic signal in which noise is removed from an acoustic signal including noise.

An acoustic signal processing method according to an aspect includes: generating a first signal by removing the noise from an input acoustic signal including a voice signal and noise; obtaining a speech presence probability in response to the first signal; obtaining a parameter related to the degree of removal of the noise; and controlling the degree of removal of the noise by synthesizing the input sound signal and the first signal based on at least one of a probability of the presence of the voice and the parameter.

The controlling may include: controlling the degree of noise removal by weighted summing the input sound signal with a higher weight than the first signal based on the parameter in a section in which the voice is likely to be present; and controlling the degree of removal of the noise by weighted summing the first signal with a higher weight than the input sound signal based on the parameter in a section where the probability of the presence of the voice is low.

The controlling may include applying a voice activity detection algorithm to the first signal to divide the first signal into a voice section and a non-voice section; weighted summing the first signal corresponding to the voice section and the input sound signal corresponding to the voice section by giving a higher weight to the input sound signal than the first signal, based on the parameter; and weighted summing the first signal corresponding to the non-voice section and the input sound signal corresponding to the non-voice section by giving a higher weight to the first signal than the input sound signal based on the parameter. can

A sound signal processing apparatus according to one side obtains an input sound signal including a voice signal and noise, removes the noise from the input sound signal, generates a first signal, and inputs the signal through a user interface related to noise control and obtaining a parameter controlling the intensity of the noise, and generating an output sound signal by weighted summing the input sound signal and the first signal based on the parameter;

at least one processor.

The processor may obtain a signal-to-noise ratio (SNR) value of the input sound signal, and determine whether to activate the user interface based on the SNR value.

In generating the output sound signal, the processor acquires a speech presence probability corresponding to each section of the first signal, and based on the speech presence probability and the parameter, the input sound signal and weighted summing the first signal for each section.

In the weighted summing, the processor performs weighted summing by giving a higher weight to the input sound signal than the first signal based on the parameter in a section in which the voice presence probability is high, and in a section in which the voice presence probability is low, Based on the parameter, the first signal may be weighted and summed with a higher weight than the input sound signal.

In the weighted summing, the processor performs weighted summing of the input sound signal and the first signal based on the parameter, whereby a second signal having a high proportion of the input sound signal and a third signal having a high proportion of the first signal A signal may be generated, and the second signal and the third signal may be weighted and summed for each section based on the voice presence probability.

When generating the first signal, the processor may generate the first signal by performing a noise removal algorithm in a time domain of the input sound signal.

When generating the first signal, the processor may generate the first signal based on a deep learning model trained to output an acoustic signal in which noise is removed from an acoustic signal including noise.

A sound signal processing apparatus according to an aspect removes the noise from an input sound signal including a voice signal and noise to generate a first signal, and a speech presence probability in response to the first signal is obtained, a parameter related to the degree of noise removal is obtained, and the degree of noise removal is determined by synthesizing the input sound signal and the first signal based on at least one of a probability of the presence of the voice and the parameter. controlling, including at least one processor.

By providing a user interface that can control the intensity of background noise in an environment in which a voice signal is received, the user can control the degree of noise reduction of the voice signal in real time while receiving the voice signal. An effect that can intuitively control appropriate background noise removal can be derived.

By synthesizing an output signal in which the connection between the section including the voice and the section without the voice is natural, a noise-removing signal of improved quality can be output.

1 is an exemplary diagram of a voice signal processing system according to an embodiment;
2A and 2B are exemplary diagrams of a configuration of a noise control interface provided to a user's terminal;
3 is an exemplary diagram of a noise control interface screen provided during a group video call;
4 is an exemplary diagram of a voice signal processing system according to an embodiment;
5 and 6 are diagrams illustrating a flow chart of a method for processing an acoustic signal according to an exemplary embodiment.

Specific structural or functional descriptions of the embodiments are disclosed for purposes of illustration only, and may be changed and implemented in various forms. Accordingly, the actual implementation form is not limited to the specific embodiments disclosed, and the scope of the present specification includes changes, equivalents, or substitutes included in the technical spirit described in the embodiments.

Although terms such as first or second may be used to describe various elements, these terms should be interpreted only for the purpose of distinguishing one element from another. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected to” another component, it may be directly connected or connected to the other component, but it should be understood that another component may exist in between.

The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, and includes one or more other features or numbers, It should be understood that the possibility of the presence or addition of steps, operations, components, parts or combinations thereof is not precluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be interpreted in an ideal or excessively formal meaning unless explicitly defined in the present specification. does not

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. In the description with reference to the accompanying drawings, the same components are assigned the same reference numerals regardless of the reference numerals, and overlapping descriptions thereof will be omitted.

1 is an exemplary diagram of a voice signal processing system according to an embodiment.

Referring to FIG. 1 , a voice signal processing system 100 according to an exemplary embodiment may correspond to a system for controlling the degree of noise removal of an acoustic signal.

The system 100 according to an embodiment may include an apparatus 110 that receives an acoustic signal including noise and generates an output acoustic signal in which a degree of noise removal is controlled according to a set noise intensity. In other words, when the noise intensity is set to be high, the device 110 generates an output sound signal having a low degree of noise removal from the input sound signal, and when the noise intensity is set to be low, the device 110 generates an output sound signal having a high degree of noise reduction from the input sound signal. By generating , the degree of noise removal can be controlled. The apparatus 110 is an apparatus for performing a method for processing a sound signal according to an embodiment, and may correspond to an apparatus for generating an output sound signal by processing an input sound signal according to the method for processing a sound signal according to an embodiment. . The apparatus 110 may include at least one processor, a memory, and an input/output device, and may correspond to, for example, a server or a user's device (eg, a mobile phone, a computer, etc.).

The memory constituting the device 110 according to an embodiment may be a volatile memory or a non-volatile memory, and may store information related to a sound signal processing method or a program in which the sound signal processing method is implemented.

At least one processor constituting the apparatus 110 according to an embodiment may perform a method of processing an acoustic signal. A processor constituting the device 110 may execute a program and control the device 110 . Codes of programs executed by the processor may be stored in a memory constituting the device 110 .

The device 110 according to an embodiment is connected to an external device (eg, a personal computer or a network) through an input/output device, can exchange data, receive input data from a user, or send output data to a user can provide

Referring to FIG. 1 , the device 110 may include a noise removing unit 111 and a noise controlling unit 112 . Each component of the voice signal processing apparatus 110 may correspond to a module, and each component may perform a process for processing an acoustic signal. The configuration of the device 110 shown in FIG. 1 is divided to describe the sound signal processing method according to an embodiment in functional units, and the configuration of the device is not limited thereto. As described above, the sound signal processing method according to an embodiment may be performed by at least one processor constituting the apparatus 110 .

The noise removing unit 111 according to an embodiment may correspond to a module that receives a voice signal and an acoustic signal 101 including noise and outputs a noise-removed signal. The noise included in the input sound signal 101 may include background noise or background noise generated when noise generated in the surrounding environment is received through the input device in addition to the voice signal. The noise removal unit 111 may perform a process of removing noise from the input sound signal 101 using a noise removal algorithm. The noise removal algorithm may include various algorithms for removing noise from an acoustic signal, such as an algorithm for removing noise using a statistical technique and an algorithm for removing noise using a deep learning model.

According to an embodiment, the noise removal algorithm may include an algorithm performed in a time domain of an acoustic signal. In other words, an algorithm for removing noise from an acoustic signal expressed in the time domain may be included. The noise removal algorithm performed in the time domain of the acoustic signal may be distinguished from the noise removal algorithm performed in the frequency domain of the acoustic signal.

For example, the noise removal algorithm uses a deep learning model trained to output a noise-removed acoustic signal from a noise-containing acoustic signal with a supervised learning technique, and a one-dimensional convolution ( convolution) and estimating the noise-removed signal. Here, the learning method of the deep learning model may include a process of mapping a result of applying one-dimensional convolution to an acoustic signal including noise expressed in the time domain to an acoustic signal from which noise has been removed. Hereinafter, the acoustic signal from which the noise is removed by the noise removal algorithm from the acoustic signal 101 including the noise and the voice signal input to the system may be referred to as a first signal.

The noise control unit 112 according to an embodiment generates an output sound signal 102 whose degree of noise removal is controlled by synthesizing a first signal from which noise has been removed from the input sound signal 101 and the input sound signal 101 . It may correspond to a module. The noise control unit 112 may perform a process of generating the output sound signal 102 whose degree of noise removal is controlled by using a noise control algorithm. The degree of noise removal may be controlled based on an input through the noise control interface 120 .

According to an embodiment, the noise control algorithm for generating the output acoustic signal 102 may include a method of weighted summing the first signal and the input acoustic signal 101 . A weight corresponding to the first signal for weighted summing and a weight corresponding to the input acoustic signal 101 may be determined based on a parameter controlling the intensity of noise obtained through the noise control interface 120 . According to an embodiment, a parameter for controlling the intensity of noise may be input from a user terminal through the noise control interface 120 .

For example, the output sound signal 102 may be expressed as in Equation 1 below.

Here, x(t), y(t), and z(t) are sound signals expressed in the time domain, which represent the sound signals as a function of time (t), respectively, an input sound signal, a first signal, and an output corresponds to the signal. α is a parameter related to the intensity of noise input from the user terminal through the noise control interface 120 and may have a real value of 0 or more and 1 or less.

According to an embodiment, the parameter α related to the intensity of the noise is determined as a weight of the first signal, and (1-α) is determined as the weight of the input sound signal 101, and according to the determined weight, the first signal and the input sound An output acoustic signal 102 may be generated by weighted summing the signals 101 .

According to an embodiment, the weighted summing of the input sound signal and the first signal may be performed for each section or each frame. A frame refers to a section in which a signal expressed in the time domain is divided into predetermined time units. Weighted summing of the input acoustic signal and the first signal for each frame is weighted summing the first frame of the input acoustic signal and the first frame of the first signal according to the weight, wherein the first frame of the acoustic signal and the first frame of the first signal A frame may correspond to the same time period. For example, when the frame size is 1s, the weighted sum of the input sound signal of the 0~1s frame and the first signal of the 0~1s frame, the input sound signal of the 1-2s frame and the first signal of the 1-2s frame In a method of weighted summing , the entire input sound signal and the entire first signal may be weighted and summed. The weight of the input sound signal for weighted summing and the weight of the first signal may be different for each frame. For example, when the frame size is 1s, the input acoustic signal x(0) of the 0~1s frame and the first signal y(0) of the 0~1s frame are α ₀ x(0)+(1-α ₀ ) ) can be weighted summed as y(0), and the input acoustic signal of 1~2s frame and the first signal of 1~2s frame are weighted as α ₁ x(1)+(1-α ₁ )y(1) can be weighted summed, and α ₀ and α ₁ can be determined independently of each other.

According to an embodiment, the noise controller 112 may generate the output sound signal 102 in consideration of a speech presence probability. This will be described in detail with reference to FIG. 4 below.

The system 100 according to one embodiment may include a noise control interface 120 . The noise control interface 120 may include a user interface (UI) that supports interaction between a user and the system 100 for noise control. The noise control interface 120 may be mounted on a device different from the device 110 performing the sound signal processing method and may be connected to the device 110 through a network to transmit/receive data. For example, the device 110 corresponds to a server related to sound signal processing, and the noise control interface 120 is mounted on the user's terminal and transmits data received from the user's terminal through the network to the apparatus 110 and , may receive data from the device 110 through the network and display it on the user terminal.

Unlike the one shown in FIG. 1 , the noise control interface 120 according to an embodiment may be mounted on the apparatus 110 for performing the method for processing an acoustic signal. For example, the device 110 may correspond to a user terminal and the noise control interface 120 may correspond to a user interface provided to the user terminal.

The acoustic signal processing apparatus 110 according to an embodiment may obtain an input related to noise control input through the noise control interface 120 . The input for controlling the noise may include an input for controlling the intensity of the noise. For example, the user may input a parameter for controlling the intensity of noise through the interface 120 , and the acoustic signal processing apparatus 110 may obtain the input parameter through the network.

The intensity of noise means the level of noise included in the output sound signal 102 , and the degree to which noise is removed from the input sound signal 101 may be determined according to the noise intensity setting. For example, when the noise intensity of noise is set to be small, since it means that the output sound signal contains less noise, the degree of noise removal may be set to be large. On the other hand, when the noise intensity is set to be large, the degree of noise removal may be set to be small.

For example, the configuration of the noise control interface provided to the user's terminal may refer to FIGS. 2A and 2B . Referring to FIG. 2A , the user may set the intensity of noise included in the output sound signal 102 by changing the position of the displayed bar 201 within the reference line 202 through the provided noise control interface. As the bar 201 is positioned to the left of the reference line 202 , it may indicate a small noise intensity, and as the bar 201 is positioned to the right of the reference line 202 , it may indicate a greater noise intensity. Referring to FIGS. 2A and 2B , since the bar 211 shown in FIG. 2B is located to the right of the reference line than the bar 201 shown in FIG. 2A , the intensity of the noise is set higher in FIG. 2B than in FIG. 2A . The noise control interface according to an embodiment may receive an input for changing the position of the bar 201 from a user, parameterize a point within the reference line 202 where the bar 201 is located, and obtain a parameter related to noise intensity. have.

The configuration of the interface shown in FIGS. 2A and 2B is an example of the configuration of the noise control interface 120 according to an embodiment. In addition, the noise control interface 120 may include an interface for numerically inputting the intensity of noise. and may include an interface supporting various inputs, such as a touch input through a touch screen, a key input through a keyboard, and a click input through a mouse.

A user may send an input related to noise control to the system by adjusting the intensity of noise included in the output acoustic signal 102 via the noise control interface 120 , and the system may send an input related to noise control received through the interface. The output acoustic signal 102 may be generated by adjusting the degree of noise removal based on the input.

According to an embodiment, the noise control interface 120 may be provided in an environment for receiving a voice signal. For example, the noise control interface 120 may be provided in an environment in which a voice signal of a call counterpart is received during a voice call or a video call.

For example, FIG. 3 is a diagram illustrating a noise control interface provided during a group video call. Referring to FIG. 3 , when a group video call is conducted through a user terminal, a noise control interface may be provided in an environment in which video and audio transmitted from the terminal of the call counterpart are received. In the case of a group video call, a noise control interface may be provided in response to a voice signal of each of a plurality of call parties, and the user may use the noise control interface displayed at the bottom of the interface where the video of the call party is displayed, the call party corresponding to the video It is possible to control the noise intensity of the voice signal received from the The user may receive the voice signal of the other party whose degree of noise removal is adjusted by controlling the noise intensity of the voice signal received from the call party through the interface corresponding to each of the plurality of call counterparts. In addition, since the noise intensity corresponding to the plurality of call counterparts may be set differently, the noise removal degree may be adjusted differently according to each set noise intensity to output an acoustic signal.

Referring back to FIG. 1 , whether to activate the noise control interface 120 according to an embodiment may be determined based on a signal-to-noise ratio (SNR) of the input sound signal 101 .

Signal-to-noise ratio (SNR) represents the ratio of the received signal to the noise signal. When the received signal strength is V _s and the noise strength is V _n , alog ₍ V _s /V _n )( dB) (where a is a positive constant). The larger the SNR, the larger the signal is than the noise. For example, when the SNR of the audio signal is 20 dB than when the SNR is 7 dB, the speech sounds louder and the noise sounds smaller, and 0 dB means that the speech and noise levels are the same, - When the SNR is negative, such as 10dB, it means that the noise is greater than the voice.

The device 110 according to an embodiment may obtain an SNR value of the input acoustic signal 101 and determine whether to activate the user interface 120 based on the SNR value. For example, when the SNR value is less than a predetermined reference value, it may be determined that a large amount of noise is included in the input sound signal 101 , and thus it may be determined to activate the noise control interface 120 .

According to an embodiment, when it is determined to activate the interface 120 , the interface 120 is activated, and the device 110 may perform a noise removal algorithm and a noise control algorithm.

Referring back to FIG. 3 , in the case of a group video call, a noise control interface may be provided in response to each voice signal of a plurality of call counterparts, and the SNR value of each input sound signal received from the plurality of call counterparts may be Based on it, it may be determined whether to activate the noise control interface. Whether the noise control interface is activated may be displayed on the interface in various ways. For example, whether the noise control interface is activated may be indicated by the color difference between the figures 301 and 302 displayed on the interface and the color difference between the figures 303 and 304 . When the noise control interface is activated, the user can move the position of the bar 305 indicating the noise intensity in order to input a parameter for controlling the noise intensity through the input device, whereas when the noise control interface is not activated, The user cannot move the position of the bar 306 representing the noise intensity through the input device.

4 is an exemplary diagram of a voice signal processing system according to an embodiment.

Referring to FIG. 4 , the apparatus for performing the method for processing an acoustic signal according to an embodiment may further include a configuration of a voice activity detector for acquiring a voice presence probability. The configuration of the device shown in FIG. 4 is divided to describe the sound signal processing method according to an embodiment in a functional unit similar to that of FIG. 1 , and the configuration of the device is not limited thereto.

The voice activity detection unit 410 according to an embodiment may correspond to a module for acquiring the voice presence probability of the first signal from which noise is removed from the input sound signal. The voice activity detection unit 410 may perform a process of acquiring the voice presence probability of the first signal using a voice activity detection (VAD) algorithm. The voice activity detection algorithm is an algorithm for discriminating a voice section and a silent section (or a non-voice section) in an acoustic signal, and may include various algorithms for detecting voice activity. As a result of the voice activity detection algorithm, a voice presence probability that is a probability that an acoustic signal used as an input of the algorithm is a voice signal may be output.

The voice activity detection unit 410 according to an embodiment may acquire a voice presence probability for each frame of the first signal according to a voice activity detection algorithm.

The noise control unit 420 according to an exemplary embodiment may generate an output sound signal by weighted summing the input sound signal and the first signal based on the parameters controlling the voice presence probability and the noise intensity.

The noise control unit 420 according to an embodiment weights and sums the input sound signal and the first signal based on a parameter for controlling the noise intensity, so that the second signal having a high proportion of the input sound signal and the first signal having a high proportion The output acoustic signal may be generated by generating the third signal and weighted summing the second signal and the third signal based on the voice presence probability of the first signal.

For example, when the parameter controlling the noise intensity input through the noise control interface is set to α ₁ (α ₁ is an arbitrary real number between 0 and 0.5), as in Equation 2 below, the input sound signal x(t ) and the first signal y(t) by weighted summing, a second signal z ₀ (t) having a high weight of the input acoustic signal x(t) may be synthesized.

Here, since α ₁ is an arbitrary real number greater than or equal to 0 and less than 0.5, the weight (1-α ₁ ) of the input acoustic signal x(t) is greater than the weight α ₁ of the first signal y(t). Accordingly, the second signal z ₀ (t) having a higher specific gravity of the input acoustic signal x(t) than the first signal y(t) may be generated.

In addition, by weighted summing the input sound signal x(t) and the first signal y(t) as shown in Equation 3 below, a third signal z ₁ (t) having a high proportion of the first signal y(t) is synthesized. can

Here, α ₂ is a parameter determined based on α ₁ input through the noise control interface, and may correspond to a parameter determined to be smaller as α ₁ is larger. For example, α ₂ may be determined as (1-α ₁ ).

Assuming that α ₂ is determined as (1-α ₁ ), corresponding to α ₁ , which is an arbitrary real number greater than 0 and less than 0.5, α ₂ is determined as a real number greater than 0.5 and less than or equal to 1, so that the input acoustic signal x(t) The weight (1-α ₂ ) of is smaller than the weight α ₂ of the first signal y(t). Accordingly, an output third signal z ₁ (t) having a higher specific gravity of the first signal y(t) than the input sound signal x(t) may be generated.

The output acoustic signal z(t) is generated by weighted summing the second signal z ₀ (t) and the third signal z ₁ (t) based on the speech presence probability p of the first signal as shown in Equation 4 below. can be

According to an embodiment, when the voice presence probability is high, the output sound signal includes more of the second signal having a high proportion of the input sound signal, and when the voice presence probability is low, the third signal having a high proportion of the first signal By synthesizing the first signal and the input sound signal so that the signal is more included in the output sound signal, an output sound signal in which the degree of noise removal is controlled for each section may be generated. weight α ₁ for synthesis of the second signal and the third signal, and By adjusting α ₂ , an output sound signal can be generated in which the connection between the portion including the audio signal and the portion not including the audio signal is natural.

According to an embodiment, when the voice presence probability p of the first signal is obtained for each frame, the noise controller 420 selects the input sound signal and the first signal based on the parameters controlling the voice presence probability and the noise intensity. An output sound signal may be generated by weighted summing for each frame. For example, when the audio presence probability is p ₁ in the frame t ₁ of the first signal, the output acoustic signal z(t ₁ ) corresponding to the frame t ₁ may be expressed as Equation 5 below.

According to an embodiment, when the voice presence probability p of the first signal is obtained for each frame, the noise control unit 420 determines whether each frame of the first signal is a frame having a high voice presence probability or a low voice presence probability. may be determined to generate an output sound signal.

Whether the specific frame of the first signal is a frame having a high voice presence probability or a low voice presence probability may be determined by comparing the voice presence probability obtained in the specific frame of the first signal with a predetermined threshold value. For example, when the speech presence probability of a specific frame of the first signal is 0.5 or more based on 0.5, a frame having a high speech presence probability may be determined, and when the speech presence probability is less than 0.5, a frame having a low speech presence probability may be determined. According to an embodiment, a frame having a high voice presence probability may be divided into a voice section, and a frame having a low voice presence probability may be divided into a non-voice section. In other words, the frames of the first signal may be divided into a voice section and a non-voice section based on a voice presence probability for each frame of the first signal output by applying a voice utterance detection algorithm to the first signal.

The noise control unit 420 according to an exemplary embodiment may weight and sum the input sound signal by weighting it higher than the first signal, based on the parameter controlling the noise intensity, in a frame with a high probability of voice presence among frames of the first signal. can The noise control unit 420 may weight and sum the first signal with a higher weight than the input sound signal, based on a parameter controlling the noise intensity, in a frame having a low voice presence probability among the frames of the first signal.

For example, when the parameter controlling the noise intensity input through the noise control interface is set to α ₁ (α ₁ is an arbitrary real number between 0 and 0.5), a frame with a high probability of voice presence among frames of the first signal In (t ₁ ), the first signal y(t ₁ ) and the input sound signal x(t ₁ ) may be synthesized as shown in Equation 6 below.

Here, since α ₁ is an arbitrary real number of 0 or more and 0.5 or less, the weight (1-α ₁ ) of the input acoustic signal x(t ₁ ) is greater than or equal to the weight α ₁ of the first signal y(t ₁ ). Accordingly, in a frame having a high probability of voice presence, an output acoustic signal z(t ₁ ) having a higher proportion of the input acoustic signal x(t ₁ ) than the first signal y(t ₁ ) may be generated.

On the other hand, in a frame having a low voice presence probability among frames of the first signal, the first signal y(t ₂ ) and the input sound signal x(t ₂ ) may be synthesized as shown in Equation 7 below.

Here, α ₂ is a parameter determined based on α ₁ input through the noise control interface, and may correspond to a parameter determined to be smaller as α ₁ is larger. For example, α ₂ may be determined as (1-α ₁ ).

Assuming that α ₂ is determined as (1-α ₁ ), corresponding to α ₁ that is an arbitrary real number between 0 and 0.5, α ₂ is determined as a real number between 0.5 and 1, so that the input acoustic signal x(t ₂ ) has a weight (1-α ₂ ) less than or equal to a weight α ₂ of the first signal y(t ₂ ). Accordingly, in a frame having a low voice presence probability, an output acoustic signal z(t ₂ ) having a higher proportion of the first signal than the input acoustic signal may be generated.

According to an embodiment, it is determined whether each frame of the first signal is a frame with a high voice presence probability or a low voice presence probability (or by determining whether it corresponds to a voice section or a non-voice section), and output The process of generating the sound signal is output by weighted summing the second signal and the third signal in Equation 4 above, with p = 1 for a frame with a high voice presence probability and p=0 for a frame with a low voice presence probability. It can be understood as generating an acoustic signal.

5 and 6 are diagrams illustrating a flow chart of a method for processing an acoustic signal according to an exemplary embodiment.

More specifically, FIGS. 5 and 6 illustrate an operation process of an acoustic signal system in an embodiment in which the acoustic signal processing apparatus is a server and a noise control interface is mounted in a user terminal.

Referring to FIG. 5 , the server may receive 510 an input sound signal input through an input device such as a microphone of the user terminal through a network. The server may obtain the first signal by performing the above-described noise removal algorithm on the received input sound signal ( 530 ). The user terminal may receive the noise control input from the user through the noise control interface and transmit it to the server, and the server may obtain 520 the noise control input received through the noise control interface. According to an embodiment, the operation of acquiring the noise control input and the operation of performing the noise removal algorithm may be performed in parallel.

The server may generate an output sound signal by performing a noise control algorithm for weighted summing the first signal and the input sound signal based on the obtained noise control input ( 540 ). The server may transmit (550) the output sound signal to the user terminal through the network, and the user terminal receiving the output sound signal may output the output sound signal through an output device such as a speaker.

According to an embodiment, the input sound signal received from the server may correspond to the sound signal on which the noise removal algorithm is performed in the user terminal. In other words, the user terminal may perform a process of removing noise from the acoustic signal received through the input device.

Referring to FIG. 6 , the server receiving the input sound signal from the user terminal obtains ( 610 ) an SNR value from the received input sound signal before performing the subsequent operations shown in FIG. You can decide whether to activate it or not. The SNR value may be calculated by a processor of the server or may be calculated from an external device and obtained in the server.

When the SNR value meets a predetermined criterion based on the obtained SNR value, the server may activate 620 the noise control interface of the user terminal. The predetermined criterion may correspond to a criterion related to an SNR value for determining whether to perform a noise removal and control algorithm with respect to the input sound signal. For example, by comparing the SNR value with a predetermined threshold value or threshold range, it may be determined to activate the noise control interface when the SNR value is greater than or less than the threshold value or when the SNR value is within the threshold range.

When the server activates the noise control interface, the noise control interface may be displayed on the user terminal, etc., and may be activated, and the user may transmit/receive data to and from the server through the noise control interface.

When the server determines to activate the noise control interface based on the SNR value, a noise removal algorithm may be performed. Also, since a noise control input can be obtained from the server through the noise control interface, a noise control algorithm can be performed.

On the other hand, when the SNR value does not correspond to the predetermined criterion, the server may not activate the noise control interface, and the noise removal algorithm may not be performed. When the noise control interface is not activated in the user terminal, the user cannot input parameters related to noise control through the interface. When the noise control algorithm is not performed in the server and a parameter related to noise control input through the noise control interface is not received, the noise control algorithm is not performed. In this case, the sound signal input to the user terminal may be output through the output device without a separate process for noise removal and control in the server.

5 and 6 show a case where the user terminal that transmits the input sound signal to the server and the user terminal that receives the output sound signal from the server are the same terminal, but according to an embodiment, when the input sound signal is transmitted to the server A user terminal that transmits to and a user terminal that receives an output sound signal from the server may correspond to different terminals. For example, in a video call or voice call environment, since an audio signal input through a specific user terminal is output through the terminal of the other party to the call, a user terminal that transmits an input sound signal to the server through the network and the server through the network The user terminals receiving the output sound signal may correspond to different terminals.

The embodiments described above may be implemented by a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the apparatus, methods, and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate (FPGA) array), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using a general purpose computer or special purpose computer. The processing device may execute an operating system (OS) and a software application running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or apparatus, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in a computer-readable recording medium.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination, and the program instructions recorded on the medium are specially designed and configured for the embodiment, or are known and available to those skilled in the art of computer software. may be Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or a plurality of software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited drawings, those of ordinary skill in the art may apply various technical modifications and variations based thereon. For example, the described techniques are performed in an order different from the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

In the sound signal processing method performed by the processor,
obtaining an input sound signal including a voice signal and noise;
generating a first signal by removing the noise from the input sound signal;
obtaining a parameter input through a user interface related to noise control and controlling the intensity of the noise; and
generating an output acoustic signal by weighted summing the input acoustic signal and the first signal based on the parameter;
containing,
Acoustic signal processing method.
According to claim 1,
obtaining a signal-to-noise ratio (SNR) value of the input sound signal; and
determining whether to activate the user interface based on the SNR value
further comprising,
Acoustic signal processing method.
According to claim 1,
The step of generating the output sound signal is
obtaining a speech presence probability corresponding to each section of the first signal; and
weighted summing the input sound signal and the first signal for each section based on the voice presence probability and the parameter
containing,
Acoustic signal processing method.
4. The method of claim 3,
The weighted summing step is
weighted summing the input sound signal with a higher weight than the first signal based on the parameter in the section in which the voice presence probability is high; and
weighted summing the first signal with a higher weight than the input sound signal based on the parameter in the section in which the voice presence probability is low
containing,
Acoustic signal processing method.
4. The method of claim 3,
The weighted summing step is
generating a second signal having a high proportion of the input sound signal and a third signal having a high proportion of the first signal by weighted summing the input sound signal and the first signal based on the parameter; and
weighted summing the second signal and the third signal for each section based on the voice presence probability
containing,
Acoustic signal processing method.
4. The method of claim 3,
The step of obtaining the voice presence probability includes:
Corresponding to each of the frames generated by dividing the first signal into specific time units,
obtaining a probability that the voice signal is included in a corresponding frame based on a voice activity detection algorithm;
containing,
Acoustic signal processing method.
According to claim 1,
generating the first signal
generating the first signal by performing a noise removal algorithm in the time domain of the input sound signal
containing,
Acoustic signal processing method.
According to claim 1,
generating the first signal
Generating the first signal based on a deep learning model trained to output an acoustic signal in which noise is removed from a noise-containing acoustic signal
containing,
Acoustic signal processing method.
In the sound signal processing method performed by the processor,
generating a first signal by removing the noise from an input sound signal including a voice signal and noise;
obtaining a speech presence probability in response to the first signal;
obtaining a parameter related to the degree of removal of the noise; and
controlling the degree of removal of the noise by synthesizing the input sound signal and the first signal based on at least one of a probability of the presence of the voice and at least one of the parameters;
containing,
Acoustic signal processing method.
10. The method of claim 9,
The controlling step is
controlling the degree of removal of the noise by weighting and summing the input sound signal with a higher weight than the first signal based on the parameter in a section in which the voice is highly probable; and
In a section where the probability of the presence of the voice is low, based on the parameter, weighting and summing the first signal with a higher weight than the input sound signal, thereby controlling the degree of removal of the noise;
containing,
Acoustic signal processing method.
10. The method of claim 9,
The controlling step is
dividing the first signal into a voice section and a non-voice section by applying a voice activity detection algorithm to the first signal;
weighted summing the first signal corresponding to the voice section and the input sound signal corresponding to the voice section by giving a higher weight to the input sound signal than the first signal, based on the parameter; and
weighted summing the first signal corresponding to the non-voice section and the input sound signal corresponding to the non-voice section by giving a higher weight to the first signal than the input sound signal based on the parameter
further comprising,
Acoustic signal processing method.
A computer program stored in a medium for executing the method of any one of claims 1 to 11 in combination with hardware.
acquiring an input sound signal including a voice signal and noise;
removing the noise from the input sound signal to generate a first signal,
Obtaining a parameter that is input through a user interface related to noise control and controls the intensity of the noise,
generating an output acoustic signal by weighted summing the input acoustic signal and the first signal based on the parameter;
comprising at least one processor;
Acoustic signal processing device.
14. The method of claim 13,
the processor is
obtaining a signal-to-noise ratio (SNR) value of the input sound signal;
determining whether to activate the user interface based on the SNR value,
Acoustic signal processing device.
14. The method of claim 13,
the processor is
In generating the output sound signal,
In response to each section of the first signal, obtaining a speech presence probability (speech presence probability),
weighted summing the input sound signal and the first signal for each section based on the voice presence probability and the parameter,
Acoustic signal processing device.
16. The method of claim 15,
the processor is
In the weighted summing,
In a section in which the voice presence probability is high, based on the parameter, weighted summing is performed by placing a higher weight on the input sound signal than the first signal,
In the section in which the voice presence probability is low, based on the parameter, weighted summing the first signal with a higher weight than the input sound signal,
Acoustic signal processing device.
16. The method of claim 15,
the processor is
In the weighted summing,
By weighted summing the input sound signal and the first signal based on the parameter, a second signal having a high proportion of the input sound signal and a third signal having a high proportion of the first signal are generated,
weighted summing the second signal and the third signal for each section based on the voice presence probability;
Acoustic signal processing device.
14. The method of claim 13,
the processor is
In generating the first signal,
generating the first signal by performing a noise removal algorithm in the time domain of the input sound signal,
Acoustic signal processing device.
14. The method of claim 13,
the processor is
In generating the first signal,
Based on a deep learning model trained to output a noise-removed acoustic signal from a noise-containing acoustic signal, generating the first signal,
Acoustic signal processing device.
generating a first signal by removing the noise from an input sound signal including a voice signal and noise;
In response to the first signal, obtain a speech presence probability (speech presence probability),
obtaining a parameter related to the degree of removal of the noise,
Controlling the degree of removal of the noise by synthesizing the input sound signal and the first signal based on at least one of a probability of the presence of the voice and the parameter,
comprising at least one processor;
Acoustic signal processing device.

Images