KR101043114B1 - Method of Restoration of Sound, Recording Media of the same and Apparatus of the same - Google Patents

Abstract

A sound recording method, a recording medium recording a sound restoration method, and a device for performing the sound restoration method are provided. A method of restoring a sound, the method comprising: receiving an observation sound signal including sound and noise, generating a plurality of independent sound signals from the observation sound signal, and using one of the plurality of independent sound signals using a frequency basis vector of the sound It may be configured to include the independent sound signal of the. Therefore, in the noisy environment, it is possible to reduce the influence of noise and improve the performance of sound source separation, thereby restoring only a specific sound. In particular, it is possible to improve the speech recognition performance of the speech recognition system by restoring a signal of a specific sound only.

Sound Restoration, Noise, Multipath Implicit Separation, Spectrogram, Nonnegative Matrix Decomposition, Frequency Basis Vector, Observation Sound Signal, Independent Sound Signal, Voice Signal

Description

Method of restoring sound, recording medium recording method of restoring sound and apparatus for performing sound restoring method

The present invention relates to a method for restoring a sound, a recording medium recording a method for restoring a sound, and an apparatus for restoring a sound. More particularly, the present invention relates to a sound restoring method using a frequency basis vector of sound, An apparatus for performing a method of restoring a recorded recording medium and sound.

BACKGROUND With the growing demand for multi modal user interfaces and the development of related technologies, user interfaces using various sensors such as voice, video, and haptic have been developed. In particular, in the case of voice, it is easy to obtain a voice signal by using a low-cost sensor, and has been attracting much attention because it is easy to develop a user interface in connection with other mobile devices because of its excellent portability. However, since voice signals are always exposed to noise and noise in real environment, the degradation of recognition performance is a major obstacle to the development of voice-based user interface.

In order to remove noise and the effects of noise in a real environment and to obtain only a clean voice signal, noise suppression or voice enhancement and sound source separation techniques are applied. In particular, the sound source separation technique is a technique for restoring independent signals from a mixed signal mixed with several signals, and thus is effective for separating the voice signal and non-voice signals from the mixed voice signal. In confined spaces such as offices and rooms, however, convolutive mixture problems are caused by signals coming back from the obstacles or walls, which is complicated in modeling, making it difficult to reduce noise or separate sound sources. do.

Recently, there have been attempts to use other additional information to improve existing sound source separation performance. Separating information other than the observation signal to enhance sound separation performance, such as a beam forming approach using geometric information between the sensors and the sound sources, or selectively separating sound sources that show the most power to the sensors. Approaches to the process have been proposed. However, these approaches mostly use secondary information about the environment in which the sound source is observed, and there is no attempt to improve the performance by applying inherent characteristic information of the sound source (voice) to the sound source separation.

It is an object of the present invention to provide a sound restoration method using a frequency basis vector of sound.

Another object of the present invention is to provide a recording medium which records a sound restoration method using a frequency basis vector of sound.

It is still another object of the present invention to provide an apparatus for performing a sound restoration method using a frequency basis vector of sound.

In the sound restoration method for achieving the above object of the present invention, in the sound restoration method, receiving an observation sound signal including the sound and noise, generating a plurality of independent sound signal from the observation sound signal And selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound.

The generating of the plurality of independent sound signals from the observed sound signal may be performed by using a convolutive blind source separation (CBSS) method.

Here, the frequency basis vector of the sound may be generated by applying a non-negative matrix factorization (NNF) method to a spectrogram matrix representing the time-frequency characteristic of the sound.

Here, the non-negative matrix factoring method may be to apply at least one of a method capable of probabilistic analysis, a method using orthogonal constraints, and a method using sparsity constraints.

Here, the step of selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound may use a probability model including the frequency basis vector of the sound.

Here, the step of selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound may be to select an independent sound signal having a large contribution of the sound from the plurality of independent sound signals. have.

The method may further include removing the noise from the selected independent sound signal using the frequency basis vector of the sound.

Here, the step of removing the noise from the selected independent sound signal using the frequency basis vector of the sound may be a method of restoring a spectrogram including only the sound.

Here, the sound may be a human voice signal.

According to another aspect of the present invention, there is provided a recording medium for recording a sound restoring method, the recording medium recording a sound restoring method comprising the steps of: receiving an observation sound signal including the sound and noise; Generating a plurality of independent sound signals and selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound.

The method may further include removing the noise from the selected independent sound signal using the frequency basis vector of the sound.

An apparatus for performing a sound restoration method for achieving the above object of the present invention is a device for performing a sound restoration method, receiving an observation sound signal containing the sound and noise, from the observation sound signal A sound restoring unit and the observation sound signal and the plurality of independent sounds, which generate a plurality of independent sound signals and select one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound. And a storage unit for storing the signal, the frequency basis vector of the sound, and the selected independent sound signal.

The sound restoration unit may further include removing the noise from the selected independent sound signal using the frequency basis vector of the sound.

Here, the sound may be a human voice signal.

According to the sound restoration method as described above, it is possible to reduce the influence of noise and improve the performance of the sound source separation in a noisy environment, through which it is possible to restore only a specific sound. In particular, it is possible to improve the speech recognition performance of the speech recognition system by restoring a signal of a specific sound only.

As the present invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description.

However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. In the following description of the present invention, the same reference numerals are used for the same elements in the drawings and redundant descriptions of the same elements will be omitted.

1 is a flowchart illustrating a sound restoration method according to an embodiment of the present invention. 2 is a conceptual diagram illustrating a sound restoration method according to an embodiment of the present invention.

1 and 2, the sound restoration method according to an embodiment of the present invention, in the sound restoration method, the step of receiving an observation sound signal including the sound and noise (step 110), the Generating a plurality of independent sound signals from the observed sound signal (step 120) and selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound (step 130). Can be configured.

First, the step of receiving the observation sound signal including the sound and the noise (step 110) may be inputting the sound and the noise to be restored using a plurality of microphones. The sound may mean a signal to be restored, and the noise may be generated from a sound source separate from the sound, and may be generated from the same sound source as the sound source, but may be different from the sound to be restored due to reflection from a reflector. It may be to include all of the signals received through.

Next, the frequency basis vector of the sound may be generated by applying a non-negative matrix factorization (NNF) method to a spectrogram matrix representing the time-frequency characteristic of the sound. The frequency basis vector of the sound is generated by learning a frequency characteristic basis vector representing a frequency characteristic of a specific sound using a specific sound. Here, the spectrogram is a popular way to look at the time-frequency characteristics of a given signal. In general, the sound to be analyzed has a very complex feature that changes in frequency and amplitude over time, and the spectrogram is a three-dimensional graph showing the change in frequency and amplitude over time. You will be able to display the characteristics well. In addition, the non-negative matrix factorization may be to apply at least one of a method capable of probabilistic analysis, a method using orthogonal constraints, and a method using sparsity constraints. In other words, the frequency basis vector may be trained using a probabilistic latent variable model technique, which is a probabilistic analysis of nonnegative matrix factorization (NNF).

(

X

행렬,

는 주파수 대역,

는 전체 프레임 수)이라고 할 때, 행렬

에 비음수 행렬 인수분해(NNF: Non-negative Matrix Factorization)를 적용하여 다음 수학식 1과 같이

을 두 개의 행렬로 분해할 수 있을 것이다.For example, the frequency basis vector of the sound is to generate a frequency basis vector by extracting unique frequency characteristic information of the sound through a learnable sound. In more detail, the spectrogram of the learnable sound signal is matrixed.

(

X

procession,

Is the frequency band,

Is the total number of frames)

Non-negative matrix factorization (NNF) is applied to the following equation (1)

Can be decomposed into two matrices.

(

X

행렬,

는 주파수 대역,

는 기저벡터의 수를 결정하는 인자)는 주파수 특성 기저벡터이며, 시간에 불변하는 주파수 특성을 갖는다. 행렬

(

X

행렬,

는 기저벡터의 수를 결정하는 인자,

는 전체 프레임 수)는 시간에 따라 가변적인 기저벡터이다.Where matrices

(

X

procession,

Is the frequency band,

Is a factor for determining the number of basis vectors) is a frequency characteristic basis vector, and has a frequency characteristic that is invariant with time. procession

(

X

procession,

Is a factor that determines the number of basis vectors,

Is the total number of frames).

에 대하여 비음수 행렬분해를 통해 행렬

및 행렬

로 분해할 수 있으며, 행렬

의 각 열은 시간의 변화에 불변하는 주파수 영역의 기저벡터이므로 행렬

를 학습함으로써, 상기 소리가 갖는 고유의 주파수 특성 정보를 추출 할 수 있다. 더불어 비음수 행렬 인수분해(NNF)의 일종인 확률적 해석이 가능한 Probabilistic Latent Variable Model 기법 등을 이용할 수 있으며, Overcomplete Representation 방법을 이용할 수도 있을 것이다.After all, the matrix

Matrix through nonnegative matrix decomposition

And matrices

Can be decomposed into matrices

Since each column of is a basis vector in the frequency domain that is invariant with time,

By learning, it is possible to extract the unique frequency characteristic information of the sound. In addition, the Probabilistic Latent Variable Model technique, which is a kind of nonnegative matrix factorization (NNF), can be used, and the Overcomplete Representation method may be used.

Next, the step (step 120) of generating a plurality of independent sound signals from the observed sound signal may be to use a multi-path blind source separation (CBSS) method. Multipath implicit sound source separation (CBSS) is a method of separating sound sources using only the information that each sound source is 'statistically independent'. In other words, it is possible to separate several sound sources from each other by using the property of statistical independence.

, 여기에서,

)는 아래의 수학식 2와 같이 정의될 수 있다.For example, the observation sound signal at time t in the presence of noise (

, From here,

) May be defined as in Equation 2 below.

는 독립음원 신호로서

를 만족하는 n개의 음원들이며,

는

지연시간의 혼합행렬,

는

의 최대값,

는 배경 잡음(백색 잡음)이다.From here,

Is an independent sound source signal

N sound sources satisfying

Is

Mixed matrix of latency,

Is

, The maximum value of,

Is the background noise (white noise).

)는 아래의 수학식 3과 같이 정의할 수 있다.In order to apply the multipath implicit sound source separation, the multipath mixed signal (

) Can be defined as in Equation 3 below.

)로부터 원래의 음원(

)을 찾는 것이므로 아래의 수학식 4에서 정의된 혼합행렬의 역인 역혼합행렬(demixing matrix)(

)를 구해야 할 것이다.The goal of sound source separation is to

From the original sound source (

) Is the inverse of the mixing matrix defined in Equation 4 below.

Will have to be saved.

는 역혼합행렬의 최대 길이이다.From here,

Is the maximum length of the inverse mixture matrix.

가 주파수일 때, 수학식 3은 수학식 5와 같이 각 주파수대의 곱으로 표현될 수 있을 것이다.The problem of sound source separation in the echo environment is simply expressed as the product of each frequency band in the frequency domain. In other words,

When E is the frequency, Equation 3 may be expressed as a product of each frequency band as shown in Equation 5.

와 음원의 분산행렬

로 다시 표현하면,Equation 5 to observe the sound signal

Matrix of sound and sound sources

In other words,

는 관측소리신호,

는 음원의 분산행렬을 나타낸다.From here,

Is an observation sound signal,

Represents the dispersion matrix of the sound source.

가 대각행렬이 된다는 것을 알 수 있으며, 이에 따라

의 비 대각성분들을 0으로 만드는 역혼합행렬을 구하는 수학식 7을 수학식 6의 관계를 이용하여 유도할 수 있다.From the fact that each sound source is statistically independent

We know that becomes a diagonal matrix,

Equation 7 for obtaining an inverse mixed matrix that sets non-diagonal components of 0 to 0 may be derived using the relationship of Equation 6.

By applying Joint Approximate Diagonalization to Equation 7, it is possible to obtain inverse mixing matrices in each frequency band, thereby separating the original sound sources from the multipath signal.

Next, the step (step 130) of selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound may use a probability model including the frequency basis vector of the sound. In addition, selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound may include selecting an independent sound signal having a large contribution from the sound among the plurality of independent sound signals. It may be.

For example, the following Equation 8 can be obtained by introducing a model capable of probabilistic analysis to derive a more efficient channel selection decision equation.

는 음원변수로서

, (

은 특정화자,

는 방해음원)를 만족하며,

는 각 음원에 대한 사전확률이며,

는 각 음원에 해당하는 기저함수이며,

는 각 음원이 주어졌을 경우의 시간에 따른 가중치이다.From here,

Is the sound source variable

, (

Is the specific speaker,

Is a disturbing sound source),

Is the prior probability for each sound source,

Is the basis function for each sound source,

Is weighted according to time when each sound source is given.

3 is an exemplary diagram of a probability graph model for explaining a sound restoration method according to an embodiment of the present invention. Referring to FIG. 3, it can be seen that the model of Equation 8 is represented by a probability graph model.

과

의 시간-주파수 표현에 각각 적용한다. 다만 여기에서

은 미리 학습해 놓은 기저벡터들을 그대로 사용하기 때문에(

, 여기에서

는 A행렬의 (f, z)번째 원소), 특정 소리의 정보를 채널 선택에 효과적으로 적용시킬 수 있을 것이다. 다중 경로 암묵 음원 분리(CBSS)를 통해 얻은 출력신호들은 특정 소리와 방해음원 및 잡음이 완벽히 분 리가 되지 않고 섞여있는 신호이기 때문에, 수학식 8과 같은 음원변수에 대한 확률모델을 세움으로써 이를 확률적으로 기술하고, 이를 통해 채널 선택의 판단기준을 마련할 수 있을 것이다.Independent sound signal obtained from the output channel of multipath implicit sound source separation (CBSS)

and

Apply to the time-frequency representation of. Just here

Since we use the basis vectors we have learned in advance,

, From here

(F, z) elements of the matrix A), it is possible to effectively apply the information of a particular sound to the channel selection. Since output signals obtained through multipath blind source separation (CBSS) are signals that are not completely separated from specific sounds, disturbed sources, and noise, they are probabilistic by establishing a probabilistic model for sound source variables such as Equation (8). It will be possible to prepare criteria for channel selection.

를 제외한

이다. 이 값들은 잠재변수가 있을 경우, 유사도를 최대화하는 (EM: Expectation and Maximization) 알고리즘으로 구할 수 있다. EM 알고리즘은 E 단계와 M 단계로 나누어져 있으며, 두 단계를 수렴할 때까지 반복적으로 수행한다.The elements to be obtained in Equation 8

excluding

to be. These values can be obtained by using the Expectation and Maximization (EM) algorithm if there are potential variables. The EM algorithm is divided into E stage and M stage, and iteratively executes until the two stages converge.

E 단계:

E stage:

M 단계:

M stage:

가 클수록 주어진 시간-주파수 표현에 특정 소리(

)의 기여도가 크다는 것을 알 수 있다. 그러므로 판단 기준식(

)은 수학식 11로 표현될 수 있을 것이다.Here, the criterion for determining the channel selection can be obtained using the prior probability of the specific sound. Intuitively

The larger is the specific sound in a given time-frequency representation.

) Is a large contribution. Therefore, the judgment criterion (

) May be represented by Equation 11.

과

에 앞서의 확률모델을 적용하여

들을 구하고 최종적으로 수학식 11을 서로 비교함으로써 특정 소리가 보다 우세하게 포함되어 있는 채널을 선택할 수 있을 것이다.Ie two time-frequency representations

and

By applying the probabilistic model

And finally compare Equation 11 to select a channel that contains a particular sound more predominantly.

Next, the sound restoration method according to an embodiment of the present invention may further comprise the step (step 140) of removing the noise from the selected independent sound signal using the frequency basis vector of the sound. Removing the noise from the selected independent sound signal using the frequency basis vector of the sound (step 140) may use a method of restoring a spectrogram including only the sound.

를 이용하여 최종적으로 특정 소리를 복원할 수 있을 것이다. 이를 위하여 먼저 특정 소리만을 담고 있는 스펙트로그램(

)을 복원하면,

의 (f,t)의 원소

는 아래의 수학식 13과 같다.For example,

You can finally restore the specific sound using. To do this, first of all, a spectrogram containing only certain sounds

),

Of (f, t) in

Is as shown in Equation 13 below.

는

음원에서 특정 확률분포로 뽑힌 수라는 점을 이용하면, 특정 소리의 스펙트로그램

은

가 이루는 이항 분포에서 총 뽑힌 횟수

가 주어졌을 때의 평균으로 정의될 수 있다.From here,

Is

Spectrogram of a particular sound using the fact that it is a number drawn with a certain probability distribution from the sound source

silver

The total number of times drawn from the binomial distribution

It can be defined as the mean given by.

과 해당 채널의 신호에서 얻은 위상정보를 이용하고, 이를 역-STFT(Short Time Fourier Transform) 변환을 통하여 최종적 으로 특정화자의 음성신호만을 복원할 수 있다.Time-Frequency Representation

And the phase information obtained from the signal of the corresponding channel, and finally, only the speech signal of the specific speaker can be restored through inverse-short time Fourier transform (STFT) transformation.

In addition, in the sound restoration method according to an embodiment of the present invention, the sound may be a human voice signal. It can be applied as a method of restoring a human voice signal, thus improving the voice recognition performance of the voice recognition device.

In the recording medium recording the sound restoration method according to another embodiment of the present invention, in the recording medium recording the sound restoration method, receiving the observation sound signal containing the sound and noise, a plurality of independent from the observation sound signal The method may include generating a sound signal and selecting one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound. In addition, the method may further include removing the noise from the selected independent sound signal using the frequency basis vector of the sound. Since the sound restoration method has already been described, a detailed description thereof will be omitted.

In addition, the apparatus for performing the sound restoration method according to another embodiment of the present invention, in the apparatus for performing the sound restoration method, receives an observation sound signal including the sound and noise, a plurality of observation sound signal from the A sound restoring unit and the observation sound signal, and the plurality of independent sound signals, which generate an independent sound signal of the sound source and select one independent sound signal from the plurality of independent sound signals using the frequency basis vector of the sound. And a storage unit for storing the frequency basis vector of the sound and the selected independent sound signal, wherein the sound restoration unit removes the noise from the selected independent sound signal using the frequency basis vector of the sound. It may be configured to include more. The sound may be a human voice signal. Since the sound restoration method has already been described, a detailed description thereof will be omitted.

4 is a model diagram for experimenting with a sound restoration method according to an embodiment of the present invention. 5 is a graph of the impact function according to the experiment of the sound restoration method according to an embodiment of the present invention.

In order to evaluate the performance of the sound restoration method according to an embodiment of the present invention, the degree of noise and echo cancellation is evaluated in an environment having real noise and echo. For the quantitative analysis, room simulation is used to simulate real noise and echo.

Referring to FIG. 4, the sizes of the rooms for the Room Simulation are 6.75m in width, 3.75m in height, and 2.5m in height. Referring to FIG. 5, the impact function response indicating the degree of reflection can be seen. S1 and S2 mean Source1 and Source2, respectively. In FIG. 4, S1 and S2 represent positions where each sound source can be located. ① indicates that S1 is located closer than S2, ② indicates that the distance between S1 and S2 is the same, and ③ indicates that S1 is farther than S2. It means. The sound source of a specific sound is located at S1, and you can perform performance evaluation at various locations by recording while changing the position of ① ~ ③. In the experiment, it is assumed that the specific sound S1 is the voice of the female speaker or the male speaker, and the disturbance source S2 is the sound of the fritter output or the trumpet playing.

Figure 6 is an illustration of the basis vector alignment of the male and female in the sound restoration method according to an embodiment of the present invention.

을 얻을 수 있다. 여기에서, 윈도우 크기는 1024, hop크기는 256 및 hamming window를 사용한다.

을 비음수 분해하기 위해 Probabilistic Latent Variable Model을 적용하고, 이때 K를 1000(>>F=512)으로 하여 Over-representation이 가능하도록 하였다. 이외에도 엔트로픽(Entropic) 사전확률을 정의할 때 요구되는 사용자 계수들은 Shashanka와 Smaragdis 연구에서 제시된 수치를 따른다.Referring to FIG. 6, it is possible to identify the frequency characteristic basis vectors of a female and a male when the experiment is performed by the following method. For learning the frequency-based basis vector, a female or male voice signal of 30 seconds in length is used as learning data separate from the voice signal. Time-frequency representation of female and male voice signals through STFT (Short Time Fourier Transform) transformation

Can be obtained. Here, the window size is 1024, the hop size is 256, and the hamming window is used.

Probabilistic Latent Variable Model was applied to non-negative decomposition, and K was set to 1000 (>> F = 512) to enable over-representation. In addition, the user coefficients required to define entropic prior probabilities follow the figures presented in the Shashanka and Smaragdis studies.

)를 아래의 수학식 16의 분리모델을 이용하여 분리한다.In addition, SDR (Source to Distortion Ratio) may be introduced to quantitatively evaluate the restoration results. SDR is a unit of measurement for evaluating the quality of sound source separation methods in a noisy environment. The larger the value, the closer the result signal to the target signal. Given signal for calculation (

) Is separated using the separation model of Equation 16 below.

은 복원하고자 하는 소리이고,

는 여러 음원에 의하여 생성된 간섭 소리이고,

는 복원하고자 하는 소리가 아닌 다른 음원에서 생성된 잡음 소리이고,

는 인공적으로 생성한 잡음 소리이다.From here,

Is the sound you want to restore,

Is interference sound produced by various sound sources,

Is noise generated from a sound source other than the sound you want to restore,

Is an artificially generated noise sound.

Based on the separation model of Equation 16, the SDR is obtained as shown in Equation 17.

), 두 입력 중 SDR이 높은 쪽 선택 2) 음원분리만을 통해 얻은 결과의 SDR값(

), 3) 본 발명의 복원 단계를 거친 최종 결과신호의 SDR값(

).In order to evaluate the performance of the sound restoration method according to an embodiment of the present invention, the SDR is measured at each step, and the increase thereof is compared with the existing CBSS. That is, the SDR is measured in every detail in large part. 1) SDR value of the observed sound signal obtained from the microphone (

), Select the higher SDR of the two inputs.

), 3) the SDR value of the final resultant signal

).

7 is a performance evaluation table of the sound restoration method according to an embodiment of the present invention.

Referring to FIG. 7, (a) is a female speaker, (b) is a SDR result of each step in the case of a male speaker, where pr represents printer noise, and tr represents the use of trumpet music as a disturbing sound source. Indicates. (a) and (b) evaluates the restoration results of the sound restoration method according to an embodiment of the present invention in various situations, when the female and the male are the specific speakers, respectively. In particular, it can be seen that through the increase of the SDR value, the sound restoration method according to the exemplary embodiment of the present invention substantially eliminates distortion caused by echo and noise, and greatly improves the separation performance of the CBSS. That is, the present invention is shown to be suitable for removing noise and echo and restoring the voice of the present speaker in a noise and echo environment.

Although described with reference to the above embodiments, those skilled in the art will understand that various modifications and changes can be made without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

1 is a flowchart illustrating a sound restoration method according to an embodiment of the present invention.

2 is a conceptual diagram illustrating a sound restoration method according to an embodiment of the present invention.

3 is an exemplary diagram of a probability graph model for explaining a sound restoration method according to an embodiment of the present invention.

4 is a model diagram for experimenting with a sound restoration method according to an embodiment of the present invention.

5 is a graph of the impact function according to the experiment of the sound restoration method according to an embodiment of the present invention.

Figure 6 is an illustration of the basis vector alignment of the male and female in the sound restoration method according to an embodiment of the present invention.

7 is a performance evaluation table of the sound restoration method according to an embodiment of the present invention.

Claims (14)

In the sound restoration method, Generating a frequency basis vector of the sound using a matrix representing the time-frequency characteristic of the sound, and generating a set of frequency characteristic basis vectors of a predetermined speaker voice using the generated frequency basis vector; Receiving an observation sound signal including sound and noise observed through various paths from the microphone; When the sound source is separated from the observed sound signal to generate a plurality of independent signals, and the sound is reconstructed using at least one frequency characteristic basis vector in the set of frequency characteristic basis vectors, the error between the sound and the reconstructed sound Selecting a small signal as an independent signal; And And removing noise from the selected independent signal using the set of frequency feature basis vectors. delete delete delete delete delete delete delete delete In the recording medium recording the sound restoration method, Generating a frequency basis vector of the sound using a matrix representing the time-frequency characteristic of the sound, and generating a set of frequency characteristic basis vectors of a predetermined speaker voice using the generated frequency basis vector; Receiving an observation sound signal including sound and noise observed through various paths from the microphone; A plurality of independent signals are generated by separating a sound source from the observed sound signal, and a signal having a small error magnitude between the sound and the reconstructed sound as an independent signal when reconstructing the sound using the set of frequency characteristic basis vectors. Selecting; And And removing noise from the selected independent signal using the set of frequency feature basis vectors. delete In the device for performing the sound restoration method A plurality of independent signals are generated by separating a sound source from the observed sound signal, and a signal having a small error magnitude between the sound and the reconstructed sound as an independent signal when reconstructing the sound using the set of frequency characteristic basis vectors. A sound restoring unit for performing selection; And And a storage unit for storing the observation sound signal, the plurality of independent sound signals, the frequency basis vector of the sound, and the selected independent sound signal. delete delete

Images