KR101838408B1 - Method and apparatus for processing audio signal - Google Patents

Abstract

An audio signal processing method and a device thereof are disclosed. According to an embodiment, the audio signal processing method can convert an audio signal into a spectrogram, obtain time axis bases and frequency axis bases from the spectrogram, and separate an audio signal of a chord musical instrument and an audio signal of a percussion instrument from the audio signal based on at least one of continuity of the time axis bases and continuity of frequency axis bases.

Description

TECHNICAL FIELD [0001] The present invention relates to an audio signal processing method,

The following embodiments are directed to techniques for processing audio signals.

Sound source separation techniques are used to separate various types of sounds from an audio signal. In order to separate the sound of Mars musical instrument and the sound of percussion instrument from the audio signal, the method of directly employing the time series signal is less effective because the amount of data required to process is larger than the information amount. The sound source separation technique is mainly performed by Fourier transform processing of audio signals. The sound source separation technique is classified into non-negative matrix factorization (NMF) or low latency component analysis using probability latent component analysis (PLCA) -rank decomposition. However, such sound source separation techniques are vulnerable to separation of vocal sound or percussion sound, and therefore, development of a technique capable of enhancing the decomposition performance of an audio signal is required.

Embodiments are intended to enhance the decomposition performance of an audio signal.

Embodiments seek to acquire more thoroughly the various types of sounds to be separated.

A method of processing an audio signal according to an exemplary embodiment includes converting an audio signal to a spectrogram; Obtaining time base bases and frequency base bases from the spectrogram; And separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument included in the audio signal based on at least one of continuity of the time base bases and continuity of the frequency axis bases.

According to one embodiment, continuity of the time base bases may be defined based on differences between components in the time base direction among the components belonging to the time base bases.

According to one embodiment, the continuity of the frequency axis bases may be defined based on differences between components in the frequency axis direction among the components belonging to the frequency axis bases.

According to one embodiment, said separating step comprises: enhancing the continuity of some bases predetermined to correspond to said Mars musical instrument among said time base bases; Enhancing the discontinuity of some of the predetermined bases to correspond to the percussion instrument among the time base bases; Enhancing discontinuities of some bases predetermined to correspond to the Mars musical instrument among the frequency axis bases; And enhancing the continuity of some of the predetermined bases corresponding to the percussion instrument among the frequency axis bases.

The method of processing an audio signal according to an embodiment includes estimating a spectrogram of the Mars musical instrument based on some bases whose continuity is enhanced to correspond to the Mars musical instrument and some bases whose discontinuity is enhanced to correspond to the Mars musical instrument step; And inversely transforming the estimated spectrogram into the audio signal of the Mars musical instrument.

According to an embodiment of the present invention, there is provided a method for processing an audio signal, comprising the steps of: estimating a spectrogram of the percussion instrument based on some discontinuity-enhanced bases corresponding to the percussion instrument and some bases whose continuity is enhanced to correspond to the percussion instrument; And inversely transforming the estimated spectrogram into the audio signal of the percussion instrument.

According to one embodiment, said separating comprises: updating said time base bases based on said spectrogram and said frequency axis bases; Enhancing the time axis continuity of the bases belonging to the Mars musical instrument group among the updated time base bases; And enhancing the time axis discontinuities of the bases belonging to the percussion group of the updated time base bases.

According to one embodiment, the step of enhancing the time axis continuity includes updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the group of Mars musical instruments and the component neighboring the time axis is small Step < / RTI >

를

로 갱신하는 단계를 포함하고, 상기 α는 0과 1사이의 실수이고, 상기

는 상기 H^(Harm) 내 k번째 행 및 n-1번째 열의 성분일 수 있다. According to one embodiment, the step of enhancing the time-axis continuity comprises the steps of: ^(a) generating a matrix H ^(Harm) representing the bases belonging to the group of Mars musical instruments,

To

, Wherein alpha is a real number between 0 and 1,

May be a component of the k-th row and the (n-1) -th column in the H ^(Harm) .

According to one embodiment, the step of enhancing the time axis discontinuity comprises updating the specific component so that the difference between the specific component in the matrix representing the bases pertaining to the percussion group and the component in the time axis is greater, . ≪ / RTI >

를

로 갱신하는 단계를 포함하고, 상기 β는 1보다 큰 실수이고, 상기

는 상기 H^{( Perc )} 내 k번째 행 및 n-1번째 열의 성분일 수 있다.According to one embodiment, the step of enhancing the time base discontinuities are in the k-th row and n th column component, H ^(Perc) matrix indicative of the base belonging to the group percussion

To

Wherein? Is a real number greater than 1,

May be a component of the k-th row and the (n-1) -th column in the H ^{( Perc )} .

According to one embodiment, the number of bases belonging to the group of Mars musical instruments and the number of bases belonging to the percussion group can be predefined.

According to one embodiment, said separating comprises: updating said frequency axis bases based on said spectrogram and said time base bases; Enhancing the frequency axis discontinuities of bases belonging to the Mars musical instrument group among the frequency axis bases; And enhancing the frequency axis continuity of the bases belonging to the percussion group of the frequency base bases.

According to one embodiment, the step of enhancing the frequency axis discontinuity includes updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the group of Mars musical instrument groups and the component neighboring the frequency axis becomes larger, .

를

로 갱신하는 단계를 포함하고, 상기 γ는 1보다 큰 실수이고, 상기

는 상기 W^(Harm) 내 m-1번째 행 및 k번째 열의 성분일 수 있다.According to one embodiment, the step of enhancing the frequency axis discontinuity is shown in the base matrix of W ^(Harm) within the m-th row and k-th column component belonging to the group of chemical conversion instrument

To

, Wherein? Is a real number greater than 1,

May be a component of the ⁽ m-1) -th row and the k-th column in the W ^(Harm) .

According to one embodiment, the step of enhancing the frequency axis continuity comprises updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the percussion group and the component neighboring the frequency axis is small Step < / RTI >

를

로 갱신하는 단계를 포함하고, 상기 δ는 0과 1사이의 실수이고, 상기

는 상기 W^{( Perc )} 내 m-1번째 행 및 k번째 열의 성분일 수 있다.According to one embodiment, the step of enhancing the frequency-axis continuity of the underlying matrix represents the W ^(Perc) within the m-th row and k-th column component belonging to the group percussion

To

, Wherein said delta is a real number between 0 and 1,

May be a component of the ⁽ m-1) -th row and the k-th column in the W ^{( Perc )} .

According to one embodiment, the number of bases belonging to the group of Mars musical instruments and the number of bases belonging to the percussion group can be predefined.

An apparatus for processing an audio signal according to an embodiment is provided for converting an audio signal to a spectrogram, obtaining time-base bases and frequency-axis bases from the spectrogram, determining at least one of continuity of the time- And a processor for separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument included in the audio signal based on the one.

Embodiments can improve the decomposition performance of an audio signal.

Embodiments can more fully acquire the various types of sounds to be separated.

1 is a flowchart illustrating an audio signal processing method according to an embodiment.
2 is a flowchart illustrating an audio signal processing method according to an exemplary embodiment of the present invention.
FIG. 3 is a flowchart for explaining an operation method of an audio signal processing method according to an embodiment.
4 is a diagram for explaining an audio signal processing method according to an embodiment.
5A and 5B are exemplary views of a spectrogram generated by an audio signal processing method according to an embodiment.
6 is an exemplary diagram illustrating the components of an audio signal processing apparatus according to an embodiment.

Specific structural or functional descriptions of embodiments are set forth for illustration purposes only and may be embodied with various changes and modifications. Accordingly, the embodiments are not intended to be limited to the specific forms disclosed, and the scope of the disclosure includes changes, equivalents, or alternatives included in the technical idea.

The terms first or second, etc. may be used to describe various elements, but such terms should be interpreted solely for the purpose of distinguishing one element from another. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

It is to be understood that when an element is referred to as being "connected" to another element, it may be directly connected or connected to the other element, although other elements may be present in between.

The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises ", or" having ", and the like, are used to specify one or more of the described features, numbers, steps, operations, elements, But do not preclude the presence or addition of steps, operations, elements, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the meaning of the context in the relevant art and, unless explicitly defined herein, are to be interpreted as ideal or overly formal Do not.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

1 is a flowchart illustrating an audio signal processing method according to an embodiment.

Referring to FIG. 1, an audio signal processing apparatus may convert an audio signal into a spectrogram (101). An audio signal processing apparatus is an apparatus for processing an audio signal, and may be implemented by a software module, a hardware module, or a combination thereof. For example, the audio signal processing apparatus can process or analyze an audio signal of a mixed sound source in which a harmonic sound and a percussive sound are mixed, and can process an audio signal of a Mars musical instrument and audio of a percussion instrument The signal can be separated. An audio signal processing device can generate a spectrogram of an audio signal mixed with a musical instrument sound and a percussion sound. The spectrogram is an analysis tool that visualizes an audio signal by a combination of a waveform and a spectrum, And the difference between the amplitudes according to the changes of the time axis and the frequency axis can be expressed as a graph representing the difference in density or color.

The audio signal processing device may receive the input audio signal and may divide the received audio signal into predefined frame intervals. The audio signal processing apparatus can convert a divided audio signal into a frequency domain using Fast Fourier Transform (FFT) or the like. The audio signal processing apparatus can generate spectrums from audio signals segmented on a frame-by-frame basis, and combine spectrums for each frame to generate a spectrogram. For example, if the spectrum represents frequency-specific amplitude, the horizontal axis of the spectrogram can be set to the time axis and the vertical axis to the frequency axis. Various methods may be employed in the manner in which the spectrogram is implemented, and an audio signal processing apparatus according to an embodiment may process an audio signal using a spectrogram that is modified and defined according to design intent, The embodiment is not limited by the type of the manner of expressing the audio signal.

Referring to FIG. 1, an audio signal processing apparatus may obtain time base bases and frequency base bases from a spectrogram (102). Generally, the percussion sound shows the characteristic that the energy spreads in a wide band, and the sound of the Mars musical instrument shows the characteristic that the sound is held for a certain time with the pitch. Thus, the sound of a musical instrument can be expressed in the form of a horizontal line having a discontinuity in the frequency axis direction with a degree of fluctuation (DF) in the time axis direction (horizontal direction) on the spectrogram. On the other hand, the percussion sound can be expressed in the form of a vertical line with continuity in the frequency axis direction (longitudinal direction) on the spectrogram and discontinuity in the time axis direction. From the spectrogram having these characteristics, an audio signal processing apparatus according to an embodiment can generate a matrix representing time base bases and a matrix representing frequency base bases, respectively, which will be described later with reference to FIGS. 3 and 4 .

1, an audio signal processing apparatus may separate an audio signal of a Mars musical instrument and an audio signal of a percussion instrument included in an audio signal based on at least one of continuity of time base bases and continuity of frequency axis bases ( 103). Here, the continuity of the time base bases can be defined based on the differences between the components in the time base direction among the components belonging to the time base bases. The continuity of the frequency axis bases can be defined based on the differences between the components in the frequency axis direction among the components belonging to the frequency axis basis.

According to an embodiment, an audio signal processing apparatus performs learning by providing characteristics of continuity and discontinuity to bases learned through a matrix decomposition algorithm, and learns the sounds of a Mars musical instrument and a percussion instrument, Finally, the audio signal of the Mars musical instrument and the audio signal of the percussion instrument can be separated from the audio signal. The learning of the bases includes repetition of operations of updating the bases required to acquire the audio signal of the Mars musical instrument and the audio signal of the percussion instrument, and learning the Mars musical instrument sound and the percussion sound is the final audio Signal (Mars musical instrument and percussion instrument). The audio signal processing apparatus can separate the audio signal of the Mars musical instrument and the audio signal of the percussion instrument from the audio signal using the continuity value (DF measure). For example, the audio signal processing apparatus can separate the audio signal of the Mars musical instrument and the audio signal of the percussion instrument from the audio signal by adjusting the continuity value of the time base bases and the frequency bases. The continuity value may be defined as a sum of the differences of neighboring terms in one vector. For example, the audio signal processing apparatus may use the continuity value defined by Equation (1).

는 연속성 값이며,

는 길이가

인 1차원 벡터이며,

는 인덱스를 의미한다. here,

Is a continuity value,

Has a length of

Dimensional one-dimensional vector,

Indicates an index.

The apparatus for processing an audio signal according to an exemplary embodiment may enhance the continuity of some bases predetermined to correspond to a Mars musical instrument in time base baselines or enhance some discontinuities of predetermined bases to correspond to percussion among time base bases, Performing at least one of enhancing the discontinuity of some bases predetermined to correspond to the Mars musical instrument among the axial bases or enhancing the continuity of some bases predetermined to correspond to the percussion instrument among the frequency axis bases, And can acquire the audio signal of the Mars musical instrument and the audio signal of the percussion instrument using the learned bases. Hereinafter, an embodiment of an audio signal processing technique through enhancement of continuity or discontinuity will be described in more detail with reference to FIG. 2 to FIG.

2 is a flowchart illustrating an audio signal processing method according to an exemplary embodiment of the present invention.

Referring to FIG. 2, an audio signal processing apparatus may convert an audio signal into a spectrogram (201). As described above, the audio signal processing apparatus can perform the fast Fourier transform by dividing the input audio signal into predefined frame intervals.

Referring to FIG. 2, an audio signal processing apparatus may initialize time base bases and frequency base bases (202). The audio signal processing apparatus can initialize time base bases and frequency base bases using randomly generated random numbers or using a predefined set of bases. According to one embodiment, an audio signal processing apparatus may matrix-decompose a spectrogram to produce a matrix representing time-base bases and a matrix representing frequency-based bases, wherein the audio signal processing apparatus includes a matrix representing time- And initialize the matrix representing the axis bases, respectively. The audio signal processing apparatus can generate the audio signal of the Mars musical instrument and the audio signal of the percussion instrument by adjusting the continuity of the frequency axis bases and the time base bases, And embodiments of the initialization processing of the bases are not limited to the above-mentioned examples, and various techniques and initialization schemes can be employed and applied.

Referring to FIG. 2, the audio signal processing apparatus may update the time base basis (203). The audio signal processing apparatus may update the time base bases based on the spectrogram and the frequency axis basis. For example, the audio signal processing apparatus updates the time base bases corresponding to the spectrogram and frequency axis bases, using a non-negative matrix factorization (NMF) algorithm and a Dirichlet constraint can do. Here, the operation of updating the time base bases is included in the operation of the audio signal processing device learning the time base bases.

Referring to FIG. 2, the apparatus for processing an audio signal may enhance the time axis continuity of bases belonging to the group of Mars musical instruments in the updated time base bases (204). According to one embodiment, an audio signal processing apparatus can update a specific component in a matrix representing bases belonging to a group of Mars musical instruments and a specific element thereof so that a difference between the specific element and a neighboring element in the time axis is small, The process can enhance the time axis continuity of the Mars instrument group.

Referring to FIG. 2, the apparatus for processing an audio signal may enhance the time axis discontinuity of bases belonging to a percussion group among the updated time base bases (205). According to one embodiment, the audio signal processing apparatus can update the specific component in the matrix representing the bases belonging to the percussion group and the specific component so that the difference between the specific component and the neighboring component in the time axis becomes larger, The time axis discontinuity of the percussion group can be enhanced. Here, the number of bases belonging to the Mars musical instrument group and the number of bases belonging to the percussion group can be predefined in the time base bases, where the predefined number can be set to a specific index. The audio signal processing apparatus distinguishes the group of Mars musical instrument groups and the group of percussion instruments in the updated time base basis based on the predefined number and updates the continuity and discontinuity of the time axis of the distinguished musical instrument group and the percussion group respectively can do. The audio signal processing apparatus can convert the time axis basis of the group of Mars musical instruments in the time axis basis to have a continuity in the time axis and convert the time axis basis of the percussion group of the time axis basis to have a discontinuity in the time axis.

Referring to FIG. 2, the audio signal processing apparatus may update the frequency axis basis (206). The audio signal processing device may update the frequency axis bases based on the spectrogram and time base bases. For example, the audio signal processing apparatus updates the frequency axis bases corresponding to the spectrogram and time axis bases, using a non-negative matrix factorization (NMF) algorithm and a Dirichlet constraint can do. Here, the operation of updating the frequency axis basis is included in the operation in which the audio signal processing device learns the frequency axis basis.

Referring to FIG. 2, the apparatus for processing an audio signal may enhance the frequency axis discontinuity of the bases belonging to the group of Mars musical instruments in the updated frequency axis basis (207). According to one embodiment, an audio signal processing apparatus can update a specific component in a matrix representing bases belonging to a group of Mars musical instruments and a specific component so that a difference between the specific component and a component adjacent to the frequency axis becomes larger, The renewal process can enhance the frequency axis discontinuity of the Mars instrument group.

Referring to FIG. 2, the audio signal processing apparatus may enhance frequency axis continuity of bases belonging to a percussion group among the updated frequency axis bases (208). According to one embodiment, the audio signal processing apparatus can update the specific component in the matrix representing the bases belonging to the percussion group and its specific component so that the difference between the specific component and the component adjacent to the frequency axis becomes smaller, The frequency axis continuity of the percussion group can be enhanced through the process. Here, the number of bases belonging to the Mars musical instrument group and the number of bases belonging to the percussion group can be predefined in the frequency axis basis, where the predefined number can be set to a specific index. The audio signal processing apparatus distinguishes the group of harmonized musical instruments and the group of percussive musical instruments in the updated frequency axis basis on the basis of the predefined number and updates the discontinuity and continuity of the frequency axis of the distinguished musical instrument group and percussion group as described above can do. The audio signal processing apparatus can convert the frequency axis bases of the Mars musical instrument group in the frequency axis basis to have a discontinuity in the frequency axis and convert the frequency axis basis of the percussion group of the frequency axis basis to have continuity in the frequency axis.

Referring to FIG. 2, the audio signal processing apparatus can repeatedly perform the operations or operations of 203 to 208. The audio signal processing apparatus may repeat the operations described above until the continuity or discontinuity of the bases converges to a predefined level and may repeat the operations described above a predetermined number of times. The audio signal processing apparatus can terminate the repetition of the above-described operations under the condition that the required level of continuity or discontinuity can be confirmed from the bases according to the design intention.

Referring to FIG. 2, the audio signal processing apparatus can convert the updated time base bases and frequency base bases into the audio signals of the Mars musical instrument and the audio signals of the percussion instrument (209). The audio signal processing apparatus can reconstruct the spectrogram of the Mars musical instrument and the spectrogram of the percussion instrument respectively using the frequency axis bases and the time base bases after completion of the above-described iterative calculation or operation. The audio signal processing apparatus can generate the audio signal of the Mars musical instrument and the audio signal of the percussion instrument from the spectrogram of the restored Mars musical instrument and the spectrogram of the percussion instrument, respectively.

According to one embodiment, an apparatus for processing an audio signal includes a base having time-axis continuity enhanced to correspond to a Mars musical instrument among the bases obtained through the above-described iterative operation, and a plurality of bases having enhanced frequency axis discontinuity to correspond to the Mars musical instrument On the basis of this, the spectrogram of the Mars musical instrument can be estimated. The audio signal processing apparatus can invert the estimated spectrogram into an audio signal of the musical instrument.

According to one embodiment, the apparatus for processing an audio signal is characterized in that the time axis discontinuity is enhanced to correspond to a percussion instrument among the bases obtained through the above-described iterative operation, and bases whose frequency axis continuity is enhanced so as to correspond to the percussion instrument , The spectrogram of the percussion instrument can be estimated. The audio signal processing apparatus can invert the estimated spectrogram into an audio signal of a percussion instrument.

FIG. 3 is a flowchart for explaining an operation method of an audio signal processing method according to an embodiment.

와

는 연속성을 강화하기 위한 계수로서, 1보다 작은 양의 실수의 범위 내에서 설정될 수 있고,

와

는불연속성을 강화하기 위한 계수로서 1보다 큰 실수의 범위로 설정될 수 있다. 도 3의 순서도 내 수학식들은 도 2를 참조하여 설명된 동작들의 구체적 예시일 수 있지만, 실시예는 도 3의 수학식들에 제한되는 것은 아니며 설계 의도에 따라 다양한 기법이 채용되거나 변형된 수학식들이 이용될 수 있다.According to one embodiment, an audio signal processing apparatus can update bases using a non-numeric matrix decomposition algorithm and perform continuity and discontinuity enhancement techniques using a Dirichlet constraint. An example of a technique for enhancing continuity or discontinuity of bases will be described with reference to Figure 3,

Wow

Is a coefficient for enhancing continuity and may be set within a range of positive real numbers less than 1,

Wow

May be set to a range of real numbers greater than 1 as a factor for enhancing discontinuity. 3 may be a concrete example of the operations described with reference to FIG. 2, but the embodiment is not limited to the equations of FIG. 3, and various schemes may be employed according to the design intent, Can be used.

Referring to FIG. 3, an audio signal processing apparatus may randomize W and H (301). Where W is the matrix representing the frequency axis basis and H is the matrix representing the time base basis. The audio signal processing apparatus can matrix-decompose spectrogram F to generate W and H. For example, an audio signal processing apparatus may matrix-decompose F with rows and columns of M and N, respectively, to produce W with rows and columns of M and K respectively and H of rows and columns of K and N, respectively. The audio signal processing apparatus can randomly initialize W and H using a random number, for example, a random number can be set to a value between 0 and 1 following a uniform distribution.

Referring to FIG. 3, the audio signal processing apparatus can learn H using Equation (2) (302).

는 W의 m번째 행 k번째 열에 있는 성분이고,

는 H의 k번째 행 n번째 열에 있는 성분이고,

은 스펙트로그램 F의 m번째 행 n번째 열에 있는 성분이며,

는 추정된 스펙트로그램

(=WH)의 m번째 행 n번째 열에 있는 성분이다. 오디오 신호 처리 장치는 비음수행렬분해 알고리즘 중 시간 축 기저들을 수학식 2를 이용하여 갱신하고, 갱신된 시간 축 기저들을 학습할 수 있다.Here, < - > is an operator for updating the left term in the right term,

Is the component in the m-th row, k-th column of W,

Is the component in the kth row, nth column of H,

Is the component in the m-th row, n-th column of spectrogram F,

Lt; RTI ID = 0.0 >

(= WH) in the m-th row n-th column. The audio signal processing apparatus can update the time base bases of the non-tone number matrix decomposition algorithm using Equation (2) and learn updated time base bases.

Referring to FIG. 3, the apparatus for processing an audio signal may enhance the time axis continuity of bases belonging to a group of Mars musical instruments in time base bases using Equation (3) (303).

는 H^(Harm) 내 k번째 행 및 n번째 열의 성분이고,

는 H^(Harm) 내 k번째 행 및 n-1번째 열의 성분이고, α는 0과 1사이의 실수이다.Here, ← is an operator for updating the left term to the right term, H ^(Harm) is a matrix representing time base bases belonging to the musical instrument group,

Is a component of the k-th row and the n-th column in H ^(Harm)

Is a component of the k-th row and the (n-1) -th column in H ^(Harm) , and alpha is a real number between 0 and 1.

를

로 갱신할 수 있다.The audio signal processing apparatus can update the bases of the Mars musical instrument group among the time base bases acquired at 302 using Equation 3 and enhance the time axis continuity of the time axis bases of the Mars musical instrument group. The audio signal processing device is a component of the kth row and the nth column in H ^(Harm) which is a matrix representing time base bases belonging to the group of Mars musical instruments

To

. ≪ / RTI >

Referring to FIG. 3, the audio signal processing apparatus may enhance the time axis discontinuity of the bases belonging to the percussion group among the time base bases using Equation 4 (304).

는 H^{( Perc )} 내 k번째 행 및 n번째 열의 성분이고,

는 H^{( Perc )} 내 k번째 행 및 n-1번째 열의 성분이고, β는 1보다 큰 실수이다.Here, H ^{( Perc )} is a matrix representing time base bases belonging to percussion group,

Is the component of the k-th row and the n-th column in H ^{( Perc )}

Is the component of the k-th row and the (n-1) -th column in H ^{( Perc )} , and β is a real number greater than one.

를

로 갱신할 수 있다.The audio signal processing apparatus can update the time base bases of the percussion group among the time base bases obtained at 302 using Equation 4 and enhance the time axis discontinuities of the time base bases of the percussion group. The audio signal processing device is a component of the kth row and nth column in H ^{( Perc )} which is a matrix representing time base bases belonging to a percussion group

To

. ≪ / RTI >

Referring to FIG. 3, the audio signal processing apparatus can learn W using Equation (5).

는 W의 m번째 행 k번째 열에 있는 성분이고,

는 H의 k번째 행 n번째 열에 있는 성분이고,

은 스펙트로그램 F의 m번째 행 n번째 열에 있는 성분이며,

는 추정된 스펙트로그램

(=WH)의 m번째 행 n번째 열에 있는 성분이다. 오디오 신호 처리 장치는 비음수행렬분해 알고리즘 중 주파수 축 기저들을 수학식 5를 이용하여 갱신하고, 갱신된 주파수 축 기저들을 학습할 수 있다.Here, < - > is an operator for updating the left term in the right term,

Is the component in the m- th row, k- th column of W,

Is the component in the kth row, nth column of H,

Is the component in the m- th row, n- th column of spectrogram F,

Lt; RTI ID = 0.0 >

(= WH) in the m- th row n- th column. The audio signal processing apparatus can update the frequency axis bases of the non-sound number matrix decomposition algorithm using Equation (5), and learn updated frequency axis bases.

Referring to FIG. 3, the apparatus for processing an audio signal may enhance the frequency axis discontinuity of bases belonging to the group of Mars musical instruments in the frequency axis basis using Equation (6).

는 W^(Harm) 내 m번째 행 및 k번째 열의 성분이고,

는 W^(Harm) 내 m-1번째 행 및 k번째 열의 성분이고, γ는 1보다 큰 실수이다.Here, < - > is an operator for updating the left term to the right term, W ^(Harm) is a matrix representing frequency axis basis belonging to a group of Mars instruments,

Is a component of an m-th row and a k-th column in W ^(Harm)

Is a component of the ⁽ m-1) -th row and the k-th column in W ^(Harm) , and?

를

로 갱신할 수 있다.The audio signal processing apparatus can update the bases of the Mars musical instrument group among the frequency axis bases acquired at 305 using Equation 6 and enhance the frequency axis discontinuity of the frequency axis basis of the Mars musical instrument group. The audio signal processing device is a component of the m-th row and the k-th column in W ^(Harm) which is a matrix representing frequency axis basis belonging to the group of Mars musical instruments

To

. ≪ / RTI >

Referring to FIG. 3, the apparatus for processing an audio signal may enhance the frequency axis continuity of bases belonging to a percussion group among frequency base bases using Equation (307).

는 W^{( Perc )} 내 m번째 행 및 k번째 열의 성분이고,

는 W^{( Perc )} 내 m-1번째 행 및 k번째 열의 성분이고, δ는 0과 1사이의 실수이다.Here, < - > is an operator for updating the left term to the right term, W ^{( Perc )} is a matrix representing frequency base basis belonging to percussion group,

Is the component of the m-th row and the k-th column in W ^{( Perc )}

Is the component of the ⁽ m-1) -th row and the k-th column in W ^{( Perc )} , and δ is a real number between 0 and 1.

를

로 갱신할 수 있다.The audio signal processing apparatus can update the bases of the percussion group among the frequency axis bases acquired at 305 using Equation 7 and enhance the frequency axis continuity of the frequency axis bases of the percussion group. The audio signal processing apparatus is a component of the m-th row and the k-th column in W ^{( Perc )} , which is a matrix representing frequency axis basis belonging to a percussion group

To

. ≪ / RTI >

Referring to FIG. 3, the audio signal processing apparatus can estimate the audio signal of the Mars musical instrument and the audio signal of the percussion instrument using Equation (308).

Here, inverseSpectrogram (*) is an operator for inversely transforming the spectrogram of *, W ^(Harm) is a matrix representing frequency axis basis belonging to a group of Mars instruments, and H ^(Harm) represents a time axis basis belonging to a group of Mars instruments matrix and, W ^(Perc) is a matrix representing the frequency domain base belonging to the percussion group, H ^(Perc) is a matrix that represents the time axis, the base belonging to the percussion group, x ^(Harm) is the audio signal of the converted instrument, x ^(Perc) is an audio signal of a percussion instrument.

The audio signal processing apparatus estimates the spectrogram of the Mars musical instrument using W ^(Harm) and H ^(Harm) obtained through repetition of the operations of 302 to 307 ^, and inversely transforms the estimated spectrogram to generate an audio signal X ^Harm can be estimated. The audio signal processing apparatus estimates a spectrogram of a percussion instrument using W ^(Perc) and H ^{( Perc )} obtained through repetition of the operations of 302 to 307 ^, and inversely transforms the estimated spectrogram to generate an audio signal x ^{( Perc )} .

4 is a diagram for explaining an audio signal processing method according to an embodiment.

Referring to FIG. 4, an audio signal processing apparatus can generate a spectrogram F (401) in which rows and columns are M and N, respectively, from an audio signal. The audio signal processing apparatus can generate matrix H 402 representing time base bases and matrix W (403) representing frequency base bases through matrix decomposition of spectrogram F (401). Here, the row and column of H 402 are K and N, respectively, and the row and column of W 403 may be M and K, respectively.

The audio signal processing apparatus repeatedly performs the operations described with reference to Figs. 2 and 3 to enhance the time-axis continuity of the bases 404 belonging to the H-402 fire-preventing instrument group, The time axis discontinuity of the time axis 405 can be enhanced. Here, the number of the time base bases 404 belonging to the Mars musical instrument group and the number of the time base bases 405 belonging to the percussion group can be respectively defined in advance. For example, if K is set to 750, the time base bases corresponding to the 1 st to 500 th rows are set to the base points 404 of the Mars musical instrument group, the time base bases in the 501 th to 750 th rows are set to base And the audio signal processing apparatus can enhance the time axis continuity and the time axis discontinuity of the bases 404 of the Mars musical instrument group and the bases 405 of the percussion group 405 in the above-described manner, respectively .

The audio signal processing apparatus repeatedly performs the operations described with reference to Figs. 2 and 3 to enhance the frequency axis discontinuity of the bases 406 belonging to the W-instrumental ensemble group 403, The frequency axis continuity of the antenna 407 can be enhanced. Here, the number of the frequency axis bases 406 belonging to the Mars musical instrument group and the number of the frequency axis basis points 407 belonging to the percussion group can be defined in advance, respectively. For example, if K is set to 750, the frequency axis bases corresponding to the 1 st to 500 th rows are set to the basis points 406 of the Mars musical instrument group, and the frequency axis basis on the 501 th to 750 th rows are the basis And the audio signal processing apparatus can enhance the frequency axis discontinuity and the frequency axis continuity of the bases 406 of the Mars musical instrument group and the bases 407 of the percussion group 407 in the above-described manner, respectively .

The audio signal processing apparatus can estimate the spectrogram F ^(Harm) 408 of the Mars musical instrument based on the bases 406 and 404 of the updated Mars musical instrument group. The audio signal processing apparatus can estimate the spectrogram F ^{( Perc )} 409 of the percussion instrument based on the bases 407 and 405 of the updated percussion group. The audio signal processing apparatus can invert the estimated spectrograms F ^(Harm) 408 and F ^{( Perc )} 409 to generate an audio signal of the Mars musical instrument and an audio signal of the percussion instrument, respectively.

5A and 5B are exemplary views of a spectrogram generated by an audio signal processing method according to an embodiment.

FIG. 5A is a spectrogram of a Mars musical instrument generated by an audio signal processing method according to an embodiment, and FIG. 5B is a spectrogram of a percussion instrument generated by an audio signal processing method according to an embodiment. 5A and 5B, it can be confirmed that no other components are mixed in the spectrogram of the Mars musical instrument and the spectrogram of the percussion instrument. An audio signal processing apparatus according to an embodiment can process an audio signal with high resolution and can acquire sounds of a Mars musical instrument and a percussion instrument to be separated from an audio signal.

6 is an exemplary diagram illustrating the components of an audio signal processing apparatus according to an embodiment.

Referring to FIG. 6, an audio signal processing apparatus 601 includes a processor 602 and a memory 603. The memory 603 may store programs for processing or controlling the operation of the audio signal processing apparatus 601 and commands, and programs for implementing the audio signal processing method. The memory may be volatile memory or non-volatile memory.

The processor 602 loads the program from the memory, performs the operations and instructions described above, executes the program in which the audio signal processing method according to one embodiment is implemented, and controls the audio signal processing apparatus 601 . The code of the program executed by the processor 602 may be stored in memory. The audio signal processing apparatus 601 is connected to an external device (for example, a sensor, a user terminal, a personal computer or a network) via an input / output device (not shown) and can exchange data.

The embodiments described above may be implemented in hardware components, software components, and / or a combination of hardware components and software components. For example, the devices, methods, and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, such as an array, a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the drawings, various technical modifications and variations may be applied to those skilled in the art. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (20)

Converting an audio signal to a spectrogram;
Obtaining time base bases and frequency base bases from the spectrogram; And
Separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument included in the audio signal based on at least one of continuity of the time base bases and continuity of the frequency axis bases
Lt; / RTI >
The separating step
Enhancing the continuity of some baselines predetermined to correspond to the Mars musical instrument among the time base bases;
Enhancing the discontinuity of some of the predetermined bases to correspond to the percussion instrument among the time base bases;
Enhancing discontinuities of some bases predetermined to correspond to the Mars musical instrument among the frequency axis bases; And
Enhancing the continuity of some bases predetermined to correspond to the percussion instrument among the frequency axis bases
≪ / RTI >
/ RTI >
The method according to claim 1,
The continuity of the time base bases
Wherein the time base is defined based on differences between components in a time axis direction among components belonging to the time base bases,
/ RTI >
The method according to claim 1,
The continuity of the frequency axis basis
Wherein the frequency axis is defined based on differences between components in the frequency axis direction among components belonging to the frequency axis basis,
/ RTI >
delete The method according to claim 1,
Estimating a spectrogram of the Mars musical instrument based on some bases whose continuity is enhanced to correspond to the Mars musical instrument and some bases whose discontinuity is enhanced to correspond to the Mars musical instrument; And
Converting the estimated spectrogram into the audio signal of the musical instrument;
≪ / RTI >
/ RTI >
The method according to claim 1,
Estimating a spectrogram of the percussion instrument based on some bases whose discontinuity is enhanced to correspond to the percussion instrument and some bases whose continuity is enhanced to correspond to the percussion instrument; And
Transforming the estimated spectrogram into the audio signal of the percussion instrument
≪ / RTI >
/ RTI >
Converting an audio signal to a spectrogram;
Obtaining time base bases and frequency base bases from the spectrogram; And
Separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument included in the audio signal based on at least one of continuity of the time base bases and continuity of the frequency axis bases
Lt; / RTI >
The separating step
Updating the time base bases based on the spectrogram and the frequency axis bases;
Enhancing the time axis continuity of the bases belonging to the Mars musical instrument group among the updated time base bases; And
Enhancing the time axis discontinuities of bases pertaining to percussion groups of the updated time base bases
/ RTI >
/ RTI >
8. The method of claim 7,
The step of enhancing the time axis continuity
Updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the group of Mars musical instruments and the component neighboring the time axis is small,
/ RTI >
/ RTI >
8. The method of claim 7,
The step of enhancing the time axis continuity
^(Harm) , which is a matrix representing bases belonging to the group of Mars musical instruments,

To

≪ / RTI >
Lt; / RTI >
Wherein alpha is a real number between 0 and 1,

Is a component of the k-th row and the (n-1) -th column in the H ^(Harm)
/ RTI >
8. The method of claim 7,
The step of enhancing the time axis discontinuity
Updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the percussion group and the component neighboring the time axis is greater,
/ RTI >
/ RTI >
8. The method of claim 7,
The step of enhancing the time axis discontinuity
^{( Perc )} , which is a matrix representing the bases belonging to the percussion group,

To

≪ / RTI >
Lt; / RTI >
Beta is a real number greater than 1,

Is a component of the k-th row and the (n-1) -th column in the H ^{( Perc )}
/ RTI >
8. The method of claim 7,
Wherein the number of bases belonging to the group of Mars musical instruments and the number of bases belonging to the percussion group are selected from predefined,
/ RTI >
Converting an audio signal to a spectrogram;
Obtaining time base bases and frequency base bases from the spectrogram; And
Separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument included in the audio signal based on at least one of continuity of the time base bases and continuity of the frequency axis bases
Lt; / RTI >
The separating step
Updating the frequency axis baselines based on the spectrogram and the time base bases;
Enhancing the frequency axis discontinuities of bases belonging to the Mars musical instrument group among the frequency axis bases; And
Enhancing the frequency axis continuity of the bases belonging to the percussion group of the frequency base bases
/ RTI >
/ RTI >
14. The method of claim 13,
The step of enhancing the frequency axis discontinuity comprises:
Updating the specific component so that the difference between the specific component in the matrix representing the bases belonging to the group of Mars musical instrument groups and the component neighboring the frequency axis becomes larger,
/ RTI >
/ RTI >
14. The method of claim 13,
The step of enhancing the frequency axis discontinuity comprises:
^(Harm) , which is a matrix representing the bases belonging to the group of Mars musical instruments,

To

≪ / RTI >
Lt; / RTI >
Wherein? Is a real number greater than 1,

Is a component of the ⁽ m-1) -th row and the k-th column in the W ^(Harm)
/ RTI >
14. The method of claim 13,
The step of enhancing the frequency axis continuity comprises:
Updating the specific component so that the difference between the specific component in the matrix representing the bases pertaining to the percussion group and the component neighboring the frequency axis is small
/ RTI >
/ RTI >
14. The method of claim 13,
The step of enhancing the frequency axis continuity comprises:
^{( Perc )} , which is a matrix representing the bases belonging to the percussion group,

To

≪ / RTI >
Lt; / RTI >
Is a real number between 0 and 1,

⁽ K-1) -th row and k-th column in the W ^{( Perc )}
/ RTI >
14. The method of claim 13,
Wherein the number of bases belonging to the group of Mars musical instruments and the number of bases belonging to the percussion group are selected from predefined,
/ RTI >
18. A computer program stored on a medium for executing the method of any one of claims 1 to 3 and 5 to 18 in combination with hardware.
Comprising: converting an audio signal to a spectrogram, obtaining time-base bases and frequency-axis bases from the spectrogram, and based on at least one of continuity of the time-base bases and continuity of the frequency- A processor for separating the audio signal of the Mars musical instrument and the audio signal of the percussion instrument
Lt; / RTI >
The processor
Enhancing the continuity of some of the time base bases predetermined to correspond to the Mars musical instrument; enhancing the discontinuity of some bases predetermined to correspond to the percussion instrument of the time base bases; Wherein the at least one of the frequency axis bases is selected from a group consisting of a plurality of frequency bands and a plurality of frequency bands,
Audio signal processing device.

Images