KR100891665B1 - Apparatus for processing a mix signal and method thereof - Google Patents

Abstract

The present invention relates to a method and apparatus for processing a mixed signal. To this end, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Obtaining first additional information; Obtaining a mix parameter; Generating a remix signal using the mix signal, the first additional information, and the mix parameter, wherein the first additional information is related to a source signal to be remixed among the source signals and the mix signal. A signal processing method is provided, wherein the second additional information indicated is modified information.

Mix signal, side information, side information transformation

Description

Method and apparatus for processing mixed signal {APPARATUS FOR PROCESSING A MIX SIGNAL AND METHOD THEREOF}

1 is a block diagram of an apparatus for encoding a first remix signal according to an embodiment of the present invention.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when using a stereo signal. FIG.

3 is a domain for processing a mix signal according to an embodiment of the present invention.

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention.

FIG. 6 is a detailed view of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. FIG.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention.

8B is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention used in combination with a conventional decoding apparatus.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention.

10 is a block diagram of an apparatus for encoding a remix signal according to an embodiment of the present invention.

FIG. 11 is a detailed block diagram of the remix signal encoding apparatus of FIG. 10. FIG.

12 is a block diagram of a remix signal encoding apparatus according to an embodiment of the present invention.

13 is a block diagram illustrating an additional information processing unit according to an embodiment of the present invention.

14 is a transcoder for converting additional information to be suitable for a newly applied mix signal according to an embodiment of the present invention.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention.

16 is a flowchart illustrating a method of processing additional information according to an embodiment of the present invention.

17 is a flowchart illustrating a signal processing method according to an embodiment of the present invention.

The present invention relates to a method and apparatus for processing a mixed signal. To date, stereo signals are most commonly generated as mix signals and most widely used by consumers. In recent years, multichannel signals have been increasingly used. However, there is a limitation that the mixed signal is processed in units of channels, not in units of source signals constituting the mix signal. Therefore, when processing a mix signal in units of channels, there is a problem in that only a specific source signal constituting the mix signal cannot be independently processed. For example, while watching a movie, it is impossible to increase the volume of background music while keeping the volume of actors' voices constant.

In order to solve the above problems, an object of the present invention is to provide an encoding method and apparatus for transforming additional information into a low information amount.

Another object of the present invention is to provide a method and apparatus for decoding a mixed signal, which can control a mixed signal in units of a source signal using modified additional information.

In order to achieve the above object, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Obtaining first additional information; Obtaining a mix parameter; Generating a remix signal using the mix signal, the first additional information, and the mix parameter, wherein the first additional information is related to a source signal to be remixed among the source signals and the mix signal. A signal processing method is provided, wherein the second additional information indicated is modified information.

In addition, to achieve the above object, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Obtaining a source signal to be remixed among the source signals; Generating first side information using the mix signal and the source signal to be remixed; And transforming the first additional information into second additional information, wherein the first additional information is information indicating a relationship between the source signal to be remixed and the mixed signal. do.

In addition, in order to achieve the above object, the present invention comprises the steps of obtaining a first mix signal and the first additional information; Obtaining a second mix signal; And modifying the first additional information and the second additional information by using a result of comparing the first mixed signal and the second mixed signal, wherein the first additional information is used to remix the first mixed signal. The second additional information is information necessary for remixing the second mixed signal.

In addition, in order to achieve the above object, the present invention provides a mixed signal decoding unit for obtaining a mixed signal including at least one source signal; An additional information decoding unit obtaining first additional information; And a remix rendering unit configured to generate a remix signal using the mix signal, the first additional information, and the mix parameter, wherein the first additional information is a relationship between the source signal to be remixed among the source signals and the mix signal. The second additional information indicating the modified information, and the mix parameter provides a signal processing apparatus, characterized in that generated using the control information obtained from the user.

In addition, in order to achieve the above object, the present invention provides an additional information generation unit for generating a first additional information using a mix signal including at least one source signal and the source signal to be remixed; An additional information transformation unit which transforms the first additional information into second additional information; And an additional information encoding unit encoding the second additional information, wherein the first additional information is information indicating a relationship between the source signal to be remixed and the mixed signal.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

The present invention provides an algorithm that can process a mix signal in a unit of a source signal. When the mix signal is processed in units of source signals, a wide variety of effects can be produced. For example, while watching a movie, it is possible to increase only the volume of the background music while keeping the volume of actors' voices constant. In the present invention, the source signal includes one or more sources (eg, pianos) constituting the mix signal. Processing in source signal units means that in processing a mix signal, characteristics (eg, localization, gain) associated with a particular source signal constituting the mix signal are " individually ". It can be modified. By “individually” it is meant that modifying the properties associated with a particular object does not affect the properties of other source signals or only has a small effect that is difficult to perceive.

The mix signal includes mono, stereo, and multichannel signals. For convenience of explanation, hereinafter, a stereo signal will be described as an example, but the present invention is not limited to the stereo signal. In the present invention, a small amount of side information is transmitted as well as a typical mix signal format (for example, PCM, MP3, MPEG-AAC). The remix signal may be generated using the mixed signal and the additional information.

1 is a block diagram of a first remix signal encoder according to an embodiment of the present invention. The first remix signal encoder includes an additional information generator 103 and an additional information encoder 105.

Referring to FIG. 1, the additional information generator 103 generates additional information 104 using a typical mix signal 101 and a source signal 102 constituting the mixed signal. The mix signal 101 may be a mono, stereo, and multi-channel audio signal. The source signal 102 may be some or all of the source signals constituting the mix signal 101. The additional information 104 refers to information used to process the mixed signal in source signal units. The additional information 104 includes mix parameters for remixing the mix signal. The mix parameter may include an encoder mix parameter generated using a source signal in an encoder, and optionally, a blind mix parameter generated using only a mix signal. Examples of the mix parameter may include a gain value and a subband power for each source signal. A detailed definition and generation method for the additional information 104 is described in FIG. 2. The present invention also includes generating the additional information 104 using only the source signal 102 constituting the mix signal. The side information encoding unit 105 encodes the generated side information 104 to generate an encoded side information signal 106. The mix signal 101 and the side information signal 106 are transmitted to a decoding apparatus.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when the mixed signal is a stereo signal. As described above, the mixed signal used in the present invention may be a mono, stereo, and multi-channel audio signal, but for convenience, it will be described with reference to the stereo signal 201.

및

는 상기 스테레오 신호를 구성하는 소스 신호들의 합으로 표현될 수 있다. 여기서, n은 타임 인덱스를 의미한다. 따라서, 상기 스테레오 신호(201)는 아래의 [수학식 1]과 같이 표현될 수 있다.The stereo signal 201

And

May be expressed as a sum of source signals constituting the stereo signal. Here, n means time index. Therefore, the stereo signal 201 may be expressed as Equation 1 below.

은 소스 신호들을 나타낸다.

및

는 각각의 소스 신호에 대한 진폭 패닝(amplitude panning) 및 게인(gain)을 결정하는 값이다. 모든

들은 서로 독립적이다. 상기

는 모두 순수한 소스 신호이거나, 또는 순수한 소스 신호에 약간의 잔향(reverberation) 및 효과음 신호성분(sound effect signal components)을 포함할 수 있다. 예를 들면, 특정한 잔향 신호성분은 2개의 소스 신호, 즉, 왼쪽 채널로 믹스된 신호와 오른쪽 채널로 믹스된 신호로 표현될 수 있다.Here, I is the number of source signals included in the stereo signal,

Represents source signals.

And

Is a value that determines the amplitude panning and gain for each source signal. all

They are independent of each other. remind

Are all pure source signals, or may include some reverberation and sound effect signal components in the pure source signal. For example, the specific reverberation signal component may be represented by two source signals, that is, a signal mixed with the left channel and a signal mixed with the right channel.

It is an object of the present invention to modify a stereo signal comprising the source signal such that M (0 <= M <= I) source signals are remixed. The source signals may be remixed into a stereo signal with different gain factors. The remix signal may be expressed as Equation 2 below.

및

는 리믹스되는 M개의 소스 신호들에 대한 새로운 게인 팩터들이다. 상기

및

는 디코더 단에서 제공될 수 있다. 이 경우에, 부가정보생성부(206)는 스테레오 신호(201) 및 M개의 소스 신호(202)를 이용하여 부가 정보(207)를 생성할 수 있다. here,

And

Are new gain factors for the M source signals to be remixed. remind

And

May be provided at the decoder stage. In this case, the additional information generator 206 may generate the additional information 207 using the stereo signal 201 and the M source signals 202.

에 대한 접근 없이, [수학식 1]로 표현되는 통상적인 믹스 신호가 주어지는 경우에, [수학식 2]로 표현되는 리믹스 신호를 지각적으로 모방하는 것을 목적으로 한다.As described above, an object of the present invention is to remix the stereo signal in the unit of the source signal when given a conventional stereo signal and some additional information. As in the present invention, it is not possible to completely generate the remix signal represented by Equation 2 from the mixed signal represented by Equation 1 using a very small amount of additional information. Thus, the present invention provides the respective source signals

It is aimed to perceptually mimic the remix signal represented by [Equation 2], given an ordinary mix signal represented by [Equation 1] without access to.

및

로 표시된다. 여기서 k는 서브밴드 신호들의 시간 인덱스(time index)이다. 유사하게, M개의 소스 신호들(202)의 서브밴드 신호들(205)은

,

,...,

로 표시된다. 명료한 표현을 위해, 서브밴드(주파수) 인덱스를 사용하지 않았다.Referring to FIG. 2, a stereo signal 201 and M source signals 202 included in the stereo signal 201 are input to a first remix signal encoding apparatus. The stereo signal 201 may be delayed to some extent to be synchronized with the side information and used directly as an output signal. To generate the additional information, the stereo signal 201 and the source signals 202 are decomposed into subband-specific signals 204 and 205 in the time-frequency domain through the filter bank 203. That is, the stereo signal 201 and the source signal are processed in the time-frequency domain, which will be described later with reference to FIG. 3. The subband-specific signal 204 is similarly processed at the center frequency of each subband. At a particular frequency, the subband pair 204 of the stereo signal 201

And

Is displayed. Where k is the time index of the subband signals. Similarly, the subband signals 205 of the M source signals 202 may be

,

, ...,

Is displayed. For clarity, no subband (frequency) index is used.

를 생성한다. 또한, 상기 부가정보생성부(206)는 스테레오 신호(201)의 서브밴드 쌍(204)을 이용하여, 서브밴드별로 게인 팩터

및

를 생성한다. 상기 게인 팩터

및

는 외부에서 직접 주어질 수 있다. 상기 서브밴드별 숏-타임 서브밴드 파워 및 게인 팩터를 이용하여 서브밴드별 부가 정보(207)가 생성된다. 상기 부가정보생성부(206)는 상기 숏-타임 서브밴드 파워 및 게인 팩터들 이외에 상기 스테레오 신호에 관련된 다른 정보를 부가 정보(207)로 생성할 수 있다. 부가정보인코딩부(208)는 상기 서브밴드별 부가정보(207)를 이용하여 부호화된 부가 정보 신호(209)를 생성한다. Given the subband signals 205 of the source signals 202, the side information generator 206 may perform short-time subband power for each subband,

Create In addition, the additional information generator 206 uses a subband pair 204 of the stereo signal 201 to obtain a gain factor for each subband.

And

Create The gain factor

And

Can be given directly from the outside. Subband-specific additional information 207 is generated using the short-time subband power and gain factor for each subband. The additional information generator 206 may generate other information related to the stereo signal as additional information 207 in addition to the short-time subband power and gain factors. The additional information encoding unit 208 generates an additional information signal 209 encoded using the additional information 207 for each subband.

및

는 고정적이 될 것이다. 만일

및

가 시간 k에 따라 가변적이라면, 상기 게인 팩터들은 시간의 함수로 생성될 것이다. 상기 게인 팩터들은 직접 양자화 및 부호화되지 않고, 먼저 양자화 및 부호화에 더 적합한 다른 값들로 전환될 수 있다. 또한,

는 스테레오 신호(201)의 서브밴드 파워에 상대적인 값으로 정규화될 수 있다. 이것은 스테레오 신호를 효율적으로 부호화하기 위해 통상적인 인코딩 장치가 이용되는 경우에, 본 발명을 상대적으로 변화에 강하도록 만들어준다. 예를 들면,

및

는 아래의 [수학식 3]으로 표현되는 게인 및 데시벨(dB) 단위의 레벨차로 전환되어 전송될 수 있다. For many stereo signals 201, the gain factor

And

Will be fixed. if

And

If is variable over time k, the gain factors will be generated as a function of time. The gain factors are not directly quantized and coded, but may first be converted to other values more suitable for quantization and coding. Also,

Can be normalized to a value relative to the subband power of the stereo signal 201. This makes the present invention relatively resistant to changes when conventional encoding devices are used to encode stereo signals efficiently. For example,

And

May be converted into a level difference in units of gain and decibel (dB) expressed by Equation 3 below, and then transmitted.

는 부가 정보로서 직접 부호화되는 것이 아니라, 아래의 [수학식 4]로 표현되는 스테레오 신호에 상대적으로 정의된 값으로 변환되어 전송될 수 있다. Also,

Is not directly encoded as additional information, but may be converted into a value defined relative to the stereo signal represented by Equation 4 below and transmitted.

는 아래의 [수학식 5]와 같이 계산될 수 있다. To produce short-time subband power, the present invention uses single-pole averaging. In other words,

May be calculated as shown in Equation 5 below.

Here, α∈ [0,1] determines the time-constant of the estimation window (estimation window) which decreases exponentially as shown in Equation 6 below.

는 서브밴드 샘플링 주파수를 나타낸다. 예를 들면, T=40 ms를 이용할 수 있다. 이하에서,

는 숏-타임 평균(short-time averaging)을 나타낸다. 만일

및

가 주어지지 않는다면, 상기

및

는 부가정보생성부(206)에서 생성될 필요가 있다.

이므로,

는 아래의 [수학식 7]과 같이 계산된다.here,

Denotes a subband sampling frequency. For example, T = 40 ms can be used. In the following,

Denotes short-time averaging. if

And

If is not given,

And

Needs to be generated in the additional information generation unit 206.

Because of,

Is calculated as shown in Equation 7 below.

는 아래의 [수학식 8]과 같이 계산된다.Similarly,

Is calculated as shown in Equation 8 below.

3 illustrates a domain for processing an audio signal according to an embodiment of the present invention. As mentioned above, the audio signal and the additional information are processed as subband-specific signals in the time-frequency domain as shown in FIG. Subband-specific signals in the time-frequency domain are perceptually derived. For example, a signal for each subband may be generated by using a short time fourier transform (STFT) having a sine wave analysis window and a sine analysis and synthesis window having a length of about 20 ms. In this case, the STFT coefficients may be grouped such that one group has a bandwidth that is about twice the equivalent rectangular bandwidth (ERB).

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention. The second remix signal encoding apparatus includes a downmixing unit 402, an additional information generating unit 403, and an additional information encoding unit 406.

Referring to FIG. 4, the downmixing unit 402 generates a sum signal 404 by adding a plurality of source signals 401. Unlike the first remix signal encoding apparatus, the second remix signal encoding apparatus transmits the sum signal 404 instead of the stereo signal. The additional information generator 403 generates the additional information 405 using the source signals 401. The additional information 405 includes a subband power and a gain factor corresponding to each source signal. In addition, the additional information 405 may include a parameter corresponding to a delay in the remix renderer. Similar to the first remix signal encoding apparatus, the additional information 405 may be converted into another value more suitable for quantization and encoding and transmitted. The additional information encoding unit 406 generates an encoded additional information signal 407 using the generated additional information 405. The generated sum signal 404 and the side information signal 407 are transmitted to the decoding apparatus. The present invention also includes an encoding apparatus that does not have a downmixing unit 402. In this case, the source signals 401 are not converted to the sum signal 404, and each source signal 401 is transmitted directly.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention. The first remix signal decoding apparatus includes a side information decoder 503 and a remix renderer 505.

Referring to FIG. 5, the mix signal 501 and the additional information signal 502 are input to the first remix signal decoding apparatus. The mix signal 501 may be a mono, stereo or multichannel audio signal. The additional information decoding unit 503 decodes the additional information signal 502 to generate additional information 504. The additional information 504 includes a gain factor and subband power of source signals included in the transmitted audio signal 501. The remix renderer 505 may input a user-mix parameter 506 generated using control information directly provided by the user. The remix renderer 505 generates the remix signal 507 using the mix signal 501, the transmitted additional information 504, and the user mix parameter 506. A detailed description of the method of generating the remix signal will be described later with reference to FIG. 6. The remix signal 507 is generated as an Eq-channel mix signal having the same number of channels as the number of channels of the transmitted mix signal, or an upchannel mix signal having more channels than the number of channels of the mix signal. (Up-channel mix signal) can be generated.

FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. As described above, the transmitted mixed signal may be a mono, stereo, and multichannel audio signal, but for convenience, the stereo signal 601 will be described.

및

로 표현된다. 부가정보디코딩부(605)는 전송된 부가 정보 신호(602)를 복호화하여, 서브밴드별 부가 정보(606)를 생성한다. 또한, 리믹스 렌더링부(607)에 사용자가 제공하는 제어 정보를 이용하여 생성된 사용자 믹스 파라미터(608)가 입력될 수 있으며, 상기 사용자 믹스 파라미터(608)는 서브밴드별로 제공될 수 있다. 전술한 것처럼, 상기 부가 정보(606)는 리믹스 될 M개의 소스 신호에 대한 서브밴드별 게인 팩터(

및

) 및

로 표현되는 서브밴드 파워를 포함한다. 리믹스 렌더링부(607)는 서브밴드별로 생성된 스테레오 신호(604), 전송된 부가 정보(606) 및 사 용자 믹스 파라미터(608)를 이용하여, 서브밴드별 리믹스 신호(609),

및

를 생성한다. 상기 리믹스 신호(609)를 생성하는 방법은 아래에서 더욱 상세하게 기술된다. 상기 리믹스 신호(609)는 역필터뱅크(610)를 통해 시간 도메인(time domain)의 스테레오 신호(611),

및

로 변환된다. Referring to FIG. 6, the stereo signal 601 is decomposed into a subband-specific signal 604 in the time-frequency domain through the filter bank 603. As shown in FIG. 6, the subband-specific signal 604 at a particular frequency

And

It is expressed as The additional information decoding unit 605 decodes the transmitted additional information signal 602 to generate additional information 606 for each subband. In addition, a user mix parameter 608 generated using control information provided by a user may be input to the remix renderer 607, and the user mix parameter 608 may be provided for each subband. As described above, the additional information 606 is a subband-specific gain factor for M source signals to be remixed.

And

) And

It includes the subband power represented by. The remix rendering unit 607 uses the stereo signal 604 generated for each subband, the additional information 606 transmitted, and the user mix parameter 608, and the remix signal 609 for each subband,

And

Create The method of generating the remix signal 609 is described in more detail below. The remix signal 609 is a stereo signal 611 in the time domain through the inverse filter bank 610,

And

Is converted to.

는 서브밴드별 소스 신호

로 교체된다. 즉, 서브밴드별 믹스 신호(604)는 아래의 [수학식 9]과 같이 표현될 수 있다. A method of generating the remix signal 609 generated by the remix renderer 607 is as follows. [Equation 1] and [Equation 2] are also valid for the subband signals 604 and 609. In this case, the source signal

Is the subband-specific source signal

Is replaced by. That is, the mix signal 604 for each subband may be expressed by Equation 9 below.

The subband remix signal 609 may be expressed by Equation 10 below.

및

가 주어지면, 아래의 [수학식 11]와 같이 서로 다른 게인들을 가지는 서브밴드별 리믹스 신호(609)가 상기 서브밴드별 믹스 신호(604)의 선형 조합으로 추정될 수 있다.To generate the remix signal 609, least squares estimation may be used. Per subband mix signal 604,

And

Given by Equation 11 below, the subband remix signal 609 having different gains may be estimated as a linear combination of the subband mix signals 604.

,

,

및

는 가중 팩터들(weighting factors)이다. 이때, 생성되는 추정 에러(estimation error)는 아래의 [수학식 12]과 같이 정의될 수 있다.here,

,

,

And

Is the weighting factors. In this case, the generated estimation error may be defined as shown in Equation 12 below.

,

,

및

는 평균제곱오차(mean square error),

및

가 최소가 되도록 서브밴드별로 생성될 수 있다. 이때, 추정 에러,

및

가

및

에 직교(orthogonal)될 때, 상기 평균제곱오차가 최소가 된다는 것을 이용할 수 있다. 생성되는

및

는 아래의 [수학식 13]과 같이 표현될 수 있다.The weighting factors,

,

,

And

Is the mean square error,

And

May be generated for each subband such that is minimized. In this case, the estimation error,

And

end

And

When orthogonal to, it can be used that the mean square error is minimized. Generated

And

May be expressed as Equation 13 below.

,

및

는 직접 생성될 수 있지만,

및

은 전송된 부가 정보(606)(예를 들면,

,

,

) 및 사용자가 제공하는 제어 정보(608)(예를 들면, 게인 팩터

및

)를 이용하여, 아래의 [수학식 14]와 같이 생성될 수 있다.here,

,

And

Can be generated directly,

And

Is transmitted additional information 606 (e.g.,

,

,

) And control information 608 provided by the user (e.g., gain factor

And

) Can be generated as shown in Equation 14 below.

및

가 아래의 [수학식 15]와 같이 생성될 수 있다.Similarly,

And

May be generated as shown in Equation 15 below.

및

는 아래의 [수학식 16]과 같이 표현될 수 있다. here,

And

May be expressed as Equation 16 below.

If the phases of the mix signal 604 are coherent or nearly synchronized with each other, a value expressed by Equation 17 below approaches 1.

In this case, the weights may be expressed as Equation 18 below.

및

)를 이용하여 각각의 소스 신호를 독립적으로 리믹스하여 생성된 리믹스 신호와 유사하게 들린다.The subband remix signal 609 generated as described above is converted into the remix signal 611 of the time-domain through the inverse filter bank 610 as described above. The remix signal 611 may generate a user mix parameter generated using control information provided by a user.

And

It sounds similar to the remix signal generated by independently remixing each source signal with

So far, the focus has been on remixing two-channel stereo signals. However, as described above, the present invention is not limited to stereo signals, but may be extended to remixing multichannel audio signals, for example, 5.1 channel audio signals. Those skilled in the art can remix multichannel audio signals, similar to the stereo signals described herein. In this case, Equation 11 may be written as Equation 19 below.

Optionally, one of the channels of the mix signal can be left without remixing. For example, for a 5.1 surround channel, it is possible to apply remixing only to the front channel without modifying the two back channels. In this case, a two or three channel remix algorithm is applied to the front channel.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. The second remix signal decoding apparatus includes a side information decoding unit 703, a spatial information integrating unit 705, and a remixing rendering unit 707.

Referring to FIG. 7, the sum signal 701 of the source signals and the additional information signal 702 are input to the second remix signal decoding apparatus. The additional information decoding unit 703 decodes the additional information signal 702 to generate additional information 704. The additional information 704 includes a gain factor, a delay constant, a subband power, and the like. The additional information integrator 705 separates the sum signal 701 into a plurality of source signals 706 by using the additional information 704. The remix renderer 707 may generate the remix signal 709 using the source signals 706. In this case, the remix renderer 707 may generate the remix signal 709 using the mix parameter transmitted as additional information. In addition, the remix renderer 707 may selectively generate the remix signal 709 using the user mix parameter 708 generated by using control information provided by the user.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. The mixed signal 801 may be encoded by the conventional encoding apparatus 803 and converted into the encoded mixed signal 805. The mix signal 801 may be a channel-specific signal or a source signal. The conventional encoding device 803 includes not only conventional encoding devices such as AAC, MP3 encoder, etc., but also encoding devices to be developed in the future. The remix signal encoding apparatus 804 according to the present invention generates the additional information signal 806 using the mixed signal 801 and the source signal 802 included in the mixed signal. The multiplexer 807 generates the bitstream 808 using the encoded mix signal 805 and the side information signal 806. As described above, the additional information signal 806 may be inserted into an auxiliary data area in the conventional mix signal format so as to be compatible with conventional devices.

8B is a block diagram of a combination of a conventional decoding device and a remix signal decoding device according to an embodiment of the present invention. The demultiplexer 810 separates the encoded mix signal 811 and the side information signal 812 from the transmitted bitstream 809. The conventional decoding device 813 then decodes the encoded mix signal 811 to produce a mix signal 814 that can be used in the remix signal decoding device 815 according to the present invention. The conventional decoding device 813 includes not only conventional encoding devices such as AAC, MP3 decoder, etc., but also encoding devices to be developed in the future. The mix signal 814 may be a channel-specific signal or a source signal. The remix signal decoding apparatus 817 according to the present invention may convert the mix signal 814 into the remix signal 816 using at least one of the additional information signal 812 and the user mix parameter 817.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention. Referring to FIG. 9, the remix signal decoding apparatus includes a mixed signal decoder 901, a parameter generator 902, and a remix renderer 908. It may optionally include an effector (911). The parameter generator 902 may include a blind mix parameter generator 903, a user mix parameter generator 904, and a remix parameter generator 905. The remix parameter generation unit 905 may include an equimix parameter generation unit 906 and may optionally include an upmix parameter generation unit 907. In addition, the remix renderer 908 may include an equimix renderer 909 and optionally include an upmix renderer 910.

The mixed signal decoding unit 901 decodes the encoded mixed signal transmitted from the encoding end to generate a mixed signal. The parameter generator 902 receives additional information and user control information (or configuration information) transmitted from an encoding stage. The user control information is not transmitted at the encoder stage but may be generated at the decoder stage. The user mix parameter generator 904 generates a user mix parameter by using user control information. The additional information transmitted from the encoder stage may include an encoder mix parameter. In addition, the blind mix parameter generator 903 may generate a blind mix parameter using the mix signal. The encoder mix parameter and the blind mix parameter are alternatively input to the remix parameter generator 905.

The remix parameter generator 905 generates a remix parameter using additional information and a user mix parameter. The remix parameter may be generated to be applied to a channel of the remix signal. Equimix parameter generation unit 906 included in the remix parameter generation unit 905 generates a remix parameter used to generate a remix signal having the same channel number as the number of channels of the mix signal, the remix parameter generation unit 905 The upmix parameter generation unit 907, which may be included in the control unit, generates a remix parameter used to generate a remix signal having a larger number of channels than the number of channels of the mixed signal. The remix parameter is input to the remix renderer 908.

Equimix renderer 909 included in the remix renderer 908, using the remix parameter and the mix signal, having an equal number of channels equal to the number of channels of the mix signal (Eq-channel remix signal ) The upmix renderer 910, which may be included in the remix renderer 908, uses more channels than the number of channels of the mix signal by using the remix parameter and the mix signal generated by the upmix parameter generator 907. Generate an up-channel remix signal having a number. The upmix renderer 910 may generate an upchannel remix signal using the remix signal generated by the Equichannel renderer 909.

Accordingly, the decoding apparatus may output the mixed signal transmitted from the encoding stage as it is, output as an equal channel remix signal, or output as an up channel remix signal. Optionally, the remix renderer may give various effects to the remixed signal by using information provided from the effector 911.

10 is a block diagram of an encoding apparatus according to an embodiment of the present invention. Referring to FIG. 10, the encoding apparatus includes an additional information processing unit 1003. The encoding apparatus may directly transmit the mixed signal 1001 to the decoding apparatus. However, when there are a plurality of the mixed signals 1001, the encoding apparatus 100 may downmix the mixed signal 1001 into one or two downmix signals and transmit the mixed signals 1001. Although not shown in the drawing, the mix signal 1001 may be encoded and transmitted for transmission efficiency of the mix signal.

The additional information processing unit 1003 generates additional information using the mix signal 1001 and the source signal 1002. The source signal 1002 may be included in the mix signal 1001 or may be a separate source signal that is not included in the mix signal 1001. The additional information includes level information, time delay information, cross correlation information, mix information, and the like.

Here, the level information may include the level of the source signal to be remixed, and may include a relative level between the source signals to be remixed or a relative level between the source signal and the mixed signal to be remixed. In addition, the level of the mix signal may be separately included in the additional information. The time delay information may include time delay information between the source signals to be remixed, or time delay information between the source signal and the mixed signal to be remixed. The cross-correlation information may include cross-correlation information between source signals to be remixed, cross-correlation information between a source signal and a mix signal to be remixed, and cross-correlation information between the mix signals. Mix information indicates how much of a particular source is mixed into the mix signal. For example, if you want to mix a particular source to the right, you can mix it so that the right channel is larger than the left channel. Can be done. As such, the mix information may indicate the degree to which each source is mixed to each channel. The mix information may include not only the size but also information such as time delay and correlation related to the mix.

The decoding apparatus generates a remix signal using the transmitted additional information and the mix signal. At this time, it is possible to hear a specific source signal louder or smaller according to the user's request. It is also possible to change the sound position of a particular source signal to achieve a particular effect. Such a process may be configured by generating original source signals through additional information and mixed signals and then remixing them. Alternatively, it is possible to generate the remix signal by modifying the mix signal using the additional information and the user control information without generating the original source signals.

The additional information processing unit 1003 may modify the generated additional information. That is, the additional information processing unit 1003 can reduce the amount of additional information by normalizing the additional information or setting some of the additional information as a default value. The additional information processing unit 1003 encodes the modified additional information and transmits the encoded additional information to the decoding apparatus.

및

는 믹스 신호(1101)에 포함되는 소스 신호(1102)의 합으로 표현될 수 있다.FIG. 11 is a detailed block diagram of the encoding apparatus of FIG. 10. Referring to FIG. 11, the encoding apparatus includes a filter bank 1103 and an additional information processing unit 1003. The additional information processing unit 1003 includes an additional information generating unit 1106, an additional information modifying unit 1108, and an additional information encoding unit 1110. In the present invention, the mix signal includes mono, stereo, and multichannel signals, and for convenience, the mix signal 1101 will be described with reference to FIG. 11. As shown in Equation 1 above, the mix signal 1101

And

May be expressed as the sum of the source signals 1102 included in the mix signal 1101.

은 소 스 신호들을 나타낸다.

및

는 각각의 소스 신호에 대한 진폭 패닝(amplitude panning) 및 게인(gain)을 결정하는 게인 팩터이다. 소스 신호(1102)는 서로 다른 게인 팩터들을 가지면서 믹스 신호(1101)에 믹스된다. 필터뱅크(1103)는 믹스 신호(1101) 및 소스 신호(1102)를 시간-주파수 도메인의 서브밴드별 신호(1104 및 1105)로 분해한다. 부가정보생성부(1106)는 서브밴드별 믹스 신호(1104) 및 서브밴드별 소스 신호(1105)를 이용하여, 게인 팩터

,

및 숏-타임 서브밴드 파워

(1107) 등과 같은 서브밴드별 부가 정보를 생성한다.Here, I is the number of source signals included in the mix signal,

Indicates source signals.

And

Is a gain factor that determines the amplitude panning and gain for each source signal. The source signal 1102 is mixed with the mix signal 1101 with different gain factors. The filter bank 1103 decomposes the mix signal 1101 and the source signal 1102 into subband-specific signals 1104 and 1105 in the time-frequency domain. The additional information generator 1106 uses a subband mixed signal 1104 and a subband source signal 1105 to obtain a gain factor.

,

And short-time subband power

Subband-specific additional information such as 1107 is generated.

,

를 값은 동일하나 새로운 게인 팩터와 새로운 소스 신 호를 이용하여

,

와 같이 표현할 수 있다. The additional information modifying unit 1108 transforms the additional information 1107 for each subband. That is, the side information transformation unit 1108 transforms the subband side information 1107 into new side information 1109 such as a new gain factor and a new short-time subband power. The side information transformation unit 1108 modifies the side information such that the mixed signal expressed using the new side information 1109 has the same value as the original mix signal 1101. That is, the mix signal 1101 is expressed as the product of the gain factor and the source signal.

,

Is the same, but using a new gain factor and a new source signal

,

It can be expressed as

,

중 하나를 디폴트값으로 설정하면, 인코딩 장치는 디폴트값으로 설정된 게인 팩터를 전송할 필요가 없게 된다. 인코딩 장치는 부가 정보를 양자화 및 부호화에 더 적합한 다른 값으로 전환할 수도 있다. 부가정보인코딩부(1110)는 변형된 부가 정보(1109)를 인코딩하여 디코딩 장치로 전송하거나, 또는 변형된 부가 정보(1109)를 양자화 및 부호화에 적합한 값으로 전환한 후, 이를 인코딩하여 디코딩 장치로 전송한다. In this case, the new gain factor

,

If one of the default values is set, the encoding apparatus does not need to transmit the gain factor set to the default value. The encoding apparatus may convert the side information into another value more suitable for quantization and encoding. The additional information encoding unit 1110 encodes and transmits the modified additional information 1109 to the decoding apparatus, or converts the modified additional information 1109 to a value suitable for quantization and encoding, and then encodes the modified additional information 1109 to the decoding apparatus. send.

12 is a block diagram of an encoding apparatus according to an embodiment of the present invention. Referring to FIG. 12, the encoding apparatus includes a downmixing unit 1202 and an additional information processing unit 1003, and the additional information processing unit 1003 includes an additional information generation unit 1203, an additional information transformation unit 1204, and An additional information encoding unit 1205 is included. The difference between the encoding device shown in FIG. 11 and the encoding device shown in FIG. 12 lies in the information used to generate additional information. The encoding apparatus illustrated in FIG. 11 generates additional information using a source signal and a mix signal, but the encoding apparatus illustrated in FIG. 12 generates additional information using only a source signal.

The downmixer 1202 downmixes the source signal 1201 and transmits the downmixed signal to the decoding apparatus. The additional information processing unit 1003 generates additional information, encodes it, and transmits the additional information to the decoding apparatus. The additional information generator 1203 may generate additional information by using all or part of the source signal 1201. The additional information transforming unit 1204 transforms the additional information generated by the additional information generating unit 1203 into new additional information for transmission efficiency. The side information transformation unit 1204 may convert the modified side information into a value suitable for quantization, encoding, and the like. The additional information encoding unit 1205 encodes the modified additional information and transmits the encoded additional information to the decoding apparatus, or converts the modified additional information for quantization, and then encodes the converted additional information and transmits the encoded additional information to the decoding apparatus.

,

및 숏-타임 서브밴드 파워

를 포함한다. 부가정보변형부(1302)는 부가 정보를 변형하여 새로운 게인 팩터

,

및 새로운 숏-타임 서브밴드 파워

등과 같은 새로운 부가 정보를 생성한다. 13 is a block diagram illustrating an additional information processing unit according to an embodiment of the present invention. Referring to FIG. 13, the additional information processing unit 1003 includes an additional information generation unit 1301, an additional information transformation unit 1302, an additional information conversion unit 1303, an additional information quantization unit 1304, and an additional information encoding unit ( 1305). The additional information processor 1003 generates additional information, processes the information, and transmits the additional information to the decoding apparatus. The additional information generation unit 1301 included in the additional information processing unit 1003 generates additional information using at least one of a mix signal and a source signal. The additional information is a gain factor

,

And short-time subband power

It includes. The side information transformation unit 1302 transforms the side information to form a new gain factor.

,

And new short-time subband power

Create new additional information such as

, 또는

을 전송한다. 여기서,

은 i번째 소스 신호를 의미한다. 상기 소스 신호의 도메인은 시간 영역일 수 있고, 또는 서브밴드 도메인일 수 있다. 이와 같은 경우, 상기 레벨 값의 변화가 커서 양자화하는데 어려움을 격을 수 있다. 이를 해결하기 위해,

와 같이 특정 소스 신호에 대한 상대적인 값을 전송하거나, 또는

과 같이 믹스 신호의 레벨에 대한 상대적인 값을 전송할 수 있다. 여기서,

은 믹스 신호를 의미한다. The additional information transformation unit 1108 transforms the additional information. The level information included in the additional information may be modified as follows. To transmit the level of each source signal

, or

Send it. here,

Denotes the i-th source signal. The domain of the source signal may be a time domain or may be a subband domain. In such a case, it is difficult to quantize because the change in the level value is large. To solve this,

Transmit a value relative to a particular source signal, such as

As such, the relative value of the level of the mixed signal may be transmitted. here,

Means mix signal.

In addition, in order to offset the influence of interference between signals that may occur in constructing the mixed signal, a value such as Equation 20 below may be transmitted.

의 단순 합이 아닌 소스마다 특정 가중치(예를 들면, 게인 팩터)를 부여하여 구성된다면, 상기 [수학식 20]의 분모를 구성하는 소스 신호 레벨 합에도 그 가 중치를 적용하여 구성할 수 있다. Here, N is the number of source signals constituting the mix signal. If the mix signal

If it is configured by giving a specific weight (for example, a gain factor) for each source instead of the simple sum of, the weighting value may be applied to the sum of the source signal levels constituting the denominator of Equation (20).

를 이용하여 믹스 신호의 게인 팩터를 각각 정규화하여

를

으로,

를

으로 변형하고,

를 새로운

로 변형하여 아래의 [수학식 21]과 같이 변형할 수 있다.For example, one of two gain factors is used to normalize each of the two gain factors to create a new gain factor. At this time, one of the gain factors is transformed into a specific constant value (ie, 1). In addition, the additional information modifying unit 1302 may modify a gain factor and a source signal. For example, the additional information transformation unit 1108 may have a gain factor.

Use to normalize the gain factor of each mix signal

To

to,

To

Transform to

New

It can be modified as shown in Equation 21 below.

,

및 숏-타임 서브밴드 파워

등과 같은 부가 정보를 새로 운 게인 팩터

,

및 새로운 숏-타임 서브밴드 파워

등과 같은 새로운 부가 정보로 변형해도 디코딩 장치가 리믹스 신호를 생성하는데 아무런 차이가 없다. 새로운 게인 팩터 중,

는 디폴트값으로 설정되므로 인코딩 장치는

를 별도로 전송할 필요가 없다. 따라서 부가 정보량이 감소한다. The value is the same value as the original mix signal. So the encoding device has a gain factor

,

And short-time subband power

New gain factor such as additional information

,

And new short-time subband power

The decoding apparatus does not make any difference in generating the remixed signal even if it is transformed into new additional information such as. Among the new gain factors,

Is set to the default value, so the encoding device

There is no need to send it separately. Therefore, the amount of additional information is reduced.

,

및

를 그대로 인코딩하여 디코딩 장치로 전송할 수도 있으나, 양자화 및 인코딩에 더 적합한 값으로 전환하여 전송할 수도 있다. 부가정보전환부(1303)는 앞의 [수학식 3] 및 [수학식 4]를 이용하여 부가 정보

,

,

를

,

,

로 전환할 수 있다.The additional information converting unit 1303 converts the additional information transforming unit 1302 into a form that is easy to transmit additional information. The encoding device has additional information

,

And

May be encoded as it is and transmitted to the decoding apparatus, but may be converted to a value more suitable for quantization and encoding and then transmitted. The additional information conversion unit 1303 is the additional information by using the above [Equation 3] and [Equation 4]

,

,

To

,

,

You can switch to

은 특정 상수값으로 디폴트 되어 있으므로 인코딩 장치는

또는

중 하나와

만을 전송하면 된다. 부가정보양자화부(1304)는

또는

중 하나와

를 양자화한다. 부가정보인코딩부(1305)는 양자화된 부가 정보를 인코딩하여 디코딩 장치로 전송한다.At this time,

Is defaulted to a specific constant value, so the encoding device

or

With one of

You only need to send it. The additional information quantization unit 1304

or

With one of

Quantize The side information encoding unit 1305 encodes the quantized side information and transmits the encoded side information to the decoding apparatus.

The additional information may include various information in addition to the above-described gain factor, short-time subband power, time delay information, cross correlation information, and mix information. For example, if additional information is not generated together with the mixed signal, there may be a problem in reproduction due to inconsistency such as time sync. Therefore, timing information may be included in the additional information. In addition, the timing information may be included in the mixed signal or may be included in both the side information and the mixed signal. Therefore, the sync problem can be solved when the additional information is reproduced together with the mixed signal by using the timing information. Here, the timing information may be information about actual time, information about relative time, or include information that may be determined according to the characteristics of the mix signal.

Even with a mixed signal mixed with the same source, a difference may occur between the signals due to the mixing method or the encoding method. For example, a difference may occur in time sync between music recorded on a CD and music converted to MP3. . In such a case, the additional information multiplexed with the MP3 may affect the accuracy of reproduction due to problems such as time sync when used in the CD. In this case, it is also possible to compare the multiplexed signal and the additional information with the newly used signal, and modify the additional information to match the newly used signal.

14 is a transcoder for converting additional information to be suitable for a newly applied mix signal according to an embodiment of the present invention. Referring to FIG. 14, input 1 1401 is a multiplexed signal and input 2 1402 is a mix signal to which additional information is newly applied. In some cases, the input 1 1401 may be input with the mix signal and additional information. The transcoder 1403 compares the mix signal of the input 1 1401 and the mix signal of the input 2 1402, and converts additional information based on the mixed signal. The output 1404 may be converted additional information, or the signal of the input 2 and the additional information may be multiplexed and output.

In addition, when generating the additional information in the encoding apparatus, it is possible to adjust the amount of the additional information according to the application (application). For example, an application that is capable of muting a particular object needs more information than an application that only pans a particular object. Therefore, it is possible to provide the decoding apparatus with guide information on the maximum control capable of guaranteeing sound quality in the encoding apparatus generating the additional information. The guide information may be equally applied to all source signals or may be independently applied to each source signal.

In addition, the additional information may include an identifier indicating a specific channel of mono or multichannel. Assume that a multi-source is input so that a mix signal and additional information are generated. The multi-source may be mono depending on the source, may be stereo with two channels, or may be multi-channels exceeding two channels. If the source has multiple channels, it is possible to assume that each channel is an independent source and process it.

For example, in the case of a source having a stereo channel, it is conceivable that the K th input source is mono and the K + 1 th input source is stereo. In this case, if the Kth input source is the nth processing source, then the first channel of the K + 1th input stereo source is the n + 1th processing source, and the second channel of the K + 1th input stereo source is n + 2 Recognizing as the first processing source, it is possible to recognize and encode one input source as one processing source, that is, two processing sources, for each channel.

In this case, when there are a total of N processing sources, it is necessary to include the type of each processing source in the bit string. For example, the nth processing source is a mono signal, the n + 1th processing source is the first channel of the stereo source, and the n + 2th processing source is the second channel of the stereo source configured with the n + 1th processing source. It is necessary to send information such as Here, assuming that the processing source of the stereo source is always transmitted adjacently, it can be seen that the processing source type is limited to three types such as mono, first channel, and second channel.

15 is a block diagram of a decoding apparatus according to an embodiment of the present invention. Referring to FIG. 15, the decoding apparatus includes a demultiplexer 1502, a mixed signal decoder 1505, an additional information decoder 1504, an additional information restorer 1505, and a remix renderer 1506. do.

When the multiplexed mix signal and the first additional information 1501 are input to the demultiplexer 1502, the demultiplexer 1502 separates the encoded mix signal and the first additional information, and the encoded mixed signal is mixed signal decoded. In block 1503, the encoded first additional information is transmitted to the additional information decoding unit 1504. The first additional information refers to information obtained by modifying second additional information used to remix the mixed signal.

The mix signal decoding unit 1503 generates a mix signal by decoding the encoded mix signal, and the side information decoding unit 1504 generates first side information by decoding the encoded first side information. Then, the additional information restoring unit 1505 restores the generated first additional information to the original second additional information. The additional information restoring unit 1505 is optionally present. That is, the decoding apparatus according to the present invention may be configured to generate the remix signal using the first additional information, and may also be configured to generate the remix signal using the second additional information. The first additional information or the second additional information and the decoded mix signal are transmitted to the remix renderer 1506. The remix renderer 1506 may generate the remix signal 1507 using the first additional information or the second additional information, the mix signal, and the user mix parameter. The user mix parameter may be generated using control information obtained from a user.

16 is a flowchart illustrating a method of processing additional information according to an embodiment of the present invention. Referring to FIG. 16, the additional information generator 1301 generates additional information using a mix signal or a source signal (S1601). The additional information transformation unit 1302 transforms the generated additional information (S1602). The additional information switching unit 1303 converts the modified additional information into another form (S1603). The additional information quantization unit 1304 quantizes the converted additional information (S1604). The additional information encoding unit 1305 encodes and transmits the quantized additional information to the decoding apparatus (S1505).

,

,

를 생성한다. 부가정보변형부(1302)는 생성된 부가 정보를 변형한 다. 즉, 부가정보변형부(1302)는 게인 팩터 중 하나의 값, 예컨대,

를 이용하여 게인 팩터 각각을 정규화하여

,

을 생성한다.For example, a method of modifying additional information according to an embodiment of the present invention is as follows. The additional information generator 1301 uses the mixed signal or the source signal to provide additional information.

,

,

Create The additional information transformation unit 1302 transforms the generated additional information. That is, the additional information transformation unit 1302 has one value of the gain factor, for example,

To normalize each gain factor

,

Create

를 정규화하면 새로운 게인 팩터

은 1이 된다. 부가정보변형부(1302)는 정규화에 사용된 게인 팩터를 이용하여 소스 신호

를

으로 변형하고, 변형된 소스 신호

를 이용하여 서브밴드별 파워

를

으로 변형한다. 변형된 부가 정보

,

는 양자화 및 인코딩하기에 더 적합한 형태인

또는

, 그리고

로 전환된다. 이 때, 정규화에 의해

은 1이 되므로, 게인 팩터 중

만 전환하면 된다. 따라서,

및

중 하나만 이용된다. 전환된 부가 정보

또는

, 그리고

는 양자화되고 인코딩되어 디코딩 장치로 전송된다.Where the gain factor

Normalizing a new gain factor

Becomes 1 The side information transformation unit 1302 uses the gain factor used for normalization to source signal.

To

Modified, and the modified source signal

Power per subband using

To

Transform into. Modified Side Information

,

Is a more suitable form for quantization and encoding.

or

, And

Is switched to. At this time, by normalization

Becomes 1, so the gain factor

You only need to switch. therefore,

And

Only one of them is used. Switched Side Information

or

, And

Is quantized, encoded and sent to the decoding apparatus.

17 is a flowchart illustrating a signal processing method according to an embodiment of the present invention. Referring to FIG. 17, the decoding apparatus receives modified additional information (S1701). The modified side information includes a modified gain factor and a modified subband power. As described above, since the value of one of the modified gain factors is defaulted to a specific value, the decoding apparatus needs to receive only the non-default gain factor and subband power among the modified additional information.

When the encoding apparatus modifies the additional information, the source signal is also modified accordingly, so that a new source signal may be generated. Therefore, the encoding apparatus transmits the newly generated source signal and additional information about the newly generated source signal. When the encoding apparatus modifies the additional information, both the gain factor and the source signal are transformed, so that the mixed signal composed of the product of the gain factor and the source signal is the same as the original signal. The decoding apparatus receives the modified side information and decodes it. The modified additional information is then used to generate the original additional information. Control information provided by a user may be input to the decoding apparatus.

Then, the modified additional information is restored to the original additional information (S1702). In some cases, the modified additional information can be directly used without restoring the original additional information. The decoding apparatus may generate a remix signal using the mix signal, the control information, and the original additional information (S1703). In addition, the decoding apparatus according to the present invention may generate a remix signal using the modified additional information instead of the original additional information. The remixed signal generated by the decoding apparatus is irrelevant to whether or not the additional information of the encoding apparatus is modified.

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

According to the present invention, it is possible to provide a signal processing method and apparatus capable of adjusting a mixed signal for each source signal using the modified additional information.

In addition, according to the present invention, by modifying the additional information to generate the additional information with a low information amount and to transmit it, there is an effect that can reduce the amount of data transmission.

Claims (22)

Obtaining a mix signal comprising one or more source signals; Obtaining first side information; Obtaining a mix parameter; And Generating a remix signal using the mix signal, the first side information, and the mix parameter, Wherein the first additional information is information obtained by modifying second additional information indicating a relationship between a source signal to be remixed among the source signals and the mixed signal. The method of claim 1, wherein the signal processing method is Restoring the first additional information to the second additional information; The generating of the remix signal is performed using the mix signal, the second additional information, and the mix parameter. The method according to claim 1 or 2, The first additional information includes power information of the source signal and correlation information of the source signal and the mix signal. The correlation information indicates a degree in which the source signal is included in the first channel of the mix signal, and a degree included in the second channel of the mix signal is determined by default. The method according to claim 1 or 2, The first additional information has a smaller bitrate than the second additional information. The method of claim 4, wherein The first additional information is generated by normalizing the second additional information. The method of claim 1, The first additional information or the second additional information includes information indicating a level of the source signal to be remixed, a level between the source signals to be remixed, or a level between the source signal to be remixed and the mixed signal. Signal processing method. The method of claim 1, Wherein the first additional information or the second additional information includes information indicating a time delay between the source signals to be remixed or a time delay between the source signal to be remixed and the mix signal. The method of claim 1, The first additional information or the second additional information includes information indicating cross correlation between the source signals to be remixed or cross correlation between the source signals to be remixed and the mixed signal. The method of claim 1, Wherein the first additional information or the second additional information includes information indicating a degree to which the source signal to be remixed is mixed with the mixed signal. The method of claim 1, The first additional information or the second additional information includes timing information for solving a time sync problem when generating the remix signal together with the mixed signal. The method of claim 1, The first additional information or the second additional information includes guide information on a maximum control value of the source signal to be remixed. The method of claim 11, The guide information is equally applied to the entire source signal to be remixed, or is independently applied to each source signal to be remixed. The method of claim 1, The first additional information or the second additional information includes channel identification information indicating a channel in which each source signal is included. Obtaining a mix signal comprising one or more source signals; Obtaining a source signal to be remixed among the source signals; Generating first side information using the mix signal and the source signal to be remixed; And Transforming the first additional information into second additional information, And the first additional information is information representing a relationship between the source signal to be remixed and the mix signal. The method of claim 14, wherein the signal processing method is And converting the second additional information into third additional information more suitable for quantization and encoding. The method of claim 14, wherein the signal processing method is And quantizing and encoding the third additional information. 15. The method of claim 14, wherein said modifying step Generating the second additional information to normalize the first additional information. Obtaining a first mix signal and first side information; Obtaining a second mix signal; And transforming the first additional information into second additional information by using a result of comparing the first mixed signal and the second mixed signal. Wherein the first additional information is information necessary for remixing the first mixed signal, and the second additional information is information necessary for remixing the second mixed signal. The method of claim 18, wherein the signal processing method is And independently outputting the second additional information or multiplexing the second additional information and the second mixed signal and outputting the second additional information. A mix signal decoding unit obtaining a mix signal including one or more source signals; An additional information decoding unit obtaining first additional information; And A remix renderer configured to generate a remix signal using the mix signal, the first additional information, and the mix parameter, The first additional information is information obtained by modifying second additional information indicating a relationship between the source signal to be remixed and the mix signal among the source signals, and wherein the mix parameter is generated using control information obtained from a user. Signal processing device characterized in that. The apparatus of claim 20, wherein the signal processing device is And an additional information restoring unit for restoring the first additional information to the second additional information. And the remix renderer generates the remix signal using the mix signal, the second additional information, and the mix parameter. An additional information generator configured to generate first additional information using a mix signal including one or more source signals and a source signal to be remixed; An additional information transformation unit which transforms the first additional information into second additional information; And A side information encoding unit encoding the second side information, And the first additional information is information indicating a relationship between the source signal to be remixed and the mix signal.

Images