KR100891666B1 - Apparatus for processing audio signal and method thereof - Google Patents

Abstract

The present invention relates to a method and apparatus for processing a mixed signal. To this end, the present invention includes the steps of extracting additional information embedded in a non-perceptual region of a mix signal including one or more source signals; Obtaining a user mix parameter; And generating a remix signal using the mix signal, the additional information, and the user mix parameter, wherein the additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal. A signal processing method is provided.

Remix, side information, source signal

Description

Method and apparatus for processing mixed signal {APPARATUS FOR PROCESSING AUDIO SIGNAL AND METHOD THEREOF}

1 is a block diagram of an apparatus for encoding a first remix signal according to an embodiment of the present invention.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when using a stereo signal. FIG.

3 is a domain for processing a mix signal according to an embodiment of the present invention.

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention.

5 is a block diagram of a first remix signal decoding apparatus according to an embodiment of the present invention;

FIG. 6 is a detailed view of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. FIG.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention.

8B is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention used in combination with a conventional decoding apparatus.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention.

10 is a block diagram of an encoding apparatus according to an embodiment of the present invention for embedding additional information.

FIG. 11 is a detailed block diagram illustrating an embedding unit constituting the encoding apparatus of FIG. 10 according to an embodiment of the present invention. FIG.

12 is a diagram for a first method of embedding additional information according to an embodiment of the present invention.

13 is a diagram of a second method for embedding additional information according to an embodiment of the present invention.

14 is a diagram of a data structure reconstructed additional information according to an embodiment of the present invention.

15 is a block diagram of a decoding apparatus for decoding a mixed signal using embedded side information according to an embodiment of the present invention.

FIG. 16 is a detailed block diagram of an embedding unit constituting an encoding apparatus according to an embodiment of the present invention for preserving an original signal and embedding additional information. FIG.

17 is a block diagram illustrating a method of preserving an original signal and embedding additional information according to an embodiment of the present invention.

18 is a block diagram of a decoding apparatus according to an embodiment of the present invention for completely restoring an original signal.

FIG. 19 illustrates a first method of embedding additional information in a mix signal of two channels according to an embodiment of the present invention. FIG.

20 illustrates a second method of embedding additional information in a mix signal of two channels according to an embodiment of the present invention.

FIG. 21 illustrates a third method of embedding additional information in a mix signal of two channels according to an embodiment of the present invention. FIG.

FIG. 22 illustrates a fourth method of embedding additional information in a mix signal of two channels according to an embodiment of the present invention. FIG.

FIG. 23 is a block diagram illustrating a method of removing embedded side information according to an embodiment of the present invention. FIG.

24 is a block diagram of a device for encoding a mixed signal with additional information embedded according to an embodiment of the present invention.

25 is a detailed block diagram of the encoding apparatus of FIG. 24 according to an embodiment of the present invention.

FIG. 26 is a block diagram of a decoding apparatus for embedding additional information in a mix signal according to an embodiment of the present invention. FIG.

The present invention relates to a method and apparatus for encoding / decoding a mix signal. To date, stereo signals are most commonly generated and most widely used by consumers. In recent years, multichannel signals have been increasingly used. However, there is a limitation that the mixed signal is processed in the channel signal unit, not in the source signal unit constituting the mix signal. Therefore, when processing the mixed signal in the channel signal unit, there is a problem that only a specific source signal constituting the mixed signal can not be processed independently. For example, while watching a movie, it is impossible to increase the volume of background music while keeping the volume of actors' voices constant.

When the mixed signal is stored in the recording medium, there is a case where an auxiliary data area for storing additional information about the mixed signal does not exist. Therefore, in this case, there is a problem that the media signal cannot be processed in the unit of the source signal. In addition, when additional information is stored separately or transmitted separately, there is a compatibility problem with a general mix signal format.

In order to solve the above problems, an object of the present invention is to provide an encoding / decoding method and apparatus for embedding additional information on the mixed signal in order to process the mixed signal by the source signal using the additional information. have.

In order to achieve the above object, the present invention comprises the steps of extracting additional information embedded in the non-perceptual region of the mix signal comprising one or more source signals; Obtaining a user mix parameter; And generating a remix signal using the mix signal, the additional information, and the user mix parameter, wherein the additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal. A signal processing method is provided.

In addition, to achieve the above object, the present invention comprises the steps of obtaining a mix signal comprising at least one source signal; Obtaining a source signal to be remixed among the source signals; Generating side information using the mix signal and the source signal to be remixed; Embedding the additional information in a non-perceptual region of the mix signal, wherein the additional information indicates a relationship between the source signal to be remixed and the mix signal.

In addition, to achieve the above object, the present invention includes the steps of extracting a combined signal embedded in the non-perceptual region of the mixed signal comprising one or more source signals; Lossless decoding the combined signal to restore a signal corresponding to the non-perceptual region; Restoring the mixed signal from which the combined signal is extracted using the restored signal, wherein the combined signal includes a signal and additional information obtained by losslessly encoding a signal located in a non-perceptual region of the mixed signal; A signal processing method is provided.

In addition, to achieve the above object, the present invention comprises the steps of lossless coding a signal located in the non-perceptual region of the mixed signal comprising one or more source signals; Generating a combined signal by combining the lossless coded signal and additional information; And embedding the combined signal in the non-perceptual region, wherein the additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal.

In addition, in order to achieve the above object, the present invention comprises the steps of extracting additional information present in the non-perceptual region of the mix signal including one or more source signals; Encoding the mix signal; And generating a bitstream using the encoded mixed signal and the additional information, wherein the additional information indicates a relationship between a source signal to be remixed among the source signals and the mixed signal. Provide a method.

In addition, in order to achieve the above object, the present invention includes the steps of obtaining an encoded mix signal and additional information; Decoding the encoded mix signal; Embedding the additional information in the non-perceptual region of the decoded mix signal, wherein the mix signal comprises one or more source signals, wherein the additional information is one of the source signals to be remixed and the mixed signal; It provides a signal processing method characterized in that the relationship with.

In addition, in order to achieve the above object, the present invention includes an embedded signal decoding unit for extracting the encoded additional information embedded in the non-perceptual region of the mixed signal including one or more source signals; An additional information decoding unit to decode the encoded additional information to generate additional information; And a remix rendering unit configured to generate a remix signal using the additional information, the mix signal, and the user mix parameter, wherein the additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal. The user mix parameter provides a signal processing apparatus, which is generated using control information obtained from a user.

In addition, in order to achieve the above object, the present invention provides an additional information generation unit for generating additional information using a mix signal including at least one source signal and the source signal to be remixed; And an embedding unit to embed the additional information in the non-perceptual region of the mix signal, wherein the source signal to be remixed is selected from the source signals, and the additional information is a relationship between the source signal to be remixed and the mix signal. It provides a signal processing apparatus characterized in that.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

The present invention provides an algorithm that can process a mixed signal in an object signal unit. When the mix signal is processed in units of source signals, a wide variety of effects can be produced. For example, while watching a movie, it is possible to increase only the volume of the background music while keeping the volume of the actors' voices constant. In the present invention, the source signal includes one or more sources (eg, pianos) constituting the mix signal. Processing in source signal units means that in processing a mix signal, characteristics (eg, localization, gain) associated with a particular source signal constituting the mix signal are " individually ". It can be modified. By “individually” it is meant that modifying the characteristics associated with a particular source does not affect the characteristics of other source signals, or only a small effect that is difficult to perceive. For convenience of explanation, hereinafter, the mixed signal will be described as an example, but the present invention is not limited only to the mixed signal. In the present invention, a small amount of side information is transmitted as well as a typical mix signal format (for example, PCM, MP3, MPEG-AAC). The additional information may be used to re-mix source signals included in the transmitted mixed signal.

1 is a block diagram of a first remix signal encoder according to an embodiment of the present invention. The first remix signal encoder includes an additional information generator 103 and an additional information encoder 105.

Referring to FIG. 1, the additional information generator 103 generates additional information 104 using a typical mix signal 101 and a source signal 102 constituting the mixed signal. The mix signal 101 may be a mono, stereo and multi-channel audio signal. The source signal 102 may be some or all of the source signals constituting the mix signal 101. The additional information 104 refers to information used to process the mixed signal in source signal units. The additional information 104 includes mix parameters for remixing the mix signal. The mix parameter may include an encoder mix parameter generated using a source signal in an encoder, and optionally, a blind mix parameter generated using only a mix signal. Examples of the mix parameter may include a gain value and a subband power for each source signal. A detailed definition and generation method for the additional information 104 is described in FIG. 2. The present invention also includes generating the additional information 104 using only the source signal 102 constituting the mix signal. The side information encoding unit 105 encodes the generated side information 104 to generate an encoded side information signal 106. The mix signal 101 and the side information signal 106 are transmitted to a decoding apparatus.

FIG. 2 is a detailed block diagram of the first remix signal encoding apparatus of FIG. 1 when the mixed signal is a stereo signal. As described above, the mixed signal used in the present invention may be a mono, stereo and multichannel mixed signal, but for convenience, the mixed signal will be described based on the stereo signal 201.

및

는 상기 스테레오 신호를 구성하는 소스 신호들의 합으로 표현될 수 있다. 여기서, n은 타임 인덱스를 의미한다. 따라서, 상기 스테레오 신호(201)는 아래의 [수학식 1]과 같이 표현될 수 있다.The stereo signal 201

And

May be expressed as a sum of source signals constituting the stereo signal. Here, n means time index. Therefore, the stereo signal 201 may be expressed as Equation 1 below.

은 소스 신호들을 나타낸다.

및

는 각각의 소스 신호에 대한 진폭 패닝(amplitude panning) 및 게인(gain)을 결정하는 값이다. 모든

들은 서로 독립적이다. 상기

는 모두 순수한 소스 신호이거나, 또는 순수한 소스 신호에 약간의 잔향(reverberation) 및 효과음 신호성분(sound effect signal components)을 포함할 수 있다. 예를 들면, 특정한 잔향 신호성분은 2개의 소스 신호, 즉, 왼쪽 채널로 믹스된 신호와 오른쪽 채널로 믹스된 신호로 표현될 수 있다.Here, I is the number of source signals included in the stereo signal,

Represents source signals.

And

Is a value that determines the amplitude panning and gain for each source signal. all

They are independent of each other. remind

Are all pure source signals, or may include some reverberation and sound effect signal components in the pure source signal. For example, the specific reverberation signal component may be represented by two source signals, that is, a signal mixed with the left channel and a signal mixed with the right channel.

It is an object of the present invention to modify a stereo signal comprising the source signal such that M (0 <= M <= I) source signals are remixed. The source signals may be remixed into a stereo signal with different gain factors. The remix signal may be expressed as Equation 2 below.

및

는 리믹스되는 M개의 소스 신호들에 대한 새로운 게인 팩터들이다. 상기

및

는 디코더 단에서 제공될 수 있다. 이 경우에, 부가정보생성부(206)는 스테레오 신호(201) 및 M개의 소스 신호(202)를 이용하여 부가 정보(207)를 생성할 수 있다. here,

And

Are new gain factors for the M source signals to be remixed. remind

And

May be provided at the decoder stage. In this case, the additional information generator 206 may generate the additional information 207 using the stereo signal 201 and the M source signals 202.

에 대한 접근 없이, [수학식 1]로 표현되는 통상적인 믹스 신호가 주어지는 경우에, [수학식 2]로 표현되는 리믹스 신호를 지각적으로 모방하는 것을 목적으로 한다.As described above, an object of the present invention is to remix the stereo signal in the unit of the source signal when given a conventional stereo signal and some additional information. As in the present invention, it is not possible to completely generate the remix signal represented by Equation 2 from the mixed signal represented by Equation 1 using a very small amount of additional information. Thus, the present invention provides the respective source signals

It is aimed to perceptually mimic the remix signal represented by [Equation 2], given an ordinary mix signal represented by [Equation 1] without access to.

및

로 표시된다. 여기서 k는 서브밴드 신호들의 시간 인덱스(time index)이다. 유사하게, M개의 소스 신호들(202)의 서브밴드 신호들(205)은

,

,...,

로 표시된다. 명료한 표현을 위해, 서브밴드(주파수) 인덱스를 사용하지 않았다.Referring to FIG. 2, a stereo signal 201 and M source signals 202 included in the stereo signal 201 are input to a first remix signal encoding apparatus. The stereo signal 201 may be delayed to some extent to be synchronized with the side information and used directly as an output signal. To generate the additional information, the stereo signal 201 and the source signals 202 are decomposed into subband-specific signals 204 and 205 in the time-frequency domain through the filter bank 203. That is, the stereo signal 201 and the source signal are processed in the time-frequency domain, which will be described later with reference to FIG. 3. The subband-specific signal 204 is similarly processed at the center frequency of each subband. At a particular frequency, the subband pair 204 of the stereo signal 201

And

Is displayed. Where k is the time index of the subband signals. Similarly, the subband signals 205 of the M source signals 202 may be

,

, ...,

Is displayed. For clarity, no subband (frequency) index is used.

를 생성한다. 또한, 상기 부가정보생성부(206)는 스테레오 신호(201)의 서브밴드 쌍(204)을 이용하여, 서브밴드별로 게인 팩터

및

를 생성한다. 상기 게인 팩터

및

는 외부에서 직접 주어질 수 있다. 상기 서브밴드별 숏-타임 서브밴드 파워 및 게인 팩터를 이용하여 서브밴드별 부가 정보(207)가 생성된다. 상기 부가정보생성부(206)는 상기 숏-타임 서브밴드 파워 및 게인 팩터들 이외에 상기 스테레오 신호에 관련된 다른 정보를 부가 정보(207)로 생성할 수 있다. 부가정보인코딩부(208)는 상기 서브밴드별 부가정보(207)를 이용하여 부호화된 부가 정보 신호(209)를 생성한다. Given the subband signals 205 of the source signals 202, the side information generator 206 may perform short-time subband power for each subband,

Create In addition, the additional information generator 206 uses a subband pair 204 of the stereo signal 201 to obtain a gain factor for each subband.

And

Create The gain factor

And

Can be given directly from the outside. Subband-specific additional information 207 is generated using the short-time subband power and gain factor for each subband. The additional information generator 206 may generate other information related to the stereo signal as additional information 207 in addition to the short-time subband power and gain factors. The additional information encoding unit 208 generates an additional information signal 209 encoded using the additional information 207 for each subband.

및

는 고정적이 될 것이다. 만일

및

가 시간 k에 따라 가변적이라면, 상기 게인 팩터들은 시 간의 함수로 생성될 것이다. 상기 게인 팩터들은 직접 양자화 및 부호화되지 않고, 먼저 양자화 및 부호화에 더 적합한 다른 값들로 전환될 수 있다. 또한,

는 스테레오 신호(201)의 서브밴드 파워에 상대적인 값으로 정규화될 수 있다. 이것은 스테레오 신호를 효율적으로 부호화하기 위해 통상적인 인코딩 장치가 이용되는 경우에, 본 발명을 상대적으로 변화에 강하도록 만들어준다. 예를 들면,

및

는 아래의 [수학식 3]으로 표현되는 게인 및 데시벨(dB) 단위의 레벨차로 전환되어 전송될 수 있다. For many stereo signals 201, the gain factor

And

Will be fixed. if

And

If is variable over time k, the gain factors will be generated as a function of time. The gain factors are not directly quantized and coded, but may first be converted to other values more suitable for quantization and coding. Also,

Can be normalized to a value relative to the subband power of the stereo signal 201. This makes the present invention relatively resistant to changes when conventional encoding devices are used to encode stereo signals efficiently. For example,

And

May be converted into a level difference in units of gain and decibel (dB) expressed by Equation 3 below, and then transmitted.

는 부가 정보로서 직접 부호화되는 것이 아니라, 아래의 [수학식 4]로 표현되는 스테레오 신호에 상대적으로 정의된 값으로 변환되어 전송될 수 있다. Also,

Is not directly encoded as additional information, but may be converted into a value defined relative to the stereo signal represented by Equation 4 below and transmitted.

는 아래의 [수학식 5]와 같이 계산될 수 있다. To produce short-time subband power, the present invention uses single-pole averaging. In other words,

May be calculated as shown in Equation 5 below.

Here, α∈ [0,1] determines the time-constant of the estimation window (estimation window) which decreases exponentially as shown in Equation 6 below.

는 서브밴드 샘플링 주파수를 나타낸다. 예를 들면, T=40 ms를 이용할 수 있다. 이하에서,

는 숏-타임 평균(short-time averaging)을 나타낸다. 만일

및

가 주어지지 않는다면, 상기

및

는 부가정보생성부(206)에서 생성될 필요가 있다.

이므로,

는 아래의 [수학식 7]과 같이 계산된다.here,

Denotes a subband sampling frequency. For example, T = 40 ms can be used. In the following,

Denotes short-time averaging. if

And

If is not given,

And

Needs to be generated in the additional information generation unit 206.

Because of,

Is calculated as shown in Equation 7 below.

는 아래의 [수학식 8]과 같이 계산된다.Similarly,

Is calculated as shown in Equation 8 below.

3 illustrates a domain for processing a mix signal according to an embodiment of the present invention. As described above, the mix signal and the side information are processed as subband-specific signals in the time-frequency domain as shown in FIG. Subband-specific signals in the time-frequency domain are perceptually derived. For example, a signal for each subband may be generated by using a short time fourier transform (STFT) having a sine wave analysis window and a sine analysis and synthesis window having a length of about 20 ms. In this case, the STFT coefficients may be grouped such that one group has a bandwidth that is about twice the equivalent rectangular bandwidth (ERB).

4 is a block diagram of a second remix signal encoding apparatus according to an embodiment of the present invention. The second remix signal encoding apparatus includes a downmixing unit 402, an additional information generating unit 403, and an additional information encoding unit 406.

Referring to FIG. 4, the downmixing unit 402 generates a sum signal 404 by adding a plurality of source signals 401. Unlike the first remix signal encoding apparatus, the second remix signal encoding apparatus transmits the sum signal 404 instead of the stereo signal. The additional information generator 403 generates the additional information 405 using the source signals 401. The additional information 405 includes subband power and gain factor corresponding to each source signal. In addition, the additional information 405 may include a parameter corresponding to a delay in the remix rendering unit. Similar to the first remix signal encoding apparatus, the additional information 405 may be converted into another value more suitable for quantization and encoding and transmitted. The additional information encoding unit 406 generates an encoded additional information signal 407 using the generated additional information 405. The generated sum signal 404 and the side information signal 407 are transmitted to the decoding apparatus. The present invention also includes an encoding apparatus that does not have a downmixing unit 402. In this case, the source signals 401 are not converted to the sum signal 404, and each source signal 401 is transmitted directly.

5 is a block diagram of an apparatus for decoding a first remix signal according to an embodiment of the present invention. The first remix signal decoding apparatus includes an additional information decoder 503 and a remix renderer 505.

Referring to FIG. 5, the mix signal 501 and the additional information signal 502 are input to the first remix signal decoding apparatus. The mix signal 501 may be a mono, stereo or multichannel mix signal. The additional information decoding unit 503 decodes the additional information signal 502 to generate additional information 504. The additional information 504 includes a gain factor and a subband power of source signals included in the transmitted mixed signal 501. The remix renderer 505 may input a user-mix parameter 506 generated using control information directly provided by the user. The remix renderer 505 generates the remix signal 507 using the mix signal 501, the transmitted additional information 504, and the user mix parameter 506. A detailed description of the method of generating the remix signal will be described later with reference to FIG. 6. The remix signal 507 is generated as an Eq-channel mix signal having the same number of channels as the number of channels of the transmitted mix signal, or an upchannel mix signal having more channels than the number of channels of the mix signal. (Up-channel mix signal) can be generated.

FIG. 6 is a detailed diagram of the first remix signal decoding apparatus of FIG. 5 when using a stereo signal. As described above, the transmitted mixed signal may be a mono, stereo and multichannel mixed signal, but will be described with reference to the stereo signal 601 for convenience.

및

로 표현된다. 부가정보디코딩부(605)는 전송된 부가 정보 신호(602)를 복호화하여, 서브밴드별 부가 정보(606)를 생성한다. 또한, 리믹스렌더링부(607)에 사용자가 제공하는 제어 정보를 이용하여 생성된 사용자 믹스 파라미터(608)가 입력될 수 있으며, 상기 사용자 믹 스 파라미터(608)는 서브밴드별로 제공될 수 있다. 전술한 것처럼, 상기 부가 정보(606)는 리믹스 될 M개의 소스 신호에 대한 서브밴드별 게인 팩터(

및

) 및

로 표현되는 서브밴드 파워를 포함한다. 리믹스렌더링부(607)는 서브밴드별로 생성된 스테레오 신호(604), 전송된 부가 정보(606) 및 사용자 믹스 파라미터(608)를 이용하여, 서브밴드별 리믹스 신호(609),

및

를 생성한다. 상기 리믹스 신호(609)를 생성하는 방법은 아래에서 더욱 상세하게 기술된다. 상기 리믹스 신호(609)는 역필터뱅크(610)를 통해 시간 도메인(time domain)의 스테레오 신호(611),

및

로 변환된다. Referring to FIG. 6, the stereo signal 601 is decomposed into a subband-specific signal 604 in the time-frequency domain through the filter bank 603. As shown in FIG. 6, the subband-specific signal 604 at a particular frequency

And

It is expressed as The additional information decoding unit 605 decodes the transmitted additional information signal 602 to generate additional information 606 for each subband. In addition, a user mix parameter 608 generated using control information provided by a user may be input to the remix renderer 607, and the user mix parameter 608 may be provided for each subband. As described above, the additional information 606 is a subband-specific gain factor for M source signals to be remixed.

And

) And

It includes the subband power represented by. The remix renderer 607 uses the stereo signal 604 generated for each subband, the additional information 606 transmitted, and the user mix parameter 608, and the remix signal 609 for each subband,

And

Create The method of generating the remix signal 609 is described in more detail below. The remix signal 609 is a stereo signal 611 in the time domain through the inverse filter bank 610,

And

Is converted to.

는 서브밴드별 소스 신호

로 교체된다. 즉, 서브밴드별 믹스 신호(604)는 아래의 [수학식 9]과 같이 표현될 수 있다. A method of generating the remix signal 609 generated by the remix renderer 607 is as follows. [Equation 1] and [Equation 2] are also valid for the subband signals 604 and 609. In this case, the source signal

Is the subband-specific source signal

Is replaced by. That is, the mix signal 604 for each subband may be expressed by Equation 9 below.

The subband remix signal 609 may be expressed by Equation 10 below.

및

가 주어지면, 아래의 [수학식 11]와 같이 서로 다른 게인들을 가지는 서브밴드별 리믹스 신호(609)가 상기 서브밴드별 믹스 신호(604)의 선형 조합으로 추정될 수 있다.To generate the remix signal 609, least squares estimation may be used. Per subband mix signal 604,

And

Given by Equation 11 below, the subband remix signal 609 having different gains may be estimated as a linear combination of the subband mix signals 604.

,

,

및

는 가중 팩터들(weighting factors)이다. 이때, 생성되는 추정 에러(estimation error)는 아래의 [수학식 12]과 같이 정의될 수 있다.here,

,

,

And

Is the weighting factors. In this case, the generated estimation error may be defined as shown in Equation 12 below.

,

,

및

는 평균제곱오차(mean square error),

및

가 최소가 되도록 서브밴드별로 생성될 수 있다. 이때, 추정 에러,

및

가

및

에 직교(orthogonal)될 때, 상기 평균제곱오차가 최소가 된다는 것을 이용할 수 있다. 생성되는

및

는 아래의 [수학식 13]과 같이 표현될 수 있다.The weighting factors,

,

,

And

Is the mean square error,

And

May be generated for each subband such that is minimized. In this case, the estimation error,

And

end

And

When orthogonal to, it can be used that the mean square error is minimized. Generated

And

May be expressed as Equation 13 below.

,

및

는 직접 생성될 수 있지만,

및

은 전송된 부가 정보(606)(예를 들면,

,

,

) 및 사용자가 제공하는 제어 정보(608)(예를 들면, 게인 팩터

및

)를 이용하여, 아래의 [수학식 14]와 같이 생성될 수 있다.here,

,

And

Can be generated directly,

And

Is transmitted additional information 606 (e.g.,

,

,

) And control information 608 provided by the user (e.g., gain factor

And

) Can be generated as shown in Equation 14 below.

및

가 아래의 [수학식 15]와 같이 생성될 수 있다. Similarly,

And

May be generated as shown in Equation 15 below.

및

는 아래의 [수학식 16]과 같이 표현될 수 있다. here,

And

May be expressed as Equation 16 below.

If the phases of the mix signal 604 are coherent or nearly synchronized with each other, a value expressed by Equation 17 below approaches 1.

In this case, the weights may be expressed as Equation 18 below.

및

)를 이용하여 각각의 소스 신호를 독립적으로 리믹스하여 생성된 리믹스 신호와 유사하게 들린다.The subband remix signal 609 generated as described above is converted into the remix signal 611 of the time-domain through the inverse filter bank 610 as described above. The remix signal 611 may generate a user mix parameter generated using control information provided by a user.

And

It sounds similar to the remix signal generated by independently remixing each source signal with

So far, the focus has been on remixing two-channel stereo signals. However, as described above, the present invention is not limited to stereo signals, but may be extended to remixing multichannel mixed signals, for example, 5.1 channel mixed signals. Those skilled in the art can remix multichannel mix signals, similar to the stereo signals described herein. In this case, Equation 11 may be written as Equation 19 below.

Optionally, one of the channels of the mix signal can be left without remixing. For example, for a 5.1 surround channel, it is possible to apply remixing only to the front channel without modifying the two back channels. In this case, a two or three channel remix algorithm is applied to the front channel.

7 is a block diagram of a second remix signal decoding apparatus according to an embodiment of the present invention. The second remix signal decoding apparatus includes an additional information decoding unit 703, a spatial information integration unit 705, and a remix rendering unit 707.

Referring to FIG. 7, the sum signal 701 of the source signals and the additional information signal 702 are input to the second remix signal decoding apparatus. The additional information decoding unit 703 decodes the additional information signal 702 to generate additional information 704. The additional information 704 includes a gain factor, a delay constant, a subband power, and the like. The additional information integrator 705 separates the sum signal 701 into a plurality of source signals 706 by using the additional information 704. The remix renderer 707 may generate the remix signal 709 using the source signals 706. In this case, the remix renderer 707 may generate the remix signal 709 using the mix parameter transmitted as additional information. In addition, the remix renderer 707 may selectively generate the remix signal 709 using the user mix parameter 708 generated using control information provided by the user.

8A is a block diagram illustrating a combination of a conventional encoding apparatus and a remix signal encoding apparatus according to an embodiment of the present invention. The mixed signal 801 may be encoded by the conventional encoding apparatus 803 and converted into the encoded mixed signal 805. The mix signal 801 may be a channel-specific signal or a source signal. The conventional encoding device 803 includes not only conventional encoding devices such as AAC, MP3 encoder, etc., but also encoding devices to be developed in the future. The remix signal encoding apparatus 804 according to the present invention generates the additional information signal 806 using the mixed signal 801 and the source signal 802 included in the mixed signal. The multiplexer 807 generates the bitstream 808 using the encoded mix signal 805 and the side information signal 806. As described above, the additional information signal 806 may be inserted into an auxiliary data area in a conventional mix signal format to be compatible with conventional devices.

8B is a block diagram of a combination of a conventional decoding device and a remix signal decoding device according to an embodiment of the present invention. The demultiplexer 810 separates the encoded mix signal 811 and the side information signal 812 from the transmitted bitstream 809. The conventional decoding device 813 then decodes the encoded mix signal 811 to produce a mix signal 814 that can be used in the remix signal decoding device 815 according to the present invention. The conventional decoding device 813 includes not only a conventional encoding device such as an AAC, an MP3 decoder, etc. but also an encoding device to be developed in the future. The mix signal 814 may be a channel-specific signal or a source signal. The remix signal decoding apparatus 817 according to the present invention may convert the mix signal 814 into the remix signal 816 using at least one of the additional information signal 812 and the user mix parameter 817.

9 is a detailed block diagram of a remix signal decoding apparatus according to an embodiment of the present invention. Referring to FIG. 9, the remix signal decoding apparatus includes a mixed signal decoder 901, a parameter generator 902, and a remix renderer 908. It may optionally include an effector (911). The parameter generator 902 may include a blind mix parameter generator 903, a user mix parameter generator 904, and a remix parameter generator 905. The remix parameter generation unit 905 may include an equimix parameter generation unit 906 and may optionally include an upmix parameter generation unit 907. In addition, the remix renderer 908 may include an equimix renderer 909 and optionally include an upmix renderer 910.

The mixed signal decoding unit 901 decodes the encoded mixed signal transmitted from the encoding end to generate a mixed signal. The parameter generator 902 receives additional information and user control information (or configuration information) transmitted from an encoding stage. The user control information is not transmitted at the encoder stage but may be generated at the decoder stage. The user mix parameter generator 904 generates a user mix parameter by using user control information. The additional information transmitted from the encoder stage may include an encoder mix parameter. In addition, the blind mix parameter generator 903 may generate a blind mix parameter using the mix signal. The encoder mix parameter and the blind mix parameter are alternatively input to the remix parameter generator 905.

The remix parameter generator 905 generates a remix parameter using additional information and a user mix parameter. The remix parameter may be generated to be applied to a channel of the remix signal. Equimix parameter generation unit 906 included in the remix parameter generation unit 905 generates a remix parameter used to generate a remix signal having the same channel number as the number of channels of the mix signal, the remix parameter generation unit 905 The upmix parameter generation unit 907, which may be included in the control unit, generates a remix parameter used to generate a remix signal having a larger number of channels than the number of channels of the mixed signal. The remix parameter is input to the remix renderer 908.

Equimix renderer 909 included in the remix renderer 908, using the remix parameter and the mix signal, having an equal number of channels equal to the number of channels of the mix signal (Eq-channel remix signal ) The upmix renderer 910, which may be included in the remix renderer 908, uses more channels than the number of channels of the mix signal by using the remix parameter and the mix signal generated by the upmix parameter generator 907. Generate an up-channel remix signal having a number. The upmix renderer 910 may generate an upchannel remix signal using the remix signal generated by the Equichannel renderer 909.

Accordingly, the decoding apparatus may output the mixed signal transmitted from the encoding stage as it is, output as an equal channel remix signal, or output as an up channel remix signal. Optionally, the remix rendering unit may give various effects to the remix signal by using information provided from the effector 911.

10 is a block diagram of an encoding apparatus for embedding additional information according to an embodiment of the present invention. The encoding apparatus includes an additional information generating unit 1103, an additional information encoding unit 1005, and an embedding unit 1007.

Referring to FIG. 10, a mix signal 1001 and a source signal 1002 are input to the encoding apparatus. As described above, the mix signal 1001 may be a mono, stereo or multichannel mix signal. In the present specification, a stereo signal will be described as an example for convenience, but the present invention is not limited thereto. The side information generating unit 1003 generates the side information 1004 using the mixed signal 1001 and the source signal 1002. The additional information 1004 may include mix information indicating a degree to which a source signal included in the mix signal is mixed in the mix signal channel, time delay information between source signals included in the mix signal, or the source signal. And time delay information between the mixed signal channel, cross correlation information between source signals included in the mix signal, or cross correlation information between the source signal and the mixed signal channel.

및

(1008)는 디코더 장치에서 입력신호(input signal)

및

(1001)과 동일한 신호라 볼 수 있다.The side information encoding unit 1005 generates an additional information signal 1006 for transmission and storage using the side information 1004. An embedding unit 1007 embeds the additional information signal 1006 in the mix signal 1001. In this case, a "digital signal embedded method" may be used. Using the "digital signal embedding technique", it is possible to embed additional information in the PCM type mix signal 1001 without sound quality distortion. The mixed signal in which the additional information is embedded is hardly distinguished from the original mix signal. That is, output signal in which additional information is embedded

And

1008 denotes an input signal at the decoder device

And

It can be regarded as the same signal as 1001.

The "digital signal embedding method" includes a bit replacement coding method, an echo hiding method, a spread-spectrum-based method, and the like. The bit-subversion encoding method inserts desired information by transforming the lowest bits of the quantized mix signal sample, and uses the property that the modification of the least significant bit has little effect on the quality of the mixed signal. Way. The echo insertion method is a method of inserting an echo of a size small enough to be inaudible to the human ear to the mix signal. The spread spectrum communication method converts a mixed signal into a frequency domain through a discrete cosine transform, a discrete fourier transform, and the like, and then converts desired information made in binary into a PN (Pseudo Noise) sequence. This method is used to spread spectrum and add it to the mixed signal converted into the frequency domain. In the present invention, a description will be made mainly on the bit substitution encoding method of the embedding method, but the present invention is not limited to the bit substitution encoding method.

FIG. 11 is a detailed block diagram illustrating an embedding unit constituting the encoding apparatus of FIG. 10 according to an embodiment of the present invention. The embedding unit includes a buffer 1103, a masking threshold computing unit 1105, a bitstream reshaping unit 1107, and a mix signal encoding unit 1109. do. By using the bit substitution encoding method, additional information may be embedded in a non-perceptual region of a mixed signal component. In this case, the size of the non-perceptive region (hereinafter, referred to as K value) may use K (K> 0) bits of a specific region according to a pre-decided method instead of using only the lowest 1 bit. . For example, the predetermined method is to obtain a masking threshold according to a psychoacoustic model and to allocate an appropriate bit according to the masking threshold.

Referring to FIG. 11, the mix signal 1101 is input to the mix signal encoding unit 1109 through the buffer 1103. The masking limit value calculator 1105 calculates a masking limit value for each section (for example, a block) by using the information 1104 related to the characteristics of the input mix signal 1101. In addition, the masking limit value calculator 1105 uses the masking limit value to obtain a K value that can be modified without aural distortion. That is, the number of bits that can be used to embed the additional information 1102 in the mix signal 1101 is allocated for each block. In the present specification, a block refers to a data unit inserted using one K value existing in a frame. One or more blocks may exist in one frame. Therefore, if the length of the frame is fixed, the length of the blocks may decrease as the number of blocks increases. The bitstream reconstruction unit 1007 may reconstruct the additional information to include the K value. In this case, the reconstructed additional information may include a sync word, an error detection code, an error correction code, or the like. The mixed signal encoding unit 1109 embeds the reconstructed additional information 1108 into the mix signal 1101 and outputs the mixed signal 1110 in which the additional information is embedded. As described above, the reconstructed side information 1108 may be embedded within K bits of the mix signal. The K value is inserted into the reconstructed additional information 1108 and transmitted to the decoding apparatus, and the decoding apparatus may extract additional information from the mix signal using the K value.

Various methods are used to embed additional information into the mixed signal for each block. The first method simply adds additional information after substituting the lower K bits of the mix signal with zeros. For example, if the K value is 3, one sample data of the mix signal is 11101101, and the additional information data to be embedded is 111, the lower 3 bits of the 11101101 are replaced with 0 to make 11101000, and then the addition The information data 111 is added to form 11101111.

The second method uses a dithering method, first subtracting additional information data from the lower K bits of the mix signal, then requantizing the mix signal based on the K value, and applying the requantized mix signal to the mixed signal. It is a method of adding additional information data. For example, if the K value is 3, one sample data of the mix signal is 11101101, and the additional information data to be embedded is 111, 111 is subtracted from 11101101 to 11100110, and requantized for the lower 3 bits or more to 11101000. (Rounding), then add 111 to 11101111.

Since the additional information embedded in the mixed signal is an arbitrary bitstream, it may not have a white noise characteristic. Since it is advantageous in terms of sound quality to add a white noise type signal to the mixed signal, it may be added to the mixed signal after whitening the additional information. The whitening may be applied to an additional information signal except syncword. Whitening in the present invention refers to making the volume of the mixed signal into a random signal having the same or almost similar magnitude in all frequency domains. In addition, in the process of embedding the additional information into the mixed signal, the acoustic shaping technique may be applied to the additional information to minimize the acoustic distortion. In the present invention, the noise shaping is to change the noise characteristic so that the energy of the quantization noise generated in the quantization process is shifted to the high frequency band of the audible frequency band or more, or obtain a masking limit value from the corresponding mixed signal, and time-varying corresponding to the masking limit value. A process of generating a time-varing filter and modifying the characteristics of noise generated in the quantization process by the filter.

12 illustrates a first method of embedding additional information according to an embodiment of the present invention. The additional information may be embedded in the mix signal in various ways. FIG. 12 corresponds to a method of embedding the additional information in the sample plane order. The first method is to distribute and embed additional information about a corresponding block in units of K bits. If the value of K is 4 and one block 1205 is composed of N samples 1204 as shown, the additional information may be embedded in the lower four bits of each sample. As described above, the present invention is not limited to embedding additional information only in the lower 4 bits of each sample. And within the lower K bits of each sample, the side information may be embedded from the upper bit (Most Significant Bit (MSB) first) or the lower bit from the lower bit (LSB (Least Significant Bit) first) as shown in FIG. Can be.

The arrow 1203 shown indicates the direction in which it is embedded, and the numbers in parentheses indicate the data sort order. If the number of bits of additional information to be embedded is less than the embeddable number of bits in the area where the additional information is embedded, fill the remaining bits with zeros (1206), insert a random signal, or replace with the original mix signal. can do. For example, when the number of samples (N) constituting the block is 100 and the K value is 4, the number of bits (W) embedable in the block is W = N * K = 100 * 4 = 400 bits. If the number of bits (V) of additional information to be embedded is 390 bits (that is, V <W), the remaining 10 bits are filled with zeros, inserted random signals, replaced with the original mixed signal, or the end of data. It can be filled with a tail sequence that indicates or a combination thereof. The terminal bit string refers to a bit string indicating the end of additional information in the block. Although FIG. 12 illustrates filling the remaining bits for each block, the present invention includes filling the remaining bits for each insertion frame by the above method.

13 shows a second method of embedding additional information according to an embodiment of the present invention. The second method is to embed the additional information 1301 in a mixed signal in bitplane order. In this case, as shown in FIG. 13, the additional information may be sequentially embedded from the lower bits of the mixed signal for each block, but the present invention is not limited thereto. For example, if the number of samples (N) constituting the block is 100 and the K value is 4, the first 100 bits constituting the lowest bitplane are filled first, followed by the 100 bits constituting the second bitplane. You can proceed.

Arrow 1303 shown indicates the direction in which it is embedded, and the numbers in parentheses indicate the data sort order. The second method may be particularly advantageous for extracting Sync Word at any location. That is, when searching for the syncword of the embedded side information from the mixed signal, it is possible by extracting and searching only the LSB. In addition, the second method can be expected to use the minimum LSB according to the number of bits (V) of the additional information to be embedded. In this case, when the number of bits (V) of the additional information to be embedded is smaller than the embeddable number of bits (W) in the region where the additional information is embedded, the remaining bits are filled with zero as described above (506) or a random signal is inputted. It can be replaced with the original mix signal, filled with a terminal bit that indicates the end of the data, or a combination thereof. In particular, it is advantageous to use the mix signal as it is in the method. Although FIG. 13 also shows filling the remaining bits for each block, the present invention includes filling the remaining bits for each insertion frame by the above method.

14 illustrates a data structure reconstructing additional information according to an embodiment of the present invention. As described above, the additional information may be reconfigured to include a sync word 1403 and a K value 1404 in the bitstream reconstruction unit. Further, in the reconstruction process, at least one error detection code or error correction code 1406 and 1408 capable of determining whether the additional information signal is damaged during transmission or storage, Hereinafter, only the error correction code) may be included in the reconstructed additional information signal. The error correction code includes a cyclic redundancy check (CRC). The error correction code may be divided into two stages, including an error correction code 1 (1406) for the header 1401 including K values and an error correction code 2 (1408) for the additional information data 1407. have. In addition, other information 1405 may be separately included in the additional information signal. The other information 1405 may include identification information on a method of embedding additional information.

15 is a block diagram of a decoding apparatus for decoding a mixed signal using embedded side information according to an embodiment of the present invention. The decoding device includes an embedded signal decoding unit 1502, an additional information decoding unit 1504, and a remixing unit 1506.

Referring to FIG. 15, the embedded signal decoder 1502 may detect the additional information signal 1503 from the mix signal 1501. The additional information decoding unit 1504 decodes the additional information signal to generate additional information 1505. The remix renderer 1506 generates a remix signal 1508 using the additional information 1505 and the mix signal 1501. In this case, the remix rendering unit 1506 may use the user mix parameter 1507 generated using control information provided by the user.

FIG. 16 is a detailed block diagram of an embedding unit constituting an encoding apparatus that preserves an original mix signal and embeds additional information according to an embodiment of the present invention. Since the encoding device is similar to the encoding device shown in FIG. 10, detailed description of the same parts will be omitted. The difference from the encoding device shown in FIG. 10 is that the embedding part constituting the encoding device is configured differently.

Referring to FIG. 16, the mix signal 1601 is input to the mix signal encoding unit 1609 through a buffer 1603. The masking limit value calculator 1605 calculates a masking limit value using information 1604 regarding the characteristics of the mix signal 1601, and calculates a K value 1606 using the masking limit value. The bitstream reconstruction unit 1607 reconstructs the mixed signal data 1611 and the additional information signal 1602 without removing the mixed signal data 1611 in the region where the additional information signal 1602 is to be embedded. The reconstruction method is described in more detail in FIG. 17. The mixed signal encoding unit 1609 may embed the reconstructed signal 1608 into the mixed signal 1601.

17 is a block diagram illustrating a method of preserving an original mix signal and embedding additional information according to an embodiment of the present invention. For convenience of explanation, the additional information is inserted into one block constituting one channel 1701 of the mixed signal. Referring to FIG. 17, a component of the mix signal 1701 may be divided into an area 1702 in which additional information is not embedded and an area 1703 in which an additional information is embedded. The mixed signal of the embedded region 1703 undergoes a lossless coding process 1704. The lossless coded mix signal and side information signal 1705 are then combined and encoded to form a combined signal 1707. The combined signal 1707 is then embedded in the mix signal, producing a mixed signal 1708 with the combined signal embedded. By doing so, the decoding apparatus can completely recover the original mix signal 1701 using the combined signal 1707 when necessary. The above method is possible because the amount of data of the additional information signal to be embedded is actually smaller than the amount of embeddable data, so that a space for inserting a lossless coded mix signal can be secured. The method of simply embedding the side information signal is not exactly the same signal, although the perceptually embedded signal and the original signal are the same, so that if the restoration of the complete original signal is necessary, the method may be used. Can be.

18 is a block diagram of a decoding apparatus for completely reconstructing an original mixed signal according to an embodiment of the present invention. The decoding apparatus includes an embedded signal decoding unit 1802, a lossless decoding unit 1804, and a replacing unit 1806.

및

를 이용하여 복호화를 수행하는 대신에,

및

를 이용하여 리믹싱 과정을 수행할 수 있기 때문에, 더욱 좋은 출력을 얻을 수 있다. Referring to FIG. 18, the embedded signal decoder 1802 extracts the combined signal 1803 from the mixed signal 1801 in which the combined signal is embedded. The lossless decoding unit 1804 restores the mixed signal 1805 corresponding to the region in which the combined signal is embedded using the combined signal 1803. The replacing unit 1806 generates the original signal 1807 using the restored mixed signal 1805. The restoration process according to FIG. 18 and the remixing process using additional information may be used simultaneously. In this case,

And

Instead of performing decryption using

And

Since the remixing process can be performed using, a better output can be obtained.

19 illustrates a first method of embedding side information in a mix signal of two channels according to an embodiment of the present invention. Although FIG. 19 illustrates a method of embedding additional information in a stereo mix signal, the present invention includes a method of embedding additional information in a mono, stereo, or multichannel mix signal. For convenience of explanation, the stereo signal will be described as an example.

As illustrated in FIG. 19, the first method divides and embeds additional information into two channels, and the order is alternately embedded in the two channels on a sample basis. Since the signal characteristics of the two channels are different, the masking limit value of each channel can be obtained separately, and the K value can be assigned to each channel differently. That is, as shown, K ₁ may be allocated to one channel and K ₂ to another channel. In addition, the K value may be different for each block. For example, the additional information is first filled in the lower K ₁ bits of sample 0 of one channel (here, left channel) and then the lower K ₂ bits of sample 0 of the other channel (here, right channel). It then fills the lower K ₁ bits of sample 1 of the original channel (left channel) and the lower K ₂ bits of sample 1 of the other channel (right channel). In the drawings, the numbers in the blocks indicate the order of filling the additional information. 19 shows filling from the MSB, it is also possible to fill from the LSB.

20 illustrates a second method of embedding side information in a mix signal of two channels according to an embodiment of the present invention. The second method embeds the additional information into two channels, and alternates the order of embedding the additional information into two channels in units of bitplanes from the LSB. In addition, since the signal characteristics of the two channels are different, masking limit values of each channel may be separately obtained, and K (K ₁ and K ₂ ) values may be differently assigned to each channel. That is, as shown, K ₁ may be allocated to one channel and K ₂ to another channel. In addition, the K value may be different for each block. For example, the first 1 bit of sample 1 of one channel (here left channel) is filled first, and the lowest 1 bit of sample 1 of the other channel (here right channel) is filled first. It then fills the least significant 1 bit of sample 2 of the original channel (left channel) and again the least significant 1 bit of sample 2 of the other channel (right channel). In the drawings, the numbers in the blocks indicate the order of filling the additional information. As described above, in the case of embedding the additional information bitstream in two channels, it is possible to assign a K value differently to each channel. In this case, it is possible to separately transmit the K value for each channel in the bitstream. In addition, when the K value is transmitted in plural, a differential encoding method may be used when encoding the K value.

21 illustrates a third method of embedding side information in a mix signal of two channels according to an embodiment of the present invention. The third method relates to a method of inserting the additional information into a mix signal having at least one channel when the frame of each channel is composed of a plurality of blocks (length B). As shown, the K value may have different values for each channel and each block, or may have the same value. K ₁ , K ₂ , K ₃ and K ₄ The values may be stored in a frame header that is transmitted once for the entire frame, which may be located in the LSB. In this case, the header may be inserted in units of bit planes, and the additional information data may be inserted alternately in a sample unit, or alternately in block units. FIG. 21 shows a case where the number of blocks in a frame is two, so that the size B of the block is N / 2. In this case, the number of bits inserted into the frame becomes (K1 + K2 + K3 + K4) * B.

22 illustrates a fourth method of embedding additional information in a mix signal of two channels according to an embodiment of the present invention. The fourth method is a method of embedding additional information into two channels, and then inserting the information alternately into the two channels in the order of LSB (or MSB) to bit plane and inserting them alternately by sample unit. The method may be performed in units of frames or in units of blocks as shown. As illustrated in FIG. 22, 1 to C (hatched portions) correspond to headers and may be inserted in the LSB (or MSB) in bitplane order to facilitate the search for the insertion frame syncword. C + 1 or more (non-hatched portion) is a part other than the header, and can be alternately inserted into two channels in a sample unit so that additional information data can be easily read. K values may have different values for each channel and block or may have the same value.

23 is a block diagram illustrating a method of removing embedded side information according to an embodiment of the present invention. For the purpose of copyright protection of contents, the method removes or modifies only the additional information from the mixed signal in which the additional information is inserted so that no additional information remains for remixing.

Referring to FIG. 23, the analyzer 2302 analyzes a mix signal 2301 in which additional information is embedded, and extracts embedding information 2303 such as information about a region in which the additional information is embedded. The remover 2304 removes the additional information from the mixed signal 2301 in which the additional information is embedded by using the embedding information 2303 to generate the mixed signal 2305 in which the additional information is removed. In this case, various methods may be used to remove the additional information. In the first method, the presence of additional information embedded from the mixed signal is determined through the analyzer 2302, and the embedded level value is found to determine the removal range. Then, exactly remove the bit string in which the additional information is embedded. The second method finds a level value (similar to K value) of embeddable random noise without distorting the sound quality through the analysis unit 2302, and adds a corresponding random signal to destroy the inserted additional information. will be. The third method is to transform the signal using an all pass filter. The fourth method is to remove the syncword information by replacing the least significant 1 bits with an arbitrary random signal.

24 is a block diagram of an apparatus for encoding a mixed signal in which additional information is embedded according to an embodiment of the present invention. Referring to FIG. 24, the encoding apparatus includes an additional information extracting unit 2401, a mixed signal encoding unit 2402, and a multiplexing unit 2403. There may occur a case where the mixed information embedded with the additional information needs to be encoded (for example, Advanced Audio Coding (AAC) encoding). At this time, in the case of general AAC encoding, since the embedded information is very inferior in robustness, it may be completely destroyed in the encoding process. However, if the mixed signal is compressed and encoded after extracting the additional information from the mixed signal in which the additional information is embedded before encoding, the additional information may be transmitted together with the compressed mixed signal. In the following, such a method will be described in detail.

First, the additional information extractor 2401 extracts additional information from the mixed signal in which the additional information is embedded. As the method of extracting the additional information, the method described in FIG. 1 may be used. Next, the mix signal encoding unit 2402 compresses and encodes the mixed signal from which the side information is extracted, and the multiplexer 2403 multiplexes the encoded mix signal and the side information to generate a bitstream. If the additional information is spatial information, the multiplexed bitstream may be a compressed bitstream with spatial information. For example, it may be an AAC + MPEG Surround signal.

In this case, if the additional information is a signal that is used by being aligned in time with the PCM type mix signal, the frame unit of the encoded mixed signal and the unit into which the additional information is inserted need to be aligned. It is necessary to adjust the encoding start position of the mixed signal by using the frame sync information known in the process of extracting additional information. In addition, the frame length of the mix signal may be determined using the frame length information of the embedded information. A method of using the frame length information will be described later. The final bit stream generated by the above method has a very low data rate but has additional information, and thus, a multi-channel signal can be generated as a signal having a very low bit rate and used very conveniently for storage and transmission.

25 is a detailed block diagram of the encoding apparatus of FIG. 24 according to an embodiment of the present invention. Referring to FIG. 25, the encoding apparatus may further include a buffering unit 2505. Further, the additional information extracting unit 2501, which is a component of the encoding apparatus, includes a sync information extracting unit 2502, a header information extracting unit 2503, and a payload extracting unit 2504.

First, the sync information extractor 2502 extracts sync information of additional information from the mixed signals Lo 'and Ro' embedded with additional information. When the sink information is found, a position corresponding to the first sample of the sink information becomes a frame sync value, and the frame sync value is transmitted to the buffering unit 2505. The buffering unit 2505 detects the start position of the frame by using the frame sync value, buffers data of the mixed signal frame length from the corresponding position, and transfers the data to the mix signal encoding unit 2506.

Meanwhile, the header information extracting unit 2503 decodes the header area of the additional information existing after extracting the sink information and extracts header information necessary for decoding. The header information may be used to decode payload information corresponding to data information included in the additional information. In this case, the frame length information of the additional information extracted from the header area may be transferred to the buffering unit 2505 or the mixed signal encoding unit 2506. The mixed signal encoding unit 2506 may determine the frame length of the mixed signal using the frame length information.

 The payload extractor 2504 extracts actual data information except sync information and header information from the additional information. The mixed signal encoding unit 2506 encodes the mixed signal using the sync information and the frame length information. Next, the multiplexer 2507 generates a bitstream using the encoded mixed signal and additional information.

26 is a block diagram of a decoding apparatus for embedding additional information in a mixed signal according to an embodiment of the present invention. Referring to FIG. 26, the decoding apparatus includes a demultiplexer 2601, a mixed signal decoder 2602, and an embedding unit 2603.

The demultiplexer 2601 demultiplexes the bitstream to extract an encoded mix signal and additional information to be embedded in the mix signal. Next, the mixed signal decoding unit 2602 decodes the encoded mixed signal and outputs the decoded mixed signals Ld and Rd. An embedding unit 2603 embeds the additional information in the decoded mix signal to generate the mixed signals Ld 'and Rd' with the additional information embedded therein. The mix signal may be stored and transmitted to a PCM signal storage medium such as a CD.

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

As described above, in coding the mixed signal, additional information having a low bit rate may be used to remix specific source signals included in the mixed signal. In this case, additional information for remixing the source signal is required, and the present invention embeds the additional information into the mix signal, thereby remixing the signal to a storage medium having no auxiliary data area or a data format having no auxiliary data area. There is an effect that can provide a method and apparatus that can be reproduced.

Claims (26)

Extracting additional information embedded in a non-perceptual region of the mix signal including one or more source signals; Obtaining a user mix parameter; And Generating a remix signal using the mix signal, the additional information, and the user mix parameter, The additional information indicates a relationship between the source signal to be remixed of the source signals and the mixed signal. The method of claim 1, wherein the extracting of the additional information is performed. Extracting size information of an area in which the additional information is embedded from a header area of the additional information; And And extracting the additional information from the mix signal using the magnitude information. The method of claim 2, The header area of the side information is embedded in the least significant bit (LSB) of the mixed signal. The method of claim 2, The size information has a fixed value in the block in which the additional information is embedded. The method of claim 1, wherein the signal processing method is Extracting an error correction code embedded in the mix signal; And And determining whether the additional information is damaged by using the error correction code. The method of claim 1, The user mix parameter is generated using the control information obtained from a user. The method of claim 1, The additional information includes level information of the source signal, relative level information between the source signals, or relative level information between the source signal and the mixed signal channel. The method of claim 1, The additional information includes information indicating the degree to which the source signal is mixed in the mixed signal channel. The method of claim 1, The additional information includes time delay information between the source signals or time delay information between the source signal and the mixed signal channel. The method of claim 1, The additional information includes cross correlation information between the source signals or cross correlation information between the source signal and the mixed signal channel. Obtaining a mix signal comprising one or more source signals; Obtaining a source signal to be remixed among the source signals; Generating side information using the mix signal and the source signal to be remixed; And Embedding the additional information in a non-perceptual region of the mix signal, And the additional information indicates a relationship between the source signal to be remixed and the mix signal. The method of claim 11, The additional information is embedded in the non-perceptual region in a sample plane order or a bit plane order. The method of claim 12, The additional information is inserted into the sample plane or the bit plane from the upper bit or the lower bit is inserted into the method. The method of claim 11, And a size of an area in which the additional information is embedded is obtained for each block in which the additional information is embedded using a masking threshold of the mixed signal. The method of claim 14, And the additional information is embedded in the area after replacing the area in which the additional information is embedded with zero. The method of claim 14, And the sub information is embedded after subtracting the sub information from the area in which the sub information is embedded and requantizing the area. The method of claim 11, wherein the embedding step, When the number of bits of the additional information to be embedded is smaller than the embeddable number of bits in the region in which the additional information is embedded, the remaining bits are filled with zeros, filled with random signals, filled with original mixed signals, or end bit strings. The method further comprises the step of filling with or in combination thereof. Extracting a combined signal embedded in a non-perceptual region of the mix signal comprising one or more source signals; Lossless decoding the combined signal to restore a signal corresponding to the non-perceptual region; And Restoring the mixed signal from which the combined signal is extracted using the restored signal; The combined signal includes a signal obtained by losslessly encoding a signal located in a non-perceptual region of the mixed signal and additional information. The method of claim 18, The additional information indicates a relationship between the source signal to be remixed of the source signals and the mixed signal. The method of claim 19, wherein the signal processing method is Obtaining a user mix parameter; And And generating a remix signal using the additional information, the mix signal, and the user mix parameter. Losslessly encoding a signal located in a non-perceptual region of the mix signal comprising one or more source signals; Generating a combined signal by combining the lossless coded signal and additional information; And Embedding the combined signal in the non-perceptual region, The additional information indicates a relationship between the source signal to be remixed of the source signals and the mixed signal. Extracting additional information present in the non-perceptual region of the mix signal including one or more source signals; Encoding the mix signal; And Generating a bitstream using the encoded mix signal and the side information; The additional information indicates a relationship between the source signal to be remixed of the source signals and the mix signal. The method of claim 22, wherein the encoding step Extracting sink information of the additional information; And And determining a frame start position of the mix signal using the sync information. Obtaining an encoded mix signal and side information; Decoding the encoded mix signal; And Embedding the additional information in a non-perceptual region of the decoded mix signal, The mixed signal includes one or more source signals, and the additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal. An embedded signal decoding unit for extracting encoded side information embedded in a non-perceptual region of a mix signal including one or more source signals; An additional information decoding unit to decode the encoded additional information to generate additional information; And A remix renderer configured to generate a remix signal using the additional information, the mix signal, and a user mix parameter, The additional information indicates a relationship between the source signal to be remixed among the source signals and the mix signal, wherein the user mix parameter is generated using control information obtained from a user. An additional information generator configured to generate additional information using a mix signal including one or more source signals and a source signal to be remixed; And An embedding unit for embedding the additional information in the non-perceptual region of the mix signal, The source signal to be remixed is selected from among the source signals, and the additional information indicates a relationship between the source signal to be remixed and the mix signal.

Images